The Impact of iCoaching on Teacher-Delivered Opportunities to Respond

Abstract

Coaching in the school setting typically follows the traditional format of preconference, observe, and postconference, where feedback on teaching performance is shared but often delayed. Professional development (PD) provides teachers with skills to enhance their teaching practice with little to no follow-up or support. The most effective way to produce change in the school setting is to show the connection between PD and student performance, and iCoaching can help to bridge the gap. In this study, four teachers participated in a focused PD session and subsequent iCoaching sessions where the researcher used iPods and Bluetooth earbuds as a bug-in-ear (BIE) device. A coach served as a live, remote observer providing coaching prompts to increase teacher-delivered opportunities to respond (OTR). The results indicated that iCoaching was effective in increasing teacher-delivered OTR and in increasing student responses and academic performance.

Keywords

iCoaching teacher coaching technology opportunities to respond professional development bug-in-ear

Feedback on employee performance is important for both supervisors and employees. The use of coaching to provide feedback is found in both public and private organizations. Traditional coaching typically provides delayed feedback (Kretlow & Bartholomew, 2010). Research has shown that effective employment coaching involves immediate, systematic, positive, constructive, and detailed feedback (Scheeler & Lee, 2002). Without that coaching, learners often practice and repeat errors, especially when they are implementing newly acquired skills. Delayed feedback can affect student performance in that students may learn skill practice incorrectly (Heward, 1997), and coaching can help to bridge this gap. Coaching and feedback should be provided immediately and often to reinforce correct responding. Error correction should also happen immediately to provide an opportunity for performance of the correct response. Coaching using immediate feedback corrects the errors in situ and results in effective instruction while practicing a skill.

Kretlow and Bartholomew (2010) found that coaching in general was a promising practice for teachers in promoting high-fidelity evidence-based practices (EBPs). Teachers noted that the need for collaboration with a coach and time for reflection were imperative to improving their teaching skills. The researchers also found that professional development (PD) for the coaches helped them support teachers in learning new ways of delivering lessons and encouraged the teachers to use their newly learned practices. In addition, Kretlow and Bartholomew found two models of coaching were most common in the literature: the traditional coaching model called supervisory coaching, which is usually accompanied by delayed feedback and recommendations for follow-up, and side-by-side coaching that provides immediate feedback and modeling in real time alongside the employee. Critical components of both coaching models include highly engaged, instructive group training sessions and follow-up observation sessions with specific feedback. Coaching also helps to prevent isolation frequently experienced by new teachers (Joyce & Showers, 1982).

In one of the first studies on delivering immediate feedback to special education preservice teachers, Scheeler and Lee (2002) provided preservice teachers immediate feedback via bug-in-ear (BIE) during a field-based practicum. Participants took a course on effective instruction before the study and were prepared for the intervention using short, specific coaching statements delivered through BIE. The preservice teachers received immediate feedback, given within 1−3 s, on their instructional delivery through BIE from a researcher in the back of the classroom. Feedback consisted of short statements to deliver corrective feedback or provided specific praise. This enabled the preservice teachers to resume active responding faster and strengthened the instructional response, resulting in more completed three-term contingencies for students. Immediate corrective feedback was more effective than traditional delayed feedback in improving instruction. Over time, the use of BIE coaching has morphed into the use of cameras (Rock et al., 2009) and iPods.

The improvement of teacher effectiveness has been the focus of most school-based coaching studies. Several EBPs have been targeted to improve both teacher practice and student engagement and achievement. One EBP, opportunities to respond (OTR), has been defined as the teacher’s direct prompt (instructional stimulus) to a student and the student’s response; this includes the combination of interaction between teacher talk, prompts and wait time, and a verbal prompt or question to promote an appropriate response (Cavanaugh, 2013; Haydon, MacSuga-Gage, Simonsen, & Hawkins, 2012; Haydon, Mancil, & Van Loan, 2009; Stichter et al., 2008). Providing students increased OTR is a way of enhancing academic and behavioral achievement (Haydon et al., 2010). Students who have more OTR in the classroom setting excel both academically and behaviorally (Sutherland, Alder, & Gunther, 2003). Low-achieving students tend to receive fewer OTR, which causes academic and behavioral frustration leading to disruptions within the classroom (Haydon et al., 2010). Providing struggling students with more OTR has the potential to increase student engagement, increase appropriate behavior, and solicit more correct academic responses (Haydon et al., 2010; Haydon et al., 2012; Haydon et al., 2009). OTR provides feedback to the student and can help to support teacher implementation of classroom management systems (Cavanaugh, 2013).

When used proactively, OTR can provide the basis for interventions, help prevent schoolwide and classroom behavior issues, and can be included in the response to intervention (RtI) model of support as a Tier 1 support in combination with other classroom management strategies (Simonsen, Fairbanks, Briesch, Myers, & Sugai, 2008). Increasing OTR can strategically be implemented with students with disabilities and with students in low-performing schools, where students experience nonsupportive, instructionally and academically poor teaching environments. Low OTRs often accompany at-risk environments and are coupled with little to no contingency reinforcement to promote or support positive behavior (Stichter et al., 2008).

A definitive standard for optimal rates of OTR within a special education classroom is still lacking. Several sources (Haydon et al., 2010; MacSuga-Gage & Simonsen, 2015; Sutherland et al., 2003; Sutherland & Wehby, 2001) cite the Council for Exceptional Children (CEC, 1987) recommendations of OTR rates of 4–6 per min with 80% accuracy during introduction of new material and 8–12 per min with 90% accuracy for independent drill and practice with students with high-incidence disabilities (e.g., learning disabilities). More recently, in a review of OTR studies, MacSuga-Gage and Simonsen (2015) cautiously recommended an optimal rate of 3–5 OTR per min. In their discussion, the rate of 3–5 OTR per min applies to direct instruction of basic facts or in drill and practice instruction.

One of the challenges found in supporting teacher adoption of new, effective practices is in connecting the change to student performance (Guskey, 2002). The most effective way to affect the process of teacher change is to show an improvement in student performance. Once a teacher implements a new teaching strategy, either through coaching or PD, he or she continues to use that strategy if an academic, behavioral, or emotional improvement is seen in student behavior. The evidence seen firsthand in student progress is the motivating factor behind implementation and continued use (Guskey, 2002).

Teacher investment in the PD content and support for implementation need to be present for teachers to effectively bridge the gap between PD and implementation (Guskey, 2002). A combination of coaching and PD is supported throughout the literature (Joyce & Showers, 1980, 1982; Knight, 2009; Kretlow, Wood, & Cooke, 2011). Coaching can provide the necessary support to enhance and refine current skills and those learned within PD that need to be applied in the classroom setting. Enhancing a teacher’s repertoire of teaching skills is easier than learning completely new skills.

The purpose of this study was to increase the number of teacher-delivered OTR to students in the classroom during instruction by using a combination of PD and coaching via iPod. Given that students with autism spectrum disorders (ASD) typically have limited communication skills, and the more opportunities a teacher provides to the students to engage in communication, the more proficient students will become at communicating, this study sought to add to the literature base on effective teacher behavior. By increasing teacher-delivered OTR, students are given more opportunities to practice communicating in the classroom. Specifically, an intervention package was created to combine PD and in situ coaching via Apple iPods, therefore, labeling it iCoaching. The study posed three Research Questions:

Will iCoaching increase teacher delivery of OTR for students during whole group instruction?

Will OTR increases remain after iCoaching is withdrawn?

Will increased OTR levels result in increased student responses and academic performance?

Method

Participants and Setting

Participants in this study were four in-service special education teachers and the students with ASD in their classes. The teachers were selected based on their certification (special education), nominated by the principal, and were in good professional standing (i.e., not currently on a PD or improvement plan). Teachers were observed to deliver consistently low OTR levels (less than 50% of the total 10-s intervals during their lessons) prior to selection and confirmed during baseline. A description of the study participants can be found in Table 1.

Table 1.

Participant Information.

Teacher	Age	Race	Number of Years Teaching	Degree	Teaching Certification(s)	Student Ages	Student grade Range	Number of Students in Class
Sarah	30	White	8	BA English MS in ESE	ESE, K–12 EE, K–6	11–14	6–8	10
Laura	34	White	7	BS, EE, ESE, K–12 concentrations	ESE, K–12 with ASD Endorsement, Pre-K–Grade 3	6–8	1–2	9
Katrina	39	Hispanic	7	BS, Finance MBA-Business, Pursuing MEd in ESE	ESE, K–12 with ASD Endorsement, EE K–6	11–14	6–8	10
Aaron	35	Black	7	BA, EE, K–6, MEd in ABA	ESE, K–12 EE, K–6	9–11	4–6	10

Note. ESE = exceptional student education; EE = elementary education; ABA = applied behavior analysis.

The study was conducted in a public K–8 charter school in Florida for students with ASD. To attend the school, students were required to have an ASD diagnosis/identification and an Individualized Education Program (IEP) stating that they needed a specialized, intensive environment to be successful. Classes in this school were smaller than traditional public school classrooms. The nature of the school with class sizes of five to six students may be defined as small group instruction. However, since all students in each teacher’s classroom participated in the study, the setting is referred to a whole class instruction.

The study was conducted remotely from the researcher’s work or home office. Closed-circuit camera feeds were used to conduct the observations. In this way, the intrusiveness of another adult in the small classrooms was minimized. The closed-circuit cameras displayed a continuous video and audio stream and were used by administration for similar observation activities. The classrooms were accessed by the researcher using a closed-circuit video application. The researcher logged in to the application at a remote location and was connected to the camera system. Remote coaching is supported by a history of research that indicates teachers find remote audio coaching unobtrusive and a good way to receive the feedback (e.g., Scheeler, McAfee, Ruhl, & Lee, 2006). The researcher called the participant using FaceTime audio on the participant’s iPod from her iPhone or MacBook to initiate the observation session and to improve the quality of the closed-circuit TV feed. The night before each observation, the researcher e-mailed the participant to confirm the time of the next day’s observation. On the day of each observation, the researcher texted participants to ensure they were ready to begin the session. Messages were sent using iMessage, the texting application on Apple products.

Materials

The researcher created the materials for the coaching session, which were provided to the teachers just prior to implementing the intervention, after the participant achieved a stable, predictable baseline. Foundational knowledge regarding OTR was assessed via a pretest, after which participants viewed a researcher-created video module (i.e., a narrated 20-min PowerPoint). The participants were provided with guided notes to complete while viewing the video. A posttest was administered upon completion of the teacher coaching module once the participant sent the researcher completed guided notes. Following the posttest, the researcher sent the participants links to videos (OTR, n.d.) created by the University of Louisville as examples of teacher-delivered OTR and conducted practice sessions with the participants until they completed the items on the Mastery Criteria Checklist. The researcher provided each participant with the following equipment for the study: iPod touch, 16 gigabytes; Bluetooth headset paired with each iPod; individual charging cords for the iPod touch and headset; and a charging station for charging both iPod and headset together.

Experimental Design

This study utilized a multiple probe across participants design, as described by Horner and Baer (1978) and Gast (2010). A variation of the multiple baseline design, multiple-probe design combines multiple baseline and probe techniques to demonstrate a functional relation between the application of the independent variable and the change in the dependent variable. Multiple-probe designs differ from multiple baseline designs in that baseline data are not continuously collected for each behavior, setting, or participant during all conditions of the study and was selected because of the increased possibility of reactivity during continuous baseline. The multiple-probe design was selected because it allowed the investigator to assess the participants during all conditions of the study (Horner et al., 2005; Kratochwill et al., 2013).

The study design consisted of three conditions including baseline, an intervention package (teacher coaching video module and iCoaching), and maintenance (three probes conducted following withdrawal of intervention). This study was conducted with each participant independently, which allowed the investigator to implement the intervention package individually. The goal of the study was to investigate if the teachers’ behavior increased if and only if they received the intervention, thus providing a demonstration of experimental control and establishing a functional relation between the independent and dependent variables.

The study used partial interval recording. Each minute of an observation session was divided into six 10-s intervals. Lessons were observed for a maximum of 15 min, yielding a maximum of 90 intervals per session. Partial interval recording was chosen because it estimates the occurrence of the target behavior without requiring continuous observation (Kennedy, 2005).

Behavioral Measures

Independent variables

The independent variables of this study included individual behavioral measures for Parts 1 and 2 of the intervention. Part 1 of the intervention included an OTR teacher preparation session developed by the researcher. The teacher preparation session consisted of the pretest to assess current knowledge of OTR, a narrated video with guided notes, a posttest, example videos, and practice sessions with equipment and coaching comments. Upon completion of the pretest, the participants viewed the video module and completed the guided notes and then returned them to the researcher. Next, the teacher completed the posttest. The pretest and posttest contained nine questions, which assessed foundational knowledge of OTR and included multiple-choice and open-ended questions (e.g., OTR include, and Name two types of response strategies). Finally, the teachers viewed videos independently where they identified examples of teacher-delivered OTR and completed practice sessions with the researcher using the equipment while receiving practice coaching comments until they indicated proficiency with the equipment via self-reporting.

Part 2 of the intervention included iCoaching. During iCoaching, teachers were prompted to deliver OTR to target students via an iPod and Bluetooth. The independent variable during iCoaching was the coaching prompt that included the student name to signal the teacher to call on that student and the word prompt. The researcher and participant reviewed specific coaching comments that were used during the postbaseline teacher coaching sessions. A FaceTime audio call was connected using the FaceTime app, available on all Apple platform products, via wireless Internet. The teacher received the coaching comment via Bluetooth from the coach. The independent variable in Part 2 was the percentage of intervals, where coaching comments were delivered to the participant each session. Each session was between 5 and 15 min, with each minute divided into 10-s intervals, with the number of intervals between 30 and 90 for a full session.

Dependent variables

The first dependent variable, OTR, included prompts to increase teacher-delivered statements or questions intended to elicit responses from the individual students or entire class. For this study, the specific prompts included verbal or gestural, academic or nonacademic, and class or individual. A detailed explanation of the dependent variables, including definitions, examples, and nonexamples, is found in Table 2.

Table 2.

Dependent Variable Definitions— OTR—Teacher Prompting.

Term/Code	Definition	Example	Nonexample
Verbal/V	Teacher addresses students or class verbally—statements, mands, requests, or questions to indicate a response is needed	Teacher uses words, names, or specific language to address students	Teacher talk not related to prompting student
Gestural/G	Teacher gestures, points, or manually guides student or class to indicate that a response is needed	Pointing, tapping on student desk or shoulder	Clapping, snapping, gestures not directly related to the students or used as a prompt
Academic/A	Prompt is directed toward an academic task	“Billy, what is number four?”	Teacher uses a hand signal to students to pay attention
Nonacademic/N	Prompt is directed toward a nonacademic task	“Billy please pick your head up off the desk”	Language or gesture directing students to provide academic information
Individual Student/I	Prompt is directed to individual student	“Billy, what is two plus two?”; teacher points at Billy	No student name specified or prompt directed toward a group
Class/C	Prompt is directed to entire class	“Class, raise your hand if you know the answer”	Teacher specifies student name or gestures at student

Note. OTR = Opportunities to Respond.

A secondary dependent variable included two student performance measures:

A measure of student responses during whole group instruction and

A measure of student attainment of Access Point (special education alternate Florida state standards) goals.

The first student measure was student response behavior. The definition of student response behavior included a response to a teacher question (verbal or nonverbal), a response to a teacher request or mand, or a response to a teacher prompt. Examples and nonexamples of student responses to teacher-delivered OTR are found in Table 3.

Table 3.

Examples and NonExamples of Student Responses to Teacher-Provided OTR.

Teacher-Provided OTR	Example Student Response	Nonexample Student Response
“Take out your book.”	Student takes out his or her book	Student echoes teacher: “Take out my book!”
“Sit up in your seat.”	Student sits up straight in seat.	Student does not sit up.
“What are you thankful for?”	Student points to picture on communication device.	Student does not respond.
“Where are we going?”	Students say, “Outside!”	Student does not respond.
“Good morning”	“Hello,” or “Good morning”	Student does not respond.
“What did you have for breakfast?”	“Waffles!”	Student echoes teacher: “Breakfast.”

Note. OTR = Opportunities to Respond.

The second student measure included a curriculum-based component. The students at the school were assessed on a specific set of alternate state standards for students with disabilities in Florida, referred to as Access Points. Access Points are categorized into levels of difficulties (participatory, supportive, and independent) to ensure that all students with disabilities were measured on the standards, which are correlated and measured with the state alternate assessment (CPALMS, 2017).

Procedures

Baseline

During baseline, at least five probes were conducted for each teacher to establish (a) whether any coaching prompts were delivered and (b) whether the teachers delivered any prompts during their usual instruction. In addition, the initial data for each student’s curricular performance were collected during this phase. This was done by collecting the data from the teacher on the Access Point directly related to the class observed. Sarah had five baseline sessions, Linda had six baseline sessions, Katrina had six baseline sessions, and Aaron had seven baseline sessions. Teachers were instructed to follow their typical routines during baseline.

Intervention Part 1—Teacher preparation session

Intervention Part 1 started with the first teacher (Sarah), while the other three teachers remained in baseline. Intervention Part 1 involved e-mailing the teacher the link for the one item upon finishing the current item in the following order with no time limits: pretest, coaching module with guided notes, and posttest.

Each participant engaged in a practice session using FaceTime to achieve proficiency with equipment and to resolve any issues using the closed-circuit cameras, iPods, Bluetooth headsets, or the FaceTime call process. The session lasted between 5 and 10 min and was geared toward proficiency in using the equipment and provided exemplars of the coaching comments that would be received while teaching. Teacher preparation sessions were delivered individually as each teacher achieved stable baseline, which was determined by visual analysis during the baseline condition and prior to receiving the coaching to increase teacher-delivered OTR. During this phase, the researcher monitored each teacher’s progress as they completed the items on the mastery checklist. All items on the mastery checklist (e.g., watched video, completed and sent guided notes to researcher, watched example videos) were completed before implementing the iCoaching.

Intervention Part 2—iCoaching

Coaching comments specific to teacher prompting behavior were delivered to each participant via an iPod and Bluetooth earpiece. A FaceTime audio call was made so the coaching could be delivered. The teacher received the coaching comment via the Bluetooth earpiece from the researcher and then the teacher prompted the specific student mentioned in the coaching comment immediately (within 3–5 s). In addition to the coaching intervention, the researcher continued to collect data on observational measures of student response, researcher coaching comments, and teacher delivery of OTR (prompts). Sessions lasted an average of 10 min, with the range from 5 to 15 min. The iCoaching intervention was delivered to Sarah, Linda, Katrina, and Aaron for nine, eight, seven, and seven sessions, respectively. The iCoaching procedure for this study was removed once the percentage of teacher-delivered OTR reached a high, stable level (across at least three data points at least 50% over baseline; Cooper, Heron, & Heward, 2007).

Maintenance

Maintenance probes were conducted once a week for 3 weeks after the intervention was withdrawn. No coaching comments were delivered, but the FaceTime audio call was initiated, and the video cameras were accessed via the computer application. The participants did not wear the Bluetooth earpiece and left the iPod on a speaker mode as they did during baseline. The researcher observed each participant during the same academic time and subject as previous phases. Finally, data on student performance on curriculum-based measures were collected when all teachers completed maintenance.

Interobserver Agreement and Procedural Fidelity

To prepare observers to use the codes reliably, the researcher and research assistant independently viewed the example videos (OTR, n.d.) used in the teacher coaching preparation session. The videos demonstrated effective teacher-delivered OTR for students with disabilities and provided examples to define and identify teacher-delivered OTR prior to beginning the intervention. The researcher and assistant compared results from the training videos, and if a significant discrepancy was found, they watched the video again together. Final agreement for interobserver agreement (IOA) training between researcher and research assistant was 89%.

A data collection sheet compared observers’ recording of observation codes on a code-by-code basis. IOA was calculated by counting the number of codes recorded in agreement by the observers, divided by the number of total codes for each teacher-delivered OTR (by category: verbal/gestural, academic/nonacademic, and individual/class), and multiplied by 100 to determine percentage of point-by-point agreement (Kazdin, 1982). This was calculated on an interval-by-interval basis, across all conditions of the study. During the study, IOA was collected for 42% of total sessions (i.e., 28 of 67 total sessions), which exceeds the recommended minimum of 20% from Kratochwill et al. (2013). Total interobserver agreement for Sarah had a range of 82–100% (M = 91%). For Katrina, IOA range was 79–100% (M = 93%). For Aaron, IOA range was 74–97% (M = 88%). For Linda, IOA range was 53–92% (M = 82%). The range of Linda’s IOA seems very large. The reason for this is that during baseline, her IOA range was 53–87%. However, during intervention, IOA range was 89–92%, and 87% during maintenance. IOA by participant and phase is presented in Table 4.

Table 4.

Interobserver Agreement by Participant.

Condition/Participant	Sarah (%)	Linda (%)	Katrina (%)	Aaron (%)
Baseline
Mean	90	67	91	81
Range	83–100	53–87	79–100	74–92
Intervention
Mean	86	91	91	86
Range	82–91	89–92	82–98	86
Follow-up
Mean	96	87	98	97
Range	93–98	87	98	97
Total
Mean	91	82	93	88
Range	82–100	53–92	79–100	74–97

To ensure procedural fidelity of the intervention, a checklist was created containing the following items: materials (iPod, earphones, and computer application for closed-circuit camera), connected via FaceTime audio, coaching statements followed same pattern, appropriate data collection form used, and the observation lasted no more than 15 min. Procedural fidelity data were recorded during intervention and maintenance for all participants. Each participant had at least two procedural fidelity checks during intervention and one during maintenance. Procedural fidelity data were not collected during baseline. The data showed 100% procedural fidelity across all 14 fidelity checks.

Results—The Impact of iCoaching on Teacher Delivery of OTR

Figure 1 shows the total teacher-delivered OTR during all experimental conditions of the study, which addresses Research Questions 1 and 2 and shows the student response data for all experimental conditions of the study, which addresses Research Question 3.

Figure 1.

Teacher-delivered Opportunities to Respond (OTR) and student response.

Intervention Part 1—Teacher Preparation Session

The teacher preparation data collected during Intervention Part 1 address Research Questions 1 and 2. A summary of the teacher scores on the pretest, guided notes from the coaching module, the posttest, and the guided notes is presented in Table 5.

Table 5.

Intervention Part 1: Teacher OTR Knowledge Assessment Results.

Teacher	OTR Pretest Score (%)	OTR Posttest Score (%)	Guided Notes (%)	Mastery Checklist (%)
Sarah	11	78	100	100
Linda	33	89	100	100
Katrina	11	100	100	100
Aaron	22	89	100	100

Note. OTR = Opportunities to Respond.

A score of 100% was considered mastery on the posttest. Only one teacher, Katrina, achieved that score initially. The other three teachers did not initially earn a score of 100%. They met with the researcher and reviewed the incorrect items on the posttest prior to moving on to the other items on the checklist.

Intervention Part 2—iCoaching

Across all participants, teachers received no coaching comments during baseline and maintenance. Percentages of coaching comments were calculated by dividing the number of intervals where a coaching comment was delivered by the total number of intervals and multiplied by 100 to achieve percent. During Intervention Part 2, coaching comments delivered to Sarah ranged from 9% to 22% (M = 15%), Linda ranged from 10% to 28% (M = 19%), Katrina ranged from 10% to 16% (M = 13%), and Aaron ranged from 10% to 20% (M = 15%).

Teacher-delivered OTR

The impact of iCoaching on Sarah’s delivery of OTR to students during whole group instruction is found in the first graph of Figure 1. During five baseline sessions, Sarah delivered a range of 22–32% (M = 26%) OTR of intervals. Once intervention was introduced, an increase was seen incrementally over the first two sessions (M = 38% and M = 42%). After the third session, an increase in teacher-delivered OTR to 76% was seen, which decreased on the fourth and fifth intervention sessions, ranging from 57% to 76%. An increase was seen from the sixth to ninth intervention sessions from 72% to 81%. During three maintenance sessions, Sarah’s teacher-delivered OTR remained stable a range of 73–79% (M = 75%).

The impact of iCoaching on Linda’s delivery of OTR to students during whole group instruction is found in the second graph of Figure 1. During six baseline sessions, Linda’s delivery of OTR ranged from 31% to 47% (M = 38%). Linda received eight iCoaching intervention sessions, which ranged from 63% to 87% (M = 77%) OTR. After the eighth session, iCoaching was removed; during the next three maintenance sessions, her teacher-delivered OTR ranged from 72% to 86% (M = 79%).

The impact of iCoaching on Katrina’s delivery of OTR for students during whole group instruction is found in the third graph of Figure 1. During six baseline sessions, Katrina’s delivery of OTR ranged from 28% to 41% (M = 35%). Katrina received seven intervention sessions, where the delivery of OTR ranged from 65% to 89% (M = 81%). After the seventh intervention session, iCoaching was removed. During the maintenance sessions, her teacher-delivered OTR ranged from 82% to 90% (M = 86%).

The impact of iCoaching on Aaron’s delivery of OTR to students during whole group instruction is found in the fourth graph of Figure 1. During seven baseline sessions, Aaron’s delivery of OTR ranged from 19% to 42% (M = 29%). Aaron received seven intervention sessions, where the teacher-delivered OTR range was 69–88% (M = 81%). After the seventh intervention session, iCoaching was removed. During the maintenance, his teacher-delivered OTR ranged from 79% to 85% (M = 82%).

Teacher-delivered OTR by Type

Data on six types of teacher-delivered OTR were collected. Types of OTR were verbal, gestural, academic, nonacademic, individual, and class. Percentages of OTR type were calculated by dividing the number of intervals of occurrence of each type by the total number of OTR intervals and multiplying by 100 to achieve percent. Regardless of student age or ability level, the teacher-delivered OTR followed a similar pattern for all teachers: verbal, academic, and individual OTRs were delivered more frequently than gestural, nonacademic, and class OTR. For example, Sarah’s class had the lowest level of communication skills and the wait time between teacher cue and student response was up to 30 s. While waiting for students to respond, Sarah was coached to increase OTR by prompting other students who were waiting. Linda was also coached to increase OTR by providing prompts to attend to the academic material on the projection screen, where she asked the students comprehension questions based on the material presented. Katrina was prompted to increase her number of prompts to her students to maintain her instructional pace. Like Sarah, Aaron was coached to use wait time to provide prompts to students, which was an effective way to increase OTR in his classroom by delivering mands to students while waiting for others to respond. It is notable that Linda had the highest nonacademic and class delivery of OTR in baseline, intervention, and maintenance. Linda’s students had the greatest need for behavioral prompting, and her students typically responded as a class, as was her style of teaching. Katrina had the highest level of individual OTR, as her teaching style was more drill-type direct instruction.

Student Outcomes

Student response data are also shown in Figure 1. This shows the change in student response across the experimental conditions in direct comparison to teachers’ increases in their delivery of OTR.

Sarah’s student response data are found in the first graph in Figure 1. During baseline, student response level ranged from 20% to 32% (M = 26%) of the intervals. During intervention, her response level ranged from 37% to 77% (M = 61%). During maintenance, Sarah’s student response level ranged from 48% to 67% (M = 60%).

Linda’s data for student responses are found in the second graph in Figure 1. During baseline, the student response level ranged from 24% to 44% (M = 33%). During intervention, her student response level ranged from 61% 83% (M = 73%). During maintenance, Linda’s student response level ranged from 71% to 78% (M = 74%).

Katrina’s data for student response are found in the third graph in Figure 1. During baseline, student response level ranged from 25% to 39% (M = 32%). During intervention, her student response level ranged from 62% to 82% (M = 75%). During maintenance, Katrina’s student response level was ranged from 65% to 90% (M = 78%).

Aaron’s data for student response are found in the fourth graph in Figure 1. During baseline, his student responses ranged from 17% to 33% (M = 24%). During intervention, his student response level ranged from 60% to 82% (M = 73%). During maintenance, Aaron’s student response level, ranged from 73% to 77% (M = 76%).

These data show that as the teacher-delivered OTR increased, the student responses also increased. The largest gains in student responses were found in Sarah’s classroom, where the range was a low of 37% to a high of 77% during intervention. The smallest gains in student responses were found in Katrina’s classroom, where the range was a low of 62% to a high of 82% during intervention. Regardless of the students’ ages and academic levels, observable gains in student responses were made. Of note, the students with the lowest academic skills (Sarah’s class) made the largest gains.

Student attainment of access points

Participants identified specific Florida State Access Points as the focus of their instruction during the study. To answer Research Question 3, student proficiency of the specific Access Point was assessed by the teacher and reported to the researcher before the intervention (i.e., during baseline data collection) and again after maintenance (i.e., the end of the study) for each student in each teacher’s class. In Sarah’s class, the prestudy percentages ranged from 0% to 60% proficiency. Poststudy, the range of proficiency in Sarah’s class was 20–80%. Students in Linda’s class had a prestudy range of 20–80%. After the study, student proficiency ranged from 20% to 100%. Katrina selected two Access Points for the study. Katrina had a prestudy range of 0–60% for students on the first Access Point and a range of 0–50% for the second Access Point. All the students in her class made gains on both Access Points. Aaron’s class had a prestudy range of 30–80%. After the study, the student proficiency ranged from 40% to 85%. The researcher had no control over the pre–post assessment; it was teacher made and teacher reported.

Social Validity

Social validity was assessed after all participants completed the maintenance phase of the study. All participants indicated that the study changed the way they felt about receiving iCoaching. One participant said, “Seeing things with another person’s pair of eyes is always helpful. The rapid feedback helped me to see just how much more my students could be engaged throughout lessons.” When asked if the iCoaching they received was too long, too short, or just right, four of the four participants said it was just right. Only one of the four teachers indicated that the iCoaching was disruptive. All four teachers found the study had an impact on the way they communicated with their students. When asked about the possibility use of iCoaching as an evaluative tool for teacher observations by supervisors, or with peers as peer coaching, three of the four indicated they would want a supervisor using iCoaching evaluatively; all four teachers indicated they would use iCoaching with colleagues as a peer coaching tool.

Discussion

Impact of iCoaching on Teacher-Delivered OTR

The results of the study indicated that iCoaching had a positive impact on teacher-delivered OTR and increased levels of student response. Prior to implementation of the intervention, teachers delivered OTR at low levels. With the combination of the teacher preparation session and the iCoaching intervention, teacher-delivered OTR rose to higher, stable levels, and remained at those levels after the intervention was withdrawn. The results demonstrated that the teacher’s behavior increased if and only if they received the intervention package. This is a demonstration of experimental control, thus establishing a functional relationship between the independent and dependent variables. A corresponding increase in student responses across the intervention and maintenance phases of the study was also seen. Teacher delivery of OTR followed a pattern for the teachers in the study in that they delivered most OTR as verbal, academic, and individual.

Because the intervention package included a PD beforehand and coaching in situ, it was difficult for participants to determine which part of the package (i.e., PD or coaching) had the greatest impact on their teaching. Teacher behaviors were changed when a comprehensive intervention package of prestudy, PD, and ongoing coaching is delivered. Based on feedback gathered from the participants, the change in teacher behavior was impacted by the entire intervention package rather than one part having more impact than another. Given the PD on OTR, and the knowledge they gained, the coaching supported practice implementation.

A structured classroom with a well-organized teacher provides an opportunity for OTRs to increase even further. Comparing two teachers, one who is structured and who includes additional opportunities for student responses in their lesson plans versus another who has a general plan but tends to wing it during instruction, demonstrates the impact of teacher-delivered OTRs on classroom management. Increasing student attention and time on task also increases the potential for student participation.

This study not only adds to the literature on improvement of teacher use of EBPs to increase student responding, but it also supports the research on the use of iCoaching as a PD strategy. iCoaching can be extended to teacher education to coach preservice teachers to increase EBPs, use of strategies learned in the university classroom setting, and a bridge between research and practice. iCoaching provides fluid support to student interns in their practicum and student teaching placements, thus helping to generalize teaching strategies learned in the university setting to the classroom setting.

The results of this study suggest that iCoaching coupled with PD can improve teacher’s use of EBPs. The coaching comments provided via iCoaching is one way to improve daily teaching routines and provides the teacher with opportunities to practice new EBPs learned in PD or for immediate error correction.

Limitations

The study was conducted with teachers of students in a specialized school for students with autism who had complex behavioral and academic needs. The average class size of 10 students with multiple adults aside from the teacher (e.g., therapists, paraprofessionals, and related service providers) made for a uniquely supported setting. There were several occasions where the researcher waited for a student to de-escalate problem behavior and regain self-control. While this did not have great impact on the study data, it altered observation times and sometimes affected the observation, as the teacher focused on redirecting one student’s behavior, resulting in lower teacher-delivered OTR for that session. To get similar results in a public school, the level of support for student behavior needs to be considered, along with a structured teaching style.

While the literature provides guidance for OTR levels for drill-type instruction (CEC, 1987; Haydon et al., 2010; MacSuga-Gage & Simonsen, 2015; Sutherland et al., 2003; Sutherland & Wehby, 2001), it does not include levels specifically for students with ASD or for other types of tasks beyond drill-type instruction. The absence of normative data drove the decision to use several criteria (improvement over baseline along with high stable levels) to determine a teacher’s proficiency in OTR delivery in this study. Having more studies that verify the rates of response for optimal learning will make this and subsequent OTR studies stronger.

Future Research

Future researchers should consider how technology issues might interfere with a replication of any study that relies on technology as the main method to collect data. This study relied heavily on a consistent audio and visual contact with the classrooms. Not having the researcher in the classroom was done to limit intrusion and reactivity in a very small classroom environment. Therefore, Internet glitches like limited accessibility and poor audio quality posed problems. There were unforeseen technical issues that made the use of an iPod/iPhone to enhance audio reliability unpredictable; those issues were not identified prior to the start of the study. Future research should plan for Internet and technology issues and consider contingency plans. Future use of this form of technology for teacher improvement should also consider the strength and consistency of the technology.

This study showed that iCoaching increases teacher use of effective practices, specifically teacher-delivered OTR. The increase in teacher-provided OTR signaled increases in student responding and student achievement. Therefore, the combination of the teacher preparation session and iCoaching is in line with previous studies that delivered coaching or immediate feedback via a BIE device.

Based on the success observed with in-service special education teachers increasing their delivery of OTR via iCoaching and the increase in student responses, it appears that more research is needed to support iCoaching as an EBP. Future iCoaching studies can be expanded to other groups of teachers including general education and preservice teachers. The current study focused on students with ASD; however, future research can be expanded to include other individuals with disabilities (e.g., learning disability and emotional/behavioral disability). Previous coaching using different types of BIE equipment focused on three-term contingencies (e.g., Goodman, Brady, Duffy, Scott, & Pollard, 2008; Scheeler et al., 2006), where this study expanded to coaching prompts (rather than feedback) and connected to a specific, designated teacher behavior (OTR). Future research can expand to other evidence-based teacher behaviors, including increasing time on task, increasing wait time, encouraging higher level thinking skills, and many more.

Opportunities to use iCoaching can be conducted remotely or in person. This study showed that, if available, classroom closed-circuit cameras could be used as part of the supervision model. In addition, FaceTime video or Internet video calling software (e.g., Skype) can be used. Future studies can use enhanced clip-on lenses (e.g., fish-eye) or computer cameras to obtain a full classroom view via FaceTime video, thus eliminating the need for additional technology or equipment.

Technology is not perfect, but if implemented with fidelity, it can be used to improve daily teaching practices and provide support to teachers. One of the participants summed it up well, “We all know technology isn’t perfect, so sessions did not always go off without a hitch, but in the grand scheme of things, this was a very valuable study, and it has greatly impacted the overall comprehension of my class.”

iCoaching is an effective way to provide coaching to increase research-based teaching behaviors. Expanding technology enables teachers to receive real-time coaching comments and can be extended to other areas, including peer coaching, supervisory coaching, and remote coaching. The use of remote technology applications in PD and coaching increases efficiency and removes geographic limitations and intrusiveness of traditional coaching. Remote technology has evolved from a large, conspicuous, equipment-intensive system to the current state of long-range transmission via covert sending and listening pocket-sized devices. The possibilities are limitless for the future of iCoaching for teacher educators.

By incorporating iCoaching, the researcher prompted the teacher to increase OTR and removed the coaching given for OTR once the teacher reached mastery criteria, creating self-sufficiency in the teacher’s own delivery and monitoring of OTR. Increasing OTR led to an increase in student responding and achievement within the teachers’ classrooms. The data show that in-service teachers’ behavior can be changed using iCoaching and aligns with literature (Albers & Greer, 1991; Greenwood & Maheady, 1997; Guskey, 2002) that recommends making PD meaningful and connecting it with student achievement to effectively change teacher behavior (Joyce & Showers, 1982; Kretlow et al., 2011). iCoaching has the potential to improve coaching practices for in-service and preservice teachers and supports teachers in acquiring and maintaining new teaching strategies learned in PD with support from the coach. iCoaching can be included in teacher PD and preservice teacher training to create coaching protocols and increase use of EBPs in schools.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

References

Albers

A. E.

Greer

R. D.

(1991). Is the three-term contingency trial a predictor of effective instruction? Journal of Behavioral Education, 1, 337–354.

Cavanaugh

(2013). Performance feedback and teachers’ use of praise and opportunities to respond: A review of the literature. Education and Treatment of Children, 36, 111–136.

Cooper

J. O.

Heron

T. E.

Heward

W. L.

(2007). Applied behavior analysis. Upper Saddle River, NJ: Pearson/Merrill-Prentice Hall.

Council for Exceptional Children. (1987). Academy for effective instruction: Working with mildly handicapped students. Reston, VA: Author.

CPALMS. (2017). Browse and search standards. Retrieved from http://www.cpalms.org/Public/search/Standard

Gast

D. L.

(2010). Single subject research methodology in behavioral sciences. New York, NY: Routledge.

Goodman

Brady

M. P.

Duffy

M. L.

Scott

Pollard

(2008). The effects of “bug-in-ear” on special education teachers’ delivery of learn units. Focus on Autism and Other Developmental Disabilities, 23, 207–216.

Greenwood

C. R.

Maheady

(1997). Measurable change in student performance: Forgotten standard in teacher preparation? Teacher Education and Special Education, 20, 265–275.

Guskey

T. R.

(2002). Professional development and teacher change. Teachers and Teaching: Theory and Practice, 8, 381–391.

10.

Haydon

Conroy

M. A.

Scott

T. M.

Sindelar

P. T.

Barber

B. R.

Orlando

A. M.

(2010). A comparison of three types of opportunities to respond on student academic and social behaviors. Journal of Emotional and Behavioral Disorders, 18, 27–40.

11.

Haydon

MacSuga-Gage

A. S.

Simonsen

Hawkins

(2012). Opportunities to respond: A key component of effective instruction. Beyond Behavior, 22, 23–31.

12.

Haydon

Mancil

G. R.

Van Loan

(2009). Using opportunities to respond in a general education classroom: A case study. Education and Treatment of Children, 32, 267–278.

13.

Heward

W. L.

(1997). Four validated instructional strategies. Behavior and Social Issues, 7, 43–51.

14.

Horner

R. D.

Baer

D. M.

(1978). Multiple-probe technique: A variation of the multiple baseline. Journal of Applied Behavior Analysis, 11, 189–196.

15.

Horner

R. H.

Carr

E. G.

Halle

McGee

Odom

Wolery

(2005). The use of single subject research to identify evidence-based practice in special education. Exceptional Children, 71, 165–179.

16.

Joyce

Showers

(1980). Improving inservice training: The messages of research. Educational Leadership, 37, 379–385.

17.

Joyce

Showers

(1982). The coaching of teaching. Educational Leadership, 40, 4–10.

18.

Kazdin

A. E.

(1982). Single-case research designs: Methods for clinical and applied settings. New York, NY: Oxford University Press.

19.

Kennedy

C. H.

(2005). Single-case designs for educational research. New York, NY: Pearson Education.

20.

Knight

(2009). Coaching. Journal of Staff Development, 30, 18–22.

21.

Kratochwill

T. R.

Hitchcock

Horner

R. H.

Levin

J. R.

Odom

S. L.

Rindskopf

D. M.

Shadish

W. R.

(2013). Single-case intervention research design standards. Remedial and Special Education, 34, 26–38.

22.

Kretlow

Bartholomew

(2010). Using coaching to improve the fidelity of evidence-based practices: A review of studies. Teacher Education and Special Education, 33, 279–299.

23.

Kretlow

A. G.

Wood

C. L.

Cooke

N. L.

(2011). Using in-service and coaching to increase kindergarten teachers’ accurate delivery of group instructional units. The Journal of Special Education, 44, 234–246.

24.

MacSuga-Gage

A. S.

Simonsen

(2015). Examining the impact of opportunities to respond on student outcomes: A systematic review of the literature. Education and Treatment of Children, 38, 211–240.

25.

Opportunities to Respond. (n.d.). Retrieved from https://louisville.edu/education/abri/primarylevel/otr/group

26.

Rock

Gregg

Thead

Acker

Gable

Zigmond

(2009). Can you hear me? Evaluation of an online wireless technology to provide real-time feedback to special education teachers-in-training. Teacher Education and Special Education, 32, 64–82.

27.

Scheeler

M. C.

Lee

(2002). Using technology to deliver immediate corrective feedback to preservice teachers. Journal of Behavioral Education, 11, 231–241.

28.

Scheeler

M. C.

McAfee

J. K.

Ruhl

K. L.

Lee

D. L.

(2006). Effects of corrective feedback delivered via wireless technology on preservice teacher performance and student behavior. Teacher Education and Special Education, 29, 12–25.

29.

Simonsen

Fairbanks

Briesch

Myers

Sugai

(2008). Evidence-based practices in classroom management: Considerations for research to practice. Education and treatment of children, 31, 351–380.

30.

Stichter

J. P.

Lewis

T. J.

Whittaker

T. A.

Richter

Johnson

N. W.

Trussell

R. P.

(2008). Assessing teacher use of opportunities to respond and effective classroom management strategies: Comparisons among high-and low-risk elementary schools. Journal of Positive Behavior Intervention, 11, 68–81.

31.

Sutherland

K. S.

Alder

Gunter

P. L.

(2003). The effect of varying rates of opportunities to respond to academic requests on the classroom behavior of students with EBD. Journal of Emotional and Behavioral Disorders, 11, 239–248.

32.

Sutherland

K. S.

Wehby

J. H.

(2001). Exploring the relationship between increased opportunities to respond to academic requests and the academic and behavioral outcomes of students with EBD: A review. Remedial and Special Education, 22, 113–121.