Abstract
All teachers need ongoing coaching that helps them recognize areas of strength and need for teaching students with disabilities. Unfortunately, the amount, quality, and speed with which teachers receive feedback from coaches, administrators, mentors, or other instructional leaders on their academic and behavior management practices is limited. In this article, we describe a multimedia professional development option for documenting teacher practice, generating feedback, and delivering targeted instruction.
Students with disabilities (SWD) often benefit from placement in general education settings, or inclusive settings. Access to the general education curriculum and their peers can positively impact long-term outcomes for students with disabilities (Harber et al., 2016). When SWD are placed in general education classrooms with high quality instructional strategies (e.g., evidence-based and high-leverage practices), they often reap the benefits and can be successful. Thus, to best support SWD, it is important that teachers in multiple settings use evidence-based or high-leverage practices designed to support student outcomes. However, general educators often feel that they do not have the training or skills needed to support these students’ intensive needs (Gable et al., 2012; Reinke et al., 2011). Thus, it is important that educators in multiple settings (e.g., general education, co-taught, self-contained) are given meaningful and data-based feedback that can improve their use of EBPs and HLPs.
Principals, coaches, and other instructional leaders are often limited in numerous ways when it comes to providing effective coaching and feedback to teachers (Sweigart et al., 2016). The same is true for teachers or those in training seeking to self-reflect on their teaching (Nagro et al., 2020). As a result, many teachers do not receive the quality and depth of feedback and ongoing support they may need to make improvements that would impact performance and behavior of students with disabilities (Cornelius et al., 2019). To address some of these barriers for observers, coaches, and teachers, research should focus on the development of coaching tools that enable observers to more readily complete observation cycles, including providing teachers with meaningful data-based feedback.
What Do We Know About Effective Coaching?
Guidelines for Effective Practice
The State Implementation and Scaling up of Evidence-based Practices Technical Assistance Center ([SISEP] Cusumano & Preston, 2018) and National Center for Systemic Improvement (NCSI, 2016) draw upon principles of implementation science to provide a synthesis of research literature outlining coaching practices that have demonstrated evidence for improving teacher practice and student outcomes. The core components of the SISEP and NCSI guides are
What do we know about observations?
Coaching (e.g., observations and follow up feedback) is a promising means of helping to improve teacher practice (e.g., Kraft et al., 2018), yet few schools have resources to provide teachers with such supports (e.g., a full-time colleague hired as a coach). Often, coaching within schools do not meet many or all of the mentioned SISEP and NCSI guidelines. Under-resourced schools in particular struggle to provide contextually relevant and tailored coaching supports to teachers in need (Cortina, 2011). Certain infrastructural challenges also thwart the progress of coaching programs. For example, coaches are often limited in access to observational instruments that provide detailed data on observed instructional and behavior management practices, without additional capabilities for monitoring changes in practice use over time (Reddy et al., 2017). With both measurement limitations and the inability to track progress over time, it can be difficult for coaches in these situations to conduct the meaningful observations recommended in the SISEP and NCSI models.
What do we know about modeling for teachers?
According to the SISEP (Cusumano & Preston, 2018) and NCSI guidelines (2016), when used during a coaching program, modeling is an effective strategy for promoting the use of targeted strategies among teachers (e.g., Biancarosa et al., 2010; Kim et al., 2011). Modeling can be done by observers or coaches in-person or using technology-based options such as PD videos (Rodgers et al., 2019). The person doing the observation is often not in a strong position to understand the academic and behavioral needs of students with disabilities (Crockett et al., 2007). In addition, observers may encounter boundaries with regard to their own knowledge about evidence-based and other effective practices in various content areas, impacting their ability to provide meaningful feedback (Garet et al., 2011) and is therefore likely unable to skillfully model a strategy for the teacher.
What do we know about effective performance feedback to teachers?
Sweigart and colleagues (2015) define performance feedback as using observation data to provide objective information about the lesson, to the greatest extent possible, to a teacher being observed. Effective performance feedback should be non-evaluative (Duchaine et al., 2011), and focused on improving implementation of specific practice(s) (Solomon et al., 2012). However, as noted above, instructional leaders and other supervisors are not always in strong position to deliver performance feedback because they lack (a) an observation instrument/ tool that aids in their collection of data, and (b) knowledge and skill for translating what they saw and data collected into meaningful messages the teacher can use. COACHED was created to help address both issues.
What do we know about alliance building in coaching?
Although the role of coach can be played by any number of professionals in a school, several challenges to implementing and sustaining effective coaching programs have been identified. One example includes the narrow focus of coaching approaches, where teachers and students are only supported in specific content areas, single target behaviors, or individual student problems - rather than addressing interrelated, academic and behavioral concerns (Reddy et al., 2017). Though these issues exist, identifying the core, evidence-based components of coaching and weaving these areas of need with programs can potentially overcome challenges (Bradshaw et al., 2018).
Introducing COACHED
The purpose of this Technology in Action piece is to introduce a new technology-based suite of tools called

Typical coacing cycle with COACHED.
COACHED can be used flexibly across age and grade levels given its foundation in theory and empirical literature around effective coaching, feedback, and professional learning. To see and test the functionality of COACHED with sample teacher coaching records, readers can log in to a test account by going to https://coachedweb.azurewebsites.net/ and using the following credentials: Username: COACHED@test.com; Password: COACHEDtest.
Classroom Teaching Scan
The Classroom Teaching (CT) Scan (Kennedy et al., 2017) is a technology-based observational tool that allows the observer to collect objective real-time data on a range of academic and behavioral practices in the classroom. The low-inference nature of the CT scan enables the observer to collect detailed data on distinct teacher practices, student actions (e.g., engagement), and classroom contextual features (e.g., co-teaching models). Time-stamped data from the CT Scan are captured and uploaded within COACHED, automatically generating detailed, yet teacher-friendly feedback. The data from the CT Scan is used to generate two main forms of feedback: a detailed narrative (see Figures 2 and 3) and a visual timeline (see Figure 4) of the observation .

Feedback form feedback on practices.

Feedback form narrative summary and counts.

CT scan time line.
To use the CT Scan the observer begins by entering observation and classroom details into the registration form. Next the observer uses the main CT Scan interface (see Figure 5) to begin the observation and capture detailed information based on the lesson content. The observer selects the broad category being taught (e.g., vocabulary instruction, explicit reading instruction, behavior management), and then a specific teacher practice within the broad category (e.g., teaches/reviews expectations, uses contingency or reinforcer, active supervision). The observer can easily and quickly change categories and practices to match teacher moves throughout the lesson using the dropdown menu and confirming the new practice. The CT Scan automatically records the length of use for each category and practice. The observer can note the specific instructional topic and/or vocabulary being addressed at any given point throughout a lesson, this information is time stamped below the category and practice in the timeline. Finally, to monitor implementation fidelity, each practice has a list of implementation markers to select from as they observe these markers occurring (see Figure 6). Based in literature pertaining to each classroom practice, implementation markers describe specific teacher actions or “look-fors” during an observation. Whether each implementation marker is observed or not is used to automatically generate the coaching feedback following the observation.

Classroom teaching (CT) scan interface.

Detail of implementation markers.
The CT Scan also allows the observer to simultaneously collect data on the number and types of discrete actions employed by teachers and students including: (a) opportunities for students to respond (OTRs; e.g., deep/probing questions, rote questions, choral, non-academic); (b) praise (academic-specific, behavior-specific, generic); (c) precorrection/prompts; (d) student behavior (e.g., disruptions); and (e) student-generated questions. As the observer selects each behavior, the CT Scan automatically tracks the number and timestamp of each occurrence. The CT Scan interface also allows the observer to document the context of the teacher’s lesson by tracking the type of teacher (e.g., general or special educator), who the teacher is providing instruction to (e.g., individual student, whole group), co-teaching model, and visual aids being used (e.g., computer, graphic organizer). At the same time, the observer can also track student actions (e.g., listening, writing, group work) and the percentage of students who are engaged throughout the observation. Finally, the observer can record qualitative, time stamped notes for any additional information. All of these data are presented in a teacher-friendly timeline (see Figures 7 and 4) that overlays each observed item (e.g., category and practice) with others that occurred at the same time (e.g., OTRs, student actions). The timeline output includes graphs that display a breakdown of teacher and student actions during the lesson as well as the counts and rates per minute of types of OTR, feedback, prompts, and student questions. Finally, these data are also used to automatically generate a narrative feedback.

CT scan pie charts of how the teacher and students spent their time during the lesson, and raw counts of countable events during lesson including rate per minute.
Feedback Reports
Once an observation using the CT scan is complete, data is saved to a secure online database within COACHED and the observer can move on to the next phase of the coaching cycle (see Figure 1). Coaching feedback reports are automatically generated from the observation data, and the observer can use the online form with pre-populated fields to edit where needed (see Figure 2). COACHED automatically fills in information on the categories, practices, implementation markers, OTRs, feedback statements, visual aids, and student actions. The only additional writing components required of observers is a brief narrative summarizing the lesson and a description of “one big thing” to work on before the next observation. Providing detailed feedback a once time intensive process, now takes only a few minutes.
The feedback form displays, in chronological order, the categories and practices that were observed during the lesson (see Figure 2). Since much of the feedback report is automatically filled in from CT Scan data this saves the observer/ coach copious amounts of time and reduces error. For example, detailed feedback sentences are automatically input into the feedback form based on which implementation markers were observed and not observed. By presenting the observation data in both the narrative feedback format and the visual CT Scan timeline, teachers get a detailed picture of what occurred during the observation.
The practices are displayed in individual boxes with associated data (e.g., duration of practice use). Below each practice in the box are the implementation markers. The automatically generated feedback for implementation markers are randomly selected from a comment bank to ensure the feedback is unique and not repetitive. Each implementation marker has several prewritten sentences for both observed and unobserved instances. To maintain the flexibility of the COACHED app, the bank of feedback sentences can be updated directly with in COACHED. Within each practice box, teachers will also see a breakdown of the number and types of OTRs and feedback statements used during that practice as well as visual aids used and student actions (e.g., taking notes, listening). Each practice box provides teachers and observers with an in-depth look at what was occurring during that practice.
Further, a section at the bottom of the feedback form for observers to provide a short, personalized narrative summarizing the observation and setting goals for the teacher (see Figure 3). In this narrative summary, observers may also choose to reference multimedia PD videos, available next to each practice box and described in the following section, for practices where improvements are needed. The feedback form is also where teachers can directly access their CT Timeline for that specific observation. Additionally, total summary counts of types of OTRs and Feedback Statements are listed. While teachers can access their feedback via logging into their individualized COACHED accounts, observers also have the option of downloading a pdf of the feedback or sending a link directly to the teacher’s email account. The entire process of generating and sending feedback has been greatly reduced, limiting the burden on observers. Additionally, the feedback does not contain a “quality” score and the observers rely on using observed data in generating the summary narrative. This style of data-based feedback enables the maintenance of a positive teacher-observer alliance.
Multimedia Modules
Following the observation of a teacher using the CT Scan and delivery of feedback, the final element of COACHED is to provide on-demand PD for areas of improvement.
CAP-TVs are multimedia modules developed to reflect Mayer’s (2020) Cognitive Theory of Multimedia Learning and associated instructional design principles. This theoretical model and design principles and their relation to CAP-TVs are described elsewhere (Rodgers et al., 2017). The key is that the videos are short (5–15 min); and use rich images with accompanying key text to provide the user with a streamlined lesson on a specific topic or practice. The short in length allows teachers to engage with the PD easily and on their own schedule.
For example, this sample CAP-TV (https://vimeo.com/448122821) in part 1 (0:00-7:00) introduces key vocabulary and elements of a practice. Part two (7:01-end) shows a model teacher implementing the practice. CAP-TVs are assigned to teachers within COACHED as an immediate opportunity to receive PD. The modules can also be posted elsewhere online or on course management sites.
Evidence for COACHED Components
The three core components of the COACHED app: the CT scan, performance feedback based on CT Scan data, and CAP-TVs have demonstrated effectiveness in improving both teacher and student outcomes. Kennedy and colleagues (2017; 2018) found, that when teaching vocabulary, inclusive middle school teachers of students with disabilities who were observed and received feedback using the CT Scan tended to use vocabulary instructional practices with increased frequency and quality relative to baseline and peers in other comparison groups (Kennedy et al., 2017, 2018). At the same time, both students with and without disabilities in these same classrooms performed better on curriculum-based measures of science vocabulary compared to their peers with and without disabilities in business-as-usual classrooms whose teachers did not receive CT Scan feedback and CAP-TV modeling. Recent work also suggests that pre-service teachers who received CT scan feedback incorporated more explicit instruction practices into their lessons (Peeples et al., 2018). Teachers have also reported a preference for the objective and visual feedback from the CT Scan timeline and associated feedback forms as opposed to arbitrary quality scores and more subjective feedback (Kennedy et al., 2017, 2018; Peeples et al., 2018).
Content acquisition podcasts (CAPs) have been used with both pre- and in-service teachers with promising results. In a recent randomized control trial, Romig et al. (2018) found that pre-service teachers who used CAP-TVs vs lecture or reading demonstrate significant improvements in their writing instruction. Pre-service teachers also report a preference for CAPs, reporting overall satisfaction as well as their usefulness for teaching students with disabilities (Kennedy et al., 2014). CAPs are flexible and easily integrated into classrooms, and teachers have also been successful in designing and creating their own CAPs (Kennedy et al., 2014, Weiss et al., 2016). Finally, teachers who received training in the form of CAP-TVs have shown improvement in their use of evidence-based instructional strategies (Ely et al., 2015).
How COACHED Can Help Administrators, Coaches, Teacher Educators, and Teachers
Each component of COACHED helps to address the core components of effective coaching and professional development noted by the State Implementation and Scaling up of Evidence-based Practices Technical Assistance Center ([SISEP] Cusumano & Preston, 2018) and National Center for Systemic Improvement (NCSI, 2016). The CT Scan helps the professional conducting the observation capture the exact practice that was used and key ancillary data. The CT Scan is superior to checklists or simple notetaking as it allows the observer to focus exclusively on the practice(s) being implemented, in addition to the overall fidelity of practice implementation. Additionally, with the ability to capture data from both academic and behavioral domains, the CT scan addresses the issue of the narrow focus of other coaching programs.
The performance feedback automatically generated by COACHED saves the observer significant time, and ensures educators being observed receive feedback on the practices they actually used. The reduced burden on observers can serve as a benefit for schools who may be struggling with access to resources and personnel and have limited time to provide teachers with quality feedback. Recommendations in the SISEP (Cusumano & Preston, 2018) and NCSI (2016) also call for feedback that is presented in multiple formats (e.g., narrative, graphical, visual; Neuman & Cunningham, 2009, Snyder et al., 2015). The CT scan timeline and the coaching feedback form address this by providing teachers with feedback in graphical form (i.e. pie charts), a timeline visual format, and with written and numerical feedback.
As noted previously, quality feedback should maintain a focus on the implementation of specific practices rather than evaluation (Duchaine et al., 2011; Solomon et al., 2012), as this can help with the alliance between the observer and teacher when the teacher does not feel their practice is being judged (NCSI, 2016). The feedback generated from COACHED does not use an evaluative quality score. Rather, data are reported on the practices used and observers can use those data to provide teachers with summary feedback and specific goals for improvement.
Further, CAP-TVs provide teachers with on-demand PD that can be viewed when needed or desired, re-contextualizing the often disconnected format of typical PD sessions. These vignettes are relatively easy to create, and do not carry costs other than the time to create them. A library of free CAP-TVs are available at www.spedintro.com. The CAP-TVs provide teachers with on-demand modeling that they can access for targeted practices. New videos can be created by instructors or other leaders for specific practices or other topics of need.
Conclusion
COACHED is a free, multimedia option for administrators, coaches, teacher educators, and teachers of any grade level or content area to conduct observations, receive coaching, and on-demand PD based on specific data captured during lessons. The system is flexible in use, and customizable to suit a range of users’ needs. A range of future research studies will help demonstrate the impact of this tool on various important outcomes for educators and students with disabilities.
Footnotes
Authors’ Note
We would like to extend a thank you to Chris DiVita and James Field of ITS Custom Applications and Consulting Services (CACS) for their continued work in develping the COACHED app. We would also like to acknowledge Dr. Bill Ferster for his work in coding the orignal CT Scan observation tool.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by grants from the U.S. Department of Education’s Office of Special Education Programs (H327S190004).The opinions expressed by the authors are not necessarily reflective of the position of or endorsed by the U.S. Department of Education.
