The Impact of Real-Time Feedback and Video Analysis on Early Childhood Teachers’ Practice

Abstract

We used a multiple-probe single-case research design to examine the effect of a professional development package that included real-time, technology-enhanced, and performance-based feedback and video analysis on three preschool teachers’ use of naturalistic instruction targeting children’s communication and child responses. We also measured the quality of the teachers’ naturalistic instruction targeting children’s communication. The professional development package was effective in enhancing teachers’ use of naturalistic instruction targeting children’s communication. In addition, children responded and teachers enhanced the quality of naturalistic instruction targeting children’s communication. Teachers maintained their use of naturalistic instruction targeting children’s communication upon removal of the intervention condition.

Keywords

technology-enhanced performance-based feedback video analysis professional development eCoaching naturalistic communication strategies reflection

One evidence-based intervention to improve children’s skills is naturalistic instruction (NI; Fox & Hanline, 1993; Snyder, Hemmeter, & Fox, 2015). NI involves using everyday activities to intentionally provide children opportunities to practice target skills within contextually relevant routines (Division for Early Childhood, 2014; Grisham-Brown, Hemmeter, & Pretti-Frontczak, 2005; Snyder, Rakap, Hemmeter, et al., 2015). Implementation of NI targeting child communication has supported the maintenance and generalization of children’s communication (Kaczmarek, Hepting, & Dzubak, 1996; Snyder et al., 2015). Time delay via choice making (providing choices) and mirroring and mapping (imitating the child’s actions and modeling language) are two examples of NI targeting children’s communication development (Kaiser & Hampton, 2017).

Although NI targeting children’s communication development has demonstrated effectiveness in improving target skills, research findings suggest that teachers experience challenges aligning these practices with children’s target skills (Rahn, Coogle, & Ottley, 2019). In addition, there is variability in teachers’ use of NI (Noh, Allen, & Squires, 2009; Rahn et al., 2019). Thus, there is a need to identify effective professional development (PD) practices that support teachers’ intentional use of NI.

PD in early childhood settings traditionally takes place outside of the classroom using didactic trainings (Snyder, Hemmeter, & McLaughlin, 2011). Although this type of PD may increase knowledge, it is ineffective in changing practice (Snyder et al., 2011). Conversely, effective PD is interactive, collaborative, and ongoing, and it aligns with target goals (Hemmeter, Snyder, Kinder, & Artman, 2011). Embedding PD within the classroom to provide relevant application and ongoing support is critical to enhance the use and fidelity of evidence-based practices (Snyder, Hemmeter, Fox, 2015).

Researchers have identified PD packages that can be used within the context of early childhood classrooms (Artman-Meeker, Hemmeter, & Snyder, 2014; Barton et al., 2018; Fox, Hemmeter, Snyder, Binder, & Clarke, 2011; Hemmeter et al., 2011; Snyder et al., 2018). For example, Fox and colleagues (2011) used workshops and face-to-face coaching to support inclusive early childhood special education (ECSE) teachers’ use of pyramid model practices (Fox et al., 2011). These teachers participated in a workshop and received coaching that included goal setting and action planning within their classroom until they met an established criterion level of implementation. This PD package was effective in enhancing teachers’ use of pyramid model practices.

Snyder and colleagues (2018) compared the effect of PD packages including PD with no coaching, self-coaching, or on-site coaching on teachers’ use of embedded instruction. Teachers in the self-coaching condition received materials and emails to prompt self-coaching, and teachers in the on-site coaching alternated between face-to-face coaching and electronic coaching sessions (email, video conferencing). During the on-site coaching sessions, the coach and teacher discussed embedded instruction, the coach provided performance feedback, and the coach and teacher developed goals and action plans together. The face-to-face coaching condition resulted in significantly more embedded instruction and the children in their classrooms used significantly more learning targets compared with teachers in the other two conditions.

Researchers have also used a combination of approaches paired with technology-enhanced performance-based feedback (TEPF) to enhance participants’ use of target practices. For example, email-delivered feedback has been effective for increasing teachers’ use of NI (Coogle, Ottley, Storie, Rahn, & Burt, 2018), pyramid model practices (Artman-Meeker et al., 2014), descriptive praise (Hemmeter et al., 2011), and practices related to the Division for Early Childhood’s recommended practices (Barton et al., 2018). Although the specific procedures used across these studies differed, technology was the active intervention component used to increase the efficiency of PD. For example, researchers have provided training and engaged in face-to-face observations (Hemmeter et al., 2011) or video-recorded observations (Artman-Meeker et al., 2014). Other scholars have prompted participants to self-select target practices, conducted face-to-face observations, provided email feedback, and have used individual adaptations (e.g., an individual meeting and goal setting) to support participants’ use of target practices (Barton et al., 2018).

Researchers have also examined the effects of conducting observations and providing real-time TEPF from an alternate location using a video conferencing system (e.g., Skype) paired with real-time TEPF via a Bluetooth device (Coogle, Storie, Ottley, Rahn, & Burt, 2019). Real-time TEPF is feedback that teachers receive “in-the-moment” while they are interacting with children. Conducting observations from an alternate location paired with real-time TEPF has increased teachers’ use of naturalistic communication strategies during the typical routines of early childhood classrooms with children who have varying communicative needs (Coogle, Ottley, Rahn, & Storie, 2018; Coogle, Ottley, Storie, Rahn, & Burt, 2017; Coogle et al., 2018; Coogle, Rahn, & Ottley, 2015; Coogle, Rahn, Ottley, & Storie, 2016; Coogle et al., 2019).

Video-based technologies have also been used within PD systems. Broadly, video-based activities such as video club that emphasizes peer mentoring (van Es & Sherin, 2010), microteaching that emphasizes coaching with feedback (Amobi, 2005), and video analysis that emphasizes self-reflection (Nagro, & Cornelius, 2013; Nagro, deBettencourt, Rosenberg, Carran, & Weiss, 2017; Nagro, & deBettencourt, 2018) all show promise for improving teacher behaviors. Whereas both video club and microteaching include the provision of feedback from others, video analysis is self-facilitated. During video analysis, teachers record their interactions with children, watch the video evidence to reflect on specific aspects of their interaction, and then identify areas of strength as well as areas for improvement. This process of record, review, reflect, and revise can be repeated over time so that teachers become more comfortable watching themselves on video, learn to notice elements of their interaction with children, and reflect on how they might improve these interactions (Nagro et al., 2017; Nagro & deBettencourt, 2018). For example, in one study, an elementary teacher candidate completing a clinical experience with students with moderate cognitive disabilities engaged in video analysis as a method for self-mentoring and was able to meet criteria and maintain improvements in (a) the number and variation of praise statements given during a lesson, (b) the rate of opportunities for student response, (c) the rate of visual scanning of the room, (d) the ratio of praise to redirection statements, and (e) implementation fidelity of all steps outlined in the lesson as a result of engaging in video analysis activities (Hager, 2012). Video analysis has also been studied in early childhood settings to document the potential of this approach on teacher practice (e.g., Nagro et al., 2017; Nagro & deBettencourt, 2018; Pianta, Mashburn, Downer, Hamre, & Justice, 2008; Wright, Ellis, & Baxter, 2012).

The flexibility of video technology, combined with watching one’s own video, allows teachers to review concrete data rather than relying on memory alone (Seidel, Stürmer, Blomberg, Kobarg, & Schwindt, 2011; Wang & Hartley, 2003). Once teachers learn to notice elements of their interaction using video evidence, they can focus on the quality of their interactions through reflection activities (Deniz, 2012; Nagro et al., 2017; Nagro & deBettencourt, 2018). However, without guidance on how to reflect during video analysis, teachers tend to focus on superficial aspects of the profession and are less likely to improve their own abilities (Calandra, Gurvitch, & Lund, 2008; Kalk, Luik, Taimalu, & Täht, 2014; Nagro et al., 2017; Nagro & deBettencourt, 2018). One approach to guiding teachers to reflect on the quality of their interactions with children is to provide teachers with a framework (e.g., The Danielson Group, 2017). Teachers can use a framework, portion of a framework, a checklist, or a matrix that includes operational definitions of teacher behaviors, quality indicators, or examples and non-examples to guide their own video review process (Nagro et al., 2017; Nagro & deBettencourt, 2018). This approach creates a structured learning experience because teachers review their video evidence looking for and reflecting on predetermined teacher behaviors outlined within the framework rather than allowing for open-ended video review and reflection.

Despite the documented effectiveness of real-time TEPF and video analysis in select contexts, the use of these activities is well ahead of supporting scientific evidence (Nagro et al., 2017; Nagro & deBettencourt, 2018; Smith & Kennedy, 2014). Accepting the newest technology-based educator training tools without appropriate research and empirical evidence of effectiveness can inadvertently have negative impacts on teacher candidates (Dieker et al., 2014; Hicks, Lee, Berson, Bolick, & Diem, 2014). In a review of 21st-century change drivers for transformative special education teacher education, Rock and colleagues (2016) explained:

When considering how transformative models could be supported and strengthened through technology, developers should proceed with caution to ensure that it is embedded thoughtfully, enabling special education teacher educators to work more effectively and efficiently. Otherwise, technology could inadvertently exacerbate workload demands and increase stress levels (p. 104).

To the best of our knowledge, researchers have not studied real-time TEPF in relation to quality of teacher practice. In addition, the body of research on video analysis has been largely incoherent due to major discrepancies in the operational definition of video analysis and dependence on perceived change as a proxy for actual improvements in teacher behaviors (Nagro, & Cornelius, 2013; Nagro et al., 2017; Nagro & deBettencourt, 2018; Tripp & Rich, 2012). These gaps in research limit what is known about potential impacts of these PD approaches. Examining these two approaches as one comprehensive PD package offers a unique opportunity to examine the effects on both the frequency and quality of target teaching practices including drawing connections to child outcomes. Therefore, we combined real-time TEPF and video analysis to determine the effect on teachers’ use of NI targeting children’s communication, the quality of the teachers’ NI targeting children’s communication, and the percent of opportunities in which children responded. The research questions we sought to answer included:

Research Question 1: Is there a functional relation between a real-time TEPF and video analysis PD package on the frequency of teachers’ use of NI targeting children’s communication?

Research Question 2: Is there a functional relation between a real-time TEPF and video analysis PD package on the quality of teachers’ NI targeting children’s communication?

Research Question 3: To what extent do children respond using functional communication when their teachers use NI targeting children’s communication?

Method

Participants and Setting

Inclusion criteria were (a) being a preschool teacher and (b) having children in their class who had an individualized education plan (IEP) with goals related to expressive communication. The first author recruited participants from an early language and literacy class that she taught. Each teacher indicated a willingness to participate in the study. Participants were three female, ECSE teachers (Andreia, Hadi, and Abigail), who taught in public preschool classrooms and were all working on graduate degrees and certification in ECSE. Andreia was Black and Hadi and Abigail were White, non-Hispanic teachers. Andreia was 32 years old with 2 years of teaching experience. Hadi was 53 years old with 11 years of teaching experience. Abigail was 51 years old with 2 years of teaching experience. Andreia and Hadi’s classrooms included eight children all with IEPs; Abigail had 12 children, 10 with IEPs.

Each participating teacher chose a focus child who had an IEP goal related to using expressive communication. The focus child in Andreia’s classroom was a 3-year-old male who was of Hispanic descent. Andreia’s focus child had a developmental delay. His primary language at home was Spanish. He was imitating approximately three words, and he would make requests when prompted, but he was not yet spontaneously using words. He would physically engage in social activities such as dramatic play; however, he experienced challenges communicating appropriately with other children. The focus child in Hadi’s classroom was a 3-year-old male who was of Bengali descent, and he was identified with autism spectrum disorder (ASD). He was using approximately 100 words in Bengali and English. Much of his communication was difficult to understand, and he was not yet using his words with a communicative purpose. He was not yet engaging with others, but rather was interested in repeating similar routines with familiar play objects such as stacking magnetic tiles. During these types of play routines, he would vocalize; however, these vocalizations were not yet directed toward another person. They were also not yet understood by the teacher or other children. The focus child in Abigail’s classroom was a 3-year-old male who was of Chinese descent. His parents used English and Mandarin equally until the child was 2 years, and then began using English as their primary language at home. He was identified with ASD. He did engage with his peers, and these relationships were highly motivating for him in terms of communication. His needs were that he continued to need support with his functional language. He was not yet answering wh-questions. He would often respond by echoing parts of the question in his answer, and he would focus on certain subjects and ask the same question repeatedly.

All sessions included the focus child, their teacher, and sometimes their peers. All sessions took place when and where they typically would, had the research not been occurring. The teachers were asked to choose activities in which they would like support, and all teachers selected teacher- and child-led routines. Teacher-led routines were group sessions where the teacher had a specific plan for the interaction such as a small-group game. Child-led routines included activities where the teacher followed the child’s lead such as morning free play.

Research Design

We used a multiple-probe single-case research design, because we did not expect behaviors to change without intervention, and a multiple-probe design allows for intermittent collection of data points as opposed to continuous data collection in multiple-baseline designs, which conserves the resources available to the research team, and more feasible as it is less time intensive (Kratochwill et al., 2013; What Works Clearinghouse, 2014). We included three opportunities for replication across the three participating teachers. In addition, we met all requirements for What Works Clearinghouse (2017) standards version 4.0 with reservations. We systematically manipulated the independent variable, included at least three sessions within each condition, measured each outcome variable using more than one observer, and established interobserver agreement (IOA; Kratochwill et al., 2013).

Measures

To be more sensitive to the occurrence of dependent variables, sessions were divided into 5-s intervals for a duration of 6 min (Yoder, Ledford, Harbison, & Tapp, 2018). We selected partial-interval coding because our previous research revealed that when using distance observations video lengths can vary and partial-interval coding allowed us to account for any potential variance that might occur across sessions. Moreover, in both our previous research and this study, we found partial-interval recording to be the best estimate of count. In our partial-interval coding system, we only coded a behavior during the interval in which it began to ensure that our data accurately reflected the dependent variables without inflating the frequency in which the behaviors occurred if the duration was across two intervals. We also used the Poisson correction method to control for error that may occur when using partial-interval coding (Yoder et al., 2018).

Teacher outcomes

We selected choice making (Andreia and Abigail) and mirroring and mapping (Hadi) as the specific NI practices targeting children’s communication as these were strategies the teachers were not yet using and which aligned with the children’s IEP goals. Choice making took place when the teacher verbally labeled choices without a correct or incorrect response (e.g., “Blocks or Legos?”). Mirroring and mapping took place when the teacher imitated what the child was doing and provided language (e.g., after the child tapped the blocks together the teacher tapped the blocks together saying, “tap blocks”). We measured both prompted (i.e., using NI targeting children’s communication when the coach provided a suggestion [this only took place during intervention]) and unprompted (i.e., using NI targeting children’s communication without the coach providing a suggestion).

Quality use of NI targeting children’s communication

We coded the quality of each NI strategy targeting children’s communication the teacher used (Kaiser & Hampton, 2017). The quality criteria included (a) environment (arranging the environment to promote choices and imitation [e.g., multiple materials available to offer choices or imitate]), (b) wait time (pausing after offering a choice or using mirroring and mapping to provide the child the opportunity to communicatively respond), (c) child response (child using communication after the delivery of choice making or mirroring and mapping), and (d) teacher response (when the child responded, the teacher responded by providing a desired choice or responding to the child’s communication). We also identified strategy-specific criteria. For choice making, this included providing different types of choices and labeling two choice options. For mirroring and mapping, this included imitation and using one- to two-word language models. For each NI strategy targeting children’s communication that the teacher used, we computed a quality score by determining the percentage of the quality indicators that were met. Because some of the quality indicators were dependent on the prior quality indicator occurring, if we did not score one of the previous quality indicators, then those following were scored as not applicable and, therefore, were not computed within the percentage. For example, with choice making this included wait time, child response, and access. If wait time was not provided, then the child did not have the opportunity to respond nor did they have the opportunity to receive access so child response and access would have been coded as not applicable. See Table 1 for the quality criteria.

Table 1.

Quality Indicators for Teacher’s Use of Naturalistic Instructional Strategies Targeting Children’s Communication Development.

Quality Indicators	Definition	Choice making	Mirroring and mapping
Environment	Setting up the environment to use the target naturalistic communication strategy by having materials to provide choices or to imitate the child	X	X
Wait time	Pausing after the teacher used a strategy to provide the child an opportunity to respond	X	X
Child response	The child’s use of functional communication via gestures, vocalizations, or words in response to the naturalistic communication strategy	X	X
Teacher response	If the child responded, we coded whether the teacher reinforced the child’s response by providing them access to a choice they made or responding to the child’s communication	X	X
Different types of choices	Choices about materials, choices about roles in a play routine, and choices about what activity the child would like to do	X
Labeling two choices	Using words to label two choices	X
Physical imitation (mirroring)	Physically imitating what the child did		X
Language model (mapping)	Using one to two words surrounding physical action		X

Note. X indicates whether each of the quality indicators was measured for choice making, mirroring and mapping, or both naturalistic instructional strategies.

Child responses

We coded whether each focus child responded to their teacher’s use of NI targeting children’s communication. After the teacher used NI targeting children’s communication, a response occurred when the child communicated via gestures, vocalizations, or words according to the coding definitions within the Individual Growth and Development Indicators: Early Communication Indicator (Juniper Gardens Children’s Project, 2011). Gestures are physical movements to communicate with another individual. Vocalizations are utterances that are not understood words. Single words are one word used in isolation, and multiple words are two or more understood words used in combination.

Interobserver agreement (IOA)

We video-recorded each session and a trained graduate student coded each video. The coach double coded all videos to ensure accuracy and IOA. We used the point-by-point method for a random 24% of videos across all participants and conditions to calculate IOA (Cooper, Heron, & Heward, 2007; Lane & Ledford, 2014). Overall, IOA was 91% (range = 60%–100%).

Materials

The coach provided each teacher with an iPad mini, Bluetooth® earpiece, and a Swivl™ (held the iPad and tracked movement). The coach used Google Hangouts™ to provide real-time feedback during intervention sessions and used Camtasia software to record all sessions. This allowed the coach to download the video-recording of the call immediately following each session for coding purposes.

Procedures

Approval was obtained from the university institutional review board as well as informed consent from the teachers and guardians of the children in each classroom. During each session across all conditions, the teachers placed the iPad in the Swivl™ and placed the Bluetooth® device in their ear. They used the iPad to call the coach using Google Hangouts™. Then the teacher began engaging with their focus child and any other children present at that time.

The fifth author used an online random number generator to randomize the order in which each participant received intervention (Randomness and Integrity Services Ltd., 2019). Randomization in the order in which each participant started intervention and replication across the three teachers increases confidence that the results were solely due to the intervention. We included at least three sessions within the baseline and intervention conditions, and we planned to collect three maintenance probes so that we could visually analyze the data. Due to child absences, one teacher was only able to complete two maintenance probes.

Baseline

During baseline, each teacher interacted with her focus child as she typically would. The coach did not provide feedback. The number of baseline sessions varied from three to five sessions. As a research team, we watched the baseline videos and conferenced about the most appropriate NI strategy targeting children’s communication for each participant based upon the teacher’s use of NI targeting children’s communication and child’s current communication development, and their IEP goals.

Intervention

At the start of intervention, the coach had 8 years of experience providing coaching for personnel preparation, PD, and research-related purposes. Prior to real-time TEPF, each teacher had a goal-setting meeting with the coach using Google Hangouts™. During this meeting, the coach suggested an NI strategy targeting children’s communication and discussed quality criteria with the teacher. All teachers agreed with the NI strategy targeting children’s communication and the quality criteria that the coach suggested. Using the randomized multiple-probe staggered schedule, each teacher received real-time TEPF (Kratochwill et al., 2013).

Real-time TEPF included prompts (suggestions to use the NI strategy targeting children’s communication) and comments (statements regarding the teachers’ use of their NI strategy targeting children’s communication). When teachers used their NI strategy targeting children’s communication, our comments were affirmative (praised teacher’s use of their NI strategy targeting children’s communication) and, as needed, corrective (providing guidance regarding how the teacher might use the NI strategy targeting children’s communication to align with the quality criteria). The delivery of our real-time TEPF prompts and comments aligned with that of Scheeler, Ruhl, and McAfee’s (2004) recommendations. Both prompts and comments were specific to what was taking place within the observation, and prompts and comments were immediate (provided in real time).

During the first minute, the coach only provided affirmative comments so that she could observe whether the teacher used an unprompted NI strategy targeting children’s communication; this also allowed the coach to understand the nature of the activity prior to providing prompts. For the remaining 5 min, prompts and comments were provided to support the teachers’ use of their NI strategy targeting children’s communication. The coach intended to deliver feedback at a rate of one feedback per minute. If the teacher used his or her NI strategy targeting children’s communication without prompts, then the real-time TEPF for that minute was at least one affirmative comment. See Table 2 for the types of feedback statements that the coach delivered across sessions and the number of prompted and unprompted strategies the teachers used.

Table 2.

Types of Teacher Behaviors and TEPF Statements per Intervention Session.

	Agatha			Hadi			Abigail
Session	Prompted		Unprompted	Prompted		Unprompted	Prompted		Unprompted
Teacher behaviors
1	0		2	0		32	0		3
2	1		6	0		25	0		4
3	1		7	0		56	1		3
4	1		5	0		39	0		8
5	0		8	0		19	0		18
6	0		3
7	2		7
8	0		22
	Agatha			Hadi			Abigail
	Affirm	Corrective	Prompt	Affirm	Corrective	Prompt	Affirm	Corrective	Prompt
TEPF statements
1	2	1	2	8	0	0	3	0	0
2	7	0	1	11	0	0	3	1	1
3	9	0	1	14	1	0	4	0	1
4	4	0	1	23	0	0	8	0	0
5	8	1	0	13	0	0	16	0	0
6	3	0	0
7	9	0	2
8	16	0	0

Note. Teacher behaviors are the number per intervention session. Teacher behaviors were either prompted (i.e., demonstrated in response to a prompt from the coach) or unprompted (i.e., demonstrated without a prompt from the coach). TEPF statements were any statements used by the coach and were either affirmative statements, corrective statements, or prompts to use the target behavior. TEPF = technology-enhanced performance feedback.

In addition to providing real-time TEPF, upon completion of each observation session, the coach sent an email to participating teachers to use their attached video file and reflection matrix to reflect upon their session that just occurred (see the reflection matrix in Figure 1). Without the necessary tools and structured guidance on how to reflect, teachers tend to focus on summarizing events superficially (Calandra et al., 2008; Kalk et al., 2014; Nagro et al., 2017; Nagro & deBettencourt, 2018). Therefore, we opted to use a matrix that included both types of reflective practice across a continuum (describe, analyze, judge, apply) and topics for reflection. Specifically, we introduced four types of reflective practice where teachers were asked to describe their teaching choices, analyze why such choices were made, judge the success of those choices based on child outcomes, and apply insights to plan for future lessons (see Nagro et al., 2017). We then paired the continuum of reflective practice with the specific elements of the NI strategy targeting children’s communication (e.g., choice making: managing the environment, creating scenarios for choice, providing two choices, providing wait time, and providing access). This matrix provided teachers with a tangible approach to video analysis because they knew what to focus on when watching their video-recorded lesson and how to reflect on the quality of their target teaching practices using the continuum of reflective practice. Participating teachers engaged in video analysis within 72 hr of teaching so that the lesson they were reviewing was still relevant. Then the teacher emailed each completed reflection matrix to the coach before their next TEPF session.

Figure 1.

Reflection Matrix.

Maintenance

Once all participants completed intervention, maintenance began when both the teacher and focus child were available. We collected maintenance probes based on child and teacher availability. For Andreia, maintenance began 2 days after intervention, and Hadi and Abigail began maintenance 8 days after intervention was complete. We used the same procedures as during the baseline condition. Hadi and Abigail were able to complete three maintenance probes. However, due to focus child absences, Andreia only completed two maintenance probes.

Data Analysis

We visually analyzed the data for level, trend, immediacy of an effect, overlap of data, and variability (see What Works Clearinghouse, 2017, Standards Version 4.0 for a description of each visual analysis component), and we considered consistency across participants (Kratochwill et al., 2013). We used our primary dependent variable (each teacher’s use of their NI strategy targeting children’s communication) to make decisions regarding changes in condition. We analyzed teachers’ quality use of their NI strategy targeting children’s communication, and we computed the percentage of NI targeting children’s communication that resulted in child responses. We descriptively analyzed social validity data to describe the acceptability, feasibility, and perceived effectiveness of the intervention.

Social Validity

To determine the perceived feasibility and effectiveness of real-time TEPF paired with video analysis, all teachers completed an electronic questionnaire that included seven open-ended questions and 24 Likert-type scale questions. Questions were related to the importance of the feedback as a component of PD for special educators, the extent to which they would recommend this intervention to other educators, the degree to which feedback helped them gain new skills, and the feasibility of feedback and video analysis.

Procedural Fidelity

The third author developed a fidelity checklist to determine the procedural fidelity of implementation (FOI) of our study procedures across all conditions. The checklist included indicators for the following: (a) feedback was delivered during the appropriate conditions (intervention only), (b) the session took place during a typical routine, (c) the video included the teacher and focus child, (d) the session included positive comments (intervention only), (e) the session included corrective comments (intervention only), (f) feedback was descriptive based upon the participant’s needs (intervention only), (g) statements were provided during pauses in each teacher’s interaction (intervention only), (h) statements targeted the specific naturalistic communication NI strategy targeting children’s communication (intervention only), (i) the amount of feedback was adjusted based on the teacher’s use of their NI strategy targeting children’s communication (intervention only), (j) the coach indicated when the session was over (intervention only), (k) the coach posted the video for the teacher to view (intervention only), and (l) the teacher completed a written reflection using the matrix template (intervention only). We identified yes when the indicator took place (e.g., prompt was provided that was specific, comment was offered). We identified no when an indicator took place, but did not meet criteria (e.g., affirmative comment was offered but it was not specific, corrective comment was offered but it was not specific). We identified not applicable when there was not an opportunity for the indicator to take place (e.g., if the teacher was using the NI strategy targeting children’s communication throughout the session, there was not an opportunity to provide a prompt). FOI was 98% (range = 80%–100%) across all conditions.

Results

Teachers’ Use of NI Strategies Targeting Children’s Communication

Andreia

As shown in Figure 2, during baseline, Andreia rarely used choice making and demonstrated a decelerating trend (range = 0–2). Upon starting intervention, we observed a gradual increase with an accelerating trend (range = 2–22). There was one overlapping data point between baseline and intervention (Session 1). Andreia demonstrated an increase in level during intervention and maintained her use of choice making (range = 16–22).

Figure 2.

Graphed data for all participants across research conditions where open squares reflect the frequency in which teachers used naturalistic instruction targeting children’s communication (based on the left axis), open circles reflect the percentage of quality indicators met by teachers when using naturalistic instruction targeting children’s communication (based on the right axis), and closed triangles reflect the percentage of strategies in which children responded to their teacher’s naturalistic instruction (based on the right axis).

Hadi

During baseline, Hadi used mirroring and mapping in a minimal number of intervals (range = 0–3), with all occurrences taking place during Session 2 of baseline. Therefore, she demonstrated a decelerating trend. Upon beginning intervention, we observed an immediate increase in her use of mirroring and mapping (range = 19–56). Although Hadi demonstrated variability within intervention, she had no overlapping data with baseline. Hadi maintained her use of mirroring and mapping at a level slightly lower than intervention (range = 21–37).

Abigail

During baseline, Abigail infrequently used choice making and demonstrated a decelerating trend (range = 0–1). Upon beginning intervention, we observed an immediate increase in level and an accelerating trend (range = 3–18). Abigail had no overlapping data with baseline. Abigail maintained her use of choice making, and her level increased (range = 12–21).

Quality of the Teachers’ Use of NI Strategies Targeting Children’s Communication

Andreia

The quality of Andreia’s use of choice making in one observed session during baseline was 67%. During intervention, we observed an immediate increase in level with an accelerating trend (range = 75%–94%). This pattern continued during maintenance (range = 95%–98%). There was low variability and no overlap of data across conditions.

Hadi

The quality of Hadi’s use of mirroring and mapping during baseline was 33%. During intervention, we observed an immediate increase in level with minimal variability (range = 74%–92%). Her quality use during maintenance was at a level slightly higher than intervention (range = 83%–86%).

Abigail

Abigail’s quality use of choice making during baseline was high (83%). During intervention, her quality level increased (range = 83%–100%). Abigail maintained her quality use at a level consistent with intervention (range = 83%–100%).

Associated Child Responses

Andreia

During the initial baseline session, when Andreia used choice making, her focus child responded to 100% of opportunities. Throughout intervention, the focus child continued to respond to Andreia’s use of choice making (range = 63%–100%). During maintenance, child response data increased (range = 89%–92%).

Hadi

Hadi’s focus child had few opportunities to respond during baseline. When he was provided opportunities (Session 2), he did not respond. During intervention, Hadi’s focus child demonstrated an immediate increase in his responses (range = 8%–89%). Hadi’s focus child demonstrated no overlap with baseline and an accelerating trend. Hadi’s focus child continued to respond during maintenance at a slightly higher level than intervention (range = 47%–55%).

Abigail

Abigail’s focus child appropriately responded to 0% to 100% of opportunities during baseline. Upon beginning intervention, there was an immediate and stable increase in the focus child’s responses as her focus child responded to 100% of opportunities provided. Abigail’s focus child continued to respond to opportunities during the maintenance condition.

Social Validity

On Likert-type scale items, participants strongly agreed (M = 5; range = 5–5) on the social significance (i.e., receiving feedback is an important component of PD for special educators, and they would recommend this intervention to other special educators) and importance (i.e., feedback helped them gain new skills, they were confident they would continue using the skills without feedback, and children increased their use of expressive language) of the intervention. Participants also strongly agreed (M = 5; range = 5–5) on the appropriateness of procedures (i.e., TEPF sessions were individualized, technology made feedback manageable, and TEPF statements were easy to receive), and they strongly agreed (Hadi and Abigail) or disagreed (Andreia) (M = 4; range = 2–5) that accessing videos and completing reflections is feasible. Participants rated the real-time TEPF technology favorably, strongly agreeing (M = 5; range = 5–5) that receiving feedback via the technology was helpful. Hadi and Abigail disagreed and Andreia strongly disagreed (M = 1.67; range = 1–2) that wearing the real-time TEPF device was distracting. Last, participants reported many aspects of the video analysis component of the intervention that were extremely helpful (M = 5; range = 5–5). These included watching their own teaching to reflect on teaching choices, analyzing and judging teaching choices during reflection, applying insight from reflection to future instruction, narrowing the focus of their reflection via the reflection matrix, and engaging in video analysis frequently. They reported that writing reflections about their teaching choices were somewhat helpful (Hadi) or extremely helpful (Andreia and Abigail) (M = 4.67; range = 4–5). They indicated that reflecting on specific elements of instruction were somewhat helpful (Andreia) or extremely helpful (Abigail and Hadi) (M = 4.67; range = 4–5). Andreia reported that using the reflection matrix when writing video-based self-reflections was somewhat unhelpful, and Hadi and Abigail indicated that it was somewhat helpful (M = 3.33; range = 2–4).

Overall, themes that emerged from participants’ open-ended responses demonstrated favorable feedback regarding the intervention. They expressed increased confidence in implementing teaching techniques effectively, improvements in children’s expressive language, and the immediacy of feedback as strengths of the intervention. Participants did note that a challenge of the intervention was balancing the time for the intervention procedures with managing their classroom and the time to complete the reflection matrix within 72 hr of teaching. Despite the challenge of time, participants expressed that they would highly recommend this PD package to other teachers.

Discussion

Our research supports previous findings, which suggest PD provided within the context of the classroom was effective in enhancing target practices, children responded to the opportunities they were provided, and teachers indicated that this PD package was socially valid. Our research adds to the literature by combining real-time TEPF and video analysis to determine the effect on teachers’ use of NI targeting children’s communication, their quality use of NI targeting children’s communication, and rate of child responses. Finally, teachers indicated that there were specific elements that were challenging for them in terms of the feasibility of this PD package and such feedback can inform future iterations of this comprehensive approach to PD.

Support to Previous Findings

Three full-time teachers focused on an NI strategy targeting children’s communication engaged in this comprehensive PD package and demonstrated visible improvements in providing children opportunities to practice their communication skills. Teachers perceived that this PD package was effective. These findings align with previous research regarding the promise of both real-time TEPF and video analysis individually (Coogle et al., 2019; Nagro et al., 2017; Nagro & deBettencourt, 2018). In addition, the ultimate goal of PD activities is to improve learning opportunities for children leading to improved child outcomes. Each focus child demonstrated opportunities to practice their communication skills as they responded to their teacher’s use of NI targeting children’s communication. This finding aligns with previous research that has demonstrated when teachers receive support, they respond favorably, as do the children in their classroom (Coogle et al., 2019; Snyder et al., 2018).

Additions to the Literature

Our research adds to the literature by combining real-time TEPF and video analysis as an effective PD package to increase the quality of target practices. We provided real-time feedback from an alternate location while also facilitating in-depth reflection and analysis of teacher practice. This PD package exposed teachers to multiple opportunities for feedback and reflection that they would not have experienced by participating in either real-time TEPF or video analysis alone. For example, during real-time TEPF, teachers had opportunities to receive feedback and make real-time changes to their teaching, which is not traditionally part of video analysis activities. Similarly, during video analysis, teachers benefited from hearing the real-time TEPF captured on the video files a second time, providing them with an opportunity to gain a deeper understanding of the feedback provided while removing the need to simultaneously teach, which occurs during traditional real-time TEPF.

Findings from this investigation suggest that the comprehensive PD package shows promise for increasing teachers’ NI targeting children’s communication as well as improving the quality of such strategies. The quality was lower during baseline as compared to the intervention and maintenance conditions for all teachers. This finding suggests that our intervention package was effective in enhancing the quality of the teachers’ NI targeting children’s communication and sustaining those outcomes after the conclusion of the PD.

We also observed variable patterns between the quality of teachers’ NI targeting children’s communication and child responses. Although the quality of Andreia’s NI strategy targeting children’s communication during the baseline condition was low (67%), her focus child responded to 100% of opportunities provided. During Sessions 5 and 7 of the intervention when her quality was higher (86% and 93%), her focus child responded to 63% of opportunities provided, and there were sessions in which her quality was lower and her focus child consistently responded (Session 1). There were also five sessions in which the quality of Andreia’s NI targeting children’s communication and child responses were more closely aligned (Sessions 2, 3, 4, 6, and 8). In comparison, the quality of Hadi’s NI targeting children’s communication correlated with her focus child responses. When Hadi increased the quality of her NI targeting children’s communication, her focus child began responding more frequently. Furthermore, when Abigail used her NI strategy targeting children’s communication, her quality use remained high, and her focus child consistently responded. The variability across participants was likely due to environmental or child characteristics and should be examined in future research. For example, some of the activities in which Andreia and her focus child engaged seemed more motivating and engaging than others. Hadi’s focus child was not yet communicating with others and was not yet responding to Hadi at the start of the study, and therefore we were able to observe growth over time. From the start of the study, Abigail’s focus child frequently responded. These findings should be interpreted with caution.

We also added to the literature by determining the number of feedback statements delivered during each coaching session (see Table 2 for feedback statements). Although our prompting and corrective response statements were low, we noticed that feedback statements were frequent due to higher rates of unprompted NI strategies targeting children’s communication during intervention, when we affirmed the teachers. This is an important finding, as it suggests that the additional supports built into this comprehensive PD package such as the initial teacher–coach goal-setting conference to set goals, coaching presence throughout the sessions to provide affirmative comments, repeated exposure to affirmative comments captured on video, and engagement in focused video self-reflection activities all supported teachers in their quality use of NI targeting children’s communication. Although we cannot determine the impact of the individual components of our PD package, we can conclude that the package shows potential for improving teacher practices.

Finally, it is important to note that teachers reported finding it challenging to engage in both real-time TEPF and video analysis when considering their job demands. In previous studies, teachers received one intervention opposed to this comprehensive intervention package. While they expressed this challenge, they also perceived it as effective and suggested that other teachers participate in this type of PD.

Limitations

Although our intervention package was effective in enhancing the quantity and quality of target teacher behaviors, it is important to consider the limitations. We did not ask participating teachers to include timestamps in their written reflections assuring that they actually used the video evidence when completing their reflection matrices. Conversely, we did not limit the number of times participating teachers could view their video-recorded lessons. The potential variance in engagement with the video analysis portion of this PD package could have influenced the outcomes in ways we did not capture. In addition, all three participants opted into this study showing a willingness to engage in PD using technology. There may be specific teacher characteristics or dispositions unique to individuals willing to engage in additional education resulting in ideal conditions for implementation of this intervention package. Similarly, the participating teachers in this study were aware of the technological demands and still willing to participate, which may represent ideal conditions for testing this type of PD. In previous video analysis research, participants noted the need for ongoing technical support, which negatively influenced their attitudes toward the experience (Nagro et al., 2017; Nagro & deBettencourt, 2018), but the participants in this study did not. Given the social desirability of the participants and the small number of participants, it is unlikely that the participants would have rated any of the social validity items with a lower score.

The PD package we used was resource intensive, which also limits the social and ecological validity. Furthermore, it may be difficult to replicate this study on a larger scale. Each technology package including the iPad mini, Bluetooth® earpiece, and a Swivl™ costs US$1,010.99. Funding for the instructional technology was awarded through a university seed grant, and therefore these costs did not fall on the teachers, school administrators, or the researchers. The coach was deeply invested in participants and provided individualized PD to meet their unique needs; it may be difficult to provide this type of PD to larger groups of teachers without multiple coaches trained in effective real-time TEPF. All of these considerations may mean rethinking and redesigning the way in which PD is delivered.

A final limitation was regarding the method. Due to time limitations, we collected a limited number of data points in the baseline condition for all participants. It would have been ideal to collect at least three baseline probes (opposed to one) for each participant who had not yet received intervention as each participant beginning intervention. This limits our ability to make conclusion about data stability.

Implications for Future Research and Practice

First, future research should ensure baseline stability prior to introducing intervention. This would allow researchers to enhance their external validity and interpret their results with more confidence. In addition, researchers might consider examining the social validity and effectiveness of this comprehensive approach compared with either real-time TEPF or video analysis in isolation. In addition, although this research adds to the literature, by identifying the number and type of feedback statements (e.g., prompts, comments), it would be helpful to gain a deeper understanding of what the coach said so that we might better understand coaching statements and how this aspect of feedback influenced the teachers’ practice. A qualitative study that examines what the coach said and why they made the statements they did would deepen our understanding of decisions that are being made when coaching. In addition, researchers might consider examining whether reflective practice or reflective ability changed over time to determine if teacher perceptions of their practice correlate with their actual use of NI targeting children’s communication.

Our positive findings suggest that teacher educators, policymakers, and administrators might consider the components of this PD package when determining how to support teachers in their use of target quality practices. Using evidence-based practices such as NI within everyday activities and routines is important to provide children opportunities to practice target skills, but there is variability in teachers’ use of NI (Coogle et al., 2019). Therefore, it is important to consider how we might enhance the quantity and quality of teachers’ practice. Results from this study demonstrate that this PD system was effective in changing teacher practice.

Conclusion

Real-time TEPF (Bambrick-Santoyo, 2013) and video analysis (Nagro et al., 2017; Nagro & deBettencourt, 2018; Kennedy, Alves, & Rodgers, 2015) are innovative examples of PD activities that integrate teacher buy-in through active learning and can translate to improved early childhood classroom practices. These technologies create an authentic learning experience that directly relate to teachers’ immediate needs while maintaining the structure and guidance required to promote lasting change. PD should enable teachers to be innovators, critical thinkers, and implementers of effective teaching strategies that enhance child outcomes (Gulamhussein, 2013). Indeed, effective PD practices include hands-on activities, expert guidance, authentic context, opportunities for collaboration and practice, and feedback and reflection to promote transfer of knowledge to practice (Coogle et al., 2019; Barton et al., 2018; Brenner & Brill, 2016; Garet, Porter, Desimone, Birman, & Yoon, 2001; Snyder et al., 2018). The comprehensive PD package used in our study embodies many of the key features of high-quality PD and shows promise for enhancing the quantity and quality of teacher practice.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

This research was supported by the College of Education and Human Development at George Mason University.

References

Amobi

F. A.

(2005). Preservice teachers’ reflectivity on the sequence and consequences of teaching actions in a microteaching experience. Teacher Education Quarterly, 32, 115–129.

Artman-Meeker

Hemmeter

M. L.

Snyder

(2014). Effects of distance coaching on teachers’ use of pyramid model practices: A pilot study. Infants & Young Children, 27, 325–344. doi:10.1097/IYC.0000000000000016

Bambrick-Santoyo

(2013). Giving and getting feedback in real time. Phi Delta Kappan, 95, 72–73. doi:10.1177/003172171309500417

Barton

E. E.

Pokorski

E. A.

Gossett

Sweeney

Qiu

Choi

(2018). The use of email to coach early childhood teachers. Journal of Early Intervention. doi:10.1177/1053815118760314

Brenner

A. M.

Brill

J. M.

(2016). Investigating practices in teacher education that promote and inhibit technology integration transfer in early career teachers. Techtrends, 60, 136–144. doi:10.1007/s11528-016-0025-8

Calandra

Gurvitch

Lund

(2008). An exploratory study of digital video editing as a tool for teacher preparation. Journal of Technology and Teacher Education, 16, 137–153. Retrieved from https://www.learntechlib.org/primary/p/23496/

Coogle

C. G.

Ottley

J. R.

Rahn

N. L.

Storie

(2018). Bug-in-ear eCoaching: Impacts on novice early childhood special education teachers. Journal of Early Intervention, 40, 87–103. doi:10.1177/1053815117748692

Coogle

C. G.

Ottley

J. R.

Storie

Rahn

N. L.

Burt

(2018). Performance-based feedback to enhance pre-service teachers’ practice and preschool children’s expressive communication. Journal of Teacher Education. doi:10.1177/0022487118803583

Coogle

C. G.

Ottley

J. R.

Storie

Rahn

N.L.

Burt

(2017). eCoaching to enhance special educator practice and child outcomes. Infants and Young Children, 30, 58–75. doi:10.1097/IYC.0000000000000082

10.

Coogle

C. G.

Rahn

N. L.

Ottley

J. R.

(2015). Pre-service teacher use of communication strategies upon receiving immediate feedback. Early Childhood Research Quarterly, 32, 105–115. doi:10.1016/j.ecresq.2015.03.003

11.

Coogle

C. G.

Rahn

N. L.

Ottley

J. R.

Storie

(2016). eCoaching across routines to enhance teachers’ use of modeling. Teacher Education and Special Education, 39, 227–245. doi:10.1177/0888406415621959

12.

Coogle

C. G.

Storie

Ottley

J. R.

Rahn

N. L.

Burt

(2019). Technology enhanced performance-based feedback to support teacher practice and child outcomes. Topics in Early Childhood Special Education. doi:10.1177/0271121419838624

13.

Cooper

J. O.

Heron

T. E.

Heward

W. L.

(2007). Applied behavior analysis (2nd ed.). Pearson.

14.

Deniz

(2012). Video recorded feedback for self regulation of prospective music teachers in piano lessons. Journal of Instructional Psychology, 39, 17–25. Retrieved from https://ir.lib.uwo.ca/cgi/viewcontent.cgi?article=6428&context=etd

15.

Division for Early Childhood. (2014). DEC recommended practices in early intervention/early childhood special education. Retrieved from http://www.dec-sped.org/recommendedpractices

16.

Fox

Hanline

M. F.

(1993). A preliminary evaluation of learning within developmentally appropriate early childhood settings. Topics in Early Childhood Special Education, 13, 308–327. Retrieved from http://mutex.gmu.edu/login?url=http://search.ebscohost.com/login.aspx?direct=true&db=eric&AN=EJ472649&site=ehost-live

17.

Fox

Hemmeter

Snyder

Binder

D. P.

Clarke

(2011). Coaching early childhood special educators to implement a comprehensive model for promoting young children’s social competence. Topics in Early Childhood Special Education, 31, 178–192. doi:10.1177/0271121411404440

18.

Garet

M. S.

Porter

A. C.

Desimone

Birman

B. F.

Yoon

K. S.

(2001). What makes professional development effective? Results from a national sample of teachers. American Educational Research Journal, 38, 915–945. doi:10.3102/00028312038004915

19.

Grisham-Brown

Hemmeter

M. L.

Pretti-Frontczak

(2005). Blended practices for teaching young children in inclusive settings. Brookes.

20.

Gulamhussein

(2013, September). Teaching the teachers: Effective professional development in an era of high stakes accountability. National School Boards Association, Center for Public Education. Retrieved from http://conference.ohioschoolboards.org/2017/wp-content/uploads/sites/17/2016/07/1pm111317A114Job-embedPD.pdf

21.

Hager

K. D.

(2012). Self-monitoring as a strategy to increase student teachers’ use of effective teaching practices. Rural Special Education Quarterly, 31, 9–17.

22.

Hemmeter

M. L.

Snyder

P. A.

Kinder

Artman

(2011). Impact of performance feedback delivered via electronic mail on preschool teachers’ use of descriptive praise. Early Childhood Research Quarterly, 26, 96–109. doi:10.1016/j.ecresq.2010.05.004

23.

Hicks

D. H.

Lee

J. J.

Berson

M. B.

Bolick

C. C.

Diem

R. R.

(2014). Guidelines for using technology to prepare social studies teachers. Contemporary Issues in Technology & Teacher Education, 14, 433–450.

24.

Juniper Gardens Children’s Project. (2011). ECI coding definitions. Retrieved from http://igdi.ku.edu/wp-content/uploads/2016/04/ECI-definitions.pdf

25.

Kaczmarek

L. A.

Hepting

N. H.

Dzubak

(1996). Examining the generalization of milieu language objectives in situations requiring listener preparatory behaviors. Topics in Early Childhood Special Education, 16, 139–167. doi:10.1177/027112149601600203

26.

Kaiser

Hampton

(2017). Enhanced milieu teaching. In McCauley

R. J.

Fey

M. E.

Gillam

(Eds.), Treatment of language disorders in children (pp. 87–120). Brookes.

27.

Kalk

Luik

Taimalu

Täht

(2014). Validity and reliability of two instruments to measure reflection: A confirmatory study. Journal of the Humanities & Social Sciences, 18, 121–134. doi:10.3176/tr.2014.2.02

28.

Kennedy

M. M.

Alves

K. D.

Rodgers

W. J.

(2015). Innovations in the delivery of content knowledge in special education teacher preparation. Intervention in School & Clinic, 51, 73–81. doi:10.1177/1053451215579268

29.

Kratochwill

T. R.

Hitchcock

J. H.

Horner

R. H.

Levin

J. R.

Odom

S. L.

Rindskopf

D. M.

Shadish

W. R.

(2013). Single-case intervention research design standards. Remedial and Special Education, 34, 26–38. doi:0741932512452794

30.

Lane

J. D.

Ledford

J. R.

(2014). Using interval-based systems to measure behavior in early childhood special education and EI. Topics in Early Childhood Special Education, 34, 83–93. doi:10.1177/0271121414524063

31.

Nagro

S. A.

Cornelius

K. E.

(2013). Evaluating the evidence base of video analysis: A special education teacher development tool. Teacher Education and Special Education, 36, 312–329. doi:10.1177/0888406413501090

32.

Nagro

S. A.

deBettencourt

L. U.

Rosenberg

M. SA.

Carran

D. T.

Weiss

M. P

. (2017). The effects of guided video analysis on teacher candidates’ reflective ability and instructional skills. Teacher Education and Special Education, 40(1), 7–25. doi:10.1177/0888406416680469

33.

Nagro

S. A.

deBettencourt

L. U.

(2018). Reflection activities within clinical experiences: An important component of field-based teacher education. In Hodges

T. E.

Baum

A. C.

(Eds.), The handbook of research on field-based teacher education (pp 565–586). IGI Global. doi:10.4018/978-1-5225-6249-8.ch024

34.

Noh

Allen

Squires

(2009). Use of embedded learning opportunities within daily routines by early intervention/early childhood special education teachers. International Journal of Special Education, 24, 1–10. Available from http://www.internationaljournalofspecialeducation.com/

35.

Pianta

R. C.

Mashburn

A. J.

Downer

J. T.

Hamre

B. K.

Justice

(2008). Effects of web-mediated professional development resources on teacher-child interactions in pre-kindergarten classrooms. Early Childhood Research Quarterly, 23, 431–451. doi:10.1016/j.ecresq.2008.02.001

36.

Rahn

N. L.

Coogle

C. G.

Ottley

J. R.

(2019) Early childhood special education teachers’ use of embedded learning opportunities within classroom routines and activities. Infants and Young Children, 32, 3–19. doi:10.1097/IYC.000000000000013

37.

Randomness and Integrity Services Ltd. (2019). Random integer generator. Retrieved from https://www.random.org/integers/

38.

Rock

M. L.

Spooner

Nagro

S. A.

Vasquez

Dunn

Leko

Luckner

J. L.

Bausch

Donehower

Jones

J. L.

(2016). 21st century change drivers: Considerations for constructing transformative models of special education teacher development [Special Issue]. Teacher Education and Special Education, 39, 98–120. doi:10.1177/0888406416640634

39.

Scheeler

M. C.

Ruhl

K. L.

McAfee

J. K.

(2004). Providing performance feedback to teachers: A review. Teacher Education and Special Education, 27, 396–407. doi:10.1177/088840640402700407

40.

Seidel

Stürmer

Blomberg

Kobarg

Schwindt

(2011). Teachers learning from analysis of videotaped classroom situations: Does it make a difference whether teachers observe their own teaching or that of others? Teaching and Teacher Education, 27, 259–267. doi:10.1016/j.tate.2010.08.009

41.

Smith

S. J.

Kennedy

M. J.

(2014). Technology and teacher education. In Sindelar

P. T.

McCray Brownell

M. T.

Lingnugaris/Kraft

(Eds.), Handbook of research on special education teacher preparation (pp. 178–193). Routledge, Taylor, & Francis.

42.

Snyder

Hemmeter

M. L.

Fox

(2015). Supporting implementation of evidence-based practices through practice-based coaching. Topics in Early Childhood Special Education, 35, 133–143. doi:10.1177/0271121415594925

43.

Snyder

Hemmeter

M. L.

McLaughlin

(2011). Professional development in early childhood intervention: Where we stand on the silver anniversary of PL 99-457. Journal of Early Intervention, 33, 357–370. doi:10.1177/1053815111428336

44.

Snyder

Hemmeter

M. L.

McLean

Sandall

McLaughlin

Algina

(2018). Effects of professional development on preschool teachers’ use of embedded instruction practices. Exceptional Children, 84, 213–232. doi:10.1177/0014402917735512

45.

Snyder

P. A.

Rakap

Hemmeter

M. L.

McLaughlin

T. W.

Sandall

McLean

M. E.

(2015). Naturalistic instructional approaches in early learning: A systematic review. Journal of Early Intervention, 37, 69–97. doi:10.1177/1053815115595461

46.

The Danielson Group. (2017). The framework. Retrieved from https://www.danielsongroup.org/framework/

47.

Tripp

T. R

Rich

P. J.

(2012). The influence of video analysis on the process of teacher change. Teaching and Teacher Education, 28, 728–739. doi:10.1016/j.tate.2012.01.011

48.

van Es

E. A.

Sherin

M. G.

(2010). The influence of video clubs on teachers’ thinking and practice. Journal of Mathematics Teacher Education, 13, 155–176. doi:10.1007/s10857-009-9130-3

49.

Wang

Hartley

(2003). Video technology as a support for teacher education reform. Journal of Technology and Teacher Education, 11, 105–138. Retrieved from https://www.learntechlib.org/primary/p/17791/

50.

What Works Clearinghouse. (2017). Procedures and standards handbook (vol.4). Retrieved from http://ies.ed.gov/ncee/wwc/documentsum.aspx?sid=19

51.

Wright

M. R.

Ellis

D. N.

Baxter

(2012). The effect of immediate or delayed video-based teacher self-evaluation on head start teachers’ use of praise. Journal of Research in Childhood Education, 26, 187–198. doi:10.1080/02568543.2012.657745

52.

Yoder

P. J.

Ledford

J. R.

Harbison

A. L.

Tapp

J. T.

(2018). Partial-interval estimation of count: Uncorrected and Poisson-corrected error levels. Journal of Early Intervention, 40, 39–51. doi:107.171/1770/5130583185115171774488407