Performance-Based Feedback to Enhance Preservice Teachers’ Practice and Preschool Children’s Expressive Communication

Abstract

A multiple-probe, single-case design was used to determine the effect of delayed and immediate performance-based feedback on preservice teachers’ use of embedded learning opportunities, an evidence-based practice in early childhood special education, as well as focus children’s expressive communication and responses to preservice teachers’ practice. Results suggest delayed performance-based feedback improved preservice teachers’ practice, which was further enhanced when immediate feedback was used. Positive outcomes were observed across children for both their responses and overall expressive communication.

Keywords

performance-based feedback embedded learning opportunities preservice teachers early childhood special education

Currently one in 68 children is diagnosed with autism spectrum disorder (ASD) each year (Centers for Disease Control, 2014), and children are being identified with ASD younger in age than ever before (Guthrie, Swineford, Nottke, & Wetherby, 2013; Woods & Wetherby, 2003). A core skill that children with ASD experience challenges with is social communication, which is reflected in the way they communicate with others (Zwaigenbaum, Bryson, & Garon, 2013). Social communication is a critical skill that necessitates intervention because communicative challenges have both academic and behavioral implications (Koegel, Matos-Freden, Lang, & Koegel, 2012). Individuals identified with ASD benefit from receiving intervention in their natural environment, as generalizing a skill to a new environment presents challenges for them (DeMarchena, Eigsti, & Yerys, 2015). For young children with ASD, the natural environment is frequently an inclusive preschool classroom (Matson, Hattier, & Williams, 2012).

Embedded Learning Opportunities to Promote Social Communication Skills

One practice that has documented positive effects on a variety of skills, including social communication for children with disabilities, is embedded learning opportunities (McGee & Daly, 2007; Snyder et al., 2018; Wong et al., 2015). During an embedded learning opportunity, the teacher intentionally embeds an antecedent, prompting the child to practice a target behavior such as a social communication skill, and concludes with a consequence that reinforces the behavior and encourages the child to utilize the target behavior again. For example, during play with a car ramp, the teacher might catch the car at the bottom of the ramp, hold it and look expectantly at the child (antecedent), wait for the child to approximate the word “car” (child behavior), and then hand the car to the child (consequence). The purpose of an embedded learning opportunity is to provide multiple, carefully planned opportunities for a child to practice and eventually attain a target skill. In the case of the child playing with the car ramp, the teacher would catch the car multiple times to provide the child with several opportunities to communicate. The teacher might expand on the embedded learning opportunity in subsequent turns by modeling “more car” and waiting for the child to approximate a two-word response.

Although embedded learning opportunities have documented effectiveness, they are used infrequently by teachers (Noh, Allen, & Squires, 2009; Pretti-Frontczak & Bricker, 2001). Teachers have indicated they do not use embedded learning opportunities frequently, because of their large class sizes, challenges individualizing instruction, and limited access to resources related to implementing embedded learning opportunities that align with their children’s specific goals. This gap between research and practice aligns with the sentiments of teachers who have indicated feeling unprepared to meet the diverse needs of children within inclusive classrooms (Busby, Ingram, Bowron, Oliver, & Lyons, 2012). There are a number of variables associated with teacher preparation that may explain these feelings. First, due to challenges associated with teacher shortages, teachers are frequently teaching in areas in which they did not receive sufficient training (Busby et al., 2012). In addition, teacher preparation requires more than teaching about evidence-based practices, but rather preservice teachers should have intentionally designed opportunities to practice using evidence-based practices (Brownell, Ross, Colon, & McCallum, 2005; Kennedy et al., 2016; Nagro & deBettencourt, 2017). This generalization from the university classroom to the field is a complex task, as there is not a prescribed method to engage preservice teachers in their transition of knowledge into practice, and limited research has been conducted on preservice teacher preparation in the field of special education (Leko & Brownell, 2011). From the research that does exist, scholars have suggested that training should include multiple examples of the practice that are contextually relevant to the setting in which preservice teachers are engaged and that performance feedback be provided to teachers in their classroom setting (Scheeler, Bruno, Grubb, & Seavey, 2009).

Performance-Based Feedback During Student Teaching Experiences

The student teaching component of teacher preparation programs is critical in preservice teacher professional development, as it provides opportunities to apply the knowledge that has been gained during the preservice preparation program (Auld, Belfiore, & Scheeler, 2010; Leko & Brownell, 2011; Macy, Squires, & Barton, 2009). For example, when preservice teachers are engaged in coursework that includes a field placement in which they can use the practices they are learning about, this combination of experiences provides them the opportunity to enhance their knowledge and skills, and ultimately improve student achievement (Scott, Gentry, & Phillips, 2014). In addition, preservice teachers have reported field placements as the most valuable aspect of their preparation programs (Bishop, Brownell, Klingner, Leko, & Galman, 2010). Although field placements are a core component of preservice teacher preparation programs, and a means to practice, develop, and apply the target skills from their preparation programs (Barton, Fuller, & Schnitz, 2016), challenges related to identifying a supervising teacher who has the time to dedicate to mentoring a preservice teacher, and finding a placement where the course/program objectives are modeled can result in a disconnect between the university classroom and school setting (Billingsley & Scheuermann, 2014; Ostrosky, Mouzourou, Danner, & Zaghlawan, 2013; Scott et al., 2014). Therefore, there is a need for quality supervision that fosters connectivity between the university and community setting and supports preservice teachers to engage and receive feedback in practices necessary to attain and deliver evidence-based practices (Division for Early Childhood, 2014; Kennedy et al., 2016; Leko & Brownell, 2011). However, due to limited resources such as time, scheduling, and funding, it has become increasingly challenging to provide adequate supervision in field placements (Scheeler, McKinnon, & Stout, 2012).

Although there is not a prescribed supervision model across teacher preparation programs, performance-based feedback has evidence of being both feasible and effective in changing preservice teacher behavior (Barton & Wolery, 2008; Coogle, Rahn, & Ottley, 2015; Hemmeter, Snyder, Kinder, & Artman, 2011; Scheeler et al., 2012). Performance-based feedback is a process in which an individual receives information regarding the actions in which they are engaging (Hattie & Timperley, 2007). Performance-based feedback in early childhood settings has included technology-enhanced methods to provide feedback from an alternate location, overcoming some of the challenges associated with adequate supervision (Barton et al., 2016; Barton & Wolery, 2008; Coogle et al., 2015). These feedback systems have varied from using delayed forms of performance-based feedback such as email (Barton et al., 2016; Barton & Wolery, 2008), to a more immediate form of performance-based feedback through bug-in-ear technology using a Bluetooth device and Skype^TM (Coogle et al., 2015). Delayed performance-based feedback has been used by conducting a face-to-face observation within a classroom and promptly sending feedback through email (i.e., within 4 hr; Barton & Wolery, 2008), whereas immediate performance-based feedback has embedded bug-in-ear technology for preservice teachers to receive feedback during the actual observation session (Coogle et al., 2015).

Although performance-based feedback has been diverse in terms of delivery method, it has resulted in a positive effect on targeted preservice teacher skills (Barton et al., 2016; Barton & Wolery, 2008; Coogle et al., 2015). For example, face-to-face observations paired with a follow-up email have resulted in preservice teachers using recommended practices (Barton et al., 2016; Barton & Wolery, 2008). In addition, immediate performance-based feedback delivered from a distance via Skype^TM and Bluetooth® has resulted in preservice teachers’ use of embedded learning opportunities targeting communication skills (Coogle et al., 2015). However, the body of evidence regarding the use of multiple forms of performance-based feedback to support preservice teachers is still emerging. In addition, there is little research examining the effect of performance-based feedback on the communication outcomes of focus children with ASD, but the emerging research suggests positive results on child outcomes after the educator in their classroom received one form of feedback (Ottley, Rahn, Coogle, Ferron, & Storie, 2018). Even so, there is a need to continue to learn more regarding the impact of feedback delivered to preservice teachers on child outcomes. Therefore, the purpose of this study was to use delayed and immediate performance-based feedback (i.e., email and bug-in-ear, respectively) to determine the effect on preservice teachers’ use of embedded learning opportunities with a focus child with ASD, focus child responses, and focus child communication. In the current study, all focus child participants had an individualized education program (IEP) goal related to the way in which they communicated with others; therefore, the embedded learning opportunities we selected in this research included antecedents that would elicit a social communicative behavior (i.e., choice making, environmental arrangement, and mands; see Table 1) and a consequence that when used with fidelity would increase the likelihood the child would continue expressively communicating (i.e., natural reinforcement). The research questions we sought to answer included as follows:

Research Question 1: Is there a functional relation between delayed performance-based feedback and preservice teachers’ use of embedded learning opportunities?

Research Question 2: Is there a functional relation between immediate performance-based feedback and preservice teachers’ use of embedded learning opportunities?

Research Question 3: Is there an association between teachers’ use of embedded learning opportunities and focus children’s responses?

Research Question 4: Is there an association between teachers’ use of embedded learning opportunities and focus children’s expressive communication?

Table 1.

Embedded Learning Opportunity Strategies.

Strategy	Definition	Example
Antecedents
Choice making	Providing choices to the child to create a communication opportunity	At lunch, Stephen’s teacher asks if he would like to have juice or milk
Environmental arrangement	Manipulating materials in the environment to provide a communication opportunity	The teacher places the child’s crackers where the child can see them but not reach them
Mand	Making a request in statement form for the child to say something	The teacher says, “Say more” when child is reaching for additional snack items
Consequences
Natural reinforcement	Upon appropriate communication attempts, providing the child with desired item or routine to naturally reinforce communication attempts	When Aalap signs, “More please,” his teacher gives him more blocks to play with

Method

Participants and Setting

Participants were three female, early childhood special education preservice teachers completing their final semester of an undergraduate, university-based, dual-licensure program in a small mid-Atlantic city. The preservice teacher participants were completing their final semester of their 4-year preparation and certification program and were enrolled in student teaching in an inclusive preschool classroom within a rural community. Two of the preservice teachers were placed in one county (different classrooms), and one was placed in a neighboring county. All classrooms were using the same state curricula and each classroom had a similar but unique schedule (i.e., all classrooms had choice time in the morning, but the precise time was different). The three participants were Olivia, Amelia, and Lucy (pseudonyms). All participants were White, non-Hispanic, and between the ages of 21 and 22 years at the start of the study. The preservice teachers’ experiences with children with and without disabilities included practicum placements that they had in their previous classes, which included approximately 300 hr of time in early childhood environments (i.e., homes and inclusive community care and preschool classrooms) across previous early childhood classes. During their student teaching experience, the preservice teachers were placed in full-day inclusive public preschool environments. Preservice teachers were expected to build rapport within the classroom, assist the classroom teacher, and finally take the role of the lead teacher while demonstrating mastery of programmatic objectives related to positive behavior support, differentiating instruction, and collaborative practices. It is important to note that Amelia was completing her internship in a classroom in which her supervising teacher was also receiving performance-based feedback targeting the same skills.

Focus children were students identified with ASD in the preservice teachers’ internship classrooms who had an IEP goal targeting expressive communication (i.e., using words to express wants and needs). The focus child in Olivia’s classroom was Liam, a 4-year-old, a non-Hispanic White boy who was also identified with developmental delays and a seizure disorder. Liam demonstrated use of three- to four-word phrases, but he was difficult to understand and had challenges effectively communicating with others. The focus child in Amelia’s classroom was Asher, a 4-year-old, non-Hispanic White boy. Asher used one- to two-word phrases, infrequently initiated conversation, and exhibited echolalia. The focus child in Lucy’s room was Jamar, a 4-year-old, non-Hispanic White boy. Jamar communicated frequently with gestures and vocalizations, with some one- and two-word phrases.

All sessions were conducted during regular teaching hours within the classrooms of all participating preservice teachers. Intervention occurred during free play, which took place when and where it typically would if the research would not have been occurring. All activities included the focus child, and sometimes included children who were developing typically due to the natural interactions within the classroom.

Research Design

We used a multiple-probe, single-case ABC research design (where A represents baseline, B represents delayed feedback, and C represents immediate feedback) to determine whether a functional relation existed between delayed and immediate performance-based feedback on preservice teachers’ use of embedded learning opportunities. We selected a multiple-probe design because the behaviors identified were not expected to change prior to intervention, and the design allows for purposefully omitting data collection on specific sessions, which lessens the burden of data collection while maintaining the integrity of the research design and data collection process (Kratochwill et al., 2013). Using the multiple-probe design, participants start intervention in a staggered fashion (i.e., Olivia on day 6, Amelia on day 12, Lucy on day 17), which strengthens the internal validity of the study because it shows that changes in participant behaviors were a result of the intervention and not due to chance. Although a stronger research design would have been an ABABACAC design in which multiple replications occur between conditions, theoretically this does not make sense for our research questions, because we do not want participants to stop using the antecedents and consequences when coaching ceases, but rather for them to sustain their practices. Consequently, we selected a multiple-probe design as the primary design and we embedded a second intervention to determine whether the effects of immediate feedback improved teacher practice above and beyond delayed feedback.

All requirements for What Works Clearinghouse were met by systematically manipulating the independent variables, using more than one observer to measure all outcome variables, and establishing interobserver agreement (Kratochwill et al., 2013). Our study met What Works Clearinghouse standards with reservations because the first teacher only received four delayed feedback sessions and consequently there were not five data points in the condition to meet the standards without reservations. This was due to researcher error and providing immediate feedback during one session too early.

Measures

We used a partial-interval coding system by dividing a 6-min session into 10-s intervals and coding (a) when preservice teachers used target antecedents and consequences, (b) how focus children responded to antecedents, and (c) focus child communication. Upon completion of each session, the coach (third author) used a feedback form to record the feedback provided and preservice teachers’ use of prompted (provided a prompt to use a target antecedent and/or consequence) and spontaneous (preservice teacher utilized a target antecedent and/or consequence without a prompt) antecedents and consequences. During baseline and delayed feedback conditions, only spontaneous antecedents and consequences were used, as there were no prompts provided. Similarly, we only documented each focus child’s response to their preservice teacher’s use of target antecedents, but not consequences, as we would not expect a child to have an immediate communicative response to our selected consequence (reinforcement). We also coded each focus child’s use of expressive communication (gestures [e.g., pointing], vocalizations [e.g., sounds that are not understood as words], single and multiple words [e.g., one word such as please, and more than one word such as more please], and total weighted communication). Total weighted communication was scored for each session by weighting each form of communication (i.e., gestures and vocalizations: weight of 1, single words: weight of 2, multiple words: weight of 3) using the Individual Growth and Development Indicator Measure: Early Communication Indicator, which has moderate to high criterion validity with other early expressive communication measures (Greenwood, Carta, Walker, Hughes, & Weathers, 2006; Juniper Gardens Children’s Project, 2011).

Materials

During each condition of the study, preservice teachers selected materials within their classrooms to use with their focus children during the sessions. The first author and coach met with each preservice teacher and provided them with an iPad Mini®, Bluetooth® earpiece, Skype™, and a Swivl™ (a device that held the iPad and tracked each preservice teacher’s movement via a lanyard). The coach used Skype™ to observe and provide performance-based feedback during the immediate performance-based feedback sessions. ECamm™ software was used to record all sessions. The coach downloaded the video recording of the Skype™ call immediately following each session to code the preservice teacher and focus child behaviors (i.e., responses and communication).

Procedure

Before beginning the study, we obtained approval from the university institutional review board, each preservice teacher’s school district, and the mentoring teacher. Informed consent from preservice teachers, mentoring teachers, and guardians of the children in each classroom was obtained. To be sure that all preservice teachers received the same information regarding embedded learning opportunities, prior to collecting data, all preservice teachers participated in a face-to-face training provided by the first and third author that summarized embedded learning opportunities including each of the antecedents and consequences. Preservice teachers also discussed their focus children and goals they were working toward in the classroom and how the embedded learning opportunities could be used to facilitate goal attainment. During this meeting, preservice teachers were trained on the use of the technology by the first and third authors. Each preservice teacher also provided details on their classroom schedules, including the time frame of free play to allow the coach to plan for 10-min sessions that did not overlap with one another (i.e., preservice would indicate free play is scheduled from 10:00 a.m.-11:00 a.m. and the coach assigned them to 10:15 a.m-10:25 a.m.). The following week, prior to beginning the study, each preservice teacher called the coach to test the technology. The order in which preservice teachers and focus child participants completed each condition was based upon the randomization schedule. During each session, to keep all conditions of the research consistent, the preservice teacher placed the iPad in the Swivl™, placed the Bluetooth® device in their ear, and called the coach using Skype™ on the iPad at the predetermined time of session. The coach answered the call, greeted the preservice teacher to make sure sound and video were working, then started recording. The preservice teacher then began the session with her focus child.

Baseline

During the baseline condition, each preservice teacher was asked to interact with her focus child during free play as she typically would if research was not occurring. The number of baseline sessions varied from five to nine sessions depending on when intervention was introduced. If any antecedents or consequences were used in baseline sessions, they were coded. When preservice teachers completed their last baseline session, the first author and coach selected a target antecedent that the preservice teacher used infrequently during baseline. For one preservice teacher (Olivia) who infrequently utilized more than one antecedent, we randomly selected one target antecedent. The consequence selected for all preservice teachers was natural reinforcement because that was identified as the most appropriate consequence for all child participants based on their level of communication. See Table 1 for definitions and examples of target antecedents and consequences.

Intervention

Intervention included two conditions. The first intervention condition was delayed performance-based feedback, which was delivered to the preservice teacher via email at the end of each session. The second intervention condition was immediate feedback delivered via bug-in-ear in real-time during the observation. Intervention was introduced following the multiple-probe staggered introduction schedule (Kratochwill et al., 2013). Intervention sessions took place during free play and were approximately 6 min in length. We used 6-min sessions because results from our prior work have demonstrated that short segments of performance-based feedback are effective in changing educator practice and educators have indicated this was a feasible amount of time to keep a young child with ASD engaged in an activity (Coogle, Ottley, Rahn, & Storie, 2018).

Delayed performance-based feedback

Upon completing the observation, delayed feedback was provided by the coach within the body of an email and as an electronic attachment sent to the preservice teacher within 1 hr of completing the observation. The email was structured to resemble a previous format that was effectively used by another research team (Hemmeter et al., 2011; see the appendix). That is, it included a general positive opening statement, intent of the email, examples of embedded learning opportunities used paired with affirmative feedback regarding the teacher’s use of these embedded learning opportunities, suggestive feedback regarding future use of embedded learning opportunities, statements or questions about scheduling future sessions, a request for a reflection about the feedback provided, and a positive closing statement. Suggestive feedback included examples of how the target antecedent or consequence could be used, and affirmative feedback included positive comments identifying the specific antecedent or consequence that the preservice teacher used. As the follow-up action, preservice teachers were asked to respond to four reflection questions based on the feedback they received by the end of the same day which feedback was received.

Immediate performance-based feedback

After the final delayed feedback session, preservice teachers began receiving immediate bug-in-ear feedback via Bluetooth® and Skype^TM. When the preservice teacher was ready, they called the coach using Skype^TM. The coach observed the activity for the first minute; only affirmative feedback was provided if the preservice teacher used a target antecedent or consequence spontaneously, which allowed the preservice teacher to start the activity without interruption and provided the coach time to understand the nature of the activity before offering suggestions. During the remaining 5 min, suggestive and affirmative feedback were provided to support the preservice teachers’ use of (or lack thereof) target antecedents and consequences. Suggestive feedback was delivered at a rate of approximately one per minute when the coach observed that the preservice teachers were not using the antecedent or consequence spontaneously. Suggestive feedback was also used to help guide the preservice teacher to use target antecedents and/or consequences correctly (i.e., when providing a choice, make sure to label each item). When target antecedents and/or consequences were used spontaneously, the coach used affirmative feedback to praise the preservice teacher (i.e., “nice job using choice making”).

Maintenance and generalization

All authors determined a priori that each preservice teacher would complete three maintenance and generalization sessions upon completion of the second intervention condition (immediate feedback). During maintenance and generalization, the preservice teachers received no feedback from the coach. Maintenance and generalization began immediately after intervention sessions concluded due to time constraints (1-2 days post-intervention). Preservice teachers completed three maintenance video sessions consecutively in which they interacted with the child during free play. During generalization sessions, preservice teachers were asked to interact with the focus child during a typical meal-time routine (breakfast, lunch, or snack). Olivia and Amelia collected one maintenance and one generalization probe per day; however, due to time constraints (end of school year), Lucy collected two maintenance probes on the same day and two generalization probes on the same day.

Data Analysis

Visual analysis of graphed data was used to answer our first two research questions according to the six aspects of visual analysis (Horner et al., 2005). We used visual analysis to determine immediacy of effect for each condition (i.e., how quickly the data changed upon receiving a new condition), overlap of data across research conditions (i.e., did data within each condition overlap), consistency of data across conditions for each teacher (i.e., what patterns emerged across participants), level (i.e., mean performance within each condition), trend (i.e., slope within each condition), and variability (i.e., change in performance; Horner et al., 2005; Kratochwill et al., 2013). Once we determined a functional relation, we utilized Tau-U to examine the intensity of the effect (Parker, Vannest, Davis, & Sauber, 2011). Tau-U is a nonparametric approach to measure the nonoverlap of data between two phases. It follows the Kendall’s S sampling distribution, similar to the Mann–Whitney U and Kendall Rank Correlation nonparametrics tests. We selected Tau-U because it is the most acceptable nonoverlap effect size currently available, as it considers both level and trend in its analysis (Vannest, Parker, & Gonen, 2011). The Tau-U analysis supplements the visual analysis methods used to answer our first two research questions about the functional relation between the feedback and preservice teachers’ practices.

Interobserver agreement and fidelity of implementation

A graduate student unfamiliar with the study but who had coding experience collected interobserver agreement and fidelity of implementation for 39% of the videos. Interobserver agreement was collected using the total agreement method (Kennedy, 2005). That is, the number of disagreements was subtracted from the total number of occurrences and was divided by the number of agreements plus disagreements. Interobserver agreement was collected for prompted use of antecedents and consequences, child responses, and child communication (gestures, vocalizations, single words, multiple words, and total communication). Interobserver agreement was 100% for prompted and 85% (range = 58%-100%) for spontaneous antecedents and consequences. Interobserver agreement was 90% (range = 50%-100%) for child responses, 86% for gestures (range = 73%-97%), 83% (range = 64%-97%) for vocalizations, 88% (range = 73%-97%) for single words, 86% (range = 58%-100%) for multiple words, and 86% (range = 72%-94%) for total communication.

Fidelity of implementation data are important in determining a functional relation between the independent and dependent variables, because if the intervention is implemented as designed, then it should produce the hypothesized effects, if it indeed is an effective intervention. Conversely, if an intervention is not implemented as designed and outcomes are weak, then this can explain why outcomes may not have been achieved as predicted. The same graduate student who coded interobserver agreement also coded fidelity of implementation across all conditions of the study to ensure that the intervention was delivered as intended (no feedback during baseline, email feedback during the delayed feedback condition, and bug-in-ear feedback during the immediate feedback condition). We determined that feedback was appropriately provided (i.e., prompts, affirmative, and corrective types of feedback were specific to identified strategies; 99%; range = 90%-100%).

Results

Preservice Teacher Outcomes

Figure 1 presents graphs of the preservice teachers’ data. Throughout baseline, the preservice teachers’ use of target antecedents and consequences varied by participant. Olivia rarely used target antecedents and consequences (M = 0.4; range = 0-1). Amelia used target antecedents and consequences during some, but not all sessions (M = 1.7; range = 0-5), and had a stable trend. Lucy used at least two antecedents or consequences per session (M = 4.8; range = 2-11), but demonstrated a decelerating trend in her performance.

Figure 1.

Teachers’ correct use of naturalistic communication strategies across phases (closed circles) and during generalization (open circles).

Our first research question was related to the effect of delayed feedback on preservice teacher’s use of embedded learning opportunities. Upon transition to the delayed feedback condition, all preservice teachers demonstrated an immediate increase in the frequency with which they spontaneously used their target antecedents and consequences. This increased level stayed consistently higher than baseline for all preservice teachers throughout the delayed feedback condition. Olivia used an average of 6.5 target antecedents or consequences per session (range = 3-14), with her performance demonstrating an accelerating trend; she started intervention using four target antecedents or consequences during the first session and increased to 14 target antecedents or consequences during the final delayed feedback session. Amelia also displayed an accelerating trend starting with seven target antecedents or consequences used during the first delayed session and ending with 12 target antecedents or consequences in the final session. Amelia demonstrated the largest growth between baseline and delayed feedback, averaging 14.2 strategies used per session (range = 7-19). Although Lucy demonstrated a slightly negative slope throughout delayed feedback, her level had increased by 3.6 target antecedents or consequences to an average of 8.4 strategies per session (range = 5-10).

Our second research question examined the effect of immediate feedback on preservice teacher’s use of embedded learning opportunities. When the preservice teachers transitioned from the delayed feedback to the immediate feedback condition, all teachers either continued to use their target antecedents or consequences at levels comparable with the delayed feedback condition (Amelia and Lucy) or showed continued growth in their performance (Olivia). Although Olivia did not demonstrate an immediate increase in her use of her target antecedents or consequences from the delayed feedback to the immediate feedback condition, overall, she demonstrated a positive trend and consistently used them at levels higher than her use in the delayed feedback condition (M = 14.5; range = 8-22). Variability was limited to two sessions where Olivia’s performance was either greater than (i.e., immediate feedback Session 4) or less than (i.e., Session 8) that which would be expected. All of Amelia’s immediate feedback data remained at least double her baseline performance (M = 16.5; range = 10-22). Amelia’s first two immediate feedback sessions were both greater than her best performance during the delayed feedback condition. However, throughout the immediate feedback condition, she demonstrated a negative trend, which resulted in the last four sessions overlapping with her performance in the delayed feedback condition. Two of Lucy’s immediate feedback sessions overlapped with her performance during the baseline and delayed feedback conditions. However, she demonstrated an accelerating trend, which resulted in an increased level during the immediate feedback condition (M = 11.2; range = 4-17).

All three preservice teachers sustained their use of target antecedents or consequences after feedback was removed. Olivia had a stable trend and slightly higher level than her performance during the immediate feedback condition (M = 15; range = 14-16). Both Amelia and Lucy had negative maintenance trends, but Amelia’s level was greater than that of her immediate feedback condition (M = 22.3; range = 20-25), whereas Lucy’s level was 0.5 strategies lower (M = 10.7; range = 9-13).

Across all teachers, variability was minimal during generalization, but a negative trend was observed for Olivia and Amelia. Amelia generalized her use of the strategies to a new routine at levels comparable with her performance during the immediate feedback condition (M = 18.7; range = 16-20). However, Olivia (M = 2; range = 0-5) and Lucy (M = 2.3; range = 2-3) did not generalize their use of the communication strategies to a new routine and their generalization performance was comparable with baseline.

Tau-U comparisons

For each participant, we first examined whether we needed to correct for any trends in the data before calculating Tau-U values. No participants had a significant trend in their baseline conditions or delayed feedback conditions, so standard calculations were used when comparing any condition with baseline as well as when comparing the immediate feedback condition with the delayed feedback condition. A significant negative trend (τ = −0.87, p = .015) was found for Amelia’s immediate feedback condition, so we corrected for trend when comparing her maintenance and generalization performance with that condition. Tau-U statistics for each participant and across all participants (omnibus) for each condition comparison is reported in Table 2. When delayed feedback, immediate feedback, and maintenance condition data were compared with baseline, values were significant across all participants and for the omnibus test. This was not the case for the generalization condition; only Amelia’s performance was significantly better than baseline.

Table 2.

Tau-U Comparison Statistics.

	B vs. I₁	B vs. I₂	I₁ vs. I₂	B vs. M	I₂ vs. M	B vs. G	I₂ vs. G
Olivia	1.0*	1.0**	.75*	1.0*	0.17	0.40	−1.0*
Amelia	1.0**	1.0**	.30	1.0*	1.4***	1.0*	0.83
Lucy	0.69*	0.76*	.40	0.85*	−0.13	−0.62	−1.0*
Omnibus	0.89***	0.92***	.48*	0.95***	0.49*	0.24	−0.39

Note. Amelia’s I₂ vs. M and I₂ vs. G values reflect correction for her negative I₂ trend; B = Baseline; I₁ = Delayed Feedback; I₂ = Immediate Feedback; M = Maintenance; G = Generalization.

p < .05. **p < .01. ***p < .001.

When we compared adjacent conditions between delayed feedback and immediate feedback, Tau-U was significant for Olivia, which aligns with our visual analysis conclusions. Similarly, when comparing the immediate feedback condition with maintenance, results were significant for Amelia. However, when we compared immediate feedback data with that of generalization, Olivia and Lucy’s performance was significantly lower during generalization. For the study as a whole, the Tau-U values were significant for delayed versus immediate feedback (favoring immediate) as well as for immediate feedback versus maintenance (favoring maintenance).

Focus Child Outcomes

Appropriate responses to the teacher’s use of target antecedents

Our third research question was to determine child responses to preservice teacher practice. There were limited opportunities for any focus children to respond to antecedents during baseline, because their teachers were infrequently using antecedents. When delayed feedback began and the preservice teachers started using antecedents, all three focus children’s appropriate responses increased marginally. Upon transition into the immediate feedback condition, focus children’s responses increased more drastically, with all three focus children more than doubling their mean response rate when compared with the delayed feedback condition. Within the maintenance condition, focus children continued to respond at levels that were comparable with the frequency with which they responded during the immediate feedback condition. Asher’s responses during generalization remained high. However, Olivia and Lucy used antecedents infrequently throughout generalization, which provided fewer opportunities for Liam and Jamar to respond, and subsequently, they responded appropriately less often. Table 3 provides descriptive information regarding each focus child’s appropriate responses during each condition of the research.

Table 3.

Mean Child Outcomes and Ranges Across Research Phases.

	B	I₁	I₂	M	G
Liam (Olivia)
Responses	0.0 (0-0)	1.75 (0-7)	6.0 (4-11)	6.0 (4-9)	0.67 (0-1)
Vocalizations	10.8 (3-16)	9.25 (7-14)	9.75 (5-14)	11.33 (10-13)	4.67 (3-7)
Gestures	7.60 (1-17)	11.75 (2-8)	5.13 (0-9)	3.67 (2-6)	5.67 (5-7)
Single words	5.00 (2-8)	7.75 (6-10)	5.38 (2-8)	7.00 (3-13)	1.33 (0-3)
Multiple words	9.80 (5-16)	20 (17-26)	12.0 (0-23)	19.0 (12-24)	13.3 (10-17)
Total	57.8 (39-61)	96.5 (86-121)	61.2 (35-91)	86.0 (62-104)	53.0 (41-67)
Asher (Amelia)
Responses	0.57 (0-2)	3.20 (1-5)	7.0 (1-11)	8.33 (7-10)	10.67 (9-14)
Vocalizations	5.00 (1-11)	1.60 (1-3)	3.67 (1-8)	7.00 (5-8)	2.33 (0-4)
Gestures	7.00 (3-15)	4.80 (2-8)	6.50 (5-9)	9.33 (6-13)	3.67 (2-5)
Single words	4.00 (1-10)	6.20 (5-9)	3.83 (0-9)	9.33 (9-10)	11.00 (8-15)
Multiple words	0.86 (0-2)	1.20 (0-3)	1.00 (0-2)	1.33 (0-3)	1.67 (1-2)
Total	22.6 (6-27)	22.4 (15-30)	20.8 (8-35)	39.0 (34-48)	33.0 (27-42)
Jamar (Lucy)
Responses	0.0 (0-0)	0.60 (0-1)	1.60 (0-6)	1.00 (0-3)	0.33 (0-1)
Vocalizations	5.22 (1-12)	12.60 (7-17)	5.20 (3-8)	4.00 (2-7)	1.67 (1-2)
Gestures	6.67 (1-12)	6.20 (3-10)	3.20 (0-6)	7.67 (2-16)	0.0 (0-0)
Single words	7.78 (3-20)	8.60 (4-14)	6.00 (4-9)	5.67 (4-7)	4.00 (2-8)
Multiple words	4.22 (2-10)	7.20 (6-10)	11.0 (6-16)	11.0 (10-12)	4.67 (4-5)
Total	40.1 (23-61)	57.6 (49-70)	53.4 (38-68)	56.0 (43-65)	23.7 (18-33)

Note. B = Baseline; I₁ = Delayed Feedback; I₂ = Immediate Feedback; M = Maintenance; G = Generalization; Responses = Appropriate Responses; Total = Total Weighted Communication.

Expressive communication

Our fourth research question was to determine associations between our intervention and focus children’s expressive communication. Focus children’s expressive communication (i.e., vocalizations, gestures, single words, multiple words) was reported next and organized by condition. Table 3 includes means and ranges for each type of communication as well as the total weighted communication.

During baseline, Liam, Asher, and Jamar were expressively communicating at varying levels, but all three were using each type of expressive communication to some extent. Liam was primarily using multiple words and unintelligible vocalizations, Asher was gesturing most often, and Jamar was using single words and gestures most frequently. Their average total weighted communication during baseline ranged from 22.6 (Asher) to 57.8 (Liam).

During the delayed feedback condition, Liam had a marked increase in the number of multiple words used, which drastically increased his total weighted communication. This increase was not sustained into the immediate feedback condition, and Liam’s communication levels were comparable to his levels in the baseline condition. Asher’s performance during both delayed and immediate feedback conditions was fairly consistent with his levels of communication during baseline for all four types of expressive communication. Jamar doubled his vocalizations from baseline to delayed feedback but then regressed back to baseline when he transitioned into the immediate feedback condition. Jamar used on average three more multiple-word utterances per delayed feedback session; Jamar’s use of multiple-word utterances continued to increase at steady rates with an average of four more utterances during the immediate feedback condition than the delayed feedback condition.

Maintenance data for Liam were comparable to his performance during the delayed feedback condition in that his use of multiple-word utterances and subsequently his total weighted communication were again high. Asher also demonstrated growth in his expressive communication during the maintenance condition. He doubled his use of vocalizations and single words, and also increased his use of gestures. These changes nearly doubled his total weighted communication during maintenance, compared with all other conditions prior. Jamar’s expressive communication during maintenance was similar to that of the delayed and immediate feedback sessions.

Liam’s use of vocalizations, gestures, and single words were low during generalization, but his use of multiple words was comparable to that of the immediate feedback condition. Overall, his total weighted communication during generalization was lower than his baseline performance. Asher’s use of vocalizations and multiple words was comparable to his performance during feedback conditions. He used fewer gestures during the generalization condition, but more single words. His total weighted communication was about 1.5 times that of his communication during baseline and feedback conditions. Jamar had very limited communication during generalization. He used no gestures and less than two vocalizations. He used about four single-word and multiple-word utterances per session. Collectively, he communicated about half as often during generalization than during any other condition of the research.

Discussion

Summary of Results

Preservice teacher preparation research in the field of special education is limited (Leko & Brownell, 2011). Because there is a disconnect that frequently occurs between the college and community setting, there is a need to identify mechanisms to support preservice teachers in field placements where they can apply and practice skills that will result in their use of evidence-based practices (Kennedy et al., 2016). This research provides support and new findings for the current literature base regarding preservice teacher preparation, performance-based feedback, and inclusive early childhood environments as settings to provide services to children with ASD. It extends former research suggesting performance-based feedback has a positive impact on preservice teachers’ use of target skills (Barton et al., 2016; Barton & Wolery, 2008; Coogle et al., 2015; Scheeler et al., 2012). However, this research adds to the literature by examining preservice teachers’ outcomes while engaging with a focus child with ASD across two feedback conditions (delayed and immediate), focus children’s responses, and communication outcomes. In addition, maintenance and generalization data provide additional insights into mechanisms to promote these outcomes for teachers.

Preservice Teacher Outcomes

The purpose of this study was to determine whether a functional relation existed between specific forms of feedback (delayed and immediate) and preservice teacher outcomes. These findings are important as it is the first study to explore two types of feedback systems (delayed and immediate) within an inclusive early childhood environment and a teacher preparation program. Our results align with previous findings suggesting that both delayed and immediate forms of feedback enhance target skills of preservice teachers (Barton et al., 2016; Barton & Wolery, 2008; Coogle et al., 2015). This finding has practical importance because it provides teacher educators flexibility in selecting the method of providing feedback as both forms of feedback delivery were effective.

Given both the design of our study and variability in preservice teacher outcomes during delayed and immediate feedback, it is not possible to identify one form of feedback as more effective; however, this research does provide a foundation for looking more in-depth at various feedback systems and provides a rationale for further exploration. For example, teachers who prefer detailed feedback and those who benefit from thinking time to reflect on that feedback may perform best with delayed feedback. Conversely, teachers who have strong knowledge of the strategies, but need support in understanding how to implement them effectively in natural environments may benefit most from immediate feedback. Importantly, for the preservice teachers in our study, all increased their use of target antecedents or consequences upon beginning delayed feedback, and one participant (Olivia) further increased her use of target skills upon receiving immediate feedback. If these results replicate in future studies, this finding may be particularly important for target skills that need to be implemented at high frequencies to be effective. Furthermore, all preservice teachers maintained their use of their target antecedents or consequences; however, only one participant consistently generalized antecedents and consequences into a new routine (Amelia). Together these findings suggest that preservice teachers may need different levels of support to successfully implement evidence-based practices. For example, while some preservice teachers did not need support in generalizing practices (i.e., Amelia), other preservice teachers may need feedback in other types of routines to support their use of evidence-based practices. The research design that we used does not allow us to compare one intervention being more effective than the other; however, visual analysis does suggest that some participants’ trend lines accelerated upon receiving immediate feedback (i.e., Olivia and Lucy), whereas Amelia’s data stayed rather stable and then decelerated, which may have implications regarding the various supports preservice teachers need to maximize their use of evidence-based practices. In addition, preservice teachers may prefer and feel that they respond better to one method of feedback over another.

Focus Child Outcomes

In addition, this study adds to the literature as it has implications regarding child outcomes. Child responses are important to examine for a child with ASD to attain a skill because they must practice the skill multiple times (Coogle et al., 2015). In this case, social communication is the target to determine if performance-based feedback provided opportunities for a child to practice social communication. Focus children’s responses differed across delayed and immediate feedback, increasing during delayed feedback and further increasing within the immediate feedback condition. It is difficult to determine whether their responses were based upon the type of antecedent and consequence, the type of feedback, or the time that had passed by the time preservice teachers were in the immediate feedback condition which allowed focus children opportunities to practice responses resulting in a desired consequence. Focus child responses were sustained into maintenance and into generalization, when opportunities to communicate were provided.

Communication has both social and academic implications (Koegel, Matos-Freden, Lang, & Koegel, 2012), and therefore, we examined the results of performance-based feedback on focus child communication more generally. Our analysis suggests that Jamar and Liam increased their use of weighted communication during delayed feedback. Once Jamar’s weighted communication increased during the delayed feedback condition, it stayed relatively consistent upon entering immediate feedback, and maintenance; however, his communication decreased during generalization (below baseline). This may be due to the decelerating trend of Lucy’s use of antecedents and consequences during maintenance and lower level of antecedents and consequences during generalization. Liam’s weighted communication increased during delayed feedback, decreased during immediate feedback, increased again during maintenance, and decreased during generalization at a level similar to baseline. This pattern is difficult to understand beyond recognizing that Olivia rarely used antecedents and consequences during generalization which is when Liam’s communication decreased to a level below baseline. Asher, however, demonstrated a consistent level of communication until an increase during the maintenance condition occurred and then was sustained into generalization. This may be related to Amelia’s sustained use of antecedents and consequences across time as she was the only preservice teacher to generalize antecedents and consequences at a level consistent with intervention and maintenance. Consistent across all focus child participants was when preservice teachers did not use antecedents and consequences, child weighted communication decreased. This finding is important to continue to explore, and suggests performance-based feedback may be a system that not only increases target skills of preservice teachers but also allows children with ASD to practice desired communication skills.

Limitations

This study was conducted with a small sample of participants within a short period of time. Maintenance results may have been stronger with more sustained intervention and/or may have been different if measured for a longer period of time post-intervention (e.g., the next school year). Second, all preservice teachers were provided performance-based feedback on a different type of antecedent based upon their baseline data, which could have also impacted the outcomes (i.e., some antecedents are easier to implement than others and children respond differently to different types of antecedents). In addition, Olivia transitioned to immediate feedback earlier than intended (one session early); thus, we met What Works Clearinghouse Standards with Reservations. It is also important to note that Amelia’s supervising teacher was also receiving similar performance-based feedback. Although Amelia could not hear the feedback that was being delivered, it is possible she had opportunities to observe her supervising teacher utilizing embedded learning opportunities and this provided her additional support in attaining skills. Finally, given that we did not return to baseline prior to entering the immediate feedback condition, it is not certain that the changes in practice from immediate feedback were the result of that condition alone, or potentially carried over from the delayed feedback condition. Even with this design limitation, our primary design was the multiple-probe design and based on that design, we met standards for the quantity and frequency of probes (i.e., no data are missing) and the session requirements, which are necessary to make casual claims from the data.

Implications for Future Practice and Research

This study has several implications for future practice and research. First, we know that teachers in inclusive early childhood environments feel unprepared to meet the diverse needs of children within their classrooms, including students with ASD, and there is a frequent disconnect between the college classroom and school classroom (Busby et al., 2012; Kennedy et al., 2016; Ostrosky et al., 2013); therefore, it is important to consider supports to enhance the preparation of preservice teachers. In traditional field settings using face-to-face observations it is difficult to provide preservice teachers the level of support necessary to take the skills they learn in the higher education setting and implement them in a school-based setting (Scheeler et al., 2012). Consequently, it is necessary to consider feasible systems that will allow teacher educators to support preservice teachers through use of performance-based feedback. The methods investigated in this research include two different performance-based feedback systems that could be used by higher education instructors as part of field placements. It is also important to consider the role of the supervising teacher and how training both the preservice teacher and the supervising teacher can improve outcomes for one or both teachers. Amelia’s teacher was participating in another research study where she was also receiving support to use embedded learning opportunities. Amelia’s results (which followed different patterns than the other two participants) may have been impacted by her supervising teacher modeling these practices, or discussing the use of embedded learning opportunities, in addition to Amelia receiving feedback regarding her practice. It is impossible for us to determine this with certainty, as we did not measure or consider supervising teacher’s use of embedded learning opportunities as part of this study. However, this reflects an important consideration for researchers and teacher educators.

In addition, the results of this study have implications for future research. Although all preservice teachers increased their use of target skills, it would be beneficial to learn more about delayed and immediate feedback through an alternating treatment single-case design targeting different skills. Although preservice teachers maintained their use of target skills, only one preservice teacher generalized these skills to a new routine. It would be beneficial to identify effective ways to program for generalization or to support participants in generalizing target skills more consistently (Scheeler, 2008). Considering a system of tiered supports for preservice teachers in their use of target skills and promoting use of these skills to generalization may be a consideration for future research as this finding aligns with our former research suggesting preservice teachers experience difficulty in generalizing target skills (Coogle et al., 2015). Confounding variables within the environment such as supervising teacher’s use of practices and how the supervising teacher impacts the preservice teacher’s use of target strategies such as embedded learning opportunities would also be an addition to the literature.

Furthermore, it would be valuable to learn more about how this work impacts child outcomes. As indicated, this study took place over a short period of time, but an outcome such as expressive communication is a more distal measure. Therefore, conducting a study like this across a longer period of time may provide more information regarding the effect on child outcomes. This could have beneficial implications for not only preservice teachers but also institutes of higher education and school-based setting partnerships who serve children with communication delays.

Conclusion

It is critical to identify feasible and effective means to support preservice teachers’ transfer of knowledge to practice because there is gap between evidence-based practices and those practices taking place in inclusive early childhood classrooms (Noh et al., 2009; Pretti-Frontczak & Bricker, 2001). One means to support this transfer is performance-based feedback. To be able to deliver performance-based feedback, it is important for institutes of higher education as well as administrators to consider those systems which are both effective and feasible, but also beneficial for child participants. Delayed and immediate feedback may be potential methods to enhance practice and ultimately child outcomes.

Footnotes

Appendix

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Author Biographies

Christan Grygas Coogle is an assistant professor of early childhood special education at George Mason University within the Early Childhood Education Program. Her research interests include supporting the use of evidence-based practices through the provision of effective preservice and inservice professional development and embedded interventions for young children with and without disabilities.

Jennifer R. Ottley is an associate professor of early childhood special education at Ohio University and program coordinator for the Special Education program within the Department of Teacher Education. Her teaching and research emphasize teacher and teacher candidates’ use of evidence-based practices that promote young children’s learning and development.

Sloan Storie is a doctoral candidate in special education at the University of Oregon. Her research interests focus on the use of implementation science to disseminate and implement evidence-based practices in rural communities.

Naomi L. Rahn is the research and planning section chief in the Division of Early Care and Education at the Wisconsin Department of Children and Families. Her current research interests focus on child care access and continuity.

Amy Kurowski-Burt is an assistant professor of occupational therapy at West Virginia University. She is also a doctoral candidate in educational psychology. Her current research focuses on autism services, sensory integration, health and wellness through the lifespan, and accessible and inclusive environments.

References

Auld

R. G.

Belfiore

P. J.

Scheeler

M. C.

(2010). Increasing pre-service teachers’ use of differential reinforcement: Effects of performance feedback on consequences for student behavior. Journal of Behavioral Education, 19, 169-183. doi:10.1007/s10864-010-9107-4

Barton

E. E.

Fuller

E. A.

Schnitz

(2016). The use of email to coach pre-service early childhood teachers. Topics in Early Childhood Special Education, 36, 78-90. doi:10.1177/0271121415612728

Barton

E. E.

Wolery

(2008). Teaching pretend play to children with disabilities: A review of the literature. Topics in Early Childhood Special Education, 28, 109-125. doi:10.1177/0271121408318799

Billingsley

G. M.

Scheuermann

B. K.

(2014). Using virtual technology to enhance field experiences for pre-service special education teachers. Teacher Education and Special Education, 37, 255-272. doi:10.1177/0888406414530413

Bishop

A. G.

Brownell

M. T.

Klingner

J. K.

Leko

M. M.

Galman

S. A.

(2010). Differences in beginning special education teachers: The influence of personal attributes, preparation, and school environment on classroom reading practices. Learning Disability Quarterly, 33, 75-92. doi:10.1177/073194871003300202

Brownell

M. T.

Ross

D. D.

Colon

E. P.

McCallum

C. L.

(2005). Critical features of special education teacher preparation: A comparison with general teacher education. Journal of Special Education, 38, 242-252. doi:10.1177/00224669050380040601

Busby

Ingram

Bowron

Oliver

Lyons

(2012). Teaching elementary children with ASD: Addressing teacher challenges and preparation needs. The Rural Educator, 33, 27-35. Retrieved from http://epubs.library.msstate.edu/index.php/ruraleducator/article/view/146

Center for Disease Control. (2014). Data & statistics. Retrieved from http://www.cdc.gov/ncbddd/autism/data.html

Coogle

C. G.

Ottley

J. R.

Rahn

N. L.

Storie

(2018). Bug-in-ear eCoaching: Impacts on novice early childhood special education teachers. Journal of Early Intervention, 40, 87-103. doi:10.1177/1053815117748692

10.

Coogle

C. G.

Rahn

N. L.

Ottley

J. R.

(2015). Pre-service teacher use of communication strategies upon receiving immediate feedback. Early Childhood Research Quarterly, 32, 105-115. doi:10.1016/j.ecresq.2015.03.003

11.

DeMarchena

A. B.

Eigsti

Yerys

B. E.

(2015). Brief report: Generalization weaknesses in verbally fluent children and adolescents with autism spectrum disorder. Journal of Autism and Developmental Disorders, 45, 3370-3376. doi:10.1007/s10803-015-2478-6

12.

Division for Early Childhood. (2014). DEC recommended practices in early intervention/early childhood special education 2014. Retrieved from http://www.dec-sped.org/recommendedpractices

13.

Greenwood

C. R.

Carta

J. J.

Walker

Hughes

Weathers

(2006). Preliminary investigations of the application of the Early Communication Indicator (ECI) for infants and toddlers. Journal of Early Intervention, 28, 178-196. doi:10.1177/105381510602800306

14.

Guthrie

Swineford

L. B.

Nottke

Wetherby

A. M.

(2013). Early diagnosis of autism spectrum disorder: Stability and change in clinical diagnosis and symptom presentation. Journal of Child Psychology and Psychiatry, 54, 582-590. doi:10.1111/jcpp.12008

15.

Hattie

Timperley

(2007). The power of feedback. Review of Educational Research, 77, 81-112. doi:10.3102/003465430298487

16.

Hemmeter

M. L.

Snyder

P. A.

Kinder

Artman

(2011). Impact of performance feedback delivered via electronic mail on preschool teachers’ use of descriptive praise. Early Childhood Research Quarterly, 26, 96-109. doi:10.1016/j.ecresq.2010.05.004

17.

Horner

R. H.

Carr

E. G.

Halle

McGee

Odom

Wolery

(2005). The use of single-subject research to identify evidence-based practice in special education. Exceptional Children, 71, 165-179. Retrieved from www.freewebs.com/lowvisionstuff/Single_Subject.pdf

18.

Juniper Gardens Children’s Project. (2011). ECI coding definitions. Retrieved from http://igdi.ku.edu/wp-content/uploads/2016/04/ECI-definitions.pdf

19.

Kennedy

C. H.

(2005). Single-case designs for educational research. New York, NY: Pearson.

20.

Kennedy

M. J.

Wagner

Stegall

Lembke

Miciak

Alves

K. D.

. . . Hirsch

S. E.

(2016). Using content acquisition podcasts to improve teacher candidate knowledge of curriculum-based measurement. Exceptional Children, 82, 303-320. doi:10.1177/0014402915615885

21.

Koegel

Matos-Freden

Lang

Koegel

(2012). Interventions for children with autism spectrum disorders in inclusive school settings. Cognitive and Behavioral Practice, 19, 401-412. doi:10.1016/j.cbpra.2010.11.003

22.

Kratochwill

T. R.

Hitchcock

J. H.

Horner

R. H.

Levin

J. R.

Odom

S. L.

Rindskopf

D. M.

Shadish

W. R.

(2013). Single-case intervention research design standards. Remedial and Special Education, 34, 26-38. doi:0741932512452794

23.

Leko

M. M.

Brownell

M. T.

(2011). Special education preservice teachers’ appropriation of pedagogical tools for teaching reading. Exceptional Children, 77, 229-251. doi:10.1177/001440291107700205

24.

Macy

Squires

J. K.

Barton

E. E.

(2009). Providing optimal opportunities: Structuring practicum experiences in early intervention and early childhood special education preservice programs. Topics in Early Childhood Special Education, 28, 209-218. doi:10.1177/0271121408327227

25.

Matson

Hattier

Williams

(2012). How does relaxing the algorithm for autism affect DSM-5 prevalence rates. Journal of Autism Developmental Disorders, 42, 1549-1556. doi:10.1007/s10803-012-1582-0

26.

McGee

G. G.

Daly

(2007). Incidental teaching of age-appropriate social phrases to children with autism. Research and Practice for Persons with Severe Disabilities, 32, 112-123. doi:10.2511/rpsd.32.2.112

27.

Nagro

S. A.

deBettencourt

L. U

. (2017). Reviewing special education teacher preparation field experience placements, activities, and research: Do we know the difference maker? Teacher Education Quarterly, 44, 7-33. Retrieved from https://files.eric.ed.gov/fulltext/EJ1148920.pdf

28.

Noh

Allen

Squires

(2009). Use of embedded learning opportunities within daily routines by early intervention/early childhood special education teachers. International Journal of Special Education, 24, 1-10. Available from http://www.internationaljournalofspecialeducation.com/

29.

Ostrosky

M. M.

Mouzourou

Danner

Zaghlawan

H. Y.

(2013). Improving teacher practices using microteaching: Planful video recording and constructive feedback. Young Exceptional Children, 16, 16-29. doi:10.1177/1096250612459186

30.

Ottley

J. R.

Rahn

N. L.

Coogle

C. G.

Ferron

J. M.

Storie

(2018). Associations among professional development, teachers’ use of naturalistic language strategies, and preschoolers’ functional communication. Early Education and Development. Advance online publication. doi:10.1080/10409289.2018.1492296

31.

Parker

R. I.

Vannest

K. J.

Davis

J. L.

Sauber

S. B.

(2011). Combining nonoverlap and trend for single-case research: Tau-U. Behavior Therapy, 42, 284-299. doi:0005-7894/11/284-299

32.

Pretti-Frontczak

Bricker

(2001). Use of the embedding strategy during daily activities by early childhood education and early childhood special education teachers. Infant-Toddler Intervention: The Transdisciplinary Journal, 11, 111-128. Retrieved from http://eric.ed.gov/?id=EJ629406

33.

Scheeler

M. C.

(2008). Generalizing effective teaching skills: The missing link in teacher preparation. Journal of Behavioral Education, 17, 145-159. doi:10.1007/s10864-007-9051-0

34.

Scheeler

M. C.

Bruno

Grubb

Seavey

T. L.

(2009). Generalizing teaching techniques from university to K-12 classrooms: Teaching preservice teachers to use what they learn. Journal of Behavioral Education, 18, 189-210. doi:10.1007/s10864-009-9088-3

35.

Scheeler

M. C.

McKinnon

Stout

(2012). Effects of immediate feedback delivered via webcam and bug-in-ear technology on preservice teacher performance. Teacher Education and Special Education, 35, 77-90. doi:10.1177/0888406411401919

36.

Scott

L. A.

Gentry

Phillips

(2014). Making preservice teachers better: Examining the impact of a practicum in a teacher preparation program. Educational Research and Reviews, 9, 294-301. doi:10.5897/ERR2014.1748

37.

Snyder

Hemmeter

M. L.

McLean

Sandall

McLaughlin

Algina

(2018). Effects of professional development on preschool teachers’ use of embedded instruction practices. Exceptional Children, 84, 213-232. doi:10.1177/0014402917735512

38.

Vannest

K. J.

Parker

R. I.

Gonen

(2011). Single case research: Web based calculators for SCR analysis (Version 1.0) [Web-based application]. College Station: Texas A&M University. Available from singlecaseresearch.org

39.

Wong

Odom

S. L.

Hume

K. A.

Cox

A. W.

Fettig

Kucharczyk

… Schultz

T. R.

(2015). Evidence-based practices for children, youth, and young adults with autism spectrum disorder: A comprehensive review. Journal of Autism and Developmental Disorders, 45, 1951-1966. doi:10.1007/s10803-014-2351-z

40.

Woods

J. J.

Wetherby

A. M.

(2003). Early identification of and intervention for infants and toddlers who are at risk for autism spectrum disorder. Language, Speech, and Hearing Services in Schools, 34, 180-193. doi:10.1044/0161-1461(2003/015)

41.

Zwaigenbaum

Bryson

Garon

(2013). Early identification of autism spectrum disorders. Behavioural Brain Research, 251, 133-146. doi:10.1016/j.bbr.2013.04.004