Abstract
The purpose of this study was to examine the effectiveness of email-delivered performance feedback to teachers working in inclusive early childhood classrooms. A concurrent multiple probe across-participants design was used to examine the relation between performance feedback delivered via email and teachers’ use of play expansions. Results indicated that email was an effective method for delivering performance feedback, subsequently increasing teachers’ use of play expansions when individualized to meet the needs of teachers; however, the complexity of children’s play behaviors did not increase.
Keywords
Play is an important context for learning and a critical milestone that contributes to the development and well-being of young children (Ginsburg, 2007). Play provides children with multiple opportunities to learn and engage with the social and physical environment (Barton, 2015; Lifter et al., 2011). Furthermore, play is the context in which children learn and interact with their environments, including caregivers and materials. It also is a context in which skills learned across several domains can be practiced (Morrison et al., 2002). Given the established benefits of play, early childhood settings should ensure that all children have multiple opportunities and the needed supports for engaging in sustained play of increasing complexity (National Association for the Education of Young Children, 2009).
There are two overarching types of play: social play (Parten, 1932) and object play (Barton, 2010, 2016). Social play is defined relative to the child’s interactions with peers, with or without objects (Parten, 1932), whereas object play is defined relative to the child’s behaviors in relation to objects, toys, or materials (Belsky & Most, 1981). Although these types of play are defined separately, they often occur simultaneously. Social and object play skills develop in a predictable sequence that corresponds with other developmental domains such as language, cognitive, and social development. Providing all children opportunities to engage in social and object play is critically important for ensuring their access to the social, physical, and instructional environment.
For many children, simply providing a nurturing and responsive environment with time for unstructured play is sufficient for ensuring they acquire the benefits of play. However, these universal supports are not always adequate for ensuring the same benefits are accessible to children with disabilities. Children with disabilities engage in play less often and with less variety than their typically developing peers (Sigman et al., 1999). These differences in play affect children with disabilities in multiple ways. First, play delays can restrict access to their social, instructional, and physical environments. For example, children who frequently engage in repetitive play often have fewer opportunities to interact with their peers and join their play. Second, play is an important skill and context for learning; complex and socially valid learning opportunities can be embedded into play. For example, in preschools, free play and center time make up a large portion of the day and can be used to promote social interactions and provide instruction (Division for Early Childhood, 2014). However, if children are engaging in less complex social and object play, there are fewer opportunities to embed learning opportunities. Teaching children to play with objects in a manner similar to their peers might be important for supporting social play (Lifter et al., 2011).
Play Expansions
There is a burgeoning body of research focused on systematic play instruction for young children with disabilities (Barton, Murray, et al., 2019). This literature demonstrates that systematic prompting and differential reinforcement are related to increased frequency and diversity of play behaviors in young children (Barton, Choi, & Mauldin, 2019; Barton, Gossett, et al., 2019). However, most of these procedures involve multistep prompting and intensive, individualized interventions. Single-step procedures that can be implemented across multiple children and during typical free play contexts are also needed. For example, play expansions are comprised of contingent adult imitation of a child’s play action and then modeling another play action that is closely related to the child’s previous play action (Frey & Kaiser, 2011). In this manner, the adult models a slightly more complex play action at a time when the child is highly likely to be attending to it. Play expansions may be more feasible for teachers to implement during typical routines and activities in early childhood classrooms when compared with other, more intensive interventions, such as system of least prompts or constant time delay, due to the ease and flexibility of use. Play expansions also are versatile and can be used with children of varying ages and across play materials, including those typically found in early childhood classrooms.
Play expansions are modeled after language expansions, which have been shown to be effective in increasing children’s vocabulary (Kaiser et al., 2000; Kaiser & Trent, 2007). Therefore, it is reasonable to predict that play expansions may increase children’s use of novel play actions. Frey and Kaiser (2011) conducted the only study to date that has examined the isolated impact of play expansions on the play behaviors of children; however, they found equivocal results with substantial variability across participants. Research has also included play expansions as one component of larger, comprehensive intervention packages, which has successfully improved child outcomes (Ingersoll & Schreibman, 2006; Wong et al., 2007; Wright et al., 2013). Across these studies, interventions were implemented by researchers rather than endogenous staff, which also limits their ecological validity.
Training Teachers to Facilitate Play Skills
Given the benefits of play and the known delays in play skills for children with disabilities, play instruction should be a regular part of preschool classrooms. Researchers have trained teachers to use response prompting procedures (e.g., system of least prompts), contingent imitation, and behavior-specific praise to increase the frequency and diversity of play in young children (Barton, 2015; Barton, Choi, & Mauldin, 2019; Barton et al., 2013; Barton, Gossett, et al., 2019; Barton & Wolery, 2010). For example, Barton (2015) trained teachers to teach generalized pretend play using a brief didactic training, followed by ongoing feedback. Researchers have also trained teachers to embed other instructional content (i.e., communications strategies, social skills, requesting, labeling colors, and shapes) into a play context (Coogle et al., 2015).
Performance feedback
Although researchers across these studies use a variety of methods to train teachers, all used some form of performance feedback. Performance feedback occurs when individuals are given verbal, written, or graphical feedback about their implementation of an intervention during an observation in an effort to improve their implementation during subsequent observations (Casey & McWilliam, 2008; Fallon et al., 2015). Researchers have found that electronic feedback (e.g., email, text message) can be an effective and efficient method for delivering performance feedback (Artman-Meeker & Hemmeter, 2013; Barton et al., 2016; Hemmeter et al., 2011; O’Flaherty et al., 2019; Simonsen et al., 2010). Furthermore, varied forms of electronic feedback (i.e., email, text message) have been found to be effective at changing teacher behaviors (Artman-Meeker & Hemmeter, 2013; Barton et al., 2016; Barton, Rigor, et al., 2019). Ongoing email performance feedback also might promote dialogue and build rapport between the coach and teacher receiving feedback, without the time commitment and scheduling challenges of a face-to-face meeting. To date, no researchers have examined the relation between email performance feedback, teachers’ use of play expansions, and child play behaviors.
Current Study
We examined the effect of email performance feedback on teachers’ use of play expansions in their classrooms. Our study was guided by the following research questions: First, does the use of brief training plus specific email performance feedback increase the number of play expansions, teachers use in early childhood classrooms? Second, after receiving a brief training plus specific email performance feedback, does teachers’ use of play expansions maintain over time with general feedback and no feedback, and during covert observations? Third, does teachers’ use of play expansions change the type, frequency, or diversity of children’s play, compared with nonintervention conditions? Finally, do teachers find this intervention to be effective and feasible?
Method
Participants, Coach, and Researchers
This study took place in a university-based inclusive preschool in a southeastern state. Four teachers were recruited based on recommendations from the preschool director (see Table 1 for additional information about teachers). Inclusion criteria required that teachers (a) work in a classroom with children with disabilities (i.e., not an itinerant teacher), (b) are a member of a teaching team who can engage with one child for 10 min a day, (c) check their email at least once every 24 hr, (d) can nominate a child with low play skills compared with their peers, and (e) used fewer than five play expansions across two 10-min direct observations, conducted by the first author across two different days. The first author served as the coach (i.e., researcher) and sent all emails; she was a 22-year-old Asian American graduate student in special education and applied behavior analysis. She had multiple applied experiences with young children during which she used play expansions; however, this was her first experience as a coach. She was supervised by a White, female faculty member in special education who was a board-certified behavior analyst-doctoral level and had over 15 years of experience coaching in early childhood settings. Research assistants (i.e., graduate students in early childhood special education) video recorded all sessions and served as reliability coders.
Teacher Participants.
Note. AA = Associate of Arts degree; BA = Bachelor of Arts degree; EC = early childhood; ECE = early childhood education; Educ. = elementary education; GS = general studies.
Kathy and Sue were working in the same classroom. bTeaching fellows were preservice teachers. cEmployed as an assistant teacher but working as lead teacher during study due to the lead teacher’s medical leave.
Seven children also were recruited for this study. One target and one alternate child were assigned to each teacher at the beginning of the study (see Table 2 for child information); hereafter, the teacher and her assigned child are referred to as a dyad. Thirty-two days after the study commenced, some children changed classrooms due to the end of the school year, so the teacher/child dyads were revised. Inclusion criteria required that child participants (a) were 12 to 48 months of age at the beginning of the study (i.e., measured via teacher report) and (b) typically used fewer diverse play actions relative to their peers based on teacher nomination.
Child Demographics and Teacher Assignments.
Note. T = target; A = alternate.
Setting
The study took place in four different classrooms with children aged 18 to 43 months (see Table 1 for specific classroom age ranges). Two to four teachers were present in each classroom when sessions occurred. Observations took place in the context of typical classroom activities (i.e., center time). The dyads were instructed to engage in play in one of the existing centers while other free play activities were occurring. Dyads were free to move around from center to center while the session was occurring as long as the teacher could engage with the child. Other adults, such as co-teachers, were present during sessions and conducted their typical classroom responsibilities. Other children were engaged in center time as usual and were free to join or play near the dyads at any time during the session.
Materials
Data were collected using a handheld video camera (i.e., Cannon Vixia HFR52) and coded using Procoder DV software (Tapp, 2003). PowerPoint presentation software was used to create and deliver the didactic training. A simple toy set was also used in this training to facilitate modeling and role-playing activities. The teachers used their own devices (i.e., smartphones, laptops) to read and respond to emails. The coach used her laptop and an email account created for this study to send all emails.
Target Behaviors and Data Collection
The primary dependent variable (DV) was teachers’ use of play expansions. Play expansions were measured using timed-event recording. A play expansion was defined as occurring when the teacher imitated exactly what the child did, added something related to the play actions, and mapped language onto her action (Barton et al., 2013; Frey & Kaiser, 2011). An example of a play expansion is if the child rolled a car down the ramp, the teacher said “Look!” and rolled a car down the ramp like the child, then turned the car around and rolled it back up the ramp and said, “Fast car!” A nonexample would be if the child rolled the car down the ramp and the teacher said, “Your car is yellow!” or simply imitated the child rolling the car down the ramp.
Data also were collected regarding the diversity, type, and frequency of play actions observed by each target child. Play actions were measured with timed-event recording using four steps. First, each discrete play action was coded as same or different. Play actions were coded as different if they had not occurred previously in the session and as the same if they had occurred previously in the session. Second, each play action was coded for play type using a taxonomy of pretend play described by Barton (2010, 2015, 2016) and adapted for use in this study. Third, the total number of same and different play actions was summed to get a total frequency of play actions per session. Finally, we recorded the number of diverse actions used across sessions and analyzed using cumulative records (Kennedy, 2005). However, there was no change in trend across conditions; data are not presented in this manuscript but are available via email to the first author.
Interobserver Agreement (IOA)
IOA data were collected for 39% of all sessions. The primary IOA coder was masked to conditions. IOA was calculated using point-by-point agreement with a 3-s window for agreements (Ledford et al., 2018). IOA was calculated by dividing the number of agreements by the number of agreements plus the number of disagreements and then multiplying this by 100. When IOA fell below 90% agreement, coders met to have a discrepancy discussion. The average IOA across all teachers, children, and conditions was 93% (see Table 3 for more details).
Interobserver Agreement Means and Ranges per Condition and Participant.
Note. GF = general feedback; SF = specific feedback..
Experimental Design
A concurrent multiple probe across-participants design was used. We used four time-lagged tiers with four conditions: baseline, specific performance feedback, general performance feedback, and maintenance (Gast et al., 2018). We used visual analysis to identify functional relations and establish experimental control (Barton et al., 2018). Experimental control was established with three demonstrations of behavior change (across participants) at three different points in time at intervention onset and not when the intervention was introduced in previous tiers. This design meets contemporary single-case design standards (Kratochwill et al., 2013).
Procedures
All observation sessions were 10 min in duration and occurred in each participant’s classroom during their regularly scheduled center times. All sessions (excluding generalization and covert sessions) were scheduled with the teacher at least 12 hr prior. Sessions occurred up to 5 times per week, based on the teacher’s schedule. Each participant was paired with the same target children (with changes occurring after a scheduled school break). Following each session, the coach coded the data using ProcoderDV (Tapp, 2003). An email was sent to each participant in the 24-hr period following their session. All emails contained a greeting, statement of thanks, question for the participant, and the date of the next observation.
During all sessions, the participants were asked to stay near their target child during the observation, with the understanding that this was not always possible. Data collection continued for the participant (i.e., teacher) even if the target child was not in the frame. If the target child was absent, sessions were conducted with the teacher and alternate child. If both the target child and alternate child were absent, no session occurred. In addition, sessions did not occur if fewer than two teachers were present in the classroom, as to minimize disruptions to the classroom routine and allow the teacher to conduct her regular classroom duties.
Baseline
In baseline conditions, all previously stated control variables remained the same (i.e., 10-min sessions conducted by a research assistant with a video camera during free play time). Following each observation, the coach sent an email with a greeting, thank you statement, question for the participant, and the date of the next observation. Neither specific feedback (SF) nor general feedback (GF) was included in these emails.
Didactic training
Once baseline stability was established and before intervention commenced, each participant received a didactic training. This training lasted 15 to 20 min and was conducted in a location convenient to the participant (e.g., teacher’s office, teacher’s classroom, nearby coffee shop, or coach’s office). During the training, the coach used a PowerPoint presentation with embedded video examples. The coach provided a definition of play expansions, described how the participants could use them in their classrooms, and outlined the procedures for implementing play expansions. After reviewing this content, the participant practiced using play expansions (e.g., via role-play with toys) and the coach provided performance-based feedback.
SF
After receiving the brief training, the coach immediately began providing SF. All previously stated control variables remained the same and the coach sent an email within 24 hr of the observation. The SF emails consisted of an (a) greeting, (b) thank you statement, (c) report of the number of play expansions used, (d) at least one supportive feedback statement including a specific example of a play expansion used during the observation, (e) at least one constructive feedback statement including an example of a play expansion that could have been implemented differently or a scenario during the observation in which the participant missed an opportunity to use a play expansion, (f) a question for the participant, and (g) the date of the next observation.
Adaptations
If teachers did not increase their use of play expansions, we planned two adaptations. The first adaptation was the addition of a goal set by the coach based on current performance. The coach set the goal within the email on the first day, but the participant could select her own goals each subsequent day (also via email). Emails were the same as the SF conditions plus a reference to the specified goal. If the participant met the goal for the day, the coach provided supportive feedback in the email for meeting the goal. If the participant did not meet the goal, the coach told the participant that they did not reach the goal, reminded them of the goal, and provided encouragement for the next day.
If play expansions did not increase with goal setting, we added in-person coaching (IP). During IP, the coach briefly reminded the participant of the steps for using play expansions prior to each session. If the participant independently used a play expansion, the coach provided immediate, SF. Approximately once per minute, the coach told the participant how to use a play expansion in that moment (i.e., “Imitate H, then jump the person”). Following IP sessions, an email identical to the SF email was sent. Only one teacher—Christine—received these adaptations.
GF
The GF condition commenced when the participant’s use of play expansions increased and stabilized during SF. The GF email consisted of a (a) greeting, (b) thank you statement, (c) general statement about the participant’s use of play expansions (e.g., “Nice job using play expansions today!”), (d) question, and (e) the date of the next observation.
Maintenance
Maintenance sessions took place following the conclusion of the GF condition. All previously stated control variables remained the same (i.e., 10-min sessions conducted by an observer with a video camera during free play time). The emails delivered during maintenance were identical to emails delivered during baseline.
Covert observations
Throughout all phases of the study (i.e., baseline, SF, GF, maintenance), covert observations were planned to occur at least once per week using a random number generator. Covert observations were conducted from small observation rooms adjacent to each classroom with a one-way mirror and speaker system. The coach could see everything and hear everything said by adults and most of what children said, but the teachers and children could not see they were being observed. The participants were aware of covert observations but did not know the days or times when they were planned to occur as they changed weekly. No emails were sent following covert observations.
Procedural Fidelity
Procedural fidelity was measured in two ways. First, each of the didactic training sessions (n=4) was video recorded and coded for procedural fidelity. The research assistant used a checklist designed for this study to record the presence or absence of each of the training components, answering participant’s questions, and providing performance feedback during the trainings (e.g., definition, examples, opportunities for practice, verbal performance feedback). Procedural fidelity was 100% for 100% of the didactic training sessions. Second, procedural fidelity data also were collected regarding the independent variable (i.e., email performance feedback; see Table 4). Emails were randomly selected using a random number generator by a research assistant; 34% of sessions were coded across conditions and participants. The research assistant used a checklist designed for this study to measure procedural fidelity across all conditions. Procedural fidelity data were calculated using the following formula: (correct behaviors / [correct behaviors + incorrect behaviors] × 100). Data were analyzed separately for each behavior, participant, and condition. Procedural fidelity of performance feedback averaged 96% across participants and conditions (see Table 5).
Email Components by Condition.
GF = general feedback; GS = goal setting; SF = specific feedback; IP = in-person coaching.
Procedural Fidelity Means and Ranges per Condition and Participant.
Social Validity
Social validity of the outcomes was measured by having masked raters (n = 12) watch randomly selected video clips from intervention and baseline condition and complete a questionnaire. The masked raters were special education graduate students. The questionnaire asked raters to indicate which video clip showed higher levels of play expansions and more diverse play. Social validity of the procedures was measured by sending a questionnaire to teacher participants (n = 4) regarding the study procedures and outcomes.
Results
Teachers’ Use of Play Expansions
Overall, there was clear behavior change with the introduction of SF followed by GF across three of the four tiers (i.e., participants) and no change prior to commencing SF. Thus, experimental control was established and a functional relation was identified. Figure 1 shows the results of teachers’ use of play expansions across all tiers (i.e., teacher participants).
Teacher’s use of play expansions.
In the first tier, Kathy had a low, stable level of play expansions in baseline. When the SF condition was introduced, there was an immediate increase in level with a variable but increasing trend; data for the other teachers remained low and stable. After eight sessions, the data showed a steady increasing trend. Data remained high with some variability in the GF condition. Maintenance began immediately after four sessions in the GF condition, during which the level of play expansions immediately decreased below GF levels but remained well above baseline levels. The variability decreased in maintenance, compared with both SF and GF conditions.
Sue had a low, stable level of play expansions in baseline, with no change when the SF condition was introduced for Kathy. After the SF condition was introduced, the data had an increasing trend, with an increase in level. After 10 sessions, the level of play expansions was increasing, and variable. When the coach introduced the GF condition, data remained relatively stable, at the same level as SF. In maintenance, data were more variable and had a slight decrease in level following a brief increase.
Joy had a low, stable level of play expansions in baseline. The data showed no change in level or trend when the SF condition was introduced for the previous participants. After the SF condition was introduced with Joy, there was an immediate increase in her level of play expansions, which remained stable. GF was then introduced, at which point there was an increase in level with an increasing trend. In the maintenance condition, there was another slight drop in level, followed by a highly variable decreasing trend.
Christine also had a low, stable level of play expansions in baseline. Baseline levels remained low and stable when the SF condition was introduced for previous participants. After the introduction of the SF condition, there was no immediate change in level, but there was a slight increasing trend. After the break, data stabilized at a lower level than previous data points. Based on this, goal setting was introduced. After six sessions of goal setting, data remained variable at a slightly higher level, so IP was introduced. Upon introduction of IP, there was an immediate increase in level, with an increasing trend. After three sessions, GF was introduced; data were at a similar level to the previous condition. Maintenance was introduced following one session of GF and showed a slight decrease in level, though still above baseline levels.
Covert observations
Data from covert observations could not be collected as planned. The coach attempted to observe from the observation booth several times a week for all participants (i.e., a minimum of 3 times per week per participant). During these attempts, teacher participants were not playing with children in a way that would permit observations focused on play expansions (i.e., they were talking to parents, cleaning up, feeding a meal, etc.). Only four brief (i.e., less than 2 min) observations were collected across two different teachers. Therefore, data were not analyzed or graphed.
Child Play Behaviors
Frequency and complexity of play
Figure 2 shows the results of target children’s frequency and complexity of play. In baseline, Leo (paired with Kathy) showed a variable number of play actions, with functional play with pretense (FPP) occurring most frequently. Upon introduction of the SF condition, data patterns did not change. In GF, there was a decrease in level, coupled with an increasing trend. He used more complex play types (i.e., object substitution, imagining absent objects and assigning absent attributes) for the first time in GF. In maintenance, there was another decrease in level, with an increasing trend. Overall, no discernible changes occurred across conditions.
Frequency and complexity of play across all child participants. Simple play consisted of sensorimotor play, relational play, and functional play. Complex play included object substitution, imagining absent objects, and assigning absent attributes.
Mason and Sam (paired with Sue before break) showed a variable number of play actions in baseline, with FPP and simple play (i.e., sensorimotor, relational and functional play) occurring most frequently. When the SF condition was introduced, the data remained consistent with baseline levels. In the GF condition, the type and number of play actions remained consistent. Variability was moderate. During maintenance, Parker (with Sue after the break) showed an increasing trend. Overall, no discernible changes occurred across conditions.
Parker (paired with Joy before break and Sue after break) and Mark (paired with Joy) had a variable number of play actions in baseline. Simple play was observed most frequently. When the SF condition was introduced, Parker’s and Mark’s data remained similar to baseline, but variability decreased. Data during GF were consistent with SF. In maintenance, there was an increase in level, followed by a decrease and stabilization with levels similar to GF and SF. Overall, no discernible changes occurred across conditions.
Max and Madison (paired with Christine) demonstrated a relatively low level of play actions, with simple play types occurring most frequently. In SF, variability of play increased with a slight increasing trend in frequency. Levels of FPP increased slightly in IP. In GF and maintenance, level and type of play remained consistent with SF levels. Overall, no discernible changes occurred across conditions.
Diversity of play within session
Figure 3 displays data regarding the diversity of play actions within sessions across all child participants. For Leo (paired with Kathy), there were moderate and variable levels of diversity in baseline. When the SF condition was introduced, there was an increasing trend with an increase in level with high variability. In GF, the data decreased in level and trend. When the maintenance condition was introduced, the level decreased and was consistent with baseline levels. Overall, an increase in diverse play actions was observed from baseline to SF, but the change did not maintain.
Within-session diversity of play actions across all child participants.
For Mason (paired with Sue), baseline data were moderate and had an increasing trend. Sam had a decrease in diversity in baseline. When SF was introduced for Mason and Sam, there was a slight decrease in level of diverse play actions but was still consistent with baseline levels. GF and maintenance levels were consistent with baseline and SF levels. Overall, no discernible changes occurred across conditions.
During baseline, data for Mark and Parker (paired with Joy) were at a moderate level and variable, with a trend that increased and then decreased. For Mark, when SF was introduced, there was an immediate increase in level and increasing trend. For Parker, levels of diverse play actions in SF were variable and consistent with baseline. For both Parker and Mark, GF and maintenance levels were consistent with baseline. As was seen with Leo, there was an increase in diverse play actions from baseline to SF, but the change did not maintain.
For Max (paired with Christine), data in baseline were highly variable. When the SF condition was introduced for Max, there was no immediate change; however, his level of diverse play actions did become more stable after four sessions. Data for Leo were stable and moderate throughout SF, GF, and maintenance, with no discernible trend.
Social Validity
When asked which video showed more play expansions by the teachers, 11 of the 12 masked raters selected the intervention video over the baseline video. When asked which video showed children engaging in more frequent play behaviors, nine of the 12 masked raters selected the intervention video over the baseline video. When asked which video showed a child engaging in more diverse play, nine of the 12 masked raters selected the intervention video over the baseline video.
All four teachers (n = 4) responded to the participant questionnaire. The questionnaire scale ranged from 1 (not willing; not feasible) to 5 (very willing; very feasible). The teachers’ average rating for their willingness to receive training and feedback on several similar interventions (including play expansions) was 4.0 and the average feasibility of implementing play expansions was 4.3. This demonstrates that the teacher participants found this intervention to be feasible in their classrooms, and that they would be willing to implement them in their classrooms. All four teachers also indicated they would prefer to receive email feedback over written, text, phone, graphical, or in person.
Discussion
Overall, teachers’ use of play expansions increased after the brief training and email feedback began. After the brief training and introduction of SF, three out of the four teacher participants (i.e., Kathy, Sue, and Joy) showed an immediate increase in their use of play expansions. The fourth teacher (i.e., Christine) showed a steady increase in play expansions over time with individualized adaptations (i.e., goal setting and IP). The intervention was effective and supports the use of performance feedback in early childhood settings. However, the child data were inconclusive and did not support the hypothesis that teachers’ use of play expansions would change the complexity, frequency, and diversity of children’s play. Furthermore, covert observations could not be conducted, which limits interpretations of the results.
The results of this study extend the research on the use of performance feedback in early childhood contexts. For example, several studies have demonstrated a functional relation between performance feedback delivered via email following a brief training and discrete teacher behaviors (Artman-Meeker & Hemmeter, 2013; Barton, Rigor, et al., 2019; O’Flaherty et al., 2019). This study extends the research on performance feedback in early childhood settings by showing the efficacy and feasibility of using email to deliver performance feedback to teachers to increase their use of play expansions with young children. Furthermore, our results support previous research and highlight the need for using individualized adaptations when coaching to ensure change in participants’ behaviors (Barton, Rigor, et al., 2019).
Ours study also contributes to the research on supporting children’s play behaviors. Frey and Kaiser (2011) found that play expansions were related to increases in child play complexity although the magnitude of the changes varied across children and results during the generalization conditions were variable. Conversely, in this study, play expansions were not related to changes in child play behaviors. These discrepancies and noneffects are important although they are more likely to be overlooked as they are rarely published (Shadish et al., 2016). Our findings suggest that teachers’ intermittent use of play expansions did not change child behaviors and teachers did not play with children during center times unless they were being observed. Although these findings are considered “noneffects,” they are critically important for ensuring we have a comprehensive and complete understanding of the relation between teacher and child behaviors. We know that the disproportionate publication of studies with clear, robust behavior change has a negative impact on future replications—which are imperative to advance science and knowledge in this area—and what we know about evidence-based practices (Cook & Therrien, 2017; Tincani & Travers, 2018). This appears to be particularly true for studies with complex variables such as teacher practices and their impact on child play behaviors (Barton et al., 2018).
This study can be used to identify the specific replications that are needed to advance research. For example, additional high-quality research is needed to examine the relation between the play expansions and child play behaviors. Researchers might ask questions related to the two important differences between this study and Frey and Kaise’s (2011) study, which were the dosage of play expansions received by target children (i.e., children in this study received fewer play expansions) and the implementers (i.e., teachers versus researchers). Furthermore, previous research has shown systematic prompting is related to increases in child play complexity (Barton, 2015; Barton, Choi, & Mauldin, 2019; Barton, Gossett, et al., 2019; Barton, Murray, et al., 2019), which should be considered when planning play interventions.
All teachers displayed low, stable levels of play expansions in baseline. Furthermore, the coach attempted to conduct covert observations during regularly scheduled free play times, which were assumed to be the highest likelihood that teachers would be playing with students. However, even with repeated attempts, trying multiple times across multiple days, the coach was unable to identify a time where teachers were engaging with children in play for more than a few brief moments. This finding is important as it suggests that teachers may not be consistently playing with children. Research has shown that children in preschool settings spend the majority of their day in play, and it provides a context for learning skills across several domains (Division for Early Childhood, 2014; McConnell, 2002). Teachers are missing valuable instruction time if they are not using play as a context for learning or supporting children’s increasingly complex play. Furthermore, all children were included in this study because their teacher reported that they had limited play skills and could benefit from additional instruction focused on play. However, this suggests that it is highly likely that teachers were not using play expansions outside of the context of this study or spending additional time with their target child teaching play skills.
Teachers implemented play expansions efficiently with minimal supports. Three out of the four teachers showed an immediate increase in their use. Specifically, the training that took place lasted between 15 and 20 min, and only occurred once per teacher. The emails took the coach approximately 10 min to write and likely only took the teachers a few minutes to read. In addition, although it was not specifically targeted by this study, teachers generalized this behavior across children without any additional support. Due to scheduling, each teacher worked with at least two target children, and some worked with three children (i.e., Sue and Christine) over the course of the study. Teachers also had a wide range of experiences (i.e., 1–11 years of experience in early childhood education) and held various positions, from lead teacher, to assistant teacher, to teaching fellow (i.e., preservice teacher). However, by the end of the study, all four teachers used play expansions effectively. This speaks to the efficacy, utility, and adaptability of our intervention.
The teachers who responded to the emails had the most robust effects (i.e., Kathy and Joy). These two participants had an email response rate during the SF phase of 100% and 83%, respectively. It is reasonable to assume that for the ongoing feedback to be effective, teachers must consume the feedback (i.e., read the email). This study supports the existing literature that shows ongoing feedback is critical to the success of an intervention (Artman-Meeker & Hemmeter, 2013; Fallon et al., 2015). Thus, performance feedback should be delivered using a format that the recipient consumes regularly. Although all teachers in this study indicated they read emails daily, there was a differential response rate to the performance feedback across teachers. Future replications might try delivering performance feedback in multiple formats (i.e., email, text, verbally) rather than just one format to increase the likelihood that the performance feedback is consumed.
Limitations
There were two major limitations in this study. First, we were not able to conduct covert observations. This suggested that teachers were not utilizing center time to teach children more complex ways to play. We do not know what teachers would have done during the covert observation sessions had they been conducted.
Second, child play behaviors did not change despite their teachers increased use of play expansions. There are several reasons that this may have happened. First, there were problems with scheduling. The child participants involved in the study were frequently pulled from the classroom during center time for therapies (e.g., speech, occupational therapy) and the alternate children were used quite frequently, in some cases almost as much as the target child (i.e., Joy’s participants). Thus, the participants were not getting a consistent dose of the intervention. Furthermore, before the break, students had been in their classroom and working with their target teacher for nearly a year, and they likely had a strong rapport. After the break, sessions were conducted in a new classroom, sometimes beginning after their first week in that class, and working with a new teacher. They likely did not have the same level of rapport or knowledge of their target child’s skills and interests, which is likely important when supporting a child’s play. Another potential reason for the lack of change in child data was the context in which sessions were conducted. In this study, we did not restrict centers; this afforded us limited control over materials and children often went to multiple centers during each session. Conversely, Frey and Kaiser (2011) used a consistent toy set throughout their intervention and had more robust changes in play behaviors. These results suggest that some children might require more robust, intensive intervention focused on play complexity.
Implications for Research and Practice
There are at least three important implications for research and practice. First, the coach anecdotally noticed that teacher responsivity increased for all participants over the course of the study (e.g., following the child’s lead, imitating the child’s actions and words, and adding language; Kong & Carta, 2013). For example, Christine was more responsive and an active play partner for her children over the course of the study. This was not the intended effect of the intervention, but it was a desirable, anecdotal change that we did not measure. Providing teachers with a specific strategy to use when interacting with children during play might be helpful for increasing their responsivity; this should be measured in future replications.
Second, the lack of data from covert observations might provide important information regarding the impact of the study. This suggests teachers might need more support and instruction related to supporting children’s play. Teachers also might need support in learning how to engage with children during center times using strategies that expand children’s play.
Third, the coach had no set criterion for the number of play expansions that teachers should be using. All teachers in this study had a different maximum number of play expansions used, ranging from 7 to 14. However, none of the children showed improvements in play behaviors. Future research should examine the rates of play expansions that are feasible to deliver and the rates of play expansions that are needed to increase child play behaviors.
Conclusion
In conclusion, we examined the effect of email performance feedback on teachers’ use of play expansions in their classrooms. We found that a brief training paired with email performance feedback was sufficient to change teacher behavior and increase their use of play expansions in inclusive early childhood classrooms. This supports the use of email performance feedback as an evidence-based practice for discrete teacher behaviors. However, teachers’ use of play expansions within the bounds of this study were not sufficient to change children’s complexity, frequency, or diversity of play. Further research should be conducted to identify ecologically valid play interventions and to evaluate ways to support teachers’ use of play expansions in the absence of the observer. Identifying efficient and effective professional development practices will enable the field to better meet the needs of teachers and improve outcomes for children.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Preparation of this article was supported in part by the U.S. Department of Education, Office of Special Education and Rehabilitative Services, Grants H325D180095 and H325K140110. However, the opinions expressed do not necessarily reflect the policy of the U.S. Department of Education and no official endorsement should be inferred.
