Abstract
Preschool classrooms provide a unique context for supporting the development of children with social-communication challenges. This study is an uncontrolled clinical trial of an adapted professional development intervention for preschool teachers (Social Emotional Engagement-Knowledge & Skills-Early Childhood). Social Emotional Engagement-Knowledge & Skills-Early Childhood is a low-resource-intensive, transdiagnostic intervention to address the learning needs of children with social-communication challenges and consists of four asynchronous online modules and three synchronous coaching sessions. The current research evaluated the feasibility and acceptability of intervention and research procedures, implemented in authentic early childhood education settings. Participants included one teacher and one target child with social-communication challenges from 25 preschool classrooms, sampled to maximize variability. Overall, the current research revealed high levels of feasibility, with 9 out of 10 benchmarks met: (a) procedures for participant recruitment reliably identified a neurodiverse sample of children with teacher-reported social-communication challenges; (b) teachers showed high levels of program engagement and Social Emotional Engagement-Knowledge & Skills-Early Childhood completion (76%); and (c) results revealed a robust pattern of gains in Social Emotional Engagement-Knowledge & Skills-Early Childhood classrooms and associations among key outcome measures (including active engagement, student teacher relationship, social-communication competencies). Implications for the design of a subsequent, larger effectiveness-implementation hybrid trial (Type 1) are discussed.
Lay Abstract
Preschool classrooms provide a unique context for supporting the development of children with social-communication challenges. This study evaluates the feasibility and acceptability of an adapted professional development intervention for preschool teachers (Social Emotional Engagement-Knowledge & Skills-Early Childhood). Social Emotional Engagement-Knowledge & Skills-Early Childhood is a low-resource-intensive, transdiagnostic intervention to address the learning needs of children with a broad range of social-communication challenges in authentic preschool classrooms. The intervention consists of four asynchronous online modules and three synchronous coaching sessions. Participants included one teacher and one target child with social-communication challenges from 25 preschool classrooms from private childcare, Head Start, and public Pre-K programs. Results reveal high levels of Social Emotional Engagement-Knowledge & Skills-Early Childhood feasibility, with 9 out of 10 feasibility benchmarks met: (a) procedures for participant recruitment reliably identified a neurodiverse sample of children with teacher-reported social-communication challenges; (b) teachers showed high levels of program engagement and Social Emotional Engagement-Knowledge & Skills-Early Childhood completion (76%); and (c) results revealed a robust pattern of gains in Social Emotional Engagement-Knowledge & Skills-Early Childhood classrooms and associations among key outcome measures (including active engagement, student–teacher relationship, social-communication competencies). This research prepares a subsequent, larger effectiveness-implementation hybrid trial (Type 1) that investigates the effectiveness of Social Emotional Engagement-Knowledge & Skills-Early Childhood for improving child outcomes and explores facilitators and barriers of program implementation and sustainability.
Keywords
The social attention, communication, and interaction of autistic 1 children differ from children without autism in characteristic ways (Bauminger-Zviely & Shefer, 2021; Sigman & Ruskin, 1999). Preschool classrooms and other natural environments where interactions occur with peers provide a unique context for supporting children’s social-communication development. Clinical and educational innovations in inclusive preschool settings focused on autism have been championed by a small number of university-affiliated model programs (Boulware et al., 2006; Capes et al., 2019; Hine & Wolery, 2006; McGee et al., 1999; Siller et al., 2022; Stahmer et al., 2011; Stahmer & Ingersoll, 2004). These programs flexibly blend high-quality practices from early childhood education (ECE; National Association for the Education of Young Children [NAEYC], 2020) and clinical autism interventions (Naturalistic Developmental Behavioral Interventions [NDBI]; Schreibman et al., 2015). Despite the promise of these model programs, few have been replicated in community settings. One possible reason is that the required blend of high-quality instruction, individualized supports, and intensive interventions is difficult to achieve in resource-constrained, community-based ECE settings.
Informed by innovative service delivery models for low- and middle-income countries, the Lancet Commission on the future of care and clinical research in autism proposed the adoption of a stepped care model “in which the least resource-intensive service is offered first, and then gradually stepped up to more intensive or specialist-delivered treatment if necessary” (p. 283; Lord et al., 2022). However, a low-resource-intensive, transdiagnostic intervention focused on social-communication challenges does not exist. The current research evaluates the feasibility/acceptability of such an intervention, implemented by teachers in authentic ECE settings. The experimental intervention was initially developed/implemented as a classroom-wide approach for Pre-K–12 general and special education settings (Social Emotional Engagement-Knowledge and Skills, SEE-KS; Rubin & Townsend, 2022; Townsend & Rubin, 2018), and adapted to target children with social-communication challenges in early childhood settings (SEE-KS-EC) for the current research.
Considering the target population
While characteristic for autistic children, problems with social communication are also observed in children with sub-clinical autism symptoms (Tufan, 2014), attention-deficit hyperactivity disorder (Marton et al., 2009), learning disorder (Bauminger et al., 2005), and disruptive behavior disorder (Gilmour et al., 2004). Furthermore, secondary problems in social communication may emerge for children with other communication disorders (e.g. speech sound or fluency disorder), children with social anxiety, or children with intellectual disability (Topal et al., 2018). A transdiagnostic, low-resource-intensive intervention focused on social-communication challenges fills three gaps. First, teacher attitudes about implementing evidence-based practices specific to autism tend to be negative (Suhrheinrich et al., 2021). A transdiagnostic intervention targeting a larger group of children may be more acceptable/feasible in general education settings, compared to interventions focused on specific conditions such as autism (such an intervention could more easily be embedded during summer pre-planning and sustained across school years). Second, while autistic children can be reliably identified by 18 months, most children are not diagnosed until 4 years (Maenner et al., 2020). A transdiagnostic, low-resource-intensive intervention could be implemented rapidly/flexibly when learning challenges are noticed prior to a referral for evaluation (Green, 2019; Lord et al., 2022). Third, infant researchers are increasingly calling for transdiagnostic intervention approaches, contending that the impact of early intervention is heightened if they target prodromal symptoms that are shared across neurodiverse groups (Hampton & Rodriguez, 2022; Talbott & Miller, 2020).
Underlying assumptions and approach of SEE-KS-EC
Learning is possible as children engage appropriately with the physical environment, materials, and other people in the classroom (Bailey & Wolery, 1992). Moreover, the quality of children’s classroom active engagement (AE) likely mediates the relation between teacher supports/instruction and children’s learning outcomes (Shih et al., 2021; Vitiello & Williford, 2016). SEE-KS-EC is a professional development intervention for preschool teachers that aims to enhance AE across three domains: investment, independence, and initiation. These three AE domains were selected because each is likely to be impacted by social-communication problems (Fitzpatrick, 2012; Searle et al., 2013; Sparapani et al., 2016; Yoder et al., 2019): (a) SEE-KS-EC stipulates that children show investment when their attention is sustained, their faces express focus/joy, and their bodies are oriented toward others. Current clinical autism interventions utilize a variety of strategies to promote children’s social motivation and interest in a range of materials, utilizing strategies such as child-preferred activities and natural reinforcement (Schreibman et al., 2015). Similarly, researchers focused on children with emotional/behavioral problems have emphasized that children’s interest, motivation, and emotional engagement is a central component of classroom engagement (Searle et al., 2013). (b) SEE-KS-EC stipulates that children show independence when they anticipate and imitate others’ actions, follow routines, and respond to visual or contextual cues. Autistic children show characteristic challenges in self-regulatory capacities, including delayed gratification, effortful control, and emotion regulation (Jahromi et al., 2019). Moreover, preschooler’s self-regulatory skills have been linked to early academic achievements (math, writing skills) and cognitive achievement in adolescence (Fitzpatrick, 2012). Particularly, in classroom settings, children’s ability to recognize emotions and regulate behaviors is essential for following classroom behavioral expectations, managing transitions, and completing activities independently (Morgan et al., 2018). (c) SEE-KS-EC stipulates that children show initiations when they communicate frequently with a range of partners (e.g. peers, teachers) and demonstrate a range of communicative modes (e.g. gaze, gestures, spoken language) and functions (e.g. requesting, commenting, affiliating). Several recent clinical trials of school-based interventions for young children with autism spectrum disorder (ASD) used observational classroom measures of child-initiated communication to capture targeted learning processes (Boyd et al., 2018; Morgan et al., 2018; Shih et al., 2021).
The current study
ECE settings in the United States include a mixture of privately funded childcare centers, nurseries, and preschools, as well as publicly funded Head Start, Pre-K, or special education programs (Friedman-Krauss et al., 2020; Laughlin, 2013; Siller et al., 2021). Service systems differ vastly regarding funding sources, regulations for licensing/operation, staff qualifications, access to professional development, and student population. This study is an uncontrolled clinical trial investigating the feasibility and acceptability of SEE-KS-EC across three types of community-based ECE settings: private childcare, Head Start, and public school-based Pre-K. This research was completed in preparation of a future, larger effectiveness-implementation hybrid trial (type 1; Landes et al., 2019) with the aims of investigating effectiveness and exploring implementation issues.
Because this study is designed as a feasibility study in preparation of a full-scale effectiveness trial, the primary focus is on implementation outcomes, aiming to identify elements of the research methods and intervention protocol that need modification and explore how intervention-related changes might occur (Bowen et al., 2009; Leon et al., 2011). Implementation outcomes are distinct from clinical treatment outcomes in that they provide proximal indicators of the implementation process. Proctor and colleagues proposed a working taxonomy of implementation outcomes that include five constructs that should be prioritized during early stages of the implementation process—acceptability, adoption, appropriateness, feasibility, and fidelity (Mettert et al., 2020; Proctor et al., 2011), which informed the five specific aims of the current study: (a) to evaluate the feasibility of enrolling target children with social-communication challenges, (b) to evaluate teacher adherence to the intervention and research protocol, (c) to complete a preliminary evaluation of implementation fidelity and teacher self-efficacy, (d) to complete a preliminary evaluation of pre-post gains in observed and teacher-reported clinical child outcomes, and (e) to complete a preliminary evaluation of associations between implementation and clinical child outcome measures.
Methods
Settings and subjects
The study took place in 25 ECE classrooms: 8 classrooms in an early learning center of one public school system, 5 classrooms in four Head Start centers, and 12 classrooms in four private childcare centers. One teacher and one child from each classroom were recruited to participate in this study. Children were eligible to participate if they attended 1 of the 25 classrooms, were between 2 and 5 years of age, and identified by their teacher as a child with social-communication challenges. To identify eligible children, teachers completed a three-step process. In Step 1, teachers nominated children diagnosed with autism, by either clinical diagnosis or educational disability classification. In classrooms that did not include a child with diagnosed autism, teachers identified children with a broader range of developmental, behavioral, or social disabilities (Step 2). In the absence of an autism diagnosis or other apparent disability, teachers identified children who did not consistently engage in classroom activities (Step 3). Recruitment of children in Step 3 was supported by a validated Diagnostic and Statistical Manual of Mental Disorders (DSM-5) teacher screening questionnaire for autism (EDUTEA; Morales-Hidalgo et al., 2017). This rating scale includes 11 items, scored from 0 to 3. A total summed score of ⩾10 is considered predictive of autism and was used as a threshold for study inclusion. The eligibility of each teacher-nominated child was confirmed in an online meeting with a research team member. Teachers then invited the child’s parent to participate. If a parent of an eligible child declined participation, the teacher repeated the process to identify another child in the classroom.
Procedures
The study was completed during the 2021–2022 school year. Teachers/children in one public school and two private childcare programs were enrolled during fall (14 classrooms). Teachers/children in four Head Start and three private childcare programs were enrolled during spring (11 classrooms). Upon enrollment, teachers and parents completed a battery of surveys and classroom videos were collected. Following baseline data collection, teachers and children participated in SEE-KS-EC. Teacher surveys and classroom videos were also collected at endpoint.
Community involvement
We used an established implementation science framework (Aarons et al., 2011) to adapt SEE-KS for implementation in ECE settings. SEE-KS is a teacher-delivered intervention designed to promote the target child’s AE during natural classroom routines/activities (Rubin & Townsend, 2022; Townsend & Rubin, 2018). 2 The adaptation process was informed by 31 regional stakeholders, including state administrators (n = 5), program directors (n = 6), and teachers (n = 20) from Head Start, private childcare, and public Pre-K. The adapted intervention is referred to as SEE-KS-EC.
Intervention
SEE-KS-EC consists of four, self-paced (i.e. asynchronous) online modules designed to instruct teachers on how to promote AE during circle time: (a) Module 1: teachers learn about the elements of AE (investment, independence, initiation); (b) Module 2: teachers learn to stimulate child investment by creating meaningful classroom routines that leverage hands-on materials and provide options for active participation; teachers learn to sustain investment by adjusting the learning environment based on child characteristics/behavior and providing tools for self-regulation (visuals, role models); (c) Module 3: teachers learn to stimulate child independence by proactively teaching behavioral expectations and presenting information in multiple ways (hands-on materials, visuals, role models, verbal language); and (d) Module 4: teachers learn to promote child initiations by responding to children’s nonverbal signals, creating frequent opportunities for initiations, and providing a variety of modes for expressions (proximal objects, visuals, gestures, movement, and role-play). The four online modules include 34 short videos of didactic presentations (M = 2.5 min, range: 1–8 min), 25 exemplary classroom videos (M = 1.1 min, range: 0.25–1.75 min), and 43 written response prompts, typically embedded as case-based learning prompts within classroom videos. The SEE-KS-EC online modules were developed during the SEE-KS adaptation process. SEE-KS-EC also includes three synchronous, web-based coaching sessions, delivered by one of two trained research clinicians. Prior to each coaching session, classroom videos were collected by a research assistant. Coaching sessions use an Appreciative Inquiry framework to reflect on classroom videos (Morgan et al., 2022), providing opportunities for teachers to appreciate what is already working and to identify opportunities for increasing the target child’s AE. During a typical 10-week SEE-KS-EC cycle, teachers complete the four online modules in weeks 2, 3, 6, and 9, and the three coaching sessions during weeks 4, 7, and 10, respectively.
Measures
Data collection included four types of measures: (a) surveys for sample characterization, (b) administrative data to capture adherence to the research and intervention protocol, (c) observational measures to capture implementation fidelity and AE, and (d) teacher ratings to capture self-efficacy, student–teacher relationship, and social-communication competencies.
Surveys for sample characterization
At study entry, parents and teachers completed demographic surveys developed for the study. Parent surveys included questions about child diagnoses, parental concerns, and intervention use. Teacher surveys included questions about professional training/experiences. Furthermore, teachers completed two norm-referenced rating scales to evaluate social-communication impairment and competencies at baseline. Feasibility benchmark: at least 80% of children show social-communication impairments 1 SD above average (t scores ⩾ 60) or social-communication competencies 1 SD below average (standard scores ⩽ 85; Table 1).
A priori set SEE-KS-EC feasibility benchmarks and outcomes.
SRS-2 = Social Responsiveness Scale, Second Edition; SSIS-SEL = Social Skills Improvement System-Social Emotional Learning Edition; √ = feasibility benchmark met; X = feasibility benchmark not met.
Out of SEE-KS-EC completers.
Social Responsiveness Scale-2
The Social Responsiveness Scale, Second Edition (SRS-2; Constantino & Gruber, 2012) (65 items) is a norm-referenced rating scale, evaluating social-communication impairments. T scores between 66 and 75 are typically observed in autistic children with moderate severity.
Social Skills Improvement System: Social Emotional Learning Edition rating form
The Social Skills Improvement System: Social Emotional Learning Edition (SSIS-SEL; Gresham & Elliot, 2017) (58 items) is a norm-referenced rating scale assessing social-communication competencies. The SSIS-SEL provides a total composite score, and subscale scores for self-awareness, self-management, social awareness, relationship skills, and responsible decision-making.
Administrative data to capture adherence to the intervention and research protocol
Adherence to the research and intervention protocol was evaluated using administrative data. A priori set feasibility benchmarks are summarized in Table 1: (a) ⩾30 teachers consented, (b) ⩾80% of consented teachers lead to the successful enrollment of a teacher–child pair, (c) ⩾75% of enrolled teacher–child pairs complete the SEE-KS-EC program, (d) ⩾75% of teachers who completed SEE-KS-EC completed at least three online modules (⩾180 min) and completed at least two coaching sessions (⩾60 min), and (e) ⩾90% of teachers who completed the SEE-KS-EC program provided usable outcome data.
Observational measures to capture implementation fidelity and child AE
Coaching fidelity
Online coaching sessions were recorded using screen capture; one session per teacher was randomly selected and coded using a coaching fidelity scale developed for this study. Coding was completed by a clinician who had no prior interaction with the evaluated teacher. The SEE-KS-EC fidelity scale evaluates adherence to structural elements (10 items; e.g. the coach states the session’s focus, reviews the online modules/videos), session content (5 items; e.g. adequately describes strategies for increasing AE), and approach (4 items; e.g. provides effective positive, appreciative feedback). Structural elements were rated as present/absent, and content/approach elements were rated on a 4-point scale, ranging from 1 = not at all to 4 = very well. Full and domain scores were expressed as percentage scores. Feasibility benchmark: ⩾90% of coaching sessions achieved at least 75% fidelity.
Teacher implementation fidelity
Classroom circle-time videos were recoded before and after the intervention period with two to three synchronized, remote-controlled cameras (M = 20.43 min, SD = 7.49, range: 8–40). Teacher implementation fidelity was rated minute-by-minute on five 3-point scales, ranging from 0 = not implemented to 2 = fully implemented. Scales evaluated teacher strategies to promote child investment, independence, and initiation. Sample teacher strategies captured in this fidelity measure are provided in the introduction (see section on Underlying Assumptions and Approach of SEE-KS-EC). Twenty percent of videos were independently coded by two observers, and intra-class correlations (ICCs) were calculated at the subscale level, ranging from good to excellent agreement (ICC = 0.72–0.99).
AE
Classroom circle-time videos collected before and after the intervention period were also coded minute-by-minute to evaluate two indicators of AE (4-point scales ranging from 0 to 3). Scales evaluated children’s productivity (capturing elements of investment and independence) and directed communication (capturing child initiations; Morgan et al., 2018). Minute-by-minute ratings were averaged to compute subscale scores for each video. Twenty percent of videos were independently coded by two observers, with ICCs revealing excellent agreement (ICC = 0.74–0.85).
Teacher ratings to capture self-efficacy, student–teacher relationship, and social-communication competencies
Teachers completed the SSIS-SEL (see above) and two additional rating scales, before and after the intervention period.
Teacher Sense of Efficacy Scale—Short Form
The Teacher Sense of Efficacy Scale—Short Form (TSES-SF; Tschannen-Moran & Woolfolk Hoy, 2001) (12 items) probes teachers’ self-efficacy for classroom management (e.g. “How much can you do to control disruptive behavior in the classroom?”), promoting student engagement (e.g. “How much can you do to motivate students who show low interest in school work?”), and using instructional strategies (e.g. “To what extent can you provide an alternative explanation or example when students are confused?”). Teachers rate items on a 9-point scale from 1 = nothing to 9 = a great deal. Average ratings are computed for the three subscales (4 items each); the total score is the average of all items.
Student-Teacher Relationship Scale—Short Form
The Student-Teacher Relationship Scale—Short Form (STRS-SF; Pianta & Steinberg, 1992) (15 items) examines teachers’ relationships with an individual child in their classroom, yielding scores for conflict (7 items; e.g. “This child and I always seem to be struggling with each other.”) and closeness (8 items; e.g. “I share an affectionate, warm relationship with this child.”). Teachers rate on a 5-point scale from 1 = definitely does not apply to 5 = definitely applies. Average scores are derived for each of the two subscales (conflict, closeness) and overall (positive relationship).
Analysis approach
Baseline measures, administrative data, and measures of coaching fidelity were analyzed and presented descriptively. Outcome measures were compared before and after SEE-KS-EC using paired sample t tests. Hedges’ g coefficient was further computed to estimate effect size (ES) with 95% confidence intervals. By convention, g coefficients of 0.2, 0.5, and 0.8 are interpreted as small, medium, and large, respectively. Results for main composite scores are reported in Table 2; results for subscale scores are reported in Supplementary Table 1. Finally, we used nonparametric Spearman correlations to explore associations between implementation, teacher, and child outcomes. Analyses were performed in SAS v.9.4 (Cary, NC, USA) and SPSS v.28, and p values were evaluated for statistical significance two-sided at the 0.05 threshold.
Paired t tests comparing pre-post gains on teacher and child outcomes focusing on a single composite score for each measure (SEE-KS-EC completers only; N = 19).
STRS-SF = Student-Teacher Relationship Scale-Short Form; TSES-SF = Teachers’ Sense of Efficacy Scale-Short Form; SSIS-SEL = Social Skills Improvement System-Social Emotional Learning Edition; ES = effect size.
Results
Sample description: teacher, child, and family demographics
Figure 1 provides a CONSORT chart, summarizing participant flow and study procedures. Of n = 31 consented teachers (feasibility benchmark met; N ⩾ 30), we successfully enrolled n = 25 (81%) teacher–child pairs (feasibility benchmark met; ⩾80%). All participating teachers (n = 25, 100%) were female, 2 (8%) identified as Hispanic, 17 (68%) as White, and 6 (24%) as Black. Teachers’ educational attainment included master’s (n = 11; 44%), bachelor’s (n = 7; 28%), and associate degrees (n = 7, 28%). On average, teachers had M = 13.0 years (SD = 11.1) of teaching experience.

CONSORT diagram.
Children’s chronological ages ranged from 25 to 59 months (M = 46.8, SD = 8.8). Of the 25 children (n = 18 boys; 72%), n = 10 (40%) were White, n = 9 (36%) Black, and n = 6 (24%) of other/mixed ethnicity/race. In terms of maternal education, n = 9 (36%) mothers completed a graduate degree, n = 11 (44%) a standard college/university degree, n = 4 (16%) a partial college degree or specialized training, and n = 1 (4%) a high school diploma.
Aim 1: to evaluate the feasibility of enrolling target children with social-communication challenges
Sample average SRS-2 scores indicate high levels of social-communication impairments (t scores: N = 23; M = 70.5, SD = 14.0, range: 46–94). Similarly, SSIS-SEL scores indicate low levels of social-communication competencies (standard scores: N = 23; M = 70.7, SD = 14.3, range: 40–99). Of all children, 83% scored 1 SD above average on SRS-2 or 1 SD below average on SSIS-SEL (feasibility benchmark met; ⩾80%). According to parent report, the sample included n = 3 children (12%) with an autism diagnosis and n = 4 additional children (16%) with an Individualized Education Plan (with parents reporting hyperactivity, emotional outbursts, and compliance as dominant concerns). Finally, n = 11 parents (44%) reported a range of developmental (n = 5, 20%; speech/language delays, difficulties with peer interactions, limited fine motor skills, sensory processing issues) and behavioral concerns (n = 8, 32%; defiance, aggression, hyperactivity, emotional outbursts), or noted that their child was receiving intervention services (n = 4, 16%; speech, occupational, or play therapy).
Aim 2: to evaluate teacher adherence to the intervention and research protocol
Study completion
Of the 25 participating teacher–child pairs, n = 19 (76%) completed the SEE-KS-EC intervention (feasibility target met; ⩾75%). All dyads (100%) who completed the intervention also provided exit data (feasibility target met; ⩾90%). On average, the 19 teachers completed M = 3.2 online modules (SD = 1.5; range: 0–4), with n = 15 (79%) completing at least three online modules (feasibility benchmark met; ⩾75%). On average, the 19 teachers committed M = 216.0 min to the online modules in total (SD = 130.0 min; range: 0–472), with n = 14 (74%) committing more than 180 min (feasibility benchmark not met; ⩾75%). Similarly, on average, the 19 teachers completed 2.4 coaching sessions (SD = 0.8; range: 1–3), with n = 15 (79%) completing at least two coaching sessions (feasibility benchmark met; ⩾75%). On average, the 19 teachers committed M = 99.6 min to the coaching sessions in total (SD = 32.0 min; range: 33–139), with n = 16 (84%) committing more than 60 min (feasibility benchmark met; ⩾75%). Finally, on average, the period between baseline videos and the last coaching session was M = 3.2 months (SD = 2.3 months; range 0.5–7.2).
Because completion patterns differed considerably between fall and spring cycles, Figure 1 presents the participant flow separately for the two cohorts. Of the 14 teacher–child pairs participating during fall, 10 (71%) completed all four online modules (committing M = 275.5 min in total; SD = 91.4 min; range: 188–472) and all three coaching sessions (committing M = 116.9 min in total; SD = 16.1 min; range: 89–139). All 10 teachers also completed exit surveys and videos. On average, the period between the baseline videos and the last coaching session was M = 5.0 months (SD = 1.8 months; range: 3.0–7.2). The four remaining teachers discontinued the study after the baseline assessments, but before the first coaching session (one because the target child unenrolled from the program). None of these four teachers provided exit data.
Of the 11 teacher–child pairs participating during spring, only one teacher completed the full intervention. Anticipating the approaching end of the school year, SEE-KS-EC content was compressed into two coaching sessions for four teachers and one coaching session for four teachers. On average, the nine teachers (82%) who completed SEE-KS-EC during spring completed M = 2.2 online modules (SD = 1.8; range: 0–4), committing M = 149.9 min in total (SD = 138.9 min; range: 0–438). Similarly, nine teachers completed M = 1.7 coaching sessions (SD = 0.7; range: 1–3), committing M = 80.0 min in total (SD = 34.9 min; range: 33–138). On average, the period between the baseline videos and the last coaching session was M = 1.3 months (SD = 0.6 months; range: 0.5–2.5). The two remaining teachers discontinued the study before the baseline videos and before the first coaching session, respectively (reason: unresponsive). Neither of these two teachers provided exit data.
Attrition analysis
To evaluate attrition pattern, all available baseline measures were compared between teachers who completed SEE-KS-EC (n = 19, 76%) and teachers who did not (n = 6, 24%). On average, SSIS-SEL standard scores were higher for teachers who completed SEE-KS-EC (M = 75.2, SD = 12.8), compared to teachers who did not (M = 59.2, SD = 12.9), t = 2.47, p < 0.05. Similarly, SRS-2 t scores were lower for teachers who completed SEE-KS-EC (M = 66.8, SD = 12.6), compared to teachers who did not (M = 83.8, SD = 10.8), t = 2.73, p < 0.05. No other significant group differences emerged.
Aim 3: to complete a preliminary evaluation of implementation fidelity and teacher self-efficacy
Observational ratings of coaching fidelity showed excellent adherence to the coaching protocol, M = 92.36% (SD = 7.96), with n = 18 (94%) coaching sessions achieving at least 75% fidelity (feasibility target met; ⩾90%). Subscale scores for structure, content, and approach were M = 90.52% (SD = 11.29), M = 93.13 (SD = 9.18), and M = 93.42 (SD = 8.19), respectively. Pre–post comparison of teachers’ implementation fidelity showed that scores were higher after (M = 1.33, SD = 0.25) than before participating in SEE-KS-EC (M = 1.21, SD = 0.22), t = 1.51, p = 0.15, g = 0.35. Exploratory analyses at the subscale level revealed that pre–post gains were more pronounced for strategies to support independence (g = 0.44) and investment (g = 0.34), compared to an inverse relationship for strategies to support child initiations (g = −0.17). Finally, pre–post comparisons revealed that teacher self-efficacy ratings (TSES-SF) were higher after (M = 7.64, SD = 0.88) than before SEE-KS-EC (M = 7.02, SD = 0.91), t = 3.83, p < 0.01, g = 0.86. At the level of subscales, pre–post gains were evident across all three TSES-SF subscales (i.e. student engagement, instructional strategies, classroom management), g = 0.62–0.85.
Aim 4: to complete a preliminary evaluation of pre–post gains in clinical child outcomes
Pre–post comparison revealed that observed AE scores were higher after (M = 2.05, SD = 0.51) than before participating in SEE-KS-EC (M = 1.84, SD = 0.53), t = 1.72, p = 0.15, g = 0.40. Similarly, pre–post comparisons revealed that student–teacher relationship scores (STRS-SF) were higher after (M = 3.95, SD = 0.61) than before participating in SEE-KS-EC (M = 3.55, SD = 0.67, t = 3.61, p < 0.01, g = 0.85). At the level of subscales, pre–post gains were evident for both the closeness (g = 0.76) and the conflict subscales (g = −0.48). Finally, pre–post comparisons revealed that children’s social-communication competencies (SSIS-SEL) were higher after (M = 81.53, SD = 16.59) than before participating in SEE-KS-EC (M = 76.18, SD = 12.56), t = 2.53, p < 0.05, g = 0.60. At the level of subscales, pre–post gains were most pronounced for children’s social awareness, relationship skills, and responsible decision-making (g = 0.61–0.73), and less pronounced for self-awareness and self-management skills (g = 0.02–0.18).
Aim 5: to complete a preliminary evaluation of associations between implementation and clinical child outcome measures
Spearman correlations between implementation factors (i.e. time engaged with online modules or coaching sessions, coaching fidelity), teacher outcomes (i.e. pre–post gains in teacher fidelity or self-efficacy), and child outcomes (i.e. pre–post gains in AE, student–teacher relationship, social-communication competencies) are presented in Table 3. Results show significant correlations between several implementation factors. For example, observed ratings of coaching fidelity were significantly associated with the duration of teachers’ engagement with online modules (rho = 0.67; p < 0.01) or coaching sessions (rho = 0.62; p < 0.01). Similarly, results show significant correlations of gain scores across several child outcomes. For example, pre–post gains in AE were significantly correlated with gains in student–teacher relationship (rho = 0.54; p < 0.05) and children’s social-communication competencies (rho = 0.66; p < 0.01). No significant associations were found between implementation factors and gains in teacher outcomes or gains in child outcomes.
Spearman correlations of implementation and clinical outcomes (SEE-KS-EC completers only; N = 19).
STRS-SF = Student-Teacher Relationship Scale—Short Form; TSES-SF = Teachers’ Sense of Efficacy Scale-Short Form; SSIS-SEL = Social Skills Improvement System-Social Emotional Learning Edition.
in minutes.
< 0.05; ** < 0.01; *** < 0.001.
Discussion
SEE-KS-EC is a low-resource-intensive, transdiagnostic intervention that aims to increase AE of children with social-communication challenges. Overall, the current research revealed high levels of feasibility, with 9 out of 10 benchmarks met (Table 1). The first aim of this research was to evaluate the feasibility of our approach for identifying and recruiting target children with social-communication challenges. That is, we started by recruiting general education preschool teachers, who then used a structured selection process to identify target children with social-communication concerns who were invited to participate in this study. Results are encouraging. First, of the 31 consented teachers, only 4 (13%) were unable to successfully refer a child for participation in this research. Second, while only 7 (28%) of the 25 enrolled children had a prior autism diagnosis or educational disability classification, the sample showed high levels of social-communication impairments and low levels of social-communication competencies (i.e. the sample average SRS-2 score was two SDs above the population average, and the sample average SSIS-SEL score was two SDs below the population average).
Barriers to implementing professional development interventions in ECE settings include limited resources, dispersed accountability, and a workforce with limited professional training (National Survey of Early Care & Education Project Team, 2014; Wackerle-Hollman et al., 2021). Despite these barriers, the current research demonstrates the feasibility of a professional development framework that integrates self-guided online modules and synchronous online coaching. Nineteen of the 25 participating teachers (76%) completed the SEE-KS-EC program. These completion rates are comparable with public school-based interventions targeting preschool and elementary students with autism (Boyd et al., 2018; Morgan et al., 2018; Stahmer et al., 2023). Furthermore, the 19 teachers who completed SEE-KS-EC demonstrated high levels of engagement with both asynchronous online modules (M = 216.0 min) and synchronous coaching sessions (M = 99.6 min). Particularly, teacher’s high engagement with the online modules is noteworthy, given that teacher schedules in ECE settings often include little protected time for professional development. Furthermore, the current project was implemented during the 2021–2022 school year, a time characterized by high ECE staff turnover and burnout due to the ongoing recovery from the COVID-19 pandemic (Crawford et al., 2021).
SEE-KS-EC was designed as a 10-week program with alternating weeks dedicated to data collection (i.e. classroom videos), asynchronous online modules, and synchronous coaching sessions. Results show that the average teacher took significantly longer (M = 3.2 months) than the intended 10 weeks to complete the program, with one teacher taking as long as 7.2 months. Reasons for delays are manifold, including child absences on data collection days, school year breaks, delays in the completion of online modules, and difficulties scheduling coaching sessions. Particularly, for teachers who completed this research during spring, the overly ambitious timeline caused the need to condense coaching sessions, impacting both program intensity and coaching fidelity. Going forward, SEE-KS-EC should (a) be implemented as a year-long professional development program beginning in fall (possibly with an expanded focus on other classroom routines such as center time or classroom transitions), (b) be incorporated into existing professional development structures (e.g. embedded in planning days), and (c) leverage technology that allows teachers to collect their own classroom videos.
Given the current study’s focus on feasibility, the sample size was chosen based on the availability of resources and study constraints, and we prioritized maximizing sampling variability (e.g. providers, educators, parents) over the inclusion of a control group (Leon et al., 2011). Despite the relatively small sample size, the current research revealed a robust pattern of treatment-related gains, including improvements in observed implementation fidelity (not sig.; g = 0.35), teacher self-efficacy (sig.; g = 0.86), student–teacher relationship (sig.; g = 0.85), AE (not sig.; g = 0.40), and SEL skills (sig.; g = 0.60). Moreover, while results failed to demonstrate significant associations between intervention receipt and child outcomes, different indicators of intervention receipt (coaching fidelity, online module completion, coaching session completion) and different indicators of child outcome gains (AE, student teacher relationship, social-communication capacities) were correlated with each other, supporting the overall validity of our measurement approach.
Limitations
This study includes two important limitations. First, the feasibility benchmarks selected for this research were primarily grounded in the investigators’ past experiences supporting providers in clinical and educational settings. Given the novelty of the SEE-KS-EC approach (e.g. focus on children with social-communication challenges across a range of ECE settings), a strong empirical basis to guide benchmark selection was not available. Second, the small sample size of this study did not allow us to evaluate potential moderators of implementation outcomes, including teacher (e.g. teachers’ educational attainment) or organizational characteristics (e.g. whether teachers were released from teaching duties to complete the online modules during working hours). Similarly, the lack of significant correlations between implementation and child outcomes may be due to insufficient statistical power and should be interpreted cautiously.
Future directions
The current research was completed in preparation of a future, larger effectiveness-implementation hybrid trial (type 1; Landes et al., 2019) with the primary aim of investigating the effectiveness of SEE-KS-EC on child outcomes. Ideally, such a study would randomize programs to SEE-KS-EC or a waitlist condition and use a stepped wedge design (Stahmer et al., 2023). A secondary aim of this hybrid trial would be to explore implementation and sustainability issues (i.e. barriers and facilitators), focusing equally on the service environment (e.g. childcare and school systems), organizational characteristics (e.g. implementation leadership/climate and support strategies), and teacher characteristics (e.g. provider attitudes and engagement in training; Aarons et al., 2011; Brookman-Frazee et al., 2020; Brookman-Frazee & Stahmer, 2018). Furthermore, we identified three specific questions for future research: First, researchers should investigate implementation strategies tailored for specific service delivery systems. For example, compared to other childcare systems, public Pre-K programs administered by local school systems provide a unique set of facilitators (e.g. relatively high levels of teacher qualifications, clear accountability structures, established systems for professional development) and barriers (e.g. centrally planned classroom activities and routines, children with relatively high support needs). Unique characteristics of ECE service systems should be leveraged for designing plausible SEE-KS-EC implementation strategies. Second, the proposed low-resource-intensive, transdiagnostic intervention should be incorporated within a comprehensive multitiered system of support (MTSS) framework, providing a hierarchy of tiered interventions that vary by intensity, frequency, and degree of individualization (Carta & Young, 2019). MTSS frameworks have been developed to ensure a rapid, flexible, and effective response in addressing children’s learning needs in the areas of early mathematics, language, literacy, and challenging behaviors (Division for Early Childhood (DEC), 2021; Shepley & Grisham-Brown, 2019). However, an MTSS framework targeting children with early social-communication challenges is currently not available. Finally, the current research focused on the impact of SEE-KS-EC on target children with social-communication challenges. While coaching sessions focused on creating opportunities for increasing AE of target children, modifications to circle-time routines and activities often increased AE opportunities for all children. Future research should also investigate the classroom-wide impact of SEE-KS-EC on all children.
Supplemental Material
sj-docx-1-aut-10.1177_13623613231179289 – Supplemental material for Feasibility and acceptability of a low-resource-intensive, transdiagnostic intervention for children with social-communication challenges in early childhood education settings
Supplemental material, sj-docx-1-aut-10.1177_13623613231179289 for Feasibility and acceptability of a low-resource-intensive, transdiagnostic intervention for children with social-communication challenges in early childhood education settings by Michael Siller, Lindee Morgan, Sally Fuhrmeister, Quentin Wedderburn, Brooke Schirmer, Emma Chatson and Scott Gillespie in Autism
Footnotes
Acknowledgements
We wish to thank Emily Rubin for her guidance in adapting the experimental intervention for implementation in early childhood education settings.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Preparation of this article was supported by grant R21HD100820 from the Eunice Kennedy Shriver National Institute of Child Health and Human Development, The Marcus Foundation, and Children’s Research Trust.
Supplemental material
Supplemental material for this article is available online.
Notes
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
