Abstract
All children benefit from intentional interactions and instruction to become socially and emotionally competent. Over the past 30 years, evidence-based intervention tactics and strategies have been integrated to establish comprehensive, multitiered, or hierarchical systems of support frameworks to guide social–emotional interventions for young children. In this study, the authors reviewed systematically the efficacy of classroom-wide social–emotional interventions for improving the social, emotional, and behavioral outcomes of preschool children and used meta-analytic techniques to identify critical study characteristics associated with obtained effect sizes. Four electronic databases (i.e., Academic Search Premier, Educational Resource Information Center, PsycINFO, and Education Full Text) were systematically searched in December 2015 and updated in January 2018. “Snowball methods” were used to locate additional relevant studies. Effect size estimates were pooled using random-effects meta-analyses for three child outcomes, and moderator analyses were conducted. Thirty-nine studies involving 10,646 child participants met the inclusion criteria and were included in this systematic review, with 33 studies included in the meta-analyses. Random-effects meta-analyses showed (a) improvements in social competence and emotional competence, and (b) decreases in challenging behavior. For social competence and challenging behavior, moderator analyses suggested interventions with a family component had statistically significant and larger effect sizes than those without a family component. Studies in which classroom teachers served as the intervention agent produced statistically significant but smaller effect sizes than when researchers or others implemented the intervention for challenging behavior. This systematic review and meta-analysis support using comprehensive social–emotional interventions for all children in a preschool classroom to improve their social–emotional competence and reduce challenging behavior.
Keywords
Social–emotional competence in the early years lays the foundation for children’s subsequent well-being and success (Shonkoff & Phillips, 2000). Social–emotional competence refers to young children’s ability to form close and secure adult and peer relationships; experience, regulate, and express emotions in socially and culturally appropriate ways; explore the environment in appropriate ways to learn (Yates et al., 2008). Research has established a positive relation between young children’s social–emotional competence and their readiness for school and success in peer relationships (La Paro & Pianta, 2000; McClelland et al., 2000).
Considerable effort has been devoted to identify evidence-based promotion, prevention, and intervention practices to support children’s social–emotional competence and to address their challenging behavior. Over the last several decades, one trend in social–emotional interventions for young children has been to combine individual evidence-based intervention tactics and strategies to establish comprehensive intervention models or hierarchical intervention frameworks now commonly referred to as multitiered systems of support (Brown & Conroy, 2011; Hemmeter et al., 2013). All young children, including those who have delays or disabilities or are at risk, benefit from social–emotional supports and instruction (Zins & Elias, 2006), including classroom-wide supports and instruction. In the context of multitiered systems of support, implementing universal social–emotional practices in early childhood classrooms represents a promising approach to improving all young children’s social–emotional competence and promoting their positive behavior (Durlak et al., 2011; Hemmeter et al., 2016). For children with or at risk for delays or disabilities, secondary and tertiary interventions provide additional targeted or individualized supports and instruction (Carta & Young, 2019).
A number of social–emotional curricula or interventions have been developed to systematically promote young children’s social–emotional competence. To date, there have been two published, peer-reviewed syntheses of comprehensive social–emotional interventions for young children from birth through 5 years of age (i.e., Barton et al., 2014; Joseph & Strain, 2003). Both classroom-based and family-focused programs/curricula designed to improve the social, emotional, and behavioral outcomes of young children were analyzed in these two syntheses, with an emphasis on the examination of study quality indicators for each program/curriculum. Although research studies evaluating the effects of each program/curriculum were analyzed for study quality, program/curriculum was the unit of analysis in these two syntheses and not meta-analyses of study findings.
Several quantitative reviews have been conducted on social skill interventions exclusively targeting school-age children with a specific diagnosis such as autism spectrum disorders and emotional and behavioral disorders (e.g., Bellini et al., 2007; Quinn et al., 1999). Vaughn and colleagues (2003) synthesized group design studies published between 1975 and 1999 on social skill interventions for improving social functioning of 3- to 5-year-old children with disabilities. The researchers identified and analyzed 23 studies and calculated effect sizes using a standardized mean difference effect size (i.e., Cohen’s d). By descriptively comparing the magnitude and range of effect sizes, the researchers suggested that positive social outcomes for children with disabilities were associated with several components of the interventions such as modeling, play-related activities, rehearsal/practice, and prompting. This synthesis focused on the effects of social skill interventions on children with disabilities, without providing evidence about possible benefits for their typically developing peers.
Another meta- analysis (January et al., 2011) examined the effectiveness of classroom-wide interventions for promoting social skills of children (including children with varying levels of social difficulties) enrolled in preschool through 12th grade. Twenty-eight peer-reviewed journal articles published between 1981 and 2007 were included in this meta-analysis, with each study being a control- or comparison-group design. This meta-analysis indicated that the overall effect of classroom-wide interventions on children’s social skills was positive but small (d = 0.15, 95% confidence interval [CI] = [0.12, 0.19]), whereas moderator analyses suggested the effectiveness varied as a function of intervention and child characteristics. Compared with older children, this meta-analysis suggested that interventions might be more efficacious for younger children, particularly those in preschool and kindergarten (January et al., 2011).
Durlak and colleagues (2011) conducted a meta-analysis of school-based universal social–emotional programs on multiple outcomes of kindergarten through high school students. They used Hedges’ g to estimate effect sizes for 213 studies published or unpublished by 2007. Durlak and colleagues found the largest effect size for social–emotional skills (g = 0.57, 95% CI = [0.48, 0.67]), followed by academic performance (g = 0.27, 95% CI = [0.15, 0.39]), positive social behavior (g = 0.24, 95% CI = [0.16, 0.32]), emotional distress (g = 0.24, 95% CI = [0.14, 0.35]), attitudes toward self and other (g = 0.23, 95% CI = [0.16, 0.30]), and conduct problems (g = 0.22, 95% CI = [0.16, 0.29]).
January et al. (2011) and Durlak et al. (2011) included school-age children in their meta-analyses; analyses of effects of classroom-wide interventions on the social, emotional, and behavioral outcomes of young children are not yet established. The primary purpose of the present systematic review and meta-analysis was to review systematically and then meta-analyze evidence on the effects of classroom-wide social–emotional interventions for improving the social, emotional, and behavioral outcomes of preschool children when compared with either a no-treatment control group or business-as-usual group. The secondary objective was to use moderator analyses to identify specific study characteristics that were associated with larger effect sizes across the social, emotional, and behavioral outcomes. Specifically, three research questions guided this systematic review and meta-analysis:
Method
An a priori review protocol was developed to plan and document methodological decisions made for conducting this synthesis (see online Supplemental Text 1). The protocol was developed to align with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA; Moher et al., 2009). A PRISMA checklist for the current systematic review and meta-analysis is shown in online Supplemental Table 1.
Search Strategy to Identify Studies
A systematic search of Academic Search Premier, Educational Resource Information Center, PsycINFO, and Education Full Text electronic databases was conducted in December 2015 and updated in January 2018. No filters (e.g., language, publication type, age/population groups) were used during the database search. An example of the electronic search strategy is shown in Supplemental Text 2. The search results were imported and combined into the reference management software EndNote X7 and deduplicated. Snowball methods (e.g., Greenhalgh & Peacock, 2005) were used to locate additional studies not identified in the electronic database search by examining citations of included studies, studies citing included studies, and reference lists from the key literature reviews.
Selection of Studies
After electronic records were deduplicated, all records in the EndNote X7 software were screened for inclusion through a two-stage procedure. First, the title and abstract of each record were screened to eliminate records that clearly did not meet inclusion criteria. As a result, a list of potentially eligible studies was identified and subjected to the next step. Full texts of the articles deemed as possibilities were then examined against the inclusion criteria. Two of the study authors completed each stage of screening independently, resolving disagreements through mediation to reach consensus.
For a study to be included in the present review, it had to meet seven inclusion criteria. First, the study had to be an empirically based research study published in a peer-reviewed journal. Gray literature was not included in the present review, given systematic searching for gray literature can be challenging, particularly when data analytic strategies and reported results differ for the same study in a peer-reviewed journal and in the prepublication form (i.e., gray literature; Schmucker et al., 2017). Second, child participants in the study were preschool-age children with or without disabilities. The average child age at the beginning of the study had to be between 36 and 60 months. Third, the study was a controlled group experimental design, specifically, a study design comparing the effects of the intervention between one group of participants who received the intervention to another group who did not experience the intervention, regardless of randomization. Fourth, the study examined the effect of a classroom-wide social–emotional intervention. Building and expanding on definitions in previous reviews (i.e., Barton et al., 2014; Joseph & Strain, 2003), classroom-wide social–emotional intervention in the present review was defined as a curriculum, multicomponent intervention, hierarchical intervention, or intervention package/program intended for use with a whole class or groups of children in a class and designed to provide universal supports for improving the social, emotional, and behavioral outcomes of preschool children. All children in a preschool classroom have the potential to receive the intervention, regardless of their level of social–emotional competence or challenging behavior. Secondary or tertiary interventions designed for children with or at risk of social, emotional, or behavioral challenges and comprehensive preschool curricula/interventions designed to guide classroom instruction in multiple learning domains were not eligible for inclusion in this review. Fifth, the study reported the child outcome data. Sixth, the study had to be conducted in typical preschool settings (e.g., preschool, Head Start, child care center, nursery school, preprimary class). Finally, the study was written in English.
Coding Variables and Procedures
To summarize the existing empirical literature on classroom-wide social–emotional interventions in preschool classrooms, a coding form and manual, which are available upon request from the first author, to extract data from identified studies, were developed. Data were extracted on variables related to participant-level, intervention-level, and study-level characteristics. Data extraction forms were developed and data were coded using Microsoft Excel®. All coding variables and effect size estimates across included studies were coded independently by two coders (i.e., the first, fourth, and fifth authors of the present study). Disagreements between coders were resolved through mediation that involved the second and third authors to reach consensus, when necessary.
We coded 16 variables related to (a) preschool classrooms/programs, (b) child participants, and (c) personnel participants. Preschool classroom/program variables consisted of the number of classrooms, type of classroom, type of program, and its operation hours. Variables related to child and personnel participants included, when available, sample size, gender, age, disability, dual language learners, family income or socioeconomic status, level of education, teaching experience, and professional role.
We coded five variables related to the classroom-wide intervention: (a) name of the intervention; (b) delivery of the intervention, including group format, setting, and intervention agent; (c) training of the intervention agent; (d) family component (i.e., parents were trained to implement the intervention or part of the intervention in home settings); and (e) dose of intervention that was quantified by recording the length of treatment per week, total number of weeks of treatment, total length of treatment (in hours), total number of sessions of treatment, number of sessions per week, and duration of each session.
We coded eight research designs and result variables: (a) study design; (b) number of groups/conditions; (c) fidelity of intervention implementation; (d) geographic location (i.e., country) where the study was conducted; (e) categories of child outcomes (i.e., social or emotional competence, challenging behavior) assessed as dependent variables; (f) data used for effect size calculation for each category of child outcomes; (g) measurement instruments used to assess each category of child outcomes; and (h) study-level risk of bias, which was assessed using Cochrane’s risk of bias tool (Higgins & Altman, 2008) with considerations for nonrandomized studies (Reeves et al., 2013). Nine domains of risk of bias were examined: sequence generation, allocation concealment, blinding of participants and personnel, blinding of outcome assessment, incomplete outcome data, selective outcome reporting, protection against contamination, baseline measurements, and procedural fidelity. Each domain was coded as low (low risk of bias), high (high risk of bias), or unclear (unclear risk of bias).
Meta-Analytic Procedures
Meta-analysis is a statistical technique by which individual study treatment effects on specified outcomes are combined into a pooled treatment effect for multiple studies (Lipsey & Wilson, 2001). In a meta-analysis, the unit of analysis is the effect size, determined based on the primary study outcomes rather than the individual participant. We used the Comprehensive Meta-Analysis (CMA) software (Borenstein et al., 2014) to calculate the effect size for each study outcome. We conducted meta-analyses for three child outcomes separately: social competence, emotional competence, and challenging behavior.
Calculation of effects and analytic strategies
Given that multiple measures with different scales were used to assess child outcomes across studies, effect sizes were calculated using the standardized mean difference between posttest means and pooled standard deviations of the treatment and control groups, with a correction for small sample bias (Hedges’ g; Hedges & Olkin, 1985). The majority of studies reported unadjusted posttest means, posttest standard deviations, or inferential test statistics for the posttreatment assessment, which were the data used for the calculation of effect size estimates. Two studies did not report unadjusted (actual) means; Morris et al. (2013) reported adjusted posttest means and Upshur et al. (2013) reported pretest/posttest change scores for each experimental condition. For both of these studies, we used the data reported in the article for calculation of the effect size estimates, including in the meta-analysis. If posttest means, standard deviations, and sample sizes were not provided in the study report, we estimated effect sizes using p-values, t-values, or F-values, following estimation procedures recommended by Lipsey and Wilson (2001). For effect sizes, we calculated one-point estimate per child outcome for each included study (here a “study” refers to a pairwise comparison in the meta-analyses). If a study reported multiple measures for one outcome, we selected the measure that had superior psychometric properties (either from the study report or from the user’s manual for the instrument). If both a teacher report and caregiver report of the same measurement instrument were used, we used the teacher data for the effect size calculation.
Although the social–emotional interventions included in our review were delivered to entire classrooms of children, most studies reported outcome data at the child level. Therefore, we used the sample size of children when calculating effect size estimates for all but four studies. Three studies (Baker-Henningham et al., 2009; Lynch et al., 2004 [Michigan 1996–1997 sample]; and Ostrov et al., 2009) reported study results at the classroom level and one study reported results at the program level (Feil et al., 2009). For these four studies, we transformed the sample sizes from classrooms/programs to children using data provided in the articles, so that the sample sizes across all studies included in the meta-analysis were similar (Deeks et al., 2008).
With respect to the meta-analysis for each category of child outcomes, we used a random-effects model to estimate the weighted mean effect size. Heterogeneity was estimated using both the Q-statistic and I2. The use of the Q-statistic for the homogeneity test has been criticized, and a nonsignificant result is recommended to not be taken as evidence of no heterogeneity (Deeks et al., 2008). Given the limitations of the Q-statistic, I2 values were emphasized in interpreting the meta-analytic results. We assessed funnel plot asymmetry using the trim and fill method (Duval & Tweedie, 2000), a nonparametric technique, to detect potential publication bias.
Moderator analysis
We conducted moderator analyses to examine whether variations among study effects were associated with study or research characteristics (i.e., family component, intervention agent, child outcome measure, research design, treatment fidelity, and dose of treatment) that we hypothesized might have a relation to outcomes based on previous meta-analyses (Durlak et al., 2011; January et al., 2011). The analyses were conducted using a method analogous to the one-way analysis of variance for categorical variables and meta-regression for continuous variables. Although there is no commonly accepted guidance about the number of studies required for conducting moderator or subgroup analysis, some researchers have suggested a minimum of four studies for each subgroup (Fu et al., 2011). Therefore, we did not conduct subgroup analyses for categorical variables with fewer than four studies in any subgroup. Given the power issue with small sample sizes in meta-regression, the ratio of covariates to studies is recommended to be about 1:10 (Baker et al., 2009). Furthermore, researchers have suggested that meta-regression should not be conducted when there are fewer than 10 studies in a meta-analysis (Deeks et al., 2008). Considering the total number of studies in our meta-analysis for each child outcome, we examined the influence of one covariate at a time by conducting a univariate meta-regression analysis analogous to a simple regression for continuous variables. All analyses were conducted using CMA (version 3.0) software.
Improvement index
In our meta-analyses, effect sizes provided information about the magnitude and direction of the posttest difference between the treatment and control groups. To interpret effect sizes in a way readers can judge the practical importance of intervention effect, we translated effect sizes into “improvement index” values. This index represents the difference between the percentile rank corresponding to the treatment group mean in the treatment group distribution and the percentile rank corresponding to the control group mean in the control group distribution (What Works Clearinghouse, 2017).
Results
A total of 31,780 records were located through the systematic search of the four electronic databases. An additional 5,625 were identified using the “snowballing” methods. After deduplication, 30,361 records remained. Following the initial screening of titles and abstracts, 379 articles remained. We then examined the full-text articles to assess eligibility for inclusion in this review. Thirty-eight journal articles describing 39 studies met the established inclusion criteria and were included in this systematic review. Given some studies did not provide sufficient information relevant for effect size estimation, 33 studies with 37 comparisons (in which effect size could be estimated for at least one category of child outcome) were included in the meta-analysis. A PRISMA flow diagram (Moher et al., 2009) documenting the search and selection process is shown in online Supplemental Figure 1.
Descriptive Characteristics of Reviewed Studies
Participant characteristics
Online Supplemental Table 2 shows the key characteristics of included studies. A total of 10,646 child participants were included across the 39 studies, with a mean of 273 children (SD = 238.60, range = 43–1,049). Children in 16 studies (41%) were partly or entirely recruited from Head Start programs. Twenty-nine studies (74%) reported the mean age of children, with a mean of 4.30 years (SD = 0.38, range of means = 3.33–4.98 years). Only three studies (Domitrovich et al., 2007; Feil et al., 2009; Hemmeter et al., 2016) reported recruiting children with special needs, with an additional four studies (Hamre et al., 2012; Morris et al., 2013; Pickens, 2009; Ştefan & Miclea, 2013) reporting the percentage of children who were dual language learners.
Intervention agents and characteristics
Classroom teachers served as the intervention agent in 29 of 39 (74%) of the included studies; other professionals who delivered the intervention to preschool children included researchers (n = 1), trained curriculum facilitators (n = 2), and a team of researchers/facilitators and classroom teachers (n = 3). The level of education or teaching experience of intervention agents was specified in only 14 studies (36%). Twenty-four studies (62%) reported the type of professionals who provided training to the intervention agents, with researchers being the primary trainers of the authentic instruction agents. As shown in Supplemental Table 2, parents were trained to implement the intervention in home settings in 13 studies (33%), which complemented the intervention delivered in the preschool settings. More than half of the studies (n = 24; 62%) did not specify the classroom activities in which the intervention was delivered to children. Whole group activities were the intervention settings for 10 studies (26%), and embedding the instruction into daily activities/routines was reported in 6 studies (15%). As shown in Supplemental Table 2, a variety of social–emotional curricula or programs were used as the intervention.
Variability existed with respect to the dose of intervention (see online Supplemental Table 3). For example, across included studies, the total length of intervention ranged from 5 weeks to about 40 weeks. The number of teaching episodes or intervention sessions delivered to children per week ranged from one session per week to a session everyday (i.e., five times per week). The duration of each teaching episode or session ranged from 5- to 180 min per session. The cumulative dose in terms of the total number of hours of intervention was able to be calculated for 23 studies (59%), with a mean cumulative intervention dose of 20.7 hr (SD = 20.46, range = 5–84 hr).
Study characteristics
Among the 39 included studies, more than half (n = 21; 54%) used experimental research designs, and the remaining 18 (46%) were quasi-experimental studies. In most of the included studies, participants were divided into two conditions/groups (i.e., treatment condition and control condition); five studies involved more than one treatment or control condition. The United States was the most common location for the research studies included in this review (n = 28; 72%). Of the remaining 11 studies, 2 were conducted in Romania, 2 in Spain, 2 in Turkey, 2 in the United Kingdom, and 1 in Norway, Jamaica, and Australia, respectively.
Risk of bias graphs were created to illustrate the percentage of studies with each of the judgments for the nine potential biases assessed (see online Supplemental Figures 2 and 3). Most studies did not adequately describe the method by which participants were assigned (i.e., sequence generation). No study included in the review was free of blinding of participants and personnel, which is common in social science research and the nature of classroom-wide interventions, in which it is very difficult to blind children who are receiving the intervention and individuals who are implementing the intervention. Nearly half of the included studies were rated as unclear risk for incomplete outcome data and protection against contamination. Greater than 50% of studies had high risk of detection bias due to outcome assessors who were not blind to children’s condition assignment. Finally, a majority of the studies also had a high risk of bias for procedural fidelity, meaning this was not reported as being measured in the included studies.
Online Supplemental Tables 4–6 show the effect size estimates for each outcome category and corresponding measurement instrument. For the category of child outcomes being reported with sufficient information for effect size calculation, 6 studies (15%) reported all three child outcome categories (i.e., social, emotional, and challenging behavior), 22 studies (56%) reported two categories of child outcomes, and 5 studies (13%) reported one child outcome category. Variability existed with respect to the measurement instruments used for the child outcome categories. Measurement of child outcomes relied predominantly on informant ratings, especially for the social (n = 23) and challenging behavior (n = 21) outcome categories.
Effects of Classroom-Wide Social–Emotional Interventions
Social competence
Thirty articles with 34 pairwise comparisons involving 8,394 children examined relationships between classroom-wide social–emotional interventions and children’s social competence. As shown in Figure 1, classroom-wide social–emotional interventions had statistically significant and noteworthy effects on the social competence of preschool children (g = 0.42, 95% CI = [0.28, 0.56]; z = 5.77, p < .001; k = 34). Nevertheless, we found large heterogeneity between studies, Q (33) = 291.37, p < .001, I2 = 88.67%. As shown in online Supplemental Figure 4, the funnel plot for the meta-analysis of the social competence outcome showed asymmetry in the left portion of the funnel, suggesting possible publication bias. An estimate of an adjusted weighted mean effect size using the trim and fill with the 11 “missing” studies would have been smaller.
Forest plot of the effects of social–emotional interventions on preschool children’s social competence.
Emotional competence
Twelve articles with 14 pairwise comparisons involving 1,799 children examined relationships between classroom-wide social–emotional interventions and children’s emotional competence. As shown in Figure 2, classroom-wide social–emotional interventions had a statistically significant and noteworthy effect on preschool children’s emotional competence (g = 0.33, 95% CI = [0.10, 0.56]; z = 2.85, p = .004; k = 14). Nevertheless, we found large heterogeneity between studies, Q (13) = 60.38, p < .001, I2 = 78.47%. As shown in online Supplemental Figure 5, the funnel plot for the meta-analysis of the emotional competence outcome showed slight asymmetry in the left portion of the funnel, suggesting two studies were “missing.” An estimate of an adjusted weighted mean effect size using the trim and fill with the two “missing” studies would have been slightly smaller.
Forest plot of the effects of social–emotional interventions on preschool children’s emotional competence.
Challenging behavior
We located 24 articles with 28 pairwise comparisons that involved 6,708 children and examined relationships between classroom-wide social–emotional interventions and children’s challenging behavior outcome. As shown in Figure 3, classroom-wide social–emotional interventions had a statistically significant and noteworthy effect on the reduction of challenging behavior of preschool children (g = −0.31, 95% CI [−0.43, −0.19]; z = −5.03, p < .001; k = 28). As indicated by the Q-statistic and I2 value, we found considerable heterogeneity between studies, Q (27) = 131.40, p < .001, I2 = 79.45%. As shown in online Supplemental Figure 6, the trim and fill method identified four “missing” studies for the challenging behavior outcome, showing slight asymmetry in the right portion of the funnel. An estimate of an adjusted-weighted mean effect size using the trim and fill with these studies would have been slightly smaller because a higher negative value for this outcome indicates a more robust effect (i.e., fewer instances of or less severe challenging behavior).
Forest plot of the effects of social–emotional interventions on preschool children’s challenging behavior.
Interpretation of obtained effect sizes in context
Online Supplemental Table 7 shows the translation of the weighted mean effect sizes into an improvement index for the three child outcomes. For social competence, an effect size g = 0.42 (95% CI = [0.28, 0.56]) translates into an improvement index of 16.28 percentile points, 95% CI = [11.03%, 21.23%]. This finding means on average, a child in the control group who scores at the 50th percentile would be expected to have scored at the 66.28th percentile of the control group distribution if he or she had received the intervention. With respect to emotional competence, an effect size of 0.33 (95% CI = [0.10, 0.56]) translates into an improvement index of 12.93 percentile points, 95% CI = [3.98%, 21.23%], suggesting the intervention is likely to result in an average child in the treatment group being ranked 12.93% percentile points higher compared with the average child in the control group who would still remain at the 50th percentile. For challenging behavior, an effect size of −0.31 (95% CI = [−0.43, −0.19]) translates into an improvement index of −12.17 percentile points, 95% CI = [−16.64%, −7.53%], suggesting that the intervention would have led to a 12.17% decrease in percentile rank (i.e., reduction in challenging behavior) for an average child in the treatment group and that 62.17% of the children in the treatment group scored below the control group mean.
Analysis of Moderator Effects
Table 1 shows the results from moderator analyses across the three child outcomes. As shown in Table 1, the majority of the moderator of intervention effect analyses that we conducted did not yield statistically significant differences. We limit the presentation of findings in this section to variables in which one of the analyses indicated a statistically significant difference, given the majority of findings were not statistically significant. The findings should be interpreted with caution given the number of univariate moderator analyses conducted and the possibility of inflation of Type I error rates.
Subgroup Analyses for Each Category of Child Outcomes.
Note. k = number of studies/comparisons; CI = confidence interval; Q = test of heterogeneity between subgroups.
Subgroup analyses were not conducted due to insufficient number of studies/comparisons included.
The bold faced values are p values.
p < .05. **p < .01.
We found a statistically significant difference for two intervention characteristics, inclusion of a family component and intervention agent. First, interventions with a family component had a statistically significant larger effect size than those without a family component for challenging behavior, Q (1) = 7.92, p = .005; and social competence, Q (1) = 7.03, p = .008. Second, the moderator analyses indicated that effect sizes were larger for studies in which intervention was delivered by nonclassroom teachers (e.g., researchers, curriculum facilitators, team) than those delivered by classroom teachers for challenging behavior, Q (1) = 6.40, p = .011; but the category of intervention agent did not moderate social competence treatment effects, Q (1) = 1.10, p = .294. For emotional competence, there were insufficient data to conduct the moderator analyses for either the family component or delivery agent.
We also found statistically significant differences in our moderator analyses for two research characteristics. First, quasi-experimental studies yielded statistically significant larger effect sizes on social competence than studies that had random assignment, Q (1) = 7.54, p = .006; but not for emotional competence, Q (1) = 0.30, p = .860, or challenging behavior, Q (1) = 0.34, p = .560. Second, we found studies in which treatment fidelity was not measured produced statistically significant larger effect sizes than studies for which fidelity was measured for both the social outcome, Q (1) = 5.79, p = .016, and the emotional outcome, Q (1) = 4.48, p = .034; but not for the challenging behavior outcome, Q (1) < 0.01, p = .989.
Given the included studies provided varying information about the amount of exposure to the intervention, we coded dose of intervention using six metrics (i.e., length of treatment per week, number of weeks of treatment, total length of treatment in hours, total number of sessions of treatment, number of sessions per week, and duration of each session). We used meta-regression techniques to examine whether there were systematic differences in outcome associated with dose, which are shown in online Supplemental Table 8. Our series of meta-regression analyses revealed that no statistically significant associations were found between effect sizes for the three child outcomes (i.e., social competence, emotional competence, and challenging behavior) and the six dosage variables that we defined.
Discussion
The purposes of this synthesis were to systematically review and meta-analyze studies examining the effects of classroom-wide social–emotional interventions on the social, emotional, and behavioral outcomes of preschool children and to identify select study or intervention characteristics that moderated obtained intervention effects. A total of 39 studies met the inclusion criteria for the systematic review, with 33 studies involving 37 unique pairwise comparisons included in the meta-analyses. The current systematic review and meta-analysis differ in emphasis from previous reviews and meta-analyses by focusing exclusively on social–emotional interventions implemented classroom-wide for preschool children with and without disabilities and evaluating their effects on children’s social competence, emotional competence, and challenging behavior.
Overall, the results of the meta-analyses showed moderate improvements in social competence, emotional competence, and challenging behavior. Emotional competence was less likely to be measured as a dependent variable than social competence in the included studies; the effect of the intervention on emotional competence was not examined in previous meta-analyses (January et al., 2011; Wilson & Lipsey, 2007). Given the construct of social–emotional competence subsumes both social and emotional skills, many instruments do not operationalize social competence and emotional competence as separate constructs (Ştefan et al., 2009). In addition, few instruments with well-established psychometric properties exist for measuring emotional competence of children. This multidimensional construct has been recognized as an area of need in measurement development in early childhood (Denham et al., 2016).
Inspection of the overall effect sizes obtained in the present meta-analyses along with those obtained on similar outcomes from existing meta-analyses of school-based social–emotional interventions (Durlak et al., 2011; Horowitz & Garber, 2006; January et al., 2011; Lösel & Beelmann, 2003; Wilson et al., 2001) indicates that effect sizes in the present review are similar to or, in some cases, larger than those reported in the comparison meta-analyses that involved school-age children. Our findings suggest classroom-wide social–emotional interventions might be particularly efficacious for preschool-age children for social competence and challenging behavior outcomes. Furthermore, the instruments identified as measuring the social, emotional, and behavioral outcomes in this systematic review and meta-analysis primarily relied on teacher reports of child behaviors rather than direct behavioral observational measures or direct performance-based assessments of child behaviors. The less frequent use of direct child assessments to measure preschool children’s social, emotional, and behavioral competence is likely a reflection of the lack of psychometrically sound direct assessment instruments and current recommended authentic assessment practices in early childhood and early childhood special education (Division for Early Childhood, 2014; Halle & Darling-Churchill, 2016).
Although we found significant heterogeneity with studies included in the present review across the three child outcomes, our moderator analyses provided limited support for specific intervention or study characteristics moderating child outcomes with two exceptions: family involvement and intervention agent. Family engagement in early childhood education has been linked to children’s academic and behavioral outcomes (Fantuzzo et al., 2004; Marcon, 1999). In the present synthesis, parent training was part of the social–emotional intervention in about one third of the included studies. Our moderator analyses suggested that interventions with a family component generally were more efficacious in increasing social competence and decreasing challenging behavior of preschool children than those without a family component. This finding supports previous studies showing family involvement as a major ingredient in the success of interventions for promoting social competence and reducing challenging behavior of young children (Dunlap et al., 2006).
Consistent with previous evidence (Marulis & Neuman, 2010; Mol et al., 2009), findings of this study indicate that the person who delivers the intervention (i.e., intervention agent) might be an important factor in the use of class-wide social–emotional interventions. Our analyses revealed that the type of intervention agent was statistically associated with the effects of classroom-wide interventions on the challenging behavior outcome; interventions provided by nonclassroom teachers had larger effect sizes on the challenging behavior outcome than those delivered by classroom teachers. This finding should be interpreted with caution, however, due to the differences in the number of studies and comparisons in each group. Nevertheless, this finding is important given the contemporary emphasis on interventions being delivered by authentic intervention agents (e.g., teachers, family members) in authentic contexts (Division for Early Childhood, 2014). As interventions from efficacy studies, where researchers or other nonauthentic agents often deliver the intervention with high fidelity, are evaluated at scale in effectiveness studies, it will be important to ensure that authentic intervention agents are implementing the intervention with fidelity when examining child outcomes (Snyder et al., 2018).
We found no statistically significant association between dose metrics and child outcomes in our meta-regression analyses. Because the dose of intervention is a function of three important subcomponents: intervention frequency, total intervention duration, and cumulative intervention intensity (Warren et al., 2007), future research should report sufficient information to permit the quantification of a dose of intervention variable to enable its examination in relation to intervention effects. In addition, dose formats should be specified through replicable descriptions of the independent variable categories, including treatment and control or comparison conditions and complemented by measures of fidelity.
Limitations
Limitations of studies included in systematic review and meta-analysis
As with most systematic reviews, the studies included in this systematic review had methodological limitations, which could have effects on our findings and interpretations. Most included studies lacked detailed descriptions of child participants (e.g., special education status, English language learner status), so it is difficult to identify the population of participants to which results may be generalized. Although many studies claimed random assignment of child participants to study conditions, strategies or methods used for random allocation were rarely reported, and therefore, experimental randomization may not have been realized. Descriptions of the characteristics of intervention agents and the methods used to train them were also absent from many studies. Future research should include providing sufficient information about the critical features of intervention agents (e.g., professional role, educational background/licensure, years of experience). Given few studies included in this review explicitly described including children with or at risk of disabilities, more research is needed to provide evidence about the effects of classroom-wide interventions for children with or at risk of disabilities.
In addition, procedural fidelity was only measured and reported in half of the included studies. Measuring procedural fidelity can strengthen the internal and external validity of the study by demonstrating that the intervention was implemented as intended and not implemented in comparison groups or conditions (Sanetti & Kratochwill, 2009). Finally, outcome assessments relied heavily on adult-reported instruments (teacher- or parent-report) instead of observation-based measures. Given interventions were more often delivered by classroom teachers and outcome assessments were also completed by these teachers, blinding of outcome assessment was impossible for many studies and could lead to a risk of outcome bias.
Limitations of current systematic review and meta-analysis
Prior to conducting this systematic review and meta-analysis, a protocol was created to minimize potential sources of bias. Nevertheless, several limitations should be considered when interpreting findings. First, we did not review studies in the gray literature in which studies with null or less robust effects would likely be found (e.g., file-drawer problem; Rosenthal, 1979). Our reported intervention effects might be overestimated given the exclusion of gray literature. Second, this synthesis included studies with both random and nonrandom group assignments. Compared with studies that randomize participants to groups, nonrandomized studies are more likely to be susceptible to bias; for example, selection biases in nonrandomized studies are more likely to occur, attrition is often worse, and outcomes are rarely assessed blind to the group allocation (Reeves et al., 2013). Third, many studies included in this synthesis used cluster designs (e.g., teachers assigned to treatment with children clustered in teachers’ classrooms) but did not take this clustering into account when analyzing child-level findings. Because the majority of studies did not account for clustering in their analyses, we were unable to examine variation due to clustering when estimating treatment effects. A sensitivity analysis was conducted to compare the results of the meta-analysis of social competence, including and not including the four studies previously described, for which the sample size had to be adjusted. The inclusion of the four studies did not have a significant impact on the results or interpretations of the meta-analysis (k = 34, g = 0.42, 95% CI = [0.28, 0.56] vs. k = 30, g = 0.45, 95% CI = [0.29, 0.62]). Similar findings were obtained for the meta-analysis of challenging behavior (k = 28, g = −0.31, 95% CI = [−0.43, −0.19] vs. k = 25, g = −0.29, 95% CI = [−0.42 to −0.15]). Fourth, given the limited number of studies included in the meta-analyses, we were not able to run multivariate meta-regression analysis with multiple covariates. It is possible that interactions between some covariates may be associated with intervention effectiveness. Finally, this review was limited to studies written in English, which might reduce the generalizability of the findings.
Conclusion
Overall, this systematic review and meta-analysis offer evidence that classroom-wide social–emotional interventions produce positive effects on the social, emotional, and behavioral outcomes of preschool children. The development of social–emotional competence is based on children’s observations and experiences, and all children benefit from intentionally planned and explicit social–emotional interactions and experiences (Hemmeter et al., 2016). The use of classroom-based interventions to promote children’s social–emotional competence gained momentum in the 1970s and has maintained the interest of practitioners and researchers since that time (Bierman & Motamedi, 2015). A modest evidence base exists to support the effects of explicit social, emotional, or behavior skill instruction delivered in the classroom setting. Findings from the present synthesis add to the growing evidence that all children, including those with or at risk of disabilities or delays, benefit from classroom-wide supports and instruction to promote their social–emotional and behavioral competence. Universal social–emotional and behavioral interventions implemented with all children are an important means for promoting social–emotional competence and preventing social–emotional difficulties or delays and challenging behavior of preschool children. Our findings suggest that these universal interventions are more efficacious when parents also are supported to implement universal strategies in the home settings. A variety of efforts are needed to provide policy supports, professional development, and technical assistance to support high-quality implementation of these universal interventions and their widespread adoption in early childhood.
Supplemental Material
Revised_Supplemental_Material – Supplemental material for Systematic Review and Meta-Analysis of Classroom-Wide Social–Emotional Interventions for Preschool Children
Supplemental material, Revised_Supplemental_Material for Systematic Review and Meta-Analysis of Classroom-Wide Social–Emotional Interventions for Preschool Children by Li Luo, Brian Reichow, Patricia Snyder, Jennifer Harrington and Joy Polignano in Topics in Early Childhood Special Education
Footnotes
Authors’ Note
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors received no financial support for the research, authorship, and/or publication of this article.
Supplemental Material
Supplemental material for this article is available on the Topics in Early Childhood Special Education website with the online version of this article.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
