Abstract
The study analyzed the factorial and concurrent validity of the Student–Teacher Relationship Scale (STRS) using an exploratory structural equation modeling (ESEM) approach. Participants were 368 Italian children aged 3 to 6 (M = 4.60, SD = 0.98). The three-factor ESEM solution fit the data better than the classical confirmatory factor analysis (CFA) model and the measurement invariance of the scale was confirmed across sex and age (3-4 vs. 5-6 years) groups. The concurrent validity of the STRS was investigated within the ESEM approach using children’s social behaviors as validity criteria. Findings supported the goodness of ESEM over CFA and attested to the validity of the STRS to understanding the teacher–child relationship quality in young children.
In the preschool and kindergarten educational context, the quality of teacher–child relationships plays an important role in explaining children’s social behavior within the classroom (Hughes, Bullock, & Coplan, 2014). Whereas secure or close teacher–child relationships have been associated with higher levels of socially competent behaviors, school liking, and peer acceptance, negative or insecure teacher–child relationships have been related to children’s behavioral difficulties (e.g., aggressive or withdrawn behaviors; Sabol & Pianta, 2012).
Given the importance of such relationships, developmental scientists have pointed out the need for developing valid and reliable tools aimed at assessing the quality of teacher–child relationships in the earliest stages of childhood development. The Student–Teacher Relationship Scale (STRS; Pianta, 2001), a 28-item scale, is considered one of the most popular teacher-report tools to evaluate three core dimensions of relationships between teachers and children (i.e., close, conflictive, and dependent), as attested by its large use across different countries/cultures (e.g., Zhang & Nurmi, 2012).
Despite the popularity of the STRS, previous studies based on exploratory factor analyses (EFA) and independent cluster models of confirmatory factor analyses (ICM-CFA) reported mixed findings about the factor structure of the STRS. For instance, Gregoriadis and Tsigilis (2008), using an EFA, found that a 26-item solution better captured the three-factor structure of the STRS. Milatz, Gluer, Harwardt-Heinecke, Kappler, and Ahnert (2014), using an ICM-CFA, dropped 16 items with poor psychometric properties to obtain a good factor solution. In addition, to the best of our knowledge, only a few studies evaluated the measurement invariance (MI) of the STRS across groups. For example, Milatz et al. (2014) supported a partial scalar invariance of the modified version of the STRS across kindergarteners, first graders, and second graders and a full scalar invariance of the scale across sex. Solheim, Berg-Nielsen, and Wichstrøm (2011) did not find full sex invariance on a modified 25-item version of the scale.
Based on these premises, the aim of the present study was to investigate the latent structure and concurrent validity of the STRS scores using an exploratory structural equation modeling (ESEM) framework, which is a flexible analytical technique that overcomes many limitations of EFA and ICM-CFA. Compared with highly restrictive basic ICM-CFA models that posit all cross-loadings on non-targeted factors to be zero, ESEM allows the free estimation of cross-loadings (such as in classical EFA) while still offering the possibility of conducting MI tests such as in ICM-CFA. Furthermore, ESEM estimates less inflated correlations among latent factors (a problem which typically occurs in ICM-CFA; Morin, Marsh, & Nagengast, 2015).
First, we expected a better fit of the ESEM model over the highly restrictive ICM-CFA. Second, we tested the MI of the STRS across age groups (preschoolers vs. kindergarteners) and sex. Finally, we conducted a concurrent validity analysis by examining the association of the STRS with children’s socially competent, aggressive, and withdrawn behaviors (see Hughes et al., 2014; Sabol & Pianta, 2012).
All these research goals were explored within the Italian preschool/kindergarten where children typically interact with same teachers and classmates for several years (Sette, Baumgartner, & Schneider, 2014).
Method
Participants
Participants were 368 Italian children (49.7% girls) between the ages of 3 and 6 (M = 4.60, SD = 0.98), enrolled in public preschools (n = 30 classrooms). Written parental consent was obtained for all participants.
Procedures
To ensure the independence of measures, two teachers were used as sources of information in each classroom: One for the assessment of the teacher–child relationship quality (Teacher A) and the other for the evaluation of children’s social behaviors (Teacher B).
Measures
The quality of the teacher–child relationship (Teacher A)
The STRS (Pianta, 2001) was used to assess teachers’ perceptions of their relationship with each child. The STRS contains 28 items rated on a 5-point Likert-type scale (1 = definitely does not apply to 5 = definitely applies) designed to assess closeness (11 items), conflict (12 items), and dependency (5 items).
Children’s social behaviors (Teacher B)
A revised version (Sette, Baumgartner, & MacKinnon, 2015) of the Social Competence and Behavior Evaluation Scale (SCBE-30; LaFreniere & Dumas, 1996) was used to evaluate children’s anxiety-withdrawal (α = .88), social competence (α = .89), and anger-aggression (α = .85).
Data Analysis
Given the ordered categorical nature of STRS items, we used the weighted least-squares mean and variance-adjusted (WLSMV) estimation method in Mplus 7.4 (Muthén & Muthén, 2012). In addition, we took into account the hierarchical structure of our data (students nested within classrooms) by using the Mplus correction of standard errors for nested data (type = complex). Because the χ2 statistic is sensitive to sample size, we also used the comparative fit index (CFI) and Tucker–Lewis index (TLI) >.90, and root mean square error of approximation (RMSEA) values < .08 with 90% confidence interval (CI) as indicative of acceptable model fit (Kline, 2011). To test differences among configural, metric, and scalar MI, we calculated Δχ2 tests. However, because the Δχ2 test is overly sensitive to minor model misspecifications, we also considered the ΔCFI test with a critical level of −.01 (Cheung & Rensvold, 2002).
Results
The ICM-CFA model showed a worse fit, χ2(347) = 740.43, p < .001, CFI = .84; TLI = .83, RMSEA = .06, 90% CI = [ .05, .06], compared with the ESEM with target rotation, χ2(297) = 444.15, p < .001; CFI = .94; TLI = .93, RMSEA = .04, 90% CI = [ .03, .05]. However, the ESEM indicated that six items presented very poor psychometric properties such as very low primary factor loadings (λ < .40 on the intended factor; Schaefer et al., 2015) or a small gap between the primary and secondary loading (i.e., .20; Schaefer et al., 2015). Accordingly, we deleted these six items and we re-ran the analyses on the reduced 22-item version of the STRS. The ESEM confirmed its better fit, χ2(168) = 272.56, p < .001, CFI = .96; TLI = .94, RMSEA = .04, 90% CI = [.03, .05], compared with the ICM-CFA model, χ2(206) = 404.45, p < .001, CFI = .92; TLI = .90, RMSEA = .05, 90% CI = [.05, .06]. The statistically significant correlations at the latent level between (a) conflict–closeness and (b) conflict–dependency were less inflated in the ESEM (rs = −.39 and .33, ps < .001, respectively) compared with those estimated in the ICM-CFA (rs = −.49 and .54, ps < .001, respectively), thereby suggesting the higher discriminant validity of the ESEM over the ICM-CFA (Morin et al., 2015). Importantly, although all items strongly loaded onto their intended factor, we also found statistically significant cross-loadings which attested to the unlikely goodness of the ICM-CFA model (see Table 1). Alpha reliability coefficients were .86, .88, and .63 for closeness, conflict, and dependency, respectively. The STRS also showed full scalar MI, thereby attesting to its strong factor consistency both across age groups and sex (see Table 2).
ESEM Standardized Factor Loadings for the Best Fitting Model.
Note. Primary factor loadings are in bold. The six items removed were as follows: 4. c. is uncomfortable with touch from me—reverse-scored item; 21. c. copies my behavior (Close TCR); 19. c. responds to my look—reverse-scored item; 25. c. cries to get something from me (Conflictive TCR); 6. hurt, embarrassed when I correct; 17. c. expresses hurt or jealousy (Dependent TCR). ESEM = exploratory structural equation modeling; TCR = Teacher–child relationship; c. = child.
p < .05. **p < .01. ***p < .001.
Multi-Group ESEM Measurement Invariance (MI) Across Age and Sex.
Note. Factor loadings (λ) and thresholds (ϑ) were constrained to equality across groups. Sixteen children were not rated on their teacher–child relationship by Teacher A and, therefore, they were missing in the MI analyses. Since χ2 values are not exact using WLSMV as method of parameter estimation, χ2 and resulting CFI values can be non-monotonic with model complexity (Muthén & Muthén, 2012). ESEM = exploratory structural equation modeling; MI = measurement invariance; χ2 = Chi-square goodness of fit; CFI = Comparative fit index; TLI = Tucker–Lewis index; RMSEA = Root mean square error of approximation; CI = confidence interval.
The ESEM used to test the validity of the STRS (Figure 1) fit the data relatively well, χ2(263) = 368.64, p < .001, CFI = .95; TLI = .93, RMSEA = .03, 90% CI = [.03, .04], and explained a moderate amount of variance of children’s anxiety-withdrawn (R2 = .17), anger-aggression (R2 = .24), and social competence (R2 = .28). Younger children showed a higher tendency to engage in dependent teacher–child relationships (β = −.21, p < .05). Girls (0 = boys, 1 = girls) were less likely than boys to have conflictive relationships with their teachers (β = −.19, p < .05), whereas they tended to establish closer teacher–child relationships than boys (β = .14, p < .05).
ESEM validity model.
Discussion
To summarize, the ESEM approach corroborated the goodness of fit of the three-factor structure of the STRS in which closeness, conflict, and dependency are three distinct but interrelated facets of the teacher–child relationship quality. However, six out of 28 items of the scale proposed by Pianta (2001) were removed because they showed poor psychometric properties and were ambiguous in their content. For instance, the item related to the child’s use of crying to get something from his or her teacher is not prototypical of the hypothesized dimension of conflict. Children at this age may cry when they want something, regardless of the quality of their relationship with teachers (secure vs. insecure). Interestingly, the removal of these items did not affect the content representation of each subscale and was consistent with previous studies on the factorial validity of the STRS (e.g., Milatz et al., 2014). Future studies are needed to replicate our findings in different cultural/educational contexts.
Importantly, the ESEM showed a better fit compared with the ICM-CFA. This is likely due to the fallacy of the items to capture a specific latent construct only as empirically attested by the significant cross-loadings of many items. The ESEM approach also confirmed its appropriateness to better differentiate the latent structure of STRS. Indeed, the associations among the three factors of the STRS were less inflated in the ESEM compared with the ICM-CFA model. We found that children who established secure relationships with their teachers were moderately less at risk to develop oppositional/conflictive relationships with them. In contrast, children with dependent relationships with teachers also displayed conflictive relationships with them (Sabol & Pianta, 2012). However, future studies should replicate our results by using both ICM-CFA and ESEM methodologies.
The ESEM results also attested to the validity of the STRS in relation to children’s social behaviors. Conflictive teacher–child relationships were related to lower levels of children’s anxious-withdrawn behaviors and social competence and higher levels of aggressive behaviors. This result is in line with previous studies which reported that conflictive teacher–child relationships were strong predictors of children’s oppositional behaviors toward peers (Doumen et al., 2008). Furthermore, our results confirmed that dependent teacher–child relationships may enhance the risk of anxiety and withdrawn behaviors (Sabol & Pianta, 2012). Hence, dependent children who have difficulties separating from their teachers may avoid interactions with peers during group activities or may have difficulties exploring independently the school environment.
In conclusion, the STRS represents a valid and reliable scale to help teachers understand the quality of their relationships with children. Italian teachers should be aware that supportive relationships with children from their earliest school experiences may prevent the onset of children’s social difficulties (Pallini & Laghi, 2012).
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.