Adaptation of the Social Competence and Behavior Evaluation Scale for Adolescents: Structural Validity and Reliability

Present Study

The aim of the study was to adapt the Slovene version of preschool SCBE for adolescents and verify internal reliability and the structural validity (i.e., one aspect of the construct validity, Messick, 1995) of the adolescent version of SCBE using modern statistical procedures (exploratory structural equation modeling—ESEM). The objective was to test the same model that was cross-culturally found in preschoolers using PCA (e.g., Dumas et al., 1997; Dumas et al., 1998; LaFrenière & Dumas, 1995; LaFrenière et al., 2001); thus, we wanted to test whether 16-item clusters load the expected three factors for the adolescents. ¹

Sample and Procedure

The sample consisted of 342 eighth and ninth graders (46.5% girls, 46% boys, 8.5% did not indicate gender; age range: 13 to 16.3 years, M = 14.4, SD = .6) from 19 basic schools in Slovenia. We used convenience sampling of schools (i.e., based on existing contacts with schools). Every fourth student in a class according to the alphabetical list was included in the study.

The adolescents’ social behavior was rated by their class teachers (N = 74) who completed the SCBE. Teachers assessed between one and six students, depending on the size of a class. Teachers had been teaching students for 1 to 4 years (M = 2.8 years). The teachers received the questionnaires and instructions by mail.

Instrument and Adaptation Procedure

In the adaptation procedure, four experts and practitioners independently reviewed the existing Slovenian version of the SCBE. They indicated which of the items describe behaviors that are not relevant for the eighth and ninth graders (and suggested modifications). Their suggestions were reviewed and discussed by the panel of three additional experts (psychologists) that together designed pilot version of the SCBE for adolescents. Almost half of the items (39 out of 80) remained the same, and in the 15 items “child” was dropped (“student” was added when appropriate). In the remaining items, for example, “cry” was dropped/replaced with “worry,” “sad,” or “complain”; “fight” and “hit” was replaced with “harass” or “is rude” (24 items). The Items 4 and 79 that describe specific internalizing behavior of preschoolers were modified more extensively to assure age-appropriate behavior of adolescents.

Statistical Analysis

After calculating descriptive statistics and reliability analysis (using SPSS), the ESEM with target rotation was applied (using Mplus). In the ESEM approach, all factor loadings are estimated—target loadings are estimated freely, while cross-loadings are “targeted” to be close to zero (Morin, Arens, & Marsh, 2015). This is different from the conditions imposed by confirmatory factor analysis (CFA) where each item is allowed to load on its target factor and has zero loadings on every other factor (Furnham, Guenole, Levine, & Chamorro-Premuzic, 2013). ESEM provides us with the benefits (compared with CFA) in terms of model fit indices, standard errors, tests of significance, and distinctiveness of the conceptually based factors (e.g., Marsh, Nagengast, & Morin, 2013). Models were estimated using robust maximum likelihood (MLR).

The fit of all models was evaluated using multiple indices: the chi-square statistic, the comparative fit index (CFI), the root mean square error approximation (RMSEA), and the standardized root mean square residual (SRMR). Several different cutoffs are cited in the literature for an acceptable (good or adequate) fit of a model: for example, CFI > .90 or .95; RMSEA< .05 or .06, or .08; SRMR< .08 (see Browne & Cudeck, 1993; Hair, Anderson, Tatham, & Black, 1998; Hu & Bentler, 1998; Marcoulides & Hershberger, 1997). As indicated by Kenny (2014), some researchers have suggested RMSEA = .10 as the cutoff criteria; values between .08 and .10 indicate mediocre fit (MacCallum, Browne, & Sugawara, 1996), and values above .10 indicate poor fit (Browne & Cudeck, 1993). Thus, in this study, we observed the following fairly lenient criteria for acceptable model fit: CFI > 0.90, RMSEA < 0.10, and SRMR < 0.08.

Descriptive Statistics and Reliability

In Table 1, descriptive statistics and reliability for the 16-item clusters are shown.

OPEN IN VIEWER

Table 1.

Descriptives and Reliability for 16-Item Clusters.

	M	SD	Skewness	Kurtosis	Cronbach’s α
Joyful	16.70	5.26	−.30	−.82	.83
Tolerant	15.40	5.31	−.20	−.48	.79
Secure	16.24	5.36	−.30	−.69	.83
Calm	14.98	5.54	−.26	−.60	.83
Integrated	14.48	5.99	−.03	−.86	.89
Prosocial	14.54	6.05	−.14	−.95	.89
Autonomous	14.61	5.03	−.09	−.75	.72
Cooperative	17.10	5.48	−.50	−.58	.83
Depressive	16.90	4.63	−.32	−.53	.67
Anxious	16.84	4.64	−.56	.06	.65
Isolated	18.34	5.06	−.81	.68	.78
Dependent	17.42	4.01	−.60	.33	.56
Angry	18.77	5.51	−.88	−.04	.83
Aggressive	22.19	4.38	−1.96	3.51	.89
Egotistical	18.63	4.71	−.75	.17	.75
Oppositional	20.05	5.49	−1.17	.58	.88

Note. Following the recommendation of Curran, West, and Finch (1996) for assuring multivariate normality, no variables needed to be transformed due to excessive skew or kurtosis (all univariate skewness < 2.0 and kurtosis < 7.0).

Internal reliability (Cronbach’s alpha) of eight bipolar scales ranged from .69 for autonomous–dependent scale to .90 for integrated–isolated scale. Alpha coefficients of summary scales were .86 for internalizing behavior, .93 for externalizing behavior, and .96 for social competence scale. Alphas were also very similar to the ones obtained for the French (LaFrenière et al., 1992), English (LaFrenière & Dumas, 1995), and Spanish versions of SCBE (Dumas et al., 1998).

Structural Validity

Using ESEM and target rotation, we tested whether the three-factor structure based on 16-item clusters found in preschool samples across several cultures and languages is found also for the adolescent version. Item clusters represent 16 negative and positive poles of the eight basic scales of the questionnaire. The model is depicted in the Figure 1.

OPEN IN VIEWER

Figure 1.

ESEM based on item clusters.

The fit of the initial model without correlated errors was as follows: CFI = .91, SRMR = .030, χ²(75) = 400.22, p = .000, RMSEA = .113, confidence interval [CI] = [.102 – .124]. After including six correlated errors between item clusters belonging to the opposite poles of the basic scales (as suggested by modification indices), the model demonstrated adequate fit: CFI = .94, SRMR = .025, χ²(69) = 279.31, p = .000, RMSEA = .094, CI = [.083 – .106]. The RMSEA is close to 0.10 indicating mediocre fit; however, other modification indices indicate good fit. The acceptable support for the presupposed three-factor model is implied also in the high target and low nontarget loadings. All the model-estimated target loadings for the indicators were significant (p < .001). Target factor loadings ranged from .53 to .90. All cross-loadings were below .33 and are thus not shown.

Social competence and internalizing behavior were negatively moderately correlated (r = −.46, p < .001). The correlation between social competence and externalizing behavior was also negative and moderate (r = −.31, p < .001). The correlation between internalizing and externalizing behavior was close to zero and nonsignificant (r = −.05, p = .31), indicating almost perfect orthogonality between the two constructs. The correlations obtained in French (LaFrenière et al., 1992) and Slovenian (LaFrenière et al., 2001) versions for preschoolers are somewhat higher, which can be attributed to different analyses used in this study (ESEM compared with PCA).

The model is concordant with previous validation of the preschool version of SCBE in Slovenia (LaFrenière et al., 2001), Spain (Dumas et al., 1998), the United States, and Canada (Dumas et al., 1997; LaFrenière & Dumas, 1995). In these studies, acceptable support was found that social adjustment of preschool children can be described with three main factors of social competence, internalizing behavior, and externalizing behavior using 16-item clusters. Interestingly, the PCAs of the SCBE on preschool samples in these studies revealed several substantial cross-loadings at the level of item clusters, whereas in the present study using ESEM, the cross-loadings were substantially lower compared with the loadings on the target factors. For example, in the previous Slovenian study, joyful, secure, integrated, and autonomous item clusters loaded higher on the internalizing behavior scale than on the (target) social competence scale (LaFrenière et al., 2001); on the Spanish sample (Dumas et al., 1997), the item cluster isolated loaded higher on the social competence scale than on the (target) internalizing scale, and item clusters calm, prosocial, and cooperative loaded higher on externalizing problems scale than on the (target) social competence scale. Thus, the findings from this study clearly support the three-factor structure based on item clusters for the Slovene adolescent version of the SCBE.

Limitations and Strengths

Several limitations mark this study. The age range of adolescents in the sample was quite narrow. Moreover, no other instrument was used in the study that would allow conclusions in regard to the other aspects of validity (i.e., external, Messick, 1995) of the adolescent SCBE. The study also has some strengths; all analyses were conducted with medium-size sample, using advanced statistical tools (MLR, ESEM).