Adaptation of the Social Competence and Behavior Evaluation Scale for Adolescents: Structural Validity and Reliability

Abstract

The Social Competence and Behavior Evaluation (SCBE), originally developed for assessing preschoolers, was adapted for the adolescents. The instrument taps social competence, externalizing and internalizing problems. In the adolescent SCBE, more than 65% of the items (54 items) remained practically the same as in the preschool version, 24 items were modified slightly, and two items were rewritten completely. The instrument was tested on 342 adolescents (M = 14.4 years, SD = .6). The summary scales showed high reliability. Using exploratory structural equation modeling (ESEM), acceptable support for the three-factor model based on 16-item clusters was found, indicating that minimal adjustments to the items of the preschool version allows for the assessment of the same constructs in adolescence. The adolescent version of the SCBE can be valid and reliable instrument for describing social adjustment in adolescents making the SCBE interesting from an international perspective.

Keywords

social competence internalizing behavior externalizing behavior adolescents exploratory structural equation modeling

There is large body of research on the measures of social competence and behavioral problems (internalizing and externalizing) from early childhood through adolescence. Some measures are designed for assessment through broader age period—from early childhood to adolescence, for example, Achenbach System of Empirically Based Assessment (Achenbach, 1991; Achenbach & Rescorla, 2001), and consist of versions for different age periods. The other measures focus on specific developmental period, for example, Social Competence and Behavior Evaluation (SCBE, LaFrenière & Dumas, 1995) which is designed for preschoolers.

In Slovenia, only the SCBE measure has been culturally adapted and standardized (LaFrenière, Dumas, Zupančič, Gril, & Kavčič, 2001), but no standardized measure of social competence exists for adolescent age. Research (e.g., longitudinal studies) and practice (e.g., counseling, measuring effects of intervention) show the need for such instrument (e.g., Vidmar, 2011; Zupančič & Kavčič, 2007). Therefore, we adapted Slovene’s SCBE for preschoolers to the adolescent age. The adolescent version is interesting also from an international perspective, particularly in the countries where preschool version already exists. Compared with Achenbach’s instruments (Achenbach, 1991; Achenbach & Rescorla, 2001), the SCBE offers more detailed measures of social adjustment (see below for more detail). This enables more focused counseling as well as more specific research.

SCBE Scale

The SCBE is a standardized instrument, developed by LaFrenière, Dumas, Capuano, and Dubeau (1992) that assesses patterns of social competence, emotion regulation, and expression and adjustment difficulties in children aged 2.5 to about 6 years in preschool setting. The instrument was adapted and validated in French (Dumas, LaFrenière, Capuano, & Durning, 1997) and English (LaFrenière & Dumas, 1995), as well as Spanish (Dumas, Martinez, & LaFrenière, 1998) and Slovene normative sample of preschoolers (LaFrenière et al., 2001). It has been used in educational practice as well as research (e.g., Sette, Baumgartner, & MacKinnon, 2014).

The instrument consists of eight basic bipolar scales, which were conceptually constructed based on the dimensions that are central to the quality of the child’s adaptation to the preschool environment (Dumas et al., 1998; LaFrenière et al., 1992). Each of basic scales consists of five positive and five negative items that form 16-item clusters. The eight positive poles of basic scales define the social competence summary scale (joyful secure, tolerant, integrated, calm, prosocial, cooperative, autonomous), the negative poles define the externalizing problems summary scale (angry, aggressive, egotistical, oppositional) and internalizing problems summary scale (depressive, anxious, isolated, dependent).

The three-factor structure of the SCBE representing three summary scales have been consistently supported in the studies using the principal component analyses (PCAs; see Dumas et al., 1997; Dumas et al., 1998; LaFrenière & Dumas, 1995; LaFrenière et al., 2001 for the French, Spanish, English, and Slovene versions, respectively). The internal consistency of basic and summary scales of the SCBE was uniformly high in several national samples of preschoolers; Cronbach’s alpha ranged from .79 to .91 in the French–Canadian sample (LaFrenière et al., 1992), from .79 to .91 in the U.S. samples (LaFrenière & Dumas, 1995), from .67 to .89 in the Spanish samples in Europe and United States (Dumas et al., 1998), and from .76 to .95 in the Slovene sample (LaFrenière et al., 2001).

Present Study

The aim of the study was to adapt the Slovene version of preschool SCBE for adolescents and verify internal reliability and the structural validity (i.e., one aspect of the construct validity, Messick, 1995) of the adolescent version of SCBE using modern statistical procedures (exploratory structural equation modeling—ESEM). The objective was to test the same model that was cross-culturally found in preschoolers using PCA (e.g., Dumas et al., 1997; Dumas et al., 1998; LaFrenière & Dumas, 1995; LaFrenière et al., 2001); thus, we wanted to test whether 16-item clusters load the expected three factors for the adolescents.¹

Method

Sample and Procedure

The sample consisted of 342 eighth and ninth graders (46.5% girls, 46% boys, 8.5% did not indicate gender; age range: 13 to 16.3 years, M = 14.4, SD = .6) from 19 basic schools in Slovenia. We used convenience sampling of schools (i.e., based on existing contacts with schools). Every fourth student in a class according to the alphabetical list was included in the study.

The adolescents’ social behavior was rated by their class teachers (N = 74) who completed the SCBE. Teachers assessed between one and six students, depending on the size of a class. Teachers had been teaching students for 1 to 4 years (M = 2.8 years). The teachers received the questionnaires and instructions by mail.

Instrument and Adaptation Procedure

In the adaptation procedure, four experts and practitioners independently reviewed the existing Slovenian version of the SCBE. They indicated which of the items describe behaviors that are not relevant for the eighth and ninth graders (and suggested modifications). Their suggestions were reviewed and discussed by the panel of three additional experts (psychologists) that together designed pilot version of the SCBE for adolescents. Almost half of the items (39 out of 80) remained the same, and in the 15 items “child” was dropped (“student” was added when appropriate). In the remaining items, for example, “cry” was dropped/replaced with “worry,” “sad,” or “complain”; “fight” and “hit” was replaced with “harass” or “is rude” (24 items). The Items 4 and 79 that describe specific internalizing behavior of preschoolers were modified more extensively to assure age-appropriate behavior of adolescents.

Statistical Analysis

After calculating descriptive statistics and reliability analysis (using SPSS), the ESEM with target rotation was applied (using Mplus). In the ESEM approach, all factor loadings are estimated—target loadings are estimated freely, while cross-loadings are “targeted” to be close to zero (Morin, Arens, & Marsh, 2015). This is different from the conditions imposed by confirmatory factor analysis (CFA) where each item is allowed to load on its target factor and has zero loadings on every other factor (Furnham, Guenole, Levine, & Chamorro-Premuzic, 2013). ESEM provides us with the benefits (compared with CFA) in terms of model fit indices, standard errors, tests of significance, and distinctiveness of the conceptually based factors (e.g., Marsh, Nagengast, & Morin, 2013). Models were estimated using robust maximum likelihood (MLR).

The fit of all models was evaluated using multiple indices: the chi-square statistic, the comparative fit index (CFI), the root mean square error approximation (RMSEA), and the standardized root mean square residual (SRMR). Several different cutoffs are cited in the literature for an acceptable (good or adequate) fit of a model: for example, CFI > .90 or .95; RMSEA< .05 or .06, or .08; SRMR< .08 (see Browne & Cudeck, 1993; Hair, Anderson, Tatham, & Black, 1998; Hu & Bentler, 1998; Marcoulides & Hershberger, 1997). As indicated by Kenny (2014), some researchers have suggested RMSEA = .10 as the cutoff criteria; values between .08 and .10 indicate mediocre fit (MacCallum, Browne, & Sugawara, 1996), and values above .10 indicate poor fit (Browne & Cudeck, 1993). Thus, in this study, we observed the following fairly lenient criteria for acceptable model fit: CFI > 0.90, RMSEA < 0.10, and SRMR < 0.08.

Results With Discussion

Analyses focused on the factor structure of newly adapted adolescent version of the SCBE. The three-factor structure obtained in preschoolers was presupposed for adolescents, and a model based on 16-item clusters was tested.

Descriptive Statistics and Reliability

In Table 1, descriptive statistics and reliability for the 16-item clusters are shown.

Table 1.

Descriptives and Reliability for 16-Item Clusters.

	M	SD	Skewness	Kurtosis	Cronbach’s α
Joyful	16.70	5.26	−.30	−.82	.83
Tolerant	15.40	5.31	−.20	−.48	.79
Secure	16.24	5.36	−.30	−.69	.83
Calm	14.98	5.54	−.26	−.60	.83
Integrated	14.48	5.99	−.03	−.86	.89
Prosocial	14.54	6.05	−.14	−.95	.89
Autonomous	14.61	5.03	−.09	−.75	.72
Cooperative	17.10	5.48	−.50	−.58	.83
Depressive	16.90	4.63	−.32	−.53	.67
Anxious	16.84	4.64	−.56	.06	.65
Isolated	18.34	5.06	−.81	.68	.78
Dependent	17.42	4.01	−.60	.33	.56
Angry	18.77	5.51	−.88	−.04	.83
Aggressive	22.19	4.38	−1.96	3.51	.89
Egotistical	18.63	4.71	−.75	.17	.75
Oppositional	20.05	5.49	−1.17	.58	.88

Note. Following the recommendation of Curran, West, and Finch (1996) for assuring multivariate normality, no variables needed to be transformed due to excessive skew or kurtosis (all univariate skewness < 2.0 and kurtosis < 7.0).

Internal reliability (Cronbach’s alpha) of eight bipolar scales ranged from .69 for autonomous–dependent scale to .90 for integrated–isolated scale. Alpha coefficients of summary scales were .86 for internalizing behavior, .93 for externalizing behavior, and .96 for social competence scale. Alphas were also very similar to the ones obtained for the French (LaFrenière et al., 1992), English (LaFrenière & Dumas, 1995), and Spanish versions of SCBE (Dumas et al., 1998).

Structural Validity

Using ESEM and target rotation, we tested whether the three-factor structure based on 16-item clusters found in preschool samples across several cultures and languages is found also for the adolescent version. Item clusters represent 16 negative and positive poles of the eight basic scales of the questionnaire. The model is depicted in the Figure 1.

Figure 1.

ESEM based on item clusters.

The fit of the initial model without correlated errors was as follows: CFI = .91, SRMR = .030, χ²(75) = 400.22, p = .000, RMSEA = .113, confidence interval [CI] = [.102 – .124]. After including six correlated errors between item clusters belonging to the opposite poles of the basic scales (as suggested by modification indices), the model demonstrated adequate fit: CFI = .94, SRMR = .025, χ²(69) = 279.31, p = .000, RMSEA = .094, CI = [.083 – .106]. The RMSEA is close to 0.10 indicating mediocre fit; however, other modification indices indicate good fit. The acceptable support for the presupposed three-factor model is implied also in the high target and low nontarget loadings. All the model-estimated target loadings for the indicators were significant (p < .001). Target factor loadings ranged from .53 to .90. All cross-loadings were below .33 and are thus not shown.

Social competence and internalizing behavior were negatively moderately correlated (r = −.46, p < .001). The correlation between social competence and externalizing behavior was also negative and moderate (r = −.31, p < .001). The correlation between internalizing and externalizing behavior was close to zero and nonsignificant (r = −.05, p = .31), indicating almost perfect orthogonality between the two constructs. The correlations obtained in French (LaFrenière et al., 1992) and Slovenian (LaFrenière et al., 2001) versions for preschoolers are somewhat higher, which can be attributed to different analyses used in this study (ESEM compared with PCA).

The model is concordant with previous validation of the preschool version of SCBE in Slovenia (LaFrenière et al., 2001), Spain (Dumas et al., 1998), the United States, and Canada (Dumas et al., 1997; LaFrenière & Dumas, 1995). In these studies, acceptable support was found that social adjustment of preschool children can be described with three main factors of social competence, internalizing behavior, and externalizing behavior using 16-item clusters. Interestingly, the PCAs of the SCBE on preschool samples in these studies revealed several substantial cross-loadings at the level of item clusters, whereas in the present study using ESEM, the cross-loadings were substantially lower compared with the loadings on the target factors. For example, in the previous Slovenian study, joyful, secure, integrated, and autonomous item clusters loaded higher on the internalizing behavior scale than on the (target) social competence scale (LaFrenière et al., 2001); on the Spanish sample (Dumas et al., 1997), the item cluster isolated loaded higher on the social competence scale than on the (target) internalizing scale, and item clusters calm, prosocial, and cooperative loaded higher on externalizing problems scale than on the (target) social competence scale. Thus, the findings from this study clearly support the three-factor structure based on item clusters for the Slovene adolescent version of the SCBE.

Limitations and Strengths

Several limitations mark this study. The age range of adolescents in the sample was quite narrow. Moreover, no other instrument was used in the study that would allow conclusions in regard to the other aspects of validity (i.e., external, Messick, 1995) of the adolescent SCBE. The study also has some strengths; all analyses were conducted with medium-size sample, using advanced statistical tools (MLR, ESEM).

Conclusion

The adolescent SCBE has psychometric characteristics that closely parallel to the preschool version in Slovenian as well as French, Spanish, and English. The adolescent version of SCBE proved to be a reliable and valid (structural validity) instrument for describing social competence and problem behavior in adolescents at the level of summary scales. The study also shows that minimal adjustments to the items of the preschool version allow for the assessment of the same constructs in adolescence making it interesting from the international perspective (for the existing SCBE language versions).

In future studies, external aspects of validity (predictive, concurrent) for adolescent SCBE would need to be examined.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Notes

References

Achenbach

T. M.

(1991). Integrative guide for the CBCL/4-18, YSR and TRF profiles. Burlington: Department of Psychiatry, University of Vermont.

Achenbach

T. M.

Rescorla

L. A.

(2001). Manual for the ASEBA school-age forms & profiles. Burlington: Research Centre for Children, Youth and Families, University of Vermont.

Browne

M. W.

Cudeck

(1993). Alternative ways of assessing model fit. In Bollen

Long

(Eds.), Testing structural equation models (pp. 136-162). Newbury Park, CA: Sage.

Curran

P. J.

West

S. G.

Finch

J. F.

(1996). The robustness of test statistics to nonnormality and specification error in confirmatory factor analysis. Psychological Methods, 1, 16-29.

Dumas

J. E.

LaFreniere

P. J.

Capuano

Durning

(1997). Profil Socio-Affectif (PSA) [Socio-emotional Profile (SEP)]. Paris, France: Les Editions du Centre de Psychologie Appliquée.

Dumas

J. E.

Martinez

LaFreniere

P. J.

(1998). The Spanish version of the Social Competence and Behavior Evaluation (SCBE)–Preschool Edition: Translation and filed testing. Hispanic Journal of Behavioral Sciences, 20, 255-269.

Furnham

Guenole

Levine

S. Z.

Chamorro-Premuzic

(2013). The NEO Personality Inventory–Revised: Factor structure and gender invariance from exploratory structural equation modeling analysis in a high-stakes settings. Assessment, 20, 14-23.

Hair

J. F.

Jr. Anderson

R. E.

Tatham

R. L.

Black

W. C.

(1998). Multivariate data analysis (5th ed.). Upper Saddle River, NJ: Prentice Hall.

Bentler

P. M.

(1998). Fit indices in covariance structure modeling: Sensitivity to underparameterized model misspecification. Psychological Methods, 3, 424-453.

10.

Kenny

D. A.

(2014). Measuring model fit. Retrieved from http://davidakenny.net/cm/fit.htm

11.

LaFrenière

P. J.

Dumas

J. E.

(1995). Social Competence and Behavior Evaluation, Preschool Edition (SCBE). Los Angeles, CA: Western Psychological Services.

12.

LaFrenière

P. J.

Dumas

J. E.

Capuano

Dubeau

(1992). Development and validation of the Preschool Socioaffective Profile. Psychological Assessment, 4, 442-450.

13.

LaFrenière

P. J.

Dumas

J. E.

Zupančič

Gril

Kavčič

(2001). Vprašalnik o socialnem vedenju otrok, SV-O priročnik [Social competence and behavior evaluation, the manual]. Ljubljana, Slovenia: Center za psihodiagnostična sredstva.

14.

MacCallum

R. C.

Browne

M. W.

Sugawara

H. M.

(1996). Power analysis and determination of sample size for covariance structure modeling. Psychological Methods, 1, 130-149.

15.

Marcoulides

G. A.

Hershberger

S. L.

(1997). Multivariate statistical methods: A first course. Mahwah, NJ: Lawrence Erlbaum.

16.

Marsh

H. W.

Nagengast

Morin

A. J. A.

(2013). Measurement invariance of big-five factors over the life span: ESEM tests of gender, age, plasticity, maturity, and la dolce vita effects. Developmental Psychology, 49, 1194-1218.

17.

Messick

(1995). Validity of psychological assessment: Validation of inferences from persons’ responses and performances as scientific inquiry into score meaning. American Psychologist, 50, 741-749.

18.

Morin

A. J.

Arens

A. K.

Marsh

H. W.

(2016). A bifactor exploratory structural equation modeling framework for the identification of distinct sources of construct-relevant psychometric multidimensionality. Structural Equation Modeling: A Multidisciplinary Journal, 23, 116-139.

19.

Sette

Baumgartner

MacKinnon

D. P.

(2015). Assessing Social Competence and Behavior Problems in a Sample of Italian Preschoolers Using the Social Competence and Behavior Evaluation Scale. Early Education and Development, 26, 46-65.

20.

Vidmar

(2011). Socialna kompetentnost in učna uspešnost v prvih razredih osnovne šole [Social competence and academic achievement in the first grades]. Šolsko polje, 22(1-2), 31-54.

21.

Zupančič

Kavčič

(2007). Otroci od vrtca do šole: razvoj osebnosti in socialnega vedenja ter učna uspešnost prvošolcev [Children from kindergarten to school: The development of personality and social behaviour, and academic achievement of the first-graders]. Ljubljana, Slovenia: Znanstvenoraziskovalni inštitut Filozofske fakultete.