Abstract
The Social Skills Rating System (SSRS) developed by Gresham and Elliott (1990) is a multirater, norm-referenced instrument measuring social skills and adaptive behavior in preschool children. The aims of the present study were (a) to test the factorial structure of the Parent Form of the SSRS for the first time with a German preschool sample (391 children) and (b) to present a modified version appropriate for 3- to 6-year-olds in German preschools. The sample consisted of 391 children (187 males, 204 females) from German preschools and their parents. Confirmatory factor analyses (CFAs) revealed a poor overall fit testing the original version. For the extended age range (3-6 years) a revised version of the SSRS with a reduced and newly composed item pool is proposed based on explorative factor analyses. Results are discussed with respect to practicability and revision of SSRS for German preschool populations.
Social competence can be broadly defined as being effective in social interactions (Rubin, Bukowski, & Parker, 2006). More precisely, Rubin and Rose-Krasnor (1992) have conceptualized social competence as the ability to achieve personal goals in social interactions while simultaneously maintaining positive relationships with others over time and across situations. In short social competence refers to the overall ability of an individual to act in a socially appropriate manner (Gresham, 1986) whereas the “social skills approach” refers to specific and distinct behaviors representing social competence (Sheridan & Walker, 1999). To develop valid and reliable instruments for self and peer assessment of social competence it seems helpful to take a social skills approach, as Gresham and Elliot (1990) did with their Social Skills Rating System (SSRS). The present study tested the psychometric properties of the Parent Form of this measure for the first time with a German preschool parent sample.
The preschool years are a sensitive period for the acquisition of social skills and related abilities (particularly perspective taking, empathy, and intersubjectivity; e.g., Benenson, Markovits, Roy, & Denko, 2003; Dunn & Cutting, 1999; Göncü, 1993; Watson, Nixon, Wilson, & Capage, 1999). Inter- and intraindividual variation in the acquisition and performance of social skills (e.g., Achenbach & Edelbrock, 1981; Cavell, 1990; Dirks, Treat, & Weersing, 2007) calls for methods to measure social skills adequately. The SSRS (Gresham & Elliott, 1990) is a popular multirater, norm-referenced instrument designed to measure social skills and adaptive behavior in preschool, elementary, and secondary school children across three domains: social skills, problem behaviors, and academic competence. Different versions for self-ratings, teacher ratings, and parent ratings have been developed, and norms are based on a large sample of 4,000 boys and girls aged 3- to 18-years gathered from the United States (see the SSRS Manual; Gresham & Elliott, 1990). In this study, only the Parent Preschool Form of the social skills part of the SSRS will be considered. The Parent Preschool Form of the SSRS measures social skills in four domains (cooperation, assertion, responsibility, and self-control). These domains resemble the first four of five categories Caldarella and Merrell (1997) defined as the main categories of social competence (peer relations, assertion, compliance, self management, and academic-related social competence). Originally, each item is rated on “frequency” (“how often?”) and “importance” (“how important?”). The importance ratings of the SSRS have not been analyzed by Gresham and Elliott (1990) to form subscales. They are, however, of clinical relevance to examine if certain deficits in social skills are relevant for the respective raters. Additionally, these importance ratings may have implications for intervention planning. As the present article has no clinical focus, the importance of the ratings will not be discussed further. Irrespective of advantages and disadvantages, this form of parent ratings are often used and can be defined as a standard for the assessment of children’s social skills during the preschool years (cf. Erdley, Nangle, Burns, Holleb, & Kaye, 2010).
When adapting inventories for implementation in different countries it is not sufficient to translate the items into the local language. One of the aspects to consider is whether institutional constraints have an impact on the implementation of inventories. Although these constraints may reflect theoretical assumptions, for example, about individual development, they are not universal. For example, in the USA the preschool years include 3- and 4-year-old children, whereas in Germany the preschool years or kindergarten years, respectively, usually range from 3 to 6 years. For practicability reasons it would be helpful to have one instrument addressing all children in one institutional context. Because the SSRS preschool version is standardized for 3- and 4-year-old (preschool) children, the present form may not be suitable for all children in German kindergartens. As, to our knowledge, we are the first to present a German version of the SSRS (Parent Preschool Form), our aim was to propose a version (based on explorative analysis) ready to use with children from ages 3 to 6 in Germany covering the whole kindergarten period with one instrument.
In addition, we wanted to test the criterion validity of the revised version of the SSRS. Several measures were selected as criteria for concurrent and divergent validity. First, based on a number of studies (e.g., Merrell, 1995, Parker, Rubin, Erath, Wojslawowicz, & Buskirk, 2006), we expected that high levels of anxious behavior are associated with low levels of social skills. Second, we expected to find a negative correlation between “negative emotionality,” representing a temperament factor, and social skills. In their review, Sanson, Hemphill, and Smart (2004) concluded that low negative emotionality is associated with the development of positive social skills and prosociality. Finally, we also expected that the temperamental factors “activity” and “persistence” are associated with high levels of social skills, whereas “social inhibition” should go along with low levels of social skills (Sanson et al., 2004). The age span between 3- and 6-years represents an important period with significant changes (i.e., improvements) in social behavior and developing social skills (for an overview see Parke & Clarke-Stewart, 2011). Therefore, as a further indicator of validity of the SSRS we tested if we can find an increase in social skills in different age groups within our sample.
Although acceptable psychometric properties have been reported for the Parent Versions of the SSRS in some reviews of social skills questionnaires (e.g., Bracken, Keith, & Walker, 1998; Demaray et al., 1995), in recent years, the factorial structure of the original parent-frequency rating scales have been discussed (Fantuzzo, Manz, & McDermott, 1998; Manz, Fantuzzo, & McDermott, 1999; Van der Oord et al., 2005; Van Horn, Atkins-Burnett, Karlin, Ramey, & Snyder, 2007; Whiteside, McCarthy, & Miller, 2007). However, most of the studies dealing with the psychometric properties of the SSRS are based on elementary samples and not on preschool children. With a Dutch sample of 239 children from elementary schools Van der Oord et al. (2005) could replicate three factors of the original Parent Elementary Scale (“assertion,” “self-control,” and “cooperation”) whereas the fourth factor (“responsibility”) could not be replicated. Whiteside et al. (2007) conducted a study with 298 children from the Midwestern of the USA. As in the study by Van der Oord et al. (2005), problems occurred with the replication of the SSRS scale “responsibility.” Van Horn et al. (2007) conducted a longitudinal study with the Elementary Parent Form of the SSRS, based on data from 4,345 school children ranging from kindergarten to third grade collected from all over the USA. The authors could not replicate the original structure of the scales. Unlike other studies they stuck to a four-factor solution (“cooperation,” “assertion,” “self-control,” and “responsibility”) with a modified distribution and a substantial reduction of items. Finally, the only study based on a preschool sample and the Preschool Parent Version of the SSRS was conducted by Manz and colleagues (1999). They tested the psychometric properties of the SSRS Preschool Parent Version with 859 Afro-American children. As a result of their factorial analyses a two-factor solution emerged. The first factor consisted of items describing appropriate behavior at home called “self-control” (14 items). The second factor consisted of items measuring skills to cope with social situations in general, called “interpersonal skills” (16 items).
In sum, there is evidence suggesting that the factorial structure of the SSRS (Parent Version) is not robust. Particularly, the subscale “responsibility,” measuring behavior concerning the ability to communicate with adults and the sense of respect for property and work, is discussed in the literature.
With respect to the ambiguous results reported above and preceding the main explorative analysis, a confirmatory analysis will be conducted to test if the original factorial structure of the parent version of the preschool SSRS can be confirmed with a German preschool sample. In addition, we will test if the alternative model proposed by Manz et al. (1999) better fits the present data than the original version. We will not test the other alternative models reviewed above because they were derived from older samples or the Elementary Parent Version of the SSRS. They were reported to corroborate findings reporting evidence that the Parent Version of the SSRS has to be judged with caution. To our knowledge, no study has reported the factorial structure of the Preschool Parent Version of the SSRS with a German preschool sample until now. We expect that modifications of the original scales will be necessary to achieve an acceptable factorial structure.
Method
Participants
A total of 391 children (187 males, 204 females) from German preschools and their parents stemming from the BELLA preschool study participated in our study. Between October 2005 and May 2006, parents from the larger KiGGS study (Kurth, 2007) with children from ages 3 to 6 were asked to participate in the BELLA preschool study. After receiving parents’ informed consent the questionnaires were sent to 450 families. A total of 391 questionnaires were returned (87% response rate). The mean age of the children was 4.6 with a range from 2 to 6 years. Most questionnaires were answered by mothers alone (81%), 3% of the questionnaires were answered by fathers alone, and in 16% of the cases both parents answered the questions. Mean age of the mothers was 34.6 years. The average age of fathers was 37.7 years. In 24% of the families the child had no sibling. We found no differences in ratings on all scales between families with one or more than one child as well as between different raters (applying univariate ANOVAs). Eight percent of the entire sample had a migrant background. As 20% from the general German population have a migrant status, the data are hardly representative in this respect (Statistisches Bundesamt, 2011). According to a multidimensional index of socioeconomic status (SES; Winkler, 1998), combining information about educational level, income, and professional status, 28% of the families had a low SES, 46% had an average status, and 26% stemmed from an upper social class. Children with low SES were rated as being less self-controlled, more active, emotionally more negative and more aggressive than children with a higher SES (univariate ANOVAs with post hoc tests, for description of the measures, see below).
Measures
Social Skills
Social skills were assessed using a German translation of the Social Skills Scale of the Preschool Parent Version of the SSRS. The items were translated and back-translated by experts. The Social Skills Scale comprises 39 items divided into four subscales; responses are provided on a 3-point Likert-type scale (“never” = 0, “sometimes” = 1, “very often” = 2). One item was assigned to two subscales, so that each scale consists of 10 items. The first subscale “cooperation” (e.g., “The child will volunteer to help family members with tasks”) measures behavior reflecting helping others, sharing with others, and complying with rules. The second subscale “assertion” (e.g., “The child will participate in organized group activities”) deals with initiating behaviors like seeking information from others, or responding to actions of others. The third subscale “responsibility” (e.g., “The child will answer the phone appropriately”) asks for behavior that demonstrates the skill to interact properly with adults. The subscale “self-control” (e.g., “The child will wait turns in games or activities”) measures behaviors, like reactions to attempts of teasing or the ability to compromise.
Behavior Problems
In this study, two subscales of the parent version of the German translation of the Child Behavior Checklist 1½-5 (Arbeitsgruppe Deutsche Child Behavior Checklist, 2002) were completed by the parents. The first one, assessing anxious/depressed behavior, comprises eight items (e.g., “is unhappy and sad”), and the second subscale, assessing aggressive behavior, comprises 19 items (e.g., “hits others,” “gets upset often”). The parents made responses on a 3-point Likert-type scale (“not true” = 1, “sometimes true” = 2, “exactly true” = 3). Reliability was measured using the intraclass correlation (ICC). Following the distinction proposed by Shrout and Fleiss (1979) for the present data structure the ICC (2, k) is appropriate, equalling the Cronbach’s Alpha value. In addition, we decided to present the 95% CI for all ICC (2, k) measures in this study. Reliabilities for the Child Behavior Checklist scales in this study were questionable (ICC [2, k]; anxiety = .68, 95% CI [0.63, 0.73]) and good (ICC [2, k] aggression = .86, 95% CI [0.83, 0.88]) and altogether judged as satisfactory concerning the purpose of the present study.
Temperament
To assess the temperament of the children, a short version of the Temperament Assessment Battery for Children (TABC; Newman, Noel, Chen, & Matsopoulos, 1998) was applied. The short version comprises 15 items, equally divided into five subscales called “activity level” (e.g., “prefers active play to quiet types of activities”), “adaptability/agreeableness” (e.g., “has difficulty adjusting to changes in daily routine”), “negative emotionality” (e.g., “has temper outbursts with screaming and/or crying”), “persistence” (e.g., “works on projects until they are finished”), and “social inhibition” (e.g., “is slow to warm up to new adults“). Responses are provided on a 5-point Likert-type scale (“never” = 1, “seldom” = 2, “sometimes” = 3, “often” = 4, “always” = 5). Newman et al. (1998) tested this short version of the TABC and judged its psychometric properties as satisfactory. In this study, reliability ranged from rather low to acceptable (ICC[2, k]activity = .56, 95% CI [0.48, 0.63]; ICC[2, k]agreeableness = .52, 95% CI [0.43, 0.59]; ICC[2, k]emotionality = .74, 95% CI [0.69, 0.78]; ICC[2, k]persistence = .63, 95% CI [0.56, 0.69]; ICC[2, k]inhibition = 0.67; 95% CI [0.61, 0.72]). As the TABC scales consist of three items only we considered the item intercorrelations as a further indicator of consistency. The item intercorrelations were adequate for the subscales “negative emotionality”—ranging from r = .44 to r = .59 (all p < .01)—and “social inhibition”—ranging from r =.30 to r = .49 (all p < .01)—only. Due to the reliability values and item intercorrelations, and different from the planned analyses proposed in the introductory section only the subscales “negative emotionality” and “inhibition” were considered for further analyses.
Statistical Procedures
To test the factorial structure of the original SSRS and the alternative item distribution proposed by Manz et al. (1999) CFAs were conducted to investigate which model fits the present data best. As the present study represents the first attempt to test the SSRS with a German preschool sample, we conducted additional exploratory factor analyses. This was done to be able to detect a factorial structure of the SSRS especially valid for German populations. Due to scaling of the items and skewness of the distributions all factorial analyses are based on maximum likelihood parameter estimates with standard errors and a chi square statistic that are not affected by nonnormality of data (MLR). For data analysis we used Mplus 5.2. To compare the original SSRS scales with the alternative solution by Manz et al. (1999) using chi square differences a correction of chi square statistics proposed by Satorra and Bentler (2001) was necessary. Missing values were replaced by using full information maximum likelihood estimators as implemented in Mplus 5.2. Psychometric properties were analyzed by computing ICCs. Based on the assumption of a metric data level on the subscale level bivariate Pearson correlations were used for validation of the scales and to evaluate interscale relationships. For validation purposes, we built two age groups to analyze age trends. The “young age group” consisted of 2- to 4-year-old children (n = 193) and the “old age group” consisted of 5- and 6-year-old children (n = 198).
Results
Replication of the Original Scales of the SSRS and the Alternative Scales Proposed by Manz et al. (1999)
To verify the original factorial structure of the SSRS, a CFA was conducted. Four indices were used to evaluate the fit of the CFA model, namely the Tucker-Lewis-Index (TLI), the comparative fit index (CFI), the root mean square error of approximation (RMSEA) with its 90% CI, and the chi square value. The first two indices are “incremental fit indices” comparing the fit of the target model with the fit of a baseline model. The baseline model is assumed to have no covariances between the variables (Geiser, 2010). The RMSEA is an approximate estimate of the model fit. Values of .90 or above at the TLI and CFI, and values of .05 or less at the RMSEA indicate a good fit of a model (Browne & Cudeck, 1993; Hu & Bentler, 1999). To compare the different models the corrected chi square differences described above were used.
The overall model fit was critical (χ2 = 1,387.10; df = 695, p < .001; TLI = .72; CFI = .81, and RMSEA = .05 [90% CI = 0.047/0.054]) with incremental fit indices below the threshold. The RMSEA implied an acceptable model fit. The alternative two-factor model proposed by Manz et al. (1999) showed slightly weaker fit indices (χ2= 901.59; df = 404, p < .001; TLI = .73; CFI = .75, and RMSEA = .06 [90% CI = 0.051/0.061]). The difference between the two models was significant (Δχ2 = 482.40; df = 291; p < .001) indicating that the original factorial structure of the SSRS fits the present data better than the alternative model proposed by Manz et al. (1999). As the fit indices of the original model were not convincing further analysis should reveal possible modifications on an exploratory level.
Modification of the SSRS
We computed exploratory factor analyses for the extended age range (3-6 years) of our sample. Results are again based on MLR estimators. As all scales measure different aspects of social skills, the resulting factors were allowed to be correlated. An oblique rotation method (Geomin) was applied. Model fit was evaluated using the chi square value, the RMSEA as well as the TLI and CFI indexes. In the main analysis, the number of factors was limited to four factors for two reasons: (a) the Scree plot (Costello & Osborne, 2005) as a measure to decide about number of factors indicated that four factors should be extracted and (b) the extraction procedure is not entirely exploratory because the item pool was already tested during the development of the SSRS and similar categories of social skills have already been found by Caldarella and Merrell (1997). Combining statistical and conceptual considerations as it is common in exploratory factor analysis (e.g., Haig, 2005) a three-factor solution proved to suit the data best (χ2 = 1,017.52; df = 627; p < .001; TLI = .82; CFI = .84; RMSEA = .04 [90% CI = 0.035/0.044]). However, eight items with factor loadings below the threshold of .30 were eliminated. This common but relative low threshold was chosen due to our sample size of more than 350 participants (Stevens, 2009; Tabachnik & Fidell, 1989). After eliminating items with low loadings an additional exploratory factor analysis was conducted to eliminate the influence of low loading distracters (Brown, 2006). Again, combining statistical information and theoretical considerations a three-factor solution emerged (χ2 = 631.26; df = 365; p < .001; TLI = .85; CFI = .88; RMSEA = .04 [90% CI = 0.036/0.047]), although incremental fit indices were still slightly below the recommended threshold.
Table 1 gives an overview of the modified subscale composition of the SSRS. A total of 31 items distributed over three subscales remained. The three scales resemble the original scales proposed by Gresham and Elliott (1990) with a reduced item pool, for the two scales “cooperation” (6 items), and “assertion and initiative” (9 items) and an enhanced item pool for the scale “self-control” (16 items). Yet, the fourth factor from the original SSRS version, “responsibility,” could not be replicated with the present data. Correlations between factors were medium in size with values of r = .31 between “self-control” and “assertion,” r = .44 between “self-control” and “cooperation” and r = .41 between “assertion” and “cooperation.”
Factorial Structure From the Exploratory Factor Analysis of the German Version of the Parent Preschool SSRS Subscales (MLR Estimates, Geomin Rotation).
Note: Exploratory factor loadings after all values below .30 were deleted. Significant loadings based on a corrected alpha of 0.05/ 87 are marked with an asterisk.
As a test for reliability again ICCs were computed. They varied between questionable (ICC[2, k]cooperation = .68, 95% CI [0.64, 0.74];), acceptable (ICC[2, k]assertion = .71, 95% CI [0.67, 0.76]), and good (ICC[2, k]self-control = .84, 95% CI [0.81, 0.86]).
Additional explorative analyses were conducted to find a factor solution with acceptable incremental fit indices. Excluding items with loadings below the threshold of .30 a solution with six factors emerged (χ2 = 282.55; df = 204; p < .01; TLI = .93; CFI = .96; RMSEA = .03 [90% CI = 0.020/0.040]). Whereas the scale “assertion and initiative” (7 items) remained almost unchanged compared with the three-factor solution the scale “cooperation” was divided into the two subscales “voluntary cooperation” (3 items; ICC[2, k]voluntary = .60, 95% CI [0.53, 0.67]) and “self-reliant cooperation” (3 items; ICC[2, k]self-reliant = .61, 95% CI [0.54, 0.67]). The rather heterogeneous scale “self-control” was divided into three subscales we called “emotion control” (2 items; ICC[2, k]emotion control = .67, 95% CI [0.60, 0.73]), “family conduct” (4 items; ICC[2, k]family conduct = .70, 95% CI [0.65, 0.75]), and “general conduct” (6 items; ICC[2, k]conduct = .66, 95% CI [0.61, 0.71]). All reliabilities can be judged as questionable.
Criterion Validity
To assess concurrent and divergent criterion validity of the SSRS scales we conducted bivariate analyses, which are summarized in Table 2.
Correlations Between Social Skills and Measures of Behavior and Temperament (Pearson’s r).
Note: *p < .05, **p < .01, ***p < .001, ns = not significant.
CBCL = Child Behavior Checklist; TABC = Temperament Assessment Battery for Children.
The modified subscales of the SSRS were diversely related to aspects of behavior and temperament. As expected, “cooperation” and “self-control” were negatively correlated with “aggression” and “negative emotionality,” meaning that high levels of individual cooperation and self-control go along with low levels of aggression and negative emotionality. “Assertion” was most clearly linked with “social inhibition” in the expected direction. Children rated as assertive show low levels of social inhibition.
As a further test of validity of the instrument age trends were analyzed. As expected, for two of the three subscales of the modified SSRS we found that older children were rated as more skilled than younger ones. There was a significant age effect for “assertion,” t(389) = −3.29, p < .01, and for “self-control,” t(389) = −3.72, p < .001. For the subscale “cooperation” however, we found no age effect, t(389) = −1.45, p = .15, not significant.
Discussion
The present study addressed the question, whether the Preschool Parent Version of the SSRS, developed by Gresham and Elliott (1990) for 3- to 4-year-olds, can be applied to a German preschool sample with 3- to 6-year-olds. The findings of the analysis concerning the replication of the original scales were mixed. On one hand, we failed to replicate the original factor structure of the SSRS. On the other hand—after some modifications—a factor structure similar to the original for three subscales was found.
To fit the German educational system with children visiting preschool from 3- to 6-years, the age range of the sample was extended compared with the original version of the preschool SSRS. Applying exploratory factor analysis with this extended sample, three of the resulting subscales were similar in content to those reported by Gresham and Elliott (1990). The exception was the “responsibility” subscale which could not be replicated at all. This part of our results is in line with recent results of studies from the Netherlands (Van der Oord et al., 2005) and the United States (Whiteside et al., 2007). Summing up, three subscales with a reduced item pool (31 items) compared with the original SSRS fitted the present data best. Although incremental fit indices are slightly below recommended values and internal consistencies are at the lower threshold of acceptability the resulting scales represent the different facets of social skills. Model fit and reliability may be sufficient considering the balance between statistical criteria and interpretational heterogeneity of the social skills construct. Additional analyses revealed that especially the scales “self-control” and “cooperation” can be object to further subdivision. We recommend using these more detailed subscales just in cases when differential measurement is necessary for research or practical purposes.
The modified subscales of the SSRS are only moderately related to each other suggesting that they measure different aspects of social skills. This heterogeneity is reflected in the results concerning the criterion validity of the SSRS. Whereas “self-control” and “cooperation” represent aspects of social skills dealing with socially accepted and appreciated behavior, “assertion” reflects the component of social skills dealing with getting into contact with others. Thus, this component is more strongly related to aspects of social inhibition and anxiety than other dimensions of social skills. Behavior like asking others for help or joining group activities reflecting the concept of assertion is not compatible with anxiety or social inhibition. Concerning age the expected differences favoring older children emerged in subscales “assertion” and “self-control” but not in the subscale “cooperation.” Taking a closer look at the items forming the subscales one can argue that “cooperation” include aspects of compliance toward parental demands, which can be expected to be present at earlier ages, whereas the other scales contain more demanding features of social interaction. In sum, our analyses indicate strong criterion validity for the modified subscales of the SSRS with a German preschool sample and point to the importance of considering different facets of social skills.
The present results indicate the need for a revision of Preschool Parent Form of the SSRS for a German preschool population ranging from 3- to 6-years of age. The need for a statistically adequate solution with respectable psychometric properties calls for a reduction and reorganization of the original item pool for the German version.
Due to some limitations, certain questions cannot be addressed with the present study. First, the present study was a single perspective study. It would have been helpful to have teacher ratings as well to see if there are similar results concerning the factorial structure of the preschool teacher version of the SSRS. Second, additional studies using German samples will be necessary to confirm our results. Third, a direct cultural comparison of results (USA–Germany) was not possible, because the sample only included German children.
Apart from the results concerning the factorial structure of the social skills part of the SSRS, some important questions concerning the development of social skills during the preschool years emerged which should be addressed in future studies. For example, referring to the differences in parent ratings depending on SES, a closer look should be taken at the role of socioeconomic disadvantage in the development of social skills and other personal resources (McLloyd, 1998). Despite the aforementioned limitations, the present study provides a useful first attempt to test the psychometric properties of the SSRS with a German preschool sample.
Footnotes
Acknowledgements
The BELLA study would not have been possible without the engagement of the members of the BELLA study group: Ulrike Ravens-Sieberer (Principal Investigator), Claus Barkmann, Susanne Bettge, Monika Bullinger, Manfred Döpfner, Michael Erhart, Beate Herpertz-Dahlmann, Heike Hölling, Franz Resch, Aribert Rothenberger, Michael Schulte-Markwort, Nora Wille, Hans-Ulrich Wittchen. Furthermore, we thank the numerous children and parents who participated in the study for their time and commitment.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The BELLA study was fully funded by the CD Foundation as part of the German Science Foundation.