The Case for the Perceived Social Competence Scale II

Abstract

Objective:

This study examines the psychometric properties of the revised Perceived Social Competence Scale (PSCS), a brief, user-friendly tool used to assess social competence among youth.

Method:

Confirmatory factor analyses (CFAs) examined the factor structure and invariance of an enhanced scale (PSCS-II), among a sample of 420 youth. Correlations between PSCS-II and self-reported social skill scores were examined. A longitudinal CFA tested the invariance of the factor structure over time with a different sample of 451 youth.

Results:

The revised 5-item PSCS-II demonstrated acceptable factorial validity, factorial invariance across time and gender as well as strong predictive validity.

Conclusion:

The PSCS-II was supported as a strengthened version of the PSCS to measure social competence in social work research and practice.

Keywords

social competence scale development youth development

Strategies to promote social competence continue to be a priority in prevention, youth development, education, and social work practice (Anderson-Butcher, Iachini, & Amorose, 2008; Bornstein, Hayn, & Haynes, 2010; Dirks, Treat, & Weersing, 2007; Obradovic & Hipwell, 2010). Social competence is defined as the degree to which youth engage in prosocial behaviors that allow them to successfully create and maintain positive social interactions with others (Gresham, 2002; Katz, McClellan, Fuller, & Walz, 1995; Masten & Coatsworth, 1998). Theories such as social cognitive theory and self-determination theory emphasize the importance of environmental influences on social development. Conceptual models of social competence have included social adjustment, social performance, and social skills (Cavell, 1990), for which social skills focus on the specific abilities and behaviors considered effective within social contexts (Dodge, Petit, McClasky, & Brown, 1986; Nangle, Grover, Holleb, Cassano, & Fales, 2010; Vahedi, Farrokhi, & Farajian, 2012). Further, the assessment of perceived social competence is increasingly important, as research consistently documents that many youth are lacking this important skill. In fact, estimates suggest that one in five children and adolescents in the United States are lacking in these valued skills and attributes (Blumberg, Carle, O’Connor, Moore, & Lippman, 2008), with greater levels of need existing among youth of color and from disadvantaged circumstances (Klebanoff & Muramatsu, 2002; Nowicki, 2003).

The long-term consequences of poor social competence can be quite serious. Often, deficits in social competence coincide with other emotional and behavioral problems, including attention deficits (Coleman, 2008; Nixon, 2001; Thorell & Rydell, 2008), aggression (Chen, Huang, Chang, Wang, & Li, 2010; Fraser et al., 2005; Frey, Hirschstein, & Guzzo, 2000), and depression (Gable & Shean, 2000; Williams & Galliher, 2006). The importance of social competence and sustaining social relationships is life long—it has been shown to help influence academic success (Chen et al., 2010; Welsh, Parke, Widaman, & O’Neil, 2001; Zins, Bloodworth, Weissberg, & Walberg, 2004), higher levels of self-esteem (Kostelnik, Gregory, Soderman, & Whiren, 2012), and lower levels of substance abuse (Griffin, Epstein, Botvin, & Spoth, 2001).

The valid assessment of social competence is crucial to understanding baseline deficits among youth as well as is important for examining outcomes associated with programs and interventions (Anderson-Butcher, Wade-Mdivanian, Paluta, Lower, Amorose, & Davis, 2014; Crowe, Beauchamp, Catroppa, & Anderson, 2011; Fetro, Rhodes, & Hey, 2012). Several tools are available for use in assessing social competence and related factors, including tools such as the Social Competence Scale of the Achenbach Child Behavior Checklist (Achenbach, 1991; Achenbach & Edelbrock, 1983), the Kohn Social Competence Scale (Kohn, & Rosman, 1974), and the social competence component of the Self-Perception Profile for Children (Harter, 1985). One user-friendly, brief measure is the Perceived Social Competence Scale (PSCS; Anderson-Butcher et al., 2008).

The PSCS is a self-report instrument comprised of 4 items assessing various elements of social competence. Strengths of the PSCS include its user-friendliness, ease of administration, use at no cost with the permission of the researchers, established reliability, and factorial and predictive validity (Corcoran & Fischer, 2013). Although the PSCS has been used in program evaluation studies and research (e.g., Anderson-Butcher, Wade-Mdivanian, Davis, Ruch, & Riley, 2013; Mertens & Wilson, 2012; Satici, Uysal, & Satici, 2013), the authors call for additional psychometric work to further demonstrate the tool’s effectiveness for research and practice (Anderson-Butcher et al., 2008, Anderson-Butcher, Amorose, Iachini, & Ball, 2013).

Needs exist in relation to strengthening the psychometric properties of the PSCS, as recent research has demonstrated that the original version of the PSCS has not necessarily functioned as expected. For instance, research has detected little growth over time in social competence among an economically disadvantaged youth sample involved in sport in the United States, which may be due to favorable perceptions of social competence among youth at baseline (Anderson-Butcher, Riley, Iachini, Wade-Mdivanian, Davis, & Amorose, 2013). In a different study, scores of the PSCS did not differentiate among high and low youth attenders in a program designed to target social competence in the United States (Anderson-Butcher et al., 2008), therefore, suggesting that the tool’s ability to tease apart group differences may be limited. Likewise, this same study found that the variance in scores among a sample of older youth was small (Anderson-Butcher et al., 2008). Older youth reported mostly favorable perceptions of their social competence, suggesting that the tool may not be sensitive to differences in perceptions among older youth. In the end, ceiling effects may allow for little variability in scores as well as leave little room for the documentation of growth over time. Given these limitations, further examination of the PSCS properties overall is warranted, especially in relation to strengthening the tool’s sensitivity to individual differences among respondents.

In addition to further examination of the psychometric properties of the PSCS, there is a need to test for factorial invariance over time, as examination of the stability of scores may help further inform the tool’s integrity and applicability as an outcome measure. In other words, if factorial invariance over time was established, the PSCS’s utility in measuring growth in social competence would be established. To date, the stability of respondent responses over time on the PSCS has not been tested. Additionally, Anderson-Butcher, Iachini, and Amorose (2008) recommend that future research should establish the convergent validity of the PSCS by determining how scores relate to other measures of social competence and related skills. Establishing that children respond comparably on similar tools measuring comparable constructs would demonstrate further support for the PSCS’s validity.

As such, the purpose of this research is to expand upon initial psychometric work related to the development of the PSCS. Building from the original 4-item PSCS, new items are added to the instrument and the response scale is modified to allow for more variance in responses. Analyses explore the reliability and factor structure of the strengthened tool. Then, the revised scale is tested for factorial and predictive validity. Additionally, the new tool is tested using longitudinal factor analysis, therefore, further exploring the tool’s stability over time, factorial invariance, and overall utility in practice and research.

Method

Participants

Data were collected from two samples of children and youth who participated in a sports-based positive youth development summer program offered on a university campus in the United States. The summer program consists of a 19-day camp designed to enhance the social and athletic competence of economically disadvantaged youth through play-based education curriculum and instructional sport activities. The program incorporates positive youth development best practices, including the use of a curriculum, staff training on positive youth development, and intentional programming designed around the anticipated outcomes. Participation in the summer program is voluntary and free of charge. For a full description of the program, see Riley and Anderson-Butcher (2012).

The participants in Sample 1 included 420 youth who responded to all items on the PSCS-II and completed other related constructs. These youth ranged in age from 9 to 15 years (M = 11.62 years, SD = 1.43) and included 253 males (60.2%) and 167 females (39.8%). The majority of the participants identified themselves as African American/Black (77.6%), mixed/multi-racial (8.8%), White/non-Hispanic (4.0%), Hispanic (2.4%), Native American (2.4%), or Asian/Pacific Islander (0.7%).

Sample 2 was comprised of 451 youth who completed the PSCS-II on three separate occasions during a 4-week youth development camp (precamp, midcamp, and postcamp). The participants ranged in age from 8 to 15 years (M = 11.48 years, SD = 1.56) and included both males (58.4%) and females (41.6%). The majority of participants were African American/Black (84.7%), mixed/multi-racial (8.2%), or White/non-Hispanic (3.5%), with all other racial/ethnic groups representing less than 1% of the sample.

Measures

PSCS-II

The original PSCS (Anderson-Butcher et al., 2008) consists of 4 items designed to measure the social competence of youth. Responses to the items were made on a 5-point Likert-type scale (1 = Not at All; 2 = A Little; 3 = Some; 4 = A Lot; and 5 = Very Much). Items include (1) I help other people, (2) I ask others if I can be of help, (3) I am good at making friends, and (4) I get along well with others. Four new items were developed by the original authors of the PSCS to further expand upon the constructs measured on the PSCS. Items were created upon examining theory and research in social competence and positive youth development. Additionally, a content expert (with over 10 years of experience in positive youth development) provided support for the new items content and face validity. The additional 4 items include (5) I show concern for others, (6) I am a good friend, (7) I show care for others, and (8) I give support to others. The response scale was also modified to be more explicit, by adding the word “true” to each response option. Specifically, youth were asked to indicate “how true” each statement was on a 5-point Likert-type scale (1 = Not at all true, 2 = A little true, 3 = Somewhat true, 4 = Pretty true, and 5 = Really true). In the end, the initial revised PSCS-II consisted of the 4 original PSCS items and the 4 new items (total of 8 items).

Social Skills Improvement System

To explore the predictive validity of the PSCS-II, youth perceptions of social skills were measured using the Social Skills Improvement System (SSIS; Greshem & Elliot, 2008). Sample 1 completed the SSIS measure in addition to the PSCS-II. The SSIS instrument is a revision of the Social Skills Rating System (Gresham & Elliot, 1990) and includes seven social skill domains, including assertion, communication, cooperation, empathy, engagement, responsibility, and self-control (Gresham & Elliot, 2008). The instrument consists of 46 items, approximately 6 or 7 items per social skill domain. The youth responded using a 4-point Likert-type scale, indicating how true they think each statement is of them (1 = not true; 4 = very true). The SSIS subscales have demonstrated acceptable reliability for the youth version (α = .72–.83; Gresham & Elliot, 2008). The SSIS tool was selected for the purpose of assessing predictive validity of the PSCS-II, based upon its youth focus, widespread usage in a variety of contexts, and establishment as a prominent social skill instrument (Frey, Elliott, & Gresham, 2011; Gould, Dixon, Najdowski, Smith, & Tarbox, 2011; McIntosh & MacKay, 2008; Wigelsworth, Humphrey, Kalambouka, & Lendrum, 2010).

Procedures

Data collection for the two samples occurred as part of a multiyear study designed to assess youth outcomes associated with participation in a sport-based positive youth development summer program. Data for Sample 1 were collected during the 2012 summer program, while data for Sample 2 were collected the following summer. Please note that both studies were approved by the institutional review board of The Ohio State University. As youth (aged 9–15) can register for the summer program multiple years in a row, 36.5% of the campers in 2013 were found to have participated in previous camps. As such, some youth may have been involved in both years of data collection.

Youth were recruited to participate in the summer program through various community programs and avenues. During registration for the summer program, parents/guardians were informed of the research and asked to consent to their child’s participation. Assent was also attained from youth aged 14 years or older. Participation in the summer program was not contingent upon research consent and/or assent. Youth enrolled in the study completed a hard copy survey, comprised of a large battery of instruments, in a classroom on the university’s campus, administrated by an institutional review board–approved researcher. Data were collected at 2 separate times in 2012 (i.e., precamp and postcamp) and 3 separate times in 2013 (i.e., precamp, midcamp, and postcamp). Each youth was assigned an identification number in order to connect the longitudinal data, upon which all personal identifiers were removed. Completion of the entire battery of instruments took approximately 30 min, whereas completion of the instruments used in this study took about 5 min. The additional constructs included in the full survey measured self-perceptions of related social and personal skills, such as effort, teamwork, and self-control.

In order to enhance participation and decrease missing data, the instrument was administered in multiple sessions to youth who did not fully complete the instrument or were absent during data collection. Missing data were collected up to 3 days past the original data collection periods. Prior to importing the data to LISREL 8.81, patterns of missing data were assessed through SPSS Statistics 21 software, for which the majority of missing data were found to be due to attrition. As such, only complete cases were used for data analysis in order to test the stability of the PSCS-II factor structure over time. Normality of the observed variables was assessed through univariate and multivariate skewness and kurtosis statistics prior to data analysis.

Data Analysis

A number of confirmatory factor analyses (CFAs) were conducted using LISREL 8.81 to provide validity evidence for the PSCS-II. First, we tested the overall factor structure and gender invariance of the revised scale using a multigroup CFA with the data from Sample 1. Next, we conducted a longitudinal CFA to test the invariance of the factor structure over time with the data from Sample 2.

In all model testing, we relied on multiple fit indices to evaluate the adequacy of the estimated models. A good fit of a model was defined by the following: nonsignificant χ² at p < .05, root mean square error of approximation (RMSEA) ≤ .06, comparative fit index (CFI) ≥ .95, and Tucker–Lewis index (TLI) ≥ .95 (Hu & Bentler, 1999). We also examined the modification indices to determine whether any local areas of strain were affecting the acceptability of the models.

When evaluating the adequacy of specific invariance constraints, we examined changes in CFI and the results of χ² difference tests between the progressively constrained and baseline models. A change in CFI ≤ .01 (Cheung & Rensvold, 2002) and a nonsignificant reduction in χ² served as our guidelines for determining the tenability of the proposed invariance constraints.

There is considerable variability in the nomenclatural and suggested sequence of steps involved in testing for measurement invariance across groups and/or time (Little, 2013; Vandenberg & Lance, 2000). We generally followed the steps and procedures recommended by Brown (2006) and Little (2013) to conduct the analyses. When looking at gender invariance and the overall factor structure of the PSCS-II in Sample 1, we began by conducting a basic CFA model separately for males and females. In both cases, each of the 8 scale items was specified to load on a single latent construct and the factor loading of one of the items, “I show care for others,” was set equal to one to establish a metric for the latent variable. Also, the factor variance was freely estimated as was the uniqueness for each item. No covariances between item uniquenesses were modeled.

The next three models tested imposed specific invariance constraints across groups as a way to explore the measurement properties of the scale. Specifically, we tested for (1) configural invariance (i.e., equal form/factor structure), (2) weak invariance (i.e., equality of factor loadings), followed by (3) strong invariance (i.e., equality of indicator intercepts). Establishing strong invariance across groups (and time with Sample 2) was our desired outcome, as this level of measurement invariance provides reasonable evidence that any observed group differences and/or changes over time are based on true construct differences rather than measurement artifacts (Little, 2013).

Following the establishment of strong measurement invariance, we tested for differences in the latent construct parameters. Specifically, we tested for differences between groups in the factor variance and in the factor mean. This was accomplished by constraining groups to be equal on each of these parameters and comparing the model fit to the strong invariance model (see Little, 2013). These results do not have implications for evaluating the factorial validity of the measure but rather provide descriptive information about the scores obtained in our data.

The same basic sequence of steps was used to test the invariance of the measure across time with the data from Sample 2. Again, this involved testing and comparing the progressively restrictive measurement models (configural, weak, and strong invariance) followed by a test of the latent construct parameters (examining similarities and differences in latent variances and means). In these models, the individual scale items collected at a specific time point (i.e., precamp, midcamp, and postcamp) were specified to load on a single latent factor representing perceived social competence at that respective time point. The factor loading for one of the items at each time point was set equal to one to establish a metric for the latent variables. This fixed loading was identical across all three time points. The latent factors were permitted to covary given they represent the same latent construct separated only by time. The factor variance at each time point was freely estimated as was the uniqueness for each item. Given that the same items are used at all three time points, it was reasonable to assume that the item-specific variance would be related across successive measurement occasions (Little, 2013). Therefore, the uniquenesses of corresponding items were allowed to covary overtime.

In addition to factor analyses and invariance testing, correlations between the PSCS-II score and SSIS social skills score as well as domain scores (e.g., communication, responsibility, self-control) were run to examine the predictive validity of the PSCS-II. A mean summated score was calculated for the PSCS-II, SSIS, and SSIS subscales in order to run the correlations. The SSIS was used to establish predictive validity, as it is a well-accepted and comprehensive social skills tool that measures a construct closely aligned with social competence (Basca, 2002). The instrument has consistently been identified as the most psychometrically sound tool assessing social skills (Basca, 2002). Significant positive relationships among scores on the PSCS-II and scores on the SSIS and its subscales were considered evidence of predictive validity, as it would be expected that scores on similar, yet related measures would be comparable.

Results

Preliminary Analyses

The distributional assumptions of the data were tested using PRELIS 2.20 (Scientific Software International, Inc., Chicago). The univariate skewness and kurtosis values for Sample 1 ranged from −0.40 to −2.08 and from −0.70 to 4.63, respectively. The univariate skewness and kurtosis values for Sample 2 ranged from −0.17 to −1.30 and from −0.89 to 1.73, respectively. Although most of the values suggested a reasonable degree of normality, the tests for multivariate skewness and multivariate kurtosis were significant in both samples (p < .01). Nevertheless, we chose to employ maximum likelihood estimation procedures, given this approach has demonstrated the ability to withstand departure from normality (see Chou & Bentler, 1995).

Main Analyses

Overall factor structure and gender invariance—Sample 1

To test the overall factor structure and gender invariance of the PSCS-II, we conducted a separate CFA for males and females.

Testing the factor structure with males

The first analysis was conducted with the male participants (n = 253). The overall fit of this model to the data was less than ideal, χ² = 81.95, df = 20, p = .00, RMSEA = .08 (90% confidence interval [CI] = [.05, .10]), CFI = .98, TLI = .97. The significant χ² and RMSEA values above the suggested criterion indicated the model did not fit the data well. Examination of the modification indices suggested that Items 3 and 6 were problematic. We therefore deleted these items and conducted the CFA again. The fit of the 6-item version of scale was relatively good, χ² = 13.43, df = 9, p = .14, RMSEA = .00 (90% CI = [.00, .07]), CFI = 1.00, TLI = .99. Based on the reduction in the RMSEA value and the emergence of a nonsignificant χ² value, we determined this version of the PSCS-II had stronger factorial validity than the 8-item version. However, an examination of the squared multiple correlations (SMCs) indicated that Item 4 was a relatively poor indicator (SMC = .29) of the latent social competence construct. Thus, we deleted this item and ran the CFA 1 more time. The 5-item version of the scale fits reasonably well, χ² = 11.30, df = 5, p = .05, RMSEA = .04 (90% CI = [.00, .11]), CFI = 1.00, TLI = .99, and no further areas of local strain were apparent. Each of the items positively and significantly (p < .05) loaded on the latent social competence factor, with completely standardized coefficients ranging from .68 to .82 (M = .75, SD = 0.05) and SMCs averaging .56 (SD = 0.09).

Testing the factor structure with females

Next, we conducted the CFA using just the female participants from Sample 1 (n = 167). Interestingly, the same general pattern of results emerged. Although the overall fit of the model testing the 8-item version of the scale was good, χ² = 25.63, df = 20, p = .18, RMSEA = .00 (90% CI = [.00, .06]), CFI = 1.00, TLI = 1.00, the SMCs for Items 3, 4, and 6 were all low (SMC = .06, .33, and .33, respectively). We therefore deleted these items and conducted the CFA again. The fit of the 5-item version of scale was also quite good, χ² = 2.28, df = 5, p = .81, RMSEA = .00 (90% CI = [.00, .04]), CFI = 1.00, TLI = 1.02, and no further areas of local strain were apparent. All 5 items positively and significantly (p < .05) loaded on the latent social competence factor, with completely standardized coefficients ranging from .68 to .74 (M = .72, SD = 0.02) and SMCs averaging .52 (SD = 0.03).

Testing for gender invariance

Having settled on moving forward with a 5-item version of the measure, which demonstrated reasonably good factorial validity for males and females separately, our next step was to specifically test for gender invariance on the scale scores. The results of this testing is presented in Table 1. Based on the recommendations of Widaman and Thompson (2003), we first specified and tested an acceptable null model which was used in calculating the incremental fit indices (i.e., CFI and TLI). Next, we began testing and comparing models specifying configural invariance (i.e., equal form/factor structure), weak invariance (i.e., equality of factor loadings), and finally strong invariance (i.e., equality of indicator intercepts). In all 3 cases, the overall fit of the model to the data was acceptable (e.g., nonsignificant χ² values, RMSEA ≤ .06; CFI and TLI ≥ .95). Furthermore, our comparison of the increasingly restrictive models supported the tenability of the proposed invariance constraints with each comparison showing a change in CFI of ≤ .01 and a nonsignificant reduction in χ². Thus, combined with the overall goodness of fit of the strong invariance model (see Table 1), these results provide solid evidence that the 5-item PSCS-II functions similarly for males and females. Table 2 shows the key parameter estimates for the strong invariance model.

Table 1.

Model Fit Statistics for the Tests of Gender Invariance in Social Competence Scores.

Model Tested	χ²	df	p	Δχ²	Δdf	p	RMSEA (90% CI)	CFI	ΔCFI	TLI	ΔTLI	Pass?
Null model	919.342	29	<.001	—	—	—	—	—	—	—	—	—
Measurement model estimates
Configural invariance	13.511	10	<.193	—	—	—	.000 [.000, .067]	.997	—	.992	—	Yes
Weak invariance	17.576	14	<.227	3.999	4	<.406	.000 [.000, .061]	.997	.000	.994	.002	Yes
Strong invariance	26.405	18	<.091	8.829	4	<.066	.024 [.000, .068]	.994	.004	.990	.004	Yes
Latent model estimates
Factor variance	28.366	19	<.077	1.961	1	<.161	.026 [.000, .068]	.989	.001	.984	.001	Yes
Factor mean	47.447	19	<.001	21.042	1	<.001	.064 [.029, .097]	.968	.023	.951	.034	No

Note. RMSEA = root mean square error of approximation; CI = confidence interval; CFI = comparative fit index; TLI = Tucker-Lewis index; Pass? = outcome of decision on whether model was deemed acceptable.

Table 2.

Parameter Estimates From Strong Invariance Model—Gender.

			Freely Estimated Parameters
	Parameters Constrained to be Invariant		Males (n = 253)		Females (n = 167)
Item	FL	INT	UNQ	SMC	UNQ	SMC
1. I help other people	.662	.656	.499	.444	.528	.517
2. I ask others if I can be of help	.743	−.112	.566	.429	.520	.466
3. I show concern for others	.890	−.789	.366	.552	.427	.645
4. I show care for others	.794	—	.348	.577	.399	.664
5. I give support to others	.706	.501	.550	.524	.400	.503
Factor variance (SD)			.687 (.829)		.543 (.737)
Factor mean			3.764		4.155

Note. FL = item loading; UNQ = item uniqueness; SMC = squared multiple correlation; INT = item intercept. Item loadings and item uniquenesses are based on Common Metric Standardized Parameter Estimates. -- indicates the value is fixed to set the scale of the construct’s parameter estimates.

Examining gender differences in scale scores

For descriptive purposes, we were interested in comparing males and females on the latent model estimates of the PSCS-II. This set of analyses has no direct bearing on the establishment of acceptable factorial validity of the PSCS-II; rather, we felt it was important to provide descriptive information about the scores from the revised measure. The model fit statistics relevant to the latent scale scores are presented in Table 1 and parameter estimates in Table 2. As shown in Table 2, the estimated factor mean was 3.76 (SD = 0.83) for males and 4.16 (SD = 0.74) for females. As reported in Table 1, constraining the variance to be equal across groups was reasonable, given that the change in CFI was ≤ .01, and there was a nonsignificant reduction in χ² in comparison to the strong invariance model. The test of the factor means, on the other hand, did not support the tenability of the invariance constraint. Constraining the means to be equal resulted in a change in CFI of .23, and there was a significant change in χ² in comparison to the strong invariance model (see Table 1). These results indicate that males and females were significantly different, with females scoring higher than males. The effect size of this difference was medium to large in magnitude with a calculated Latent Cohen d = .72 (see Little, 2013).

Longitudinal Invariance Testing—Sample 2

The next set of analyses was designed to provide evidence that the measurement properties of the PSCS-II remained constant over time using data from Sample 2. All testing were done on the 5-item version of the scale that emerged from the gender invariance testing, as this was the version of the scale that demonstrated acceptable factorial validity. As before, we tested and compared progressively restrictive measurement models (configural, weak, and strong invariance) and then tested and compared latent construct parameters (examining similarities and differences in latent variances and means). Males and females were pooled together for all model testing.

Testing invariance over time

The model fit statistics for the invariance testing across the three time points (precamp, midcamp, and postcamp) are presented in Table 3, with the key parameter estimates for the strong invariance model provided in Table 4. Once again, the results from the acceptable null model (Widman & Thompson, 2003) were used in calculating the incremental fit indices (also see Little, 2013). Although the χ² value for each model was significant (p > .05), overall the fit of the configural, weak, and strong invariance model were shown to be acceptable with RMSEA values ≤ .06 and CFI and TLI values ≥ .95. The comparison of each of the increasingly restrictive measurement models provided at least some support for the tenability of the invariance constraints. Specifically, there was a nonsignificant reduction in χ² and a change in CFI of ≤ .01 when moving from configural to weak invariance. In other words, the results provide solid evidence that the factor loadings remained the same across the three time points. The support for equal item intercepts was a bit weaker, although still reasonable. The change in χ² value was significant when comparing the strong invariance to the weak invariance model; however, the change in CFI was ≤.01. Thus, we concluded there was reasonable support for strong invariance of the measure over time.

Table 3.

Model Fit Statistics for the Tests of Invariance in Social Competence Scores Across Three Time Points.

Model Tested	χ²	df	p	Δχ²	Δdf	P	RMSEA (90% CI)	CFI	ΔCFI	TLI	ΔTLI	Pass?
Null model	3616.161	123	<.001	—	—	—	—	—	—	—	—	—
Measurement model estimates
Configural invariance	173.982	72	<.001	—	—	—	.040 [.028, .052]	.971	—	.950	—	Yes
Weak invariance	183.395	80	<.001	9.413	8	<.309	.039 [.028, .050]	.970	.000	.955	.004	Yes
Strong invariance	217.133	88	<.001	33.738	8	<.001	.044 [.034, .055]	.963	.007	.948	.006	Yes
Latent model estimates
Factor variance	222.175	90	<.001	5.042	2	<.080	.044 [.034, .054]	.962	.001	.948	.000	Yes
Factor mean	308.034	90	<.001	90.901	2	<.001	.061 [.052, .070]	.938	.026	.915	.034	No

Table 4.

Parameter Estimates From Strong Invariance Model—Time.

	Precamp			Midcamp			Postcamp			INT
	FL	UNQ	SMC	FL	UNQ	SMC	FL	UNQ	SMC	INT
I help other people	.669	.552	.448	.770	.407	.593	.760	.423	.577	.094
I ask others if I can be of help	.575	.669	.331	.686	.530	.470	.676	.543	.457	−.608
I show concern for others	.683	.533	.467	.770	.407	.593	.741	.451	.549	−.507
I show care for others	.737	.456	.544	.797	.365	.635	.789	.377	.623	—
I give support to others	.665	.557	.443	.783	.388	.612	.802	.357	.643	.120
Factor variance (SD)	.474 (.689)			.571 (.756)			.515 (.718)
Factor mean	3.919			4.083			4.286

Note. FL = item loading; UNQ = item uniqueness; SMC = squared multiple correlation; INT = item intercept. Item loadings and item intercepts were constrained to be equal across time. Item loadings and item uniquenesses are based on completely standardized parameter estimates. -- Indicates the value is fixed to set the scale of the construct’s parameter estimates.

Examining changes in scale scores over time

The remaining analyses focused on exploring similarities and differences in the latent construct parameters—namely, the latent variances and means. These analyses are not necessarily relevant to establishing acceptable factorial validity of the PSCS-II but instead provide relevant descriptive information about the scale scores. As shown in Table 4, the estimated factor means were 3.92 (SD = 0.69), 4.08 (SD = 0.76), and 4.29 (SD = 0.72) for precamp, midcamp, and postcamp, respectively. As reported in Table 3, constraining the variance to be equal across the three time points was reasonable, given that the change in CFI was ≤ .01, and there was a nonsignificant reduction in χ² in comparison to the strong invariance model. Thus, the variability of the factor scores on the PSCS-II appears to remain constant over time. However, the test of the factor means suggested that the scores on the 5-item PCSC-II changed over time. Constraining the means to be equal resulted in a change in CFI of .26, and there was a significant change in χ² in comparison to the strong invariance model (see Table 3). The effect sizes of the differences were in the small to medium range with calculated Latent Cohen’s d values = .22, .29, and .53 for the precamp versus midcamp, midcamp versus postcamp, and precamp versus postcamp, respectively.

Predictive Validity

To establish predictive validity of the 5-item PSCS-II, Sample 1 was used to examine correlations between the PSCS-II and SSIS, using mean summated scores of the measures. The SSIS tool can be divided into seven domains (i.e., assertion, communication, cooperation, empathy, engagement, responsibility, and self-control). The mean summated score for the comprehensive measure as well as seven subscales was utilized to examine predictive validity. Using the 5-item PSCS-II (M = 3.79, SD = 0.85, range = 1–5) and SSIS overall scale (M = 96.50, SD = 19.60, range = 75–300) from Sample 1, social competence and overall social skills were correlated and found to have a strong positive relationship (r = .597) at the p < .01 level of significance. The correlations between social competence and the SSIS subscales were also significant (p < .001) and positive, ranging from r = .364 to .644. The correlations between the 5-item PSCS-II and the various SSIS scales support the predictive validity of the proposed social competence tool.

Discussion and Applications to Social Work

The purpose of this study was to revisit and enhance the psychometric properties of the original PSCS. The results of the study support a 5-item scale, called the PSCS-II, which demonstrated acceptable psychometric properties. Tests of factorial invariance suggest that this measure operates similarly for males and females, with females reporting significantly more favorable perceptions than males. In addition, the longitudinal factor analysis provided evidence for invariance across time, for which the measurement properties, factor loadings, and variability of factor scores remained constant across three time points. This suggests that changes in social competence over time, as measured by the PSCS-II, are actually due to a change in the latent variable of interest, as opposed to a change in the way participants are responding to scale items (see Little, 2013; McArdle & Nesselroade, 1994). The PSCS-II also demonstrated good predictive validity, correlating positively with various subscales on the SSIS. Overall, the revised scale appears to be a solid tool to assess perceptions of social competence among youth. More specifically, the tool is brief yet still sensitive enough to differentiate between groups as well as stable enough to be useful in assessing growth in perceptions over time. In fact, early pilot work on the new tool has demonstrated its utility in measuring pre- to postchanges in youth development research (Anderson-Butcher et al., 2013). The limitation, however, is that the PSCS-II is not context specific and may not adequately measure trait and behaviors of interest to researchers in this area. More expansive conceptual models of social competence also include constructs such as social adjustment and performance (Cavell, 1990), and therefore the PSCS-II’s brevity may limit its utility.

Assessing social competence and related outcomes among youth is central to social work practice. Social work interventions designed to prevent problem behaviors and promote prosocial ones are common in youth development organizations, schools, faith-based organizations, and other social settings. The modified PSCS-II can help social workers, educators, youth development professionals, and others better assess youth perceptions of social competence and target their strategies to address key items measured in the tool. More specifically, the PSCS-II may be used to determine baseline levels of perceived social competence, thus allowing for the identification of those at risk for poor social skills and of who might be in need of targeted interventions. As such, social workers and other practitioners might have youth complete the PSCS-II in order to determine needs among individuals and groups and in turn design interventions to address limited competencies. Specific scores on individual items may also be examined (i.e., I show care for others; I ask others if I can be of help) to provide direction on specific strategies used to support targeted children and/or groups of young people.

The PSCS-II also is a useful tool that may be used to track progress over time from pre- to postintervention, therefore serving as an important resource for measuring change and perceived growth. Social workers and other practitioners might use the tool to evaluate potential growth in skills among the youth they are working with in practice, thus demonstrating successes (or lack thereof) of specific social work interventions. Further, the PSCS-II is useful when used among large groups of youth, as aggregate data could inform universal interventions aimed at promoting social competence and selective interventions for targeted groups. For instance, social workers in schools might measure social competence among entire student populations in order to examine school-wide, grade-level, gender-specific needs. These data collectively can inform school-wide improvement planning efforts to promote positive social skills and prosocial behaviors among aggregate groups of youth. Given that social competence is related to a host of positive youth development outcomes, data that inform programming to improve this construct may be useful in supporting broader youth outcomes. As such, the PSCS-II serves as a reliable, valid, and easily administered public domain tool for practice and research.

Although the PSCS-II is a useful measurement tool, there are some limitations to the tool’s applicability. First, the PSCS-II measures youths’ self-reported perceptions of social competence only. Although the use of youth self-report is supported (Danielson & Phelps, 2003), parent/guardian and/or teacher/practitioner reports also are valuable because they provide an outsider perspective. It should be noted, however, that reporting biases may also result when adult ratings are used (Danielson & Phelps, 2003; Youngstrom, Loeber, & Stouthamer-Loeber, 2000), and perhaps a combination of approaches is optimal.

Second, the PSCS-II is also purposefully brief. Although its brevity conveys advantages, the constructs measured are quite complex (Lee & Smith, 1999; Shouse, 1996). This complexity may call for more comprehensive assessments of children and experiences and perceptions of youths, particularly in studies that aim to examine the multiple facets of interpersonal relationships in detail. Other variables related to social competence (such as social responsibility) are not assessed. The external validity of this tool is also not fully established. Further research is needed to confirm the applicability of the PSCS-II in other developmental settings, especially in real-life social work settings to ensure its overall sensitivity and applicability. In addition, future work on the PSCS-II might also explore larger samples of youth from diverse settings, which may in turn allow for the establishment of normative data and/or cutoff scores identifying levels of risk. Similarly, tests of invariance highlight gender differences, another area of future research that might be explored in more detail.

In the end, this study modified the original PSCS, refining the measure and in turn producing a revised tool with strong psychometric properties. The resultant PSCS-II represents one brief, easily administered, user-friendly, public domain, self-report tool that measures youth perceptions of social competence, an important protective factor related to positive youth outcomes. Social workers and others can use the PSCS-II within their practice to assess baseline needs and measure perceived growth.

Footnotes

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

References

Achenbach

T. M.

(1991). Integrative guide for the 1991 CBCL/4-18, YSR, and TRF profiles. Burlington: University of Vermont Department of Psychiatry.

Achenbach

T. M.

Edelbrock

(1983). Manual for the child behavior checklist and revised child behavior profile. Burlington: University of Vermont.

Anderson-Butcher

Amorose

A. J.

Iachini

Ball

(2013). Community and youth collaborative initiative school experience surveys: Social skills scale in elementary schools. Columbus, OH: College of Social Work, The Ohio State University.

Anderson-Butcher

Iachini

Amorose

(2008). Initial reliability and validity of the perceived social competence scale. Research on Social Work Practice, 18, 47–54.

Anderson-Butcher

Riley

Iachini

Wade-Mdivanian

Davis

Amorose

(2013). Exploring the impact of a summer sport-based youth development program. Evaluation and Program Planning, 37, 64–69.

Anderson-Butcher

Wade-Mdivanian

Davis

Ruch

Riley

(2013). OSU LiFE Sports 2012 annual report. Columbus, OH: College of Social Work. The Ohio State University.

Anderson-Butcher

Wade-Mdivanian

Paluta

Lower

Amorose

Davis

(2014). OSU LiFE Sports 2013 annual report. Columbus, OH: College of Social Work. The Ohio State University.

Basca

(2002). Literature review of youth development/asset tools. Burbank, CA: EMT Group.

Blumberg

S. J.

Carle

A. C.

O’Connor

K. S.

Moore

Lippman

L. H.

(2008). Social competence: Development of an indicator for children and adolescents. Child Indicators Research, 1, 17–50.

10.

Bornstein

M. H.

Hahn

Haynes

M. O.

(2010). Social competence, externalizing, and internalizing behavioral adjustment from early childhood through early adolescence: Developmental cascades. Development and Psychopathology, 22, 717–735.

11.

Brown

T. A.

(2006). Confirmatory factor analysis for applied research. New York, NY: Guildford Press.

12.

Cavell

T. A.

(1990). Social adjustment, social performance, and social skills: A tri-component model of social competence. Journal of Clinical Child Psychology, 19, 111–122.

13.

Chen

Huang

Chang

Wang

(2010). Aggression, social competence, and academic achievement in Chinese children: A 5-year longitudinal study. Development and Psychopathology, 22, 583–592.

14.

Cheung

G. W.

Rensvold

R. B.

(2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9, 233–255.

15.

Chou

Bentler

P. M.

(1995). Estimates and tests in structural equation modeling. In Holye

R. H.

(Ed.), Structural equation modeling: Concepts, issues, and applications (pp. 37–55). Thousand Oaks, CA: Sage.

16.

Coleman

W. L.

(2008). Social competence and friendship formation in adolescents with attention-deficit/hyperactivity disorder. Adolescent Medicine: State of Art Review, 19, 278–299.

17.

Corcoran

Fischer

(2013). Measures for clinical practice and research: Couples, families and children (5th ed., Vol. 1). New York, NY: Oxford University Press.

18.

Crowe

L. M.

Beauchamp

M. H.

Catroppa

Anderson

(2011). Social function assessment tools for children and adolescents: A systematic review 1989-2011. Journal of Neurotrauma, 29, 1277–1291.

19.

Danielson

C. K.

Phelps

C. R.

(2003). The assessment of children’s social skills through self report: A potential screening instrument for classroom use. Measurement and Evaluation in Counseling Development, 35, 218–229.

20.

Dirks

M. A.

Treat

T. A.

Weersing

V. R.

(2007). Integrating theoretical, measurement, and intervention models of youth social competence. Clinical Psychology Review, 27, 327–347.

21.

Dodge

K. A.

Pettit

G. S.

McClaskey

C. L.

Brown

M. M.

(1986). Social competence in children. Monographs of the Society for Research in Child Development, 51, 2.

22.

Fetro

J. V.

Rhodes

D. L.

Hey

D. W.

(2012). Perceived personal and social competence: Development of valid and reliable measures. The Health Educator, 42, 19–26.

23.

Fraser

M. W.

Galinsky

M. J.

Smokowski

P. R.

Day

S. T.

Terzian

M. A.

Rose

R. A.

Guo

(2005). Social information-processing skills training to promote social competence and prevent aggressive behavior in the third grades. Journal of Consulting and Clinical Psychology, 73, 1045–1055.

24.

Frey

J. R.

Elliott

S. N.

Gresham

F. M.

(2011). Preschoolers’ social skills: Advances in assessment for intervention using social behavior ratings. School Mental Health, 3, 179–190.

25.

Frey

K. S.

Hirschstein

M. K.

Guzzo

B. A.

(2000). Second step: Preventing aggression by promoting social competence. Journal of Emotional & Behavioral Disorders, 8, 102–112.

26.

Gable

S. L.

Shean

G. D.

(2000). Perceived social competence and depression. Journal of Social and Personal Relationships, 17, 139–150.

27.

Gould

Dixon

D. R.

Najdowski

A. C.

Smith

M. N.

Tarbox

(2011). A review of assessments for determining the content of early intensive behavioral intervention programs for autism spectrum disorders. Research in Autism Spectrum Disorders, 5, 990–1002.

28.

Gresham

F. M.

(2002). Best practices in social skills training. In Thomas

Grimes

(Eds.), Best practices in school psychology (4th ed., Vol. 2, pp. 1029–1040). Bethesda, MD: National Association of School Psychologists.

29.

Gresham

F. M.

Elliott

S. N.

(1990). Social skills rating system. Circle Pines, MN: American Guidance Service.

30.

Gresham

F.M.

Elliott

S.N.

(2008). Social Skills Improvement System Rating Scales. Minneapolis, MN: NCS Pearson.

31.

Griffin

K. W.

Epstein

J. A.

Botvin

G. J.

Spoth

R. L.

(2001). Social competence and substance use among rural youth: Mediating role of social benefit expectancies of use. Journal of Youth and Adolescence, 30, 485–498.

32.

Harter

(1985). Manual for the self-perception profile for children. Denver, CO: University of Denver.

33.

Bentler

K. A.

(1999). Cutoff criteria for fit indices in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1–55.

34.

Katz

L. G.

McClellan

D. E.

Fuller

J. O.

Walz

G. R.

(1995). Building social competence in children: A practical handbook for counselors, psychologist, and teachers. ERIC Elementary and Early Childhood Education Clearinghouse. Washington, DC: U.S. Department of Education.

35.

Klebanoff

Muramatsu

(2002). A community-based physical education and activity intervention for African American preadolescent girls: A strategy to reduce racial disparities in health. Health Promotion Practice, 3, 276–285.

36.

Kohn

Rosman

B. L. A.

(1974). Social competence scale and symptom checklist: For the preschool child: Factor dimensions, their cross-instrument generality, and longitudinal persistence. Developmental Psychology, 6, 430–444.

37.

Kostelnik

M. J.

Gregory

K. M.

Soderman

A. K.

Whiren

A. P.

(2012). Guiding children’s social development and learning (7th ed.). Belmont, CA: Wadsworth.

38.

Lee

Smith

(1999). Social support and achievement for young adolescents in Chicago: The role of school academic press. American Educational Research Journal, 36, 907–945.

39.

Little

T. D.

(2013). Longitudinal structural equation modeling. New York, NY: Guilford Press.

40.

Masten

A. S.

Coatsworth

J. D.

(1998). The development of competence in favorable and unfavorable environments: Lessons from successful children. American Psychologist, 53, 205–220.

41.

McArdle

J. J.

Nesselroade

J. R.

(1994). Using multivariate data to structure developmental change. In Cohen

Reese

(Eds.), Life-span developmental psychology (pp. 223–267). Hillsdale, NJ: Lawrence Erlbaum.

42.

McIntosh

MacKay

L. D.

(2008). Enhancing generalization of social skills: Making social skills curricula effective after the lesson. Beyond Behavior, 18, 17–24.

43.

Mertens

D. A.

Wilson

A. T.

(2012). Program evaluation and practice: A comprehensive guide. New York, NY: Guilford Press.

44.

Nangle

D. W.

Grover

R. L.

Holleb

L. J.

Cassano

Fales

(2010). Defining competence and identifying target skills. In Nangle

D. W.

Hansen

D. J.

Erdley

C. A.

Norton

P. J.

(Eds.), Practitioner’s guide to empirically based measures of social skills (pp. 3–20). New York, NY: Springer.

45.

Nixon

(2001). The social competence of children with Attention Deficit Hyperactivity Disorder: A review of the literature. Child Psychology and Psychiatry Review, 6, 172–180.

46.

Nowicki

E. A.

(2003). A meta-analysis of the social competence of children with learning disabilities compared to classmates of low and average to high achievement. Learning Disability Quarterly, 26, 171–188.

47.

Obradovic

Hipwell

(2010). Psychopathology and social competence during the transition to adolescence: The role of family adversity and pubertal development. Development and Psychopathology, 22, 623–636.

48.

Riley

Anderson-Butcher

(2012). Participation in a summer sport-based youth development program for disadvantaged youth: Getting the parent perspective. Children and Youth Services Review, 34, 1367–1377.

49.

Satici

S. A.

Uysal

Satici

(2013). An investigation of the relationship between deontic justice and perceived social competence. Procedia: Social and Behavioral Sciences, 140, 3–8.

50.

Shouse

(1996). Academic press and sense of community: Conflict, congruence, and implications for student achievement. Social Psychology of Education, 1, 47–68.

51.

Thorell

L. B.

Rydell

A. M.

(2008). Behaviour problems and social competence deficits associated with symptoms of attention-deficit/hyperactivity disorder: Effects of age and gender. Child: Care, Health, and Development, 34, 584–595.

52.

Vahedi

Farrokhi

Farajian

(2012). Social competence and behavior problems in preschool children. Iran Journal of Psychiatry, 7, 126–134.

53.

Vandenberg

R. J.

Lance

C. E.

(2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3, 4–70.

54.

Welsh

Parke

R. D.

Widaman

O’Neil

(2001). Linkages between children's social and academic competence: A longitudinal analysis. Journal of School Psychology, 39, 463–482.

55.

Widaman

K. F.

Thompson

J. S.

(2003). On specifying the null model for incremental fit indices in structural equation modeling. Psychological Methods, 8, 16–37.

56.

Wigelsworth

Humphrey

Kalanbooka

Lendrum

(2010). A review of key issues in the measurement of children’s social and emotional skills. Educational Psychology in Practice, 26, 173–186.

57.

Williams

K. L.

Galliher

R. V.

(2006). Predicting depression and self–esteem from social connectedness, support, and competence. Journal of Social and Clinical Psychology, 25, 855–874.

58.

Youngstrom

E. A.

Loeber

Stouthamer-Loeber

(2000). Patterns and correlates of agreement between parent, teacher, and male youth behavior ratings. Journal of Consulting and Clinical Psychology, 68, 1–13.

59.

Zins

Bloodworth

Weissberg

Walberg

(2004). The scientific base linking social and emotional learning to school success. In Zins

Weissberg

Wang

Walberg

H. J.

(Eds.), Building academic success on social and emotional learning: What does the research say? (pp. 1–22). New York, NY: Teachers Press, Columbia University.