Validation of the Chinese Version of the Social Achievement Goal Orientation Scale

Abstract

This study examined the validity of a Chinese version of the Social Achievement Goal Orientation Scale (C-SAGOS), a measure testing the trichotomous framework of achievement goal orientations in a social domain. A total of 208 college students (51% female) aged 18 to 23 participated in the study. Factor analyses showed that the three-factor model best fits the current sample and is invariant across gender groups. Results indicate that the C-SAGOS has acceptable psychometric properties, and the framework of social achievement goal is applicable to a Chinese culture.

Keywords

Social Achievement Goal Orientation Scale reliability validity Chinese undergraduates

Academic achievement goals have been studied extensively in affecting school success (Elliot, 2005). However, students also desire to achieve social goals and have ranked social goals as more important (Horst, Finney, & Barron, 2007). Correspondingly, a social achievement goal framework has been developed to understand why students want to achieve socially (Ryan & Shim, 2006). This framework has a trichotomous structure comprising three types of goals: social mastery goals (SMA), focusing on improving relationship quality and developing social competence; social performance-approach goals (SPA), reflecting desires to demonstrate one’s social competence and win positive evaluation; and social performance-avoidance goals (SPAV), focusing on avoiding being rated as socially incompetent or unpopular. Previous studies have indicated that these goals could uniquely contribute to students’ academic, social, and psychological adjustment (Horst et al., 2007; Kuroda & Sakurai, 2011; Ryan & Shim, 2006; Shim & Finch, 2014; Shim & Ryan, 2012). Therefore, it is meaningful to apply these goals to other cultural paradigms—for example, China. Little research has investigated the achievement goals of Chinese toward social competence.

Researchers from different cultures have made efforts to develop scales for measuring students’ social achievement goals (Kuroda & Sakurai, 2011; Ryan & Shim, 2006). One of the valid measurements for this purpose is the 13-item Social Achievement Goal Orientation Scale (SAGOS; Horst et al., 2007). The 13-item SAGOS has demonstrated a trichotomous factor structure, a superior model fit for college students from a broad study background, good internal consistency across undergraduate samples, and concurrent and longitudinal external validity. In this study, we aimed to validate this scale’s cross-cultural properties using a Chinese undergraduate sample. Following Ryan and Shim (2006), we conceptualized social achievement goals as desires to develop and demonstrate social competence and win positive judgments. Three types of goals reflect this conceptualization.

Method

Participants

Participants were 208 (51% female) undergraduates who attended an introductory psychology class at a large university in the Henan Province of China. They majored in communication (20.9%), civil engineering (13.6%), education (9.2%), electronic commerce (24.3%), and finance (32%) and aged from 18 to 23 (M = 20.34, SD = 3.26), and 99% of them were Han Chinese. The sample consisted of 29% sophomores, 61% juniors, and 10% seniors. The students from six available classes were invited to join the study. After being informed of the research purpose, 94% of the invited students voluntarily filled out a content letter and questionnaire during a class time.

Measurements

C-SAGOS

The C-SAGOS was obtained by translating the original English items (Horst et al., 2007), followed by a translation and back-translation process. To improve content validity, we first assessed whether the translated items reflected the meaning of the three social achievement goals in Chinese context and adjusted several unfit expressions; then we explained the concepts of social achievement goals to four undergraduates and asked them to classify the Chinese items by three goals; after an agreement on the classification was reached, students were asked to give suggestions to improve the expressions. No substantial changes were made to any of the items during this process. The C-SAGOS was administrated to 21 undergraduates (8 males, M_age = 20.33) for pilot testing. After a slight revision, a 13-item C-SAGOS was formed (available upon request). It consists of SMA (e.g., “It is important to me to have friends who really understand me”), SPA (e.g., “It is important to me to be seen as having a lot of friends”), and SPAV (e.g., “I am often concerned that others won’t like me”). Participants rated each item from 1 (not at all true of me) to 5 (very true of me).

Depressive symptoms

Depressive symptoms were measured by the Depression subscale from the Chinese version of the Depression Anxiety Stress Scales (DASS; Gong, Xie, Xu, & Luo, 2010). The Depression subscale contains seven items (e.g., “I felt downhearted and blue”). Students rated each item from 1 (did not apply to me at all) to 4 (applied to me most of the time). The validity and reliability of DASS have been well established using Chinese student samples. Cronbach’s alpha of the subscale for this study was .83.

Study engagement

We used a translated student version of the Utrecht Work Engagement Scale (UWES-CS; Schaufeli, Salanova, Gonzalez-Roma, & Bakker, 2002), a 17-item scale measuring student engagement characterized by learning-related vigor, dedication, and absorption (e.g., “When I am studying, I forget everything else around me”). Students rated each item from 1 (never) to 7 (always). The scale demonstrated high internal consistency reliability and structural validity using Chinese student samples (Zhang & Gan, 2005). Cronbach’s alpha of UWES-CS for this study was .94.

Results and Discussion

The data set had 1.5% randomly missing data, which were imputed using the expectation-maximization (EM) imputation algorithm. We conducted a multivariate skew and kurtosis statistics using an SPSS macro provided by DeCarlo (1997), which suggested that our data were multivariate non-normal.

Confirmatory Factor Analysis (CFA)

We conducted a series of CFAs using Mplus 6 to test the theoretical model. To evaluate model-data fit, we considered three approximate fit indexes suggested by Hu and Bentler (1999): root mean square error of approximation (RMSEA; best if .06 or less), standardized root mean square residual (SRMR; best if .08 or less), and comparative fit index (CFI; best if .95 or greater). The C-SAGOS will be considered as having a satisfactory structural validity if these indices signify a good model-data fit. Considering the multivariate non-normality of the data, we used adjusted maximum likelihood estimation method which provides Satorra–Bentler scaled chi-square (SBχ²) and adjusted model fit indices (Muthén & Muthén, 1998-2012).

The original CFA model resulted in a marginal model fit, SBχ² = 104, df = 62, RMSEA = .06 (.04-.08), SRMR = .07, CFI = .92. All of the items significantly loaded on their targeted factors, with a factor loading above .43 except Item 13 (“I want to have friends who are interested in me”) for SMA and Item 5 (“My goal is to avoid doing things that would cause others to make fun of me”) for SPAV. The factor loadings of these two items were below .3. After removing these two items, the CFA model resulted in an adequate model fit, SBχ² = 79.97, df = 41, RMSEA = .07 (.05-.09), SRMR = .06, CFI = .93. The standardized loadings ranged from .43 to .87 (Figure 1), supporting the items’ membership and convergent structure proposed by Horst et al. (2007). In addition, we tested two alternative models (an approach-avoidance two-factor model and a mastery-performance two-factor model) for the 11-item scale, which presented poor model fit (RMSEAs > .11, SRMRs > .07, and CFIs < .80). Consistent with Horst et al. (2007), the three-factor model is superior to alternative models and best describes the present data. Correlations among three factors are shown in Figure 1. The magnitude of correlations is consistent with previous studies (Kuroda & Sakurai, 2011; Shim & Ryan, 2012).

Figure 1.

Three-factor Structural Model for Social Achievement Goals.

Three steps were then followed to test the measurement invariance across gender groups (Table 1). First, the baseline model with no equality constraints on model parameters across groups (M0) resulted in an acceptable fit. Second, the factorial invariance model with all factor loadings being equal across groups (M1) fit the data reasonably well. The SBχ² difference comparing M0 and M1 was not statistically significant (ΔSBχ² = 8.17, Δdf = 8, p > .05), indicating a factorial invariance of the scale. Third, factor variances and covariances were constrained to be equal across groups (M2). The SBχ² difference between M1 and M2 was not statistically significant (ΔSBχ² = 9.55, Δdf = 6, p > .05), signifying a structural invariance of the scale. These results suggest that men and women are likely to interpret the scale in a conceptually similar manner.

Table 1.

Invariance Constraints for the C-SAGOS Across Gender Groups.

Model	SBχ²	df	RMSEA	SRMR	CFI
M0 = baseline model	127.62	82	.07 (.05-.09)	.07	.915
M1 = M0 with invariant factor loadings	135.38	90	.07 (.04-.09)	.08	.915
M2 = M1 with invariant factor variances and covariances	144.94	96	.07 (.05-.09)	.09	.908

Note. C-SAGOS = Chinese version of the Social Achievement Goal Orientation Scale; SBχ² = Satorra–Bentler scaled chi-square; df = degrees of freedom; RMSEA = root mean square error of approximation; SRMR = standardized root mean square residual; CFI = comparative fit index.

Reliability Analysis and Regression Analysis

The final 11-item scale showed a moderate internal consistency reliability (Cronbach’s α = .76). Cronbach’s alpha for SMA was .69, .68 for SPA, and .75 for SPAV. All corrected item-total correlations were above .36. SMA and SPA subscales had somewhat low internal consistency reliability and were lower than the original English measure. It is likely that cultural differences result in a disagreement between American and Chinese undergraduates regarding the understanding of social achievement goals. Means, standard deviations, and correlations among the subscales are presented in Table 2.

Table 2.

Means, Standard Deviations, Correlations, and Standardized Regression Coefficients.

	Correlations			Regression Coefficients
	SMA	SPA	SPAV	LE	DES
1. Gender^a	.01	.17*	.05	.14*	−.19***
2. SMA	—			.21**	−.11
3. SPA	.25***	—		.18*	−.26***
4. SPAV	.25***	.39***	—	−.13	.28***
5. LE	.21**	.16*	.01
6. DES	−.10	−.15*	.14*
ΔR^2b				.08	.10
M	4.39	2.91	3.25	4.20	1.74
SD	0.49	0.68	0.82	1.07	0.43

Note. SMA = social mastery goals; SPA = social performance-approach goals; SPAV = social performance-avoidance goals; LE = learning engagement; DES = depressive symptoms.

Male = 1, female = 2.

R² with gender being controlled.

p < .05. **p < .01. ***p < .001.

Next, two regression analyses were performed. The predictors were gender and social achievement goals, and dependent variables were learning engagement and depressive symptoms (Table 2). Consistent with previous studies (Kuroda & Sakurai, 2011; Shim & Finch, 2014), SMA significantly predicted learning engagement but not depressive symptoms. SPA negatively predicted depressive symptoms, whereas SPAV positively predicted depressive symptoms. Moreover, SPA, which have not shown an association with learning engagement in previous studies, significantly predicted learning engagement. However, the specific contribution of three goals to the two dependent variables was small (R² < .10). One explanation might be that unknown variables, for instance, self-esteem which is affected longitudinally by social achievement goals, mediate the effects of social achievement goals on two criteria (Shim, Ryan, & Cassady, 2011); at the same time, it contributes strongly to future depression and learning engagement (Sowislo & Orth, 2013; Zeigler-Hill et al., 2013). Another possibility is that the culture values behind the pursuit of social success have eased the associations between social achievement goals and adjustment profiles.

Limitations, Future Directions, and Conclusion

This study aimed to validate the SAGOS using a Chinese undergraduate sample. Two limitations should be considered in interpreting our findings. First, our sample was a convenient sample, which limited the generalizability of the current findings to a broad population. Second, the strength of the relationships between social achievement goals and two selected criteria was weaker than expected. Future studies may investigate the underlying mechanisms of how social achievement goals affect students’ adjustment and the effects of cultural moderators on the pursuit of social achievement goals. Moreover, additional study is necessary to examine whether achievement goals are distinct across the social and academic domains within a Chinese culture. In conclusion, this study supports the applicability of the trichotomous framework of social achievement goals to a Chinese-speaking context and the potential of the C-SAGOS as a useful tool for measuring these goals.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The first author received financial support for the research, authorship, and/or publication of this article from Henan University.

References

DeCarlo

L. T.

(1997). On the meaning and use of kurtosis. Psychological Methods, 2, 292-307.

Elliot

A. J.

(2005). A conceptual history of the achievement goal construct. In Elliot

Dweck

(Eds.), Handbook of competence and motivation (pp. 52-72). New York, NY: Guildford Press.

Gong

Xie

Luo

(2010). Psychometric properties of the Chinese version of DASS-21 in Chinese college students. Chinese Journal of Clinical Psychology, 18, 443-446.

Horst

S. J.

Finney

S. J.

Barron

K. E.

(2007). Moving beyond academic achievement goal measures: A study of social achievement goals. Contemporary Educational Psychology, 32, 667-698.

Bentler

P. M.

(1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1-55.

Kuroda

Sakurai

(2011). Social goal orientations, interpersonal stress, and depressive symptoms among early adolescents in Japan: A test of the diathesis-stress model using the trichotomous framework of social goal orientations. The Journal of Early Adolescence, 31, 300-322.

Muthén

L. K.

Muthén

B. O.

(1998-2012). Mplus user’s guide (7th ed.). Los Angeles, CA: Author.

Ryan

A. M.

Shim

S. S.

(2006). Social achievement goals: The nature and consequences of different orientations toward social competence. Personality and Social Psychology Bulletin, 32, 1246-1263.

Schaufeli

W. B.

Salanova

Gonzalez-Roma

Bakker

(2002). The measurement of burnout and engagement: A confirmatory factor analytic approach. Journal of Happiness Studies, 3, 71-92.

10.

Shim

S. S.

Finch

W. H.

(2014). Academic and social achievement goals and early adolescents’ adjustment: A latent class approach. Learning and Individual Differences, 30, 98-105.

11.

Shim

S. S.

Ryan

A. M.

(2012). What do students want socially when they arrive at college? Implications of social achievement goals for social behaviors and adjustment during the first semester of college. Motivation and Emotion, 36, 504-515.

12.

Shim

S. S.

Ryan

A. M.

Cassady

(2011). Self-esteem change during the first year of college: The role of achievement goals. Educational Psychology, 32, 149-167.

13.

Sowislo

J. F.

Orth

(2013). Does low self-esteem predict depression and anxiety? A meta-analysis of longitudinal studies. Psychological Bulletin, 139, 213-240.

14.

Zeigler-Hill

Masri

Smith

Vonk

Madson

M. B.

Zhang

(2013). Self-esteem instability and academic outcomes in American and Chinese college students. Journal of Research in Personality, 47, 455-463.

15.

Zhang

Gan

(2005). The Chinese version of Utrecht Work Engagement Scale: An examination of reliability and validity. Chinese Journal of Clinical Psychology, 13, 268-270.