Development and Validation of the Perceived Social Work Competence Scale in China

Abstract

Objective:

This article reports a study that developed and validated the Perceived Social Work Competence Scale (PSWCS) for assessing social work students’ competence in Mainland China.

Method:

The indicators were generated by a broad empirical review of recent literature, confirmed by experts, and indigenized by means of two focus groups of students. Two separate studies were conducted, using samples of social work students. Exploratory factor analyses and reliability tests were conducted on a cross-validation sample (n ₁ = 291) of social work students. Confirmatory factor analyses and tests of predictive validity were conducted on the second sample (n ₂ = 300).

Results:

The 48-indicator PSWCS (including nine subscales) demonstrated excellent internal consistency, acceptable test–retest reliability, satisfactory factorial validity, and positive correlation with the students’ grade point average and their satisfaction with their field experience.

Conclusions:

The PSWCS is important for enabling students to assess their competence and for enabling educators to improve field education.

Keywords

perceived competence social work students Mainland China scale validation

Rapid economic development, an aging society, the erosion of family values, and rising residential mobility have triggered a wide array of social problems and led to a new set of social risks for the people of China (Wang, Ruan, & Shi, 2014). Rising social tensions have led to a huge demand for social work professionals to address social problems (Leung, 2013). The 6th plenum of the 16th Central Committee Meeting of the Chinese Communist Party on October 11, 2006, launched the policy and announced the strategic decision to develop a large team of highly qualified social workers in China to assist in the development of a harmonious socialist society (RA-1). Furthermore, social work education and training have flourished, since social workers were identified as one of the six categories of key strategic human resources, and their profession was prioritized for future development, in the National Development Mid- and Long-term Framework on Human Capital 2010–2020 plan launched by the Chinese Government in 2010. On the basis of this plan, the number of social workers in China was projected to grow to 3 million by 2020, though this figure was later revised to one and a half million. With the huge demand for social workers, the state’s expansion of university education, and the relaxation of central control over the establishment of new programs at local universities (Leung, 2013), social work education programs expanded from 20 in 1994 to 298 in 2012 for undergraduate students and from 0 to 61 for graduate students (Wang et al., 2014).

According to the researchers’ comprehensive review (by means of the internet) of the program information of all the social work programs in China, more than 10,000 bachelor of social work (BSW) students and nearly 2,000 master of social work (MSW) students graduate from universities all over China each year. In addition, in 2014, 42 new MSW programs were approved to enroll students; therefore, it is reasonable to predict that the number of social work graduates will continue to increase in the coming years. However, there are still a number of shortcomings affecting the development of social work education: the inadequate curricula and lack of experienced social work educators in training, the lack of qualified instructors in the field, and the underdevelopment of social service agencies that provide placement opportunities for students (Li, Han, & Huang, 2012). These problems result in ill-equipped social work graduates (Leung, 2013). On the one hand, there are increasing complaints from service agencies about the difficulty of recruiting qualified social work graduates who can confidently provide social services; and on the other hand, students increasingly say that they do not get adequate field training, which makes them lack confidence in delivering social services (Ting & Zhang, 2012). In view of these problems, there is an urgent need to improve social work education (Li et al., 2012). The quality of education is to be judged by assessing whether students are equipped with the necessary competence (Leung, 2013; Xu, 2013). In this regard, now is an appropriate time to use a systematic and scientific instrument to evaluate students’ social work competence.

Statement of Purpose

There is a rich body of research on the evaluation of social work students’ and social workers’ competence. These studies focus on the following areas: (a) comprehensive knowledge—relevant here are instructor-evaluated competence (Bogo, 2010), perception of competence (Parker, 2006), and scenarios/vignettes (Regehr, Bogo, Regehr, & Power, 2007); (b) field education—relevant here are social work self-efficacy (Holden, Meenaghan, Anastas, & Metrey, 2002), competencies in macro practice (Regehr, Bogo, Donovan, Lim, & Anstice, 2012), and the Practice Skills Inventory (PSI; O’Hare, Collins, & Walsh, 1998); and (c) cultural aspects—relevant here are self-evaluated multi-cultural counseling (Green et al., 2005). However, few researchers have studied the evaluation of social work students’ competence in a Chinese context. At present, scientific studies of Chinese social work students’ competence in field education are still in their infancy. In this regard, there is a lack of validated competence instruments developed indigenously in China; nor is there any such instrument translated from tools developed/used overseas. Furthermore, in China, many social work students are placed in community-based agencies. It is therefore necessary, when evaluating students’ community-related competence, to add the aspect of community practice to such aspects as relationship formation (RF) and the like that are taken into account in scales developed in the Western context. Thus, the primary purpose of this study was to develop an appropriate competence scale for assessing Chinese students’ social work competence in their field education. The secondary purpose was to heuristically explore the reliability and validity of the developed scale in the Chinese context.

Perceived Social Work Competence

In the assessment of students’ social work competence, three main perspectives—those of school and field instructors, of the “standardized client simulation” or service user, and of the student himself or herself—have been identified in the literature (Bogo, 2010; Green et al., 2005; Holden et al., 2002; O’Hare et al., 1998; Parker, 2006; Regehr et al., 2012). While each perspective has its own advantages and disadvantages, researchers have found that the instructor and standardized client simulation perspectives are not applicable in the context of China. First, Liu, Sun, and Anderson’s (2013) study found that, of the 15 MSW programs in China that were surveyed, 9 were found to be lacking a sufficient number of faculty instructors, 8 reported having inadequate field agencies, and 7 did not have a sufficient number of qualified social workers at the field agencies. This reveals that the use of field instructors’ evaluation of students’ practice could be problematic. Second, although the standardized client simulation perspective might be a promising approach in assessing student competence in the Western countries, in which social work education is relatively advanced (Logie, Bogo, Regehr, & Regehr, 2013), it is not applicable in mainland China at present for two main reasons. First, the development of social work is still in its infancy there (Leung, 2013): MSW programs officially started only in 2009, with the recruitment of social work students beginning only in 2010. Second, the underdevelopment of social work in China means there are not the requisite conditions for conducting standardized client simulations, such as there being qualified evaluators and trained volunteers serving as simulated clients, a sufficient number of simulation cases, and such technical resources as video cameras (Logie et al., 2013).

Given the above constraints, at the current stage of the development of social work field placement in China, it is apparent that the perspective of student self-evaluation might be the most effective and efficient and is probably the only feasible way to assess their competence. In addition to its appropriateness for China’s specific context of the underdevelopment of social work, the concept of perceived social work competence from the student’s perspective has its own merit. The concept refers to a kind of assessment related to self-awareness (Parker, 2004) and has increasingly been adopted as a central concept in self-evaluation and applied in studies for evaluating students’ social work competence over the past two decades globally (Parker, 2006; Pedrazza, Trifiletti, Berlanda, & Bernardo, 2013). With due recognition of China’s specific context of the underdevelopment of social work, the concept is appropriate for evaluating students’ social work competence in field education in the Chinese context. Accordingly, this study employs perceived competence as the principal concept for analyzing students’ competence in social work in China. It is hoped that the findings will lead to the improvement of social work education programs.

Theoretical Framework

Interest in social work competence has led researchers to study its underlying properties and attempt to quantify its dimensions (Bogo, 2010; O’Hare et al., 1998; Parker, 2006). After considering the different competence models, we adopted Bogo’s hierarchy competence model in order to develop a perceived competence model. Following decades of research into social work competencies and 19 in-depth interviews with experienced field instructors, Bogo propounded a “hierarchy of competence,” which delineates first-order and second-order competencies. The meta-competencies are considered first order and the procedural competencies are considered second order, as shown in Figure 1.

Figure 1.

Theoretical framework of perceived social work competence. Based on Bogo’s Professional Competence Model (2010).

The meta-competencies enable mastery and performance of second-order competencies. The meta-competencies involve four interrelated domains—of cognitive/conceptual, interpersonal/relational, personal/professional, and values and ethics competencies—that are the kernel of social work practice (Bogo, 2010). Cognitive or conceptual competencies refer to problem solving, critical thinking, and analytic capabilities as well as to creativity. Interpersonal or relational competencies refer to social workers’ ability to connect with, support, and collaborate with team members, clients, and (in the case of students) field instructors; these competencies concern positive relationships. Personal or professional competencies refer to a cluster of abilities related to learning and performing in an organization and in a professional role. The identification of social work values and ethics are critically important because they are the foundation of social work practice. The second-order competencies concern procedural, operational, or clinical aspects of practice, including assessment, intervention, and professional communication (Bogo, 2010). They may be articulated at two levels: (a) generic to the profession and (b) specific to specializations within the profession. Whether the competencies are categorized as meta- or procedural competencies, they are all identified as involving numerous behavioral skills (Bogo, 2010). In this study, the choice of the items measuring the various types of competencies was based on their appropriately matching these conceptual dimensions.

Method

The current study developed and validated the Perceived Social Work Competence Scale (PSWCS), using a Chinese social work student sample, by means of two studies. Before completing Study 1 and Study 2, the PSWCS was prepared in three stages. First, 53 items were selected from various social work competence-related scales to form an indicator pool that might be appropriate for assessing students’ social work competence in fieldwork in mainland China. Second, the PSWCS was translated into Chinese, and its cultural relevance and content validity were evaluated by expert panel members. Third, a pilot study was conducted to approximately determine the required time and complexity of the questionnaire. Then, invitation letters were sent out to invite social work students to participate in the online survey (via Survey Monkey) at two time points, forming Study 1 and Study 2.

Scale Construction

The initial items of the PSWCS were developed using scale construction methods in two steps. First, a review of the literature on competence and related constructs was conducted. This provided descriptions of perceived competence that formed a pool of terms from which the scale items were developed. Second, items from existing general social work competence scales were reviewed, and phraseology related to the concept of perceived competence was incorporated into the construction of the scale items. On the basis of the theoretical framework of perceived social work competence (Figure 1), each of the six conceptual dimensions in the meta-competencies and the procedural competencies was allotted its own subscales, resulting in a scale with 10 operationalized subscales for initial testing (see Figure 2).

Figure 2.

Conceptual framework of perceived social work competence.

The PSWCS consisted of 53 items covered in 10 subscales: (1) RF, (2) team working (TW), (3) professional knowledge development (PKD), (4) professional resilience development (PRD), (5) professional values and ethics (PVE), (6) therapeutic skills (TS), (7) insight skills (IS), (8) case management skills (CMS), (9) supportive skills (SS), and (10) community work skills (CWS). The respondents were asked to indicate how confident they felt about executing the social work related tasks. A 5-point Likert-type scale was adopted from Parker’s (2006) scale, ranging from 1 (I am not at all confident I can do this) to 5 (I am very confident I can do this).

First, the subscales of TS, CMS, SS, and IS were derived from the PSI with 23 items (Holden et al., 2002; O’Hare & Collins, 1997; O’Hare et al., 1998). The PSI was developed by O’Hare and Collins and revised by Holden et al.; it is used to assess social work students’ different practical skills. There are two versions of the PSI developed by O’Hare and Collins; one version has 33 items and one has 23 items. In this research, the version comprising 23 items was selected because it accounted for about 9% more variance (60.6% vs. 52.1%) with about one third fewer items (23 vs. 33). Second, the PVE subscale was adapted from the Competence-based Evaluation Scale (CBE; Bogo, 2010) and contributed 5 items to the PSWCS. Bogo’s (2010) CBE consisted of 57 indicators and seven domains: values and ethics, the different uses of the self, empathy and alliance, assessment, intervention, report writing, and presentation skills. Because the CBE was designed for field instructors to rate students’ competence, we used only the subscale of PVE as a complementary domain to the PSWCS. Third, the subscales of RF, TW, PKD, and PRD were derived from the Perception of Competence Instrument (Parker, 2006); these contributed 18 items to the PSWCS. Most of the subscales of the Perception of Competence Instrument were adopted in the PSWCS to assess social work students’ general social work competence; however, in light of the current situation in mainland China, the service user groups and written communication subscales were deleted. In mainland China, social work students are likely to have fewer opportunities than Western social work students to work with a wide range of service user groups (Leung, 2013). In addition, written communication was deleted because the relevant kind of documentation is not widely furnished in China (Ting & Zhang, 2012). Fourth, the CS subscale was derived from Parris’ (2012) book An Introduction to Social Work Practice and contributed 7 items to the PSWCS. Social work programs in most of the cities and provinces in China (though not in Guangdong) have placed students in their local communities, due to the lack of social work agencies (Liu, Sun, & Anderson, 2013). Therefore, community social work competence was a very important dimension and needed to be included in the PSWCS.

Face and Content Validity

The 53 items for the PSWCS were translated into Chinese, and nine experts were invited from Hong Kong, Taiwan, Canada, and mainland China to evaluate the content validity and cultural relevance of PSWCS. First, the PSWCS was translated into Chinese by the researcher and a Chinese PhD student who majored in (American) English; then it was translated back into English by two professional translators. The discrepancies between the English and Chinese versions were evaluated. Then, through an iterative review process, the discrepancies were gradually reduced with improvement at every step. The members of the expert panel comprised six teachers with rich experience in social work education and field education, one person with a senior position in a social work association in China, and two experienced social workers based in Guangdong province. Each panel member fulfilled at least two of the following criteria: (1) the member has more than 5 years of social work teaching/work-related experience; (2) the member has published, in academic journals, articles related to social work education; (3) the member works in a university that provides BSW and/or MSW programs; and (4) the member understands social work and/or social work education in the Chinese context. Most of the experts accepted the overall coherence of the test indicators, the cultural relevance, and the representativeness of the PSWCS in relation to its suitability for assessing social work students’ competence in mainland China. Four panel members, however, thought the scale was too long and complicated, and they suggested conducting a pilot study first to confirm that social work students could comprehend it and that the language level was suitable. The participants had to complete a consent form before completing the questionnaire, which comprised 53 items of the PSWCS; two background/sociodemographic questions (regarding the respondents’ gender and previous year’s family income); and three additional questions about their grade point average (GPA) of the previous semester, their perceived overall social work competence (rated from 0 to 100), and their satisfaction with their field education.

Sample and Procedure

Ethical approval was obtained from the Human Research Ethics Committee for Non-Clinical Faculties of The University of Hong Kong in March 2014. All the students in Study 1 and Study 2 were informed of their right to voluntary participation and the confidentiality of the information collected from them, and they were invited to sign a consent form that specified the aims and objectives of the study.

A pilot study was conducted before the surveys of Study 1 and Study 2. Thirteen social work students were invited to fill in the paper-based scale with 53 items in order to estimate the length of time required, to assess whether the language used in the items was comprehensible, and to identify any other potential problems. Six of the students were from one BSW program in Shenzhen city and the other seven students were from one MSW program in Guangzhou city. After the 13 social work students had completed the questionnaire, the researchers also conducted two focus groups, one with the BSW and one with the MSW students, to gather their views on the cultural relevance of the scale. Then the researcher rephrased some wordings and rearranged the sequence of some indicators to clarify the indicators (except for those in the PVE category) in the Chinese context. Eleven of the students had concerns about the PVE subscale, commenting that the items were hard to understand and apply in the Chinese context. However, due to the important role of values and ethics in social work, the PVE subscale was retained in the PSWCS validation.

A preliminary online survey (via Survey Monkey) was conducted in May 2014 with the aim of recruiting social work students for Study 1. Another online survey was conducted in March 2015 with the aim of recruiting a group of social work students for Study 2. Because of privacy concerns and the lack of a sampling frame, the researchers adopted nonprobability methods—chain-referral methods, such as snowball and respondent-driven sampling (Heckathorn, 2002)—for nationwide recruitment of social work students who were to graduate in 2014 (for Study 1) and 2015 (for Study 2). Two approaches were used in collecting the data. First of all, the researchers asked the 13 pilot study respondents to refer those of their classmates or friends who were social work students in 2014 to participate in the online survey. In addition, the researchers invited social work educators in mainland China to promote the online survey, in order to cover more social work programs and regions. The social work students took 10–15 minutes to complete the survey questionnaire.

Strategy for the Data Analysis

The Statistical Package for Social Science Version 21.0 was used to run the descriptive statistics, to identify missing data and the assumptions for the confirmatory factor analysis (CFA), to conduct the exploratory factor analysis (EFA), to evaluate the internal reliabilities, and to determine the criterion validity of the PSWCS. The AMOS Version 21.0 software program was used to conduct CFA. (There were no missing data because the online survey was set up to require each respondent to answer all the questions and complete the questionnaire; if any questions were left unanswered, the questionnaire was automatically rejected.) In addition, bivariate scatter plots were utilized to examine the linearity, and this condition was adequately met. Normality, univariate and multivariate skewness, and kurtosis were then examined (Kline, 2005). These analyses did not reveal any problems with normality.

Results

Study 1

Sociodemographic Characteristics of the Sample

The participants were BSW or MSW social work students in 2014. In total, 363 social work students signed the consent forms to participate in the survey. Of these, 291 students filled in all the items of the online questionnaire (the response rate was 80.2%; the mean age was 24; the age range was 20–31; 78.7% were females; and 52.6% were MSW students). The participants were from social work programs located in the following cities and provinces: Beijing (28.1%), Shanghai (7%), Guangdong province (42.2%), Henan province (7.4%), Shanxi province (4.4%), Jiangsu province (5.2%), and Hebei province (5.6%). All the programs were in urban cities in China, either in the municipalities or in the provincial capitals. Most of the participants were placed in local communities (55.7%) or schools (19.2%), and they served individual persons (50.2%) and communities (52.2%) in their field placement (there was a multiple-choice item which allowed students to choose more than one served party).

EFA (n = 291)

Recently, many studies have investigated the determinants of “reliable factor recovery” and minimum sample size (Gagné & Hancock, 2006; MacCallum, Widaman, Preacher, & Hong, 2001; Winter, Dodou, & Wieringa, 2009). Research has shown that determining whether the sample size is adequate to identify reliable factors depends on several parameters: the level of commonalities (>.5), the indicator loadings (>.4), the number of variables per factor (>2 indicators), and the number of factors (Gagné & Hancock, 2006). In addition, research has demonstrated that a small sample can be used to identify reliable factors when certain criteria are met. Given the high level of commonalities (.583), the high number of indicators (53), the adequate subject-to-item ratio (5.5:1), and the number of factors (nine) of this study, we estimated that an adequate sample for conducting reliable analyses would be between 150 and 200 participants (Winter et al., 2009). Therefore, the sample size (n = 291) is likely to be adequate for the factor analysis and likely to yield good-quality results.

The factor structure of the perceived competence scale was identified by principal components analysis (PCA) and principal axis factoring (PAF). To determine the appropriateness of the data set to conduct PCA and PAF, the Kaiser–Meyer–Olkin (KMO) and the χ² value of Bartlett’s test were calculated; these indicated that the data set was adequate (KMO = 0.939, p < .001). PCA was used to estimate the underlying number of factors with promax rotation (the factors were interrelated). The criteria used to determine the factors and their items included the following: (1) a factor has an eigenvalue greater than 1, (2) an item has a factor loading greater than 0.4, (3) a factor has at least 3 items, and (4) an identified factor and the retained items are interpretable in a theoretical context. Then we identified nine components: (1) RF, (2) TW, (3)PKD, (4) PRD, (5) PVE, (6) CMS, (7) SS, (8) CWS, and (9) therapeutic and insight skills (TIS). It should be noted that the hypothesized TS were merged with IS and these (previously TS and IS, respectively) were renamed TIS.

With this nine-factor solution, we then conducted PAF with promax rotation to a preselected nine-factor solution. Five items were deleted (because they were either loaded in more than two factors or had a factor loading lower than 0.4): “recognize the ethical tensions inherent in the work” in PKD; “manage the stress that you feel in a fast-paced working and learning environment” in PRD; “increase their confidence that you really can help them” in CMS; and “ demonstrate congruence between one’s activities and professional values and ethics” and “take into account all value systems, including one’s own, that impinge on the practice situation” in PVE. The nine factors, comprising 48 items, explained 58.34% of the total variance in the structure matrix and the cumulative explanations were TIS (36.96%), CWS (41.45%), SS (45.28%), PRD (48.45%), TW (51.12%), CMS (53.31%), RF (55.16%), PKD (56.8%), and PVE (58.34%). The pattern matrix of the nine-factor scale is reported in the Appendix.

Reliability

Internal consistency: Cronbach’s α (all nine factors; n = 291)

The internal consistency obtained for the scale was excellent, as reflected by the Cronbach’s α of .964 (Kline, 2005), shown in Table 1. In addition, the internal consistencies for the subscales ranged from acceptable to excellent; the Cronbach’s αs were between .722 and .925.

Table 1.

Reliability of the Nine Factors Used in Study 1.

Effect Size	All Indicators	RF	TW	PKD	PRD	TIS	CMS	CWS	SS	PVE
Cronbach’s α	.964	.833	.839	.821	.869	.925	.857	.893	.882	.722
95% Confidence interval	[.46, .55]	[.51, .62]	[.50, .61]	[.48, .60]	[.57, .67]	[.50, .61]	[.50, .60]	[.50, .59]	[.55, .65]	[.40, .53]
Number. of items	48	4	"4	4	4	12	5	7	5	3

Note. n = 29. TIS = therapeutic and insight skills; CWS = community work skills; SS = supportive skills; PRD = professional resilience development; TW = team working; CMS = case management skills; RF = relationship formation; PKD = professional knowledge development; PVE = professional values and ethics.

Retest reliability (n = 34)

After 4 weeks, 40 randomly selected participants were invited (by email) to complete the same questionnaire again; of these, 34 students responded (a response rate of 85%). The 4-week test–retest reliability, as indicated by Pearson’s r, was .73. In addition, the reliability of the subscales was separately indicated by a Pearson’s r of between .40 and .60 (TIS .61, CWS .56, SS .43, PRD .60, PKD .58, TW .46, RF .53, CMS .48, and PVE .48). Furthermore, a paired t-test was conducted to analyze the difference in the scores on the 48-item PSWCS in the test and the retest. The result (t = −.987, p = .331, confidence intervals [CI] [−.15, .05]) showed that there is no difference in the test and the retest.

Differences Between the Samples in Study 1 and Study 2

The second study involved administering the newly generated PSWCS to a second sample in order to confirm its factor structure and evaluate the criterion validity. As described below, the second sample was comparable to the first sample. It should be stressed that there was no significant difference between the two samples (as determined by t-tests and indicated in what immediately follows by the p values and 95% CI of the differences) with regard to age (p = .793, CI [−.11, .15]); gender (p = .265, CI [−.03, .1]); the 48-item scale score (p = .186, CI [−.14, .03]); or any of the subscale scores: TIS (p = .699, CI [−.07, .12]); CWS (p = .199, CI [−.04, .17]); SS (p = .148, CI [−.03 .18]); PRD (p = .587, CI [−.16, .09]); PKD (p = .192, CI [−.18, .04]); TW (p = .051, CI [−.22, .0]); RF (p = .645, CI [−.08, .13]); CMS (p = .149, CI [−.19, .03]); and PVE (p = .096, CI [−.2 .02]).

Results

Study 2

Sociodemographic Characteristics of the Participants in Study 2

The participants were recruited from MSW students who were to graduate in 2015. This was a nationwide survey; an invitation was sent to all the 61 universities providing MSW programs, and 37 (60.7%) responded. These covered 19 provinces (82.6% of the 23 provinces that enrolled MSW students in 2015). The MSW programs in Liaoning, Inner Mongolia, Zhejiang province, and Chong Qing municipality were not included. In total, 300 online questionnaires were completed (the mean age of the respondents was 26 years, and 82.3% were females). Most of the participants were placed in communities (60%) and schools (25.3%) and most served individual persons (66.3%) and communities (55%) in their field placement.

CFA (n = 300)

The nine-factor structure model CFA, with 48 indicators, was conducted on a sample collected in Study 2 to test the stability of the extracted model. In order to identify the model, the maximum likelihood estimation method was used and the first loading of each factor was fixed to 1. No item was deleted because the factor loading of all the 48 indicators was more than .55. The model fit is satisfactory when (CMIN/df) is less than 2; the comparative fit index (CFI) and Tucker–Lewis ρ (TLI ρ²) are equal to or above .9; the root mean square error of approximation (RMSEA) is less than .06; and the standardized root mean square residual (SRMR) is less than .08 (Brown, 2006; Hu & Bentler, 1999). The original Model 1A yielded an unsatisfactory CFI and TLI ρ² but acceptable RMSEA and SRMR. Large modification indices were calculated in eight pairs of error covariance. These parameters were allowed to be free because they belonged to the same factor, and they led to Model 1B. A closer examination of these pairs revealed that their contents were very similar in Chinese meaning (see Table 2). The model fit of Model 1B was better: CMIN/df = 1.762, CFI = .904, TLI ρ² = .895, RMSEA = .05, and SRMR = .0512.

Table 2.

Summary of the Goodness-of-Fit Indices for the Two CFA Models.

Model	Description	χ²/df	CFI	GFI	PGFI	TLI	RMSEA	SRMR
1A	Nine-factor model without any error covariance	2.115	0.858	0.758	0.673	0.847	0.061	0.053
1B	Nine-factor model with eight error covariance	1.762	0.904	0.795	0.700	0.895	0.050	0.051
Pair 1 (TW): Work with field supervisor ⌒ Work with school supervisor.
Pair 2 (CWS): Research particularly collaborative and participatory action research with members of the community ⌒ Monitor and evaluate community.
Pair 3 (RF): Establish contact with clients ⌒ Form working relationships with clients.
Pair 4 (CMS): Advocating on behalf of clients ⌒ Providing information about other services available to clients.
Pair 5 (SS): Helping clients to feel good about themselves ⌒ Helping clients to feel motivated to work on their problems in treatment.
Pair 6 (TIS): Help clients to learn from past experiences ⌒ Helping clients to learn from past attempts to solve problems.
Pair 7 (TIS): Collaborating with clients on plans to help them cope with occasional relapses of a problem ⌒ Measuring client progress over time with scales attempts to solve problems
Pair 8 (TIS): Teaching clients how to manage their own problem behaviors ⌒ Teaching clients how to accomplish tasks more effectively.

Note. n = 300. CFA = confirmatory factor analysis; CFI = comparative fit index; GFI = goodness-of-fit index; PGFI = parsimony goodness-of-fit index; TLI = Tucker–Lewis index; RMSEA = root mean square error of approximation; SRMR = standardized root mean square residual; TW = team working; CWS = community work skills; RF = relationship formation; CMS = case management skills; SS = supportive skills; TIS = therapeutic and insight skills.

In this model, the indicator loadings were all significant (p < .001) and higher than .56, as shown in Table 3. In addition, it is suggested that the unstandardized regression weight (URW), the p values (<.001), standardized regression weight (SRW), standardized errors (SE), and the square multiple correlations (SMC) should be reported in CFA (Jackson, Gillaspy, & Purc-Stephenson, 2009), therefore, the URW, SRW, SE, p values, and SMC are all reported in this study.

Table 3.

Unstandardized and Standardized Values for the 48 Items.

The 48 Items	Factors	URW	SE	p	SRW	SMC
Establish contact with clients	RF	1.01	0.10	***	0.67	0.45
Form working relationships with clients		1.04	0.10	***	0.71	0.50
Provide relevant information and advice		1.09	0.10	***	0.74	0.54
Collect relevant information		1.00			0.70	0.49
Work with other social work students	TW	1.02	0.11	***	0.75	0.57
Work with agency staffs		1.30	0.13	***	0.89	0.79
Work with your field supervisor		1.08	0.09	***	0.65	0.42
Work with your school supervisor		1.00			0.56	0.32
Apply theoretical models appropriately	PKD	1.00	0.08	***	0.73	0.53
Identify your own learning needs		0.90	0.07	***	0.76	0.58
Evaluate your own practice		0.97	0.08	***	0.74	0.55
Make plans to increase your own learning opportunities		1.00			0.75	0.56
Manage the frustration you will feel when working with agency bureaucracy	PRD	0.89	0.08	***	0.63	0.40
Manage the feelings that you will have when working with clients experiencing emotional and psychological distress		1.06	0.07	***	0.80	0.65
Manage the feelings you will have when clients or their families blame you for things going wrong		1.18	0.07	***	0.86	0.75
Manage the feelings you will have when team members from other disciplines blame you for things going wrong		1.00			0.80	0.64
Help clients to learn from their past experiences	TIS	1.00			0.69	0.51
Helping clients to learn from past attempts to solve problems		0.97	0.09	***	0.68	0.47
Exploring with clients how past relationships contribute to the understanding of current problems		1.04	0.10	***	0.65	0.34
Gentle probing to help them uncover troubling feelings		1.17	0.10	***	0.71	0.46
Teaching clients specific skills to deal with certain problems		1.10	0.10	***	0.71	0.52
Showing clients how to reward themselves for progress with problem		1.08	0.09	***	0.71	0.54
Teaching clients how to manage their own problem behaviors		1.14	0.10	***	0.73	0.50
Teaching clients how to accomplish tasks more effectively		1.18	0.10	***	0.72	0.51
Teaching clients how to monitor their own behaviors		1.06	0.10	***	0.68	0.50
Collaborating with clients on plans to help them cope with occasional relapses of a problem		1.10	0.11	***	0.59	0.42
Measuring client progress over time with scales		0.95	0.09	***	0.69	0.46
Engendering confidence so that clients can cope better with their problems		1.09	0.09	***	0.71	0.48
Helping clients to feel good about themselves	SS	1.00			0.73	0.53
Helping clients to feel motivated to work on their problems in treatment		1.02	0.07	***	0.73	0.54
Providing emotional support for clients		1.08	0.09	***	0.77	0.60
Helping clients feel like they want to open up to		1.24	0.10	***	0.77	0.59
Pointing out their successes in order to increase their self-confidence		1.14	0.09	***	0.77	0.60
Assessing their level of material resources	CMS	1.00			0.67	0.45
Advocating on behalf of clients		1.14	0.12	***	0.65	0.42
Making referrals to other services		1.26	0.11	***	0.76	0.57
Providing information about other services available to clients		1.25	0.11	***	0.81	0.66
Networking with agencies to co-ordinate services		1.03	0.11	***	0.62	0.38
Engage with people on different levels	CWS	0.96	0.08	***	0.76	0.58
Assess at the level of the community		0.93	0.09	***	0.69	0.47
Negotiate and resolve conflicts		1.03	0.09	***	0.73	0.53
Obtain and manage resources		0.97	0.09	***	0.70	0.49
Research particularly collaborative and participatory action research with members of the community		0.88	0.08	***	0.67	0.44
Monitor and evaluate in the community		0.88	0.09	***	0.65	0.42
Communicate in a wide range of situations and contexts		1.00			0.70	0.50
Demonstrate respectful behavior for various cultural norms, values systems, ethics, and moral beliefs in interaction with colleagues and client groups	PVE	1.06	0.10	***	0.76	0.58
Be accountable to the client system, the agency, and the profession		1.06	0.10	***	0.80	0.63
Describe behavior in nonjudgmental terms		1			0.67	0.44

Note. The respondents were asked to indicate—using a 5-point Likert-type scale that ranged from 1 (I am not at all confident I can do this) to 5 (I am very confident I can do this)—to what extent they considered themselves able to do these things. TIS = therapeutic and insight skills; CWS = community work skills; SS = supportive skills; PRD = professional resilience development; TW = team working; CMS = case management skills; RF = relationship formation; PKD = professional knowledge development; PVE = professional values and ethics.

***p < .001.

Convergent and discriminant validity

According to Brown (2006), it is also very important to assess the validity of the factor solution on the basis of parameter estimates. Therefore, the convergent and discriminant validity was evaluated according to parameter estimates. If factor correlations are greater than .8, the scale may indicate poor discriminant validity (Brown, 2006). In this model of the PSWCS, most of the correlations of two of the nine factors—but not the correlation between TIS and SS—ranged from .39 to .79 (as shown in Table 4), which indicates acceptable discriminant validity. In addition, strong factor loadings (>.6) that do not cross load may indicate good convergent validity (Garson, 2010). All of the nine subscales showed factor loading greater than .66, which indicates the scale has good convergent validity.

Table 4.

Correlations Among the Nine Factors.

Nine Factors	TW	TIS	CWS	SS	RF	PRD	CMS	PVE	PKD
TW	—
TS	0.57***	—
CWS	0.52***	0.70***	—
SS	0.52***	0.83***	0.56***	—
RF	0.70***	0.79***	0.63***	0.72***	—
PRD	0.54***	0.70***	0.54***	0.63***	0.58***	—
CMS	0.50***	0.71***	0.71***	0.60***	0.62***	0.44***	—
PVE	0.39***	0.64***	0.56***	0.64***	0.53***	0.49***	0.48***	—
PKD	0.58***	0.77***	0.68***	0.61***	0.75***	0.60***	0.65***	0.55***	—

Note. TIS = therapeutic and insight skills; CWS = community work skills; SS = supportive skills; PRD = professional resilience development; TW = team working; CMS = case management skills; RF = relationship formation; PKD = professional knowledge development; PVE = professional values and ethics.

***p < .001.

Criterion validity

Criterion validity was assessed by examining the associations of the criterion measurements that were empirically found to be related to social work competence. It was expected that social work competence would show a positive association with GPA (Bogo et al., 2002), the level of satisfaction with field experience (Lee & Fortune, 2013), and students’ perceived overall social work competence, and that there would be no relationship with family income. The correlation between all the scores for the nine factors of the PSWCS and the criterion variables were all statistically significant (as with satisfaction, GPA, and self-rated overall competence) or insignificant (as with family income) as expected and as Table 5 demonstrates.

Table 5.

Correlations Between the Nine Factors and Other Criterion Variables.

Criterion Variables		RF	TW	PKD	PRD	TIS	CMS	CWS	SS	PVE
Satisfaction with field experience	Pearson’s r	.432**	.415**	.421**	.292**	.407**	.291**	.327**	.326**	.215**
GPA	Pearson’s r	.231**	.194*	.245**	.161*	.199*	.165*	.120*	.175*	.141*
Overall competence	Pearson’s r	.416**	.387**	.568**	.428**	.529**	.353**	.456**	.399**	.351**
Family income	Pearson’s r	−.034	.042	.016	−.055	.025	.035	.097	−.015	.040

**Mean p < .01. *Mean p < .05.

Discussion and Application to Social Work

The present study was aimed at developing and validating an instrument to measure students’ perceived social work competence. The findings, using the data collected from two samples of social work students, demonstrate that the nine-factor, 48-indicator PSWCS has a reliable and valid structure. The evaluation of the PSWCS adds to what is known about evaluating the levels of meta-competencies and procedural competencies (Bogo, 2010) in several important ways.

The psychometric properties of the PSWCS lend empirical support to our study’s theoretical claim that perceived competence consists of a variety of discrete but overlapping constructs. The internal consistency of the scale was strong, indicating the measure is indeed tapping into a commonly understood concept labeled as “perceived competence” in the research. Factor analyses reinforce the idea that perceived competence in social work practice consists of various dimensions and it appears to be a complex, interrelated, and dynamic grouping of various behavioral skills and aptitudes.

Social work education has developed very quickly in mainland China over the past decade (Leung, 2013). We acknowledge that social work competence may be evaluated by instructors or service users. However, due to the severe shortage of instructors and placement agencies, it is more practical in the context of China (where social work education is still in its embryonic stage) to use the student perspective in assessing student competence in social work field education. Therefore, we suggest that the validated PSWCS be widely used for assessing students’ self-evaluation of their competence (i.e., perceived social work competence) and to allow comparison of social work programs so as to afford a better understanding of the effectiveness of social work field education in China. In addition, Chinese social work educators may understand students’ competencies in the nine dimensions and may supervise social work students on the basis of these dimensions, which are the following: those involving skills in providing therapy and being insightful (TIS), working in a team (TW), forming relationships (RF); those needed in case management (CMS), in community work (CWS), and in being supportive (SS); those involving the development of professional knowledge (PKD) and professional resilience (PRD); and finally those involving values and ethics (PVE)—and may supervise social work students on the basis of these dimensions.

The findings of the study also suggest that there may be benefits in using the PSWCS to assess the effectiveness of social work education in China. The results reveal that social work students’ competencies are associated with their GPA and their satisfaction with field education, as expected. This indicates that the PSWCS can be adopted in attempts to understand the antecedents and consequences of competence via multiple regression analyses or structural equation modeling. In addition, the availability of such a perceived competence scale—and the findings concerning the relationships between social work competence and other variables—may generate critical input for the government in formulating nationwide policies, and for universities in designing curricula, that develop effective social work education.

Assessing student competence is crucial for developing and revising social work education curricula worldwide. Many competence-related scales have been developed and studied in the past 10 years and have demonstrated good reliability. However, seldom have these studies reported the construct validity by employing both EFA and CFA. Therefore, it is possible that a similar version of the PSWCS could be used in other contexts to assess students’ competence by making reference to Bogo’s competence theory and employing indicators selected from the existing scales developed in Western countries.

It is important to emphasize that scale development and validation are iterative processes. This study has advanced the endeavor not only in developing and validating a psychometrically sound scale for measuring social work students’ competence in China but also in raising three important issues for future research to consider. First of all, the study focused on the students’ self-reporting results. To supplement this, it is highly recommended that students’ perceptions in measuring their learning process (such as in assessing their learning activities and the quality of supervision) should also be examined in future studies. Including the views of other significant stakeholders in field education (such as field instructors and school instructors) may provide a more comprehensive and balanced picture (Vitali, 2011). Nevertheless, it is reasonable to allow students themselves, rather than instructors, to rate their competence using the basic concepts of the PSWCS. Furthermore, it should also be clarified that, in China, the student self-reporting approach to understanding competence might be better than the instructor-reporting approach, due to the shortage and inadequate training of instructors there. Second, it should also be noted that there might be a response shift bias and a social desirability bias. However, the online survey approach to data collection employed in this study should reduce the social desirability bias (compared with paper-based or classroom-based data collection strategies), as the identity of the respondents is not disclosed to the interviewer or researcher. Third, the data might not fully represent the population of social work students in China. Thus, generalization of the findings to other regions should be conducted with caution. However, in terms of the development level of social work education, the study included diverse social work programs not only in developed regions (Beijing, Shanghai, and Guangdong province) but also in comparatively less well-developed regions (such as the provinces of Henan, Shanxi, Jiangsu, and Hebei).

The current study provides empirical evidence of the psychometric properties of the PSWCS. We believe that a clear understanding of Chinese social work students’ competence following field education—an understanding based on the application of a psychometrically sound instrument—will significantly contribute to the enhancement of social work practice and research in the future.

Footnotes

Appendix

Authors’ Note

Y. Wang conducted the initial analyses and wrote the initial draft. Ernest Chui strengthened the organization of an article and the interpretation of the findings. Both authors contributed equally to the final version of the paper before submission.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

References

Bogo

(2010). Achieving competence in social work through field education. Toronto, Canada: University of Toronto Press.

Bogo

Regehr

Hughes

Power

Globerman

(2002). Evaluating a measure of student field performance in direct service: Testing reliability and validity of explicit criteria. Journal of Social Work Education, 38, 385–401.

Brown

(2006). Confirmatory factor analysis for applied research. New York, NY: Guilford Press.

Gagné

Hancock

(2006). Measurement model quality, sample size, and solution propriety in confirmation factor models. Multivariate Behavioral Research, 41, 65–83.

Garson

(2010). Statnotes: Topics in multivariate analysis: Factor analysis. Retrieved from http://faculty.chass.ncsu.edu/garson/pa765/statnote.htm

Green

R. G.

Kiernan-Stern

Bailey

Chambers

Claridge

Jones

… Walker

(2005). The multicultural counseling inventory: A measure for evaluating social work student and practitioner self-perceptions of their multicultural competencies. Journal of Social Work Education, 41, 191–208.

Heckathorn

D. D.

(2002). Respondent-driven sampling II: Deriving valid population estimates from chain-referral samples of hidden populations. Social Problems, 49, 11–34. doi:10.1525/sp.2002.49.1.11

Holden

Meenaghan

Anastas

Metrey

(2002). Outcomes of social work education: The case for social work self-efficacy. Journal of Social Work Education, 38, 115–133.

Bentler

(1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1–55. doi:10.1080/10705519909540118

10.

Jackson

Gillaspy

Purc-Stephenson

(2009). Reporting practices in confirmatory factor analysis: An overview and some recommendations. Psychological Methods, 14, 6–23.

11.

Kline

R. B.

(2005). Principles and practice of structural equation modeling (2nd ed.). New York, NY: Guilford Press.

12.

Lee

Fortune

A. E.

(2013). Patterns of field learning activities and their relation to learning outcome. Journal of Social Work Education, 49, 420–438.

13.

Leung

(2013). The development of social work and social work education in China: Issues and prospects. In Noble

Henrickson

Han

(Eds.), Social work education in the Asia Pacific region: Issues and debates (pp. 83–95). Sydney, Australia: University of Sydney Press.

14.

Han

Huang

(2012). Development of social work education in China: Background, current status, and prospects. Journal of Social Work Education, 48, 635–653.

15.

Liu

Sun

Anderson

(2013). Challenges in social work field education in China: Lessons from the Western experience. Social Work Education, 32, 179–196.

16.

Logie

Bogo

Regehr

(2013). A critical appraisal of the use of standardized client simulations in social work education. Journal of Social Work Education, 49, 66–80.

17.

MacCallum

Widaman

Preacher

Hong

(2001). Sample size in factor analysis: The role of model error. Multivariate Behavioral Research, 36, 611–637.

18.

O’Hare

Collins

(1997). Development of a valid scale for measuring social work practice skills. Research on Social Work Practice, 7, 228–238.

19.

O’Hare

Collins

Walsh

(1998). Validation of the practice skills inventory with experienced clinical social workers. Research on Social Work Practice, 8, 552–563.

20.

Parker

(2004). Effective field education in social work. Exeter, England: Learning Matters.

21.

Parker

(2006). Developing perceptions of competence during field education. British Journal of Social Work, 36, 1017–1036.

22.

Parris

(2012). An introduction to social work practice. Maidenhead, England: Open University Press.

23.

Pedrazza

Trifiletti

Berlanda

Bernardo

(2013). Self-efficacy in social work: Development and initial validation of the self-efficacy scale for social workers. Social Science, 2, 191–207.

24.

Regehr

Bogo

Donovan

Lim

Anstice

(2012). Identifying student competencies in macro practice: Articulating the practice wisdom of field instructors. Journal of Social Work Education, 48, 307–319.

25.

Regehr

Bogo

Regehr

Power

(2007). Can we build a better mousetrap? Improving the measures of practice performance in the field practicum. Journal of Social Work Education, 43, 327–344.

26.

Ting

Zhang

(2012). Flourishing in the spring? Social work, social work education and field education in China. China Journal of Social Work, 5, 201–222.

27.

Vitali

(2011). The acquisition of professional social work competencies. Social Work Education, 30, 236–246.

28.

王思斌，阮曾媛琪，史柏年. [ Wang

Ruan

Shi

] (2014). 中国社会工作教育的发展. [The development of social work education in Mainland China]. 北京大学出版社. Beijing, China. ISBN:9787301249666

29.

Winter

Dodou

Wieringa

(2009). Exploratory factor analysis with small sample sizes. Multivariate Behavioral Research, 44, 147–181.

30.

徐迎春. [ Xu

] (2013). 本土处境与现实策略——近十年来社会工作实习教育研究文献综述. [The challenges and strategies—a literature review on the research of social work field education in recent ten years]. 社会工作, 5, 120–127.