The Cultural Competency for Working With Asian American Clients Scale

Abstract

Objectives:

Many Asian Americans tend to report receiving lower quality services from direct service providers. Improving the cultural competence of social workers and other professionals who work with Asian Americans may address this dissatisfaction. To date, there are few tools to help educators and supervisors evaluate the level of cultural competence of social workers and other health professionals. This study reports on the development and testing of a cultural competence measure to evaluate social workers’ preparedness for working with Asian American clients.

Method:

Based on a sample of 294 student social workers, the final 29-item instrument is a reliable and valid tool.

Results:

The instrument shows strong content validity according to expert panelists, robust reliability, excellent model fit, and preliminary evidence of construct validity.

Conclusion:

The validated Cultural Competency with Asian American Clients Scale could be used by social work educators in planning courses and assessing student readiness for work with Asian Americans.

Keywords

cultural competency Asian Americans scale development psychometric study

In the United States, there is a variety of people from different racial and ethnic groups. In particular, Asian populations are increasing in the United States. According to the U.S. Census Bureau (2011), the total population of Asian Americans increased by 46% over the past 10 years. Also, the census shows that this was the largest increase of any racial group. However, a combination of racism, language barriers, and cultural differences has an impact on Asian Americans’ quality of life, resulting in the need for engagement with health and social services (Shea & Yeh, 2008; Smedley, Stith, & Nelson, 2009).

Unfortunately, when accessing these services, Asian Americans tend to report lower service quality and satisfaction and, as a consequence, Asian Americans drop out of service programs at a higher rate than Caucasian Americans (Atkinson & Gim, 1989; Hwang, 2006; Wang & Kim, 2010). Seemingly, a lack of cultural competence on the part of service providers, such as social workers, nurses, psychologists, and doctors, plays a role in hindering Asian Americans’ access to quality health and social services, thereby exacerbating negative health outcomes (Gim, Atkinson, & Kim, 1991). Given that social workers have an important role in facilitating access to service for marginalized groups, it is essential that the professional competence in working with this population is improved.

Cultural Competence in Social Work Practice

In recent years, it has become expected of accrediting bodies and national associations (i.e., Council on Social Work Education [CSWE] and National Association of Social Workers [NASW]) that social workers receive training in cultural competence when training for social work practice (Boyle & Springer, 2001; CSWE, 2015; Lum, 2010; NASW, 2007). The CSWE’s 2015 Educational Policy and Accreditation Standards show that social workers are required to acknowledge in their practice the understanding that differences in culture, race, gender, language, immigrant status, political perspective, and additional factors can affect a person’s life journey. This understanding is an important component of cultural competence, which is defined as “having the belief, knowledge, and skills necessary to work effectively with individuals different from one’s self; that cultural competence includes all forms of difference” (Krentzman & Townsend, 2008, p. 8). The definition incorporates key concepts (e.g., belief, knowledge, and skills) that have been relevant to previous cultural competence measures (e.g., Sue, Arredondo, & McDavis, 1992).

Many researchers point out that the definition of cultural competence is ambiguous because it is stated in broad terms (Boyle & Springer, 2001; Gallegos, Tindall, & Gallego, 2008). Boyle and Springer (2001) highlight that focusing on the framework of cultural competency in general terms does not always equate to attaining skills when working with particular groups. Hence, some evidence points to the positive outcome of training social workers using generic principles within a cultural competency framework, which practices reflexivity, and takes into account the systemic oppression across racialized groups. Some population-specific competence may serve to increase social workers’ capacity to reach marginalized populations who experience challenges accessing health and social services (Boyle & Springer, 2001). Knowledge and skills aimed at working with particular groups such as Asian Americans appears warranted although currently lacking.

An abundance of measures designed to assess cultural competence for direct service providers exist. Furthermore, these measures have been applied at different levels, such as service providers, service organizations, and consumers (Bernal & Froman, 1987; Kelley & Meyers, 1995; Krentzman & Townsend, 2008; Kumas-Tan, Beagan, Loppie, MacLeod, & Frank, 2007; Rooda, 1993). According to Boyle and Springer (2001), many people utilize the Cross-Cultural Counseling Inventory–Revised (LaFromboise, Coleman, & Hernandez, 1991); the Multicultural Counseling Awareness Scale–Form B (Ponterotto, Sanchez, & Magids, 1991); the Multicultural Awareness, Knowledge, and Skills Survey (MAKSS; D’Andrea, Daniels, & Heck, 1991); and the Multicultural Counseling Inventory (Sodowsky, Taffe, Gutkin, & Wise, 1994) as cultural competence instruments. These instruments were developed for counseling and apply to most human service fields, such as nursing, medicine, and psychology. These instruments, however, do not consider specific cultural perspectives for working with particular minority groups.

Addressing the lack of specific cultural perspective of minority groups, Lum (2003) developed the Social Work Cultural Competencies with Diverse Groups of Color and Social and Economic Justice Pretest/Posttest, which considers the perspective of diverse groups, such as First Nations peoples, European Americans, African Americans, Latinos, Asians, Muslims, women, and so on. However, Krentzman and Townsend (2008) indicate that this scale did not report reliability and validity. Also, the scale provides only eight questions for use by social workers with Asian American clients. The eight questions used by this scale might not adequately measure cultural competence for social workers who work with Asian American clients because the items are very general and are not tailored adequately to the culture. For example, 1 item states, “I understand the broad and varied social service needs of specific Asian American and Pacific Islander groups”; the respondent then selects a Likert-type answer with four options ranging from unlikely to definitely. A researcher could insert any type of ethnic or minority group into that sentence. The question does not reflect the values and needs of the various Asian groups.

Building on the important work of Lum (2003), the following study aims to develop and test a measure of social work competency for social work practice with Asian American populations. The scale is validated with current social work students who are in the Bachelor of Social Work (BSW) and the Master of Social Work (MSW) programs. The validated scale could be used by social work educators in planning courses and assessing student readiness for work with this population.

Method

Scale Development

Cultural competence is defined as “having the belief, knowledge, and skills necessary to work effectively with individuals different from one’s self; that cultural competence includes all forms of difference” (Krentzman & Townsend, 2008, p. 8). Krentzman and Townsend (2008) adopted this view of cultural competence from Sue and colleagues (1982, 1992) and the NASW (2001). Their definition incorporates key concepts (e.g., belief, knowledge, and skills) that have been relevant to previous cultural competence measures (e.g., Sue et al., 1992). There are several components of cultural competence:

awareness of one’s own heritage, values, limitation, and biases, as well as respect and sensitivity to differences [beliefs] … understanding of sociopolitical history of minorities in the U.S [knowledge] … [and] the ability to send and receive information verbally and nonverbally to understand and be understood, in a variety of cultural styles [skills]. (Krentzman & Townsend, 2008, p. 9)

Among the three domains, knowledge and skill domains were used as the framework for this current study. The beliefs domain was not utilized because people’s beliefs are not easily verifiable. Also, knowledge and skills can be transmitted or taught, but beliefs are often difficult to change. Although the knowledge and ability domains were incorporated from Krentzman and Townsend, this current study considered cultural competence for working with Asian American clients. The definition of cultural competency was modified to apply to social work professionals who work with Asian Americans. The definition of cultural competency utilized in this study is that a culturally competent social worker has the knowledge and skills needed to respect a client’s own values as well as cultural background in order to provide adequate help to the client. In light of this conceptualization, 36 items were developed to reflect two domains of cultural competence: (1) cultural knowledge (defined as a social worker’s understanding of Asian American culture, including individual, family, and community-level systems) and (2) cultural skills (defined as a social worker’s ability to apply knowledge or take action in professional practice relationships with Asian American clients, including assessment, communication, and/or referral).

To measure cultural skills and cultural knowledge, items were developed by reviewing the literature on Asian American cultural values, beliefs, preferences, and the more general cultural competence literature for content relevant to each of the domains. For these two domains, items were made to allow respondents to indicate their agreement or disagreement with item statements on a 7-point Likert-type scale. In the original item pools, the global Cultural Competency with Asian American Clients (CCAAC) Scale totaled 36 items with two intended subscales: Cultural Skills and Cultural Knowledge. Following the psychometric analyses detailed below, the final instrument consisted of 29 items. After reverse coding specified items, a global CCAAC score can be calculated by averaging responses to all scale items. Scores for individual subscales can be calculated by averaging subsets of item responses with the cultural skills and cultural knowledge domains. High scores on the new scale identify social workers who believe they have a higher cultural competence while working with Asian American clients.

Content Validation

To examine the content validation of the CCAAC Scale, expert panelists were consulted. The expert panelists were purposely recruited from the United States and consisted of social workers as well as professors in New York, Boston, San Francisco, and Tallahassee. The expert panelists were recruited if they met at least one of the three criteria: (1) experience working as a social worker with Asian American clients, (2) familiarity with Asian American culture, and (3) methodological experience in developing and testing new measurement instruments. Specifically, six of the eight panelists were social workers who provided services to Asian American clients in community agencies and were familiar with Asian American culture. The remaining two panelists were professors who were experts in developing and testing new measurement instruments.

All eight invited potential panelists agreed to participate in the reviewing of the proposed instrument. Reviewers received an e-mail containing both the proposed instrument and the cover letter with instructions on how to give feedback. For example, the reviewers were asked “in order to guide understanding as to how these items fit within the proposed domains, please read the domain definition and then rate your selection, from 1 (poor) to 5 (excellent), to indicate how well each items fits within that group.” Given these instructions, the principal investigator received feedback from seven of the eight expert panel reviewers within the 1-week period.

Quantitative feedback from the expert panelists was analyzed by obtaining the mean score of each item on the proposed instrument. If the mean score was above 3.0, the items were kept in the final instrument. If the mean score was below 3.0, the qualitative feedback from the reviewers was examined and a decision was made to either revise the item or to remove it altogether. All 36 items had a mean score of three or greater and were retained for the proposed instrument.

However, qualitative feedback from panelists suggested slight revisions for some items under the cultural skills and the cultural knowledge sections. For example, under the cultural knowledge section, reviewers recommend using the statement “Asian Americans tend to highly value education” rather than original item “Asian American adults tend to be highly educated.” This original item might be true in the second and third generations. But, many older adults tend not to be highly educated, so they want their children to have higher education. This was also supported by two reviewers who work at the Asian American community centers in New York.

Sampling and Data Collection

Following approval by the institutional review board, a nonprobability sample of both bachelor’s and master’s students enrolled in face-to-face social work courses at a major southeastern public university in the United States were recruited. Prior to the start of the study, 20 professors who taught social work courses were approached by the principal investigator in person or by e-mail to obtain permission to address their students. If professors were willing, a day for such administration was scheduled.

For those recruited in person, the principal investigator attended social work classes, in which the principal investigator briefly introduced the study and the process of informed consent to the students. In an effort to ensure that potential participants understood that their anonymity was ensured, the portion of the consent form that discussed anonymity was verbally highlighted. In addition, the principal investigator asked the participants to take the survey only once because some of participants took several social work classes, so they were likely to take the survey more than once.

The instrument consisted of the following components: the newly developed CCAAC Scale, demographic items, and three additional standardized instruments, as detailed below. The Asian American Values Scale (AAVS; Kim, Li, & Ng, 2005), the MAKSS (D’Andrea et al., 1991), and the Multidimensional Scale of Perceived Social Support (MSPSS; Zimet, Dahlem, Zimet, & Farley, 1988) were administered with the newly developed CCAAC items for testing of evidence of convergent and discriminant validity. For maximum usability, the pools of items were written at a Flesch–Kincaid reading level between seventh and eighth grade.

Convergent and Discriminant Validity Evidence

To examine evidence of discriminant construct validity of newly developed CCAAC Scale, the study utilized Zimet, Dahlem, Zimet, and Farley’s (1988) MSPSS. This validated scale reports high reliability (Cronbach’s α = .88) for the total score. As a measure of discriminant validity, it was hypothesized that cultural competency—cultural skills and cultural knowledge (CCAAC)—should not correlate with social support. Cultural competency was related to a social worker’s knowledge and clinical skills. These were abilities that should not be affected by the amount of physical or emotional support a social worker has available to them. Additionally, no relationships between social support and cultural competency have been identified in the research literature. For this reason, MSPSS was an appropriate discriminant indicator for newly developed CCAAC Scale.

The AAVS-Multidimensional (AAVS-M) was selected and administered to analyze evidence of convergent construct validity of the newly developed Cultural Knowledge subscale. This validated scale has similar content of Asian American cultural knowledge as the newly developed CCAAC Scale and reports high reliability (Cronbach’s α = .89) for the total score. Concurrent and discriminant validity were also established (Kim et al., 2005). However, the AAVS-M scale focused on Asian values and help-seeking attitudes by Asian Americans rather than cultural competency for social workers who work with Asian American clients (as the newly developed CCAAC Scale does). Despite the difference in intended population and purpose of the study, the AAVS-M items served as an adequate approximation of convergent validity for the newly developed CCAAC Scale. Specifically, it was hypothesized that a high score on the AAVS-M Scale should positively correlate with a high score on the Cultural Knowledge subscales of the newly developed CCAAC Scale.

The MAKSS was also selected to analyze evidence of convergent construct validity of the newly developed Cultural Skills subscale. The MAKSS captured similar constructs of multicultural competencies (Multicultural Counseling Awareness, Multicultural Counseling Knowledge, and Multicultural Counseling Skills.) as the newly proposed Cultural Skills subscale. Also, the MAKSS Scale reports high reliability among most subscales (Cronbach’s α = .75 for Multicultural Counseling Awareness, .90 for Multicultural Counseling Knowledge, and .96 for Multicultural Counseling Skills; D’Andrea et al., 1991). However, the aim of the MAKSS examined general multicultural competencies for counselors, while the newly proposed subscale focuses on competencies specifically related to working with Asian American clients. The MAKSS’s Multicultural Counseling Skills subscale is considered to be an appropriate convergent indicator for the newly proposed CCAAC Scale because this subscale examined skills that can be used with clients of various cultures. Specifically, it was hypothesized that a high score on the Multicultural Counseling Skills subscale should positively correlate with a high score on the Cultural Skills subscale of the newly developed CCAAC Scale. Furthermore, a single item indicator of cultural knowledge was selected to examine evidence of convergent construct validity of the knowledge subscale of the CCAAC. It was hypothesized that a higher score on the Cultural Knowledge subscale of the CCAAC should positively correlate with a high score on single item indicator of cultural knowledge. Finally, a single item indicator of cultural skills was selected and administered to analyze evidence of convergent construct validity of the skills subscales of the CCAAC. It was hypothesized that a higher score on the Cultural Skills subscale of the CCAAC should positively correlate with a high score on single item indicator of cultural skill.

Data Analysis

First, data were analyzed for data cleaning and violations of assumptions of skewness and kurtosis by using SPSS 20.0. Second, a missing values analysis was performed for the newly developed CCAAC items. However, there was no missing data. Third, the principal investigator was assessed item responses for internal consistency using coefficient α, calculations score of stratified α for the CCAAC, assessing α-if-item-deleted statistics, and calculations score of standard errors of measurement (SEMs) for the CCAAC. Fourth, by using LISREL (8.8) statistical software package, factor structure using confirmatory factor was analyzed to test the hypothesized model fit based on creations that included the χ², χ²/df ratio, the comparative fit index (CFI), the Tucker–Lewis index (TLI), the root mean square of approximation (RMSEA), and the standardized root mean square residual (SRMR). Finally, bivariate correlations were examined to test the convergent and discriminant construct validity hypotheses by using SPSS 20.0.

Results

Sample Demographics

Among a total number of 294 participants, the majority of participants were female (n = 256, 87.1%) and White/Caucasian (n = 195, 66.3%). As can be seen in Table 1, participants had a mean age of 23 (SD = 5.90) and ranged in age from 18 to 65. Social work student respondents were working toward bachelor’s (n = 172, 58%) and master’s (n = 122, 41%) degrees. Seventy-eight student respondents had clinical social work experiences and reported a mean of 3.55 months (SD = 12.17) of experience with a range of 1–120 months. Furthermore, 244 participants had social work–related volunteer services and reported a mean of 19.9 months (SD = 26.33) with a range of 1–180 months. Only 18% of the participants (n = 53, SD = 3.9) indicated working experience with Asian American clients.

Table 1.

Demographics of Sample.

	Mean (SD)	Frequency	%
Age	23 (5.90)
18–22 Years		184	62.6
23–30 Years		80	27.2
Over 30 Years		30	10.2
Gender
Male		38	12.9
Female		256	87.1
Race/ethnicity
African American/Black		40	13.6
Asian American or Pacific Islander		6	2.0
Hispanic or Latina/Latino		34	11.6
American Indian or Alaskan native		4	1.4
White/Caucasian		195	66.3
Biracial		12	4.1
Other		3	1.0
Current degree
Bachelor’s		172	58.5
Master’s		122	41.5
Year in current degree program
1st Year		99	33.7
2nd Year		50	17.0
3rd Year		75	25.5
4th Year		53	18.0
Other		17	5.8
Experiences taking cultural competency course or training
Yes		157	53.4
No		137	46.6
Clinical social work experience	3.55 (12.17)
Yes		78
1–6 months		41	13.9
7–12 months		21	7.1
Over 12 months		16	5.4
No		216
Social work–related volunteer services	19.90 (26.33)
Yes		244
1–6 months		81	27.6
7–12 months		53	18.0
Over 12 months		110	37.4
No		50
Experiences working with Asian American clients
Yes		53	18.0
No		241	82.0

Note. N = 294.

Participants reported a high degree of knowledge on Asian American Value (AAVS-M mean score = 4.52, SD = 0.58, where 7 = strongly agree) and receiving a high level of social support (MSPSS mean score = 6.13, SD = 0.82, where 1 = very strongly disagree and 7 = very strongly agree). Also, participants reported a high level of multicultural knowledge (MAKSS mean score = 2.78, SD = 0.46, where 4 = very good). For the CCAAC subscales and global scale, detailed in the following analyses, scale scores were high. Respondents’ mean scores (where 1 = strongly disagree and 7 = strongly agree) for cultural skills were 5.86 (SD = 0.55), for cultural knowledge were 5.54 (SD = 0.37), and for global were 5.68 (SD = 0.35). In this sample, there were no missing data excepting two standardized scales on MSPSS and MAKSS. Since the MSPSS and the MAKSS were utilized only for construct validity, these analyses had different sample sizes.

Initial Reliability Estimates

Cronbach’s coefficient α was computed based on the initial 36 items to evaluate the internal consistency of responses within subscales as well as for the hypothesized global score. The initial α coefficient obtained for cultural skills was .781 and for cultural knowledge was .730. The global Cronbach’s α was .784. A score of .70 and above indicates acceptability. However, in an effort to improve the reliability among items within subscales, α-if-item-deleted coefficients for each item were examined to see if elimination of any items would improve internal consistency of the domain (see Table 2). Ultimately, based on both reliability and factor analytic results, Items 4, 9, and 16 from the Cultural Skills subscale and Items 19, 23, 32, and 34 from Cultural Knowledge subscale were removed from the original 36 item pool, yielding final reliability estimates on the 29-item pool.

Table 2.

α-If-Item-Deleted for Original (36 Item) and Final (29 Item) Pools.

Cultural Skills				Cultural Knowledge
Item	Original α-if-item-deleted	Revision 1 α-if-item-deleted	Final α-if-item-deleted	Item	Original α-if-item-deleted	Revision 1 α-if-item-deleted	Final α-if-item-deleted
1	.762	.823	.837	17	.711	.753	.757
2	.755	.818	.831	18	.725	.768	.772
3	.773	.832	.845	19	.731	.774^a	—
4	.823^a	—	—	20	.706	.748	.753
5	.768	.827	.840	21	.713	.761	.764
6	.768	.827	.840	22	.718	.759	.763
7	.760	.820	.834	23	.736^a	—	—
8	.760	.821	.836	24	.710	.753	.756
9	.781	.848^a	—	25	.720	.761	.765
10	.761	.822	.838	26	.703	.748	.751
11	.764	.830	.844	27	.713	.758	.761
12	.767	.827	.840	28	.722	.765	.768
13	.759	.820	.835	29	.719	.763	.767
14	.762	.823	.836	30	.707	.756	.760
15	.757	.818	.833	31	.707	.752	.756
16	.783^a	—	—	32	.734^a	—	—
SEM = .24	α = .781	α = .836	α = .843	33	.718	.761	.764
				34	.752^a	—	—
				35	.720	.765	.769
				36	.714	.760	.763
				SEM = .21	α = .730	α = .770	α = .774

Note. SEM = standard errors of measurement.

^aDenotes that item removed to improve α.

Initial Factorial Validity

By using LISREL (8.8) statistical software package, a confirmatory factor analysis (CFA) was run. Thirty-six observed and two latent variables were entered into the path analysis. The model had χ² value of 1,052.54 (p < .01), which indicated poor model fit. However, χ² was impacted by sample size, so a χ²/df ratio provided another consideration in assessment of model fit. An acceptable χ²/df ratio should be below two or three, suggesting acceptable model fit (Abell, Springer, & Kamata, 2009). Within the 36-item model, a χ²/df ratio of 1.77 indicated acceptable model fit (see Table 3).

Table 3.

Model Fit Indices for Original and Revised Item pools.

Parameter	Original 36-Item Pool	29-Item Pool	29-Item Pool (Respecified)
χ²	1,052.54***	723.06***	527.97***^a
df	593	376	364
χ²/df	1.77	1.92	1.45
RMSEA	0.054	0.058	0.040
SRMR	0.073	0.065	0.059
CFI	0.90	0.92	0.96
TLI	0.89	0.91	0.96

Note. TLI = Tucker–Lewis index; SRMR = standardized root mean square residual; RMSEA = root mean square of approximation; CFI = comparative fit index.

^aPermitting seven error covariance.

***p < .001.

Furthermore, other fit indices were examined, including the CFI, the TLI, the RMSEA, and the SRMR. To be specific, the critical values for CFI and the TLI should be above .90, which indicates acceptable, or above .95 indicating excellent fit. This model for CFI (.90) indicated acceptable model fit. However, for the obtained TLI (.89) fell below this criterion. Furthermore, the RMSEA should be ≤.05, indicating good fit or ≤.08 indicating acceptable. The model for RMSEA score was .054, proving acceptable. The SRMR value should be considered less than .10, suggesting good model fit. The initial model for SRMR score was .0073, indicating good fit. Overall, the initial model for χ²/df ratio, CFI, RMSEA, and SRMR met these standards. However, the TLI (.89) indicated a stronger degree of fit was desirable (see Table 3).

Prior to respecifying the factor model, the factor loadings of the 7 items (4, 9, 16, 19, 23, 32, and 34) were inspected because these items were flagged as problematic based on α-if-item-deleted statistics. Using a liberal cutoff > .3, which indicated the problematic items (Kline, 1994), factor loadings for the following item scores failed to meet these criteria: Item 4 (−0.28), Item 19 (0.08), Item 23 (0.09), Item 32 (0.20), and Item 34 (−0.01). Although both Item 9 (0.44) and Item 16 (0.47) met these criteria, these items were flagged as problematic by expert panelists assessing content validation (i.e., qualitative feedbacks). At this stage, based on the potential improvements in internal consistency and model fit, it was determined that all 7 items were to be removed, resulting in a 29-item pool.

Final 29-Item Reliability

After removing 7 items, the revised 29-item pool yield improved α coefficients for both subscales (Cultural Skills α = .848; Cultural Knowledge α = .774), and the final global Cronbach’s α was. 822. However, because α may underestimate reliability, stratified αs were computed (Abell et al., 2009). Stratified α for the final 29-item pool was .843. To estimate the range of true score, SEM was also used. The SEM statistics for each subscale (Cultural Skills SEM = .24; Cultural Knowledge SEM = .21) and for the global scale (SEM = .17) meet the criterion threshold of < 5% (0.30) of the possible CCAAC item response range from 1 to 7.

Final 29-Item Pool Factorial Validity

After reduction yielding a 29-item pool, a CFA was run in LISREL 8.8. Twenty-nine observed and two latent variables (13 items on the skills, 16 items on the knowledge) were entered into the path analysis. The model had χ² value of 723.06 (p < .01). A χ²/df ratio of 1.92 indicated acceptable model fit (Table 3). Furthermore, CFI (.92) and TLI (.91) scores for this model indicated acceptable model fit. RMSEA score for the model was .058, proving acceptable. The SRMR score was .065, indicating good fit. Overall, the fit indices of the model with a 29-item pool did meet these standards without further modification. However, allowing the addition of the seven theoretically consistent error covariances within the respecified model improved model fit by all indicators (see Figure 1). For the final factor model, the respecified values, CFI (.96), TLI (.96), RMSEA (.04), SRMR (.05), χ² (527.97), and χ²/df ratio (1.45), all indicated excellent model fit (see Table 3).

Figure 1.

Final respecified 29-item path analysis of the Cultural Competency with Asian American Clients depicting factor loadings, errors, and correlated errors.

Evidence of Construct Validity: Convergent and Discriminant

In order to examine the validity of the CCAAC, correlations between the standardized scale and the newly developed global and subscale scores were examined. To examine the convergent construct validity evidence for the CCAAC, we utilized AAVS-M, the MAKSS, and two single item indicators capturing construct definition. These analyses provided evidence in support of the convergent construct validity of the CCAAC.

First, we hypothesized that a higher score on the AAVS-M was significantly correlated with respondents’ knowledge score on the CCAAC. The resulting correlation was statically significant for the Knowledge subscale (r = .31, r ² = .10), which provided evidence of convergent validity. Second, a higher score on the Cultural Skills subscale of the MAKSS was significantly correlated with the respondents’ skills score on the CCAAC. The correlation was statistically significant (r = .26, r ² = .07), which provided evidence of convergent validity. Third, a higher score on the Cultural Knowledge subscale of the CCAAC was significantly correlated with respondents’ score on the single item indicator of cultural knowledge (I understand Asian American culture, including individual, family, and community-level system). This hypothesis was not supported through the data (p = .211, r = .08). Also, small effect sizes (r ² = .02) between the Cultural Knowledge subscale of the CCAAC and a single item indicator of cultural knowledge indicated weak evidence for this validity (Cohen, 1988). Fourth, as hypothesized, a higher score on the Cultural Skills subscale of the CCAAC was significantly correlated with a higher score on the single item indicator of cultural skills (I have culturally appropriate skills to apply knowledge or take action in professional practice relationships with Asian American clients). The resulting correlation was statistically significant (r = .18, r ² = .04), which provided evidence of convergent validity.

On the other hand, in order to examine the discriminant construct validity of the CCAAC, we hypothesized that the MSPSS would not be significantly correlated with the Global CCAAC Scale scores. This hypothesis was supported through the data (r = .14, r ² = .02) in discriminant validity. Overall, results provided preliminary evidence of construct validity for the CCAAC. The magnitudes of these relationships are indicated in r ² statistics (effect size) located in Table 4.

Table 4.

Convergent and Discriminant Construct Validity Evidence for CCAAC.

	Variable	n	Cultural Skills		Cultural Knowledge		Global
	Variable	n	r	r ²	r	r ²	r	r ²
Convergent	MAKSS skills	293	.26**	.07	—	—	—	—
	A single item for Cultural Skills	294	.18**	.04	—	—	—	—
	Mean		.22	.055
	AAVS-M	294	—	—	.31**	.10	—	—
	A single item for Cultural Knowledge	294	—	—	.08	.01	—	—
	Mean				.195	.055
Discriminant	MSPSS	293	—	—	—	—	.140	.02

Note. MAKSS = The Multicultural Awareness, Knowledge, and Skills Survey; AAVS-M = Asian American Values Scale–Multidimensional; MSPSS = Multidimensional Scale of Perceived Social Support; CCAAC = Cultural Competency with Asian American Clients.

Discussion

Psychometric Summary

The objective of this study is to develop a reliable and valid measure of cultural competence for working with Asian American clients. This initial validation provides evidence that the CCAAC is a reliable and valid tool for measuring social workers’ CCAAC. To be specific, the instrument shows strong content validity according to the expert panels, robust reliability, excellent model fit, and preliminary evidence of construct validity. Furthermore, as a rapid assessment instrument, the CCAAC can be scored by computing the mean of item responses for global and subscales (Cultural Skills and Cultural Knowledge) and is easily interpretable using the same 7-point range (from 1 = strongly disagree to 7 = strongly agree) incorporated in its response options (see Figure 2). The CCAAC items were written at a Flesch–Kincaid reading level between seventh and eighth grade, indicating that the scale should be readily accessible for the intended users.

Figure 2.

Final version of the Cultural Competency with Asian American Clients (CCAAC) Scale with instructions for response.

Applications for Social Work Education and Practice

The CCAAC Scale fills a gap in the literature by providing a tool for assessing social workers’ cultural competency when working with Asian American clients. Previous cultural competency scales did not consider specific Asian cultural components. By including skills and knowledge that were based on Asian American cultural components, the CCAAC Scale will be useful to both social work students and social work professionals. It is necessary for social workers to check their abilities to offer culturally competent services to Asian American clients, as it is vital for social workers to be culturally competent. The CCAAC Scale can be used alongside other culturally specific and more global cultural competency scales to assess overall readiness for practice with marginalized groups.

Although the CCAAC Scale will be useful for social workers and social work students alike, social work students were the target population in the current study for the initial validation of the CCAAC Scale. The CCAAC Scale could be validated for social work professionals who provide services to Asian American clients by obtaining a sample of professionals. In the future, the CCAAC Scale might also provide a valuable self-evaluation tool among social work professionals who want to improve their competence and respect their clients’ values and diversities.

Furthermore, the CCAAC Scales can be utilized as a suitable tool to measure the outcomes of social work educational programs that include cultural competence education for working with Asian Americans. Social work curriculums highlight diversity and difference to train students to be social workers who are culturally competent and offer sensitive practice to clients from different ethnic and racial groups (Boyle & Springer, 2001). In social work education and classes, social work students might learn knowledge about different cultural groups; for example, the distinctive cultural heritage of the various Asian American groups that can be applied to actual clients in practice settings. By using the CCAAC Scale, social work students can assess their cultural competence for working with Asian American clients. Although the CCAAC Scale tried to include common cultural values, attitudes, and physical characteristics with regard to Asian Americans, this group does not constitute a monolith. Each Asian American culture has its own characteristics and social workers must be mindful of that. With the aid of the CCAAC Scale, instructors and social work students should first consider this cultural plurality and then provide effective services and treatments to their Asian American clients.

In social work agencies, based on the results of the CCAAC Scale, supervisors in social work agencies can provide suitable educational guidelines for social work students. For example, if the results of the CCAAC Scale show that social work students do not know about some aspects of Asian American culture, supervisors can provide additional education about Asian culture before students begin providing service to Asian American clients.

Limitations and Recommendations for Future Research

Although the initial CCAAC validation study shows promising results and contributes to filling the gap in the literature, the study has limitations that should be considered for future studies. First, while four of the five convergent and discriminant validity hypotheses were supported in this study, one hypothesis failed. There were no significant correlations in the Cultural Knowledge subscale of the CCAAC score based on a single item indicator of cultural knowledge. The Culture Knowledge subscale included individual, family, and community content based on Asian culture. However, a single item did not provide sufficient explanation to participants in the survey. From this perspective, participants might not think that these two scales are compatible, so that they might have different scores in each scale. Further study should carefully consider using a single item to asses construct validity. Also, unique hypotheses should be used as validity indicators.

The second limitation is that even though most convergent and discriminant validity hypotheses were supported in this study, the effect sizes were quite small (from .01 to.10). The effect sizes indicate that the relationships between the CCAAC Scale and standard scales were weak.

The third limitation is that only 18% of participants had working experience with Asian American clients and none of them had experience greater than 6 months. The scores of participants with experience working with Asian Americans and participants who did not have experience were compared by running a t-test. The analysis comparing the two groups found that the CCAAC scores were not statistically different between the two groups, t(292) = .361, p = .718 (working experience group: M = 5.78, SD = 0.403; no working experience group: M = 5.80, SD = 0.390). Interestingly, the two groups were not statistically different on the subscales on CCAAC as well as other standardized scales.

Future research could consider additional factors in order to effectively use the CCAAC Scale. The current study only targeted social work students in a limited area. To additionally validate the CCAAC Scale, we need to survey social work students in different settings such as both urban and rural areas or specific areas where Asian Americans are a significant demographic. In doing so, future studies could have more rigorous results. In addition, future studies should use more objective measurements for cultural competence and blend social workers’ self-reports with their clients’ reports. Such studies would result in a higher level of awareness of the differences in cultural competency. This is important as cultural competency ensures that social workers can provide effective services for Asian American clients.

Footnotes

Declaration of Conflicting Interests

The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author received no financial support for the research, authorship, and/or publication of this article.

References

Abell

Springer

D. W.

Kamata

(2009). Developing and validating rapid assessment instruments. New York, NY: Oxford University Press.

Atkinson

D. R.

Gim

R. H.

(1989). Asian-American cultural identity and attitudes toward mental health services. Journal of Counseling Psychology, 36, 209.

Bernal

Froman

(1987). The confidence of community health nurses in caring for ethnically diverse populations. Journal of Nursing Scholarship, 19, 201–203.

Boyle

D. P.

Springer

(2001). Toward a cultural competence measure for social work with specific populations. Journal of Ethnic and Cultural Diversity in Social Work, 9, 53–71.

Cohen

(1988). Statistical power analysis for the behavior sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.

Council on Social Work Education. (2015). 2015 Educational Policy and Accreditation Standards (EPAS) for Baccalaureate and Master’s Social Work Programs. Retrieved from http://www.cswe.org/File.aspx?id=81660

D’Andrea

Daniels

Heck

(1991). Evaluating the impact of multicultural counseling training. Journal of Counseling and Development, 70, 143–150.

Gallegos

J. S.

Tindall

Gallegos

S. A.

(2008). The need for advancement in the conceptualization of cultural competence. Advances in Social Work, 9, 51–62.

Gim

R. H.

Atkinson

D. R.

Kim

S. J.

(1991). Asian American acculturation, counselor ethnicity and cultural sensitivity and ratings of counselors. Journal of Counseling Psychology, 38, 57–62.

10.

Hwang

W. C.

(2006). The psychotherapy adaptation and modification framework: Application to Asian Americans. American Psychologist, 61, 702.

11.

Kelley

Meyers

(1995). The cross-cultural adaptability inventory. Minneapolis, MN: NCS Assessments.

12.

Kim

B. K.

L. C.

G. F.

(2005). The Asian American values scale–Multidimensional: Development, reliability, and validity. Cultural Diversity and Ethnic Minority Psychology, 11, 187.

13.

Kline

(1994). An easy guide to factor analysis. London, England: Routledge.

14.

Krentzman

A. R.

Townsend

A. L.

(2008). Review of multidisciplinary measures of cultural competence for use in social work education. Journal of Social Work Education, 44, 7–32.

15.

Kumas-Tan

Beagan

Loppie

MacLeod

Frank

(2007). Measures of cultural competence: Examining hidden assumptions. Academic Medicine, 82, 548–557.

16.

LaFromboise

T. D.

Coleman

H. L.

Hernandez

(1991). Development and factor structure of the Cross Cultural Counseling Inventory-Revised. Professional Psychology: Research and Practice, 22, 380–388.

17.

Lum

(2003). Culturally competent practice: A framework for understanding diverse groups and justice issues (2nd ed.). Pacific Grove, CA: Brooks/Cole.

18.

Lum

(2010). Culturally competent practice: A framework for understanding diverse groups and justice issues (4th ed.). Belmont, CA: Brooks/Cole.

19.

National Association of Social Workers. (2001). Standards for cultural competence in social work practice. Washington, DC: Author.

20.

National Association of Social Workers. (2007). Indicators for the achievement of NASW standards for cultural competence in social work practice. Washington, DC: Author.

21.

Ponterotto

J. G.

Sanchez

C. M.

Magids

(1991). Initial development of the Multicultural Counseling Awareness Scale. Paper presented at the annual meeting of the American Psychological Association, San Francisco, CA.

22.

Rooda

L. A.

(1993). Knowledge and attitudes of nurses toward culturally different patients: Implications for nursing education. The Journal of Nursing Education, 32, 209–213.

23.

Shea

Yeh

C. J.

(2008). Asian American students’ cultural values, stigma, and relational self-construal: Correlates of attitudes toward professional help seeking. Journal of Mental Health Counseling, 30, 157–172.

24.

Smedley

B. D.

Stith

A. Y.

Nelson

A. R.

(Eds.). (2009). Unequal treatment: Confronting racial and ethnic disparities in health care (with CD). Washington, DC: National Academies Press.

25.

Sodowsky

G. R.

Taffe

R. C.

Gutkin

T. B.

Wise

S. L.

(1994). Development of the Multicultural Counseling Inventory: A self-report measure of multicultural competencies. Journal of Counseling Psychology, 41, 137.

26.

Sue

D. W.

Arredondo

McDavis

R. J.

(1992). Multicultural counseling competencies and standards: A call to the profession. Journal of Counseling & Development, 70, 477–486.

27.

Sue

D. W.

Bernier

Durran

Feinberg

Pedersen

P. B.

Smith

E. J.

Vasquez-Nuttal

(1982). Position paper: Cross-cultural counseling competencies. The Counseling Psychologist, 10, 45–52.

28.

U.S. Census Bureau. (2011). Facts for features: Asian/Pacific American heritage month: May 2011. Retrieved September 5, 2013, from http://www.census.gov/newsroom/releases/archives/facts_for_features_special_editions/cb11-ff06.html

29.

Wang

Kim

B. S.

(2010). Therapist multicultural competence, Asian American participants’ cultural values, and counseling process. Journal of Counseling Psychology, 57, 394.

30.

Zimet

G. D.

Dahlem

N. W.

Zimet

S. G.

Farley

G. K.

(1988). The multidimensional scale of perceived social support. Journal of Personality Assessment, 52, 30–41.