Development and Initial Validation of Cultural Competence Inventory

Abstract

Despite the critical role of emotions in multicultural teacher education, no attempt has been made to develop an instrument including affect as a dimension in measuring cultural competence for preservice teachers. To bridge this gap, the present three-study research used three distinct samples of 456 preservice teachers to develop and estimate the reliability of scores and validity of inferences for Cultural Competence Inventory–Preservice Teachers (CCI-PT), an 18-item instrument including three subscales: emotional cultural competence (ECC), behavioral cultural competence (BCC), and cognitive cultural competence (CCC). The three-factor model was supported using exploratory factor analysis (N = 203) in Study 1 and confirmatory factor analyses (N = 199) in Study 2. Correlation results provided preliminary evidence for the convergent and discriminant validity of CCI-PT in Study 1, and regression results in Study 3 (N = 113) provided initial evidence of predictive validity of inference for the instrument. Further, the scores of cultural competence assessed through three independent studies also provided initial evidence of the reliability of CCI-PT.

Keywords

scale development/testing measurement scale validation preservice teachers multicultural education cultural competence emotional cultural competence behavioral cultural competence cognitive cultural competence

Interest in studying cultural competence in the field of teaching has mushroomed due to increasing student diversity in contrast with the relatively homogeneous teaching force (Boser, 2011; U.S. Census Bureau, 2017a, b). Teacher educators are expected to integrate multicultural education to help preservice teachers create an inclusive learning environment that ensures all learners succeed (Council for the Accreditation of Educator Preparation, 2013). To assess preservice teachers’ preparedness in creating an inclusive environment that embraces student diversity and examine the effectiveness of multicultural education, there is a need to measure cultural competence for preservice teachers. Although multicultural education literature spans nearly four decades (Banks, 1981), psychometric studies of cultural competence for teaching are still in their infancy (Prieto, 2012). Earlier attempts to develop cultural competence measures often either failed to provide essential psychometric information (e.g., Burstein, Cabello, & Hamann, 1993) or resulted in measures with psychometric properties that are less than desirable (e.g., Boyd, 2003). The more recent psychometric studies are inconclusive due to divergent factor structures and are often limited to testing factorial invariance with statistical manipulation for cross-validation instead of using different independent samples (Prieto, 2012; Spanierman et al., 2011; Yang & Montgomery, 2011). Therefore, in this research, we attempted to develop and initially validate Cultural Competence Inventory–Preservice Teachers (CCI-PT), a measure to capture cultural competence for preservice teachers, through three studies with distinct samples.

Extant Cultural Competence Measures

A systematic review of the cultural competence instrumentation literature (key search words included “cultural competence,” “cultural competency,” “diversity awareness,” “diversity beliefs,” “cultural awareness,” “multicultural teaching,” “assessment,” and “measurement” via databases including “ERIC,” “ProQuest,” and “PsycINFO”; psychometric studies in nonteaching fields such as counseling or health care were excluded) generated an exhaustive list of 18 empirical studies (see Table 1) that revealed not only the progress of instrument development and utilization since the 1980s (Henry, 1986) but also several inherent shortcomings. First, only four measurement studies aimed to measure cultural competence in working with student diversity (Boyd, 2003; Prieto, 2012; Spanierman et al., 2011; Yang & Montgomery, 2011). Most of the earlier research (e.g., Pettus & Allain, 1999; Pohan & Aguilar, 2001) only attempted to develop instruments to measure awareness, attitudes, and/or beliefs about diversity. Although attitudes and beliefs about diversity are important components of cultural competence, these instruments only measure a limited aspect of cultural competence. Second, only a handful of studies exclusively focused on preservice teachers (e.g., Artiles & McClafferty, 1998; Yang & Montgomery, 2011). Most of the prior work focused on a broad range of samples from business students (De Meuse & Hostager, 2001) and working adults (Henry, 1986) to school teachers (Dunn, Beasly, & Buchanan, 1994) and higher education faculty (Prieto, 2012). Preservice teachers may well differ from school teachers and other older age professional groups as they are in a unique preparatory stage of their career in which they receive multicultural education, take certification tests, and prepare to become classroom teachers. Third, six of the earlier measurement studies provided no psychometric information (e.g., Artiles & McClafferty, 1998; Dunn et al., 1994; see Table 1), one study only provided criterion-related validity (De Meuse & Hostager, 2001), one study performed only a secondary data analysis (which made the results irreplicable to further investigation), and six studies only relied on Cronbach’s alpha as the most notable reliability evidence (e.g., Boyd, 2003; Stanley, 1996). Only four studies (Prieto, 2012; Spanierman et al., 2011; Wang et al., 2003; Yang & Montgomery, 2011) tried to develop an instrument with known reliability of scores and validity of references (Kane, 2013). Upon a closer look at the four studies, we found that Wang and colleagues (2003) sought to measure ethnocultural empathy instead of cultural competence, Prieto (2012) focused on higher education faculty, and Spanierman and colleagues (2011) focused on both preservice and inservice teachers. As preservice and inservice teachers are at different stages of their career, it is unclear whether the factor structure would hold across the two samples. Only one study thus far (Yang & Montgomery, 2011) validated a cultural competence scale involving preservice teachers exclusively through statistical manipulation for cross-validation. However, it is unknown if the factor structure would hold across different populations. Therefore, we sought to estimate the reliability and validity for CCI-PT with three different independent populations consistent with the recommended standards (American Educational Research Association, American Psychological Association, & National Council on Measurement in Education, 2014).

Table 1.

Existing Measurement Studies and Related Cultural Competence Components.

Authors (years)	Instrument	Dimensions/domains/factors/components	Psychometric property	Sample	Cultural competence components
Authors (years)	Instrument	Dimensions/domains/factors/components	Psychometric property	Sample	Emotion	Behavior	Cognition
1. Artiles and McClafferty (1998)	Concept Map and Survey	Cognitive transformation	No information	Preservice teachers (N = 17)			X
2. Boyd (2003)	Multicultural Education and Cultural Competency Assessment	(1) Multicultural knowledge/skills; (2) multicultural awareness/consciousness	a₁ = .95; a₂ = .67	Preservice and inservice teachers (N = 162)		X	X
3. De Meuse and Hostager (2001)	Reaction-to-Diversity Inventory	(1) Emotional reactions, (2) judgments, (3) behavioral reactions, (4) personal consequences, (5) organizational outcomes	Criterion-related validity	Business students (N = 253)	X	X	X
4. Dunn, Beasly, and Buchanan (1994)	Personal Belief/Bias Scale	Teachers beliefs about how minority students learn	No information	School teachers			X
5. Henry (1986)	The Cultural Awareness Inventory	(1) Values and beliefs, (2) communication, (3) social relationships, (4) basic diet and food preparation, and (5) dress or common costume	No information	Adults dealing with school children		X	X
6. Milner, Flowers, Moore, Moore, and Flowers (2003)	Cultural Diversity Awareness Inventory	(1) General cultural awareness, (2) the culturally diverse family, (3) cross-cultural communication, (4) assessment, and (5) the multicultural environment	a = .56	Preservice teachers (N = 99)			X
7. Moore and Reeves-Kazelskis (1992)	Survey of Multicultural Education Concepts	Beliefs and attitudes about multicultural education	No information	Preservice teachers (N = 31)			X
8. Pettus and Allain (1999)	Cultural and Educational Issues Survey	Educators’ attitudes concerning cultural and educational issues	a = .92	Preservice teachers (N = 123)			X
9. Pohan and Aguilar (2001)	Personal and Professional Beliefs About Diversity Scales	(1) Personal beliefs, (2) professional beliefs	a₁ = .71-.81, a₂ = .78-.90	Preservice and inservice teachers (N = 1,941)			X
10. Ponterotto, Baluch, Grieg, and Rivera (1998)	Teacher Multicultural Attitude Survey	Multicultural awareness and sensitivity	a = .86	Preservice and inservice teachers (N = 429)			X
11. Prieto (2012)	Multicultural Teaching Competencies Inventory	(1) Acquired cultural knowledge, (2) sensitivity to student culture	CFI = .92, RMSEA = .08	Higher education faculty	X		X
12. Solomon and Levine-Rasky (1994)	Professional Development Subscale	Perspectives on multicultural and antiracist policies	No information	Inservice teachers (N = 1,000)			X
13. Spanierman et al. (2011)	Multicultural Teaching Competency Scale	(1) Multicultural teaching skill, (2) Multicultural teaching knowledge	χ²/df = 1.69,CFI = .95, RMSEA = .05	Preservice and inservice teachers (N = 506)		X	X
14. Stanley (1996)	Pluralism and Diversity Attitude Assessment	(1) Appreciate cultural pluralism, (2) value cultural pluralism, (3) implement cultural pluralism, and (4) uncomfortable with cultural diversity	a = .72-.85	Preservice teachers (N = 215)	X	X
15. Tatto (1996)	Teacher Education and Learning to Teach Study	Beliefs and values in teaching diverse students	Secondary data analysis–irreplicable	Preservice teachers (N = 817) and professors (N = 113)			X
16. Tran, Young, and DiLella (1994)	Cultural Survey	Attitudes toward ethnic groups	No information	Preservice teachers (N = 55)			X
17. Wang et al. (2003)	Scale of Ethnocultural Empathy	(1) Empathic feeling and expression; (2) empathic perspective taking; (3) acceptance of cultural differences; (4) empathic awareness	a = .73-.91,χ²/df =1.20, NFI = .98, RMSEA=.03	Undergraduate students (N = 364)			X
18. Yang & Montgomery (2011)	Multicultural Teaching Scale	(1) Knowledge, (2) praxis	χ²/df =2.50, CFI = .91, RMSEA = .06	Preservice teachers (N = 793)		X	X

Note. CFI = comparative fit index; RMSEA = root mean square error of approximation; NFI = normed fit index.

Although there have been a number of models of cultural competence (e.g., Bennett, 1993; Campinha-Bacote, 2002), the model that is most commonly used in teacher education thus far identifies three components: knowledge, skills, and awareness (Sue, 1982). However, the model was developed in counseling, a practice distinct from teaching, and neither of the two most recent psychometric study results supported the model in the context of teaching (Prieto, 2012; Spanierman et al., 2011). Prieto’s (2012) Multicultural Teaching Competencies Inventory suggested a two-factor solution: Acquired Cultural Knowledge and Sensitivity to Student Culture, whereas Spanierman and colleagues’ (2011) Multicultural Teaching Competency Scale suggested a different two-factor solution: Multicultural Teaching Skill and Multicultural Teaching Knowledge. Prieto’s study results supported cognitive (knowledge) and emotional (sensitivity) domains of cultural competence, whereas Spanierman et al. supported cognitive (knowledge) and behavioral (skill) domains which were similar to the findings of other researchers (Yang & Montgomery, 2011). Combining the three most recent study results, we proposed that cultural competence has three components: emotions, behaviors, and cognitions. Prieto (2012) concluded his study appealing for more psychometric investigations, exploring additional factors, and testing factor invariance across various samples. In light of this, the current research endeavored to further the psychometric investigation of cultural competence for preservice teachers and test factor invariance through independent samples.

As a critical component of cultural competence, cognition has been empirically documented in almost all previous study results (see Table 1), covering a wide range of cognitive cultural competence (CCC) from awareness of cultural differences, knowledge about diversity and multicultural issues (Chambers & Fischer, 2002; D’Andrea, Danniels, & Noonan, 2003), to beliefs about how student diversity should be treated in the classroom (Burstein & Cabello, 1989; Burstein et al., 1993; Tran, Young, & DiLella, 1994). Another essential component of cultural competence is behavior, which is supported by six of the study results listed in Table 1, ranging from effective communication about diversity and multicultural issues (Henry, 1986) to the actions of appreciating, valuing, and implementing diversity (Stanley, 1996). In a recent study involving 793 preservice teachers (Yang & Montgomery, 2011), praxis (i.e., multicultural action) was confirmed to be a major component of multicultural teaching competence.

Although previous literature has consistently supported the cognitive and behavioral components of cultural competence for teaching (e.g., Spanierman et al., 2011; Yang & Montgomery, 2011), an emotional component seems to be relatively neglected despite its important role (e.g., Bennett, 1993; Wang et al., 2003). A previous study on the relationship between preservice teachers’ causal attribution and cultural competence (Yang & Montgomery, 2011) suggests the importance of addressing emotions in the course of multicultural education to improve preservice teachers’ cultural competence. Positive emotions are a sign of multicultural maturity in dealing with diversity issues (Wang, 2008; Wang et al., 2003), which entails skillful identification and interpretation of the emotions of others and is considered to be a set of abilities separate from one’s own attitudes (Mayer & Salovey, 1993; Wang et al., 2003). When working with a diverse student population, the proper utilization of intercultural empathy and positive affect ensures effective communication and helps build the foundation for a good relationship. However, absence of empathy and positive affect leads to a lack of cultural sensitivity (Zhu, 2011). According to Zhu (2011), ignorance of cultural differences may lead to negative emotional reactions such as apathy, which may lead to reinforcement of prejudice and stereotypes. The importance of emotions in cultural competence is supported not only in empirical studies but also in some primary theoretical models: the process model (Campinha-Bacote, 2002) that views cultural competence as a continually evolving process, and the developmental model (Bennett, 1993) that views intercultural sensitivity as a developmental process from ethnocentric to ethnorelative stages. Both models focused on health care providers rather than teacher preparation, but they both suggested the importance of addressing emotions in acquiring cultural competence: health care providers need to develop healthy and positive emotional reactions toward client diversity. Although teaching is a distinct practice from health care services, both involve human interactions and relationships which necessitate the attention to emotions in the process (e.g., Wang, 2008; Yang & Montgomery, 2011). Thus, we believe emotions are an important component of cultural competence for preservice teachers and intentionally incorporated them in the current research.

Research Aim and Hypotheses

The present research aimed to develop a new instrument CCI-PT to capture cultural competence for preservice teachers and investigate its factorial structure and validity through three studies with independent samples. Considering the importance of emotions in cultural competence, we intentionally included emotional items in CCI-PT. Based on the three dimensions of cognition, behavior, and emotions from the literature, cultural competence for preservice teachers in the present research is defined as their ability of being knowledgeable about student diversity, developing healthy emotions toward student diversity, and engaging in actions that incorporate student diversity. To advance the instrumentation research in cultural competence for preservice teachers, we took a number of steps to overcome the earlier limitations in the present research (Kane, 2013). First of all, we developed a comprehensive inventory of cultural competence for preservice teachers by adapting items from measures of cultural competence for teaching to cover all three intended dimensions of cultural competence (cognition, behavior, and emotion) to improve confidence in the extrapolation inference of CCI-PT scores. Second, we provided psychometric property information by examining construct validities of CCI-PT through exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) and estimating reliability coefficients for internal consistency. Third, we cross-validated the CCI-PT with data collected from three independent samples of preservice teachers with different characteristics. Fourth, we tested the validity of CCI-PT in multiple ways. As past research has demonstrated relationships between cultural competence and teacher characteristics and dispositions (e.g., Chen, 2016; Ponterotto, Baluch, Grieg, & Rivera, 1998), we examined the convergent validity of the CCI-PT by examining the relationship between scores on the subcomponents of the CCI-PT and preservice teachers’ openness to experience and self-efficacy. Furthermore, we examined the criterion-related predictive validity through investigating the predictive power of cultural competence for preservice teachers and their ratings by their mentor teachers and faculty advisors, offering further information for extrapolation of CCI-PT scores (Kane, 2013).

Method

Instrument Development

Based on the systematic literature review, we intended to develop CCI-PT instrument encompassing three factors (emotion, behavior, and cognition), with each factor including five to eight items in the final version of the scale. In doing so, we tried to obtain as many items as possible in the initial item pool and select best items while going through iterative processes of measurement validation. After closely studying the previous instrumentation research of cultural competence, we compiled 98 suitable instrument items to represent cultural competence among preservice teachers addressing various emotional, cognitive, and behavioral responses to student diversity. The major instruments from which we adopted and revised the items included the Scale of Ethnocultural Empathy (SEE, Wang et al., 2003), Cultural Awareness Scale (CAS, Rew, Becker, Cookston, Khosropour, & Martinez, 2003), Modern Racism Scale (MRS, McConahay, 1986), Workplace Diversity Survey (WDS, De Meuse & Hostager, 2001), and Multicultural Teaching Scale (MTS, Wayson, 1993). Next, a panel of six experts in multicultural teacher education from several higher education institutions were asked to review the items for content relevance and clarity. Based on their detailed and informative review comments, 17 items were revised to improve clarity and nine items were excluded either due to low face validity or irrelevance to the preservice teacher population. The panel performed several iterations, resulting in a total of 89 items of the updated instrument, which was evaluated with a total of 456 preservice teachers through three studies.

Participants and Procedure

In Study 1, data from 203 preservice teachers in a teacher preparation program at a large land-grant Midwestern comprehensive public university were collected. The sample was predominantly Hispanic (85%), female (80%), and senior-level education majors (62%). Participants were provided with a link to an online survey, which consisted of CCI-PT and other measures including Teacher Efficacy Scale (Tschannen-Moran & Woolfolk Hoy, 2001) and Openness to Experience subscale from the five-factor model (FFM, Costa & McCrae, 1992) to measure convergent and discriminant validity, and Marlowe–Crowne Social Desirability Scale (Reynolds, 1982) to measure social desirability of CCI-PT as a self-report survey (Crowne & Marlowe, 1960). A 7-point response scale was used for all the measures ranging from 1 (strongly disagree) to 7 (strongly agree).

In Study 2, 199 preservice teachers from a large southeastern university took the survey of CCI-PT together with demographics. Unlike the Hispanic dominant sample in Study 1, the participants in Study 2 were mostly White (69%) and African American (23%), although it is still predominantly female (84%), which is typical of the teaching profile in the United States. Another demographic difference between the two samples is that the Study 2 sample was more diverse spanning from junior second semester (54%) to senior education majors (35%).

Study 3 involved 113 participants, including 63 preservice teachers (all females; nine were excluded from the study due to missing data using listwise deletion method) and five faculty supervisors from a southcentral private Christian university, and 54 mentor teachers in five schools where the preservice teachers taught classes (kindergarten to fifth grade) for their internship for one semester. The five university faculty supervisors taught content courses and observed the field teaching of the preservice teachers several times, and the mentor teachers coached and worked with preservice teachers on a one-on-one basis throughout the semester. Preservice teachers completed the measures of CCI-PT in a classroom setting at the start of a semester. The supervisors and mentor teachers were given a link to an online survey containing six items to evaluate the multicultural teaching performance of the preservice teachers at the end of the semester. They responded to statements such as “This intern teacher has demonstrated cultural competence in his or her teaching” on a 1 (strongly disagree) to 7 (strongly agree) response scale.

Results

The present research utilized a three-step approach for the development and initial validation of the proposed instrument to provide multiple sources of generalization and extrapolation inferences of CCI-PT (Kane, 2013). First, EFA was performed to test the factor structure of CCI-PT in Study 1. Second, the factorial structure was cross-validated through CFA in Study 2. Third, regression as well as correlation analyses were performed to provide additional validity evidence including criterion-related validity in Study 3.

EFA

The Kaiser–Meyer–Olkin (KMO) measure of sampling adequacy (.84) and significant Bartlett’s test of sphericity (p < .001) indicated that the sample was appropriate for EFA. We performed principal component analysis with varimax rotation using SPSS 22. We examined two- and three-factor solutions, which both met the eigenvalue criteria (i.e., eigenvalue > 1.0; Kaiser, 1960). An examination of the scree plot suggested a three-factor solution. We examined item communalities and deleted 10 items below .20 from the solution (Worthington & Whittaker, 2006). The decision to drop items was based on statistical analysis results and thorough reviews of item content (Costello & Osborne, 2005). Cross-loading and nonsignificant loading items were sequentially removed until a simple structure was reached. Furthermore, we manually removed items that loaded on the other factors. For example, a supposedly cognitive item “I believe that students from other cultures are getting too demanding in their push for equal rights” loaded on a factor that was predominantly represented by emotional items. A closer look at the item revealed its confusing nature as it could be more relevant to emotions given the statement “getting too demanding” than to cognition, hence its removal from the pool. The final factor solution was based on multiple criteria including a minimum of three items per factor, sufficient internal consistency, interpretability of each factor, consistency with our conceptualization of cultural competence for preservice teachers (Tabachnick & Fidell, 2007), as well as parsimony considerations (Netemeyer, Bearden, & Sharma, 2003) while ensuring comprehensiveness of the inventory. The final CCI-PT was reduced to an 18-item scale with six items loaded on each of the three factors accounting for 52.85% of the total variance (see Table 2 for final factor loadings). Although the item reduction was significant in our study, it was not unusual, given the majority of the psychometric study results losing more than 50% of the initial item pool (Morgado, Meireles, Neves, Amaral, & Ferreira, 2017).

Table 2.

Exploratory Factor Analysis Results (N = 203).

Items	Cultural competence
Items	ECC	CCC	BCC
^aI feel impatient when communicating with students from other racial or ethnic backgrounds.^R	.81
I often feel irritated by students of a different culture.^R	.77
I only feel at ease with people from my culture.^R	.72
Student diversity is stressful for me.^R	.72
I feel frustrated by student diversity.^R	.71
I feel annoyed when students do not speak standard English.^R	.62
^aKnowing about the different experiences of students improves my understanding of my own problems.		.77
Knowing students from different ethnic groups improves my understanding of myself.		.74
Knowing about the experiences of students of different cultures increases my self-understanding.		.72
Knowledge of how a student differs from me greatly enhances our friendship.		.70
Understanding how a student is both similar and different from me helps me get to know him or her.		.67
I think students from other cultures can teach me things I could not learn elsewhere.		.60
^aI help students work through problem situations caused by stereotypical attitudes toward culture differences.			.75
I assist all students to understand the perspectives of people from other ethnic groups.			.70
I effectively utilize ethnic resources in the community.			.69
I investigate how language difference affects performance on certain test items.			.68
I present diversity of cultures as a strong positive feature of American heritage.			.63
I seek out cultural biases in materials used in instruction.			.45
% of Variance	28.70	14.00	10.15
Cronbach’s alpha	.81	.82	.75
Means/standard deviations	5.92/.97	4.42/.58	5.23/.83

Note. ECC = emotional cultural competence; CCC = cognitive cultural competence; BCC = behavioral cultural competence.

Items with the highest loadings on respective factors.

Reverse coded.

Items loaded on Factor 1 reflect preservice teachers’ feelings toward student diversity suggesting their emotional cultural competence (ECC) when working with students from diverse cultures. For example, one item asks participants to respond to the statement “I feel impatient when communicating with students from other racial or ethnic backgrounds.” Items loaded on Factor 2 suggest preservice teachers’ awareness and knowledge about student diversity representing their CCC. For example, one item asks participants to respond to the statement “Knowing about the different experiences of students improves my understanding of my own problems.” Items on Factor 3 reflect preservice teachers’ actions and practices associated with student diversity representing their behavioral cultural competence (BCC). For example, one item asks participants to respond to the statement “I help students work through problem situations caused by stereotypical attitudes toward culture differences.” The internal consistency coefficients and 95% confidence intervals for the three factors were all acceptable (Cortina, 1993): ECC (α = .81), 95% confidence interval = [.79, .87]; CCC (α = .82), 95% confidence interval = [.76, .85]; and BCC (α = .75), 95% confidence interval = [.68, .81].

The item statistics of the finalized CCI-PT is presented in Table 3. Given the size of our sample, we chose a traditional method for calculating the item discrimination index (d) (Panjaitan, Irawati, Sujana, Hanifah, & Djuanda, 2018). Scores indicated that all items discriminated between the highest scorers and lowest scorers on the scale (all d’s > .0, see Table 3). The lowest d came from Item CCC₆ (d = .12), but it still had a strong item-total correlation and factor loading, so we chose to retain the item in the final scale.

Table 3.

Item Statistics of CCI-PT (N = 203).

Items	M	SD	CITC	d
ECC₁: I feel impatient when communicating with students from other racial or ethnic backgrounds.^R	5.72	1.43	.41	.22
ECC₂: I often feel irritated by students of a different culture.^R	6.29	1.08	.55	.18
ECC₃: I only feel at ease with people from my culture.^R	5.99	1.22	.37	.23
ECC₄: Student diversity is stressful for me.^R	5.77	1.39	.43	.22
ECC₅: I feel frustrated by student diversity.^R	5.95	1.35	.47	.23
ECC₆: I feel annoyed when students do not speak standard English.^R	5.79	1.55	.38	.16
CCC₁: Knowing about the different experiences of students improves my understanding of my own problems.	5.33	1.39	.45	.21
CCC₂: Knowing students from different ethnic groups improves my understanding of myself.	5.54	1.29	.55	.24
CCC₃: Knowing about the experiences of students of different cultures increases my self-understanding.	5.88	1.19	.53	.20
CCC₄: Knowledge of how a student differs from me greatly enhances our friendship.	5.23	1.47	.36	.18
CCC₅: Understanding how a student is both similar and different from me helps me get to know him or her.	6.09	0.97	.56	.22
CCC₆: I think students from other cultures can teach me things I could not learn elsewhere.	6.40	0.88	.50	.12
BCC₁: I help students work through problem situations caused by stereotypical attitudes toward culture differences.	5.48	1.11	.56	.23
BCC₂: I assist all students to understand the perspectives of people from other ethnic groups.	5.64	1.06	.52	.19
BCC₃: I effectively utilize ethnic resources in the community.	4.67	1.26	.37	.18
BCC₄: I investigate how language difference affects performance on certain test items.	5.21	1.33	.39	.15
BCC₅: I present diversity of cultures as a strong positive feature of American heritage.	5.78	1.15	.58	.23
BCC₆: I seek out cultural biases in materials used in instruction.	4.59	1.51	.45	.21

Note. CCI-PT = Cultural Competence Inventory–Preservice Teachers; ECC = emotional cultural competence; CCC = cognitive cultural competence; BCC = behavioral cultural competence; CITC = corrected item–total correlation; d = item discrimination index.

Reverse coded.

CFA

The CFA of 18-item CCI-PT representing three-factor structure was performed using AMOS 22 in Study 2. Model fit indices met the criteria for a good fitting model (Hu & Bentler, 2000), χ²(df = 132) = 192.78, comparative fit index (CFI) = .90, goodness of fit index (GFI) = .95, Tucker–Lewis index (TLI) = .96, and root mean square error approximation (RMSEA) = .05. The modification indices suggested allowing the error variance of two cognition items “Knowing students from different ethnic groups improves my understanding of myself” and “Knowing about the different experiences of students improves my understanding of my own problems” to covary. Therefore, we allowed the error terms of these two items to covary in the final model, which significantly improved model fit for the data, χ²(df = 131) = 178.51, CFI = .96, GFI = .91, TLI = .97, and RMSEA = .04. All standardized factor loadings were moderate to strong (ranging from .38 to .80) and significant at p < .001 level (Figure 1). Based on the multiple indices as recommended in literature (Hu & Bentler, 2000; Kline, 2004), the robust model fit to a different sample in Study 2 provided further validity evidence of CCI-PT.

Figure 1.

Final confirmatory factor analysis model of the Cultural Competence Inventory–Preservice Teachers (N = 199).

Criterion-Related Validity Estimates

A common criticism of self-report scales on cultural competence is that they are prone to social desirability (Constantine & Ladany, 2000). To test the potential social desirability concern of CCI-PT as a self-report instrument, zero-order correlation analyses were performed between overall scale score and its three subscales with social desirability. The low positive correlations of CCI-PT and the subscales with the Marlowe–Crowne Scale of Social Desirability scores as displayed in Table 4 suggest that this bias is small and does not entirely mask the meaningful relationships with other variables (Kocarek, Talbot, Batka, & Anderson, 2001). Furthermore, the significant correlations between cultural competence with its subscales and teacher self-efficacy (Table 4) provide initial evidence for the convergent validity of CCI-PT, echoing previous research suggesting that teacher self-efficacy is related to cultural competence (Lievens, Harris, Van Keer, & Bisqueret, 2003). Finally, despite the moderate correlation of behavioral dimension of cultural competence and openness to experience (r = .30, p < .01), the nonsignificant correlation between the CCI-PT composite score and openness to experience in Table 4 demonstrates that cultural competence is generally independent of openness to experience, providing preliminary evidence of the discriminant validity of inferences for CCI-PT.

Table 4.

Convergent and Discriminant Validity and Social Desirability Estimates of CCI-PT (N = 203).

	ECC	CCC	BCC	CCI-PT composite score
Openness to experience	.00	.06	.30**	.15
Teacher self-efficacy	.35**	.21*	.36**	.42**
Social desirability	.24**	.16*	.11	.24**

Note. CCI-PT = Cultural Competence Inventory–Preservice Teachers; ECC = emotional cultural competence; CCC = cognitive cultural competence; BCC = behavioral cultural competence.

p < .05. **p < .01 (two-tailed).

To further assess the criterion-related validity of CCI-PT, we tested whether preservice teachers’ ECC, BCC, and CCC scores on CCI-PT would predict their mentor teachers’ and faculty supervisors’ ratings of their multicultural teaching competence through hierarchical regression analyses in Study 3. Control variables (school and grade level) were entered on the first step and the three dimensions of CCI-PT were entered on the second step. Although the cognitive dimension (β = .28, p < .05) was the only significant predictor of performance rated by supervisors, accounting for 12% of the variance, both the emotional (β = .33, p < .05) and the behavioral dimensions (β = .31, p < .05) were significant predictors of performance rated by mentor teachers, accounting for 15% of the variance (Table 5). Therefore, we found initial evidence of the predictive validity: preservice teachers’ self-reported cultural competence positively predicted both their faculty supervisors’ and school mentor teachers’ evaluations of their multicultural teaching performance, although in a different manner.

Table 5.

Predictive Validity Estimates of CCI-PT (N = 113).

	B	SE	β	p
Supervisor evaluations
ECC	.18	.14	.19	.22
BCC	−.06	.17	−.05	.72
CCC	.32	.16	.28	.05
Mentor teacher evaluations
ECC	.31	.14	.33	.04
BCC	.33	.17	.31	.05
CCC	.28	.17	.24	.10

Note. Predictors: preservice teachers’ ECC, BCC, and CCC scores. Control variables: school and grade level. Dependent variables: mentor teachers’ and faculty supervisors’ evaluations of preservice teachers’ multicultural teaching competence. For supervisor evaluations: Step 1: F(6, 45) = 1.86, p = .11, R² =.07; Step 2: F(9, 42) = 2.20, p = .04, ΔR² = .12. For mentor teacher evaluations: Step 1: F(10, 37) = 1.56, p = .15, R² =.06; Step 2: F(13, 34) = 2.07, p = .04, ΔR² = .15. School and grade level were controlled for Step 1 using dummy coded variables. CCI-PT = Cultural Competence Inventory–Preservice Teachers; ECC = emotional cultural competence; BCC = behavioral cultural competence; CCC = cognitive cultural competence.

Discussion

We have reported on the development and initial validation of CCI-PT, a scale to measure cultural competence among preservice teachers. EFA results resonated with the proposed three-factor structure of CCI-PT, which includes ECC (emotional responses to student diversity), CCC (knowledge and awareness of student diversity), and BCC (instructional practice embracing and celebrating student diversity). CFA results with a different sample showed a robust fit for the three-factor model. Collectively, the two studies provided empirical evidence to support the construct validity of CCI-PT as a suitable instrument to assess cultural competence for preservice teachers, including ECC, CCC, and BCC as three distinct constructs incorporated from previous literature (Bennett, 1993; Campinha-Bacote, 2002; Sue, 1982).

Criterion-related validity of inferences for CCI-PT were evidenced through significant correlations between cultural competence for preservice teachers and other related and distinct constructs (self-efficacy beliefs, openness to experience, and social desirability). Our study results suggest CCI-PT is an instrument not only with low social desirability concern, but also with convergent and discriminant validity. Teacher self-efficacy showed the strongest and most consistent correlations with all aspects of cultural competence, whereas openness to experience was only positively correlated with the behavioral dimension of cultural competence. This is consistent with previous literature demonstrating the links between cultural competence and teacher characteristics including self-efficacy (Chen, 2016; Ponterotto et al., 1998) and research showing that openness to experience positively correlates with greater ability in navigating cross-racial relationships (Silvestri & Richardson, 2001; Walker, 2005), a behavioral aspect of cultural competence. The results from the study imply that cultural competence and teaching efficacy may have a mutual influence on each other and highlight the benefits of staying open-minded as educators.

In addition, regression analyses results provided further evidence of criterion-related validity of CCI-PT. In Study 3, preservice teachers’ cognitive dimension of their cultural competence positively predicted their multicultural teaching performance rated by their faculty supervisors, and their scores on both the emotional and behavioral dimensions of cultural competence predicted their multicultural teaching performance as rated by their mentor teachers. Interestingly, faculty supervisor ratings and school mentor teachers’ ratings of preservice teachers’ multicultural teaching performance were predicted by different dimensions of cultural competence of preservice teachers. This result may have to do with different evaluation criteria used by both parties and different levels of expectations for preservice teachers. Specifically, as faculty supervisors were primarily responsible for teaching multicultural knowledge, they likely focused on evaluating the cognitive aspect of cultural competence for preservice teachers. However, school mentor teachers had more opportunities to observe how preservice teachers feel (ECC) and behave (BCC) in their teaching experience, they likely focused on evaluating their emotional and behavioral aspects of cultural competence. This differential predictive pattern also confirmed that emotion, behavior, and cognition are three important but distinct factors of cultural competence as we proposed.

The present research added to the literature that the emotion dimension is an integral part of cultural competence of preservice teachers, which was not addressed in previous multicultural teaching instrumentation studies. This dimension presents a range of emotions preservice teachers experience in responding to student diversity. The significant correlation of emotion with teacher self-efficacy in Study 1 suggests that the more positive preservice teachers feel about student diversity, the more efficacious they feel when working with students from diverse cultures. It indicates the importance of addressing emotions in improving preservice teachers’ efficacy beliefs in working with student diversity and resonates with previous research highlighting the importance of addressing preservice teachers’ emotions in the process of multicultural education (e.g., Stanley, 1996; Wang, 2008; Zhu, 2011).

Taken together, the results from the three studies suggest that the newly developed CCI-PT can be used as a reliable and valid tool to measure the effectiveness of teacher education that focuses on cultivating preservice teachers’ cultural competence. In the present research, the CCI-PT was validated through three populations, which are geographically and ethnically distinct. Although the three samples differed substantially, the bounds of recommendation of this instrument can still be restricted. Additional research is warranted to validate the instrument in other samples. Therefore, we suggest users exert caution if using CCI-PT within different regions and education systems.

Conclusion

This study has advanced our understanding of the concept of cultural competence for preservice teachers and the development and initial validation of the CCI-PT provides a new tool to measure preservice teachers’ cultural competence. In particular, CCI-PT is among the first to include an emotional dimension in the instrument, filling the gap of the important role of affect in multicultural teacher education in contrast with its absence in existing measures. The three aspects of cultural competence assessed with CCI-PT through three independent studies in the present research also provided initial evidence of the reliability of scores and validity of inferences for the instrument. However, more psychometric studies of CCI-PT are needed to provide additional evidence of its validity and reliability. Future study can also replicate inservice teacher population to further test its extrapolation inferences (Kane, 2013).

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iDs

Yan Yang

YoonJung Cho

References

American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.

Artiles

A. J.

McClafferty

(1998). Learning to teach culturally diverse learners: Charting changes in preservice teachers’ thinking about effective teaching. Elementary School Journal, 98, 189-220.

Banks

J. A.

(1981). Multiethnic education: Theory and practice. Boston, MA: Allyn & Bacon.

Bennett

M. J.

(1993). Towards ethnorelativism: A developmental model of intercultural sensitivity. In Paige

R. M.

(Ed.), Education for the intercultural experience (pp. 21-71). Yarmouth, ME: Intercultural Press.

Boser

(2011, November 09). Teacher diversity matters: A state-by-state analysis of teachers of color. Retrieved from http://www.americanprogress.org/issues/education/report/2011/11/09/10657/teacher-diversity-matters/

Boyd

T. E.

(2003). Investigating teachers’ capacity to develop and demonstrate multicultural competency (Unpublished doctoral dissertation). University of Connecticut, Storrs.

Burstein

N. D.

Cabello

. (1989). Preparing teachers to work with culturally diverse students: A teacher education model. Journal of Teacher Education, 40(5), 9-16.

Burstein

N. D.

Cabello

Hamann

(1993). Teacher preparation for culturally diverse urban students: Infusing competencies across the curriculum. Teacher Education and Special Education, 16(1), 1-13.

Campinha-Bacote

(2002). The process of cultural competence in the delivery of healthcare services: A model of care. Journal of Transcultural Nursing, 13, 181-184.

10.

Chambers

Fischer

J. M

. (2002). Multicultural awareness, knowledge and skills: Directions for adult education training programs. Journal of Adult Education, 30(1), 32-47.

11.

Chen

(2016). Teaching in offshore programmes: An assessment of university faculty’s self-efficacy, cultural competence and preparedness. Journal of Education for Teaching, 42, 188–204.

12.

Constantine

M. G.

Ladany

(2000). Self-report multicultural counseling competence scales: Their relation to social desirability attitudes and multicultural case conceptualization ability. Journal of Counseling Psychology, 47, 155-164.

13.

Cortina

. (1993). What is coefficient alpha: An examination of theory and applications. Journal of Applied Psychology, 78, 98-104. doi: 10.1037/0021-9010.78.1.98

14.

Costa

P. T.

McCrae

R. R.

Jr. (1992). Revised NEO personality inventory (NEO-PI-R) and NEO five-factor inventory (NEO-FFI) manual. Odessa, FL: Psychological Assessment Resources.

15.

Costello

A. B.

Osborne

J. W.

(2005). Exploratory factor analysis: Four recommendations for getting the most from your analysis. Practical Assessment, Research, and Evaluation, 10(7), 1-9.

16.

Council for the Accreditation of Educator Preparation. (2013). The CAEP standards. Retrieved from http://caepnet.org/standards/introduction

17.

Crowne

D. P.

Marlowe

D. A.

(1960). A new scale of social desirability independent of psychopathology. Journal of Consulting Psychology, 24, 349-354.

18.

D’Andrea

Daniels

Noonan

M. J

. (2003). New developments in the assessment of multicultural competencies. In Pope-Davis

D. B.

Coleman

H. L. K.

Liu

W. M.

Toporek

R. L.

(Eds.), Handbook of multicultural competencies in counseling and psychology (pp. 154–167). Thousand Oaks, CA: Sage.

19.

De Meuse

K. P.

Hostager

T. J

. (2001). Developing an instrument for measuring attitudes toward and perceptions of workplace diversity: An initial report. Human Resource Development Quarterly, 12, 33-51.

20.

Dunn

Beasly

Buchanan

(1994). What do you believe about how culturally diverse students learn? Emergency Librarian, 22, 8-14.

21.

Henry

(1986, October). Cultural diversity awareness inventory = Inventario sobre el reconocimiento de diversas culturas (ERIC Document Reproduction Service No. ED282657). Hampton, VA: Mainstreaming Outreach Project, Hampton University.

22.

Bentler

P. M.

(2000). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1-55.

23.

Kaiser

J. L.

(1960). The application of electronic computers to factor analysis. Educational and Psychological Measurement, 20, 141-151.

24.

Kane

M. T.

(2013). Validating the interpretations and uses of test scores. Journal of Educational Measurement, 50(1), 1-73.

25.

Kline

R. B.

(2004). Principles and practice of structural equation modeling (2nd ed.). New York, NY: Guilford Press.

26.

Kocarek

C. E.

Talbot

D. M.

Batka

J. C.

Anderson

M. Z.

(2001). Reliability and validity of three measures of multicultural competency. Journal of Counseling & Development, 79, 486-496.

27.

Lievens

Harris

M. M.

Van Keer

Bisqueret

(2003). Predicting cross-cultural training performance: The validity of personality, cognitive ability, and dimensions measured by an assessment center and a behavior description interview. Journal of Applied Psychology, 88, 476-489.

28.

Mayer

J. D.

Salovey

(1993). The intelligence of emotional intelligence. Intelligence, 17, 433-442.

29.

McConahay

J. B.

(1986). Modern racism, ambivalence, and the modern racism scale. In Dobidio

J. F.

Gaertner

S. L.

(Eds.), Prejudice, discrimination, and racism (pp. 35-60). New York, NY: Academic Press.

30.

Milner

H. R.

Flowers

L. A.

Moore

Jr. Moore

J. L.

III Flowers

T. A.

(2003). Preservice teachers’ awareness of multiculturalism and diversity. High School Journal, 87, 63-70.

31.

Moore

Reeves-Kazelskis

(1992, November). The effects of formal instruction on preservice teachers’ beliefs about multicultural education (ERIC Document Reproduction Service No. ED354231). Paper presented at the annual meeting of the Mid-South Educational Research Association. Retrieved from https://archive.org/stream/ERIC_ED354231/ERIC_ED354231_djvu.txt

32.

Morgado

F. F. R.

Meireles

J. F. F.

Neves

C. M.

Amaral

A. C. S.

Ferreira

M. E. C.

(2017). Scale development: Ten main limitations and recommendations to improve future research practices. Psicologia: Reflexão e Crítica, 30, Article 3. Retrieved from http://psycnet.apa.org/record/2017-03880-001

33.

Netemeyer

R. G.

Bearden

W. O.

Sharma

(2003). Scaling procedures: Issues and applications. Thousand Oaks, CA: Sage.

34.

Panjaitan

R. L.

Irawati

Sujana

Hanifah

Djuanda

(2018). Item validity vs. item discrimination index: A redundancy? Journal of Physics: Conference Series, 983(1), 012101.

35.

Pettus

Allain

V. A.

(1999). Using a questionnaire to assess prospective teachers’ attitudes toward multicultural education issues. Education, 119, 651-657.

36.

Pohan

C. A.

Aguilar

T. E.

(2001). Measuring educators’ beliefs about diversity in personal and professional contexts. American Educational Research Journal, 38, 159-182.

37.

Ponterotto

J. G.

Baluch

Grieg

Rivera

(1998). Development and initial score validation of the teacher multicultural attitude survey. Educational and Psychological Measurement, 58, 1002-1016.

38.

Prieto

L. R.

(2012). Initial factor analysis and cross-validation of the Multicultural Teaching Competencies Inventory. Journal of Diversity in Higher Education, 5, 50-62.

39.

Rew

Becker

Cookston

Khosropour

Martinez

(2003). Measuring cultural awareness in nursing students. Journal of Nursing Education, 42, 249-257.

40.

Reynolds

W. M.

(1982). Development of reliable and valid short forms of the Marlowe-Crowne Social Desirability Scale. Journal of Clinical Psychology, 38, 119-125.

41.

Silvestri

T. J.

Richardson

T. Q.

(2001). White racial identity statuses and NEO personality constructs: An exploratory analysis. Journal of Counseling & Development, 79, 68-76. doi:10.1002/j.1556-6676.2001.tb01945.x

42.

Solomon

Levine-Rasky

(1994). Accommodation and resistance: Educators’ response to multicultural and antiracist education. A Report to the Federal Department of Canadian Heritage. Toronto, Canada: York University.

43.

Spanierman

L. B.

Heppner

P. P.

Neville

H. A.

Mobley

Wright

C. V.

. . .Navarro

(2011). The multicultural teaching competency scale: Development and initial validation. Urban Education, 46, 440-464. doi:10.1177/0042085910377442

44.

Stanley

L. S.

(1996). The development and validation of an instrument to assess attitudes toward cultural diversity and pluralism among preservice physical educators. Educational and Psychological Measurement, 56, 891-897.

45.

Sue

D. W.

(1982). Position paper: Cross-cultural counseling competencies. Counseling Psychologist, 10, 45-52.

46.

Tabachnick

B. G.

Fidell

L. S.

(2007). Using multivariate statistics (5th ed.). Boston, MA: Allyn & Bacon.

47.

Tatto

M. T.

(1996). Examining values and beliefs about teaching diverse students: Understanding the challenges for teacher education. Educational Evaluation and Policy Analysis, 18, 155-180.

48.

Tran

M. T.

Young

R. L.

DiLella

J. D.

(1994). Multicultural education courses and the student teacher: Eliminating stereotypical attitudes in our ethnically diverse classroom. Journal of Teacher Education, 45, 183-189.

49.

Tschannen-Moran

Woolfolk Hoy

(2001). Teacher efficacy: Capturing an elusive construct. Teaching and Teacher Education, 17, 783-805.

50.

U.S. Census Bureau. (2017a). Current population survey, 1995-2016. Retrieved from https://www.census.gov/programs-surveys/cps.html

51.

U.S. Census Bureau. (2017b). 2016 American community survey 1-year estimates. Retrieved from https://factfinder.census.gov/faces/tableservices/jsf/pages/productview.xhtml?pid=ACS_16_1YR_S1601&prodType=table

52.

Walker

S. M.

(2005). An exploratory analysis of relationships between personality constructs, anxiety, racial identity, and reactions to an experiential learning tool about race relations (Unpublished doctoral dissertation). Lehigh University, Bethlehem, PA.

53.

Wang

(2008). "Red eyes": Engaging emotions in multicultural education. Multicultural Perspectives, 10, 10-16. doi:10.1080/15210960701869330

54.

Wang

Davidson

Yakushko

O. F.

Savoy

Tan

J. A.

Bleier

J. K.

(2003). The scale of ethnocultural empathy: Development, validation, and reliability. Journal of Counseling Psychology, 50, 221-234. doi:10.1037/0022-0167.50.2.221

55.

Wayson

W. W.

(1993). Multicultural teaching scale. Plain, OH: Synergetic.

56.

Worthington

R. W.

Whittaker

T. A.

(2006). Scale development research: A content analysis and recommendations for best practices. The Counseling Psychologist, 34, 806-838.

57.

Yang

Montgomery

. (2011). Exploratory and confirmatory factor analysis of Multicultural Teaching Scale. Journal of Psychoeducational Assessment, 29(3), 261-272. doi: 10.1177/0734282910378983

58.

Zhu

(2011). From intercultural awareness to intercultural empathy. English Language Teaching, 4, 116-119.

Development and Initial Validation of Cultural Competence Inventory–Preservice Teachers

Abstract

Keywords

Extant Cultural Competence Measures

Research Aim and Hypotheses

Method

Instrument Development

Participants and Procedure

Results

EFA

CFA

Criterion-Related Validity Estimates

Discussion

Conclusion

Footnotes

Declaration of Conflicting Interests

Funding

ORCID iDs

References