The Development and Validation of an Intercultural Communicative Competence Scale for Students

Abstract

Intercultural Communicative Competence (ICC) plays a significant role in learning a foreign language. Therefore, it is important to measure the ICC level of language learners before taking actions to improve the educational process. The present study aimed at developing an ICC Scale to measure the perceived ICC level of Academic English students studying in the Foreign Language Division of an international university. The participants of the study consisted of 325 first year students in their second semester in the 2021 to 2022 academic year enrolled in 48 English-medium instructed (EMI) departments. In order to produce a reliable and valid ICC Scale, statistical data were analyzed through software programs. Further, the Exploratory Factor Analysis generated an 18-item Intercultural Communicative Competence (ICC) Scale with four factors, namely knowledge, awareness, attitude, and skills. Additionally, the Confirmatory Factor Analysis results confirmed and validated the four factors. Consequently, the ICC Scale was found to be highly reliable in measuring the perceived ICC level of students and had a high internal consistency. The newly designed ICC Scale proves to be a useful instrument for measuring the perceived ICC level of language learners.

Plain Language Summary

The study aimed at developing an Intercultural Communicative Competence (ICC) Scale to measure the perceived ICC level of Academic English students studying in the Foreign Language Division of an international university in North Cyprus. At the beginning, the newly designed ICC Scale involved 57 items; 31 items were adapted from different ICC scales and 26 items were generated by the researchers from scratch. The newly designed ICC Scale was administered to 325 first year Academic English students in the second semester of the 2021 to 2022 academic year. In order to produce a reliable and valid ICC Scale, firstly, the Exploratory Factor Analysis (EFA) was computed for statistical analyses and the designed scale was found reliable. The EFA generated an 18-item Intercultural Communicative Competence (ICC) Scale with four factors, namely knowledge, awareness, attitude, and skills. Then, the Confirmatory Factor Analysis (CFA) was implemented. Finally, the newly designed 18-item ICC Scale for students with four factors was found to be a highly reliable instrument (Cronbach Alpha reliability = .829). Furthermore, it was found a fit model (χ²/df value = 2.342) to measure the Intercultural Communicative Competence of students in a multicultural educational setting. In this study, only quantitative data about students’“perceived” ICC levels can be collected and analyzed using this ICC Scale. The ICC level as perceived by students may not necessarily be exactly the same as the ICC level practiced. To overcome this possible limitation, it is strongly suggested that researchers collect qualitative data as well (e.g., through interviews, student journals/diaries, portfolios, class observations, field notes) in order to triangulate with the quantitative data from the Scale so as to obtain more realistic pictures of the participants’ ICC levels.

Keywords

academic English competence factor analysis intercultural communication intercultural competence scale development

Introduction

The aftermath of World War II marked a new era in most parts of the world. Since the post-war period, there has been a need for communication among people from different countries and nations for business and employment, navigation, science, information technology, and education (Hutchinson & Waters, 1987). Therefore, the English language has become a lingua franca (ELF)—a means for communication among people who have different nations, different cultures and different mother tongues (e.g., Jenkins, 2006; Seidlhofer, 2006). In addition, the English language has been used as a medium of instruction (EMI) in many countries for several reasons. The two common reasons are the consequence of being a British colony in the past and the need to have better communication with other countries (e.g., Hogan et al., 2015). Another reason is to make profit and gain economic privilege (e.g., Pihama, 2019). Hence, there has been an increase in the number of multilingual and multicultural students in language education throughout the world.

Due to globalization, interaction among students from different cultures has gained importance. Kramsch (1993) pointed out that foreign language students should be able to engage in interaction with learners who have different cultures. However, learners may find themselves in no man’s-land where they need to learn and use the target language (either L2 or FL) in order to communicate with other learners who have different first languages and different cultures. In order to become successful in learning the target language, they need to interact effectively with students from other cultures and to understand each other. In other words, language learners need to develop their Intercultural Communicative Competence (ICC) which refers to the ability to interact effectively and appropriately with people from other cultures using a common (foreign or not) language (Byram, 1997).

A number of studies have investigated the Intercultural Communicative Competence (ICC) of language learners (e.g., Alptekin, 2002; Badrkoohi, 2018; Baroudi, 2017; Munezane, 2019; Pourakbari et al., 2018). While some studies have focused specifically on pre-service teachers’ ICC (e.g., Akcaoğlu & Arsal, 2021; Duisembekova, 2021; Hismanoglu, 2011; Lei, 2020; Sarıçoban & Oz, 2014; Sevimel-Sahin, 2020; Üzüm et al., 2020), one study has examined more holistically the state of affairs of a corpus of studies on multicultural education around the world published in the journals indexed by Scopus database between 2012 and 2017 (e.g., Uzunboylu & Altay, 2021). Yet a few other studies have investigated the effect of communication technology on students’ developing their Intercultural Communicative Competence (Nurullina et al., 2018; Toscu & Erten, 2020). ICC studies so far have revealed that ICC has an important effect on language learners’ learning a foreign language. Besides, there have been a few ICC-related studies conducted in the local context (e.g., Baroudi, 2017; Erden, 2018; Ozbilgehan & Celenk, 2021). However, best to the researchers’ knowledge, there is only one study conducted on the ICC of learners (e.g., Baroudi, 2017) at the same university where the current study was conducted. Moreover, Academic English students’ ICC has not been investigated yet. Therefore, the researchers decided to develop a new ICC Scale in order to measure the perceived ICC level of Academic English learners. Another reason for such a need was that the existing ICC scales lacked some important aspects of ICC. The lacking concepts involve the knowledge, awareness and attitude of the participants. In order to fill this gap, the newly designed ICC Scale was designed to measure participants’ knowledge of “different values and different taboos of other cultures,” participants’ attitude toward “having friends from other cultures and interacting with people from other cultures daily.” Moreover, the awareness section of the new ICC Scale involves some topics which were not addressed in previous scales. The items in the awareness section involve topics such as (i) people from other cultures may think and behave in different ways; (ii) they may have different beliefs, values, and morals; and (iii) they may have different taboos, religions and attitudes.

Literature Review

For effective interaction with people from other culture(s), it is essential to develop cultural awareness which refers to having an interest in other cultures, being sensitive towards people from other cultures, and becoming aware of cultural differences (Bhawuk & Brislin, 1992). However, cultural awareness is not enough for effective communication with people from other cultures. Additionally, developing “intercultural awareness” which can be defined as being able to understand, respect and tolerate cultural differences (M. Bennett, 1993) is also necessary. Likewise, it was found that for effective and meaningful communication, behaving and speaking in an appropriate way with people from other cultures was required (Byram, 1997). Therefore, Byram (1997) came up with a new term—Intercultural Communicative Competence (ICC), which represented an individual’s ability to interact effectively and appropriately with members of other cultures using a common (foreign or not) language (Byram, 1997). A similar definition of the ICC is concerned with an effective interaction in cross-cultural situations and behaving appropriately in different cultural contexts (J. M. Bennett & Bennett, 2004). Ideally, speakers should have both “cultural awareness” and “intercultural competence.” First, they need to become aware of cultural differences around them, and then try to understand, accommodate and respect cultural differences in order to develop their ICC.

There is a direct relationship between foreign language education (FLE) and the development of ICC (Byram, 1997). Byram (1997) stressed that FLE should not only focus on language teaching, but also pay attention to developing ICC level of learners as indicators of effective and successful education. Therefore, measuring language learners’ ICC level is of paramount importance. For that reason, some studies have focused on language learners and ICC. In most of these studies, EFL learners were chosen as participants because of a tremendous increase in the number of multilingual and multicultural learners of English as a foreign language due to globalization all over the world (Bouchard, 2017).

The studies conducted on EFL learners and ICC mainly focused on the attitudes of undergraduate EFL learners on ICC (e.g., Pourakbari et al., 2018; the individual differences that affect ICC (e.g., Munezane, 2019); the relationship of ICC with other factors (e.g., Badrkoohi, 2018; Hismanoglu, 2011); and EFL learners’ ICC development through telecollaboration (e.g., Toscu & Erten, 2020). However, best to the researchers’ knowledge, there is no study that measures the perceived ICC level of English language learners. The newly designed ICC Scale in this study targets to measure the perceived ICC level of Academic English learners. Moreover, although some of the existing ICC questionnaires and scales aim to measure the participants’ ICC, they do not measure the ICC levels of the participants reliably. They tend to measure the “desires” of the participants in engaging in interaction with people from other cultures, not their competence in interaction with people from other cultures. Therefore, the present study aimed to adapt and develop a reliable ICC Scale to measure the perceived ICC level of students in their interaction with people from other cultures.

The study was conducted in one of the biggest international universities in North Cyprus, which is among 22 international universities (https://www.yok.gov.tr/universiteler/diger-yuksekogretim-kurumlari) offering higher education in English. The research context is one of the few higher education institutions hosting around 16,000 students and possessing 11 faculties, five schools, including the English Preparatory School. International students from 104 different countries in the world make up more than 35% of the total student population. In addition to its diverse student body, the university appears to be within the 501 to 600 band in the university rankings according to the Times Higher Education World University Rankings (https://www.timeshighereducation.com/world-university-rankings/2022/world-ranking), and 47 of its programs have been accredited by 12 different international accrediting bodies such as ABET for engineering, FIBAA for business administration and tourism, AQAS for architecture and education, and EAQUALS for English Preparatory School.

Methodology

In this section, the stages of the ICC Scale development, and the data collection and analyses procedures are explained in detail.

Research Design

A descriptive research design aims to accurately and systematically describe a population, situation or phenomenon (McCombes, 2020). Therefore, in the present study, a descriptive research design was used in which the researchers targeted to administer the ICC Scale (for piloting) to the whole population (N=648) of the participants who were taking the Academic English course in an international university in North Cyprus. However, 325 voluntary students responded to the ICC Scale. Furthermore, the study started off by investigating and examining the contents of the existing ICC scales in the literature. The researchers found some of the items in the existing ICC scales useful as a starting point and adapted 31 items from various ICC scales by respective scholars in the field (Fantini & Tirmizi, 2006; Kazykhankyzy & Alagozlu, 2019; Zhou, 2011). No adapted item was taken as verbatim. Since the adapted content was short in coverage, another thematically-formulated 26 items were generated from scratch by the researchers. At the end of this process, a more comprehensive ICC Scale with a total of 57 items was designed.

The newly designed ICC Scale was administered to 325 Academic English students. The scale was implemented in the English language because the Academic English course was a B2 level course according to the Common European Framework of References (CEFR) for languages and the students taking Academic English course had a good command of English. Except a few words at the piloting stage, the students did not report any problems with the language. After being implemented, the ICC Scale was checked in terms of content validity by the researchers. The newly designed ICC Scale was sent to four field experts for review. Then, the experts’ written and oral feedback on the ICC Scale was obtained. First, the experts sent back their written feedback via email to the researchers. Next, the researchers held face-to-face meetings with each expert and discussed the given feedback with them. Soon after receiving minor modifications on the items, the scale was revised. Afterwards, in order to have a pre-test for the usability of the newly designed ICC scale, it was administered to a randomly selected group of Academic English students (N = 23). The results of the scale were analyzed and the students who completed the scale were interviewed face-to-face. Based on the participants’ oral feedback on the clarity of the items, a few modifications were made on a several items. The modifications included simplification of a sentence or replacing a word with a synonym to make the meaning clearer for the students. Once the modifications were applied, the final draft of the scale was piloted to see whether the designed ICC Scale was a reliable tool to measure students’ perceived ICC levels.

Research Sample

The study was conducted in the Foreign Languages Division (FLD) of the Foreign Languages and English Preparatory School (FLEPS) in one of the biggest international universities in North Cyprus, which hosted over fifteen thousand multicultural students from 104 different countries in the Spring semester of the 2021 to 2022 academic year. Moreover, the participants received a 4-hour instruction per week in their Academic English course. The course was compulsory and delivered to the whole population consisting of 684 students from 74 different countries studying in 22 groups in 48 English-medium departments. Out of 684 potential participants, 325 (47.5% of the total population) volunteered to participate, which constituted the sample of the study. Those who volunteered signed the written consent form face-to-face before they completed the ICC Scale.

Data Collection Procedure

First, since the study involved human participants, approval from the Director of the Foreign Languages and English Preparatory School (FLEPS), the Head of the Department of Foreign Language Education (FLE), and the Scientific Research and Publication Ethics Board of the university (ETK00-2022-1337) was obtained. Then, the content items of the drafted ICC Scale were strengthened using the field experts’ feedback as well as the second author’s expertise. Next, the ICC Scale was distributed to all the Academic English groups for the purpose of data collection. The ICC Scale was completed by 325 voluntary students. Finally, Exploratory Factor Analysis (EFA) and Confirmatory Factor Analysis (CFA) were performed on the collected data. The details of the phases of the analyses are presented in the data analysis section below.

Data Analysis

The data were analyzed in two phases. First, the Exploratory Factor Analysis (EFA) was conducted using SPPS (ver.22) software program, which was followed by the Confirmatory Factor Analysis (CFA) to better confirm the reliability of the ICC Scale using the Amos (ver.24) software program. Then, in order to measure the reliability of the items (N = 57) on the scale and the reliability of the overall ICC Scale, the Cronbach Alpha internal consistency coefficient value was calculated (.829). As a result of the performed EFA and CFA, a reliable research measurement tool was achieved with a good fit value (χ²/df = 2.342; Wheaton et al., 1977).

Phase 1: Exploratory Factor Analysis (EFA)

The Exploratory Factor Analysis (EFA) involved four steps. For the first step, the Kaiser-Meyer-Olkin (KMO) test and the Bartlett’s Sphericity Test were performed. The KMO value was found .820 and the Bartlett Sphericity value was obtained as 2.342. Following the required procedures by Yaşlioglu (2017), items which had factor loadings below .50 were removed (1, 6, 9, 16, 17, 20, 21, 22, 23, 24, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 45, 47, 49, 51, 55, and 57). Furthermore, the implemented removals were confirmed by Tabachnick and Fidell’s (2011) claim regarding the lowest factor loading value below .32 to be used in newly developed scales. After removing the items with lower values, the Kaiser-Meyer-Olkin (KMO) test and the Bartlett Sphericity test were re-performed. Based on the repeated tests, the KMO value was found .820 and four factors were obtained. However, items 8, 48, and 56 were not replaced under any factor. Therefore, they were removed from the scale. As for the third step, the Kaiser-Meyer-Olkin (KMO) test and the Bartlett’s Sphericity test were re-conducted. The KMO value was calculated as .813 and the obtained four factors were named as Knowledge, Awareness, Attitude, and Skills.

In the final step, the Cronbach Alpha internal consistency coefficient was calculated to measure the overall reliability of the ICC Scale. The Cronbach Alpha reliability value was found to be .829. Then, item total correlations were calculated to check the internal reliability of each sub-dimension. The reliability of Factor 1 was calculated as .856; the reliability of Factor 2 was found .763; the reliability of Factor 3 was calculated as .745, and the reliability of Factor 4 was found to be .746.

Phase 2: Confirmatory Factor Analysis (CFA)

As a result of the Exploratory Factor Analysis (EFA) performed, a 20-item ICC measurement tool with four sub-dimensions was obtained. Further, to confirm the reliability of the obtained ICC Scale, the Confirmatory Factor Analysis (CFA) was conducted twice by AMOS (ver.24) software program. As a result of the first performance of the CFA, the loading values of two items (i.e., items 7 and 19) were found lower than .50. Therefore, they were removed before calculating the reliability of the ICC Scale one more time. Consequently, the CFA confirmed that the overall ICC Scale involving 18 adapted and developed items and four factors was a highly reliable research tool to be used in educational contexts to determine the perceived ICC level of the students.

Data Collection Instrument

The Intercultural Communicative Competence (ICC) Scale for students was partially adapted and partially developed by the researchers through digesting the existing literature on ICC and examining the available ICC scales. The ICC Scale used a 5-point Likert scale with the options of “5 = strongly agree, 4 = agree, 3 = uncertain, 2 = disagree, and 1 = strongly disagree.” In total, 31 of the items in the ICC Scale were adapted from other existing ICC scales in the literature and 26 of the items were developed by the researchers. Twelve of the items (items 6, 7, 11, 14, 15, 16, 30, 31, 32, 51, 56, and 57) were adapted from Fantini and Tirmizi’s (2006) ICC Scale, 15 of the items (items 19, 33, 38, 39, 40, 41, 42, 43, 44, 46, 47, 48, 49, 50, and 53) were adapted from Kazykhankyzy and Alagozlu’s (2019) ICC Scale, and seven of the items (items 9, 12, 32, 37, 48, 51, and 54) were adapted from Zhou’s (2011) ICC questionnaire. However, among these adapted items, two were adapted referring to two different sources since some ICC scale items had similar meanings. In the present ICC Scale, item 32 was adapted from both Fantini and Tirmizi’s (2006) and Zhou’s (2011) ICC scales. Item 49 was adapted from both Zhou’s (2011) and Kazykhankyzy and Alagozlu’s (2019) ICC scales. On the other hand, 26 of the items (items 1, 2, 3, 4, 5, 8, 10, 13, 17, 18, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 34, 35, 36, 45, and 52) on the scale were developed by the researchers. For the content validity, the ICC Scale was examined and analyzed by four experts in the field in addition to the second author. Verbal and written feedback from all the field experts was taken into account to make the necessary modifications to the items on the scale.

After analyzing the collected data by using the SPSS (ver.22) software program for the EFA and the Amos (ver.24) software program for the CFA, 39 items (1, 6, 7, 8, 9, 16, 17, 19, 20, 21, 22, 23, 24, 26, 28, 27, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 45, 47, 48, 49, 51, 55, 56, and 57) were removed from the scale due to having factor loadings lower than .50 (Yaşlioglu, 2017). The finalized version of the ICC Scale had a 5-point Likert scale with “5 = strongly agree, 4 = agree, 3 = uncertain, 2 = disagree, and 1 = strongly disagree” options and consisted of 18 items with four factors, namely Knowledge, Awareness, Attitude, and Skills. The factors were given names considering Byram’s (1997) conceptualization of ICC which involved knowledge, skills, attitude and awareness. Factor 1 had 6 items, Factor 2 had 4 items, Factor 3 had 4 items, and Factor 4 involved 4 items.

To sum up, 10 out of 18 items of the newly designed ICC Scale were adapted and modified. Besides, eight items were developed by the researchers. Four of the items (items 2, 5, 6, and 17) were adapted from Fantini and Tirmizi’s (2006) ICC Scale, four of the items (items 13, 14, 15, and 18) were adapted from Kazykhankyzy and Alagozlu’s (2019) ICC Scale, and two of the items (item 3 and 16) were adapted from Zhou’s (2011) ICC Questionnaire. On the other hand, eight of the items (items 1, 4, 7, 8, 9, 10, 11, and 12) on the newly designed ICC Scale was developed by the researchers at the end of the study.

Results

The results of the study are presented in two stages below. The EFA results are followed by the CFA results.

Exploratory Factor Analysis (EFA) Results

The KMO value of the scale displayed in Table 1 below was calculated as .813, which indicated that the sampling size was highly adequate for factor analysis (Field, 2009). Bartlett’s Test of Sphericity results indicated (χ² = 2,066.657, p < .000) that the obtained value was considered statistically significant with a minimum coefficient of .60 (Tabachnick & Fidell, 1996). Therefore, based on the values obtained from the analysis, the sampling size of the study was found to be appropriate to conduct the factor analysis.

Table 1.

Kaiser–Meyer–Olkin (KMO) Test and Bartlett’s Sphericity Test.

		.813
Kaiser-Meyer-Olkin measure of sampling adequacy	χ²	2,066.657
	df	190
	p	.000

As illustrated in the Scree Plot chart below (Figure 1), there were four cut-off points on the red line which represented the point value 1. The four different cut-off points on the red line outlined that the designed ICC Scale consisted of four sub-dimensions.

Figure 1.

Scree plot chart.

Table 2 below outlined the statistical analysis of the finalized ICC Scale. 39 items with loading values below .50 were removed from the ICC Scale and the data were reanalyzed with 18 items. As a result, a valid and reliable ICC Scale with four factors was developed. Factor 1 = Knowledge included items 10, 11, 12, 13, 14, and 15. Factor 2 = Awareness involved items 2, 3, 4, and 5. Factor 3 = Attitude involved the items 18, 25, 44, and 46. Factor 4 = Skills involved the items 50, 52, 53, and 54.

Table 2.

Distribution of the Scale by Factors, Item Factor Loads, and Factor Variances.

New item number	Old item number	Communalities extraction variance	Fac. 1	Fac. 2	Fac. 3	Fac. 4
1	V10	.641	.766
2	V11	.685	.809
3	V12	.635	.788
4	V13	.507	.689
5	V14	.582	.740
6	V15	.496	.691
7	V2	.559		.730
8	V3	.628		.782
9	V4	.545		.730
10	V5	.497		.685
11	V18	.641			.767
12	V25	.492			.651
13	V44	.518			.665
14	V46	.554			.699
15	V50	.614				.728
16	V52	.454				.653
17	V53	.707				.822
18	V54	.546				.644
Eigenvalues			2.587	4.929	2.037	1.562
Explained variance value %			12.936	24.645	10.185	7.810
Total variance %			55.577

The total explained variance rate was calculated as 55.577%, which was above the lowest limit (.50) as stated by Yaşlioglu (2017). Factor 1 consisted of six items and its Eigenvalues value was found 2.587. Further, the highest factor loading value was .766 and the lowest factor loading value was calculated as .691. Factor 2 consisted of four items and its Eigenvalues value was 4.929. Moreover, the highest factor loading value was .730 while the lowest factor loading value was .685. Factor 3 = Attitude involved four items and its Eigenvalues value was found 2.037. Further, the highest factor loading value was .767 and the lowest factor loading value was calculated as .699. Factor 4 = Skills consisted of four items and its Eigenvalues value was found 1.562. Further, the highest factor loading value was .728 and the lowest factor loading value was calculated as .644.

The overall Cronbach’s Alpha reliability of the ICC Scale was found .829 (see Table 3 below). This meant that the designed scale was reliable. Moreover, the Cronbach Alpha value for each factor was computed. Besides, DeVellis (2003) suggested that an alpha coefficient of .70 for each sub-dimension was acceptable for a new instrument. In the present study, the alpha coefficient of each factor was found above .70 (see Table 3).

Table 3.

Reliability Statistics.

	Cronbach’s alpha	No of items
Overall scale	.829	18
Factor 1—Knowledge	.856	6
Factor 2—Awareness	.763	4
Factor 3—Attitude	.743	4
Factor 4—Skills	.746	4

The Cronbach’s alpha coefficients of the factors in the ICC Scale were found to be .856, .763, .743, and .746, respectively. To sum up, the findings indicated that each factor had a high reliability level.

Confirmatory Factor Analysis (CFA) Results

The findings of the CFA are shown in Table 4 below. The values obtained in this study are displayed along with the acceptable fit values in the literature. The acceptable fit values were varied in the literature. For instance, the acceptable χ²/df value was supposed to be below 5 according to Wheaton et al. (1977). However, Tabachnick and Fidell (2011) had a different view and they claimed that the excellent χ²/df value was supposed to be below 2. The χ²/df value in this study was found 2.342 and met the value of good fit. The Root mean square error of approximation RMSEA value pointed out how well the model was obtained (Byrne, 1998).

Table 4.

Confirmatory Factor Analysis Model Fit Values.

Model fit values	Academic English students’ perceived ICC level scale values	Acceptable fit values
χ²/df	2.342	≤4–5
RMSEA	0.064	0.06–0.08
NFI	0.849	≥0.95
TLI	0.889	>0.80
IFI	0.907	0.94–0.90
CFI	0.906	0.90–0.95

When the RMSEA value was considered, it was proposed by Yaşlioglu (2017) that the closer the RMSEA value to 0.1, the worse the fit, and the closer it is to 0, the better the fit. The RMSEA value in this scale was calculated as 0.064. Hu and Bentler (1999) suggested that the Normed-fit index; NFI ≥ 0.95 as the threshold. However, in this model, the NFI value was found 0.849, which was slightly below the suggested fit value (0.94–0.90) (Bentler & Bonett, 1980) but in situations where small samples were used, the NFI value could stipulate poor fit despite other statistics pointing towards good fit (Bentler, 1990; Tabachnick & Fidell, 2011). The cut-off point for the Tucker-Lewis index—TLI value was supposed to be >0.80 according to Byrne (2011). In this model, it was calculated as 0.889 which was an excellent value. The Incremental Fit Index (IFI) adjusts the Normed Fit Index (NFI) for sample size and degrees of freedom (Bollen, 1990) and the IFI value over 0.90 is a good fit. The IFI value of this scale was found 0.907 which was between the acceptable fit values (0.94–0.90). The Comparative Fit Index (CFI) is a revised form of the NFI which takes the sample size into account (Byrne, 1998). The CFI performs well even when sample size is small (Tabachnick & Fidell, 2011). The CFI was calculated as .906 in this model and it was considered acceptable as a cut-off criterion for CFI ≥ 0.90 was found acceptable according to Hooper et al. (2008). Further, Figure 2 below outlines the correlations between the factors.

Figure 2.

Confirmatory factor analysis factor distribution path diagram.

Discussion and Conclusion

The aim of the current study was to develop a scale to measure the Intercultural Communicative Competence (ICC) of Academic English students. The end product of the process, therefore, was a newly designed ICC Scale for students. The data analysis process completed in developing our newly designed ICC Scale is in line with the data analysis steps outlined in Duisembekova’s (2021) and Kazykhankyzy and Alagozlu’s (2019) ICC scale development. Duisembekova (2021) developed a new scale to investigate pre-service teachers’ beliefs about ICC in Turkey. Similarly, Kazykhankyzy and Alagozlu (2019) generated an ICC Scale to measure the ICC of Kazakh and Turkish pre-service teachers.

The scale development procedure in this study involved several stages. At the first stage—data analysis stage—an exploratory factor analysis (EFA) was used to measure the reliability of the items in the available scales. At the second stage, a confirmatory factor analysis (CFA) was performed. As a result of EFA and CFA, our scale consisted of 18 items, a relatively smaller number of items compared to Duisembekova’s (2021) scale with 34 items, and Kazykhankyzy and Alagozlu’s (2019) scale with 52 items.

On the other hand, the way we designed our ICC Scale is, in some respects, different from the formation of other ICC scales. For example, while Chao (2014), who developed the Intercultural Competence Scale for university EFL learners in Taiwan, and Guyton and Wesche (2005), who developed the Multicultural Efficacy Scale, used the items verbatim from the existing instruments in the literature, but we did not. Instead, not only did we adapt some of the items from the existing ICC scales in the literature, but also we developed some new items.

At the beginning of our study, the ICC Scale consisted of 57 items—31 items were adapted from different ICC scales (Fantini & Tirmizi, 2006; Kazykhankyzy & Alagozlu, 2019; Zhou, 2011) and 26 items were developed by the researchers. Firstly, the EFA was computed for statistical analyses and the designed scale was found reliable with the overall Cronbach’s Alpha reliability of .829. Moreover, the reliability of each factor was analyzed and the alpha coefficient of each factor was found highly reliable (above .70 which is acceptable for a newly designed instrument (DeVellis, 2003). The Cronbach’s alpha coefficients of the “knowledge” factor was found to be .856; the “awareness” factor as .763; the “attitude” factor as .743 and the “skills” factor as .746. To sum up, the findings indicated that each factor had a high reliability level.

After the EFA and the CFA were conducted, 39 items (1, 6, 7, 8, 9, 16, 17, 19, 20, 21, 22, 23, 24, 26, 28, 27, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 45, 47, 48, 49, 51, 55, 56, and 57) with loading values below .50 were removed and a new 18-item (items 2, 3, 4, 5,10, 11, 12, 13, 14, 15, 18, 25,44, 46, 50, 52, 53, and 54) ICC Scale was developed. The factor loading values of the items varied between .651 and .822 and explained 50.57% of the total variance. As a result of EFA, the items were clustered under four sub-dimensions (factor): knowledge (items 1–6), awareness (items 7–10), attitude (items 11–14) and skills (items 15–18).

The relevance and the potential contribution of the present study to the existing literature is discussed in greater detail here. The existing ICC scales mostly aimed to investigate the ICC of EFL students only (e.g., Badrkoohi, 2018; Baroudi, 2017; Chao, 2014; Duisembekova, 2021; Guyton & Wesche, 2005; Hismanoglu, 2011; Kazykhankyzy & Alagozlu, 2019; Lei, 2020; Munezane, 2019; Pourakbari et al., 2018; Sevimel-Sahin, 2020; Toscu & Erten, 2020; Üzüm et al., 2020). However, our newly designed ICC Scale aims to measure the perceived ICC level of Academic English students at university. Furthermore, our newly designed ICC Scale is different from other existing ICC scales contentwise, that is, in terms of measuring the knowledge, awareness, and attitude of the participants in some ways. These sections involve some items which were not mentioned in other ICC scales before and those items involve aspects which the researchers identified as missing in other ICC scales and thus they developed those new items in the scale. In the “knowledge” section of the ICC Scale, the participants’ knowledge about the “different values of other cultures” and “the religion of other cultures” is targeted to be measured. In the “awareness” section, awareness of the participants on some topics such as “people from different cultures may think and behave in different ways than I do,”“people from different cultures may have different beliefs, values and morals than I do,”“people from different cultures may have different taboos,”“people from different cultures may have different religions, beliefs and attitudes.” Furthermore, in “attitude” section, whether participants “have friends from other cultures” and whether they “interact with people from other cultures daily” are aimed to be asked.

Some scales which aim to measure the Intercultural Communicative Competence (ICC) of the participants are Fantini and Tirmizi’s (2006) ICC Scale; Kazykhankyzy and Alagozlu’s (2019) ICC Scale, and Zhou’s (2011) ICC Questionnaire. Both Kazykhankyzy and Alagozlu (2019) and Fantini and Tirmizi (2006) aimed to investigate the ICC of students whereas Zhou (2011) aimed to measure the ICC of EFL teachers. These existing ICC scales mostly aimed to measure the “desires” of the participants in communicating with people from other cultures. On the other hand, our newly designed ICC Scale is different from the existing ICC scales in that it is designed to measure to what extent Academic English students are interculturally competent in their interaction with people from other cultures.

An example of a similar study to ours is Kazykhankyzy and Alagozlu’s (2019) on the ICC of pre-service English teachers, from which four of the items (items 13, 14, 15, and 18) in our scale were adapted. Kazykhankyzy and Alagozlu’s (2019) aimed to measure Turkish and Kazakhstani ELT pre-service teachers’ ICC. They used Byram’s (1997) conceptualization of ICC involving knowledge, skills, attitudes and awareness while preparing the ICC scale. Similarly, in our newly designed ICC scale, we used Byram’s (1997) conceptualization of ICC to name the four factors found as a result of the EFA and the CFA. Their 52-item ICC scale was found highly (χ²/df = 1.8, p = .000) reliable with Cronbach Alpha reliability of .958. In their scale, the factor loading values of the items varied between .324 and .774. On the other hand, in our scale, the factor loading values of the items varied from .651 to .822. Therefore, the items in our newly designed ICC Scale were found highly reliable. Kazykhankyzy and Alagozlu’s (2019) scale’s explained total variance rate was calculated as 50.48% whereas in our study, the total explained variance rate was calculated as 55.577% which was above the lowest limit (.50) stated by Yaşlioglu (2017).

Fantini and Tirmizi’s (2006) study could be regarded as another parallel study to ours although their ICC tool aimed to measure the ICC of the participants in the Federation of The Experiment in International Living (FEIL) programs regarding their lives and work. Four of the items on our ICC scale were adapted from their ICC scale. In their research project of the Federation of The Experiment in International Living (FEIL), Fantini and Tirmizi (2006) aimed to analyze Intercultural Competence of participants in civic service programs in three Member Organizations (Great Britain, Ecuador, and Switzerland) in a learning process. Their scale involved 30 items and four factors (knowledge, attitude, skills, and awareness). Their scale was found a highly reliable research measurement tool (Cronbach Alpha reliability = .824, the total explained variance rate = 69.53%).

Finally, the newly designed 18-item ICC Scale for students with four factors was found to be a highly reliable instrument (Cronbach Alpha reliability = .829). Furthermore, it was found a fit model (χ²/df value = 2.342) to measure the Intercultural Communicative Competence of students in a multicultural educational setting (Tabachnick & Fidell, 2011; Yaşlioglu, 2017).

Limitations of the Study

In order to measure the ICC level of students, the researchers designed a new ICC Scale. However, it should be noted that it may not be possible to measure the “real” level of students’ ICC or to make generalizations about the implementation of this scale. In other words, only quantitative data about students’“perceived” ICC levels can be collected and analyzed using this ICC Scale. The ICC level as perceived by students may not necessarily be exactly the same as the ICC level practised. To overcome this possible limitation, we strongly suggest that researchers collect qualitative data as well (e.g., through interviews, student journals/diaries, portfolios, class observations, field notes) in order to triangulate with the quantitative data from the Scale so as to obtain more realistic pictures of the participants’ ICC levels.

In the current study, the perceived ICC level of the Academic English students were measured with the newly designed ICC Scale. However, the ICC Scale can also be used in measuring the perceived ICC level of students taking General English or English for Specific Purposes (ESP). Moreover, the newly designed ICC Scale can also be used to measure the perceived ICC level of students studying in any multicultural context such as secondary and high school as long as their English proficiency level is adequate to fully understand the items. Furthermore, the scale can be translated into other languages. Translated versions of the Scale can be used to determine the perceived ICC level of students of a single culture studying in a multicultural context. Furthermore, the Scale can also be used in investigating the perceived ICC level of students learning other languages, for example, French, Spanish, or German.

The study investigated the ICC of Academic English students only in one university in North Cyprus. On the other hand, the perceived ICC level of Academic English students studying in different contexts could be investigated. Moreover, the perceived ICC level of students from one particular nation can be researched.

Footnotes

Authors Contributions

ZE performed the initial data collection and data analysis. All authors revised and approved the submitted version of the manuscript.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Ethical Approval

This study was carried out with the permission of the Ethics Committee of Eastern Mediterranean University (approval number ETK00-2022-0137) with written informed consent from all subjects.

ORCID iDs

Zehra Ertay

Ilkay Gilanlioglu

Data Availability Statement

Data used for this study will be made available upon reasonable request.

References

Akcaoğlu

M. Ö.

Arsal

(2021). The effect of multicultural education on preservice teachers’ attitude and efficacy: Testing Bank’s content integration dimension. Participatory Educational Research (PER), 9(2), 343–357. https://doi.org/10.17275/per.22.44.9.2

Alptekin

(2002). Towards intercultural communicative competence in ELT. ELT Journal, 56(1), 57–64. https://dx-doi-org.web.bisu.edu.cn/10.1093/elt/56.1.57

Badrkoohi

(2018). The relationship between demotivation and intercultural communicative competence. Cogent Education, 5(1), 1531741. https://doi.org/10.1080/2331186X.2018.1531741

Baroudi

(2017). Intercultural communicative competence (ICC) in ELT classrooms [Master’s thesis, Foreign Language Education Department, Eastern Mediterranean University].

Bennett

(1993). Towards ethnorelativism: A developmental model of intercultural sensitivity. In Paige

(Ed.), Education for the intercultural experience. Intercultural Press.

Bennett

J. M.

Bennett

M. J.

(2004). Developing intercultural sensitivity. In Bennett

J. M.

Bennett

M. J.

Landis

(Eds.), Handbook of intercultural training (3rd ed., pp. 147–165). Sage.

Bentler

P. M.

(1990). Comparative fit indexes in structural models. Psychological Bulletin, 107(2), 238–246. https://doi.org/10.1037/0033-2909.107.2.238

Bentler

P. M.

Bonett

D. G.

(1980). Significance tests and goodness-of-fit in the analysis of covariance structures. Psychological Bulletin, 88, 588–600. https://doi.org/10.1037/0033-2909.88.3.588

Bhawuk

D. P. S.

Brislin

(1992). The measurement of intercultural sensitivity using the concepts of individualism and collectivism. International Journal of Intercultural Relations, 16(4), 413–436. https://doi.org/10.1016/0147-1767(92)90031-O

10.

Bollen

K. A.

(1990). Overall fit in covariance structure models: Two types of sample size effects. Psychological Bulletin, 107, 256–259. https://doi.org/10.1037/0033-2909.107.2.256

11.

Bouchard

(2017). Ideology, agency, and Intercultural Communicative Competence. Singapore: Springer Nature.

12.

Byram

(1997). Teaching and assessing intercultural communicative competence. Multilingual Matters.

13.

Byrne

B. M.

(1998). Structural equation modeling with LISREL, PRELIS and SIMPLIS: Basic concepts, applications and programming. Lawrence Erlbaum Associates.

14.

Byrne

B. M.

(2011). Structural equation modelling with AMOS basic concepts, applications, and programming (Multivariate Applications Series). Routledge.

15.

Chao

(2014). The development and application of an intercultural competence scale for university EFL learners. English Teaching & Learning, 38(4), 79–124. https://doi.org/10.6330/ETL.2014.38.4.04

16.

DeVellis

R. F.

(2003). Scale development: Theory and applications (2nd ed.). Sage.

17.

Duisembekova

(2021). Beliefs about intercultural communicative competence: The development and validation of a new instrument. International Journal of Instruction, 14(2), 103–116. https://doi.org/10.29333/iji.2021.1427a

18.

Erden

(2018). African university students’ intercultural experiences with impeding factors: Case from Northern Cyprus. Eurasian Journal of Educational Research, 18(77), 81–104.

19.

Fantini

Tirmizi

(2006). Exploring and assessing intercultural competence. World Learning Publications.

20.

Field

(2009). Discovering statistics using SPSS (3rd ed.). Sage.

21.

Guyton

E. M.

Wesche

M. V.

(2005). The multicultural efficacy scale: Development, item selection, and reliability. Multicultural Perspectives, 7(4), 21–29. https://doi.org/10.1207/s15327892mcp0704_4

22.

Hismanoglu

(2011). An investigation of ELT students’ intercultural communicative competence in relation to linguistic proficiency, overseas experience and formal instruction. International Journal of Intercultural Relations, 35, 805–817. https://doi.org/10.1016/j.ijintrel.2011.09.001

23.

Hogan

Sellar

Lingard

(2015). Commercialising comparison: Pearson puts the TLC in soft capitalism. Journal of Education Policy, 31(3), 243–258. https://doi.org/10.1080/02680939.2015.1112922

24.

Hooper

Coughlan

Mullen

M. R.

(2008). Structural equation modelling: Guidelines for determining model fit. The Electronic Journal of Business Research Methods, 6(1), 53–60.

25.

L. T.

Bentler

P. M.

(1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1–55. https://doi.org/10.1080/10705519909540118

26.

Hutchinson

Waters

(1987). English for specific purposes: A learner-centered approach. Cambridge University Press.

27.

Jenkins

(2006). Current perspectives on teaching World Englishes and English as a lingua franca. TESOL Quarterly, 40(1), 157–181. https://doi.org/10.2307/40264515

28.

Kazykhankyzy

Alagozlu

(2019). Developing and validating a scale to measure Turkish and Kazakhstani ELT pre-service teachers’ intercultural communicative competence. International Journal of Instruction, 12(1), 931–946. https://doi.org/10.29333/iji.2019.12160a

29.

Kramsch

(1993). Context and culture in language teaching. Oxford University Press.

30.

Lei

(2020). A survey on preservice English teachers’ intercultural communicative competence in China. English Language Teaching, 14(1), 37–47. https://doi.org/10.5539/elt.v14n1p37

31.

McCombes

(2020). Descriptive research. Scribbr. https://www.scribbr.com/methodology/descriptive-research/

32.

Munezane

(2019): A new model of intercultural communicative competence: Bridging language classrooms and intercultural communicative contexts, Studies in Higher Education, 46(8), 1664–1681. https://doi.org/10.1080/03075079.2019.1698537

33.

Nurullina

G. M.

Muraviyov

A. F.

Martyanova

A. A.

Yarmakeev

I. E.

(2018). Project technology in the development of communicative competence in schoolchildren: Extracurricular classes of Russian language. Cypriot Journal of Educational Sciences, 13(4), 461–468. https://doi.org/10.18844/cjes.v13i4.3897

34.

Ozbilgehan

Celenk

(2021). A review of multicultural education in Northern Cyprus and Turkish learning levels of students from different cultural backgrounds. Revista De Cercetare Si Interventie Sociala, 73, 114–132. https://doi.org/10.33788/rcis.73.8

35.

Pihama

(2019). Colonisation, neoliberalism and Māori education. New Zealand Journal of Educational Studies, 54(1), 5–19. https://doi.org/10.1007/s40841-019-00130-7

36.

Pourakbari

A. A.

Tabrizi

H. H.

Chalak

(2018). An investigation of Iranians and international English students’ attitudes towards intercultural communicative competence. The Journal of English Language Pedagogy and Practice, 11(23), 151–179. https://doi.org/10.30495/JAL.2019.664544

37.

Sarıçoban

(2014). Research into pre-service English teachers’ intercultural communicative competence (ICC) in Turkish context. The Anthropologist, 18, 523–531. https://doi.org/10.1080/09720073.2014.11891570

38.

Seidlhofer

(2006). English as a lingua franca in the expanding circle: What it isn’t. In Rubdy

Saraceni

(Eds.), English in the world: Global rules, global roles (pp. 40–50). Continuum.

39.

Sevimel-Sahin

(2020). A survey on the intercultural communicative competence of ELT undergraduate students. Eurasian Journal of Applied Linguistics, 6(2), 141–153. https://doi.org/10.32601/ejal.775793

40.

Tabachnick

B. G.

Fidell

L. S.

(1996). Using multivariate statistics (3rd ed.). HarperCollins.

41.

Tabachnick

B. G.

Fidell

L. S.

(2011). Using multivariate statistics. Pearson.

42.

Toscu

Erten

I. H.

(2020). Developing intercultural communicative competence by the means of telecollaboration. Education and Information Technologies, 25, 4517–4534. https://doi.org/10.1007/s10639-020-10174-8

43.

Üzüm

Akayoglu

Yazan

(2020). Using telecollaboration to promote intercultural competence in teacher training classrooms in Turkey and the USA. ReCALL, 32(2), 162–177. https://doi.org/10.1017/S0958344019000235

44.

Uzunboylu

Altay

(2021) State of affairs in multicultural education research: A content analysis. Compare: A Journal of Comparative and International Education, 51(2), 278–297. https://doi.org/10.1080/03057925.2019.1622408

45.

Wheaton

Muthen

Alwin

D. F.

Summers

(1977). Assessing Reliability and Stability in Panel Models. Sociological Methodology, 8, 84–136.

46.

Yaşlioglu

M. M.

(2017). Factor analysis and validity in social sciences: using exploratory and confirmatory factor analysis. Journal of İstanbul University Faculty of Business Administration, 46, 74–85.

47.

Zhou

(2011). A study of Chinese university EFL teachers and their intercultural competence teaching [Unpublished doctoral dissertation, University of Windsor, Ontario].