Abstract
In the past 20 years, global integration has peaked, the movement of people around the world has increased, and migration has more than doubled (Strauss, 2020). Increased globalization in the workplace has brought about increases in international assignments, travel, and collaborations (Hammond and Grosse, 2003; Shaffer et al., 2012), all of which require organizational employees to engage in any number of cross-cultural interactions. Accordingly, the extent to which individuals succeed in these cross-cultural interactions is an important focal topic for organizational leaders and researchers. Successful performance in cross-cultural settings has been studied in a number of different organizational groups and contexts, including expatriate employees (e.g. Wang et al., 2013), international students (e.g. Burke et al., 2009), non-governmental organizations (e.g. Charleston et al., 2018), military populations (e.g. Hardison et al., 2009), global leaders (e.g. Caligiuri and Tarique, 2012), and global or multicultural teams (e.g. Yitmen, 2013). Given the increased globalization in multiple organizational settings, the need to appropriately define and measure cross-cultural performance is becoming more pronounced.
To date, much of the cross-cultural management research has centered its focus on the operationalization of predictors that influence how individuals perform in cross-cultural situations. There is a wide body of research examining constructs such as cross-cultural competence, cultural intelligence, and intercultural competence, which are terms that have sometimes been used interchangeably (Deardorff, 2015), but are generally conceptualized as some combination of knowledge, skills, abilities, and other characteristics that enable individuals to perform effectively in cross-cultural settings (Ang and Van Dyne, 2008; Charleston et al., 2018; Chiu et al., 2013; Gallus et al., 2014; Hammer et al., 2003; Jyoti and Kour, 2017; Li, 2020; Velez-Calle et al., 2018).
Upon closer examination of the research on cross-cultural competence, cultural intelligence, intercultural competence, and related constructs, most of the time, the predicted outcome of these constructs has not been well-defined or consistently measured. There is generally a lack of research that explicitly focuses on providing an unambiguous definition regarding what constitutes successful performance in cross-cultural interactions, or what we will refer to as
In other words, most research examining cross-cultural performance defines the concept using very general and vague terms, such as effective performance, effective interaction, success, functioning, and appropriate behavior in cross-cultural contexts (e.g. Charleston et al., 2018; Li, 2020; Velez-Calle et al., 2018) without providing any careful analysis regarding what those terms actually mean, and how they should be measured. In sum, cross-cultural management research seems to be experiencing a criterion problem (Austin and Villanova, 1992) in which there is not a consistent framework for how to define and measure cross-cultural performance. Without an agreed upon conceptualization and operationalization, it is difficult to systematically study the nomological network surrounding cross-cultural performance in a way that moves toward scientific consensus.
Criteria and the Criterion Problem
Before delving into the ways in which performance criteria have been addressed (or have failed to be addressed) within the domain of cross-cultural research, it is important to first briefly explore the notion of “criterion” as a concept, and what, more specifically, contributes to it being characterized as problematic. A criterion, at its most fundamental level, reflects a standard or basis for comparison (Warren, 1934). The “what” being compared is often the performance of individuals, but can extend to groups of individuals (i.e. teams; LePine et al., 2008) and organizations (Boon et al., 2019), as well. Criteria are perhaps most well-known for their role within the context of validation, with a “criterion” being the outcome or behavior that a particular predictor measure (e.g. a test, assessment, or survey) is trying to predict (Sireci and Sukin, 2013). In this sense, a criterion is akin to the dependent variable in a regression equation, with the primary difference being that criteria are defined for the purpose of making a judgment or decision (Campbell, 2012; Oswald et al., 2004), whether that judgment be about the overall predictive validity of a test, or the performance of an employee relative to others. Finally, the conceptual criterion agreed upon by organizational decision makers or researchers must be translated into a concrete operational criterion measure (Slife et al. 2016), such that data can be collected, comparisons can be tabulated, and decisions or inferences made.
Criteria measurement has been particularly important in applied research domains where performance is often the primary focus. Perhaps not surprisingly, some of the earliest discussions related to criteria measurement occurred within the industrial-organizational psychology literature concerning the role of tests or measures in optimizing the selection of individuals into particular jobs (Münsterberg, 1913; Scott, 1917). Since then, the use of criteria has been ubiquitous. However, it is perhaps the ubiquity of criteria’s role as the “outcome variable” that leads to it being overshadowed by efforts to develop and validate predictor measures (Dunnette, 1963). This dynamic is a large factor in what has come to be known as the criterion problem in organizational research. The criterion problem is not new, nor is it unique to the domain of cross-cultural research. The measurement of criteria has been recognized as a critical component of performance-related research for nearly as long as academic fields concerned with the measurement of human performance have existed.
Brief Review of Cross-Cultural Performance Research
In preparation for our special issue, we conducted a selective review of the literature to sample representative published research surrounding the conceptualization, measurement, and study of cross-cultural performance. Below we briefly summarize some of the key patterns and themes distilled from this reviewed literature.
Definitions of Cross-Cultural Performance
The first question we explored within the published literature is how published research is defining and conceptualizing the criterion space within cross-cultural management studies. It is important to note that most of this literature does not use the exact term “cross-cultural performance” but instead uses other terms (e.g. sociocultural performance; Wisecarver et al., 2014; international performance; Burke et al., 2009) to refer to some type of criterion variable examined in a cross-cultural work performance setting. Research tends to define the criterion space in these studies in one of four major ways: (1) defining and labeling the criterion as cross-cultural competence (Wang et al., 2013; Charleston et al., 2018; Friedman et al., 2013), (2) defining the criterion as general performance in a non-culture specific sense (e.g. Deshpende and Viswesvaran, 1992; Presbitero and Toledano, 2018; Rose et al. 2010), (3) defining and labeling the criterion as subjective psychological adjustment of the employee (e.g. Lin et al., 2012; Okpara, 2016; Iskhakova, 2018; Tucker et al. 2004; Selmer and Lauring, 2016), and (4) not providing any formal definition of the criterion (e.g. Lee and Donohue, 2012; Lima et al., 2016).
The research that defines the criterion as cross-cultural competence is looking at some set of knowledge, affect, and skill competencies that individuals hold and that develop in response to experience, training, and education (e.g. Abbe et al., 2007). This research is very beneficial in that it establishes that these competencies can be improved through interventions like cross-cultural training (e.g. McCloskey et al., 2010), but it stops short of actually measuring performance in cross-cultural settings in a behavioral sense. In other words, there is a difference between measuring an individual’s capability to perform in a culturally competent way and measuring whether or not an individual is actually enacting that cultural competence during a specific cultural interaction or assignment.
A proportion of the research has studied more traditional forms of performance within cross-cultural settings. For example, Lee and Donohue (2012) developed a scale that measures task performance, communication performance, demonstrating effort, maintaining personal discipline, team and leadership performance, and management and administration performance in expatriates (Campbell et al., 1990). This approach is commendable in terms of acknowledging that performance is a complex and multifaceted domain, but it does not conceptualize performance as including any elements specific to performing in cross-cultural contexts as part of the criterion space. Additionally, because performance ratings were either self-reported or provided by supervisors, and it is unclear the extent to which these supervisors actually had direct knowledge of the performance of their expatriates while on assignment, it is also unclear whether ratings on these performance dimensions truly reflect the concept of behavioral cross-cultural performance.
Finally, a large proportion of the cross-cultural management literature examines the experience of adjustment, which is defined as a psychological outcome regarding the degree of comfort individuals feel in their new role (e.g. Lin et al., 2012; Okpara, 2016). Psychological adjustment is no doubt one important aspect of performing well in cross-cultural contexts, however, feeling psychologically comfortable does not necessarily mean that a person is performing effectively in cross-cultural settings. In fact, it is possible that performing effectively could require an individual to put themselves in uncomfortable situations or to adjust their behavior in a way that is unfamiliar, in which case feeling psychologically comfortable may not be the best indicator of effective performance. Overall, while the above approaches to conceptualizing the criterion space in cross-cultural management provide useful insights and solid starting points, they also provide ample opportunity for furthering exploration of what it truly means to perform effectively in cross-cultural work situations.
There are a few exceptions to the above categories of research where studies provided more complex or formal definitions of their cross-cultural criterion space. For example, Shaffer et al. (2006) combine several of the above perspectives to look at expatriate effectiveness in a multifaceted way by measuring expatriate adjustment, withdrawal cognitions, and job (i.e. task and contextual) performance. In another example, Rockstuhl et al. (2011) studied cross-border leadership effectiveness and defined that criterion as the “effectiveness of observable actions that managers take to accomplish their goals in situations characterized by cross-border cultural diversity. This aspect of global leaders’ effectiveness explicitly recognizes and emphasizes the unique challenges of heterogeneous national, institutional, and cultural contexts (Shin et al., 2007, pp 826).”
Finally, Ferro (2014) put forth maybe the only explicit standalone theory of a culture-specific performance criterion: a theory of cultural adaptive performance, which they posit is composed of two components: learning behaviors and integrating behaviors. Learning behaviors are those that facilitate the effective adjustment of performing tasks/behaviors in a new cultural environment. Integrating behaviors reflect actions that indicate one is modifying his or her behavior based on what was observed and learned through past learning behaviors and correcting for past mistakes or replacing ineffective behaviors with behaviors that are more likely to be effective in a different culture. However, this theory has yet to be explicitly tested.
In sum, there are several clear themes in terms of the gaps in criterion conceptualizations within the cross-cultural management space. First, there are very few dedicated theoretical efforts focused on clarifying this criterion space. Second, studies that define the criterion as psychological adjustment and performance in general are undoubtedly predicting important outcomes for individuals working in culturally distant and diverse contexts, but it is still unclear if these concepts alone adequately or completely reflect the cross-cultural performance criterion space. Cross-cultural management research thus far may be missing important elements of what it means to perform effectively in cross-cultural contexts (e.g. tolerating discomfort, cultural learning) simply because the criterion space wasn’t theorized very carefully. Third, studies looking at cross-cultural competence, or any other trait-like variable (e.g. cultural intelligence) as the criterion, are valuable in terms of studying if these competencies can be trained, but they generally don’t make a clear theoretical distinction between the dispositional variable (i.e. trait) and the behavioral performance (i.e. state) variable. Given many other areas of study have emphasized the importance of maintaining a distinction between trait and state (e.g. affect, Watson et al., 1988; self-regulation, Inzlicht et al., 2021), this seems like a ripe opportunity for moving the science of cross-cultural management forward.
Measures of Cross-Cultural Performance
Examining the approaches and measures used to assess cross-cultural performance also reveals an area ripe for study. There are existing validated measures of cross-cultural competence (Matsumoto and Huang, 2013; Van Driel and Gabrenya, 2013), cultural intelligence (e.g. Van Dyne et al., 2008), and intercultural competence (e.g. Hammer et al., 2003), and, generally, more research has focused on assessing the predictors of cross-cultural performance than on assessing cross-cultural performance itself (e.g. Lima et al., 2016; Velez-Calle et al., 2018). While many theoretical and methodological issues remain for existing cross-cultural competence measures (Van Driel and Gabrenya, 2013), the measurement of cross-cultural performance is even less well developed. Cross-cultural performance has been assessed using validated measures of adaptability (Lima et al., 2016; Matsumoto and Huang, 2013), adjustment (Chen et al., 2014; Okpara, 2016), and job performance (e.g. Jyoti and Kour, 2017), but no widely accepted and validated measure specific to cross-cultural performance currently exists.
Furthermore, criterion measures of cross-cultural performance also often overlap conceptually with predictor measures. In fact, in some cases, cross-cultural competence measures have been used and interpreted as criterion measures of performance (e.g. Lima et al., 2016; Velez-Calle, 2018) rather than as the predictor. Given the overlap, these attempts at criterion measurement succumb to the same criticisms of existing cross-cultural competency measures, namely subjective bias (Borman, 1991), and accessibility/observability of the assessed construct (Gabrenya et al., 2011). Some researchers have argued in favor of more performance-based measures of cross-cultural competence and its related constructs (e.g. Deardorff, 2015; Chiu et al., 2013). In other words, there exists ample conceptual muddiness around these phenomena that suggests a need to detangle the conceptualization and measurement of cross-cultural competence from the conceptualization and measurement of cross-cultural performance.
In sum, there remain several challenges in accurately conceptualizing and measuring cross-cultural performance, but with focus, these challenges are surmountable. The most noteworthy challenge is the varied and/or lacking definitions regarding performance in the international space. Stemming from that gap, another significant challenge is the resulting frameworks, analogous to the work of Campbell et al. (1990), which delineate the dimensions comprising cross-cultural performance. Definitions inform frameworks, and frameworks enable quality indicator development. Once those gaps are closed, the subsequent challenges in improving cross-cultural performance across the globe, while real, are tactical in nature, and can be addressed through best practices.
Papers in this Special Issue
Although possibly taboo to admit, despite our passion for the topic, our call for papers did not elicit an overwhelming number of submissions. We speculate that, in part, this may have been driven by the lingering negative effects of the COVID-19 pandemic on researchers' time and energy. We understand that if we were affected as editors, then of course, potential authors were affected as well. In part, however, we also speculate that there simply is still not enough thinking and research being done explicitly on the “criterion side” of cross-cultural management research, which only reinforces the critical and timely nature of this special issue. Ultimately, we were pleased to receive and accept several high quality articles that move the cross-cultural performance research forward in a meaningful way. Our set of six articles provide novel perspectives on improving the criterion space surrounding cross-cultural performance including a deep dive into the criterion issue within cross-cultural management, several theoretical articles that each conceptualize cross-cultural performance in a different way, one qualitative study exploring a unique aspect of cross-cultural performance, and one case study of a high stakes, real-world criterion measure development effort.
Our first article by Harrison Kell sets the stage for the other articles by providing a deep-dive into the criterion problem in social science research, both in a general sense and also specifically within cross-cultural management. More specifically, this paper outlines a number of cross-cutting problems that affect the quality of criteria including dimensionality, behaviors versus results, timeframes, and operationalization of criteria. This article will serve as a useful primer for scholars and practitioners interested in improving their development and measurement of cross-cultural performance as a criterion, which could have broad-reaching ramifications for improving the state of the science within cross-cultural management.
Our next several papers provide unique theoretical perspectives on the conceptualization of cross-cultural performance. Paula Caligiuri and colleagues revisit prior work from 1997 to reimagine and broaden the criterion space surrounding cross-cultural performance, highlighting the distinct dimensions of task performance, skill development, and cross-cultural adjustment. Regarding task performance, they refine previous thinking to argue that two criteria are critical to fully understanding task performance in cross-cultural settings: (1) how accurately employees read the contextual demands of the cross-cultural context and (2) how effectively they respond to those contextual demands. Regarding skill development, they posit that the development of cross-cultural competencies is in and of itself part of what it looks like to engage in successful cross-cultural performance. Finally, regarding adjustment, they move beyond the traditional “one-size-fits-all” approach to suggest a more individualized person-environment fit approach. This approach provides several interesting new avenues for future research on and measurement of cross-cultural performance.
Jennifer Feitosa and colleagues take a very different perspective on reconceptualizing cross-cultural performance by adopting a diversity, equity, and inclusion (DEI) lens. They define cross-cultural performance as the integration of multiple cultural perspectives with a specific goal to enable a diverse, inclusive, and equitable environment. Furthermore, they identify three main components of cross-cultural performance as defined from this DEI perspective: catalyzing cultural differences, using engaging communication, and promoting allyship. They also take a very specific multilevel and behavioral perspective on cross-cultural performance, viewing it as a criterion in itself, but also as a predictor of other more distal outcomes at the individuals, team, and organizational levels. We find this to be an intriguing and refreshing take on the criterion space of cross-cultural performance, and we agree with these authors that the fields of cross-cultural management and DEI have much to potentially gain from thoughtful cross-pollination of ideas.
Xiaowen Chen and colleagues zoom in on one of the most well-known contexts for studying cross-cultural performance: the expatriate employee experience. Drawing from the broader conceptualizations of job performance, they define expatriate effectiveness as a complex and multidimensional concept consisting of task, contextual, and adaptive performance. Their focus on contextual performance and adaptive performance are particularly insightful in that they suggest several specific sub-dimensions (e.g. building and maintaining relationships, transferring knowledge, following host organizations rules, learning and respecting the host country’s culture) that speak to some of the unique challenges and needs of cross-cultural work settings in particular. The authors go on to provide several concrete guidelines for more accurately operationalizing expatriate effectiveness, making this another article in the special issue that will have clear practical implications for those wanting to assess and/or study cross-cultural performance themselves.
Norhayati Zakaria and colleagues provide one of our only empirical investigations of cross-cultural performance in the form of a qualitative interview study exploring the phenomenon of cultural code-switching within global virtual teams. Using multinational corporation employees from Malaysia, this research explores how global virtual team members switch their behaviors from a high-context style towards a more low-context style, namely using directness in speech, openness in knowledge sharing, and keeping a focus on the task, in order to enable better collaboration and knowledge sharing in diverse teams. This work provides a rich, narrative description regarding why and how cultural code-switching may reflect one aspect of cross-cultural performance, hopefully driving future research to more fully consider the concept of code-switching as part of the broader cross-cultural performance criterion space.
Last, but certainly not least, Jonathan Kochert and colleagues describe a fascinating case study of a cross-cultural performance assessment development project sponsored by the U.S. Army. This effort resulted in two uniquely tailored measures of cross-cultural performance aimed at assessing Soldier performance on either 10 or five key dimensions of cross-cultural performance. This case study is then used to highlight five critical issues to be considered when designing and developing measures of cross-cultural performance: criterion conceptualization, rater source, standardization, retrospective versus in-situ sayings, and rater usability. For researchers and practitioners that have not yet engaged in the complex process of criterion measure development, this article serves as an insightful inside look at key steps in the process and considerations along the way. Taken together, we are hopeful that this special issue will encourage researchers and practitioners alike to more deeply think about, more carefully conceptualize, and more accurately measure cross-cultural performance.
Footnotes
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.