Cultural patterns of evasive answer bias in surveys

Abstract

Complete and accurate survey data are key input for research, policy, and decision making in many disciplines. However, survey respondents do not always fully cooperate, such that they skip some items or overuse the “don’t know” answer option. Evasive answer bias reflects different information than overall survey response rates, leading to item missing data and causing substantial inaccuracies in survey results. Using data from the World Values Survey, this article identifies the magnitude of the problem, then relies on individual data and country-level cultural values to derive patterns of and reasons for this evasive answer bias. While skipping answers happens less often in collectivistic and low power distance cultures, the choice of the “don’t know” option is not significantly influenced by any cultural dimension. Across countries, the effect of cultural values is stronger for female than for male respondents. Accordingly, cross-cultural researchers are advised to use advanced imputation rather than deletion methods for handling missing data.

Keywords

Cross-national differences evasive answer bias item missing data item nonresponse survey questionnaires

Introduction

Sample surveys are an important basic research method and tool in the social sciences, fueling an industry with a multibillion-dollar annual turnover (Wright and Marsden, 2010). Increasingly, government agencies and economic enterprises require complete and accurate data to respond to their constituents’, customers’ and other stakeholders’ needs. They use data to learn about society, and provide their patrons with targeted information, advertisements, and means for self-expression, often in exchange for discounts or small gifts (Messner, 2004; Steiner and Maas, 2018). An important source of these data are the patrons themselves (Cazier et al., 2007), such as when they complete survey questionnaires (Bowling, 2005). To understand and access international markets, organizations increasingly conduct surveys across national borders too (e.g. Bellman et al., 2004; Brown et al., 2007).

Despite this spread, people may react negatively to their data being collected, profiling efforts, and unpredictable or unwanted uses of their information (Cazier et al., 2007). Growing exposure to marketing efforts and over-surveying have intensified such negative perceptions (Gummer, 2019; Presser and McCulloch, 2011) and diminished respondents’ willingness to cooperate, which in turn raises concerns about survey accuracy (Fuchs et al., 2013; Gummer, 2019; Wagner and Stoop, 2018). That is, improving survey response rates is critical, but response rates represent only a part of the issue (Durand et al., 1983).

In addition to nonresponse bias, evasive answer bias manifests itself in item nonresponse, that is, in skipping questions or overusing the “don’t know” option (Kamakura and Wedel, 2000). To be clear, evasive answer bias is different from nonresponse bias (Couper and De Leeuw, 2003). While the latter relates to survey response rates (aka, unit response rates), the former describes the phenomenon of nonresponse at the item level (aka, item missing data). This article is exclusively about evasive answer bias at the item level.

Furthermore, respondents’ answers may reflect their conscious or unconscious biases (Osborne and Blanchard, 2011), such as consistency bias, social desirability bias, or biases reflecting inflated views of one’s own performance (Harzing et al., 2012). Respondents also exhibit systematic tendencies in their responses, regardless of the content (Baumgartner and Steenkamp, 2001), including acquiescence, disacquiescence, middle, and extreme response styles (Harzing, 2006; Watkins and Cheung, 1995).

Unmotivated survey respondents “can cause substantial misestimation of results, biasing results towards the null hypothesis” (Osborne and Blanchard, 2011: 1), leading to data quality problems. Evasive answer bias hides actual responses that could be meaningful for analysis (Little and Rubin, 2002). Moreover, researchers often resolve item missing data with list- or pairwise deletion (Myers, 2011). If the bias is random, the cleaned data set is representative of the sample and the statistical inferences remain valid. But if the bias can be explained systematically by specific factors, the cleaned data is no longer representative of the sample and the resulting inferences will be problematic (Jensen et al., 2010).

Similar to differences in consumer behavior linked to national cultures (Malhotra et al., 2005), international surveys often reveal response style biases that vary across countries, cultures, or languages (Harzing, 2006; Harzing et al., 2012; Messner, 2017). Accordingly, multigroup confirmatory factor analysis has become a method of choice for establishing measurement equivalence in cross-cultural research (Welzel et al., 2019). Yet, “there is a lack of research that investigates how cultural differences across countries affect consumers’ willingness to share personal information” (Schumacher and Maas, 2019, CC41), and little is known about how such differences affect unit or item response rates (Lyness and Kropf, 2007). For example, differences in answer bias between countries could look like differences in content, leading researchers to derive inaccurate insights from the incorrectly assumed differences (Van Rosmalen et al., 2010). Nor does extant research detail how the influence of individual demographic characteristics and psychological traits shifts across cultures (Jans, et al., 2018; Roxas and Stoneback, 2004). Attempts to link cultural and survey nonresponse theory are relatively new in extant research (Jans, et al., 2018; Johnson et al., 2010), and so Zhou et al. (2017) suggest researching whether respondents’ cultural and socialization background influence their answer behavior.

In response, the current study seeks to explore the reasons for evasive answer bias, examine trends across countries and cultures, and reveal how individual-level drivers might be moderated by cultural values. Observational data from the World Values Survey (WVS; Inglehart et al., 2014) combined with the GLOBE study’s cultural framework (House et al., 2004) help demonstrate how personal characteristics and cultural predispositions influence, in sometimes conflicting ways, two measures of evasive answer bias: skipping questions and choosing the “don’t know” answer. The article’s final section then details implications. It advises cross-cultural researchers to address missing data in international surveys through advanced imputation rather than deletion methods.

Resource exchange and leverage saliency theory

Resource exchange theory demonstrates how people’s willingness to disclose information relates to the benefits they receive (Brinberg and Wood, 1983). When completing surveys, for each item, respondents typically undergo a four-step process: (1) comprehend the question, (2) recall information from memory, (3) map retrieved information to the question asked, and (4) communicate a response (Bowling, 2005; Tourangeau, 1984). But respondents also may elect to skip certain questions (Warner, 1965) or choose an “I don’t know” answer option (Dobronte, 2014). In particular, they tend to avoid questions that require a longer view into the future, cover remote or controversial topics, are too revealing, come along with complex instructions (Converse, 1976), or require absolute answers (Shoham, 1998).

Leverage saliency theory proposes that respondents’ characteristics determine their likelihood of cooperating in a survey (Groves et al., 2000; Gummer, 2019), and that different people place a different level of importance to survey requests (Seifert, 2008). Previous research indicates that age, education, gender, and income determine the number of non-substantive responses people provide (e.g. Craig and McCann, 1978; Feick, 1989; Francis and Busch, 1975), though research into item nonresponse phenomena has been hindered by a lack of sufficient data (Ferber, 1966), leaving limited insights into random and evasive responders (Furnham et al., 2015). Still, the possibilities of modern technologies and associated risks of data fraud have propelled information disclosure topics into the limelight in current academic research and management practice (Steiner and Maas, 2018).

Hypotheses

In the following, the hypotheses are structured into individual- and group-level effects, pertaining to how respondents’ personal characteristics and cultural predisposition affects evasive response bias. Table 1 lists the sequence of hypotheses, and Figure 1 shows them in the context of the research model.

Table 1.

Overview of hypotheses.

	Hypothesis	Type	Measured by number of
	Hypothesis	Type	Unanswered questions	Don’t know answers
1	Female (Male) respondents evade questions more (less)	Individual-level effect	Supported	Supported
2	Older (younger) respondents evade questions more (less)	Individual-level effect	Supported	Supported
3	Respondents with higher (lower) income evade questions less (more)	Individual-level effect	Supported	Supported
4	Respondents who are generally thorough (careless) evade questions less (more)	Individual-level effect	Supported	Rejected
5	Respondents who generally trust (do not trust) other people evade questions less (more)	Individual-level effect	Rejected	Rejected
6	Respondents who generally trust (do not trust) major companies evade questions less (more)	Individual-level effect	Rejected	Supported
7	Respondents from countries with predominantly collectivistic (individualistic) values evade questions less (more)	Group-level direct effect	Supported	Rejected
8	The effect of female gender on evasive answer bias is stronger in countries with predominantly individualistic values	Cross-level interaction	Supported	Rejected
9	Respondents from countries with predominantly high (low) power distance values evade questions more (less)	Group-level direct effect	Supported	Rejected
10	The effect of female gender on evasive answer bias is stronger (weaker) in countries with predominantly high (low) power distance values	Cross-level interaction	Supported	Rejected

This table lists the hypotheses. The results are recorded separately for the unanswered questions (NOANSW) and questions marked with “don’t know” (DNTKNW).

Figure 1.

Multilevel research model and variables. This is the multilevel research model. The hypotheses H1 to H6 examine individual-level effects (level 1); H7 and H9 model group-level direct effects (level 2); H8 and H10 test cross-level interactions between level 2 and 1. Table 1 provides further information about the hypotheses; Table 2 records the variables and their abbreviations.

Individual-level effects

Gender identity theory argues that gender phenomena are multifactorial, consisting of biological sex, gender role attitudes, and psychological traits (McCabe et al., 2006; Spence, 1993; Spence and Helmreich, 1978). In general, women show more expressive, interpersonally oriented, and caring psychological traits than men. They are more emotional and adopt different perspectives. Men instead tend to be more ambitious, autonomous, aggressive, confident, and proactive (e.g. Kidder and Parks, 2001). Bossuyt and Van Kenhove (2018) find that the gender difference in assertiveness remains robust even after controlling for social desirability bias. Even during the COVID-19 pandemic, women suffered greater emotional and life distress than men (Ding et al., 2021).

According to gender socialization theory, social factors and influences also cause distinct moral developments in women and men, leading to differences in value orientation (Glass and Cook, 2018). In a similar situation, women and men respond differently (Betz et al., 1989). This theory also maintains that men prefer success and competition to following rules, whereas women prefer to perform tasks properly while maintaining amicable relationships (Roxas and Stoneback, 2004). Female respondents in turn may underreport undesirable answers (Collins, 2000; Randall and Fernandes, 1991; Schoderbek and Deshpande, 1996).

Ethics studies generally suggest that women are more ethical than men (Collins, 2000; Grimshaw, 1991; McCabe, Ingram, & Dato-on, 2006), and men are more likely to break rules (Roxas and Stoneback, 2004). Deciding how to respond when one does not quite know the answer is akin to an ethical dilemma. Differences between female and male respondents could thus be an enduring effect of their differential socialization (Rapoport, 1982). Yet, most surveys measure only ethical intentions, which are highly susceptible to social desirability bias (Dalton and Ortegren, 2011); studies testing or measuring actual ethical behavior are extremely rare (Bossuyt and Van Kenhove, 2018).

In the context of social and political science, an additional factor comes into play. Women and men differ with respect to the issues that interest them (Campbell and Winters, 2008; Coffé, 2013). In political science research, numerous studies find that women exhibit lower levels of political knowledge than men (Bennett and Bennett, 1989; Dolan, 2011). They are sometimes attributed to gender differences in social roles and responsibilities, and access to resources associated with political engagement (Verba et al., 1997). While these differences are not always large, they are consistent and persistent (Dolan, 2011). And depending on the content domain of a survey, appropriate levels of political knowledge can be important for effective participation. Taken together:

H1: Female (male) respondents evade questions more (less).

Other demographic characteristics also might anticipate differences in survey response behavior (Scholz and Zuell, 2012). For example, willingness to participate in surveys changes with age (Gummer, 2019; Gergen and Back, 1965). For certain domains, especially politics, younger respondents tend to be less experienced than more mature respondents, because they have spent less active time participating in these domains. Mature respondents tend to be more deliberate and careful in their answers, and thus more likely to admit a lack of knowledge, so they may refuse to answer questions more so than younger respondents. This effect continues with old-age, albeit for different reasons. For example, a health survey reports lower response rates among respondents older than 65 years than among younger respondents (Brazier, et al., 1992; Jenkinson et al., 1993). Similarly, in epidemiologic questionnaires, Eaker et al. (1998) identify old age as the most important predictor of partial nonresponse, such that the mean percentage of missing answers is 5% higher in the 70–79 years age group than in the 30–49 years cohort. Consumer settings concur, such that Ferber (1966) cites a below-average item nonresponse rate among older people. Accordingly,

H2: Older (younger) respondents evade questions more (less).

Comparatively low knowledge and experience also may encourage a self-perception of personal views as unusual or socially unacceptable. Especially when questions ask about public opinion, “lower-educated groups in the society are at some disadvantage” (Converse, 1976: 515). If less experienced people feel intimidated or threatened by survey questions (Krosnick and Presser, 2018), they likely exhibit nonresponse behavior to certain questions (Catania et al., 1986; Leigh and Martin, 1987). Beyond a lack of access to specific information, respondents with lower income levels may sense a status at a peripheral level of society, excluded and alienated, which they express by refusing to be interviewed or to answer all the questions (Francis and Busch, 1975). In contrast, if respondents have relevant considerations available in memory to answer a question, they need to construct an overall answer that integrates those considerations. This cognitively demanding process may exceed some respondents’ motivation or ability (Krosnick et al., 2002), such that they seek ways to avoid this work. Such avoidance may be more common among people with lower income and education levels. While education is one of the main determinants of income distribution across countries (De Gregorio and Lee, 2002), income as the consumption component of education (del Mar Salinas-Jiménez et al., 2011) is often found to be a stronger predictor than education in research models (e.g. Sabanayagam and Shankar, 2012). In turn:

H3: Respondents with higher (lower) income evade questions less (more).

Careless responses, signaling insufficient effort (Curran, 2016), also can lead to psychometric problems and affect correlational outcomes, especially between regular and reverse-scored items (Kam and Chan, 2018; Kam and Meyer, 2015). Yet the need to check for careless responses often is overlooked by researchers (Kam and Chan, 2018). The lack of screening in turn prevent insights into its prevalence or impact (Maniaci and Rogge, 2014). Furthermore, conscientiousness as a personal trait has been linked to valid, non-random responding (Furnham et al., 1998; Furnham et al., 2015; Hülsheger and Maier, 2010). Existing studies mainly address insufficient effort responding (or satisficing) and study attrition (Ward et al., 2017), without considering the potential influence of the evasive answer bias. Therefore, the next hypothesis suggests a route to investigate the effect of conscientiousness on people’s tendency to evade survey questions:

H4: Respondents who are generally thorough (careless) evade questions less (more).

As organizations solicit more customer information to gather data, enhance their offerings, and better target their marketing efforts, customers are becoming increasingly protective of the data and information they are willing to disclose (Zimmer et al., 2010). Privacy concerns inhibit information sharing (Ogan et al., 2017); trust can overcome those risk perceptions and encourage increasing information disclosure (Cazier et al., 2007; Dinev and Hart, 2006; Levin and Cross, 2004).

Trust refers to a person’s willingness to be vulnerable to someone else (Cazier et al., 2007), including sharing personal information or opinions, which places the person doing the sharing in a potentially vulnerable situation. Because trust is the “variable most universally accepted as a basis for any human interaction or exchange” (Gundlach and Murphy, 1993: 41), it can support information disclosure (Steiner and Maas, 2018) and reduce the likelihood of opportunism (Bradach and Eccles, 1989), thereby increasing the probability that a person will enter a vulnerable situation through voluntary information sharing. In a survey setting, the interviewer has a prominent role, with considerable effects on survey responses (Loosveldt and Beullens, 2014), according to the trust that respondents develop in this interviewer. In this sense, people who generally trust others may exhibit less evasive answer behavior:

H5: Respondents who generally trust (do not trust) other people evade questions less (more).

Consumer trust also might inhere to the organization collecting the data. In online settings, the organization even may become the primary recipient of consumer trust (Chow and Holden, 1997), because of the lack of a human representative. Regardless of how the questionnaire is administered, in the absence of sufficient time, cognitive capacity, or willingness to evaluate risk consciously, trust in the organization becomes a heuristic that decreases perceived risks (Vischers and Siegrist, 2008) and increases respondents’ motivation to disclose information (Kehr et al., 2015). Therefore:

H6: Respondents who generally trust (do not trust) major companies evade questions less (more).

Group-level effects

Jennings and Farah (1980) reported strong cross-national differences in evasive answer bias, which they could explain only partly with individual-level variables. Almost two decades later, De Leeuw and De Heer (2002) identified relatively large differences in refusal rates to household surveys across countries. Another 20 years later, Purdam et al. (2020) noticed considerable variation in the “don’t know” response across European countries. Yet, little research has sought to establish how individual-level characteristics manifest across cultures (Roxas and Stoneback, 2004) or how cultural values as group-level effects influence the evasive answer bias (Schumacher and Maas, 2019; Zhou, et al., 2017).

Leung et al. (2005) define national culture as the values, beliefs, norms, and behavioral patterns of a national group. A country’s culture shapes its people’s perceptions, dispositions, and behaviors (Triandis, 1989). Acquiring culture is a “slow process of growing into society” (Hofstede, 1991: 4), also referred to as socialization (Furth, 1990; Hofstede, 2001), by which people “learn various patterns of interaction that are based on the norms, rules, and values of their culture” (Gudykunst, et al., 1996: 510). A “complex configuration of values” (Woodside et al., 2011: 785) refers to standards that guide attitudes, actions, and the presentation of the self to others (De Mooij, 2017).

In turn, perceptions of ethical dilemmas, choices of alternative actions, and evaluations of their consequences influence decision making (Hunt and Vitell, 1986). Bartels (1967) defines ethics as “a basis for judgment in personal interaction,” pertaining “to the fulfilment or violation of expectations” (p. 21). Cultural values are critical in shaping ethical perceptions (Ho, 2010; Martin et al., 2009; Roxas and Stoneback, 2004), and a substantial body of research compares the ethical sensitivities of people from different countries (Collins, 2000), such that the current “call for globalization of ethics indicates there are nontrivial differences” (Roxas and Stoneback, 2004: 151). If a person perceives a set of alternatives, it evokes both deontological (right or wrong) and teleological (related to the goal) evaluations. In the deontological evaluation, the decision maker evaluates the rightness or wrongness of the behavior implied by each alternative, comparing it with a set of predetermined behavioral norms and values that reflect beliefs about honesty, cheating, and treating others fairly, as well as confidentiality, anonymity, and deceptiveness. In a teleological evaluation, the decision maker examines the consequences of the action with respect to its consequences for stakeholders, their probability, the desirability of the consequences, and the importance of each stakeholder (Hunt and Vitell, 1986). When answering survey questions, the most salient stakeholders are the interviewer and the organization conducting the survey. Because interviews involve social interactions with another person, respondents likely take social norms into account when responding (Bowling, 2005). Zey-Ferrell and Ferrell (1982) indicate that the organizational distance and relative authority of the stakeholder influences people’s beliefs and behaviors in ethical decision-making situations.

In-group collectivism is “the degree to which individuals express pride, loyalty, and cohesiveness in their organizations or families” (House and Javidan, 2004: 12). Dealing with the relationship between self and group, it is “perhaps the most important dimension of cultural difference in social behavior across diverse cultures of the world” (Triandis, 1989: 60). Its central element is the distinction between independent and interdependent self-construal (Aaker and Lee, 2001; Hofstede, 2001), that is, the image of self as separate from others (Singelis and Sharkey, 1995). As a cultural value, collectivism emphasizes ties between people. Collectivists tend to be interdependent, recognize their duties and obligations to others (Triandis, 1995), interact in a cooperative mode (Doney et al., 1998), and generally conform (Bond and Smith, 1996). Because they value relational harmony, they likely adjust to fit the environment in which they find themselves (Morling et al., 2002). If they are being interviewed, they adjust accordingly and try to answer every question. Because of their strong emphasis on avoiding conflict, collectivists likely expect to deploy a conflict-reducing technique (Forbes et al., 2011), such as skipping fewer questions, because they know answering the questions is expected of them in an interview situation, and failing to meet these expectations could cause friction. This effect may be especially powerful if the survey interview is conducted in the interviewee’s home, such that the interviewer takes the role of a guest or in-group member of society (Brewer and Kramer, 1985; Robbins and Krueger, 2005; Sumner, 1906). Conversely, if the survey is conducted in an out-group setting (e.g. shopping mall) or anonymously through electronic tools, antagonistic behaviors such as consciously skipping questions may seem more tolerated (Triandis, 1995). Collectivists are then likely to avoid answering socially sensitive topics. In a face-to-face interview, however, the immediate pressure caused by the presence of the interviewer is likely stronger than the desire to avoid a more remote and abstract social conflict.

On the other hand, people from individualistic societies prioritize themselves and their own well-being, so if they do not know the answer to a survey question, or if answering the question requires too much effort, they may just evade the question altogether. These considerations prompt the following hypothesis:

H7: Respondents from countries with predominantly collectivistic (individualistic) values evade questions less (more).

Many previous studies have suggested that ethical differences between women and men are consistent worldwide, yet social values can influence ethical values (Chen et al., 2016; Turiel, 1994). In a certain social situation, value differences between women and men cause them to adhere to their cultural values in different ways (Taras et al., 2010). Gender traits also may be stimulated by cultural values with similar qualities (Chen et al., 2016). Moreover, women’s ethical judgments often are not fixed but vary with the context, unlike the more abstract, rule-based ethical judgments that men tend to exhibit, by abstracting the ethical problem away from an interpersonal situation to find an objective way to choose a course of action (Lee et al., 2000). Beekun et al. (2010) show that women’s choices of actions in response to ethical dilemmas are significantly affected by individualism, whereas men’s ethical decisions are more universal and unrelated to culture. In summary, female ethics appear influenced more by cultural values than do male ethics:

H8: The effect of female gender on evasive answer bias is stronger in countries with predominantly individualistic values.

Power distance is the degree to which people “expect and agree that power should be stratified and concentrated at higher levels” (House and Javidan, 2004: 12). Some societies play down these differences; others allow the differences to surface and increase (Armstrong, 1996). Power distance beliefs appear related to decreased information transparency and disclosure but greater conformity and agreeableness.

First, Turilli and Floridi (2009) define transparency in terms of information visibility, that is, “the choice of which information is to be made accessible” (p. 105), and Vaccaro and Madsen (2006) add the “degree of completeness of information” (p. 146) to this definition. Jain and Jain (2018) instead associate transparency with the process of disclosing information that can enable others to make judgments. Because transparency encourages openness, it increases privacy concerns (Ball, 2009). A preference for such information transparency and disclosure is negatively impacted by high power distance values (Hofstede, 2001; Hofstede et al., 2010; Jain and Jain, 2018), considering that “a hallmark of a high power distance cultures is … the consequent control of information” (Jain and Jain, 2018: 137). People may fear that survey responses, even if officially anonymous, could be monitored by government agencies that draw inferences from their responses. Therefore, in environments with less freedom, nonresponse may be more likely (Jensen et al., 2010). Conversely, low power distance cultures prefer flat hierarchies and endorse open access to information (Jain and Jain, 2018), implying that survey respondents from low power distance cultures likely answer more questions.

Second, Taras et al. (2010) find in a meta-analysis that power distance is positively related to conformity at the individual level and to agreeableness at the country level. People with high power distance beliefs tend to enter into role-constrained interactions with authorities; they are more strongly regimented by the relative position of a superior (Lee et al., 2000). In a dyadic relation, a more powerful other can restrict the available choices and make a person conform to role expectations (Kahn et al., 1964; Zey-Ferrell and Ferrell, 1982). Survey respondents in high power distance cultures may be afraid or unwilling to express disagreement (Hofstede, 2001; Khatri, 2009), and their nonresponse offers a shield from possible reprisals (Jensen et al., 2010); they would rather evade a question than say something that might be contrary to perceived expectations. Therefore,

H9: Respondents from countries with predominantly high (low) power distance values evade questions more (less).

Finally, women try to change rules to preserve relationships (Gilligan, 1982), but changing rules is more difficult in a high power distance culture. With respect to whistleblowing in audit firms, Taylor and Curtis (2013) find that men are relatively less sensitive to variations in power distance than women. The last hypothesis therefore states:

H10: The effect of female gender on evasive answer bias is stronger (weaker) in countries with predominantly high (low) power distance values.

Research context

Data set

This study uses the aggregated data set of the World Values Survey (WVS; Inglehart et al., 2014) with 348,430 respondents from 100 countries, constructed from various surveys and waves between 1981 and 2014. The WVS is an international academic project studying human values, self-descriptions, and attitudes across the globe. The WVS data are used widely by cross-cultural psychologists, anthropologists, and social scientists (Minkov, 2012). These data are collected in face-to-face interviews at respondents’ places of residence by professional organizations. The samples are representative of people residing in private households in each country. Questionnaires are translated into any languages that serve as a first language for more than 15% of the population (WVS, n.d.). However, overall response rates are not consistently reported across countries (Inglehart, 2000).

Operationalization of culture

To classify culture, this study turns to the in-group collectivism and power distance dimensions from the GLOBE framework (House et al., 2004). Introduced in 2004, the GLOBE framework is a useful cross-cultural framework (Smith, 2006). For both dimensions, this study considers societal to-be values, rather than as-is practices, because people are mainly driven by cultural values, whereas organizations reflect the influences of practices (Hofstede et al., 1990; Hofstede and Peterson, 2000). Though not without criticism (Hofstede, 2006), GLOBE shapes contemporary international research on cultural value differences (Beugelsdijk et al., 2017; Stahl and Tung, 2015). Notably, the GLOBE power distance measure refers to control of others, being conceptually different from Hofstede’s version of power distance, which implies acceptance and expectations of exhibitions of power and authority (De Mooij, 2017; Hofstede, 2006).

Research model and variables

Figure 1 shows the research model, including theoretical constructs discussed in the previous section and the key variables, which Table 2 defines further. The first dependent variable, NOANSW refers to the percentage of survey questions not answered by a respondent, and the second, DNTKNW, represents “don’t know” answers:

N O A N S W = \frac{C N T N O A N S W}{A L L Q U E - A B S E N T - C N T D N T K N W} * 100

D N T K N W = \frac{C N T D N T K N W}{A L L Q U E - A B S E N T - C N T N O A N S W} * 100

A B S E N T = C N T M S S N G + C N T N O T A P P + C N T N O T A S K

Table 2.

Study variables.

Variable name	Variable type	Level	Source	Source reference	Description/question	Number type/scale
AGE	Independent	1	WVS	X003	Age	Numeric (integer)
ALLQUE		1	WVS	Computed	Number of available WVS questions	Numeric (integer)
CNTDNTKNW		1	WVS	Computed	Number of questions answered as “don’t know,” coded −1	Numeric (integer)
CNTMISSING		1	WVS	Computed	Number of missing or unknown questions, coded −5	Numeric (integer)
CNTNOANSW		1	WVS	Computed	Number of questions not answered, coded −2	Numeric (integer)
CNTNOTAPP		1	WVS	Computed	Number of questions answered as “not applicable,” coded −3	Numeric (integer)
CNTNOTASK		1	WVS	Computed	Number of questions not asked, coded −4	Numeric
DNTKNW	Dependent	1	WVS	Computed	Percentage of survey questions answered with “don’t know”	Numeric (percentage)
GCO2SV	Independent	2	GLOBE		Collectivism II societal values (In-group collectivism)	1 = low, …, 7 = high
GENDER	Independent	1	WVS	X001	Sex	1 = male, 2 = female
GPOWSV	Independent	2	GLOBE		Power distance societal values	1 = low, …, 7 = high
HOFIND	Independent	2	Hofstede		Individualism dimension	0 = low, … 100 = high
HOFPOW	Independent	2	Hofstede		Power distance dimension	0 = low, … 100 = high
INCOME	Independent	1	WVS	X047	Scale of incomes	1 = lower step, …, 11 = highest step
NOANSW	Dependent	1	WVS	Computed	Percentage of survey questions not answered	Numeric (percentage)
THORGH	Independent	1	WVS	A220	“I see myself as someone who does a thorough job.”	1 = disagree, …, 5 = agree
TRUST1	Independent	1	WVS	A165	“Most people can be trusted.”	1 = yes, 2 = can’t be too careful
TRUST2	Independent	1	WVS	G007_001	Trust: Other people in country	1 = trust completely, …, 5 = not trust at all
TRUST3	Independent	1	WVS	E069_13	Confidence: Major companies	1 = a great deal, …, 4 = none at all

This table records the provenance of the variables, listed in alphabetical order.

In these formulae, CNTNOANSW counts the number of questions not answered by the respondent (coded −2 in WVS). Then ALLQUE = 1367 denotes the total number of possible WVS questions; CNTMISSNG is the number of system missing questions for reasons unknown or not documented by WVW (coded −5); CNTNOTAPP and CNTDNTKNW count the number of questions that the respondent indicates are “not applicable” (coded −3) or “don’t know” (coded −1), respectively; and CNTNOTASK indicates the number of questions not asked in the survey (coded −4). For example, for a 64-year-old female respondent from Poland with unique WVS-ID 6160240223, $N O A N S W = \frac{12 * 100}{1367 - 0 - 3 - 2 - 1222} = 8.571$ , and $D N T K N W = \frac{2 * 100}{1367 - 0 - 3 - 12 - 1222} = 1.538$ . These variables only correlate at 0.111, p < .001, across all respondents, affirming that they offer two different operationalization of evasive answer bias.

Descriptive statistics and observations

Among all WVS respondents, 56.966% skip at least one question (198,487 respondents with NOANSW > 0). The item nonresponse average is 1.319%, with standard deviation of 3.244. Then 70.585% indicate they cannot answer at least one question (246,012 respondents with DNTKNW > 0). For an average of 2.812% of items (standard deviation 5.043), respondents choose the “don’t know” option. Figure 2 reveals the frequency distribution of both types of evasive answer bias across all countries; Table 3 specifies the mean percentages at country level. Overall, the magnitude of evasive answer bias in the WVS appears substantial and worth investigating.

Figure 2.

Frequency of evasive answer bias in WVS (by respondent). These histograms show how many respondents (y-axis Frequency) have evaded what percentage of questions (x-axis Evasive answer bias). The figures are given as a percentage of all WVS respondents, across countries.

Table 3.

Sample sizes, country-level means, and GLOBE cultural values.

Country	Sample	NOANSW	DNTKNW	GCO2SV	GPOWSV
Albania	1999	0.846	4.466	5.739	4.620
Algeria	2482	4.006	3.459
Andorra	1003	0.584	0.915
Azerbaijan	3004	1.235	3.143
Argentina	6398	2.104	2.667	5.507	5.641
Australia	6174	1.909	0.641	4.174	4.740
Bangladesh	3025	0.209	3.792
Armenia	3100	0.983	2.994
Bosnia Herzegovina	1200	0.267	2.287
Brazil	5911	1.310	1.422	5.176	5.326
Bulgaria	2073	1.617	6.044
SrpSka Republic	400	1.349	2.600
Belarus	4642	1.249	3.337
Canada	4095	0.433	2.295	4.265	4.822
Chile	5700	1.252	2.337
China	7791	3.033	5.605	5.804	5.038
Taiwan	3245	0.316	2.127	5.594	5.182
Colombia	10,562	0.414	1.435	5.727	5.564
Croatia	1196	0.976	2.584
Cyprus	2050	0.973	0.969
Czech Republic	2071	2.345	0.158	3.179	3.589
Dominican Rep	417	0.316	4.174
Ecuador	1202	0.092	0.306	5.807	5.600
El Salvador	1254	0.034	2.891	5.346	5.677
Ethiopia	1500	2.877	1.666
Estonia	2554	0.660	3.203
Finland	3004	1.119	1.484	4.070	4.889
France	1001	1.548	0.423	4.367	5.277
Georgia	4710	0.765	3.113	6.189	5.222
Palestine	1000	0.578	3.187
Germany	6136	0.996	2.074	4.022	5.252
Ghana	3086	0.581	0.813
Guatemala	1000	0.259	1.054	5.630	5.602
Haiti	1996	0.026	0.579
Hong Kong	2252	1.197	0.729	5.321	4.959
Hungary	3121	2.625	1.271	5.246	5.558
India	12,621	0.818	7.351	5.917	5.466
Indonesia	3015	1.405	2.892	5.683	5.179
Iran	5199	2.709	3.212	6.025	5.433
Iraq	6226	1.628	3.745
Israel	1199	0.609	1.300	4.704	4.726
Italy	1012	2.236	2.128	4.940	5.427
Japan	8170	3.290	7.142	4.632	5.109
Kazakhstan	1500	1.578	0.180	5.258	5.312
Jordan	3623	1.656	1.523
South Korea	7070	1.397	0.879	5.536	5.607
Kuwait	1303	4.123	2.795	5.802	5.124
Kyrgyzstan	2543	0.293	1.356
Lebanon	1200	0.347	2.787
Latvia	1200	1.211	2.945
Libya	2131	1.093	4.069
Lithuania	1009	2.296	5.005
Malaysia	2501	0.105	0.678	5.509	5.174
Mali	1534	5.091	2.836
Mexico	9973	1.186	2.230	5.710	5.224
Moldova	3038	0.483	3.555
Montenegro	1296	0.281	5.671
Morocco	3651	4.093	5.086	5.865	5.802
Netherlands	2952	1.797	3.508	3.701	4.108
New Zealand	2996	3.808	4.023	3.670	4.887
Nigeria	6778	1.131	0.925	5.552	5.798
Norway	2152	0.289	0.906
Pakistan	3933	0.209	3.532
Peru	5422	1.587	2.145
Philippines	3600	0.085	0.559	6.362	5.442
Poland	4057	1.439	3.542	5.521	5.102
Puerto Rico	1884	0.331	1.776
Qatar	1060	0.306	0.310	4.713	4.733
Romania	4518	1.410	4.414
Russia	8534	1.510	4.659	5.627	5.522
Rwanda	3034	0.279	2.573
Saudi Arabia	1502	0.084	3.412
Serbia	3700	1.390	4.075
Singapore	3484	0.049	0.935	5.635	4.986
Slovakia	1561	0.886	1.978
Vietnam	2495	0.318	3.046
Slovenia	3113	1.628	2.755	5.427	5.333
South Africa	16,786	1.493	2.896	5.086	4.107
Zimbabwe	2501	0.149	2.083	5.571	5.667
Spain	6319	0.694	3.692	5.454	5.519
Sweden	4172	1.607	1.263	3.659	4.851
Switzerland	3853	0.936	2.787	3.970	4.897
Thailand	2734	1.360	0.660	5.702	5.627
Trinidad and Tobago	2001	0.504	2.009
Tunisia	1205	0.445	5.716
Turkey	9290	0.920	1.957	5.881	5.572
Uganda	1002	0.275	3.087
Ukraine	5311	2.246	5.287
Macedonia	2050	0.935	3.646
Egypt	7574	0.423	1.440	5.641	4.918
UK	2134	0.677	1.876	4.082	5.153
Tanzania	1171	1.798	1.605
USA	8548	1.605	0.796	4.254	4.882
Burkina Faso	1534	3.058	4.492
Uruguay	3000	1.509	2.445
Uzbekistan	1500	3.145	0.267
Venezuela	2400	0.066	2.449	5.529	5.400
Yemen	1000	0.150	7.420
Zambia	1500	1.872	2.504	5.841	5.314
Bosnia	800	1.038	1.676

This table lists the number of respondents per country, the average percentage of questions skipped (NOANSW) or marked with “don’t know” (DNTKNW). It further gives the GLOBE values for collectivism (GCO2SV) and power distance (GPOWSV).

Ferber (1966) shows that the pattern of item nonresponses to different questions in surveys is very similar for various types of questions. Even though, the interpretation of the “don’t know” answer (and to some extent also the skipping of questions) is related to the kind of question asked. It can both be an evasive reaction or a legitimate response, that is, a declaration of a lack of knowledge (Presser, 1981). Figure 3 examines the frequency of both biases for a subset of 130 questions of the WVS that have been answered by more than 200,000 respondents. On average and across countries, a question has been skipped by 1.114% respondents (standard deviation 0.014), and 2.759% respondents (standard deviation 0.0321) have answered a question with don’t know. In Figure 3 and for both biases respectively, only 5.384% and 6.153% of questions are in the long right tail (defined as exceeding the average plus two standard deviations). These are sensitive questions (Krumpal, 2013) about the willingness to fight for one’s country (WVS item code E012), political positioning (E033, E115, E179WVS), and confidence in unions (E069_05), movements (E069_14, E069_15), and institutions (E069_20). The research model and the individual-level hypotheses H2 and H3 include demographic factors and individual information to help control for these effects.

Figure 3.

Frequency of evasive answer bias in WVS (by item). These histograms show – in percentage – how often items (y-axis Frequency) have been evaded by how many respondents (x-axis Evasive answer bias). The figures are given as a percentage of 130 WVS items answered by more than 200,000 respondents.

Figure 4 examines the existence of a potential questionnaire fatigue effect over time for all respondents across all countries. While the cumulative percentage of NOANSW nearly follows the trendline, DNTKNW shows a slight time-bound effect in that initially less questions are answered with “don’t know.” Taken together and following Ferber (1966), a view across all questions is justified to examine a respondent’s evasive bias.

Figure 4.

Fatigue effect in answering questions. This plot shows the influence of fatigue on evasive answer bias, separately for questions answered with “don’t know” (DNTKNW, black line) and questions not answered (NOANSW, grey line). The dashed lines are the trendlines through the origin.

Analysis and results

To simultaneously test individual-, cultural group-level effects, and cross-level interactions on the number of unanswered questions (NOANSW) and questions marked with don’t know (DNTKNW), two-level linear models using full maximum likelihood were separately calculated in HLM 7.03 software. All predictors were group mean centered at level 1 and grand mean centered at level 2; for the spotlight analysis (Spiller et al., 2013), all predictors were uncentered to ease interpretation (Baguley, 2009; Enders, 2013). The HLM notation by Raudenbush and Bryk (2002) is used; all variables are abbreviated according to Table 2 to avoid ambiguity.

The hypotheses are tested with a hierarchical series of research models detailed in Table 4, which, one after the other, add and remove independent variables. This procedure was set up for two reasons: First, the dependent variables NOANSW and DNTKNW are greatly impacted by measurement error, because evasive answer bias is, by its very nature, an abnormal and not very pronounced effect (Cheung and Lau, 2015). A very large sample size is thus needed to make inferences about the population based on the sample. In any regression and even more so in a multilevel regression model, the sample size requirements further increase by the number of independent variables present in a model. With 348,430 respondents from 100 countries, the aggregated WVS is already one of the largest international survey data sets available. In other words, there is no realistic way of examining such a small effect with many control variables and sufficient power. Second, not all level-1 variables are available in all countries and waves, nor are all the WVS countries covered by the GLOBE study. Entering all independent variables together into one model would reduce the sample size and the country coverage, making the analysis of an abnormal and small effect almost impossible. While automated stepwise procedures are often criticized (Ruengvirayudh and Brooks, 2016), a hierarchical analysis with a controlled entry of predictor variables based on theoretical considerations and planned prior to looking at the data is, in practice, considered to be acceptable. It capitalizes less on chance, and thereby assures that results are maximally interpretable (Malek et al., 2007; Kerlinger, 1986: 545). Table 4 provides the documentation of variable entry, and shows how the available level-1 and level-2 units vary in each step.

Table 4.

Models and predictor variables.

Mo-del	Units at level		Variables at level 1							Variables at level 2		Standard errors
Mo-del	1	2	GENDER	AGE	INCOME	THORGH	TRUST1	TRUST2	TRUST3	GCO2SV	GPOW2SV	Standard errors
A	348,530	100										—
B	233,525	51										Robust
C	228,870	51	x									Robust
D	21,222	13	x			x						Asymptotic
E	226,038	51	x	x								Robust
F	202,953	51	x	x	x							Robust
G	217,672	51	x				x					Robust
H	17,641	12	x					x				Asymptotic
J	204,404	50	x						x			Robust
K	228,870	51	x							x		Robust
K_R1	107,322	25	x							x		Robust
K_R2	244,877	56	x							x		Robust
L	202,953	51	x	x	x					x		Robust
M	217,672	51	x				x			x		Robust
N	17,641	12	x					x		x		Asymptotic
O	204,404	50	x						x	x		Robust
P	228,870	51	x							x	x	Robust
Q	228,870	51	x							x	x	Robust
Q_R2	244,877	56	x							x	x	Robust
R	228,870	51	x								x	Robust

Not all WVS countries are covered by the GLOBE study, and not all level-1 variables are available in all WVS countries and waves. This table provides a breakdown of the available level-1 and level-2 units in each research model used in this study, together with the included explanatory variables. Variable abbreviations are available in Table 2.

Dependent variable NOANSW

Unconditional model

The examination of the homogeneity of the means of NOANSW across all 100 countries and 348,530 respondents (Model A) produces the sample sizes and means listed in Table 3. Levene’s test indicates unequal variances, F(99; 348,430) = 526.263, p < .001, and the Kruskall-Wallis test reveals statistically significant differences in NOANSW among countries, H(99) = 65,834.566, p < .001. Removing countries not covered by the GLOBE study results in an unconditional model (Model B) with 51 countries and 233,525 respondents:

Level 1: NOANSW_ij = β_0j + r_ij.

Level 2: β_0j = γ₀₀ + u_0j.

The average effect is γ₀₀ = 1.345, SE = 0.145, p < .001. There is a relatively small variation between countries with 8.6% of the variation in item completion occurring between countries and about 91.4% within countries (intraclass correlation coefficient, ICC = 0.086). Yet, this variation is statistically significant (u₀ = 1.100, p < .001), and so, it is reasonable to proceed with the multilevel linear model.

Individual-level effects

In the first random intercepts random slopes model, GENDER serves as the level-1 predictor (Model C, Table 5):

Table 5.

HLM context, Model C

NOANSW: Fixed effects	Coefficient	SE	t-ratio	P	Reliability	f ²
Mean country NOANSW (β₀) intercept (γ₀₀)	1.292	0.141	9.166	<.001	0.996	0.154
Mean country GENDER (β₁) intercept (γ₁₀)	0.193	0.040	4.750	<.001	0.829	0.062

Random effects (variance components)	Variance	df	χ²	P
Variance between country intercepts (τ) (u₀)	1.030	50	18,334.776	<.001
Variance between country GENDER (τ) (u₁)	0.072	50	396.353	<.001

DNTKNW: Fixed effects	Coefficient	SE	t-ratio	P	Reliability	f ²
Mean country DNTKNW (β₀) intercept (γ₀₀)	2.292	0.232	9.841	<.001	0.997	0.003
Mean country GENDER (β₁) intercept (γ₁₀)	0.677	0.103	6.555	<.001	0.941	0.101

Random effects (variance components)	Variance	df	χ²	p
Variance between country intercepts (τ) (u₀)	2.813	50	40,067.509	<.001
Variance between country GENDER (τ) (u₁)	0.072	50	2265.091	<.001

This is the HLM context for Model C (see Table 5), in which GENDER serves as a level-1 predictor. The effect size measure f² relates to variance explained for the overall model, and is computed as $f^{2} = \frac{R^{2}}{1 - R^{2}}; R^{2} = 1 - \frac{σ_{C}^{2} + τ_{C}^{2}}{σ_{B}^{2} + τ_{B}^{2}}$ , wherein the indices refer to the empty (unconditional) model B and the full model C (following Snijders and Bosker, 2012; Lorah, 2018). An effect size of about 0.02 is small, 0.15 medium, and 0.35 large (following Cohen 1992). Variable abbreviations are available in Table 2.

Level 1: NOANSW_ij = β_0j + β_1j*(GENDER_ij) + r_ij.

Level 2: β_0j = γ₀₀ + u_0j; β_1j = γ₁₀ + u_1j.

On average and across countries, GENDER is positively and statistically significantly related to NOANSW, with an average effect of γ₁₀ = 0.193, SE = 0.040, p < .001, in support of H1. Female respondents leave more questions unanswered than male respondents, and the variation between countries is statistically significant, u₀ = 1.030, p < .001.

The variables AGE, entered as a level-1 predictor (Model E), reveals a positive and statistically significant link to NOANSW, with an effect of 0.010, SE = 0.003, and p = .003. The effect of GENDER is 0.196, SE = 0.041, p < .001. This outcome aligns with H2, such that older respondents leave slightly more questions unanswered than younger respondents. The variation between countries is u₀ = 0.995, p < .001.

When INCOME is entered as an additional level-1 predictor (Model F), GENDER and AGE continue to be positively and statistically significantly related to NOANSW (GENDER 0.148, SE = 0.039, p < .001; AGE 0.007, SE = 0.002, p = .011). Yet INCOME is negatively and statistically significantly related to NOANSW (−0.049, SE = 0.013, p < .001). In support of H3, respondents who earn higher income levels leave fewer questions unanswered.

When GENDER is supplemented with THORGH as an additional level-1 predictor (Model D), the effect of GENDER falls to 0.138, SE = 0.059, p = .038, and the effect of THORGH is −0.035, SE = 0.015, p = .045. This result confirms H4, because people who are generally thorough leave fewer questions unanswered.

Regarding the effect of trust on questionnaire completion, in Model G, the binary variable TRUST1 is not statistically significantly related to NOANSW, with an effect of 0.014, SE = 0.059, and p = .803. Moreover, the variable TRUST2 (Model H) reveals an effect that is not significant (0.012, SE = 0.021, p = .570). Similarly, the variable TRUST3 (Model J) exhibits no significant effect (0.024, SE = 0.028, p = .391). These findings conflict with H5 and H6 and indicate instead that the level of trust respondents have in others and in organizations does not significantly influence the percentage of questions they answer.

Group-level effects

The cross-level interactions of the level-2 predictors were first tested separately, then as part of the same model (Aguinis et al., 2013). With GCO2SV as a level-2 predictor for the intercepts and slopes (Model K, Table 6),

Table 6.

HLM context, Model K

NOANSW: Fixed effects	Coefficient	SE	t-ratio	p	Reliability	f ²
Mean country NOANSW (β₀) intercept (γ₀₀)	1.292	0.136	9.470	<.001	0.996	0.066
GCO2SV (γ₀₁)	−0.608	0.287	−2.117	.039		0.142
Mean country GENDER (β₁) intercept (γ₁₀)	0.192	0.037	5.163	<.001	0.806	0.155
GCO2SV (γ₁₁)	−0.280	0.080	−3.484	.001		0.142

Random effects (variance components)	Variance	df	χ²	p
Variance between country intercepts (τ) (u₀)	0.984	49	17,648.045	<.001
Variance between country GENDER (τ) (u₁)	0.061	49	334.281	<.001

DNTKNW: Fixed effects	Coefficient	SE	t-ratio	p	Reliability	f ²
Mean country DNTKNW (β₀) intercept (γ₀₀)	2.291	0.232	9.878	<.001	0.997	0.002
GCO2SV (γ₀₁)	−0.342	0.657	−0.522	.604		0.076
Mean country GENDER (β₁) intercept (γ₁₀)	0.677	0.102	6.616	<.001	0.941	0.101
GCO2SV (γ₁₁)	−0.198	0.266	−0.745	.460		0.076

Random effects (variance components)	Variance	df	χ²	p
Variance between country intercepts (τ) (u₀)	2.849	49	38,811.361	<.001
Variance between country GENDER (τ) (u₁)	0.527	49	2183.922	<.001

This is the HLM context for Model K (see Table 5), in which collectivism (GCO2SV) serves as a level-2 predictor. For effect size (f²) interpretation refer the legend to Table 5; variable abbreviations are available in Table 2.

Level 1: NOANSW_ij = β_0j + β_1j*(GENDER_ij) + r_ij, and

Level 2: β_0j = γ₀₀ + γ₀₁*(GCO2SV_j) + u_0j; β_1j = γ₁₀ + γ₁₁*(GCO2SV_j) + u_1j.

Thus, NOANSW depends on the level of GCO2SV (γ₀₁ = −0.608, SE = 0.287, p = .039), see Figure 5. In support of H7, people from collectivistic countries leave fewer questions blank than people from individualistic countries. Between male and female respondents, the average effect for NOANSW is represented as an increase to γ₁₀ = 0.192, SE = 0.037, p < .001. The cross-level interaction of GCO2SV on GENDER/NOANSW is γ₁₁ = −0.280, SE = 0.080, p = .001. As Aguinis et al. (2013) recommend, Figure 6 presents a depiction of the cross-level interaction at low (GCO2SV = 5.15), medium (5.68), and high (6.25) collectivism values with a spotlight analysis (all variables uncentered). According to Gelfand et al. (2004), these values reflect the midmost values of three societal in-group collectivism value bands in the GLOBE study. The gender effect is weaker for respondents from high collectivism countries, in support of H8.

Figure 5.

Group-level effects on unanswered questions. These scatterplots illustrate the cross-level interactions of the level-2 predictors collectivism (GCO2SV) and power distance (GPOWSV) for unanswered questions (NOANSW). Every dot represents a country. The trendlines are calculated without outliers (t-test < 0.01; Kuwait and Morocco; shown as hollow dots): NOANSW = −0.370 GCO2SV + 3.134, R² = 0.102 and NOANSW = −0.423 GPOWSV + 3.413, R² = 0.043.

When adding AGE and INCOME into the model (Model L), the cross-level interaction of GCO2SV on AGE/NOANSW and INCOME/NOANSW becomes clearly non-significant. None of the cross-level interactions of any trust-related variables (TRUST1, TRUST2, and TRUST3) on either the intercept or slope of the GENDER/NOANSW relationship is statistically significant either (Models M, N, and O).

Next, GPOWSV can be added as an alternative level-2 predictor (Model P):

Level 1: NOANSW_ij = β_0j + β_1j*(GENDER_ij) + r_ij.

Level 2: β_0j = γ₀₀ + γ₀₁*(GPOWSV_j) + γ₀₂*(GCO2SV_j) + u_0j; β_1j = γ₁₀ + γ₁₁*(GPOWSV_j) + γ₁₂*(GCO2SV_j) + u_1j.

The effect of both GPOWSV and GCO2SV on NOANSW loses statistical significance (GPOWSV γ₀₁ = 0.498, SE = 0.425, p = .247; GCO2SV γ₀₂ = −0.359, SE = 0.399, p = .373), though it persists in the hypothesized direction. Across the 51 countries, GPOWSV and GCO2SV are correlated at −0.496, p < .001, which suggests it may be preferable to remove GPOWSV as a level-2 predictor for the intercept (Model Q, Table 7), such that:

Table 7.

HLM context, Model Q.

NOANSW: Fixed effects	Coefficient	SE	t-ratio	p	Reliability	f ²
Mean country NOANSW (β₀) intercept (γ₀₀)	1.292	0.136	9.469	<.001	0.996	0.066
GCO2SV (γ₀₁)	−0.608	0.287	−2.117	.039		0.143
Mean country GENDER (β₁) intercept (γ₁₀)	0.194	0.035	5.442	<.001	0.792	0.156
GPOWSV (γ₁₁)	0.145	0.059	2.441	.018		0.143
GCO2SV (γ₁₂)	−0.210	0.070	−2.980	.005		0.143

Random effects (variance components)	Variance	df	χ²	p
Variance between country intercepts (τ) (u₀)	0.984	49	17,648.047	<.001
Variance between country GENDER (τ) (u₁)	0.055	48	313.740	<.001

This is the HLM context for Model Q (see Table 5). Collectivism (GCO2SV) serves as a level-2 predictor for both intercept and slopes; power distance (GPOWSV) serves only as a level-2 predictor for the slopes. For effect size (f²) interpretation refer the legend to Table 5; variable abbreviations are available in Table 2.

Level 1: NOANSW_ij = β_0j + β_1j*(GENDER_ij) + r_ij.

Level 2: β_0j = γ₀₀ + γ₀₁*(GCO2SV_j) + u_0j; β_1j = γ₁₀ + γ₁₁*(GPOWSV_j) + γ₁₂*(GCO2SV_j) + u_1j.

In this case, GCO2SV has a statistically significant negative effect on NOANSW (γ₀₁ = −0.608, SE = 0.287, p = .039). Between male and female respondents, the average effect of NOANSW is represented as an increase to γ₁₀ = 0.194, SE = 0.036, p < .001. The cross-level interaction of GPOWSV on GENDER/NOANSW is positive, γ₁₁ = 0.145, SE = 0.073, p = .055, and that of GCO2SV is negative, γ₁₂ = −0.210, SE = 0.096, p = .033. Figure 7 depicts the cross-level interaction through a spotlight analysis at low (GPOWSV = 5.15), medium (5.68), and high (6.25) power distance values (always medium GCO2SV = 5.68; predictors entered uncentered). In line with Carl et al. (2004) bands, these GPOWSV values reflect the extreme ends and midpoint of the societal power distance. The results confirm H9; an increase in power distance exerts a negative effect on questionnaire completion. This effect is stronger for female than for male respondents, so H8 also receives support.

Figure 6.

Cross-level interactions, Model K. For Model K (see Table 5) and unanswered questions (NOANSW), this diagram depicts the cross-level interaction through a spotlight analysis at low (GCO2SV = 5.15), medium (5.68), and high (6.25) collectivism values. All variables are uncentered.

Figure 7.

Cross-level interactions, Model Q. For Model Q (see Table 5) and unanswered questions, this diagram depicts the cross-level interaction through a spotlight analysis at low (GPOWSV = 5.15), medium (5.68), and high (6.25) power distance values. Collectivism is at medium level (GCO2SV = 5.68). All variables are uncentered.

Dependent variable DNTKNW

Unconditional model

Table 3 details the country-level means for DNTKNW. Similar to the results for NOANSW, Levene’s test indicates unequal variances for DNTKNW, F(99; 348,430) = 692.700, p < .001, and the Kruskall-Wallis test confirms statistically significant differences between the countries, H(99) = 68,678.910, p < .001. The ICC in Model B is 0.109, and variation between countries is statistically significant, u₀ = 2.822, p < .001. The average effect is γ₀₀ = 2.263, SE = 0.233, p < .001.

Individual-level effects

Model C (Table 5) confirms H1; GENDER is positively and statistically significantly related to DNTKNW, with an average effect of γ₁₀ = 0.677, SE = 0.104, p < .001. Variation between countries is statistically significant, u₀ = 2.813, p < .001. The results reveal that AGE is positively and statistically significantly related to DNTKNW in Model E, with an effect of 0.023, SE = 0.004, p < .001. Because the effect of GENDER is 0.674, SE = 0.104, p < .001, the result confirms H2. The variation between countries in this analysis is u₀ = 2.779, p < .001. In addition, INCOME is negatively and statistically significantly related to DNTKNW in Model F, at −0.192, SE = 0.027, p < .001. Both GENDER and AGE continue to be positively and statistically significantly related to NOANSW (GENDER 0.559, SE = 0.087, p < .001; AGE 0.017, SE = 0.003, p < .001), in support of H3.

In Model D, the effect of GENDER is 0.474, with SE = 0.130, and p = .003, but the effect of THORGH is not significant, −0.047, with SE = 0.027, and p = .115. In this case, H4 must be rejected. In Models G and H, neither TRUST1 nor TRUST2 is statistically significantly related to DNTKNW, with effects of 0.092, SE = 0.094, p = .333 and −0.013, SE = 0.069, p = .851, respectively. But TRUST3 (Model J) reveals a significant effect, 0.111, SE = 0.030, p < .001, indicating no support for H5 but confirmation of H6. Respondents who generally trust organizations choose the “don’t know” answer option less frequently. However, trust in other people plays no significant role.

Group-level effects

In the cross-level model (Model K, Table 6), DNTKNW does not depend on the level of GCO2SV as a level-2 predictor with statistical significance (γ₀₁ = −0.342, SE = 0.657, p = .604). The average effect for DNTKNW between male and female respondents again reveals an increase to γ₁₀ = 0.677, SE = 0.102, p < .001, but the cross-level interaction of GCO2SV on GENDER/NOANSW is not statistically significant, such that γ₁₁ = −0.198, SE = 0.266, p = .460. Similarly, in Model R, DNTKNW does not depend on the level of GPOWSV with any statistical significance (γ₀₁ = 0.044, SE = 0.551, p = .936; γ₁₁ = −0.047, SE = 0.173, p = .787). Thus, H7–H10 are all rejected, and cultural values appear to have no significant influence on how often respondents choose the “don’t know” answer option.

Robustness tests

A series of robustness tests informs the results. First, Model K_R2 is assessed with Hofstede’s individualism dimension (HOFIND) as an alternate proxy of culture, which correlates with GCO2SV at −0.265, p = .068 across the countries of this study. Accordingly, NOANSW continues to depend on HOFIND with statistical significance (γ₀₁ = 0.013, SE = 0.004, p = 0.010), but the cross-level interaction of HOFIND on GENDER/NOANSW has lost significance (γ₁₁ =0.001, SE = 0.001, p = .264). The effects for DNTKNW continue to be nonsignificant. In Model Q_R2, GPOWSV is also replaced with HOFPOW. HOFIND has a statistically significant effect on NOANSW (γ₀₁ =0.013, SE = 0.004, p = .004), but the cross-level interaction of HOFPOW and HOFIND on GENDER/NOANSW are no longer significant (γ₁₁ = 5.510 × 10⁻⁴, SE = 0.001, p = .678; γ₁₂ =0.001, SE = 0.001, p = .264). In summary, only the cultural effects of collectivism can be replicated with Hofstede’s dimensions.

Second, the other cultural dimensions of the GLOBE study—institutional collectivism, assertiveness, future orientation, performance orientation, gender egalitarianism, and humane orientation—potentially could explain further variance. Ashraf et al. (2017) argue for limiting analyses to culture dimensions that are strongly tied to the research focus, yet arguably, “ignored cultural factors … are contributing as much if not more to the observed effects” (Sivakumar and Nakata, 2001: 556). In iterative replacements of GCO2SV with other GLOBE dimensions in Model K, future orientation, performance orientation, and humane orientation exhibit statistical significance, but they lose significance in combination with GCO2SV.

Third, as Frank et al. (2013) suggest, it is useful to quantify the robustness of the effects using the application by Rosenberg et al. (2018). Because the underlying populations of Models A through R are different and the WVS data set is collected by many different interviewers with potentially different interviewing styles, this test is especially important. To invalidate the effect of GENDER on NOANSW in Model K, about 62.230% of the data would have to be replaced with samples for which there is no gender effect; and to invalidate the main (slope) effect of GCO2SV, about 7.481 (44.001) percent of the data would have to be due to bias, which is improbable.

Fourth, as the distributions of NOANSW and DNTKNW are highly skewed (see Figure 2), the models in Table 4 are rerun with a Poisson model. Because of the huge data size, HLM 7.03 software runs out of memory and often stops the computation after six or eight iterations. Yet, the resulting coefficients and their significance are very similar to the models with assumed normal distributions.

Discussion

Summary

Transparency is a ubiquitous notion, ranging from a legal requirement in tax declarations to an implicit expectation in surveys (Jain and Jain, 2018). Without complete and correct survey data, researchers and practitioners cannot draw meaningful conclusions. But when survey respondents evade answers consciously or unconsciously, they violate the requirement of transparency.

Considering the high nonresponse rates to items in surveys such as the WVS, “it is important to learn about the determinants of nonresponse behavior” (Riphahn and Serfling, 2005: 522). This article examines the evasive answer bias in surveys, which might manifest as skipping questions or consciously choosing a “don’t know” answer option. Both measures correlate only very loosely, indicating that they are distinct manifestations of evasive answer bias. They relate to demographic and attitudinal variables at the individual respondent level, as well as to cultural values at the national level (see the overview of hypotheses in Table 1).

First, the individual-level variables exert similar influences on both types of evasive answer bias, such that, female respondents (H1), older respondents (H2), and respondents with higher incomes (H3) evaded questions more than male, younger, and lower income respondents. Respondents, who characterize themselves as generally careless, skip more questions than conscientious respondents (H4). Though, conscientiousness is not significantly related to how often the respondents choose the “don’t know” answer option. Hence, including a “don’t know” answer option in surveys is not a catch-all option for careless responses. It rather provides a veritable measure that can reduce noise when respondents lack a strong opinion (non-attitude reporting; Dobronte, 2014) or do not want to share their opinions.

This research also reveals that skipping answers is not an expression of a lack of trust in either the interviewer or the organization conducting the survey (H5 and H6). Though, the frequency of the “don’t know” option is related to respondents’ general lack of trust in organizations. Trust in other people, which includes the interviewer, does not seem to influence respondents’ evasive answer behavior, but more “don’t know” answers may signal respondents’ lack of trust in the organization conducting the survey.

Second and at the country level, cultural values exert distinct effects across the two dependent measures (H7 to H10). How often respondents choose the “don’t know” answer option does not seem related to any of the cultural dimensions. In contrast, skipping answers happens less often in both collectivistic and low power distance cultures (H7 and H9), likely because people in these cultures tend to be more transparent in terms of their information sharing (Turilli and Floridi, 2009) and more ethically determined by their devotion “to the fulfilment or violation of expectations” (Bartels, 1967: p. 21). Across countries, the effect of cultural values is significantly stronger for female than for male respondents (H8 and H10).

Implications

First, evasive responses must be measured and monitored by researchers who should then “take this information into consideration when interpreting the score” (Charter, 2000: 315), with the clear recognition that “we can learn not only from truthful responses…, but also from the systematic patterns of nonresponses” (Jensen et al., 2010: 1496). These recommendations are particularly critical if important decisions, with substantial implications for stakeholders, rely on survey results in settings in which the survey respondents have not been suitably motivated (Osborne and Blanchard, 2011). Developing an understanding of the mechanisms that drive item nonresponse in surveys can support the further development of techniques to either reduce evasive answer bias or meaningfully analyze it (Riphahn and Serfling, 2005) and thereby substantively increase the managerial value of surveys.

Second, evasive responses cause missing data through a nonrandom process. Yet, researchers typically do not spend much time implementing effective strategies to address this issue. “A tacit understanding that missing data is a trivial nuisance seems to be the rule” (Myers, 2011: 298). The most frequently chosen and default approach in many statistical packages is listwise (aka, casewise) deletion (Tanguma, 2000), that is, simply discarding any respondent with a missing measurement on any item. But listwise deletion disproportionally reduces the effective sample size for certain groups of respondents, such as female, older and more well-off respondents (refer the hypotheses examined in this article, Table 1). Besides, disproportionally more responses from females are lost in countries characterized by individualistic and high power distance values (H8 and H10). Pairwise deletion is an alternative and lesser-used method, which discards cases only when the estimate requires a response to an item. This also results in biased estimates. For example, in regression analysis, different respondents are included in the estimate of each separate coefficient. Mean substitution imputes the mean of a variable in missing places and returns a complete data set with all respondents. Unfortunately, this approach artificially deflates the variation of a variable and potentially changes the value of regression estimates. Given the biases caused, researchers are generally advised to avoid the aforementioned methods and utilize statistically more appropriate approaches (Newman, 2009).

Then, multiple imputation (Rubin, 1996) replaces each missing value with a probability sampled set of values, ultimately creating several and slightly different data sets. The statistical analysis would then be performed separately on each data set and the results combined. Hot deck imputation (Andridge and Little, 2010; Myers, 2011) or k-nearest neighbor (Fix and Hodges, 1989) are somewhat easier to use; both attempt to find similar respondents in the data set and predict the missing data from them. Whereas all missing data treatments are naturally imperfect (Newman, 2014), the latter two approaches allow for retention of the complete data set and perform realistic imputations based on answers observed by other respondents. It is specifically recommended that cross-cultural researchers use imputation rather than deletion approaches on data sets from international surveys.

Third, greater understanding of the relationships of individual-level and group-level variables with answer bias in surveys also increases the chances of designing surveys that provide valid responses. The results of this study reveal, for both researchers and practitioners, key differences and similarities in people’s reactions when it comes to answering survey questions. Cross-cultural researchers specifically need to recognize that “off the shelf” questionnaires are not necessarily appropriate for measuring what they want to measure in a certain target group (Jenkinson et al., 1993). It may be necessary to deploy questionnaires that reflect considerations of the respondents’ demographics and cultural background, rather than generic ones. To identify questions that create potential item nonresponse problems, questionnaire pretesting should be done across cultural fault lines and devote particular attention to female and older participants, as well as those with lower income.

Limitations and further research directions

Although prior literature contains some studies of response bias in surveys, relatively few aim at understanding evasive answer bias. The importance of accurate data in the advanced information age suggests the critical need for further exploration of this issue. The current study taps the large WVS data set to measure two elements of respondents’ evasive answer bias. However, across WVS waves and countries, various questions have been modified or dropped, so the percentages of evasive answer bias had to be calculated and compared. Absolute numbers could not be obtained. For that reason, it is not practical to calculate a respondent’s evasive bias for thematic groups of WVS questions. Because the calculation of the response bias would be based on fewer questions, this would reduce the explanatory power of the analysis. Evasive answer bias is an irregular and deviant behavior, and its detection needs as large and robust a data set as possible. While examining the effect for different question types needs to be assigned to future research, few other databases offer such a massive number of comparable respondents and questions as the WVS though. Future research should also investigate reasons for evading items. A suggestion would be to follow up the WVS survey with a qualitative interview asking why the respondent has evaded questions.

Furthermore, the WVS Association, as a global network of social scientists, is widely perceived as a benevolent organization, so respondents to such social surveys might express less evasive response bias than they would in surveys conducted by for-profit marketing organizations. This prediction should be investigated in further research. The household survey also involves interviews conducted inside respondents’ homes; they complete the paper-and-pencil survey in the presence of the interviewer. Although trust in other people did not significantly affect evasive response bias in this study, it would be interesting to determine if the absence of the interviewer or an outgroup scenario (Forbes et al., 2011), created by surveys conducted outside respondents’ homes, affects evasive response bias. Similarly, differences between paper-and-pencil questionnaires and electronic surveys, longer and shorter surveys could be investigated in continued research.

Footnotes

Acknowledgements

The author is very thankful to an anonymous reviewer for suggesting to describe the study’s implications for data cleaning of international surveys. Moreover, the author gratefully acknowledges generous support and funding from the Darla Moore School of Business and the Center for International Business Education and Research (CIBER) at the University of South Carolina.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Wolfgang Messner

References

Aaker

Lee

(2001) “I” seek pleasures and “we” avoid pains: the role of self-regulatory goals in information processing and persuasion. Journal of Consumer Research 28(1): 33–49.

Aguinis

Gottfredson

Culpepper

(2013) Best-practice recommendations for estimating cross-level interaction effects using multilevel modeling. Journal of Management 39(6): 1490–1528.

Andridge

Little

(2010) A review of hot deck imputation for survey non-response. International Statistical Review 78(1): 40–64.

Armstrong

(1996) The relationship between culture and perception of ethical problems in international marketing. Journal of Business Ethics 15(11): 1199–1208.

Ashraf

Thongpapanl

Menguc

, et al. (2017) The role of m-commerce and readiness in emerging and developed marketes. Journal of International Marketing 25: 25–51.

Baguley

(2009) Standardized or simple effect size: what should be reported? British Journal of Psychology 100(3): 603–617.

Ball

(2009) What is transparency? Public Integrity 11(4): 293–308.

Bartels

(1967) A model for ethics in marketing. Journal of Marketing 31(1): 20–26.

Baumgartner

Steenkamp

J-BE

(2001) Response styles in marketing research: a cross-national investigation. Journal of Marketing Research 38(2): 143–156.

10.

Beekun

Stedham

Westerman

, et al. (2010) Effects of justice and utilitarianism on ethical decision making: a cross-cultural examination of gender similarities and differences. Business Ethics: A European Review 19(4): 309–325.

11.

Bellman

Johnson

Kobrin

(2004) International differences in information privacy concerns: a global survey of consumers. The Information Society 20(5): 313–324.

12.

Bennett

(1989) Enduring gender differences in political interest. The impact of socialization and political disposition. American Politics Quarterly 17(1): 105–122.

13.

Betz

O'Connell

Shepherd

(1989) Gender differences in proclivity for unethical behavior. Journal of Business Ethics 8(5): 321–324.

14.

Beugelsdijk

Kostova

Roth

(2017) An overview of Hofstede-inspired country-level culture research in international business since 2006. Journal of International Business Studies 48(1): 30–47.

15.

Bond

Smith

(1996) Culture and conformity: a meta-analysis of studies using Asch’s (1052b, 1956) line judgement task. Psychological Bulletin 119(1): 111–137.

16.

Bossuyt

Van Kenhove

(2018) Assertiveness bias in gender ethics research: why women deserve the benefit of the doubt. Journal of Business Ethics 150(3): 727–739.

17.

Bowling

(2005) Mode of questionnaire administration can have serious effects on data quality. Journal of Public Health 27(3): 281–291.

18.

Bradach

Eccles

(1989) Price, authority, and trust: from ideal types to plural forms. Annual Review of Sociology 15(1): 97–118.

19.

Brazier

Harper

Jones

, et al. (1992) Validating the SF-36 health survey questionnaire: new outcome measure for primary care. Bmj: British Medical Journal 305(6846): 160–164.

20.

Brewer

Kramer

(1985) The psychology of intergroup attitudes and behavior. Annual Review of Psychology 36(1): 219–243.

21.

Brinberg

Wood

(1983) A resource exchange theory analysis of consumer behavior. Journal of Consumer Research 10(3): 330–338.

22.

Brown

Micklewright

Schnepf

, et al. (2007) International surveys of educational achievement: how robust are the findings? Journal of the Royal Statistical Society. Series A (Statistics in Society) 170(3): 623–646.

23.

Campbell

Winters

(2008) Understanding men’s and women’s political interests: evidence from a study of gendered political attitudes. Journal of Elections, Public Opinions and Parties 18(1): 53–74.

24.

Carl

Gupta

Javidan

(2004) Power distance. In: House

Hanges

Javidan

, et al. (eds), Culture, Leadership, and Organizations. The GLOBE Study of 62 Societies. Thousand Oaks, CA: SAGE, 513–563.

25.

Catania

McDermott

Pollack

(1986) Questionnaire response bias and face-to-face interview sample bias in sexuality research. The Journal of Sex Research 22(1): 52–72.

26.

Cazier

Shao

Louis

StRD

(2007) Sharing information and building trust through value congruence. Information Systems Frontiers 9(5): 515–529.

27.

Charter

(2000) Determining random responding to objective tests. Journal of Psychoeducational Assessment 18(4): 308–315.

28.

Chen

C-w

Tuliao

Cullen

, et al. (2016) Does gender influence managers' ethics? A cross-cultural analysis. Business Ethics: A European Review 25(4): 345–362.

29.

Cheung

Lau

(2015) Accuracy of parameter estimates and confidence intervals in moderated mediation models: a comparison of regression and latent moderated structural equations. Organizational Research Methods 20(4): 746–769.

30.

Chow

Holden

(1997) Toward an understanding of loyalty: the moderating role of trust. Journal of Managerial Issues 9(3): 275–299.

31.

Coffé

(2013) Women stay local, men go national and global? Gender differences in political interest. Sex Roles 69(5–6): 323–338.

32.

Cohen

(1992) A power primer. Quantitative Methods in Psychology 112(1): 155–159.

33.

Collins

(2000) The quest to improve the human condition: the first 1500 articles published in Journal of Business Ethics. Journal of Business Ethics 26(1): 1–73.

34.

Converse

(1976) Predicting no opinion in the polls. Public Opinion Quarterly 40(4): 515–530.

35.

Couper

de Leeuw

(2003) Nonresponse in cross-cultural and cross-national surveys. In: Harkness

van de Vijver

(eds), Cross-cultural Survey Methods. Hoboken, NJ: Wiley-Interscience, 157–177.

36.

Craig

McCann

(1978) Item nonresponse in mail surveys: extent and correlates. Journal of Marketing Research 15(2): 285–289.

37.

Curran

(2016) Methods for the detection of carelessly invalid responses in survey data. Journal of Experimental Social Psychology 66: 4–19.

38.

Dalton

Ortegren

(2011) Gender differences in ethics research: the importance of controlling for the social desirability response bias. Journal of Business Ethics 103(1): 73–93.

39.

De Gregorio

Lee

J-W

(2002) Education and income inequality: new evidence from cross-country data. Review of Income and Wealth 48(3): 395–416.

40.

de Leeuw

de Heer

(2002) Trends in household survey nonresponse: a longitudinal and international comparison. In: Groves

Dillman

Eltinge

, et al. (eds), Survey Nonresponse. New York, NY: Wiley, 41–54.

41.

de Mooij

(2017) Comparing dimensions of national culture for secondary analysis of consumer behavior data of different countries. International Marketing Review 34(3): 444–456.

42.

del Mar Salinas-Jiménez

Artés

Salinas-Jiménez

(2011) Education as a positional good: a life satisfaction approach. Social Indicators Research 103(3): 409–426.

43.

Dinev

Hart

(2006) An extended privacy calculus model for e-commerce transactions. Information Systems Research 17(1): 61–80.

44.

Ding

Yang

, et al. (2021) Women suffered more emotional and life distress than men during the COVID-19 pandemic: the role of pathogen disgust sensitivity. International Journal of Environmental Research and Public Health 18(8539): 1–9.

45.

Dobronte

(2014, Jan) Pitfalls of “don't Know/no Opinion” Answer Options in Surveys. Retrieved Jun 01, 2019, from CheckMarket: https://www.checkmarket.com/blog/dont-know-no-opinion-answer-option/.

46.

Dolan

(2011) Do women and men know different things? Measuring gender differences in political knowledge. The Journal of Politics 73(1): 97–107. DOI: 10.1017/S0022381610000897.

47.

Doney

Cannon

Mullen

(1998) Understanding the influence of national culture on the development of trust. Academy of Management Review 23(3): 601–620.

48.

Durand

Guffey

Planchon

(1983) An examination of the random versus nonrandom nature of item omissions. Journal of Marketing Research 20(3): 305–313.

49.

Eaker

Bergström

, et al. (1998) Response rate to mailed epidemiologic questionnaires: a population-based randomized trial of variations in design and mailing routines. American Journal of Epidemiology 147(1): 74–82.

50.

Enders

(2013) Centering predictors and contextual effects. In: Scott

Simonoff

Marx

(eds), The SAGE Handbook of Multilevel Modeling. London: SAGE, 89–108.

51.

Feick

(1989) Latent class analysis of survey questions that include don't know responses. Public Opinion Quarterly 53(4): 525–547.

52.

Ferber

(1966) Item nonresponse in a consumer survey. Public Opinion Quarterly 30(3): 399–415.

53.

Fix

Hodges

(1989) Discriminatory analysis. Nonparametric discrimination: consistency properties. International Statistical Review/Revue Internationale de Statistique 57(3): 238–247.

54.

Forbes

Collinsworth

Zhao

, et al. (2011) Relationships among individualism-collectivism, gender, and ingroup/outgroup status, and responses to conflict: a study in China and the United States. Aggressive Behavior 37(4): 302–314.

55.

Francis

Busch

(1975) What we now know about “I don’t knows”. Public Opinion Quarterly 39(2): 207–218.

56.

Frank

Maroulis

Duong

, et al. (2013) What would it take to change an inference? Using Rubin’s causal model to interpret the robustness of causal inferences. Education, Evaluation and Policy Analysis 35: 437–460.

57.

Fuchs

Bossert

Stukowski

(2013) Response rate and nonresponse bias - Impact of the number of contact attempts on data quality in the European Social Survey. Bulletin of Sociological Methodology 117(1): 26–45.

58.

Furnham

Forde

Cotter

(1998) Personality scores and test taking style. Personality and Individual Differences 24: 19–23.

59.

Furnham

Hyde

Trickey

(2015) Personality and value correlates of careless and erratic questionnaire responses. Personality and Individual Differences 80: 64–67.

60.

Furth

(1990) The ‘radical imaginary’ underlying social institutions: its developmental base. Human Development 33: 202–213.

61.

Gelfand

Bhawuk

Nishii

, et al. (2004) Individualism and collectivism. In: House

Hanges

Javidan

, et al. (eds), Culture, Leadership, and Organizations. The GLOBE Study of 62 Societies. Thousand Oaks, CA: Sage Publications, 437–512.

62.

Gergen

Back

(1965) Communication in the interview and the disengaged respondent. Public Opinion Quarterly 30(3): 385–398.

63.

Gilligan

(1982) In a Different Voice: Psychological Theory and the Women's Movement. Cambridge, MA: Harvard University Press.

64.

Glass

Cook

(2018) Do women leaders promote positive change? Analyzing the effect of gender on business practices and diversity initiatives. Human Resource Management 57(4): 823–837.

65.

Grimshaw

(1991) The idea of a female ethic. In: Singer

(ed), A Companion to Ethics. Malden, MA: Blackwell Publishers, 491–499.

66.

Groves

Singer

Corning

(2000) Leverage-saliency theory of survey participation. Public Opinion Quarterly 64(3): 299–308.

67.

Gudykunst

Matsumoto

Ting-Toomey

, et al. (1996) The influence of cultural individualism-collectivism, self construals, and individual values on communication styles across cultures. Human Communication Research 22(4): 510–543.

68.

Gummer

(2019) Assessing trends and decomposing change in nonresponse bias: the case of bias in cohort distributions. Sociological Methods & Research 48(1): 92–115.

69.

Gundlach

Murphy

(1993) Ethical and legal foundations of relational marketing exchanges. Journal of Marketing 57(4): 35–46.

70.

Harzing

A-W

(2006) Response styles in cross-national survey research. International Journal of Cross-Cultural Management 6(2): 243–266.

71.

Harzing

A-W

Brown

Koester

, et al. (2012) Response style differences in cross-national research. Dispositional and situational determinants. Management International Review 52(3): 341–363.

72.

(2010) Ethical perception: are differences between ethnic groups situation dependent? Business Ethics: A European Review 19(2): 154–182.

73.

Hofstede

(1991) Cultures and Organizations: Software of the Mind. New York, NY: McGraw-Hill.

74.

Hofstede

(2001) Culture’s Consequences: Comparing Values, Behaviors, Institutions, and Organizations across Nations. Thousand Oaks, CA: Sage.

75.

Hofstede

(2006) What did GLOBE really measure? Researchers’ minds versus respondents’ minds. Journal of International Business Studies 37(6): 882–896.

76.

Hofstede

Minkov

(2010) Cultures and Organizations: Software of the Mind. Third edition. New York, NY: McGraw-Hill.

77.

Hofstede

Peterson

(2000) Culture: national values and organizational practices. In: Ashkanasy

Wilderorn

CPMM

Petersen

(eds), Handbook of Organizational Culture and Climate. Thousand Oaks, CA: Sage, 401–415.

78.

Hofstede

Neuijen

Ohayv

, et al. (1990) Measuring organizational cultures: a qualitative and quantitative study across twenty cases. Administrative Science Quarterly 35(2): 286–316.

79.

House

Javidan

(2004) Overview of GLOBE. In: House

Hanges

Javidan

, et al. (eds), Culture, Leadership, and Organizations: The GLOBE Study of 62 Societies. Thousand Oaks, CA: Sage Publications, 9–28.

80.

House

Hanges

Javidan

, et al. (2004) Culture, Leadership, and Organizations: The GLOBE Study of 62 Societies. Thousand Oaks, CA: Sage Publications.

81.

Hülsheger

Maier

(2010) The careless and the conscientious: who profits most from goal progress? Journal of Vocational Behavior 77(2): 246–254.

82.

Hunt

Vitell

(1986) A general theory of marketing ethics. Journal of Business of Macromarketing 6(1): 5–16.

83.

Inglehart

(2000, Feb) World Values Surveys and European Values. Retrieved Nov 03, 2019, from ICPSR: https://web.stanford.edu/group/ssds/dewidocs/icpsr2790_superseded/cb2790.pdf.

84.

Inglehart

Haerpfer

Moreno

, et al. (2014) World Values Survey: All Rounds - Country-Pooled Datafile Version. Madrid: JD Systems Institute. Retrieved Dec 06, 2018, from http://www.worldvaluessurvey.org/wvsdocumentationwvl.jsp.

85.

Jain

(2018) Power distance belief and preference for transparency. Journal of Business Research 89: 135–142.

86.

Jans

McLaughlin

Viana

, et al. (2018) Geographic correlates of nonresponse in California: a cultural ecosystems perspective. In: Johnson

Pennell

B-E

Stoop

, et al. (eds), Analytics in Comparative Survey Methods. Hoboken, NJ: John Wiley & Sons, 835–857.

87.

Jenkinson

Coulter

Wright

(1993) Short form 36 (SF 36) health survey questionnaire: normative data for adults of working age. BMJ: British Medical Journal 306(6890): 1437–1440.

88.

Jennings

Farah

(1980) Ideology, gender and political action: a cross-national survey. British Journal of Political Science 10(2): 219–240.

89.

Jensen

Rahman

(2010) Understanding corruption and firm responses in cross-national firm-level surveys. Journal of International Business Studies 41(9): 1481–1504.

90.

Johnson

Cho

Lee

(2010) Examining the association between cultural environments and survey nonresponse. Survey Practice 3(3): 1–14.

91.

Kahn

Wolfe

Quinn

, et al. (1964) Organizational Stress: Studies in Role Conflict and Ambiguity. New York, NY: Wiley.

92.

Kam

Chan

GH-h

(2018) Examination of the validity of instructed response items in identifying careless respondents. Personality and Individual Differences 129: 83–87.

93.

Kam

Meyer

(2015) How careless responding and acquiescence response bias can influence construct dimensionality: the case of job satisfaction. Organizational Research Methods 18(3): 512–541.

94.

Kamakura

Wedel

(2000) Factor analysis and missing data. Journal of Marketing Research 37(4): 490–498.

95.

Kehr

Kowatsch

Wentzel

, et al. (2015) Blissfully ignorant: the effects of general privacy concerns, general institutional trust, and affect in the privacy calculus. Information Systems Journal 25(6): 607–635.

96.

Kerlinger

(1986) Foundations of Behavioral Research. New York, NY: Holt, Rinehart and Winston.

97.

Khatri

(2009) Consequences of power distance orientation in organisations. Vision 13(1): 1–9.

98.

Kidder

Parks

(2001) The good soldier: who is (s)he? Journal of Organizational Behavior 22(8): 939–959.

99.

Krosnick

Presser

(2018) Question and questionnaire design. In: Vannette

Krosnick

(eds), The Palgrave Handbook of Survey Research. Cham: Palgrave Macmillan, 439–455.

100.

Krosnick

Holbrook

Berent

, et al. (2002) The impact of “no opinion” response options on data quality. Non-attitude reduction or an invitation to satisfice? Public Opinion Quarterly 66(3): 371–403.

101.

Krumpal

(2013) Determinants of social desirability bias in sensitive surveys: a literature review. Quality & Quantity 47(4): 2025–2047.

102.

Lee

Pillutla

Law

(2000) Power-distance, gender and organizational justice. Journal of Management 26(4): 685–704.

103.

Leigh

Martin

(1987) “Don’t know” item nonresponse in a telephone survey: effects of question form and respondent characteristics. Journal of Marketing Research 24(4): 418–424.

104.

Leung

Bhagat

Buchan

, et al. (2005) Culture and international business: recent advances and their implications for future research. Journal of International Business Studies 36(4): 357–378. DOI: 10.1057/palgrave.jibs.8400150.

105.

Levin

Cross

(2004) The strength of weak ties you can trust: the mediating role of trust in effective knowledge transfer. Management Science 50(11): 1477–1490.

106.

Little

Rubin

(2002) Statistical Analysis with Missing Data. Hoboken, NJ: John Wiley & Sons.

107.

Loosveldt

Beullens

(2014) A procedure to assess interviewer effects on nonresponse bias. SAGE Open 4(1): 1–12.

108.

Lorah

(2018) Effect size measures for multilevel models: definition, interpretation, and TIMSS example. Large-Scale Assessments in Education 6(1): 1–11.

109.

Lyness

Kropf

(2007) Cultural values and potential nonresponse bias. A multilevel examination of cross-national differences in mail survey response rates. Organizational Research Methods 10(2): 210–224.

110.

Malek

Berger

Coburn

(2007) On the inappropriateness of stepwise regression analysis for model building and testing. European Journal of Applied Physiology 101(2): 263–264.

111.

Malhotra

Ulgado

Agarwal

, et al. (2005) Dimensions of service quality in developed and developing economies: multi-country cross-cultural comparisons. International Marketing Review 22(3): 256–278. DOI: 10.1108/02651330510602204.

112.

Maniaci

Rogge

(2014) Caring about carelessness: participant inattention and its effects on research. Journal of Research in Personality 48: 61–83.

113.

Martin

Resick

Keating

, et al. (2009) Ethical leadership across cultures: a comparative analysis of German and US perspectives. Business Ethics: A European Review 18(2): 127–144.

114.

McCabe

Ingram

Dato-on

(2006) The business of ethics and gender. Journal of Business Ethics 64(2): 101–116.

115.

Messner

(2004) The beauty and importance of quality customer information. The Marketing Review 4(3): 279–290.

116.

Messner

(2017) Market research in India: does the choice of language cause questionnaire contamination? Journal of Indian Business Research 9(2): 149–168.

117.

Minkov

(2012) World Values Survey. Retrieved May 31, 2019, from The Wiley-Blackwell Encyclopedia of Globalization: DOI: 10.1002/9780470670590.wbeog840

118.

Morling

Kitayama

Miyamoto

(2002) Cultural practices emphasize influence in the United States and adjustment in Japan. Personality & Social Psychology Bulletin 28(3): 311–323.

119.

Myers

(2011) Goodbye, listwise deletion: presenting hot deck imputation as an easy and effective tool for handling missing data. Communication Methods and Measures 5(4): 297–310.

120.

Newman

(2009) Missing data techniques and low response rates: the role of systematic nonresponse parameters. In: Lance

Vandenberg

(eds), Statistical and Methodological Myths and Urban Legends. New York, NY: Routledge, 7–36.

121.

Newman

(2014) Missing data: five practical guidelines. Organizational Research Methods 17(4): 372–411.

122.

Ogan

Giglou

d'Haenens

(2017) Challenges of conducting survey research related to a social protest movement: lessons learned from a study of Gezi protests involving the Turkish diaspora in three European countries. The Information Society 33(1): 1–12. DOI: 10.1080/01972243.2016.1248615.

123.

Osborne

Blanchard

(2011) Random responding from participants is a threat to the validity of social science research results. Frontiers in Psychology 1: 1–7.

124.

Presser

(1981) Questions and Answers in Attitude Surveys: Experiments on Question Form, Wording, and Context. San Diego, CA: Academic Press.

125.

Presser

McCulloch

(2011) The growth of survey research in the United States: government-sponsored surveys, 1984-2004. Social Science Research 40(4): 1019–1024.

126.

Purdam

Sakshaug

Bourne

, et al. (2020) Understanding ‘don’t know’ answers to survey questions - an international comparative analysis using interview paradata. Innovation: The European Journal of Social Science Research 33: 1–23.

127.

Randall

Fernandes

(1991) The social desirability response bias in ethics. Journal of Business Ethics 10(11): 805–817.

128.

Rapoport

(1982) Sex differences in attitude expression: a generational explanation. Public Opinion Quarterly 46(1): 86–96.

129.

Raudenbush

Bryk

(2002) Hierarchical Linear Models: Applications and Data Analysis Methods. Newbury Park, CA: Sage.

130.

Riphahn

Serfling

(2005) Item non-response on income and wealth questions. Empirical Economics 30(2): 521–538.

131.

Robbins

Krueger

(2005) Social projection to ingroups and outgroups: a review and meta-analysis. Personality and Social Psychology Review 9(1): 32–47.

132.

Rosenberg

Frank

(2018) Konfound-It! Quantify the robustness of causal inferences. Retrieved from http://konfound-it.com

133.

Roxas

Stoneback

(2004) The importance of gender across cultures in ethical decision-making. Journal of Business Ethics 50(2): 149–165.

134.

Rubin

(1996) Multiple imputation after 18+ years. Journal of the American Statistical Association 91(434): 473–489.

135.

Ruengvirayudh

Brooks

(2016) Comparing stepwise regression models to the best-subset models, or, the art of stepwise. General Linear Model Journal 42(1): 1–14.

136.

Sabanayagam

Shankar

(2012) Income is a stronger predictor of mortality than education in a national sample of US adults. Journal of Health, Population and Nutrition 30(1): 82–86.

137.

Schoderbek

Deshpande

(1996) Impression management, overclaiming, and perceived unethical conduct: the role of male and female managers. Journal of Business Ethics 15(4): 409–414.

138.

Scholz

Zuell

(2012) Item non-response in open-ended questions: who does not answer on the meaning of left and right? Social Science Research 41(6): 1415–1428.

139.

Schumacher

Maas

(2019) The effect of cultural differences on consumers’ willingness to share personal information. In: Lam

Giesler

Luo

(eds), AMA Winter Academic Conference. Understanding Complexity, Transforming the Marketplace. Austin, TX: American Marketing Association, CC41–CC42.

140.

Seifert

(2008) Leverage-saliency theory. In: Lavrakas

(ed), Encyclopedia of Survey Research Methods. Thousand Oaks, CA: Sage, 423–425.

141.

Shoham

(1998) Export performance: a conceptualization and empirical assessment. Journal of International Marketing 6(3): 59–81.

142.

Singelis

Sharkey

(1995) Culture, self-construal, and embarrassability. Journal of Cross-Cultural Psychology: 622–644.

143.

Sivakumar

Nakata

(2001) The stampede toward Hofstede's framework: avoiding the sample design pit in cross-cultural research. Jounral of International Business Studies 32(3): 555–574.

144.

Smith

(2006) When elephants fight, the grass gets trampled: the GLOBE and Hofstede projects. Journal of International Business Studies 37(6): 915–921.

145.

Snijders

Bosker

(2012) Multilevel Analysis: An Introduction to Basic and Advanced Multilevel Modeling. Second edition. Los Angeles, CA: Sage.

146.

Spence

(1993) Gender-related traits and gender ideology: evidence for a multifactorial theory. Journal of Personality and Social Psychology 64(6): 624–635.

147.

Spence

Helmreich

(1978) Masculinity and Feminity. Their Psychological Dimensions, Correlates and Antecedents. Austin, TX: University of Texas Press.

148.

Spiller

Fitzsimons

Lynch

, et al. (2013) Spotlights, floodlights, and the magic number zero: simple effects tests in moderated regression. Journal of Marketing Research 50(2): 277–288.

149.

Stahl

Tung

(2015) Towards a more balanced treatment of culture in international business studies: the need for positive cross-cultural scholarship. Journal of International Business Studies 46(4): 391–414.

150.

Steiner

Maas

(2018) When customers are willing to disclose information in the insurance industry. A multi-group analysis comparing ten countries. International Journal of Bank Marketing 36(6): 1015–1033.

151.

Sumner

(1906) Folkways. Boston, MA: Ginn.

152.

Tanguma

(2000) A review of the literature on missing data. In: Annual Meeting of the Mid-south Educational Research Association. Bowling Green, KY, 1–24. Retrieved from https://files.eric.ed.gov/fulltext/ED448174.pdf.

153.

Taras

Kirkman

Steel

(2010) Examining the impact of culture’s consequences: a three-decade, multilevel, meta-analytic review of Hofstede’s cultural value dimensions. Journal of Applied Psychology 95(3): 405–439.

154.

Taylor

Curtis

(2013) Whistleblowing in audit firms: organizational response and power distance. Behavioral Research in Accounting 25(2): 21–43.

155.

Tourangeau

(1984) Cognitive sciences and survey methods. In: Jabine

Straf

Tanur

, et al. (eds), Cognitive Aspects of Survey Methodology: Building a Bridge between Disciplines. Washington, DC: National Academy Press.

156.

Triandis

(1989) The self and social behavior in differing cultural contexts. Psychological Review 3: 506–520.

157.

Triandis

(1995) Individualism and Collectivism. Boulder, CO: Westview.

158.

Turiel

(1994) Morality, authoritarianism, and personal agency in cultural contexts. In: Sternberg

Ruzgis

(eds), Personality and Intelligence. Cambridge: Cambridge University Press, 271–302.

159.

Turilli

Floridi

(2009) The ethics of information transparency. Ethics and Information Technology 11(2): 105–112.

160.

Vaccaro

Madsen

(2006) Firm information transparency: ethical questions in the information age. In: Berleur

Numinen

Impagliazzo

(eds), IFIP International Federation for Information Processing, Social Informatics: An Information Society for All? in Remembrance of Rob Kling. Boston, MA: Springer, 223, 145–156.

161.

van Rosmalen

van Herk

Groenen

(2010) Identifying response styles: a latent-class bilinear multinomial logit model. Journal of Marketing Research 47(1): 157–172.

162.

Verba

Burns

Schlozman

(1997) Knowing and caring about politics: gender and political engagement. The Journal of Politics 59(4): 1051–1072.

163.

Vischers

Siegrist

(2008) Exploring the triangular relationship between trust, affect, and risk perception: a review of the literature. Risk Management 10(3): 156–167.

164.

Wagner

Stoop

(2018) Comparing nonresponse and nonresponse biases in multinational, multiregional, and multicultural contexts. In: Johnson

Pennell

B-E

Stoop

, et al. (eds), Advances in Comparative Survey Methods. Hoboken, NJ: John Wiley & Sons, 807–833.

165.

Ward

Meade

Allred

, et al. (2017) Careless response and attrition as sources of bias in online survey assessments of personality traits and performance. Computers in Human Behavior 76: 417–430.

166.

Warner

(1965) Randomized response: a survey technique for eliminating evasive answer bias. American Statistical Association Journal 60(309): 63–69.

167.

Watkins

Cheung

(1995) Culture, gender, and response bias. An analysis of responses to the self-description questionnaire. Journal of Cross-Cultural Psychology 26(5): 490–504.

168.

Welzel

Brunkert

Inglehart

, et al. (2019) Measurement equivalence? A tale of false obsessions and a cure. World Values Research 11(3): 54–84.

169.

Woodside

Hsu

S-Y

Marshall

(2011) General theory of cultures’ consequences on international tourism behavior. Journal of Business Research 64(8): 785–799.

170.

Wright

Marsden

(2010) Survey research and social science: history, current practice, and future prospects. In: Marsden

Wright

(eds), Handbook of Survey Research. Second edition. Bingley: Emerald, 3–25.

171.

WVS . (n.d.) Fieldwork and Sampling. Retrieved May 31, 2019, from World Values Survey: http://www.worldvaluessurvey.org/WVSContents.jsp.

172.

Zey-Ferrell

Ferrell

(1982) Role-set configuration and opportunities as predictors of unethical behavior in organizations. Human Relations 35(7): 587–604.

173.

Zhou

Wang

Zhang

, et al. (2017) Who tends to answer open-ended questions in an e-service survey? The contribution of closed-ended answers. Behaviour & Information Technology 36(12): 1274–1284.

174.

Zimmer

Arsal

Al-Marzouq

, et al. (2010) Knowing your customers: Using a reciprocal relationship to enhance voluntary information disclosure. Decision Support Systems 48(2): 395–406.