Response Styles and Personality Traits

Abstract

In two studies, we examined the shared and unique meaning of acquiescent, extreme, midpoint, and socially desirable responding in association with the Occupational Personality Questionnaire (OPQ32), a forced-choice format personality measure designed to be less affected by these response styles, compared with personality inventories with Likert-type scales. Country-level response style indexes were derived from six waves of the International Social Survey Programme and from a meta-analysis of a social desirability scale. In the country-level correlational analysis, the four response styles formed a general response style (GRS) factor which was positively associated with (a) dominance (vs. submission) in interpersonal relationships, (b) competitive (vs. modest and democratic) feelings and emotions, and (c) data rational thinking. In a multilevel analysis, age showed a positive and education a negative effect on the individual-level GRS. Negative effects of country-level socioeconomic development and individualism and positive effects of competitiveness and data rational thinking on the individual-level response style were found. We conclude that country-level response styles are systematically associated with country personality measured by the OPQ32, suggesting that they can be viewed as having substantive meaning (i.e., culturally influenced response amplification vs. moderation). Implications are discussed.

Keywords

response styles acquiescence extremity midpoint responding social desirability OPQ32 Big Five personality

This article focuses on the association of response styles and personality traits in a cross-cultural context. Response styles are defined as respondents’ systematic tendency to answer questionnaire items on some basis other than the specific item content (Paulhus, 1991). Response styles can create validity threats in cross-cultural studies, because observed score differences across cultures may not reflect true differences in the target construct, due to confounding response styles (e.g., van Herk, Poortinga, & Verhallen, 2004). We investigate the shared and unique meaning of the four most studied response styles, namely, acquiescent response style (ARS: the tendency to agree to a statement regardless of content), extreme response style (ERS: the tendency to overuse the end points of a scale), midpoint response style (MRS: the tendency to overuse the midpoint of a scale), and socially desirable responding (SDR: the tendency to answer questions in a way to make oneself look good). ARS, ERS, and MRS can be categorized as uniform response biases (i.e., the tendency to distort responses in a particular direction more or less regardless of the content of the stimulus), whereas SDR is more susceptible to the direction of the content (Cheung & Chan, 2002).

The consistency and stability of these response styles (e.g., Weijters, Geuens, & Schillewaert, 2010) point to their trait-like properties and possibly even links to personality. Researchers have found associations among response styles and personality traits at both individual and country levels. At individual level, ARS was associated with impulsiveness and extraversion (Couch & Keniston, 1960). ERS was positively related to intolerance of ambiguity, simplistic thinking, decisiveness, extroversion, and conscientiousness (Austin, Deary, & Egan, 2006; Naemi, Beal, & Payne, 2009). MRS was associated with evasiveness (Ayidiya & McClendon, 1990). SDR was associated with the general personality factor which consists of the combination of agreeableness, conscientiousness, extroversion, openness, and emotional stability (Bäckström, 2007; Schermer & MacDougall, 2013). However, a clear patterning of correlations between personality and response styles has not been established at country level, mainly because relatively few studies have been conducted. Harzing (2006) reported a positive association of extroversion with ERS and a negative one with MRS; McCrae, Terracciano, and 79 Members of the Personality Profiles of Cultures Project (2005a) reported a negative association of conscientiousness with ARS. Smith (2011) reported a negative association of openness with ARS and none of the Big Five personality traits related to ERS.

Findings on these reported associations could be challenged on two grounds. First, the operationalizations of response styles were different in various studies. For instance, when constructed as the endorsement rate of the agree and strongly agree categories (e.g., 4 and 5 in a 5-point scale), ARS becomes a weak form of ERS (e.g., van Herk et al., 2004); when only the agree category is taken as ARS (e.g., 4 in a 5-point scale) to avoid confounding with ERS, ARS is more related to MRS (e.g., He & van de Vijver, 2013). Second, and more importantly, personality traits, often used as the validity measures of response styles, are typically measured with Likert-type scales, which themselves are likely to be tainted by the same response styles as are being investigated (Bentler, Jackson, & Messick, 1971).

The present study aims to address the two problems raised. To create consistency in the operationalizations of response styles, we integrate the four response styles in a general response style (GRS) to achieve more robust estimates, in addition to examining each response style separately. We predict response styles with personality traits using the forced-choice format version of the Occupational Personality Questionnaire (OPQ32) provided by SHL, a talent measurement solutions company (SHL, 1999, 2006, 2013). SHL is a part of CEB (Corporate Executive Board), which provides talent management services for organizations all over the world. The major difference in the forced-choice format and the conventional Likert-type scale personality instrument is that the former is robust to uniform response biases (i.e., ARS, ERS, and MRS), which allows us to measure country-level personality traits free from these uniform effects of response styles and to explain country variations in response styles that are found using other formats (e.g., Likert-type scales). In Study 1, we constructed country-level response style indexes and studied their integration and correlations with aggregated personality traits. Based on the country-level correlates identified, we applied a multilevel design in Study 2 to examine the unique variance explained by aggregated personality traits in individual-level response styles.

Study 1: Country-Level Analysis of Response Styles and Personality

We test the personality correlates of response styles and examine the convergence of results from personality traits measured with various formats. We focus on relationships of response styles with the forced-choice item format version of the OPQ32. We also use other personality instruments, which employ conventional Likert-type scales, to compare results of instruments that are more susceptible to influences of response styles and to test the generalizability of our results.

Specific Response Style and GRS

It has been found that there is a predictable pattern of correlations among ARS, ERS, MRS, and SDR. ERS and MRS are negatively related to each other, reflecting the contrast of promotion and prevention focus (Cabooter, 2010). ERS and SDR are positively related, as both are associated with desirable traits, such as extroversion and conscientiousness (Austin et al., 2006; Schermer & MacDougall, 2013). ARS, when operationalized as endorsing only the agree category, shows a negative correlation with ERS (Morren, Gelissen, & Vermunt, 2013). To integrate the full range of response styles, a GRS factor can be extracted with ERS and SDR as positive indicators and ARS and MRS as negative indicators (He & van de Vijver, 2013). This GRS factor serves as a filter of questionnaire responses, by moderating or amplifying responses. In other words, the GRS factor has the function of a volume button that can play up or play down responses.

OPQ32

The OPQ32 measures 32 work-related personality traits defined as individual styles or preferences at work in three domains: relationships with people, feelings and emotions, and thinking style (see Table 1 for an overview; SHL, 2009). It was developed with both a normative (Likert-type scales, OPQ32n) and a forced-choice version (OPQ32i, in which respondents choose one statement as most like me and one as least like me in blocks of four statements; SHL, 1999, 2006). The OPQ32i was further evolved, resulting in the OPQ32r (SHL, 2013), which has 104 triplets selected from the original version. Responses to the triplets are scored as paired comparisons to which a multidimensional item response modeling model is applied. It can recover normative scale data from the forced-choice item format (SHL, 2006, 2013). Compared with the OPQ32n and OPQ32i, the OPQ32r measures the same constructs, demonstrates good psychometric properties and external validity, provides a good indication of the trait standing, and, like the OPQ32i, is robust to the effects of uniform response biases.

Table 1.

Definitions of the OPQ32 Scales.

OPQ32	Definition
Domain 1. Relationships with people
Persuasive	Likes to change other people’s views
Controlling	Likes to be in charge
Outspoken	Freely expresses opinions
Independent minded	Prefers to follow own approach
Outgoing	Lively and animated in groups
Affiliative	Likes to be around people
Socially confident	Feels comfortable when first meeting people
Modest	Keeps quiet about personal success
Democratic	Involves others in decision making
Caring	Sympathetic and considerate toward others
Domain 2. Thinking style
Data rational	Enjoys analyzing statistical information
Evaluative	Critically evaluates information
Behavioral	Tries to understand motives and behavior
Conventional	Prefers well-established methods
Conceptual	Enjoys discussing abstract concepts
Innovative	Generates new ideas
Variety seeking	Likes changes to regular routine
Adaptable	Changes behavior to suit the situation
Forward thinking	Takes a long-term view
Detail conscious	Likes to be methodical and organized
Conscientious	Focuses on getting things finished
Rule following	Follows rules and regulations
Domain 3. Feelings and emotions
Relaxed	Finds it easy to relax
Worrying	Worries about things going wrong
Tough minded	Not easily offended
Optimistic	Expects things will turn out well
Trusting	Sees others as reliable and honest
Emotionally controlled	Can conceal feelings from others
Vigorous	Thrives on activity
Competitive	Enjoys competitive activities
Achieving	Likes to work to demanding goals and targets
Decisive	Makes fast decisions

Note. OPQ32 = Occupational Personality Questionnaire.

Through combining 26 out the 32 personality scales in the OPQ32, the Big Five personality scores can be produced (SHL, 2006). Furthermore, the OPQ32 has been adapted for use in 30 languages, and the construct and scalar equivalence have been demonstrated across 39 countries (Bartram, 2013a, 2013b); thus, this country-level measure is proven to be suitable for cross-cultural comparative studies.

The Present Study

Based on what has been reviewed, we aimed to answer three research questions in the present study. First, can a GRS factor, that was found previously at individual level (He & van de Vijver, 2013), be replicated at country level? We expect to find the same GRS factor with positive loadings of ERS and SDR and negative loadings of ARS and MRS at country level.

Second, what are the associations of response styles with expressions of personality traits that are free of uniform response biases? If response styles reflect culturally preferred communication styles (e.g., Smith, 2004), we expect that the preference to dominate in relationships, characterized by high levels of being controlling and outspoken and low levels of being affiliative, modest, and democratic, would be positively related to response styles (in the direction of ERS and SDR vs. ARS and MRS). In the feeling and emotions domain, we expect that being competitive and decisive would be associated with higher levels of response styles (Naemi et al., 2009). In the thinking domain, the preference to analyze statistical information and base inferences on facts and figures (i.e., data rational) to reduce ambiguity may boost the tendency to use response styles.

Third, is there convergence in the associations of response styles with the Big Five personality traits measured with the forced-choice format (i.e., the OPQ32r) and other Likert-based inventories? If response styles only reflect scale usage, we would expect that a forced-choice questionnaire would show other correlations with measured personality traits than a Likert-based measure would. However, if response styles share trait variance with personality, we would expect more convergence between the two types of measures. Bartram (2013a) compared the 31 country-level Big Five scale means extracted from the OPQ32i with three other data sets: the Likert-based self-report of the Revised NEO Personality Inventory (self-report NEO) and other-report of this instrument (other-report NEO), and the Likert-based self-report Big Five Inventory (BFI). He found moderate convergence among these four data sets.

Method

Data sources for ARS, ERS, and MRS

We located six waves of the International Social Survey Programme (ISSP 2003, 2004, 2005, 2006, 2007, and 2008 from www.issp.org) with national representative samples from over 30 countries that had items suitable to construct ARS, ERS, and MRS indexes. Detailed information of the surveys and use of items for response style indexes is presented in Table 2.

Table 2.

Overview of Surveys.

Survey	n of countries	n of respondents	n of items	n of Likert points	Average inter-item correlation
ISSP 2003: National identity	35	45,993
ARS			5	5	−.01
ERS			20	4	.19
MRS			24	5	.08
ISSP 2004: Citizenship	38	52,550
ARS			9	5	.05
ERS			8	7	.20
MRS			8	7	.18
ISSP 2005: Work orientation	31	43,440
ARS			18	5	.09
ERS			9	5	.07
MRS			9	5	.06
ISSP 2006: Role of government	33	48,641
ARS			8	5	.05
ERS			14	5	.03
MRS			14	5	.04
ISSP 2007: Leisure time and sports	34	49,729
ARS			5	5	.06
ERS			11	5	.10
MRS			10	5	.12
ISSP 2008: Religion	40	59,986
ARS			9	5	.07
ERS			17	4	.17
MRS			12	5	.11

Note. ISSP = International Social Survey Programme; ARS = acquiescent response style; ERS = extreme response style; MRS = midpoint response style.

We computed indicators of ARS, ERS, and MRS, using a variety of items that measured different constructs with Likert-type scales in each survey. The average inter-item correlations among items for each response style index ranged from −.01 to .20, indicating sufficient heterogeneity in item content (e.g., Greenleaf, 1992b). We used different items to compute each index, which ensured data independence and avoided statistical artifacts in testing the interrelatedness of response styles. ARS scores were derived from a set of items with five response options, ranging from 1 (strongly agree) to 5 (strongly disagree); the frequency of choosing 2 (agree) was taken as an ARS score (the endorsement of 1 = strongly agree was not used to avoid the confounding with ERS).¹ Due to the limited number of Likert-type scale items with the strongly agree to strongly disagree anchors available in the ISSP data sets, ARS indexes were constructed with smaller numbers of items compared with ERS and MRS. ERS was derived from another set of items with various response options (e.g., not at all to very much, not important to extremely important) other than agree–disagree scales; the frequency of choosing the end points (e.g., 1 and 5 in a 5-point scale) was taken as an ERS score. MRS was constructed with a third set of items with various response options; the frequency of choosing the middle category (e.g., 3 in a 5-point scale) was taken as an MRS score. For each style, we first calculated the individual index for each respondent, and then averaged the indexes across members of the same country to obtain a country-level index.

To get a global indication of country-level response styles, we replaced the missing values of the three response style indexes across the six surveys (20.45% missing) using Expectation-Maximization imputation, given that the missing values were missing completely at random (MCAR) (Little’s MCAR test: χ²(120) = 137.90, p = .13; Little & Rubin, 2002). We treated the six indexes of each response style as scale items, and used the means across the six surveys as scores of ARS, ERS, and MRS. The values of Cronbach’s alpha for the ARS, ERS, and MRS indexes were .85, .88, and .84, respectively.

Data sources for SDR

Scores on the country-level SDR were derived from an instrument-based meta-analysis (van Hemert, 2003) of the Marlowe–Crowne Social Desirability Scale from 42 countries (Domínguez Espinosa & van de Vijver, 2012). This scale was chosen because of its popularity over the years and prolific publications around the world. A total of 1,052 publications from 1960 to 2011, which used the Marlowe–Crowne Social Desirability Scale or its various short forms, were reviewed. Data from 153 journal articles, 4 unpublished theses, and 1 technical report were retained after excluding studies with clinical, army, and inmate samples, and studies with “faking” or experimental instructions. Among all the studies, the scale was used in English (49.7%); French (10.1%); Spanish (8.2%); Chinese (6.9%); German (5.7%); Danish (3.8%); Portuguese (2.5%); Japanese, Swedish, Hebrew, Turkish, and Arab/Iranian (1.9% each); Italian and Dutch (1.3% each); and Croatian, Greek, Norwegian, Indonesian, and Hindu (0.6% each). Means and standard deviations of the Marlowe–Crowne Social Desirability Scale were recorded for each study and aggregated to country level. We used the mean scores of SDR at country level in the present study (for more details of the meta-analysis and country rankings, refer to Domínguez Espinosa & van de Vijver, 2012).

Aggregated personality traits

The country-level sten scores (standardized scores ranging from 1 to 10, with a mean of 5.5 and a standard deviation of 2) of the 32 narrow personality traits as well as the Big Five personality traits from the OPQ32r in 39 countries were obtained from 113,480 working adults with over 20 languages in the data pool of SHL (Bartram, 2013b). These data were collected as part of occupational assessments (i.e., for selection or development purposes in the workplace).

The country-level scale means of the Big Five personality traits from 31 countries with the self-report NEO were obtained from McCrae (2002). The respondents were college students and working adults. The country-level scale means of the Big Five personality traits from 50 countries with the other-report NEO were obtained from McCrae et al. (2005b). The respondents were college students who were instructed to rate someone they knew well.

The country-level scale means of the Big Five personality traits from 56 countries with the self-report BFI were obtained from Schmitt, Allik, McCrae, and Benet-Martínez (2007). The respondents were mainly college students and in a few countries community-based samples.

Results

We report the findings in three parts. We first examined the interrelatedness of the four response styles at country level and the extraction of a GRS factor; then we correlated the response styles with the 32 personality scales from the OPQ32r; and finally, we compared the correlation patterning of response styles with the four sets of Big Five personality data.

Interrelatedness among response styles

Table 3 presents the correlations among the four response styles at country level. ERS and SDR were positively related, and both were negatively related to MRS. The correlations of ARS with the other three response styles were nonsignificant. A one-factor solution was supported from the principal components analysis with the four response styles. This factor (explaining 54.76% of the variance) had positive loadings of ERS (loading .87) and SDR (.72), a negative loading of MRS (−.96), and a loading close to zero for ARS (−.05). This finding largely replicates the GRS representing response moderation to amplification (He & van de Vijver, 2013). The only difference with the previous study was that in the present study ARS had a negative loading closer to zero. We used the factor score as an index of GRS.

Table 3.

Correlations Among the Response Style Indexes at Country Level.

	ARS	ERS	MRS	SDR
ARS	1
ERS	.07	1
MRS	−.21	−.83**	1
SDR	.05	.34	−.60**	1

Note. ARS = acquiescent response style; ERS = extreme response style; MRS = midpoint response style; SDR = socially desirable responding.

p < .01.

Associations of response styles with the 32 personality scales

The response style indexes, including GRS, ARS, ERS, MRS, and SDR, were correlated with the 32 scales of personality traits from the OPQ32r (Table 4). GRS was positively related to Competitive and Data Rational, and negatively related to Affiliative, Modest, Democratic, Caring, and Relaxed. Although not statistically significant, Persuasive, Controlling, and Worrying showed positive associations, and Adaptability, Trusting, Emotionally Controlled, and Vigorous showed negative associations with GRS. The contrast in correlational patterns between ERS and MRS was most salient; ERS had positive associations with Persuasive, Controlling, Outspoken, and Innovative, and negative associations with Affiliative, Adaptable, Democratic, and Emotionally Controlled, whereas MRS showed an opposite pattern. ARS was not significantly associated with these personality traits. SDR was positively related to Independent Minded and negatively related to Affiliative.

Table 4.

Zero-Order Correlation of Response Styles With the OPQ32r at Country Level.

	GRS (n = 19)	ARS (n = 27)	ERS (n = 27)	MRS (n = 27)	SDR (n = 22)
Persuasive	.43	−.14	.55**	−.44*	.02
Controlling	.45	−.09	.65**	−.47*	−.02
Outspoken	.34	−.12	.48*	−.49**	.06
Independent minded	.21	.06	−.23	.06	.48*
Outgoing	−.11	−.23	−.22	.23	−.21
Affiliative	−.48*	−.34	−.44*	.58**	−.51*
Socially confident	−.37	−.08	−.10	.23	−.27
Modest	−.47*	.18	−.24	.29	−.02
Democratic	−.66**	−.11	−.64**	.58**	−.38
Caring	−.48*	.17	−.25	.32	−.28
Data rational	.58**	−.18	.82**	−.62**	.17
Evaluative	.08	−.32	.24	−.06	−.27
Behavioral	−.08	−.22	.04	.28	−.35
Conventional	−.19	−.01	−.02	.10	−.12
Conceptual	−.04	.10	.15	.04	.05
Innovative	.23	.00	.50**	−.38*	−.08
Variety seeking	.00	−.04	−.09	.06	−.15
Adaptable	−.41	.00	−.71**	.63**	.13
Forward thinking	.02	−.01	.52**	−.33	−.01
Detail conscious	−.30	−.15	.05	.07	−.19
Conscientious	−.22	−.16	.13	.01	−.28
Rule following	−.11	−.16	.11	−.09	−.17
Relaxed	−.47*	−.28	−.32	.32	−.33
Worrying	.40	.17	.19	−.27	.16
Tough minded	−.20	.23	−.05	.02	.01
Optimistic	−.32	−.06	−.15	.16	−.21
Trusting	−.44	−.02	−.32	.34	−.31
Emotionally controlled	−.43	−.06	−.50**	.40*	.18
Vigorous	−.41	−.11	−.14	.29	−.31
Competitive	.64**	−.05	.74**	−.54**	.21
Achieving	.21	−.03	.52**	−.37	−.06
Decisive	.16	−.17	.29	−.33	−.12

Note. OPQ32 = Occupational Personality Questionnaire; GRS = general response style; ARS = acquiescent response style; ERS = extreme response style; MRS = midpoint response style; SDR = socially desirable responding.

p < .05. **p < .01.

Associations of response styles with the Big Five personality traits

We correlated the response style indexes with the Big Five personality traits measured by the OPQ32r, self-report NEO, other-report NEO, and self-report BFI (Table 5). GRS was strongly correlated with the OPQ-based agreeableness (negative) and the NEO-based self-report conscientiousness (positive). Emotional stability and extroversion from all the four sets of data were unrelated to any of the response styles. However, the main finding of the analysis was the weak convergence of correlations across instruments. Across all traits, correlations between the correlations shown in Table 5 across instruments were only significant for the two NEO measures, r(25) = .80, p < .001. Notably, the absence of any relationship between the two Likert-based self-report measures is remarkable, even if a lack of significant correlations is a recurrent problem in this field. Moreover, we could not replicate the association of extroversion with ERS and MRS that was found by Harzing (2006), probably due to the different personality measures used (in Harzing’s study, extroversion was from the Eysenck Personality Questionnaire that stresses expressiveness and liveliness). Contrary to McCrae et al. (2005a), we found a positive association of ARS with conscientiousness, which could be explained by the very different operationalization of ARS in their study (i.e., the sum of the raw scores of the 240 NEO personality items as ARS). The failure to replicate the negative association of ARS with openness as in Smith (2011) may be attributed to a similar cause (ARS there was calculated using twice the number of scale point “5” responses, plus the number of scale point “4” responses in a 5-point scale). Our findings suggest that response styles are instrument specific; yet, we cannot rule out the impact of sample incomparability as all measures are based on convenience sampling.

Table 5.

Correlations of Response Styles With Big Five Personality Traits.

	GRS	ARS	ERS	MRS	SDR
Agreeableness
OPQ32	−.71**	.01	−.54**	.54**	−.49*
Self-report NEO	−.38	.18	−.22	.51*	−.03
Other-report NEO	−.14	.12	−.20	.18	−.02
BFI	.40	.28	.50*	−.32	−.03
Conscientiousness
OPQ32	−.26	−.14	.17	.01	−.27
Self-report NEO	.69**	.53*	.69**	−.50*	.44
Other-report NEO	.33	.15	.23	−.39	.21
BFI	.32	.18	.42*	−.24	−.05
Emotional stability
OPQ32	−.45	−.14	−.23	.27	−.26
Self-report NEO	−.20	.00	.07	.25	−.06
Other-report NEO	−.16	.38	−.14	.20	−.05
BFI	.19	.35	.22	−.09	.25
Extroversion
OPQ32	−.02	−.23	.16	.01	−.31
Self-report NEO	−.02	.03	.29	.08	−.42
Other-report NEO	−.16	.09	.14	.19	−.58**
BFI	−.03	.22	.04	−.03	.04
Openness
OPQ32	.15	−.02	.23	−.13	−.07
Self-report NEO	−.22	.15	−.36	.29	−.09
Other-report NEO	−.31	.33	−.45*	.33	−.01
BFI	.52*	.40	.56**	−.38	−.05

Note. n of countries = 19, 27, 27, 27, and 22 for GRS, ARS, ERS, MRS, and SDR in OPQ32; n of countries = 15, 20, 20, 20, and 18 for GRS, ARS, ERS, MRS, and SDR in self-report NEO-PI-R; n of countries = 15, 21, 21, 21, and 19 for GRS, ARS, ERS, MRS, and SDR in other-report NEO-PI-R; n of countries = 16, 23, 23, 23, and 20 for GRS, ARS, ERS, MRS, and SDR in BFI. GRS = general response style; ARS = acquiescent response style; ERS = extreme response style; MRS = midpoint response style; SDR = socially desirable responding; OPQ32 = Occupational Personality Questionnaire; NEO = the Revised NEO Personality Inventory; BFI = Big Five Inventory.

p < .05. **p < .01.

To summarize, we found some significant personality correlates of response styles, and we did not find strong convergence of the associations between response styles and Big Five personality traits. However, it seems that even if we use a response scale that cannot reflect ARS, ERS, or MRS, the Big Five personality scales were still related to response style indexes, which strongly suggest that these indexes have a substantive meaning.

Discussion

We carried out country-level correlational analyses with response styles extracted from national representative respondents from multinational surveys and personality traits from the OPQ32r and other Likert-based inventories. We found a GRS factor (i.e., the shared meaning of ERS and SDR vs. ARS and MRS) that reflected the tendency to amplify or to moderate responses. The significant correlations of some but not all personality traits measured with the forced-choice OPQ32r supported our expectation that response styles have some substantive meaning, given that the uniform response style bias cannot influence this instrument and response styles such as ARS, ERS, and MRS cannot be computed on the basis of the forced-choice instrument. In other words, scores on the OPQ32r cannot reflect scale usage and should have some substantive meaning if there would be significant correlations between scores on this personality inventory and response style indexes. Specifically, we found support that response styles at country level reflect aspects of culture (Smith, 2004), as evidenced in (a) dominating versus submitting to interpersonal relationship, (b) competitive versus modest and democratic feelings and emotions, and (c) data rational thinking. Therefore, response styles can be viewed as culturally moderated communication filters that amplify or moderate responses. Response styles cannot be dismissed as only reflecting scale usage; instead, it appears that the way in which individuals use scales is telling of their personality. These effects are even more pronounced when scores are aggregated at country level.

Study 2: A Multilevel Analysis of Response Styles

Building upon Study 1, we studied the psychological meaning of individual-level ARS, ERS, and MRS (individual-level data on SDR were not available, thus not included) in a multilevel design with country-level OPQ32r scale scores.

Individual (Demographic) Correlates of Response Styles

The effects of age, gender, and education on response styles have been frequently examined. Some studies reported that both ARS and ERS were more commonly used among elderly (Greenleaf, 1992a; Meisenberg & Williams, 2008) and the less educated (Marin, Gamba, & Marin, 1992), whereas MRS was more frequently used by the more educated (Sturgis, Roberts, & Smith, 2010). Yet some studies reported the opposite effects; for example, Austin et al. (2006) found a negative effect of age on ERS. Effects of gender were generally inconsistent and small (Van Vaerenbergh & Thomas, 2012). A possible explanation for such conflicting results could be the sampling scheme in different studies (using either convenience or probability samples). The national representative samples from the ISSP, as used in the current study, are expected to have adequate variations on these background variables and findings based on such samples may afford better ground for generalization.

Country-Level Correlates of Response Styles

We investigate the socioeconomic development, the overarching cultural value (i.e., individualism), and aggregated personality traits at country level that were found to be relevant for response styles. It has been reported that the socioeconomic development of a country has a strong relationship with response styles (van Hemert, van de Vijver, Poortinga, & Georgas, 2002). It may be related to individual opportunities in life; people living in countries with lower levels of socioeconomic development may need to use whatever accessible resources and strategies to survive and thrive, resulting in more use of amplifying response styles.

Individualism (vs. collectivism), the most frequently studied cultural value dimension (Hofstede, 2001), was found to be associated with response styles in many studies. Yet findings were not unequivocal. Harzing (2006) reported a negative association of individualism with ARS and MRS, and no association with ERS. de Jong, Steenkamp, Fox, and Baumgartner (2008) reported a positive association of individualism with ERS. van Herk et al. (2004) reported a negative association of individualism with ERS and ARS. Again, sampling frames could be a confounding factor. We explore the effects of individualism on response styles from our multiple national representative samples.

Drawing from the first study, we selected four personality traits at country level: Affiliative, Democratic, Data Rational, and Competitive. These four personality traits were chosen, because they showed the strongest and most consistent correlations with GRS at country level.

Method

Data sources of individual-level response styles

We used the same ISSP data sets as in Study 1. We calculated the individual-level ARS, ERS, and MRS in each of the six ISSP data sets. We factor analyzed the three response styles at individual level in each set, and found a one-factor solution (with explained variances ranging from 48.51% to 65.10%) with a positive loading of ERS (loadings ranged from .19 to .82 with a mean of .70) and negative loadings of ARS (from −.04 to −.76 with a mean of −.35) and MRS (from −.72 to −.82 with a mean of −.78) for all the six data sets. Pairwise comparisons of the structural equivalence of the factor solutions in the six data sets were carried out by means of the calculation of Tucker’s phi (van de Vijver & Leung, 1997). We found an average value of .90, indicating acceptable structural equivalence. The factor scores were taken as the GRS scores in this study. In addition to these individual-level response styles, we used five dummy variables to gauge ISSP survey-specific effects.

Individual-level predictors

For each respondent in all the six ISSP surveys, data on age (in years), gender (male as 1 and female as 0), and education measured as the highest degree achieved from 1 (no formal qualification) to 5 (university completed) were available.

Country-level predictors

The Human Development Index (HDI) was used to indicate the socioeconomic development of a country. HDI is a composite measure of the average achievements in a country in three basic dimensions of human development: life expectancy, enrollment in education, and Gross Domestic Product per capita. Individualism scores for 70 countries were taken from Hofstede (2009). The sten scores of Affiliative, Democratic, Data Rational, and Competitive from the OPQ32r were the same as in Study 1.

Analysis

We conducted the analyses with HLM version 6 (Raudenbush & Bryk, 2002). In total, data covered 184,433 respondents in 26 countries. The values of the intraclass correlation coefficients were 13% for GRS, 6% for ARS, 15% for ERS, and 10% for MRS, suggesting enough variation at country level to conduct multilevel analysis (van de Vijver & Poortinga, 2002).

In accordance with Enders and Tofighi (2007), we standardized all predictors to z scores, and centered all individual-level predictors around their respective country means and the country-level predictors around the grand mean, except for dummy variables at individual level to distinguish survey differences which were neither standardized nor centered.

In line with the model building proposed by Nezlek (2008), we used the forward-stepping procedure to add predictors, first with individual-level variables and then country-level variables for each model. To be specific, we first checked if there were mean differences across surveys by adding the dummy variables, and then we added age, gender, and education at individual level. Next, we added HDI, then individualism, and finally each of the chosen personality traits from the OPQ32r. As Bartram (2013a) reported that country-level OPQ32 scales were strongly associated with HDI and individualism, adding country-level predictors in such a sequence would allow us to examine the unique variance explained by the aggregated personality traits. We evaluated the significance of the change of chi-square statistics after adding each predictor, from which we determined whether to retain the predictor in the model. All multilevel analyses employed random intercepts and fixed slopes.

Results

Table 6 presents the model building for GRS, ARS, ERS, and MRS. We excluded country-level variables that resulted in an increase in chi-square statistics for each response style index. Table 7 presents the regression coefficients for each response style. At individual level, means of indexes varied across surveys (as indicated in the significant regression coefficients in the survey-specific dummy variables), which pointed to domain-specific effects of response styles (van Dijk, Datema, Piggen, Welten, & van de Vijver, 2009). Age was positively related to GRS and ERS, and negatively related to ARS and MRS. Males scored lower on GRS, and higher on ARS and MRS. Education had a negative effect on GRS and ERS, and a positive effect on MRS. However, all these effects were small.

Table 6.

Model Building in Multilevel Analysis.

	χ²	df	Δχ²	Δdf
(a) Model comparison for GRS
(1) HDI	8,615	24	—	—	—
(2) HDI + IND	6,776	23	1,839	1	(2) − (1)
(3) HDI + IND + Affiliative	6,977	22	−201	1	(3) − (2)
(4) HDI + IND + Democratic	6,937	22	−161	1	(4) − (2)
(5) HDI + IND + Data Rational	3,935	22	2,841	1	(5) − (2)
(6) HDI + IND + Data Rational + Competitive	3,539	21	396	1	(6) − (5)
(b) Model comparison for ARS
(1) HDI	6,058	24	—	—	—
(2) HDI + Individualism	6,079	23	−21	1	(2) − (1)
(3) HDI + Affiliative	4,669	23	1,389	1	(3) − (1)
(4) HDI + Affiliative + Democratic	4,675	22	−6	1	(4) − (3)
(5) HDI + Affiliative + Data Rational	4,675	22	−6	1	(5) − (3)
(6) HDI + Affiliative + Competitive	4,682	22	−13	1	(6) − (3)
(c) Model comparison for ERS
(1) HDI	9,911	24	—	—	—
(2) HDI + Individualism	8,362	23	1,549	1	(2) − (1)
(3) HDI + Individualism + Affiliative	8,544	22	−182	1	(3) − (2)
(4) HDI + Individualism+ Democratic	7,448	22	914	1	(4) − (2)
(5) HDI + Individualism + Democratic + Data Rational	3,757	21	3,691	1	(5) − (4)
(6) HDI + Individualism+ Data Rational + Democratic + Competitive	2,962	21	795	1	(6) − (5)
(d) Model comparison for MRS
(1) HDI	5,258	24	—	—	—
(2) HDI + Individualism	3,568	23	1,690	1	(2) − (1)
(3) HDI + Individualism + Affiliative	3,678	22	−110	1	(3) − (2)
(4) HDI + Individualism + Democratic	3,630	22	−62	1	(4) − (2)
(5) HDI + Individualism + Data Rational	2,633	22	935	1	(5) − (4)
(6) HDI + Individualism + Data Rational + Competitive	2,584	21	49	1	(6) − (5)

Note. Italicized variables were excluded in the model due to the increase of the chi-square values when they were added. GRS = general response style; HDI = Human Development Index; ARS = acquiescent response style; ERS = extreme response style; MRS = midpoint response style.

Table 7.

Regression Coefficients in Multilevel Analysis.

	GRS	ARS	ERS	MRS
Individual-level predictors
Age	.0960**	−.0027**	.0083**	−.0122**
Gender (male)	−.0111**	.0044**	.0017	.0016**
Education	−.0442**	−.0008	−.0047**	.0057**
ISSP 2004	−.0137^†	.4132**	.1100**	−.0888**
ISSP 2005	.0207**	−.0151**	−.1290**	−.0551**
ISSP 2006	−.0802**	.0248**	−.1808**	.0543**
ISSP 2007	−.0856**	.0454**	.0037	.0592**
ISSP 2008	.0419**	−.0388**	.1506**	.0830**
Country-level predictors
HDI	−.1870**	.0442**	−.0303**	.0227**
Individualism	−.0895^†	—	−.0162*	.0143**
Affiliative	—	−.0387**	—	—
Data rational	.1232*	—	.0235*	−.0132*
Democratic	—	—	.0024	—
Competitive	.0473	—	.0219^†	−.0025

Note. GRS = general response style; ARS = acquiescent response style; ERS = extreme response style; MRS = midpoint response style; ISSP = International Social Survey Programme; HDI = Human Development Index.

†

p < .10. *p < .05. **p < .01.

At country level, GRS was characterized by low HDI, low Individualism, high Data Rational, and high Competitive; the same patterning was found for ERS, and the opposite patterning for MRS. ARS was predicted only by high HDI and low Affiliative.

Discussion

Using national representative samples from 26 countries, we studied the individual-level response styles in associations with age, gender, and education at individual level, and socioeconomic development, individualism, and aggregated personality traits at country level. The regression coefficients for GRS at individual level were consistent, and they were stronger than those of specific response styles, indicating that the use of a GRS results in more robust estimates. The effects of age and education were largely in line with Meisenberg and Williams (2008); older and less educated respondents tend to use amplifying response styles more, whereas younger and more educated respondents tend to use moderating response styles more. It seems that being from a younger generation with higher education is associated with a preference for more nuanced views, which may work against the use of ERS and may be more in line with preferences of middle categories of response scales. Gender differences were much smaller than those of age and education; therefore, gender is a less salient predictor for response styles.

At country level, HDI and individualism were negatively associated with ERS and positively associated with MRS. In our view, this patterning suggests that individuals in less affluent and more collectivistic countries show a stronger tendency to stress their conformity and to amplify what the members of their group are supposed to respond. Aggregated personality traits, although closely linked with cultural values (Bartram, 2013a; Hofstede & McCrae, 2004), could explain additional variance in response styles, which furthered our understanding of the mechanism underlying response styles. The positive effects of Competitive on GRS and ERS and the opposite effect on MRS demonstrate that response amplification is more prevalent in countries where people enjoy competitive activities more. The effects of the Data Rational scale provide evidence for the rational thinking–based amplifying response style. It seems counterintuitive that GRS is positively associated with data rational thinking, as data rational thinking is expected to be positively related to education level at individual level and affluence at country level. Enjoying the analysis of statistical information and basing decision on facts and figures may be positively associated with education; however, the unwillingness to deal with opinions and feelings points to rigidity and inflexibility, which could lead to the endorsement of ERS rather than ARS and MRS. To conclude, the multilevel analysis of the large set of response style with predictors from both levels presents a complex picture of the response style constructs.

General Discussion

We studied the psychological meaning of response styles in a correlational and a multilevel study with the OPQ32r and other country characteristics. We found that response styles are interrelated, thus a GRS can be extracted with positive loadings of ERS and SDR (when available), and negative loadings of ARS and MRS, at both individual and country levels. The general as well as specific response styles are systematically related to socioeconomic development, individualism, and aggregated personality traits (in particular Competitive and Data Rational) from the OPQ32r at country level, although the convergence of associations with the Big Five personality across measures is not very high. At individual level, response styles are systematically related to age and education.

Interpretation of Response Styles

There has been an ongoing debate on the nature of response styles. On one hand, response styles are viewed as mere measurement artifacts; they may pose serious challenges to the validity and comparability of scores (Clarke, 2001) or have negligible effects (Ones, Viswesvaran, & Reiss, 1996). On the other hand, response styles are a promising candidate in characterizing aspects of national cultures (Smith, 2009), as they may represent preferred ways of communication that are embedded in individual and cultural values and personality traits (Harzing, 2006; Smith, 2004, 2011). Our findings provide some evidence for the substance interpretation of response styles. The personality traits measured by the forced-choice format OPQ32r provided the unique opportunity for response styles to manifest themselves as systematic effects on scores other than uniform response biases that prevail in the analysis of Likert-based scales. At country level, there are significant positive associations of GRS (i.e., ERS and SDR vs. ARS and MRS) with Competitive and Data Rational, and negative associations with Affiliative, Democratic, and Modest. Moreover, controlling for the effects of socioeconomic development and individualism, we still see unique effects of Competitive and Data Rational on individual-level response styles; therefore, it is reasonable to conclude that response styles are more about substance than nuisance (e.g., Schwartz, Verkasalo, Antonovsky, & Sagiv, 1997).

As we found significant correlations of response styles with traits measured by a personality instrument that cannot display ARS, ERS, or MRS, response styles seem to be integrated in personality traits or at least their expression. Thus, response styles may not be a red herring as suggested by Ones et al. (1996). In practice, researchers should be cautious in applying any measure to control for response styles (Fischer, 2004). If we were to correct for such response styles, we would eliminate valid personality variance. We conclude that GRS, the integration of the four styles, can be interpreted as a filter that influences all self-reports, yet the extent of the influence may vary across constructs. Response styles share trait variance with constructs related to dominance, competitiveness, and data rational thinking; therefore, response styles are better not be corrected for in the measurement of these constructs.

Convergence and Divergence of Personality Instruments

We set out to study response styles with a personality instrument that is robust to the effects of uniform response biases and found different levels of convergence with other personality instruments. In predicting response styles with the 32 individual personality traits of the OPQ32r, the significant personality trait correlates were in line with expectation, yet the aggregated Big Five personality traits showed a less consistent pattern. The lack of convergence may be due to sample differences (e.g., small country-level sample sizes, incomparable samples from convenience sampling), instrument characteristics (e.g., different response formats in which different levels of uniform biases are embedded), and construct differences (e.g., the OPQ-based Big Five personality shared the working definitions of the five traits with the NEO). Mõttus et al. (2012) found that country rankings of self-reported conscientiousness from Likert-type scales would change if response styles were taken into consideration, which demonstrated the strong uniform response biases resulting from the Likert-type scales. All in all, caution should be taken when interpreting the association of response styles or other constructs with the Big Five personality traits derived from different measures, especially when forced-choice format personality measures, such as the OPQ32r, can provide valuable additional information.

Limitations and Further Directions

The present study has a few limitations. First, our response style indexes were constructed with items measuring other constructs, therefore it is difficult to ascertain that these response style indexes are adequately disentangled from the target constructs. However, the independent assessment and the high reliability in each response style alleviate these measurement concerns. Second, we operationalized ARS as the endorsement rate of the agreeing category with limited numbers of items, which may explain the very low loading of ARS on the GRS and its weak correlations with personality traits. However, irrespective of the various operationalizations of ARS, integrating different response styles into a general factor, which represents response amplification to moderation, helps to create consistency in findings. For these with a special interest in ARS, alternative data sources may be pursued. Items with dichotomized response options such as Agree/Disagree can be used to derive ARS indexes that are not confounded with ERS or MRS. Third, the OPQ32r personality data were collected among working adults, which may pose some sample bias in generalizing the findings. Further efforts should improve the measurements of response styles, such as to develop direct measures of response styles or to locate external evidence to validate the indirect measures of response styles. When studying the underlying mechanism of response styles across cultures, it is also imperative to include unobtrusive measures such as HDI, measures with response formats that are resistant to response styles, and other techniques to effectively control for response styles (e.g., anchoring vignettes). In conclusion, the integration of specific response styles and the efforts to employ validity measures of different response formats are promising in deepening our understanding of the nature of response styles and ensuring valid cross-cultural comparisons.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Notes

References

Austin

E. J.

Deary

I. J.

Egan

(2006). Individual differences in response scale use: Mixed Rasch modelling of responses to NEO-FFI items. Personality and Individual Differences, 40, 1235-1245. doi:10.1016/j.paid.2005.10.018

Ayidiya

S. A.

McClendon

M. J.

(1990). Response effects in mail surveys. Public Opinion Quarterly, 54, 229-247. doi:10.1086/269200

Bäckström

(2007). Higher-order factors in a five-factor personality inventory and its relation to social desirability. European Journal of Psychological Assessment, 23, 63-70. doi:10.1027/1015-5759.23.2.63

Bartram

(2013a). Scalar equivalence of OPQ32: Big five profiles of 31 countries. Journal of Cross-Cultural Psychology, 44, 61-83. doi:10.1177/0022022111430258

Bartram

(2013b). A cross-validation of between country differences in personality using the OPQ32. International Journal of Quantitative Research in Education, 1, 182-211. doi:10.1504/IJQRE.2013.056460

Bentler

P. M.

Jackson

D. N.

Messick

(1971). Identification of content and style: A two-dimensional interpretation of acquiescence. Psychological Bulletin, 76, 186-204. doi:10.1037/h0031474

Cabooter

E. F. K.

(2010). The impact of situational and dispositional variables on response styles with respect to attitude measures (Doctor in Applied Economic Sciences dissertation). Ghent University, Ghent, Belgium.

Cheung

M. W. L.

Chan

(2002). Reducing uniform response bias with ipsative measurement in multiple-group confirmatory factor analysis. Structural Equation Modeling, 9, 55-77. doi:10.1207/s15328007sem0901_4

Clarke

III . (2001). Extreme response style in cross-cultural research. International Marketing Review, 18, 301-324. doi:10.1108/02651330110396488

10.

Couch

Keniston

(1960). Yea-sayers and nay-sayers: Agreeing response set as a personality variable. Journal of Abnormal and Social Psychology, 60, 151-172. doi:10.1037/h0040372

11.

de Jong

M. G.

Steenkamp

J.-B. E. M.

Fox

J.-P.

Baumgartner

(2008). Using item response theory to measure extreme response style in marketing research: A global investigation. Journal of Marketing Research, 45, 104-115. doi:10.1509/jmkr.45.1.104

12.

Domínguez Espinosa

A. D. C.

van de Vijver

F. J. R.

(2013). A cross-cultural meta-analysis of social desirable responding. Manuscript submitted for publication.

13.

Enders

C. K.

Tofighi

(2007). Centering predictor variables in cross-sectional multilevel models: A new look at an old issue. Psychological Methods, 12, 121-138. doi:10.1037/1082-989x.12.2.121

14.

Fischer

(2004). Standardization to account for cross-cultural response bias: A classification of score adjustment procedures and review of research in JCCP. Journal of Cross-Cultural Psychology, 35, 263-282. doi:10.1177/0022022104264122

15.

Greenleaf

E. A.

(1992a). Improving rating scale measures by detecting and correcting bias components in some response styles. Journal of Marketing Research, 29, 176-188. doi:10.2307/3172568

16.

Greenleaf

E. A.

(1992b). Measuring extreme response style. Public Opinion Quarterly, 56, 328-351. doi:10.1086/269326

17.

Harzing

A.-W.

(2006). Response styles in cross-national survey research: A 26-country study. International Journal of Cross Cultural Management, 6, 243-266. doi:10.1177/1470595806066332

18.

van de Vijver

F. J. R.

(2013). A general response style factor: Evidence from a multi-ethnic study in the Netherlands. Personality and Individual Differences, 55, 794-800. doi:10.1016/j.paid.2013.06.017

19.

Hofstede

(2001). Culture’s consequences: Comparing values, behaviors, institutions, and organizations across nations. Thousand Oaks, CA: SAGE.

20.

Hofstede

(2009). Dimension data matrix. Retrieved from http://www.geerthofstede.nl/research–vsm

21.

Hofstede

McCrae

R. R.

(2004). Personality and culture revisited: Linking traits and dimensions of culture. Cross-Cultural Research, 38, 52-88. doi:10.1177/1069397103259443

22.

Little

R. J. A.

Rubin

D. B.

(2002). Statistical analysis with missing data (2nd ed.). New York, NY: Wiley.

23.

Marin

Gamba

R. J.

Marin

B. V.

(1992). Extreme response style and acquiescence among Hispanics. Journal of Cross-Cultural Psychology, 23, 498-509. doi:10.1177/0022022192234006

24.

McCrae

R. R.

(2002). NEO-PI-R data from 36 cultures: Further intercultural comparisons. In McCrae

R. R.

Allik

(Eds.), The Five-Factor model of personality across cultures (pp. 105-125). New York, NY: Kluwer Academic.

25.

McCrae

R. R.

Terracciano

79 Members of the Personality Profiles of Cultures Project. (2005a). Personality profiles of cultures: Aggregate personality traits. Journal of Personality and Social Psychology, 89, 407-425. doi:10.1037/0022-3514.89.3.407

26.

McCrae

R. R.

Terracciano

79 Members of the Personality Profiles of Cultures Project. (2005b). Universal features of personality traits from the observer’s perspective: Data from 50 cultures. Journal of Personality and Social Psychology, 88, 547-561. doi:10.1037/0022-3514.88.3.547

27.

Meisenberg

Williams

(2008). Are acquiescent and extreme response styles related to low intelligence and education? Personality and Individual Differences, 44, 1539-1550. doi:10.1016/j.paid.2008.01.010

28.

Morren

Gelissen

J. P. T.M.

Vermunt

J. K.

(2013). Exploring the response process of culturally differing survey respondents with a response style: A sequential mixed methods study. Field Methods, 25, 162-181. doi:10.1177/1525822x12453535

29.

Mõttus

Allik

Realo

Rossier

Zecca

Ah-Kion

. . . Johnson

(2012). The effect of response style on self-reported conscientiousness across 20 countries. Personality and Social Psychology Bulletin, 38, 1423-1436. doi:10.1177/0146167212451275

30.

Naemi

B. D.

Beal

D. J.

Payne

S. C.

(2009). Personality predictors of extreme response style. Journal of Personality, 77, 261-286. doi:10.1111/j.1467-6494.2008.00545.x

31.

Nezlek

J. B.

(2008). An introduction to multilevel modeling for social and personality psychology. Social & Personality Psychology Compass, 2, 842-860. doi:10.1111/j.1751-9004.2007.00059.x

32.

Ones

D. S.

Viswesvaran

Reiss

A. D.

(1996). Role of social desirability in personality testing for personnel selection: The red herring. Journal of Applied Psychology, 81, 660-679. doi:10.1037/0021-9010.81.6.660

33.

Paulhus

D. L.

(1991). Measurement and control of response biases. In Robinson

Shaver

Wrightsman

(Eds.), Measures of personality and social psychological attitudes (Vol. 1., pp. 17-59). San Diego, CA: Academic Press.

34.

Raudenbush

S. W.

Bryk

A. S.

(2002). Hierarchical linear models: Applications and data analysis methods (2nd ed.). Newbury Park, CA: SAGE.

35.

Schermer

J. A.

MacDougall

(2013). A general factor of personality, social desirability, cognitive ability, and the survey of work styles in an employment selection setting. Personality and Individual Differences, 54, 141-144. doi:10.1016/j.paid.2012.08.012

36.

Schmitt

D. P.

Allik

McCrae

R. R.

Benet-Martínez

(2007). The geographic distribution of Big Five personality traits. Journal of Cross-Cultural Psychology, 38, 173-212. doi:10.1177/0022022106297299

37.

Schwartz

S. H.

Verkasalo

Antonovsky

Sagiv

(1997). Value priorities and social desirability: Much substance, some style. British Journal of Social Psychology, 36, 3-18. doi:10.1111/j.2044-8309.1997.tb01115.x

38.

SHL. (1999). The OPQ32 manual. Surrey, UK: Author.

39.

SHL. (2006). The OPQ32 technical manual. Surrey, UK: Author.

40.

SHL. (2009). The OPQ32r user manual. Surrey, UK: Author.

41.

SHL. (2013). The OPQ32r technical manual. Surrey, UK: Author.

42.

Smith

P. B.

(2004). Acquiescent response bias as an aspect of cultural communication style. Journal of Cross-Cultural Psychology, 35, 50-61. doi:10.1177/0022022103260380

43.

Smith

P. B.

(2009). On finding improved ways of characterizing national cultures. In Wyer

R. S.

Chiu

C.-y.

Hong

Y.-y.

(Eds.), Understanding culture: Theory, research, and application (pp. 151-161). New York, NY: Psychology Press.

44.

Smith

P. B.

(2011). Communication styles as dimensions of national culture. Journal of Cross-Cultural Psychology, 42, 216-233. doi:10.1177/0022022110396866

45.

Sturgis

Roberts

Smith

(2010). Middle alternatives revisited: How the neither/nor response acts as a face-saving way of saying I don’t know (S3RI Methodology Working Papers, M10/01). Southampton, UK: Southampton Statistical Sciences Research Institute.

46.

van de Vijver

F. J. R.

Leung

(1997). Methods and data analysis of comparative research. Thousand Oaks, CA: SAGE.

47.

van de Vijver

F. J. R.

Poortinga

Y. H.

(2002). Structural equivalence in multilevel research. Journal of Cross-Cultural Psychology, 33, 141-156. doi:10.1177/0022022102033002002

48.

van Dijk

T. K.

Datema

Piggen

A.-L. J. H. F.

Welten

S. C. M.

van de Vijver

F. J. R.

(2009). Acquiescence and extremity in cross-national surveys: Domain dependence and country-level correlates. In Gari

Mylonas

(Eds.), Quod erat demonstrandum: From Herodotus’ ethnographic journeys to cross-cultural research (pp. 149-158). Athens, Greece: Pedio Books.

49.

van Hemert

D. A.

(2003). Cross-cultural meta-analyses. Online Readings in Psychology and Culture (Unit 2). Retrieved from http://scholarworks.gvsu.edu/orpc/vol2/iss2/5

50.

van Hemert

D. A.

van de Vijver

F. J. R.

Poortinga

Y. H.

Georgas

(2002). Structural and functional equivalence of the Eysenck Personality Questionnaire within and between countries. Personality and Individual Differences, 33, 1229-1249. doi:10.1016/s0191-8869(02)00007-7

51.

van Herk

Poortinga

Y. H.

Verhallen

T. M. M.

(2004). Response styles in rating scales: Evidence of method bias in data from six EU countries. Journal of Cross-Cultural Psychology, 35, 346-360. doi:10.1177/0022022104264126

52.

van Vaerenbergh

Thomas

T. D.

(2012). Response styles in survey research: A literature review of antecedents, consequences, and remedies. International Journal of Public Opinion Research, 25, 195-217. doi:10.1093/ijpor/eds021

53.

Weijters

Geuens

Schillewaert

(2010). The stability of individual response styles. Psychological Methods, 15, 96-110. doi:10.1037/a0018721