The Influence of Item Characteristics on Acquiescence among Latino Survey Respondents

Abstract

Acquiescence is often defined as the systematic selection of agreeable (“strongly agree”) or affirmative (“yes”) responses to survey items, regardless of item content or directionality. This definition implies that acquiescence is immune to item characteristics; however, the influence of item characteristics on acquiescence remains largely unexplored. We examined the influence of eight item characteristics on acquiescence in a telephone survey of 400 Latinos and non-Latino Whites: qualified wording, mental comparisons, negated wording, unfamiliar terms, ambiguous wording, knowledge accessibility, item length, and polysyllabic wording. Negated and ambiguous wording was associated with reduced acquiescence for the full sample as well as subsamples stratified by ethnicity and sociodemographic characteristics. This effect was strongest among younger, more educated, and non-Latino white respondents. No other item characteristics had a significant influence on respondent acquiescence. Findings from this study suggest that acquiescence may be affected by interactions between respondent and item characteristics.

Acquiescence has been defined as the tendency to systematically agree with survey items, regardless of item content or directionality (Baumgartner and Steenkamp 2001). Acquiescence can threaten data validity by inducing error in survey data (Billiet and Davidov 2008; Billiet and McClendon 2000; Cheung and Rensvold 2000; Hoffman et al. 2013; Weijters et al. 2008) and has been shown to be socially patterned, with higher acquiescence observed among certain social groups. However, the underlying causes of acquiescence are not well understood, and it is not clear why some groups evince higher acquiescence than others.

This research examines the influence of item characteristics on acquiescence during telephone surveys with U.S. Latinos¹ who represent an important survey sociodemographic. Latinos have been shown to be more likely to acquiesce than non-Latino Whites (NLWs; Aday et al. 1980; Marín et al. 1992; Ross and Mirowsky 1984), but it is not known why this ethnic difference persists. Latino subgroups are heterogeneous, which may lead to distinct patterns of acquiescence among Latino subgroups that may be obscured in studies that combine Latinos into a single analytical group. Latinos are the largest and second-fastest growing racial/ethnic minority population in the United States (Flores 2017); thus, understanding the survey response process among Latinos is critical to the conduct of both Latino-focused and national surveys.

Across racial and ethnic groups, acquiescence has been consistently associated with older age (De Beuckelaer et al. 2010; Meisenberg and Williams 2008; Ross and Mirowsky 1984; Weijters et al. 2010) and lower education (Billiet and McClendon 2000; Davis et al. 2011; De Beuckelaer et al. 2010; Meisenberg and Williams 2008; Weijters et al. 2010). It remains unclear, however, how age and education affect a respondent’s proclivity to acquiesce. One explanation is that older age and lower education reduce a respondent’s ability to process survey items. Narayan and Krosnick (1996) suggest that individuals with less education may have more limited cognitive resources for engaging in the effortful processing of survey items, which may encourage acquiescence as a way to reduce the effort of answering. Acquiescence may therefore be a form of satisficing (Krosnick 1991). This conceptualization is partially supported by a study with older adults in which respondents with lower cognitive abilities were more likely to acquiesce (Lechner and Rammstedt 2015). Other studies have found increased survey response times for older and less-educated adults, indicating that these respondent characteristics increase item processing difficulty (Yan and Tourangeau 2008).

Item Characteristics and Acquiescence

At least two studies have observed that acquiescence affects some items within a questionnaire more than others (De Beuckelaer et al. 2010; Kam and Zhou 2015), suggesting that item characteristics may play a contextual role in encouraging or discouraging acquiescence. However, only a few studies have examined item characteristics and acquiescence. Respondent and item characteristics may interact, such that when survey items contain characteristics that increase response difficulty, the tendency of respondents to acquiesce may be accentuated when such items are administered to acquiescence-prone individuals. Consistent with research on respondent characteristics, respondents may be more prone to acquiesce when their cognitive resources are overburdened, whether due to cognitive limitations, task difficulty, or insufficient motivation. Thus, survey items that demand greater cognitive effort may yield higher acquiescence. The current study examined three ways in which survey items may intensify cognitive effort: amplifying cognitive load, increasing question comprehension difficulty, and requesting inaccessible knowledge.

Cognitive Load

When respondents have limited cognitive ability to process items or are unwilling to engage in the requested mental effort, it has been proposed that items that increase cognitive load (the mental effort necessary to process a survey question) may yield greater acquiescence when agreement is the default response (Krosnick 1991; Narayan and Krosnick 1996). However, the limited research on cognitive load and acquiescence has been mixed. One study indicates that respondents are more likely to acquiesce while experiencing higher cognitive load (Knowles and Condon 1999), while another study found no relationship between acquiescence and cognitive load (Lenzner et al. 2010).

One item characteristic that may increase cognitive load is qualified wording, which requires simultaneous consideration of the core question as well as qualifiers that restrict one’s thinking to certain conditions or circumstances. For example, to respond to “If it looks like it will rain, I bring an umbrella to work,” a respondent must concurrently evaluate “I bring an umbrella to work” and the qualifier “if it looks like it will rain.” Assuming that qualified wording would induce higher cognitive load, we predicted that items with qualified wording would be associated with increased acquiescence (Hypothesis 1a). Items that require respondents to compare two concepts (e.g., “Apples taste sweeter than grapes”) may also induce higher cognitive load, as they similarly ask respondents to hold two concepts in mind (the sweetness of apples and sweetness of grapes) rather than a single concept (“Apples taste sweet”). We therefore hypothesized that mental comparisons would induce higher acquiescence (Hypothesis 1b).

Questions including negations (e.g., “not,” “never”) may increase cognitive load because, to accurately interpret them, respondents must reverse the meaning of specific parts of a statement in their minds. Previous research indicates that negated wording can increase the cognitive difficulty of responding to survey items, as evidenced by longer response processing time (Kamoen et al. 2017) and increased rereading of questions (Kamoen et al. 2011, 2017). Another form of negation, which Converse and Presser (1986) referred to as “implicit negatives” (e.g., “illegal” and “unnoticed”), may similarly enhance item processing difficulty (Weijters and Baumgartner 2012). As respondents with inadequate ability or motivation to process negated wording may choose to acquiesce in lieu of investing more cognitive energy (Knowles and Condon 1999), we predicted that items containing negated wording would be associated with higher acquiescence (Hypothesis 1c).

Question Comprehension

A variety of item characteristics can affect respondents’ understanding of the question and their response task. Unfamiliar terms, ambiguous wording (Fowler 1992; Johnson et al. 2006; Lenzner et al. 2011; Tourangeau et al. 2000), increased item length (Andrews 1984; Johnson et al. 2006, 2015; Laurent 1972), and higher reading levels, even for orally administered items (Holbrook et al. 2016; Johnson et al. 2006), are all believed to increase question comprehension difficulty. Since we could not locate a method for calculating item reading levels in English and Spanish, we examined the numbers of total words and polysyllabic words (Kincaid et al. 1975; McLaughlin 1969), which are often assessed when estimating reading levels. As we predicted that respondents would default to acquiescence when items comprehension difficulty increased, we hypothesized that unfamiliar terms (Hypothesis 2a), ambiguous wording (Hypothesis 2b), item length (Hypothesis 2c), and polysyllabic words (Hypothesis 2d) would be positively associated with acquiescence.

Knowledge Accessibility

It is a tenet of questionnaire design that researchers should only ask questions that respondents can answer (e.g., Fowler 1995). When respondents have insufficient knowledge to answer a question, they may acquiesce as a form of satisficing (Krosnick 1991). This process may explain why researchers have observed higher acquiescence among respondents with lower ability and/or cognitive energy (Krosnick 1999). We predicted that when knowledge accessibility was high for a survey item (i.e., responding required knowledge that respondents were likely to believe they had), responding would require less cognitive effort and, consequently, acquiescence would be lower (Hypothesis 3).

Goals of the Current Research

In the current study, we examined the influence of four respondent and eight item characteristics on acquiescence among NLW, Cuban American, Mexican American, and Puerto Rican respondents. Focusing on Latinos allowed us to examine acquiescence among a sizable and growing respondent population in the United States that tends to exhibit acquiescence as well as to explore differences in acquiescence across three of the largest Latino subgroups (Ennis et al. 2011). NLWs were included to explore whether ethnic differences would be observed and explained by item characteristics. The data for this study consisted of a unique corpus of 90 items that we specifically selected and/or created to contain the item characteristics of interest.

Method

Participants

Data for this study were obtained through a telephone survey of 400 Cuban American, Mexican American, Puerto Rican, and NLW adults. All participants were recruited from a listed sample of landline and cellular telephone numbers from a commercial vendor, which targeted numbers associated with addresses located in the five largest markets for each Latino group in the United States, including Puerto Rico, individuals with no more than 12 years of education, and a household income of $25,000 or less. These criteria were selected to ensure that the sample included Latinos and NLWs as well as participants who would be more likely to acquiesce, since lower education (De Beuckelaer et al. 2010) and income levels (Greenleaf 1992a; Meisenberg and Williams 2008; Warnecke et al. 1997) have been associated with acquiescence. Telephone numbers were randomly selected for inclusion within the markets targeting each Latino subgroup, which also included numbers associated with NLWs. For cellular numbers, the person who answered the telephone was identified as the respondent. For landline numbers, an adapted version of a within-household selection method was used in an attempt to minimize gender bias (Lavrakas 2008). Respondents were considered eligible to participate if they were between the ages of 18 and 90, spoke English and/or Spanish, or self-identified as NLW or one of the three Latino subgroups. The survey response rate was 10.1% (American Association for Public Opinion Research [AAPOR] Response Rate 3; AAPOR 2016).

Data Collection

Data were obtained through a telephone survey conducted between November 2015 and January 2016 by bilingual interviewers. Research suggests that acquiescence may be more prevalent during telephone-administered surveys than face-to-face (Jordan et al. 1980), paper-and-pencil, or online surveys (Weijters et al. 2008). The survey took an average of 36.5 minutes to complete, including recruitment, eligibility screening, and informed consent. Respondents who completed the survey were mailed a $20 gift card to a retail store. This study was approved by a university-affiliated institutional review board.

Measures

All study materials were developed in English, translated into Spanish by a company specializing in Spanish translations, and reviewed by multiple, bilingual study team members. English and Spanish wordings were revised to improve clarity and consistency in meaning across languages.

Acquiescence

The acquiescence variable used responses to 90 heterogeneous items that were administered in random order and intentionally selected to include the item characteristics under investigation (Online Appendix A). All items consisted of attitude statements, used an endpoint-only labeled response scale ranging from 1 = “strongly disagree” to 7 = “strongly agree,” and assessed a variety of topics as recommended by Baumgartner and Steenkamp (2001) and Greenleaf (1992a, 1992b). Original responses were recoded, such that responses of 1, 2, 3, or 4 were converted to 0s, responses of 5 were converted to 1s, responses of 6 were converted to 2s, and responses of 7 were converted to 3s. This coding is similar to that used by Baumgartner and Steenkamp to capture the presence and intensity of acquiescence. The acquiescence variable is the mean of these recoded responses (range: 0–3).

Cognitive Load

Three characteristics were identified as indicators of higher cognitive load during item processing: qualified wording, mental comparisons, and negated wording. Two authors independently coded each item for these characteristics, while a third served as a tiebreaker.

Qualified wording

Items included qualified wording if they contained one or more terms that requested respondents to consider the queried phenomena only at certain times or under certain conditions. Examples of qualified wording include “only,” “at times,” “unless,” and “in the last 30 days.” Zero indicated that the item did not include qualified wording, and 1 indicated that the item included qualified wording.

Mental comparisons

A value of 0 indicated that no mental comparisons were included in an item. A value of 1 indicated that an item included mental comparisons.

Negated wording

Negated wording included standard negations (e.g., “not,” “never”) as well as wording that reversed item directionality, such as “more limits” in “The U.S. should put more limits on who can own guns.” Zero indicated that the item did not include negation, while 1 indicated that negated wording was included.

Question Comprehension

Four characteristics were used to indicate question comprehension difficulty.

Unfamiliar terms

The presence of unfamiliar terms in each item was determined using the Question Understanding Aid (QUAID) software, which was developed by language and cognition experts to identify potential item comprehension problems (Graesser et al. 2006; http://quaid.cohmetrix.com/). Each item was entered individually into QUAID, which generated a report of terms rarely used in a large set of English language corpora. Since QUAID only accepts English text, this process was only conducted for the English items. Few items contained more than one unfamiliar term. Thus, the analytic variable was coded as 1 if an item included one or more unfamiliar terms and 0 if an item did not contain an unfamiliar term.

Ambiguous wording

QUAID was also used to count the number of vague, imprecise, or ambiguous terms in the English items. As the majority of items containing ambiguous terms only included a single ambiguous term, the analytic variable was coded such that 0 indicated no ambiguous terms and 1 indicated one or more ambiguous terms.

Item length

The number of words in each item stem was counted, yielding two variables per item: English item length and Spanish item length.

Polysyllabic words

The number of words with three or more syllables in each item stem was counted by native speakers of each language, resulting in two variables per item: the number of polysyllabic words in English and in Spanish.

Knowledge Accessibility

Using a consensus process, two authors assessed each item according to the likelihood that respondents would have believed they had the knowledge needed to answer each question. A rating of 1 indicated no knowledge accessibility and was strictly applied to a limited set of items that queried nonexistent concepts (e.g., “The Porino whales should be protected”) or purposefully used incomprehensible wording (e.g., “I trust social movements”). These items were impossible to answer correctly. A 2 indicated low knowledge accessibility, suggesting that respondents were unlikely to have had the knowledge needed to answer the question (e.g., “Vegetarianism is harmful to the environment”). A 3 indicated moderate knowledge accessibility, for which respondents were likely to believe they had access to the knowledge needed to form a response (e.g., “Global warming is a myth”). A 4 indicated high knowledge accessibility, meaning that most respondents would have had the knowledge needed to answer the question (e.g., “I feel that I have a number of good qualities”). Few no- or low-knowledge items were included in the survey, as pretesting suggested that such items were often noticeable and/or objectionable to respondents.

Number of Problem Characteristics

In addition to assessing each item characteristic independently, the cumulative influence of the number of problematic item characteristics was estimated. This variable was created by assigning a 1 to each item if it possessed each of the following characteristics: qualified wording, mental comparisons, negated wording, one or more unfamiliar terms, one or more ambiguous terms, and no or low knowledge accessibility. These values were then summed, yielding a range of 0–4. However, as only two items were categorized as having four problem characteristics, items with three and four problem characteristics were collapsed to produce a final problem characteristic variable with four levels ranging from 0 to 3.

Respondent Characteristics

Interview language was assessed during eligibility screening by computing the mean of participants’ responses to three items adapted from the National Latino and Asian American Study (Massachusetts General Hospital 2019). Participants were asked to confirm or disconfirm the calculated language preference, which determined interview language assignment. Respondent age, gender, education, income, race, ethnicity, and Latino subgroup were also assessed. Acculturation was measured using an adapted, 27-item version of the Acculturation Rating Scale for Mexican Americans-II (Cuellar et al. 1995). The acculturation scale contained a Latino orientation subscale that explored respondents’ Spanish use, Latino subgroup identification, affinity for their subgroup’s culture, and interaction with people from their subgroup (15 items; α = .84), and a NLW orientation subscale that queried respondents’ English use, identification as Anglo-American, affinity for Anglo-American culture, and interaction with NLWs (12 items; α = .90).

Data Analysis

All analyses were conducted using version 9.4 of the SAS® software program for Windows (copyright © 2013, SAS Institute Inc., Cary, NC). First, an item-level data set was created that contained the item characteristic values and acquiescence variables for each of the 90 heterogeneous items. As the data set was organized by items instead of respondents, respondent characteristics could not be added to this data set. To examine acquiescence for particular respondent characteristics, additional item-level acquiescence variables were created using data drawn only from respondents with selected respondent characteristics. Stratified analyses were then conducted within these respondent subgroups to examine acquiescence and item characteristics by respondent ethnicity (four subgroups), education (high school or less vs. more than high school), age (18–50 vs. 51–90 years), interview language (English vs. Spanish), and gender (male vs. female). The limited cell sizes prohibited simultaneous examination of more than one respondent characteristic (e.g., estimating acquiescence for 18- to 50-year-old Mexican Americans). All hypotheses were tested through general linear models regressing acquiescence on multiple item characteristics. Since acquiescence was assessed at the item and not the respondent level, no control variables were included.

Results

Summary statistics for respondent and item characteristics are presented in Tables 1 and 2. The majority of respondents was female and had less than a college degree. The proportion of acquiescent responses by ethnic group was 52% for NLWs, 62% for Mexican Americans and Puerto Ricans, and 64% for Cuban Americans (data not shown).

Table 1.

Respondent Characteristics.

Respondent Characteristics
Mean age in years (standard deviation [SD])	50.8 (18.0)
Gender (% female)	69.5
Education (%)
Less than 7th grade	9.5
7th–12th grade or equivalent	39.7
Some college or technical/vocational school	21.3
4-Year college degree	18.7
Graduate degree	10.8
Annual household income (%)
Less than $20,000	37.4
$20,000–$39,999	22.9
$40,000–$59,999	12.9
$60,000 or greater	26.8
Ethnicity (n)
Mexican American	100
Cuban American	100
Puerto Rican	101
Non-Latino white	99
Interview language (% Spanish)	51.3
Acculturation subscale means (Latinos only; SD)
Latino orientation subscale	2.5 (0.7)
Anglo orientation subscale	3.2 (0.5)

Note: n = 400.

Table 2.

Item Characteristics.

Item Characteristics
Number of items with qualified wording	39
Number of items with mental comparisons	14
Number of items with negated wording	23
Number of items with unfamiliar terms	37
Number of items with ambiguous terms	46
Number of items by knowledge accessibility:
No knowledge accessibility	7
Low knowledge accessibility	10
Moderate knowledge accessibility	29
High knowledge accessibility	44
Mean item length, English (standard deviation [SD]; range: 3–22)	10.7 (4.3)
Mean item length, Spanish (SD; range: 5–22)	11.7 (4.3)
Mean number of polysyllabic words, English (SD; range: 0–4)	1.1 (1.1)
Mean number of polysyllabic words, Spanish (SD; range: 0–9)	3.4 (1.8)
Number of problem characteristics
0 problem characteristics	5
1 problem characteristic	29
2 problem characteristics	32
3 problem characteristics	24

Note: n = 90 items.

Estimated associations between item characteristics and acquiescence by ethnic group are presented in Table 3. Item length and polysyllabic words were estimated separately by interview language (see Online Appendix B), as values for these variables differed between the English and Spanish items. Among the models shown in Table 3, negated wording was significantly associated with reduced acquiescence for three of the ethnic groups and marginally associated with lower acquiescence among Puerto Ricans (Hypothesis 1c). These effects were particularly strong among Cuban Americans and NLWs, although the overall p values for these two models were marginally significant. Ambiguous wording was significantly associated with reduced acquiescence among Mexican Americans and NLWs but only marginally associated with lower acquiescence among Cuban Americans and Puerto Ricans (Hypothesis 2b). Separate models (not shown) found no evidence of interactions between negated wording and ambiguous terms. High knowledge accessibility was only related to increased acquiescence among NLWs (Hypothesis 3). No other item characteristics had a significant influence on acquiescence among the four ethnic groups.

Table 3.

General Linear Regression Model Estimates of the Influence of Item Characteristics on Acquiescence by Ethnicity.

Dependent Variable: Mean Acquiescence	Mexican Americans (n = 100)	Cuban Americans (n = 100)	Puerto Ricans (n = 101)	Non-Latino Whites (n = 99)
Dependent Variable: Mean Acquiescence	B (SE)	B (SE)	B (SE)	B (SE)
Qualified wording	−.15 (.14)	−.14 (.15)	−.13 (.15)	−.14 (.15)
Mental comparisons	.04 (.19)	−.04 (.20)	.03 (.19)	−.06 (.20)
Negated wording	−.31 (.16)*	−.41 (.17)*	−.31 (.16)^#	−.43 (.17)*
Unfamiliar terms	.15 (.14)	.06 (.15)	.07 (.15)	.17 (.16)
Ambiguous terms	−.29 (.13)*	−.25 (.14)^#	−.25 (.14)^#	−.34 (.14)*
Knowledge accessibility (no knowledge accessibility = 0):
Low knowledge accessibility	−.15 (.31)	−.18 (.33)	−.02 (.32)	−.04 (.33)
Moderate knowledge accessibility	−.09 (.26)	−.02 (.27)	.03 (.26)	.28 (.28)
High knowledge accessibility	.25 (.25)	.23 (.27)	.39 (.26)	.57 (.27)*
R ²	.18	.15	.15	.21
Model p value	.04	.08	.09	.01

Note: n = 90 items.

^# p < .10.

* p < .05.

** p < .01.

*** p < .001.

****p < .0001.

As shown in Online Appendix B, the results of additional models examining the influences of respondent and item characteristics on item-level acquiescence were consistent with those reported in Table 3. These models indicate that negated wording (Hypothesis 1c) was significantly associated with lower acquiescence among the whole sample, respondents with both less and more than a high school education, respondents aged 18–50 and 51–90, respondents who completed the interview in English, and both male and female respondents. Negated wording was marginally associated with lower acquiescence among respondents who completed the interview in Spanish. Due to a moderate correlation between item length and polysyllable words in Spanish (r = .52), only item length was included in the Spanish model shown in Online Appendix B to avoid introducing collinearity. However, a separate Spanish model replacing item length with the number of polysyllable words (not shown) yielded the same pattern of results. Ambiguous wording (Hypothesis 2b) was significantly associated with lower acquiescence among the whole sample, respondents with more than a high school education, respondents aged 18–50 and 51–90, respondents who completed the interview in English, and both male and female respondents. Ambiguous wording was marginally associated with reduced acquiescence among respondents with a high school education or less. Models testing interactions (not shown) found no evidence of interactions between negated wording and ambiguous terms. No other item characteristics had a significant influence on acquiescence.

Results presented in Online Appendix B also indicate that the cumulative influence of the number of problem characteristics was inversely associated with acquiescence. As compared to 0 problem characteristics, acquiescence significantly decreased for items with 1 problem characteristic, 2 problem characteristics, and 3–4 problem characteristics. Separate models (not shown) including item length and the number of polysyllabic words indicated that these variables were not significant.

Discussion

Although results from this study do not identify item characteristics that increase acquiescence, they do suggest that two item characteristics—negated wording (Hypothesis 1c) and ambiguous terms (Hypothesis 2b)—reduce acquiescence, particularly among social groups with a lower tendency to acquiesce. Specifically, these results indicate a slightly stronger pattern of reduced acquiescence among younger, more educated, English-speaking (negated wording only), and NLW respondents. Acquiescence also decreased as the number of problem item characteristics increased, though this analysis was limited by the small number of items with 0 problem characteristics. Together, these findings provide evidence that respondent and item characteristics interact, such that items containing negated wording, ambiguous terms, or different total numbers of problem characteristics may yield slightly different responses across respondents. As the sample size of the current study was limited, additional research with larger sample sizes is needed to examine the combined influence of multiple respondent and item characteristics on acquiescence.

Expanding on work by Gilbert (1991), Knowles and Condon (1999) proposed that when confronting negated wording, respondents first comprehend items in the affirmative direction and then reintroduce any negations during a subsequent reconsideration stage. If true, then questions containing negations would require greater overall processing effort, and acquiescence may ensue when respondents are unable or unwilling to invest additional cognitive effort and instead opt for lower-effort, affirmative responses (Knowles and Condon 1999). Our observation of an inverse relationship between negated wording and acquiescence, however, fails to support this conceptualization. It is possible that negated wording did not increase cognitive load. Some evidence for this explanation was found in our study in that the pattern of results for negated wording was distinctly different from those found for two other variables used to indicate increased cognitive load, qualified wording (Hypothesis 1a), and mental comparisons (Hypothesis 1b), which had no influence on acquiescence. However, it is also possible that these variables were poor indicators of cognitive load.

An alternative explanation is that items with negated wording caught respondents’ attention (Podsakoff et al. 2003), leading respondents to invest more cognitive effort in responding to the negated items than they did to nonnegated items. Consistent with findings by Lenzner and colleagues (2010), another possibility is that negated wording increased cognitive load. But, instead of choosing affirmative responses, respondents selected negative responses when they did not invest sufficient cognitive effort to process the negations because they did not want to agree with a negatively valenced statement (due to a bias toward positivity), the negations became more salient during the item reconsideration phase, and/or negative responses were superficially consistent with the negative direction of the items. These lower-effort explanations are consistent with findings from Kamoen et al. (2017) who observed that negated wording was associated with increased response processing time, more frequent question rereading, and more disagreement (vs. agreement) responses.

The finding from this study that respondents were less likely to acquiesce when items contained ambiguous wording was contrary to our expectations. One possible explanation is that survey respondents were reluctant to agree with items that they did not understand. However, this explanation was not supported by the findings on knowledge accessibility, which, with the exception of high knowledge accessibility among NLWs, had no influence on acquiescence. Alternatively, ambiguous wording may convey that respondents’ answers should be similarly tempered by ambiguity. For example, the QUAID program coded words such as “much,” “more,” and “sometimes” as ambiguous in items used in this study. These words introduce uncertainty into the meaning of the items, which, in turn, may have made respondents less likely to commit to answers that conveyed definite agreement (acquiescence). This pattern may have been stronger among more educated respondents because they more easily identified the nuances introduced by ambiguous terms than respondents with less education.

We found little to any support for the influence of additional item characteristics on acquiescence (Hypotheses 2a, 2c, 2d, and 3) for the full study sample or among subsamples stratified by ethnicity, education, age, interview language, or gender. Some of the item characteristics were assessed using coder ratings, which may have been substantially different than ratings that would have been obtained from respondents. It is also possible that our findings were confounded by the relatively long survey duration, which may have increased acquiescence when items were located closer to the end of the questionnaire. Since the items were randomly ordered, this duration effect should have been distributed relatively evenly across items. However, the overall prevalence of acquiescence may have been higher than would have been observed in a shorter survey. There were few items in the “no” and “low” knowledge accessibility categories. We have found such items difficult to include in surveys. Respondents often notice when asked about nonexistent concepts, which can diminish trust and reduce respondents’ willingness to provide thoughtful responses.

This research adds to a limited body of knowledge about the influence of item characteristics on acquiescence. Both negated and ambiguous wording were found to decrease acquiescence, possibly due to the increased effort and uncertainty that they introduce into the response process. Findings from this study therefore concur with previous studies that negated and ambiguous wording should be avoided in questionnaire items, even if they artificially reduce acquiescence. Findings from this study also suggest that the prevailing belief that acquiescence is immune to item content or directionality merits further scrutiny. More research is needed to examine additional item characteristics as well as interactions between respondent and item characteristics. For example, this study did not examine whether social desirability interacts with acquiescence. Social desirability may be associated with respondent (e.g., individuals and social groups have differing standards for what is considered socially desirable) and item characteristics (e.g., item content influences social desirability). It is also possible that social desirability trumps acquiescent tendencies when respondent perceptions and/or item content maximize the urge to engage in socially desirable responding. Unlike acquiescence, social desirability can result in negative or affirmative responses. Further investigation of the underlying roots of acquiescence and other response styles will enhance our ability to explain differences across social groups and improve our capacity to obtain valid and comparable data across sociodemographically diverse social groups.

Supplemental Material

Supplemental Material, davis.etal.appendix1.rev - The Influence of Item Characteristics on Acquiescence among Latino Survey Respondents

Supplemental Material, davis.etal.appendix1.rev for The Influence of Item Characteristics on Acquiescence among Latino Survey Respondents by Rachel E. Davis, Sunghee Lee, Timothy P. Johnson, Frederick Conrad, Ken Resnicow, James F. Thrasher, Anna Mesa and Karen E. Peterson in Field Methods

Supplemental Material

Supplemental Material, davis.etal.appendix2.rev - The Influence of Item Characteristics on Acquiescence among Latino Survey Respondents

Supplemental Material, davis.etal.appendix2.rev for The Influence of Item Characteristics on Acquiescence among Latino Survey Respondents by Rachel E. Davis, Sunghee Lee, Timothy P. Johnson, Frederick Conrad, Ken Resnicow, James F. Thrasher, Anna Mesa and Karen E. Peterson in Field Methods

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Funding for this research was provided by the National Cancer Institute (R01 CA172283).

Supplemental Material

Supplemental material for this article is available online.

Note

References

Aday

L. A.

Chiu

G. Y.

Andersen

. 1980. Methodological issues in health care surveys of the Spanish heritage population. American Journal of Public Health 70:367–74.

American Association for Public Opinion Research. 2016. Standard definitions: Final dispositions of case codes and outcome rates for surveys. https://www.aapor.org/Standards-Ethics/Standard-Definitions-(1).aspx (accessed September 1, 2016).

Andrews

F. M.

1984. Construct validity and error components of survey measures: A structural modeling approach. Public Opinion Quarterly 48:409–42.

Baumgartner

Steenkamp

J.-B. E. M.

. 2001. Response styles in marketing research: A cross-national investigation. Journal of Marketing Research 38:143–56.

Billiet

J. B.

Davidov

. 2008. Testing the stability of an acquiescence style factor behind two interrelated substantive variables in a panel design. Sociological Methods & Research 36:542–62.

Billiet

J. B.

McClendon

M. J.

. 2000. Modeling acquiescence in measurement models for two balanced sets of items. Structural Equation Modeling 7:608–28.

Cheung

G. W.

Rensvold

R. B.

. 2000. Assessing extreme and acquiescence response sets in cross-cultural research using structural equations modeling. Journal of Cross-cultural Psychology 31:187–212.

Converse

J. M.

Presser

. 1986. Survey questions: Handcrafting the standardized questionnaire. Beverly Hills, CA: Sage.

Cuellar

Arnold

Maldonado

. 1995. Acculturation Rating Scale for Mexican Americans-II: A revision of the original ARSMA scale. Hispanic Journal of Behavioral Sciences 17:275–304.

10.

Davis

R. E.

Resnicow

Couper

M. P.

. 2011. Survey response styles, acculturation, and culture among a sample of Mexican American adults. Journal of Cross-cultural Psychology 42:1219–36.

11.

De Beuckelaer

Weijters

Rutten

. 2010. Using ad hoc measures for response styles: A cautionary note. Quality & Quantity 44:761–75.

12.

Ennis

S. R.

Rios-Vargas

Albert

N. G.

. 2011. The Hispanic population: 2010. In 2010 Census Briefs. Washington, DC: U.S. Census Bureau. https://www.census.gov/prod/cen2010/briefs/c2010br-04.pdf (accessed September 24, 2016).

13.

Flores

2017. How the U.S. Hispanic population is changing. Pew Research Center. http://www.pewresearch.org/fact-tank/2017/09/18/how-the-u-s-hispanic-population-is-changing/ (accessed January 5, 2018).

14.

Fowler

F. J.

1992. How unclear terms affect survey data. Public Opinion Quarterly 56:218–31.

15.

Fowler

F. J.

1995. Improving survey questions: Design and evaluation. Thousand Oaks, CA: Sage.

16.

Gilbert

D. T.

1991. How mental systems believe. American Psychologist 46:107–19.

17.

Graesser

A. C.

Cai

Louwerse

M. M.

Daniel

. 2006. Question Understanding Aid (QUAID): A web facility that tests question comprehensibility. Public Opinion Quarterly 70:3–22.

18.

Greenleaf

E. A.

1992a. Improving rating scale measures by detecting and correcting bias components in some response styles. Journal of Marketing Research 29:176–88.

19.

Greenleaf

E. A.

1992b. Measuring extreme response style. Public Opinion Quarterly 56:328–51.

20.

Hoffman

Mai

Cristescu

. 2013. Do culture-dependent response styles distort substantial relationships? International Business Review 22:814–27.

21.

Holbrook

A. L.

Johnson

T. P.

Ik Cho

Shavitt

Chavez

Weiner

. 2016. Do interviewer errors help explain the impact of question characteristics on respondent difficulties? Special issue, Survey Practice 9. doi: 10.29115/SP-2016-0009.

22.

Johnson

T. P.

Ik Cho

Holbrook

A. L.

O’Rourke

Warnecke

R. B.

Chávez

. 2006. Cultural variability in the effects of question design features on respondent comprehension of health surveys. Annals of Epidemiology 16:661–68.

23.

Johnson

T. P.

Shariff-Marco

Willis

Ik Cho

Breen

Gee

G. C.

Krieger

, et al. 2015. Sources of interactional problems in a survey of racial/ethnic discrimination. International Journal of Public Opinion Research 27:244–63.

24.

Jordan

L. A.

Marcus

A. C.

Reeder

L. G.

. 1980. Response styles in telephone and household interviewing: A field experiment. Public Opinion Quarterly 44:210–22.

25.

Kam

C. C. S.

Zhou

. 2015. Does acquiescence affect individual items consistently? Educational and Psychological Measurement 75:764–84.

26.

Kamoen

Holleman

Mak

Sanders

van den Bergh

. 2011. Agree or disagree? Cognitive processes in answering contrastive survey questions. Discourse Processes 48:355–85.

27.

Kamoen

Holleman

Mak

Sanders

van den Bergh

. 2017. Why are negative questions difficult to answer? On the processing of linguistic contrasts in surveys. Public Opinion Quarterly 81:613–35.

28.

Kincaid

J. P.

Fishburne

R. P.

Jr Rogers

R. L.

Chissom

B. S.

. 1975. Derivation of new readability formulas (automated readability index, fog count and Flesch reading ease formula) for navy enlisted personnel. Millington, TN: U.S. Department of Defense.

29.

Knowles

E. S.

Condon

C. A.

. 1999. Why people say “yes”: A dual-process theory of acquiescence. Journal of Personality and Social Psychology 77:379–86.

30.

Krosnick

J. A.

1991. Response strategies for coping with the cognitive demands of attitude measures in surveys. Applied Cognitive Psychology 5:213–36.

31.

Krosnick

J. A.

1999. Survey research. Annual Review of Psychology 50:537–67.

32.

Laurent

1972. The effects of question length on reporting behavior in the survey interview. Journal of the American Statistical Association 67:298–305.

33.

Lavrakas

P. J.

2008. Within-household respondent selection: How best to reduce total survey error? Media Rating Council. http://www.mediaratingcouncil.org/MRC%20Point%20of%20View%20-%20Within%20HH%20Respondent%20Selection%20Methods.pdf (accessed October 13, 2015).

34.

Lechner

C. M.

Rammstedt

. 2015. Cognitive ability, acquiescence, and the structure of personality in a sample of older adults. Psychological Assessment 27:1301–11.

35.

Lenzner

Kaczmirek

Galesic

. 2011. Seeing through the eyes of the respondent: An eye-tracking study on survey question comprehension. International Journal of Public Opinion Research 23:361–73.

36.

Lenzner

Kaczmirek

Lenzner

. 2010. Cognitive burden of survey questions and response times: A psycholinguistic experiment. Journal of Social Psychology 13:1003–20.

37.

Marín

Gamba

R. J.

Marín

B. V.

. 1992. Extreme response style and acquiescence among Hispanics: The role of acculturation and education. Journal of Cross-cultural Psychology 23:498–509.

38.

Massachusetts General Hospital, Department of Medicine, Disparities Research Unit. 2019. National Latino and Asian American Study documents. https://www.massgeneral.org/disparitiesresearch/Research/pastresearch/NLAAS-documents.aspx (accessed August 2, 2019).

39.

McLaughlin

G. H.

1969. SMOG grading—A new readability formula. Journal of Reading 12:639–46.

40.

Meisenberg

Williams

. 2008. Are acquiescent and extreme response styles related to low intelligence and education? Personality and Individual Differences 44:1539–50.

41.

Narayan

Krosnick

J. A.

. 1996. Education moderates some response effects in attitude measurement. Public Opinion Quarterly 60:58–88.

42.

Podsakoff

P. M.

MacKenzie

S. B.

Lee

J.-Y.

Podsakoff

N. P.

. 2003. Common method biases in behavioral research: A critical review of the literature and recommended remedies. Journal of Applied Psychology 88:879–903.

43.

Ross

C. E.

Mirowsky

. 1984. Socially-desirable response and acquiescence in a cross-cultural survey of mental health. Journal of Health and Social Behavior 25:189–97.

44.

Tourangeau

Rips

L. J.

Rasinski

. 2000. The psychology of survey response. New York: Cambridge University Press.

45.

Warnecke

R. B.

Johnson

T. P.

Chávez

Sudman

O’Rourke

D. P.

Lacey

Horm

. 1997. Improving question wording in surveys of culturally diverse populations. Annals of Epidemiology 7:334–42.

46.

Weijters

Baumgartner

. 2012. Misresponse to reversed and negated items in surveys: A review. Journal of Marketing Research 49:737–47.

47.

Weijters

Geuens

Schillewaert

. 2010. The stability of individual response styles. Psychological Methods 15:96–110.

48.

Weijters

Schillewaert

Geuens

. 2008. Assessing response styles across modes of data collection. Journal of the Academy of Marketing Science 36:409–22.

49.

Yan

Tourangeau

. 2008. Fast times and easy questions: The effects of age, experience and question complexity on web survey response times. Applied Cognitive Psychology 22:51–68.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.11 MB

0.16 MB