The Illusion of Political Tolerance: Social Desirability and Self-Reported Voting Preferences

Abstract

Tolerance for diversity in America is often indexed by direct measures, such as self-reported “willingness to vote” polls. However, pressures to be or appear unprejudiced may bias such estimates, yielding misleading and overly optimistic inferences about tolerance in America. The current research investigated the degree to which direct and indirect measures of political candidate preferences converge and diverge across six target groups varying in acceptability of stigmatization (atheists, African Americans, Catholics, gay men and lesbians, Muslims, and women) and across relevant participant demographics. Overall, participants (N = 3,000, nationally representative) reported less willingness to vote for target groups when measured indirectly, relative to directly. Additionally, the divergence between the direct and indirect measures was especially evident for social groups for which overt stigmatization is normatively inappropriate. This research provides a vital benchmark that quantifies the gulf between the direct and indirect measures of tolerance for various oft-stigmatized groups in America.

Keywords

social desirability stigma stereotyping and prejudice politics voting

All the world’s a stage,

And all the men and women merely players
—Shakespeare, As You Like It, II, VII Are Americans merely good at playing the role of an egalitarian or are Americans genuinely tolerant and accepting of diversity? Over time, surveys have consistently shown that Americans’ self-reported attitudes toward historically stigmatized groups (such as African Americans) are becoming more positive and inclusive over time (e.g., Bergsieker, Leslie, Constantine, & Fiske, 2012). However, the question remains whether these increasingly positive attitudes reflect people’s increasingly positive beliefs toward stigmatized groups or whether people are simply becoming better at hiding their prejudices.

Tolerance for diversity in America is often indexed by direct measures, such as self-reported “willingness to vote” polls. In these polls, people are asked whether they would vote for a qualified candidate who happened to be, for example, a woman, Black, or Muslim. People’s willingness to vote for these candidates can serve as a barometer for cultural inclusion (or exclusion) of various groups. That is, the extent to which people are willing to vote for a qualified candidate from a marginalized group reflects the extent to which people are accepting of that marginalized group generally (e.g., Payne et al., 2010). Such barometers tend to paint a picture of steadily increasing tolerance for most groups in American society over the previous several decades.

Although the willingness to vote polls do not directly relate to actual political contests, recent elections have raised interested debates over whether factors such as a candidate’s race, gender, and religion should inform voting. For example, some popular media headlines suggest the acceptability of voting (or not) for a candidate based on the candidate’s gender: “…It is OK to care about gender on the ballot” (Valenti, 2016), whereas other headlines argue it is not appropriate: “The immaturity of supporting [a candidate] because she’s a woman” (Prager, 2016). As a result of this debate, people may fear that honestly reporting their opinions in a “willingness to vote” poll may inadvertently make them appear prejudiced or disclose socially unacceptable prejudice against certain groups. Although some individuals may be motivated to express prejudice (e.g., Devine, Monteith, Zuwerink, & Elliot, 1991; Kinder, & Sears, 1981; Monteith, Devine, & Zuwerink, 1993), a fear of appearing prejudiced makes it difficult to determine whether self-reported voting preferences accurately reflect people’s tolerance for diversity in America or whether people are obscuring their attitudes from pollsters to avoid judgment.

Certain situations may assuage or amplify the fear of appearing prejudiced. Public, accountable behavior may amplify concerns of appearing prejudiced, whereas private, anonymous behavior may assuage it (e.g., Crandall & Eshleman, 2003). National “willingness to vote” polls often use methods which enhance public accountability. For example, Pew and Gallup, two prominent polling agencies, often call households and ask: “If your party nominated a generally well-qualified person for president who happened to be African American, would you vote for that person?” Because the hypothetical candidate is otherwise qualified, deciding not to vote for this candidate is thought to reveal the individual’s prejudice against African Americans. In this highly personal and public context, people may feel unwilling to report their voting preferences for fear of appearing prejudiced to the pollster. As a result, polls such as these may drastically overestimate tolerance in the United States.

In some situations, however, expressing honest opinions about certain political candidates may not evoke concerns of appearing prejudiced. This probably occurs when the candidate is not from a marginalized social group (e.g., White men) or when the candidate is from a social group which is deemed socially appropriate to stigmatize (e.g., atheists: Gervais, 2014; or Muslims: Kteily, Hodson, & Bruneau, 2016). Concerns of appearing prejudiced may bias self-reports of tolerance for marginalized social groups which are deemed socially inappropriate to stigmatize (Crandall, Eshleman, & O’Brien, 2002), such as women (e.g., Glick & Fiske, 1996) and African Americans (e.g., Plant & Devine, 1998). Therefore, people may be particularly motivated to deceive pollsters when their voting preferences reveal prejudice against a group which is normatively deemed socially inappropriate to stigmatize, but people may more accurately report their voting preference for a group against which they are less concerned about appearing prejudiced.

One way to circumvent social desirability concerns is through the use of indirect measures. By indirect measures, we mean any measure that avoids direct self-reporting of attitudes and, therefore, alleviates social desirability concerns. Indirect polling techniques may mitigate social desirability concerns and provide less biased estimates of voting preferences for socially marginalized candidates. Therefore, indirect measures of voting for a qualified candidate from a particular social group may serve as a metanarrative about cultural inclusion for this social group in contemporary American society.

The current research investigated the degree to which direct and indirect measures of tolerance for diversity converge and diverge across a number of target groups varying in acceptability of stigmatization. We investigated this question in the context of voting for a political candidate. For the direct measure, we used the standard “willingness to vote” measure described above and used by Pew and Gallup. For the indirect measure, we used the unmatched count technique (UCT: Dalton, Wimbush, & Daily, 1994; Raghavarao & Federer, 1979). The UCT relies on the comparison of base rates of responses across two conditions. In the baseline condition, respondents see a list of innocuous descriptors (e.g., “I own a dog,” “I know how to ride a motorcycle,” “I enjoy pineapple on pizza”) and tally up the number that applies to them. For instance, a respondent might own a dog, not know how to ride a motorcycle, and enjoy pineapple on their pizza. Therefore, this respondent’s tally would be two. In the sensitive condition, the respondents see the same list of descriptors plus one sensitive item—the item of interest (e.g., “I have cheated on my significant other”). In this condition, the tally would depend on whether the added descriptor—the sensitive item of interest—was true of the respondent. For instance, the respondent might own a dog, not know how to ride a motorcycle, enjoy pineapple pizzas, and cheated on his spouse, so the tally would be three. If the respondent had not cheated on his spouse, the tally would be two instead. The advantage of using the UCT as the indirect measure is that respondents are not listing which items are true of them, but rather the total number of items, allowing respondents to avoid social desirability concerns.

To estimate the prevalence of a given behavior, researchers compare the average tally given by respondents in each condition. Given that the lists are identical except for the added item of interest, and assuming the groups are comparable due to random assignment, one can infer that the difference between the two conditions is, on average, attributable to the addition of the sensitive item (Dalton et al., 1994). For example, if the sensitive condition had a mean .23 higher than the control condition, this suggests that approximately 23% of people in the population have cheated on a significant other; a mean difference of .04 would suggest that only 4% of the population was unfaithful. Additionally, researchers can compare the difference between the two UCT conditions and the direct measure to estimate the willingness to disclose sensitive behaviors. For example, if the indirect estimate is 24%, and the direct estimate is 12%, this difference suggests that half of the people who have cheated on their spouse are unwilling to openly report this indiscretion. As a result of using both the direct and indirect measures, we can obtain an estimate of the proportion of people who endorsed the sensitive item as well as an estimate of the proportion of people who are not willing to directly self-report a given behavior.

Using both the direct and indirect approach described above can help researchers determine whether or not items are socially sensitive. For innocuous items, we would not expect such a method to elicit differences between conditions. In other words, when the added item is not socially sensitive, indirect and self-report prevalence estimates should converge; when the added item is socially sensitive, indirect and self-report estimates should diverge. Moreover, if the added item is socially undesirable (e.g., “I cheated on my spouse”), self-report estimates will tend to be lower than indirect estimates; if the item is socially desirable (e.g., “I volunteer on a regular basis,” “I donate to disaster relief funds”), self-report estimates will likely be higher than indirect estimates. Thus, UCT can be used to determine whether or not social desirability is influencing responding and to determine and the prevalence of socially sensitive attitudes and behaviors, regardless of whether those attitudes and behaviors are socially desirable or undesirable, and can be an effective measure for situations when self-reports can prove to be unreliable (e.g., Dalton et al., 1994).

Present Study

We sought to explore the degree to which self-reported tolerance for various stigmatized groups of people is overstated. To do so, we used the UCT to estimate rates of political tolerance for various groups of people and compared this indirect estimate to direct self-reported tolerance for the same groups using the standard “willingness to vote” measure. We hypothesized that the direct and indirect measures would diverge sharply for social groups for which overt stigmatization is normatively inappropriate. We used a nationally representative sample (N = 3,000) and measured political candidate preferences for candidates who represent a wide range of social groups—atheist, Black, Catholic, female, homosexual, and Muslim. Thus, this research provides an important benchmark that quantifies, based on a representative sample, the gulf between the direct and indirect measures of cultural inclusion of various social groups simultaneously.

Method

Using a nationally representative sample of 3,000 U.S. adults, we administered both the direct and indirect measures of voting preferences for six target candidates. In the direct condition, participants were asked whether they would support an otherwise qualified candidate for president from their political party if that candidate happened to be from the following stigmatized social groups: atheist, Black, Catholic, female, homosexual, or Muslim. Question wording mirrored the questions used in Gallup polls to measure political preferences as a proxy for broad cultural inclusion and exclusion of different groups. Indirect measurements relied on the UCT (Dalton et al., 1994; Raghavarao & Federer, 1979) to infer base rates of inclusion of political candidates from the same stigmatized social groups. We were, therefore, able to quantify both self-reported and indirectly measured political tolerance for six target groups, as well as the divergence between self-reports and indirect measurements.

Participants

We contracted a large, nationally representative sample (N = 3,000) of American adults from YouGov, a firm that specializes in nationally representative political polling. For descriptive demographic information, see Table 1. Data were collected in June 2016 from U.S. residents.

Table 1.
Percent Breakdowns of Sample Demographics.

Demographic Group Self-Report Indirect Overall

Gender

Female 55.3 54.9 55.0

Male 44.7 45.2 45.0

Race

White 69.0 70.6 70.8

Black 12.4 11.0 11.6

Hispanic 12.2 12.4 11.5

Asian 2.8 2.0 2.3

Mixed 1.4 1.5 1.5

Other 2.2 2.5 2.4

Political ideology

Democrat 38.5 38.9 38.8

Independent 28.5 26.1 26.9

Republican 24.5 24.1 24.2

Other/unsure 8.5 10.9 10.1

Religion

Christian 55.4 55.3 55.4

Atheist/agnostic 13.4 10.7 12.5

Other 31.2 34.1 32.2

Education

Max high school 42.5 41.5 41.8

Beyond high school 57.5 41.5 58.2

Age

Mean 47.7 47.7 47.7

Range 19–90 19–91 19–91

Design

In a between-subjects design, participants were randomly assigned to one of three conditions (for experimental materials, see https://goo.gl/Tl4g7q). The first condition was a traditional self-report of willingness to vote for candidates from a list of target groups of interest. The other two conditions combined to form the indirect measure. YouGov ensured that no respondents ended up in more than one condition. All measures are reported below.

Procedure

To directly measure willingness to vote for different targets, we asked respondents to consider their willingness to vote for a candidate who, holding all else constant, was a member of a given target group: atheist, Black, Catholic, female, homosexual, and Muslim. Phrasing for the direct measure was taken straight from standard Gallup polling language and included target groups traditionally included in Gallup polls (Gallup, 2015). To indirectly measure willingness to vote for different targets, we used the UCT (Dalton et al., 1994; Raghavarao & Federer, 1979). This technique has been used to circumvent socially desirable responding in attaining population prevalence estimates for unacceptable or illegal behaviors, such as risky sexual behavior and alcohol use (LaBrie & Earleywine, 2000), poaching (Nuno, Bunnefeld, Naiman, & Milner-Gulland, 2013), antigay hate crimes (Rayburn, Earleywine, & Davison, 2003), atheism in the United States (Gervais & Najle, 2018), and more (Coutts & Jann, 2011). Furthermore, these counts tend to only show divergence from self-reports of socially undesirable traits (Coffman, Coffman, & Ericson, 2016).

Participants in the indirect conditions were randomly assigned to see either the baseline list or the socially sensitive list. In the baseline list condition, participants were given a list of five candidate descriptors. These descriptors were chosen so each list included one descriptor unlikely to be endorsed (e.g., bankrupt), one descriptor likely to be endorsed (e.g., a small business owner), and three generic descriptors (e.g., a coffee drinker). In the sensitive list condition, participants were given the same list of five candidate descriptors that the control group saw and one target group descriptor—atheist, Muslim, Catholic, Black, homosexual, or female. The difference between the baseline and target group conditions can, on average, be attributed to the inclusion of the socially sensitive item, yielding an indirect base rate for the behavior while mitigating socially desirable responding. That is, we indirectly infer willingness to vote in the unmatched count by assessing the mean difference in scores between participants in the two UCT conditions. Then, the indirect base rate from the UCT can be compared to the base rates acquired through the self-report. The format and wording for the self-report and an example of the UCT are given in Table 2. All materials are available at https://goo.gl/Tl4g7q.

Table 2.
Willingness to Vote for a Given Candidate.^a

UCT Baseline UCT Target Groups Self-Report Items

Imagine the five different people listed below. Now, separately consider the following statement for each of those five people:

“If my party nominated a generally well-qualified person for president who happened to be , I would vote for that person.”

A small business owner

An avid cyclist

From a rich family

A coffee drinker

Bankrupt

Simply count how many of those different people you would vote for.

In the space below, please write how many of these potential candidates (from 0 to 5) you would vote for:

of these candidates

Imagine the six different people listed below. Now, separately consider the following statement for each of those six people:

“If my party nominated a generally well-qualified person for president who happened to be , I would vote for that person.”

A small business owner

An avid cyclist

From a rich family

A coffee drinker

Bankrupt

Catholic

Simply count how many of those different people you would vote for.

In the space below, please write your total count of how many of these potential candidates (from 0 to 6) you would vote for:

____ of these candidates

We are interested in your political attitudes. Please answer “yes” or “no” to the following questions.

If your party nominated a generally well-qualified person for president who happened to be _, would you vote for that person?

Black (Y/N)

A woman (Y/N)

Gay or lesbian (Y/N)

Muslim (Y/N)

An atheist (someone who does not believe in God) (Y/N)

Catholic (Y/N)

A B C

Note. N = no; UCT = unmatched count technique; Y = yes.

^aA and B show an example of the unmatched count technique. Each participant in the indirect sample completed either the baseline or target group version for six different lists, with target groups having one list for each target in the self-report. C. Participants in the self-report sample indicated yes or no for each of the six targets.

Participants in both the baseline and sensitive conditions completed six trials of the UCT. To avoid order effects, each trial was displayed in a random order. Phrasing for the UCT trials was developed using standard UCT procedures (Dalton et al., 1994). On each trial of this task, participants were given the following question stem: “If my party nominated a generally well-qualified person for president who happened to be ____, I would vote for that person.” Participants were then given a list of candidate descriptors and asked how many—but not which—of these potential candidates they would vote for. The candidate descriptors were also displayed in a random order to avoid order effects. No participant had to admit (un)willingness to vote for any specific candidate.

Demographic Information

Although we did not explicitly include demographic questions in our materials, YouGov collects basic demographic information from participants and makes this information available to clients. For exploratory purposes, we investigated several demographic contrasts. In particular, we conducted demographic contrast analyses based on gender, age, ethnicity, political affiliation, religion, and education. Some participants may be more unskilled and/or unmotivated to conceal prejudice than other participants, which would result in convergence between the direct and indirect measures (see Crandall & Eshleman, 2003 for a review). We only had one specific hypothesis regarding these contrast analyses: We predicted the greatest divergence between the direct and indirect measures of candidate preferences among those who are highly educated because they are highly cognitively skilled at suppressing prejudice (e.g., Allport & Kramer, 1946; Meertens & Pettigrew, 1997). All other analyses were conducted sans specific hypotheses and thus should be interpreted accordingly.

Results

Analytic Approach

Data and analysis code are publicly available at https://goo.gl/Tl4g7q. We sought to quantify both self-reported and indirectly measured political support for members of various social groups. Our analyses relied on Bayesian hierarchical modeling using the “Rethinking” package in R (McElreath, 2016; R Core Team, 2017). Bayesian approaches provide researchers a number of pragmatic benefits, including the use of intuitive probabilistic statements regarding the relative credibility or plausibility of different potential parameter values (Kruschke, 2010, Chap. 2; Wagenmakers, Morey, & Lee, 2016). In addition, hierarchical (e.g., multilevel) models can mitigate some problems associated with multiple comparisons (Gelman, Hill, & Yajima, 2012)—comparisons that could be especially concerning in the present study, which evaluated support for six different targets across 15 separate demographic comparisons (see Westfall, Van Boven, Chambers, & Judd, 2015 for a similar multilevel analytic approach, given similarly structured data). This means that every comparison was analyzed simultaneously, while controlling for the other variables in the model.

Posterior distributions are the primary inferences drawn in our analyses. Posterior distributions represent the plausibility of different potential parameter values, contingent on the data and the model. The posterior distribution can be summarized in a variety of different ways. In our results, we describe our inferences in terms of a point estimate and associated estimate uncertainty. The posterior mean represents our inference regarding a most plausible parameter point value. Posterior density intervals reflect the range in which the most plausible parameter values lie. We describe estimate uncertainty using the 97% highest posterior density intervals (HPDIs), which represent the range in which the 97% most credible parameter values lie. Put differently, we can be 97% certain that the parameter value lies within a provided HPDI, given our data and model. This approach is in contrast to frequentist confidence intervals, which merely provide a range of values that will contain the true parameter value with known long-term error rates across repeated tests, although frequentist confidence intervals are often intuitively misunderstood as if they had the properties of Bayesian posterior density intervals (Hoekstra, Morey, Rouder, & Wagenmakers, 2014). Finally, we describe inferences regarding differences between the direct and indirect measurements using posterior probabilities of differences. This approach evaluates the probability that a given parameter is greater than a given value, contingent on the data and the model. For example, we can provide the posterior probability that self-reported tolerance is higher than indirectly measured tolerance for a given group. In some ways, this is analogous to the way that people often intuitively misinterpret frequentist p values (Oakes, 1986).

We conducted two separate multilevel models: one for self-reports and one for indirect measures. Both models evaluated political support for each of the six targets, adjusting for individual differences in gender, ethnicity, education, political affiliation, age, and religious affiliation. Both indirect and self-report analyses modeled both random slopes and intercepts across levels (with electoral target group nested within participant). The self-report inferences relied only on data from participants in the self-report condition and used a hierarchical model with a binary (logit) outcome variable as a direct measure of willingness to vote for different candidates. In contrast, the indirect measure stems from a comparison of participants randomly assigned to the two UCTs. This required a hierarchical linear model evaluating the differences in means between the two conditions: Willingness to vote for a candidate was indirectly inferred from the mean differences between those participants receiving either 5 or 6 potential candidates (assigned between conditions). Comparison of the two posterior distributions yielded posterior probabilities of differences between the two estimates. Analysis code, raw data, and samples extracted from the posterior distribution are available at https://goo.gl/Tl4g7q.

Confirmatory Overall Analysis

In our study preregistration (https://osf.io/whsae/), we predicted that self-reports would show higher tolerance than indirect measures for Black, gay, Muslim, and female candidates; little difference between measures for Catholic candidates; and potentially higher support for atheists on indirect measures (while also pointing out that different conceptual frameworks make very different predictions for atheist candidates). However, our preregistration also made it clear that “our primary goal is quantifying divergences [between self-reports and indirect measures], rather than promoting our mystical ability to accurately predict them.”

Our overall analyses revealed strong evidence that people self-report greater tolerance for all six targets, relative to indirect measures (see Figure 1 and Table 3). The largest discrepancies between the self-report measure and indirect measure of voting occurred when the target candidate was Black or female. For example, on self-reports, only 8% of people would not vote for Black or female candidates. Indirect point estimates were more than 3 times as large, with 30% and 27% of people being unwilling to vote for Black and female candidates, respectively. The smallest discrepancies were observed for the two least accepted groups, atheists and Muslims. Even though there was still reasonably strong evidence that self-reports inflate tolerance for atheists and Muslims, people were apparently largely comfortable voicing their political disapproval for these openly reviled religious outgroups.

Figure 1.
Indirect measurement reveals markedly lower political acceptance of six groups, relative to self-reports. Point estimates reflect posterior means, and error bars reflect 97% highest posterior density intervals.

Table 3.
Self-Report and Indirect Measure Parameter Estimates of Political Acceptance of Six Groups.^a

Target Group Self-Report Indirect Pr (Diff)

Atheist 56% [52, 60] 44% [29, 57] .96

Black 92% [90, 94] 70% [56, 84] >.999

Catholic 92% [90, 94] 77% [63, 91] .98

Gay 73% [69, 76] 56% [42, 70] .99

Muslim 50% [46, 54] 38% [23, 52] .95

Woman 92% [90, 94] 73% [60, 88] >.999

^aPosterior means [with 97% highest posterior density intervals, HPDIs in brackets] are provided for all six groups. Pr(diff) reflects the posterior probability that self-reported tolerance exceeds indirectly measured tolerance for each group.

Exploratory Demographic Analyses

In addition to our overall confirmatory analyses, we were also able to explore tolerance for all six targets across individual differences in gender, education, ethnicity, political affiliation, religious affiliation, and age. It is important to note that these analyses highlight a single demographic contrast while still adjusting for all other demographic covariates. So, for example, a contrast based on political affiliation can compare predicted direct and indirect support for candidates among democrats and republicans, adjusting for all other demographic information (e.g., such a contrast will show the political divergence at a mean level of education, age). As full presentation of these data and inferences are rather unwieldy, we designed a web app that allows readers to explore them at their leisure (https://willgervais.shinyapps.io/honest_votes/). We highlight a few demographic differences for purely illustrative purposes.

Education, age, political affiliation, and religious affiliation all produced notable patterns of results. Interestingly, education (beyond a high school degree) primarily predicts self-reporting greater tolerance—especially for atheist, gay, and Muslim candidates—without appreciably predicting indirectly measured tolerance (see Figure 2A). This difference implies that higher education predicts public, but not private, tolerance for certain stigmatized groups.

Figure 2.
Select demographic contrasts for (A) education, (B) age, (C) politics, and (D) religion. Point estimates reflect posterior means, and error bars reflect 97% highest posterior density intervals.

In contrast, age (comparing millennials to baby boomers) seems to primarily predict unbiased self-reporting of tolerance. Millennials and baby boomers showed similar patterns in self-reports, but baby boomers showed much larger discrepancies between self-reported and indirectly measured tolerance across the board (see Figure 2B). In terms of political party affiliation, self-reports were both more tolerant and less discrepant from indirect measures among democrats than among republicans (see Figure 2C). Finally, concerning religion, atheists and agnostics reported rather high tolerance for almost all targets, with very little discrepancy between self-reports and indirect measures. Christians, on the other hand, tended to show large discrepancies between self-reports and indirect measures for Black, gay, and female candidates, with much lower and less discrepant reported tolerance for atheist and Muslim candidates (see Figure 2D). Each of these contrasts may warrant additional focused investigation.

Discussion

The present research provides a benchmark that quantifies, based on a large and representative sample, tolerance for diversity in America. Overall, the findings demonstrate a consistent and robust gulf between the direct and indirect measures of voting for candidates from various social groups. The direct measure suggested much greater tolerance for diversity than did the indirect measure. Additionally, the divergence between the direct and indirect measures was particularly large for Black, gay, and female candidates such that people reported more tolerance for these candidates in the direct measure but markedly less tolerance in the indirect measure. However, even openly reviled groups such as atheists and Muslims were still viewed less favorably when measured indirectly (vs. directly). These results suggest that self-reported tolerance for various outgroups reflects both genuine attitudes and a component of socially desirable reporting, especially for targets from a social group which is normatively deemed socially inappropriate to stigmatize. Thus, the United States is quite unlikely to be nearly as culturally inclusive as self-reports suggest, and our results offer some clues as to just how discrepant self-reported tolerance is across different targets and among different participant demographics.

Alas, these findings cannot directly speak to inaccurate polling in actual political contests. Recently, there has been much discussion surrounding polling inaccurately predicting the winner of a political contest. Many have argued that this inaccuracy is, in part, due to the “Bradley effect” (also known as the “Wilder effect”), which is a gap between the positive support for a socially marginalized candidate and the candidate’s performance in an actual election (e.g., Hopkins, 2009; Keeter & Samaranayake, 2007; Reeves, 1997). This gap is often attributed to voter bias and social desirability concerns. Presumably, voters want to appear egalitarian by reporting their intention to vote for the marginalized candidate, but their voting behaviors do not match their self-reported voting intentions. Although there is some empirical evidence for the Bradley effect influencing the election of African American candidates from the 1980s to early 1990s, there is little evidence that this effect extends to other socially marginalized candidates, or to more recent elections (e.g., Hopkins, 2009). Therefore, we do not believe that the Bradley effect was a primary cause of the recent surprise outcome in the U.S. presidential election (see also Enten, 2016).

In addition, the current findings cannot speak to specific candidate contests due to lack of specificity. In the current research, we are assessing preferences for a generic candidate who happens to be from a specific social group. This lack of specificity means that voters know nothing about the candidate’s specific policy platform. We expect candidate’s policy platform information to drastically influence voting preferences. Therefore, although a person may report willingness to vote for an African American candidate in the abstract, this willingness may not extend to specific contests, such as when the candidate’s platform deviates from the voter’s opinions.

Although the current findings do not directly apply to recent elections, the findings provide an interesting comment on political contests nonetheless. Well-qualified candidates from the voter’s preferred political party may still be penalized for their demographic characteristics. And, indirect measures reveal markedly more intolerance for diverse candidates than previously assumed. Interestingly, some groups (e.g., younger generations, less educated individuals) may be more willing to admit their voting preferences than other groups. Our findings are consistent with other research, suggesting that indirect measures may help elucidate the penalty of bias against candidates from socially marginalized groups. For example, Payne and colleagues (2010) found that implicit bias against African Americans may have negatively impacted the margin of victory for President Obama in the 2008 election. In addition, Stephens-Davidowitz (2014) used google searches for racially charged terms as a predictor of Obama’s vote share. This indirect measure was a robust negative predictor of Obama’s vote share. Together, these findings suggest that indirect measures may be more accurate predictors of election outcomes, particularly when the candidate is representative of a socially marginalized group.

Limitations

The current research also suffers from a few limitations. First, although we tried to keep all conditions as closely matched as possible, the direct and indirect conditions still deviated in a few ways. In particular, in the direct condition, participants saw all items simultaneously, whereas in the sensitive condition of the UCT, participants saw the sensitive items sequentially and surrounded by filler items. Although we know of no research which contends that these differences in conditions influence participants’ responding, these methodological differences limit our ability to perfectly match all conditions.

Second, although we have a nationally representative sample, we lack the data granularity to investigate how local geographic context may influence tolerance for diversity. Individuals from metropolises or urban areas may report more tolerance in both the direct and indirect measures because they are more likely to interact with diverse individuals. Alternatively, it may be that frequent practice interacting with diverse individuals may lead to publicly nonprejudiced attitudes but may not influence private attitudes. As a result, people who live in metropolises or urban areas may simply be better at hiding their prejudices than are people who live in more rural areas. Future research would benefit from collecting more nuanced demographic information.

Third, the current research serves as a barometer for cultural inclusion (or exclusion) of various groups but cannot speak to trends over time. It would be useful to collect longitudinal data to investigate how both public and private attitudes toward variable social groups change over time. Given that younger participants showed more positive attitudes toward all social groups (see also Gervais & Najle, 2018), the future may show that both public and private attitudes toward diversity are positive. Moreover, the relevance of certain social groups may be specific to the current time period.

Finally, the current data are unable to investigate whether a given social category helps or hurts a candidate for a specific demographic group. We would imagine that people may prefer candidates from their in-group over out-group candidates. Thus, for some people, a socially stigmatized group membership may be a relative advantage, whereas for others it may be a disadvantage. The structure of the current data cannot investigate whether a person may pick one candidate over another candidate. As a result, we cannot directly speak to the relative penalty versus advantage of a given candidate’s group membership.

Conclusion

The United States is often considered a melting pot of diverse ethnicities, races, and religions. Despite this diversity, people’s attitudes toward diversity may not be unequivocally positive. Tolerance for diversity in America is often indexed by direct measures, yet pressures to be or appear unprejudiced may bias such estimates, yielding misleading and overly optimistic inferences about tolerance in America. The current research found that people were much more tolerant of diversity when measured using a direct, rather than indirect, measure. Perhaps Americans are better at playing the role of an egalitarian actor rather than genuinely embracing tolerance and acceptance of diversity.

Demographic Group	Self-Report	Indirect	Overall
Gender
Female	55.3	54.9	55.0
Male	44.7	45.2	45.0
Race
White	69.0	70.6	70.8
Black	12.4	11.0	11.6
Hispanic	12.2	12.4	11.5
Asian	2.8	2.0	2.3
Mixed	1.4	1.5	1.5
Other	2.2	2.5	2.4
Political ideology
Democrat	38.5	38.9	38.8
Independent	28.5	26.1	26.9
Republican	24.5	24.1	24.2
Other/unsure	8.5	10.9	10.1
Religion
Christian	55.4	55.3	55.4
Atheist/agnostic	13.4	10.7	12.5
Other	31.2	34.1	32.2
Education
Max high school	42.5	41.5	41.8
Beyond high school	57.5	41.5	58.2
Age
Mean	47.7	47.7	47.7
Range	19–90	19–91	19–91

UCT Baseline	UCT Target Groups	Self-Report Items
Imagine the five different people listed below. Now, separately consider the following statement for each of those five people: “If my party nominated a generally well-qualified person for president who happened to be ______, I would vote for that person.” A small business owner An avid cyclist From a rich family A coffee drinker Bankrupt Simply count how many of those different people you would vote for. In the space below, please write how many of these potential candidates (from 0 to 5) you would vote for: ____________ of these candidates	Imagine the six different people listed below. Now, separately consider the following statement for each of those six people: “If my party nominated a generally well-qualified person for president who happened to be ______, I would vote for that person.” A small business owner An avid cyclist From a rich family A coffee drinker Bankrupt Catholic Simply count how many of those different people you would vote for. In the space below, please write your total count of how many of these potential candidates (from 0 to 6) you would vote for: ____________ of these candidates	We are interested in your political attitudes. Please answer “yes” or “no” to the following questions. If your party nominated a generally well-qualified person for president who happened to be _______, would you vote for that person? Black (Y/N) A woman (Y/N) Gay or lesbian (Y/N) Muslim (Y/N) An atheist (someone who does not believe in God) (Y/N) Catholic (Y/N)
A	B	C

Target Group	Self-Report	Indirect	Pr (Diff)
Atheist	56% [52, 60]	44% [29, 57]	.96
Black	92% [90, 94]	70% [56, 84]	>.999
Catholic	92% [90, 94]	77% [63, 91]	.98
Gay	73% [69, 76]	56% [42, 70]	.99
Muslim	50% [46, 54]	38% [23, 52]	.95
Woman	92% [90, 94]	73% [60, 88]	>.999

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflict of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of the article: This research was funded by the Templeton Foundation (48275).

References

Allport

G. W.

Kramer

B. M

. (1946). Some roots of prejudice. The Journal of Psychology: Interdisciplinary and Applied, 22, 9–39.

Bergsieker

H. B.

Leslie

L. M.

Constantine

V. S.

Fiske

S. T.

(2012). Stereotyping by omission: Eliminate the negative, accentuate the positive. Journal of Personality and Social Psychology, 102, 1214–1238.

Coffman

K. B.

Coffman

L. C.

Ericson

K. M. M.

(2016). The size of the LGBT population and the magnitude of antigay sentiment are substantially underestimated. Management Science, 3168–3186. doi:10.1287/mnsc.2016.2503

Coutts

Jann

(2011). Sensitive questions in online surveys: Experimental results for the randomized response technique (RRT) and the unmatched count technique (UCT). Sociological Methods & Research, 40, 169–193.

Crandall

C. S.

Eshleman

(2003). A justification-suppression model of the expression and experience of prejudice. Psychological Bulletin, 129, 414–446.

Crandall

C. S.

Eshleman

O’Brien

(2002). Social norms and the expression and suppression of prejudice: The struggle for internalization. Journal of Personality and Social Psychology, 82, 359.

Dalton

D. R.

Wimbush

J. C.

Daily

C. M.

(1994). Using the unmatched count technique (UCT) to estimate base r. Personnel Psychology, 47, 817–828.

Devine

P. G.

Monteith

M. J.

Zuwerink

J. R.

Elliot

A. J.

(1991). Prejudice with and without compunction. Journal of Personality and Social Psychology, 60, 817.

Enten

. (2016). ‘Shy’ voters probably aren’t why the polls missed Trump. Five Thirty Eight. Retrieved from https://fivethirtyeight.com/features/shy-voters-probably-arent-why-the-polls-missed-trump/

10.

Gallup. (2015). Presidential voting trends. Retrieved from http://www.gallup.com/file/poll/183731/Voting_for_Candidates_Characteristics_I_150622.pdf

11.

Gelman

Hill

Yajima

(2012). Why we (usually) don’t have to worry about multiple comparisons. Journal of Research on Educational Effectiveness, 5, 189–211.

12.

Gervais

W. M

. (2014). Everything is permitted? People intuitively judge immorality as representative of atheists. PLoS One, 9. doi:10.1371/journal.pone.0092302

13.

Gervais

W. M.

Najle

M. B

. (2018). How many atheists are there? Social Psychology & Personality Science, 9, 3–10.

14.

Glick

Fiske

S. T.

(1996). The ambivalent sexism inventory: Differentiating hostile and benevolent sexism. Journal of Personality and Social Psychology, 70, 491–512.

15.

Hoekstra

Morey

R. D.

Rouder

J. N.

Wagenmakers

E.-J.

(2014). Robust misinterpretation of confidence intervals. Psychonomic Bulletin & Review, 21, 1157–1164.

16.

Hopkins

D. J.

(2009). No more Wilder Effect, never a Whitman Effect: When and why polls mislead about Black and female candidates. Journal of Politics, 71, 769–781.

17.

Keeter

Samaranayake

(2007). Can you trust what pools say about Obama’s electoral prospects? Pew Research Center Report. Retrieved from http://pewresearch.org

18.

Kinder

D. R.

Sears

D. O

. (1981). Prejudice and politics: Symbolic racism versus racial threats to the good life. Journal of Personality and Social Psychology, 40, 414–431.

19.

Kruschke

(2010). Doing Bayesian data analysis: A tutorial introduction with r. San Diego, CA: Academic Press.

20.

Kteily

Hodson

Bruneau

(2016). They see us as less than human: Meta-dehumanization predicts intergroup conflict via reciprocal dehumanization. Journal of Personality and Social Psychology, 110, 343–370.

21.

LaBrie

J. W.

Earleywine

(2000). Sexual risk behaviors and alcohol: Higher base rates revealed using the unmatched-count technique. Journal of Sex Research, 37, 321–326.

22.

McElreath

(2016). Statistical rethinking: A Bayesian course with examples in R and Stan. Boca Raton, FL: CRC Press.

23.

Meertens

R. W.

Pettigrew

T. F

. (1997). Is subtle prejudice really prejudice? Public Opinion Quarterly, 61, 54–71.

24.

Monteith

M. J.

Devine

P. G.

Zuwerink

J. R

. (1993). Self-directed versus other directed affect as a consequence of prejudice-related discrepancies. Journal of Personality and Social Psychology, 64, 198–210.

25.

Nuno

Bunnefeld

Naiman

L. C.

Milner-Gulland

E. J.

(2013). A novel approach to assessing the prevalence and drivers of illegal bushmeat hunting in the Serengeti. Conservation Biology, 27, 1355–1365.

26.

Oakes

(1986). Statistical inference: A commentary for the social and behavioural sciences. Chichester, England: Wiley.

27.

Payne

B. K.

Krosnick

J. A.

Pasek

Lelkes

Akhtar

Tompson

(2010). Implicit and explicit prejudice in the 2008 American presidential election. Journal of Experimental Social Psychology, 46, 367–374.

28.

Plant

E. A.

Devine

P. G.

(1998). Internal and external motivation to respond without prejudice. Journal of Personality and Social Psychology, 75, 811–832.

29.

Prager

(2016, 6 21). He immaturity of supporting Hillary Clinton because she’s a woman. Retrieved from http://www.nationalreview.com/article/436859/hillary-clinton-woman-card-identity-politics-are-bad-reason-vote-her

30.

R Core Team. (2017). R: A Language and Environment for Statistical Computing. Retrieved from https://www.R-project.org/

31.

Raghavarao

Federer

W. T.

(1979). Block total response as an alternative to the randomized response method in surveys. Journal of the Royal Statistical Society Series B (Methodological), 41, 40–45.

32.

Rayburn

N. R.

Earleywine

Davison

G. C.

(2003). An investigation of base rates of anti-gay hate crimes using the unmatched-count technique. Journal of Aggression, Maltreatment & Trauma, 6, 137–152.

33.

Reeves

(1997). Voting hopes or fears? White voters, Black candidates, and racial politics in America. New York, NY: Oxford University Press.

34.

Stephens-Davidowitz

(2014). The cost of racial animus on a Black presidential candidate: Evidence using Google search data. Journal of Public Economics, 118, 26–40.

35.

Valenti

(2016, 1 15). Hillary Clinton supporters: It is ok to care about gender on the ballot. The Guardian. Retrieved from https://www.theguardian.com/commentisfree/2016/jan/15/go-ahead-and-vote-for-hillary-clinton-because-she-is-a-woman

36.

Wagenmakers

E.-J.

Morey

R. D.

Lee

M. D.

(2016). Bayesian benefits for the pragmatic researcher. Current Directions in Psychological Science, 25, 169–176.

37.

Westfall

Van Boven

Chambers

J. R.

Judd

C. M.

(2015). Perceiving political polarization in the United States party identity strength and attitude extremity exacerbate the perceived partisan divide. Perspectives on Psychological Science, 10, 145–158.