Answering Questions About Race: How Racial and Ethnic Identities Influence Survey Response

Abstract

Given the fundamental role that race and ethnicity play in U.S. society, sensitive survey items on this subject can often lead individuals to underreport their true attitudes. Previous studies have shown that the absence of an interviewer reduces the pressure to provide socially desirable responses. The 2012 and 2016 American National Election Studies (ANES), where both interviewer and self-administered surveys were used, allows us to test whether mode effects emerge in the way respondents answer survey items related to racial attitudes. We also expect mode effects to vary based on the extent to which individuals are politically socialized in the United States. We find that respondents tend to underreport their racial animosity in interview-administered versus online surveys. Moreover, underreporting is nonexistent in the responses provided by foreign-born Latinos, but emerges for U.S.-born Latinos, Blacks, and Whites. These findings pose a number of implications for our understanding of racial attitudes and survey mode.

Keywords

survey methodology survey mode racial attitudes racial resentment survey response

Race is a fundamental cleavage in American society. Many scholars have recognized the importance of race in shaping American public opinion and political behavior, stretching back to the early years of modern social science (e.g., Adorno, Frenkel-Brunswick, Levinson, & Sanford, 1950; Allport, 1954). Whether expressed in outright racism or prejudice (Allport, 1954), or a less overt “symbolic racism” or similar concepts (Bobo, 1983; Kinder & Sears, 1981), race and ethnicity continue to play important roles in determining attitudes about policies and issues, as well as political behavior. One can barely read the news today without reading about killings of unarmed Black men by White police officers, concerns about race-based affirmative action policies, or the issue of immigration. Just as issue of race and immigration played a prominent role in the 2016 presidential election, it appears that they will continue to factor in future elections as well.

The social norms on race have evolved significantly over time (Mendelberg, 2001). Where expressions of old-fashioned, or open, racism were once acceptable and the norm for most of the nation’s history, the last several decades have ushered in an era where a strong norm of equality exists (Mendelberg, 2001). In light of this shift in norms, survey respondents are under pressure to offer a socially desirable response on questions pertaining to racial attitudes (Berinsky, 1999, 2002; Huddy & Feldman, 2009). That is, many individuals feel compelled to report racial attitudes in a way that conforms to the socially acceptable norm on race, for fear of being perceived as prejudiced or racist (Berinsky, 1999, 2002). As a result, survey items that directly ask about one’s racial attitudes typically trigger social desirability concerns.

As alluded to above, a significant concern in collecting sensitive information is the tendency for survey participants to provide socially desirable responses ((Crowne & Marlowe, 1960; Crowne & Marlowe, 1964); (Kreuter et al., 2008); Tourangeau & Smith, 1996; Tourangeau and Yan, 2007)). This concern is particularly acute when surveys are interviewer-administered, as opposed to self-administered (Aquilino, 1994; Atkeson, Adams, & Alvarez, 2014; (Kreuter et al., 2008; Tourangeau & Smith, 1996; Tourangeau and Yan, 2007). For instance, respondents taking self-administered surveys via the Internet, versus those participating in interviewer conducted surveys, are more willing to respond to sensitive survey questions pertaining to illicit drug use and sexual behavior (Aquilino, 1994; Atkeson et al., 2014; Kreuter et al., 2008; Tourangeau & Smith, 1996; Tourangeau and Yan, 2007). For sensitive questions pertaining to racial attitudes, it is also highly likely that the race/ethnicity of the interviewer will affect one’s responses. As these previous research efforts have documented, the absence of an interviewer greatly reduces the social norms governing certain types of undesirable behaviors and attitudes. The consistency in these findings has led scholars to conclude that mode effects play an important role in the reporting of socially undesirable attitudes.

As such, this article uses survey mode to study the reporting of a highly sensitive topic, such as racial attitudes, in surveys of the U.S. public. Whereas previous studies rely on different survey modes to examine the reporting of sensitive behaviors (e.g., Kreuter, Presser, & Tourangeau, 2008; Tourangeau & Smith, 1996), to the best of our knowledge, we are the first to study the relationship between survey mode and responses to a wide array of topics pertaining to racial sentiment. Our study takes advantage of the unique survey design of the 2012 and 2016 American National Election Studies (ANES); for the first time in its time series history, face-to-face (FTF) interviews were supplemented with data collection on the Internet. These efforts were conducted in the two modes independently, using separate (and relatively large) samples. And for the most part, the same survey items were asked in both the Internet and FTF samples.¹ We focus on a battery of race-based questions (e.g., group stereotypes, group feeling thermometers) as well as the racial resentment questions developed by Sears, Sidanius, and Bobo (2000).²

Consistent with previous research (Kreuter et al., 2008; Tourangeau & Smith, 1996), we expect mode effects to be present in the way respondents answer these survey items. In particular, respondents should be more likely to underreport their racial attitudes in the FTF versus the online survey mode.

Importantly, we also expect mode effects to vary based on the extent to which individuals are politically socialized in the United States. The varying political opportunity structures afforded to Blacks, Latinos, and Asian Americans has meant that their experience and orientation to politics differs vastly from those of White Americans (Abrajano & Alvarez, 2010; Dawson, 1994; DeSipio, 1996; Garcia Bedolla, 2009; Sanchez & Garcia, 2008). As such, we expect these differential experiences to play a critical role in how racial/ethnic groups respond to survey items on race. For Latinos, particularly those who are born outside of the United States, mode effects should be minimal or nonexistent their responses, compared with Whites and Blacks. Given that 40% of Latinos are foreign-born, the way they learn about U.S. politics and norms occurs through channels outside the classic model of parental socialization (Campbell et al., 1960). Exposure and socialization to matters on racial norms and attitudes are therefore likely to be absent. However, for Blacks, U.S.-born Latinos, and Whites, we contend that underreporting of negative racial sentiment will emerge, as a function of survey mode.

Overall, our analyses support these expectations. First, and consistent with previous research, the results suggest that mode effects are present in responses to racial attitude survey items. Individuals are more likely to underreport negative racial sentiment in the FTF versus the Internet mode. These results are robust to the inclusion of a whole host of covariates such as demographics and political knowledge. We also find support for our argument that the pattern of underreporting socially undesirable attitudes (in this case, negative racial sentiment) varies according to one’s degree of political socialization. We find no mode differences in the responses to these sensitive items among foreign-born Latinos, but mode differences do emerge in the responses provided by U.S.-born Latinos, and particularly for Black and White Americans.

These findings pose a number of implications for future research. First, because we demonstrate the prevalence of mode effects for one of the most frequently used political surveys (the ANES), scholars need to be cognizant of these mode differences, particularly on sensitive survey items. Moreover, as the ANES, like many other surveys, considers moving away from FTF interviews to an entirely online format, researchers should also be careful of comparing the same survey items across years and across decades. Finally, while our findings are the first, to the best of our knowledge, to demonstrate cross-cultural variations in the way individuals respond to a highly sensitive subject matter such as race, our research is by no means the last word on this subject. In light of the rapidly and continually changing demographic composition of the United States, along with changes in the way surveys are carried out, there is still much work to be done in this area.

Measuring Racial Attitudes in the United States

Capturing public sentiment on issues of race in the United States is no easy task (Berinsky, 1999, 2002; Gilens, Sniderman, & Kuklinski, 1998; Huddy & Feldman, 2009). The challenge of doing so has, at times, been heightened by the fraught nature of race relations in the country, as well the way racial norms have evolved and changed over time (Mendelberg, 2001). Given the way race structures and shapes nearly every aspect of American society, scholars have thought long and hard about the different ways of measuring racial sentiment and prejudice.

The earliest survey researchers relied on questions that directly asked about racial attitudes, such as those captured in racial stereotype survey items (e.g., “Blacks are hardworking,” “Blacks are intelligent”). However, subsequent scholars raised concerns regarding the ability of such questions to truly capture racial prejudice, due to issues of social desirability (Berinsky, 1999, 2002; Gilens et al., 1998; Huddy & Feldman, 2009; Kuklinski, Sniderman et al., 1997). The presence of an interviewer, as well as the race of the interviewer, can also make survey respondents less likely to provide their true opinions on race, particularly if they are racially resentful ones (Atkeson et al., 2014; for race of interviewer effects, see Krysan & Couper, 2003). Recently, scholars have attempted to alleviate some of these issues by using new methodologies, such as list experiments, to avoid explicit signals on sensitive subject matters such as race (Abrajano, Elmendorf, & Quinn, 2018; Hainmueller, Hangartner, & Yamamoto, 2015).

Among those studies that focus on racial attitudes, a recent study by Piston (2010) finds that racial stereotypes were a strong predictor of White opposition to Obama in 2008. His study reveals that open prejudice continues to play an important role in current U.S. politics. Another set of questions that tap racial attitudes are feeling thermometer questions that ask respondents to evaluate the major ethnic/racial groups in the United States as well as “illegal immigrants” on a 0 to 100 scale. Given that these survey items measure affect for the particular group in question, they can also be considered a sensitive survey item. Moreover, Kinder and Sanders (1996) developed several questions that capture racial resentment or symbolic racism. These survey items attempt to capture anti-Black affect, along with traditional Anglo-Saxon Protestant values of hard work and individualism (Sears et al., 2000). Such questions ask individuals whether (a) Blacks should work their way up without any special favors, (b) past slavery makes it more difficult for Blacks to advance, (c) Blacks have gotten less than they deserved, and (d) Blacks must try harder to get ahead. These survey items have been asked in the ANES for more than three decades now and therefore regarded by scholars as a reliable indicator of racial sentiment in the United States.

Cross-Cultural Variations in Survey Response

How individuals respond to sensitive survey items is not equal across the board. The existing research notes that some individuals are more susceptible to providing socially desirable answers than others. Johnson and Van de Vijver (2003) stress the importance and need to consider cross-cultural variations in survey responses. In the context of the United States, Aquilino finds mode effects in the survey responses provided by racial/ethnic minorities. Blacks were less likely to report illicit drug relative to Whites in the interviewer versus self-administered survey, with no difference between Hispanics and Whites (Aquilino, 1994). He attributes this difference to the stigma surrounding drug use in the Black community and, as a result, underreporting of this socially undesirable behavior emerges, vis-à-vis White respondents. And while some research suggests that Blacks and Mexican Americans score higher on a social desirability scale than do non-Hispanic Whites (Warnecke et al., 1997), others have failed to document this effect (Okazaki, 2000). In addition, Johnson and Van de Vijver (2003) contend that the pressure to offer socially desirable responses may be due to an individual’s level of social power within a society, so that those with lower levels “tend to be more concerned with impression management and hence display more socially desirable behavior.” (p. 196)

These previous research efforts serve as the foundation for our study, and we derive several hypotheses on the differential role of survey mode in the reporting of socially undesirable attitudes pertaining to race. By considering the research on survey methodology with studies of racial and ethnic politics in the United States, we hypothesize that responses to the sensitive survey items about racial attitudes may vary across the major racial/ethnic groups in the United States. Past research documents the varying political socialization process for racial/ethnic minorities, particularly for Latinos (Abrajano & Alvarez, 2010). Therefore, on such matters like race relations in the United States, many Latinos, especially for the 40% born outside of the United States, the racial norms in the United States are very likely to be altogether foreign and unfamiliar.³ And even for Latinos born in the United States, parental socialization on such issues is largely absent. In light of these distinct socialization experiences, we would expect minimal to nonexistent mode effects in the survey responses provided by foreign-born Latinos. However, mode effects should be more evident among U.S.-born Latinos, especially for third and later generation Latinos.

More formally stated, our main hypotheses are as follows. First, we expect mode effects to be present in the reporting of socially undesirable survey items. Respondents should therefore be more likely to express negative racial sentiment in the Internet mode versus the FTF mode (H1). Second, we expect mode effects to be more prominent among U.S.-born individuals (Black, Whites, and U.S.-born Latinos) relative to foreign-born Latinos (H2). Understanding the nuances of societal norms and expectations is a process that requires time and experience, as well as the opportunity to interact with the native-born population, especially White Americans. Given that such interactions occur over years and even decades, foreign-born Latinos will have had fewer opportunities to learn about these norms when compared with U.S.-born Latinos. For this very reason, we hypothesize mode effects to be nonexistent in explaining the racial attitudes of foreign-born Latinos, relative to U.S.-born Latinos (H3). We next discuss how we test these three hypotheses.

Research Design

To test these hypotheses, our dependent variables are all of the survey items from the 2012 and 2016 ANES that pertain to racial sentiment. Altogether, we identified 17 survey items that fulfilled these criteria. These include the racial resentment questions that we discussed earlier, as well as a battery of stereotype questions on intelligence and work ethic (hardworking vs. lazy) for each ethnic/racial group.⁴ We also include the feeling thermometer questions for each ethnic/racial group (Black, Latino, Asian, “illegal” immigrants, and White). The feeling thermometer questions are measured on a scale from 0 to 100, with a “100” indicating great warmth for the group in question, and “0” indicating the least warmth.

One important methodological difference exists in how these questions were asked in the two survey modes. In the FTF mode, interviewers did not directly ask respondents the stereotype and feeling thermometer questions. Instead, respondents answered these survey items using a computer-assisted survey instrument (CASI). The ANES attempted to minimize the issue of social desirability bias by providing respondents with a tablet where they could record their response to these sensitive topics. For the remaining survey items pertaining to race, the interviewers posed these questions to interviewees.

In these two election studies, the 2012 ANES interviewed a larger number of racial/ethnic minorities than did the 2016 ANES. For instance, the 2012 ANES interviewed 1,005 Latinos, whereas the 2016 ANES interviewed only 450 Latinos. Among Blacks, 1,132 were interviewed in the 2012 ANES, but in 2016 they only comprised 468 of all respondents. As such, our main analysis uses pooled data from both election cycles.⁵

As survey mode was not randomly assigned to survey respondents, concerns may arise regarding the comparability of the survey of two populations. We conducted a balance check on all the covariates in our mode and the results are available in Table A1 of the supplementary appendix. Virtually all of the covariates are balanced across the two samples, though we see that the FTF mode contains more Democrats and liberals than the Internet sample (43% vs. 51% Democrats, and 30% vs. 40% for liberal), and that the sample of respondents who are married or partnered is slightly larger in the Internet than in the FTF sample. Given that the imbalances across modes are minor, there is little utility in attempting to adjust these minor imbalances statistically or in dropping cases via matching as there is no indication that more complex methodologies would produce results different from what we present below. Thus, we proceed with our multivariate analyses using all of the data available in these two surveys and use multivariate discrete choice models to test our hypotheses. By controlling for a large array of covariates in our multivariate models, in particular using variables like age, education, and political knowledge that could differ between an online and FTF sample, we are likely accounting for these minor imbalances in the original data without estimating models that are difficult to interpret and which would arbitrarily drop a large number of cases from our analysis with a consequent loss of statistical power.

In our multivariate models, our primary explanatory variable of interest is survey mode, which is coded so that a 1 indicates that it was an Internet survey and a 0 indicates an FTF interview. We also include other covariates that have been found to explain racial attitudes, such as demographics, political preferences and affiliations, as well as a variable indicating the election year (Sears et al., 2000). To test our second and third hypotheses, we estimate these multivariate models by disaggregating the data by racial/ethnic group; this allows us to compare mode effects across these groups.

Results

Before turning to the multivariate estimates, we first examine responses to the various racial attitude questions by survey mode. These analyses are presented in Figures 1 to 3. In Figure 1, we compare the feeling thermometer score evaluations for respondents in the FTF versus the Internet mode. Recall that higher evaluations indicate greater warmth for the group in question. As the error bars indicate, we see that affect for Latinos, “illegal” immigrants, and Blacks is statistically significantly different across survey mode.⁶ Those respondents who took the survey online report lower affect for these groups, when compared with their counterparts in FTF survey. What is particularly striking is persistence of these mode effects, despite the best efforts by the ANES to have respondents record their responses using CASI. In addition, these distributions suggest the lack of a mode effect in the evaluations of Whites or Asians. This finding is a recurring pattern that emerges in our analyses; we attribute such differences to the racial ordering that exists in the United States, with Whites at the top, Asians in the middle, and Blacks and Latinos at the bottom (Kim, 2003; Masuoka & Junn, 2013).

Figure 1.

Mean FT scores, by survey mode.

Figure 2.

Mean responses stereotype question (lazy) by survey mode.

Figure 3.

Mean responses to racial resentment questions, by survey mode.

Figure 2 presents the mean responses to the stereotype questions by survey mode. Similar to the patterns that emerged in Figure 1, mode effects only emerge in the reporting of stereotypical perceptions toward Blacks and Latinos. Survey respondents who took the survey online are more willing to agree with the stereotype that Black and Latinos are lazy when compared with their FTF counterparts. In other analyses that we present in the supplementary appendix, the stereotype question on intelligence from the 2012 ANES follows the same pattern as we see here, with a larger percentage of respondents reporting Blacks and Latinos as intelligent in the Internet survey mode, when compared with the FTF survey mode.

Next, Figure 3 presents the average responses to the racial resentment questions by survey mode. Higher values indicate that respondents disagree that (a) Blacks should work their way up without special advantages, (b) slavery makes it more difficult, (c) Blacks have gotten less than they deserve, and (d) Blacks must try harder to get ahead. Comparable to the patterns that emerged with the other racial attitude questions, statistically significant mode effects emerge. Online respondents report higher levels of racial resentment relative to FTF respondents and these differences are particularly acute for the survey items pertaining to slavery and Blacks receiving less than they deserve.

While our bivariate analyses suggest an association between survey mode and responses to sensitive survey items, our multivariate analyses allow us to control for other factors that can explain racial sentiment. We present these estimates in Tables 1 to 3. In Table 1, we present the estimates from the models where the dependent variables are the racial resentment questions. Consistent with previous research, mode effects are indeed present in the responses provided to these sensitive survey items. With the exception of a single survey item, Internet survey respondents are more likely to provide socially undesirable responses (negative racial sentiment), relative to those in the interviewer-administered survey. As such, these estimates lend support for H1—Individuals are more likely to display negative racial sentiment in the Internet mode than in the FTF mode, even when controlling for a whole host of covariates. These findings support the contention that the presence of an interviewer is associated with the survey responses they provide. The estimated magnitude of effect for survey mode (presented in Table 3) is relatively modest compared with other predictors, such as partisanship and demographics, ranging from .01 to .05 in the racial resentment models.⁷

Table 1.

Racial Resentment Questions, Pooled Data.

	Blacks should work way up w/o advantages	Slavery makes it difficult for Blacks	Blacks have gotten less than they deserve	Blacks should try harder to get ahead
Internet Survey	0.352*** [0.24, 0.046]	0.150*** [0.038, 0.26]	−0.0331 [−0.14, 0.078]	0.276*** [0.165, 0.387]
Pol knowledge	0.028 [0.0825, 0.156]	0.0265 [−0.01, 0.064]	0.110*** [0.072, 0.147]	0.082*** [0.04, 0.120]
Female	−0.0261 [−0.104, 0.0517]	0.0385 [−0.0380, 0.115]	−0.0438 [−0.12, 0.033]	0.192*** [0.112, 0.273]
Employed	−0.0323* [−0.05, −0.007]	−0.00489 [−0.03, 0.02]	−0.0376** [−0.06, −0.009]	−0.0223 [−0.047, 0.003]
HS degree	0.033 [−0.513, −0.329]	0.0715 [0.107, 0.284]	0.187 [0.0143, 0.193]	0.235* [0.0446, 0.425]
Some college	0.207* [0.0272, 0.387]	0.0218* [0.004, 0.04]	0.282** [0.10, 0.46]	0.616*** [0.437, 0.795]
College	.801*** [0.610, 0.992]	−0.269** [−0.455, −0.0838]	−0.0205 [−0.11, 0.07]	1.246*** [1.056, 1.436]
Grad school	1.102*** ([0.903, 1.301)	−0.568*** [−0.76, −0.37]	−0.284** [−0.48, −0.086]	1.408*** [1.21, 1.60]
Married	−0.154*** [−0.240, −0.0685]	0.105* [0.021, 0.190]	0.165*** [0.08, 0.249]	−0.120** [−0.20, −0.035]
Millenial	0.471*** [0.33, 0.61]	0.0257 [−0.107, 0.158]	−0.0657 [−0.20, 0.068]	0.0454 [−0.088, 0.18]
Gen X	0.227*** [0.101, 0.349]	0.223*** [0.17, 0.41]	0.212*** [0.165, 0.411]	−0.0503 [−0.17, 0.071]
Baby boomer	0.0941 [−0.0187, 0.207]	0.208*** [0.092, 0.32]	0.184*** [0.067, 0.30]	−0.0339 [−0.144, 0.0758]
Democrat	0.504*** [0.389, 0.618]	−0. 486****** [−0.60, −0.37]	−0.607*** [−0.72, −0.49]	0.408*** [0.294, 0.522]
Republican	−0.657*** [−0.784, −0.531]	0.586*** [0.462, 0.710]	0.636*** [0.511, 0.762]	−0.584*** [−0.70, −0.46]
Liberal	0.528*** [0.40, 0.647]	−0.575*** [−0.69, −0.45]	−0.321*** [−0.44, −0.20]	0.508*** [0.389, 0.626]
Conservative	−0.339*** [−0.45, −0.22]	0.140* [0.004, 0.22]	0.346*** [0.189, 0.41]	−0.250*** [−0.367, −0.134]
2012 election	0.211*** [0.097, 0.324]	−0.197*** [−0.31, −0.085]	−0.186** [−0.298, −0.0738]	0.298*** [0.188, 0.409]
Weight	−0.00892 [−0.0632, 0.045]	0.0742** [0.021, 0.127]	0.0432 [−0.010, 0.096]	0.0469 [−0.001, 1.00]
cut1	0.102 [−0.04, 0.43]	−1.97***[−2.6, −2.1]	−2.22**[−11.9, −7.9]	−0.17[−0.67, −0.19]
cut2	1.53*** [1.28, 1.78]	−.69***[−0.94, −0.46]	−3.09*** [−3.35, −2.84]	.99*** [0.75, 1.23]
cut3	2.52*** [2.27, 2.77]	−0.03 [−0.27, 0.21]	−1.64*** [−1.89, −1.39]	2.05*** [1.81, 2.29]
cut4	3.68*** [3.42, 3.94]	1.12*** [0.88, 1.37]	−0.30* [−0.54, −.058]	3.30*** [3.05, 3.55]
cut5			0.929*** [0.68, 1.17]
N	8,639	8,643	8,628	8,640

Note. 95% confidence intervals in brackets. Cut1- cut5 refer to the cutpoints.

p < .05. **p < .01. ***p < .001.

Table 2.

Feeling Thermometer Questions (0-100), Pooled ANES Data.

	Asians	Latinos	Blacks	Illegal immigrant	Whites
Internet Survey	1.103 [−0.12, 2.32]	−2.037** [−3.35, −0.71]	−1.552* [−2.84, −0.26]	−2.610*** [−4.14, −1.07]	−0.486 [−1.66, 0.69]
Pol knowledge	1.327*** [0.91, 1.74]	0.829*** [0.37, 1.28]	−0.335 [−0.78, 0.11]	0.122 [−0.40, 0.64]	0.620** [0.21, 1.02]
Female	0.959* [0.06, 1.85]	2.111*** [1.14, 3.07]	3.441*** [2.49, 4.38]	1.505** [0.38, 2.62]	2.390*** [1.528, 3.25]
Employed	0.513** [0.20, 0.82]	0.187 [−0.14, 0.52]	0.0699 [−0.25, 0.39]	−0.0194 [−0.41, 0.36]	0.0597 [−0.23, 0.35]
HS grad	2.459* [0.41, 4.50]	−2.170 [−4.38, 0.04]	0.541 [−1.62, 2.70]	−5.690*** [−8.255, −3.125]	2.504* [0.528, 4.48]
Some college	5.761*** [3.83, 7.68]	−0.0938 [−2.17, 1.98]	1.472 [−0.56, 3.50]	−5.240*** [−7.65, −2.82]	2.653** [0.79, 4.51]
College grad	8.929*** [6.88, 10.97]	2.725* [0.52, 4.92]	3.013** [0.85, 5.16]	−2.221 [−4.77, 0.334]	3.040** [1.071, 5.01]
Postgrad	9.303*** [7.15, 11.45]	2.493* [0.17, 4.81]	2.189 [−0.07, 4.45]	0.118 [−2.57, 2.805]	2.001 [−0.07, 4.07]
Married	0.0428 [−0.89, 0.98]	0.881 [−0.13, 1.89]	−0.239 [−1.23, 0.75]	0.441 [−0.73, 1.61]	−0.0665 [−0.96, 0.83]
Millenial	0.183 [−1.38, 1.75]	0.650 [−1.04, 2.34]	−2.193** [−3.85, −0.54]	4.434*** [2.47, 6.39]	−6.608*** [−8.11, −5.09]
Gen X	−0.416 [−1.84, 1.01]	0.473 [−1.06, 2.01]	−1.665* [−3.17, −0.16]	0.774 [−1.01, 2.56]	−4.265*** [−5.64, −2.89]
Baby boomer	−0.892 [−2.19, 0.41]	−1.012 [−2.41, 0.39]	−0.0296 [−1.40, 1.34]	−0.453 [−2.08, 1.17]	−2.971*** [−4.22, −1.72]
Democrat	0.686 [−0.58, 1.95]	3.151*** [1.78, 4.52]	7.198*** [5.86, 8.54]	6.405*** [4.81, 7.99]	0.982 [−0.24, 2.20]
Republican	0.119 [−1.26, 1.50]	−1.796* [−3.28, −0.35]	−2.381** [−3.84, −0.92]	−8.247*** [−9.98, −6.52]	3.730*** [2.39, 5.06]
Liberal	4.298*** [2.97, 5.62]	4.173*** [2.74, 5.60]	2.272** [0.87, 3.67]	6.613*** [4.95, 8.27]	0.981 [−0.29, 2.25]
Conservative	0.703 [−0.60, 2.01]	0.707 [−0.71, 2.12]	0.291 [−1.09, 1.67]	−1.709* [−3.35, −0.06]	0.569 [−0.69, 1.83]
2012 election	2.641*** [1.39, 3.89]	2.489*** [1.14, 3.83]	1.509* [0.19, 2.82]	0.663 [−0.9, 2.22]	0.466 [−0.73, 1.66]
Weight	−0.733* [−1.32, −0.14]	−2.300*** [−2.93, −1.66]	−1.668*** [−2.28, −1.05]	−2.418*** [−3.15, −1.68]	0.00807 [−0.56, 0.57]
Constant	52.92*** [49.9, 55.9]	61.88*** [58.6, 65.1]	64.61*** [61.4, 67.8]	44.47*** [40.7, 48.2]	67.74*** [64.8, 70.6]
N	7,788	7,787	7,787	7,799	7,788

Note. 95% confidence intervals in brackets. ANES = American National Election Studies.

p < .05. **p < .01. ***p < .001.

Table 3.

Group Stereotype Questions—“Group Is Lazy.”.

	Whites	Blacks	Latinos	Asians
Internet Survey	−0.0096 [−0.120, 0.100]	0.230*** [0.120, 0.339]	0.220*** [0.111, 0.329]	0.014 [−0.096, 0.124]
Pol knowledge	−0.0300 [−0.0682, 0.008]	−0.00199 [−0.039, 0.035]	−0.0204 [−0.058, 0.017]	−0.0990*** [−0.13, −0.061]
Female	−0.0914* [−0.17, −0.011]	−0.157*** [−0.237, −0.076]	−0.00152 [−0.081, 0.078]	−0.0735 [−0.153, 0.006]
Employed	0.00188 [−0.026, 0.029]	0.00837 [−0.019, 0.036]	−0.0261 [−0.053, 0.001]	−0.0261 [−0.053, 0.001]
HS grad	−0.307** [−0.500, −0.114]	−0.0461 [−0.239, 0.147]	0.160 [−0.033, 0.354]	−0.143 [−0.33, 0.045]
Some college	−0.224* [−0.406, −0.04]	−0.122 [−0.304, 0.0603]	0.123 [−0.059, 0.30]	−0.419*** [−0.597, −0.241]
College	−0.119 [−0.310, 0.0715]	−0.187 [−0.378, 0.004]	0.189 [−0.002, 0.38]	−0.583*** [−0.771, −0.396]
Postgrad	−0.0523 [−0.252, 0.148]	−0.237* [−0.436, −0.037]	0.230* [0.0297, 0.431]	−0.638*** [−0.835, −0.441]
Married	0.0585 [−0.0257, 0.143]	0.163*** [0.0787, 0.247]	0.0257 [−0.0581, 0.110]	0.0662 [−0.0178, 0.150]
Millenial	0.504*** [0.362, 0.646]	0.125 [−0.0155, 0.27]	−0.417*** [−0.558, −0.276]	0.140 [−0.0004, 0.281]
Gen X	0.376*** [0.249, 0.504]	0.218*** [0.0911, 0.345]	−0.185** [−0.31, −0.059]	0.190** [0.0627, 0.318]
Baby boomer	0.165** [0.0503, 0.280]	−0.0314 [−0.147, 0.084]	−0.0449 [−0.160, 0.0697]	0.110 [−0.0051, 0.22]
Democrat	0.0182 [−0.0965, 0.133]	−0.415*** [−0.529, −0.301]	−0.307*** [−0.421, −0.193]	−0.175** [−0.29, −0.06]
Republican	−0.360*** [−0.484, −0.235]	0.261*** [0.137, 0.385]	−0.0226 [−0.14, 0.10]	−0.144* [−0.26, −0.021]
Liberal	0.0970 [−0.0223, 0.216]	−0.113 [−0.23, 0.006]	−0.118 [−0.23, 0.0003]	−0.130* [−0.25, −0.011]
Conservative	0.0430 [−0.0746, 0.161]	0.116 [−0.0024, 0.23]	−0.0369 [−0.15, 0.080]	−0.0977 [−0.21, 0.019]
2012 election	−0.00941 [−0.12, 0.10]	−0.0460 [−0.157, 0.065]	−0.231*** [−0.342, −0.121]	−0.191*** [−0.302, −0.079]
Weight	−0.0273 [−0.080, 0.025]	0.0749** [0.022, 0.13]	0.166*** [0.113, 0.219]	0.0795** [0.0269, 0.132]
cut1_cons	−2.25***[−2.54, −1.9]	−2.56***[−2.8, −2.2]	−1.6***[−1.8, −1.3]	−2.1***[−2.4, −1.8]
cut2_cons	−0.87***[−1.15, −0.59]	−1.5***[−1.87, −1.3]	−0.43**[−0.71, −0.15]	−0.81***[−1.09, −0.54]
cut3_cons	0.272 [−0.07, 0.55]	−0.523***[−0.80, −0.24]	0.601*** [0.32, 0.87]	0.16 [−0.11, 0.44]
cut4_cons	2.19*** [1.91, 2.48]	1.015*** [0.73, 1.29]	2.18*** [1.9, 2.46]	1.91*** [1.62, 2.19]
cut5_cons	3.586*** [3.27, 3.89]	2.22*** [1.94, 2.51]	3.43*** [3.1, 3.73]	3.17*** [2.8, 3.49]
cut6_cons	4.58*** [4.22, 4.94]	3.45*** [3.15, 3.75]	4.76*** [4.39, 5.12]	4.44*** [4.0, 4.8]
N	7,792	7,796	7,794	7,795

Note. 95% confidence intervals in brackets.

p < .05. **p < .01. ***p < .001.

In Table 2, we present the results from the models where we examine the relationship between survey mode and responses to the group feeling thermometer questions. Again, the thermometer questions focus on the following groups: Asians, Blacks, “illegal” immigrants, Latinos, and Whites. Our estimates indicate that survey mode factors into the evaluations for those ethnic/racial groups who are at the bottom of the racial hierarchy—Blacks, Latinos, and undocumented immigrants. In these three models, the mode coefficients are statistically significant and negatively signed, indicating that respondents who took the survey online provided less favorable evaluations of these ethnic/racial groups than those taking the FTF survey. The magnitude of these mode effects is greatest when survey respondents are asked to evaluate “illegal” immigrants; here, thermometer ratings are 2.6 points lower among respondents taking the Internet survey versus those taking the FTF survey. Given all the negative rhetoric regarding undocumented immigrants and Latinos over the past 10 years, particularly in the 2016 presidential election and the current political environment, it is understandable why some may feel pressure to mask their true feelings toward this group.

It is also worth noting that the magnitude of survey mode effects is greater than those of political ideology and gender, and nearly the same as partisanship. As such, the effect of survey mode is quite substantively significant and, as these analyses highlight, comparable in magnitude to other important covariates that have been associated with group perceptions and evaluations. By controlling for these additional factors, it allows us to recognize that survey mode is not a trivial matter when considering the factors that explain racial attitudes and opinions.

Moving on to Table 3, we present the estimates estimating the relationship between survey mode and racial stereotypes. More specifically, the survey item common to both the 2012 and 2016 ANES pertains to the laziness-hardworking stereotype with respect to Asians, Blacks, Latinos and Whites. As the results indicate, the mode coefficient is positive and statistically significant for the models in which the stereotype focuses on Blacks and Latinos. Such findings reaffirm the pattern that we documented in our bivariate analyses earlier. Internet respondents have a higher probability of perceiving both Blacks and Latinos as lazy rather than hardworking when compared with respondents who completed the survey in the FTF mode. Given that this stereotype is most frequently associated with Blacks and Latinos (Bobo, 1983; Bobo & Gilliam 1990), it should come as little surprise that social desirability effects only emerge in the evaluations of these two groups. In additional analysis that looks at the intelligent-unintelligent stereotype across these racial/ethnic groups from the 2012 ANES, we find similar results.⁸

Notably, the results also indicate the lack of mode effects for these stereotype questions when they are focused on Whites or Asians. Given that these particular stereotypes are not associated with either of these groups, it understandable why these social desirability effects fail to emerge.⁹ Thus far, our analysis suggests that mode effects are evident in responses to socially sensitive questions, such as race. Those who took the survey in complete anonymity appear to be more willing to express negative perceptions of Blacks, Latinos, and undocumented immigrants, versus those who took the survey with an interviewer.

We test our second and third hypotheses regarding the effect of socialization on survey response by estimating the same models as above but disaggregate by racial/ethnic group (and nativity for Latinos). Doing so allows us to compare the effect of survey mode by ethnic/racial group. Table 4 presents the estimates for the models where the dependent variables are the racial/ethnic group stereotype questions. What these estimates suggest is that mode effects are most present among White respondents, followed by Black respondents. In three out of the six questions, mode is a statistically significant predictor in the responses provided by Whites. The mode coefficient is also signed in the expected manner, so that Internet respondents are more likely to perceive both Latinos and Blacks as lazy and unintelligent compared with those taking the FTF survey. In contrast, mode effects fail to significantly predict Latinos’ racial stereotype opinions, regardless of nativity. While we hypothesized that the behavior of U.S.-born Latinos would more closely mirror the behavior of Blacks and White Americans, our results suggest otherwise. This phenomenon may be due to the barriers that both foreign-born and U.S.-born Latinos face in the political socialization process, thereby delaying the process of fully being immersed in societal norms over race (Abrajano & Alvarez, 2010).

Table 4.

Mode Effects on Racial Stereotype Questions, by Racial and Ethnic Group.

	Whites	Blacks	U.S.-born Latinos	Foreign-born Latinos
Asians—unintelligent	0.07 [−0.06, 0.21]	0.26 [−0.01, 0.53]	0.08 [−0.27, 0.43]	−0.33 [−0.82, 0.16]
Asians—lazy	−0.08 [−0.20, 0.04]	0.15 [−0.13, 0.41]	−0.13 [−0.07, 0.63]	−0.36 [−0.60, 0.32]
Blacks—unintelligent	0.21** [0.07, 0.34]	0.29* [0.02, 0.55]	−0.01 [−0.35, 0.33]	0.09 [−0.40, 0.59]
Blacks—lazy	0.10 [−0.002, 0.22]	0.27 [−0.23, 0.36]	0.04 [−0.46, 0.54]	0.06 [−0.38, 0.47]
Latinos—unintelligent	0.25** [0.12, 0.39]	0.35** [0.08, 0.62]	0.02 [−0.32, 0.36]	−0.25 [−0.74, 0.25]
Latinos—lazy	0.17*[0.04, 0.29]	−0.18 [−0.34, 0.59]	−0.09 [−0.51, 0.32]	0.15 [−0.30, 0.61]
N	5,994	1,264	741	488

Note. Row entries are survey mode coefficients and 95% confidence intervals are in brackets from the ordered logit model where the dependent variables are the racial stereotype questions and controls include demographics, political variables, and election year. The stereotype question on intelligence was only asked in the 2012 ANES. ANES = American National Election Studies.

We next calculate the marginal effects for these models to assess the magnitude of mode on these racial attitudes (available in Table A2 of the supplementary appendix).¹⁰ Recall that White respondents were most affected by survey and they were more likely to agree that Blacks are unintelligent and lazy, relative to those taking the FTF survey.¹¹ For Whites, the marginal effect is in the range of .02 to 06. Among Black respondents, they were .03 more likely to perceive Latinos as being unintelligent if they took the survey online as opposed to FTF. The magnitude of the mode differences is slightly larger when Blacks evaluated their own group on the stereotype question pertaining to intelligence: Black Internet survey respondents were .04 more likely to agree that Blacks are intelligent, relative to Blacks who participated in the FTF version of the survey.

We further test H2 and H3 by investigating whether survey mode effects can be detected for the feeling thermometer group questions; these results are presented in Table 5. Again, these models are specified in an identical fashion as those presented in Table 2, but we disaggregate respondents based on their racial/ethnic identity and, for Latinos, by their nativity. Similar to the previous set of estimates, mode effects are most evident in the group feeling thermometer questions provided by White respondents and, to a lesser extent, for Black respondents. Survey mode effects, however, continue to have no influence in U.S.-born and foreign-born Latinos’ perceptions toward these various racial/ethnic groups. Among White respondents, those who took the survey online rated “illegal” immigrants about 2 points lower and Latinos approximately 1.5 points on the 0 to 100 scale than were White respondents taking the FTF survey. Interestingly, mode effects also exist in Whites’ feeling thermometer assessments toward Asians, but in the opposite direction. Those who participated in the Internet survey rated Asians 2 points higher relative to those taking the survey in the FTF setting. Among Blacks, we see that social desirability exerts a larger impact on their group assessments. For instance, Blacks who took the Internet survey rated Whites 5.63 points lower on the 0 to 100 scale compared with Black respondents in the FTF survey mode. A similar pattern also exists in how Blacks assess their own group, with those who participated in the Internet survey offering a less favorable assessment (about 3.6 points lower), relative to Black respondents in the FTF mode. These findings are consistent with the research by White et al. (2014), who find that Blacks are pressured by both racial group interest versus self-interest in their political decisions. In this case, we find that Blacks feel less pressure to provide socially desirable responses in the online versus FTF survey mode. Among Blacks, such effects may be even more pronounced when the interviewer is also a coethnic/racial as White et al. find that racialized social pressure depresses self-interested behavior among Blacks. These results also raise the possibility that conclusions on the strength of linked fate among Blacks (e.g., Dawson, 1994; White et al., 2014) could vary by survey mode.

Table 5.

Mode Effects on Feeling Thermometer Evaluations, by Racial/Ethnic Group.

Feeling thermometer group	White Respondents	Black Respondents	U.S.-born Latinos Respondents	Foreign-born Latino Respondents
Latinos	−1.53* [−3.01, −0.05]	−2.01 [−5.81, 1.78]	−0.96 [−5.81, 3.89]	−3.18 [−9.48, 3.16]
Blacks	0.68 [−0.75, 2.1]	−3.66* [−6.74, −0.57]	3.17 [−2.15, 8.48]	3.33 [−6.81, 8.46]
Illegal immigrants	−2.17** [−3.88, −0.46]	−1.13 [−5.34, 3.17]	3.30 [−2.62, 9.23]	−4.99 [−12.87, 2.85]
Whites	−0.04 [−1.31, 1.23]	−5.63** [−9.47, −1.79]	−0.89 [−5.98, 4.20]	0.34 [−5.14, 8.59]
Asians	2.01** [0.65, 3.37]	−2.71 [−6.38, 0.96]	−4.31 [−0.72, 9.34]	2.44 [−5.79, 10.68]
N	6,611	1,261	738	320

Note. Row entries are survey mode coefficients and 95% confidence intervals are in brackets from the ordered logit model where the dependent variables are the feeling thermometer questions and controls include demographics, political variables, and election year.

An additional test of our second and third hypotheses examines the relationship between survey mode and the racial resentment survey items. Once again, this analysis disaggregates respondents by racial/ethnic group and nativity in the case of Latinos. We present the ordered logit estimates as well as the marginal effects in Table 6. The ordered logit estimates suggest that survey mode produces distinct effects from one ethnic/racial group to another. Among both White and Black respondents in the 2012 to 2016 ANES, mode effects seem to have an equal impact. The survey mode coefficient achieves statistical significance and is signed in the expected direction for two out of the four questions. Among U.S.-born Latinos, however, the effect of survey mode is less pronounced as it only helps to explain their levels of racial resentment on a single survey item. Consistent with our third hypothesis, mode effects fail to predict foreign-born Latinos’ responses to these racial resentment questions.

Table 6.

The Effects of Social Desirability on Racial Attitudes by Racial/Ethnic Group.

Pooled ANES	Whites	Blacks	U.S.-born Latinos	Foreign-born Latinos
Disagree—Blacks should work their way up w/o special advantages	0.15** [0.04,0.26](.01)	0.71*** [0.47,0.95](.16)	0.38*** [0.07,0.69](.04)	0.27 [−0.08,0.73]
Disagree—Past slavery makes it more difficult for Blacks	0.26*** [0.15,0.36](.02)	−0.08 [−0.32,0.15]	−0.09 [−0.38,0.21]	25 [−0.19,0.71]
Disagree—Blacks have gotten less than they deserve	0.027 [−0.08,0.13]	−0.16 [−0.39,0.07]	−0.28 [−0.58,0.02]	−0.16 [−0.61,0.29]
Disagree—Blacks must try harder to get ahead	0.038 [−0.07,0.48]	0.41*** [0.18,0.65](.12)	0.22 [−0.07,0.52])	0.28 [−0.17,0.74]
N	6,027	1,269	745	324

Note. Row entries are survey mode coefficients and 95% confidence intervals are in brackets from the ordered logit model where the dependent variables are the racial resentment questions and controls include demographics, political variables, and election year. Entries in parentheses are the marginal effects estimates where the dependent variable = 5 and all other coefficients are held at their mean/mode. ANES = American National Election Studies.

When we assess the magnitude of survey mode on racial resentment, it appears as if social desirability exerts a larger effect on Blacks’ responses in comparison with those of Whites. On the question of whether Blacks should work their way up without any special advantages, Blacks who take the survey online are .16 more likely to disagree than Black respondents who take the FTF survey. In contrast, White respondents taking the online survey are only .01 more likely to disagree with this sentiment as opposed to those taking the FTF survey. Such differences suggest that Blacks are more to sensitive social desirability on questions that pertain to assessments of their own racial group, when compared with Whites. Once again, such findings are consistent with earlier research by White et al. (2014). We see a similar pattern in Black respondents’ opinions toward the survey item asking whether “Blacks must try harder to get ahead.” Here, Blacks who took the online survey are .12 more likely to strongly disagree with this statement than are Black respondents taking the survey with an interviewer.

We also conducted a series of robustness checks to determine whether mode differences appear on survey items that are not highly sensitive in nature. We estimate the same models but replace the dependent variables with other non-race-related survey items that were asked in both the FTF (using CASI) and Internet modes. These questions include gun ownership, sexual orientation, life satisfaction, attitudes on inequality, wordsum (a test that is correlated with IQ),¹² political knowledge, and investing in the stock market.¹³ For the most part, survey mode failed to exert as strong and consistent of an effect as did the survey items that tapped into racial sentiment. These additional analyses provide further reassurance that the relationship between survey mode and racial attitudes are not spuriously correlated with one another.

Discussion

Our findings suggest that previous FTF surveys may have underestimated the degree to which racial animus predicts core political behaviors, such as vote choice and other political attitudes and preferences. As Tesler and Sears (2010) and Tesler (2012) demonstrate, the Obama presidency ushered in an era where racial attitudes emerged as a salient and important factor in explaining presidential vote choice, health care policy, and affect toward President Obama. In other words, even when controlling for a whole host of other factors, racial resentment and prejudice strongly predicts public opinion and affects toward the nation’s first African American president. Given that Tesler and Sears’ (2010) analyses uses the 2008 ANES (which was entirely FTF), what we know about the effects of racial resentment on vote choice is likely understated.

To help shed some light on this possibility, we use the variation in survey mode in the 2012 ANES to estimate several models that examine the relationship between racial resentment and political behaviors/attitudes related to Obama (likelihood of voting for Obama and favorability toward Obama). Our analysis disaggregates the sample by survey mode.¹⁴ If mode matters, we would expect racial resentment to be a stronger predictor of anti-Obama attitudes in the sample with the online respondents, relative to the sample comprised of FTF interviewees. These estimates, which are available in Tables A3 to A4 of the supplementary appendix,¹⁵ support this expectation. While the coefficient capturing racial resentment is a statistically significant predictor of voting for Obama (with those who are racially resentful being less likely to support him), the size of the coefficient is considerably larger for the Internet survey respondents (–.654), as opposed to the FTF survey respondents (–.365). In comparing the magnitude of these effects, racially resentful respondents from the Internet sample are .16 less likely to vote for Obama, whereas racially resentful respondents in the FTF sample are .06 less likely to support Obama.¹⁶ Thus, surveys using FTF interviewing are likely to underestimate, by about one third, how strongly racial animus predicts support for Obama.

In columns 3 to 4 of Table A3, we present the ordinary least squares (OLS) estimates examining the relationship between racial resentment and favorability toward Obama. We measure Obama favorability using the feeling thermometer question, which once again is a scale that ranges from 0 to 100. Similar to the earlier results, the coefficient size of racial resentment is larger in magnitude among the online survey respondents than it is for the FTF survey respondents. Internet respondents who score high on the racial resentment scale evaluate Obama 6.01 points lower on the 0 to 100 scale, whereas for racially resentful FTF survey respondents, they evaluate Obama 4.1 points lower on this scale. While one may argue that the magnitude of these estimated differences is not large substantially, they are nonetheless present. It is also worth reiterating these effects are particularly problematic for survey items that are sensitive in nature, urging scholars who rely on such questions to be particularly acute to these issues.

Conclusion

Overall, our findings contribute to existing research both substantively and methodologically. From a substantive vantage point, our study is the first to examine, to the best of our knowledge, mode effects on the reporting of racial sentiment. While previous research documents the persistent effect of survey mode in the reporting of socially undesirable behaviors (e.g., illicit drug use), it would also be reasonable to think that the reporting of attitudes that violate strong societal norms of egalitarianism and equality would also be prone to social desirability concerns. Furthermore, we also demonstrate cross-cultural variations in the way individuals report these socially undesirable sentiments. Indeed, our analysis suggests that survey mode is a consistent predictor of the way individuals report their racial attitudes. Survey respondents are more likely to underreport negative racial sentiment in the FTF versus the Internet versus mode. Even when we consider a whole host of factors that predict racial attitudes, such as education level and age, the effect of survey mode continues to persist. Our results also indicate that the tendency to underreport socially undesirable attitudes (in this case, negative racial sentiment) varies according to the degree to which one has been socialized about the racial norms in the United States.

As for alternative explanations for the types of mode effects that we document, it is possible that respondents taking online surveys may be more distracted than respondents in the FTF format, or in general they might be less attentive to the survey than when there is an interviewer present. While that is possible, distraction or inattentiveness should manifest in other types of survey response issues—high item nonresponse or straight-lining, for example. We see no reason to believe that distraction or inattention would lead to the systematic differences in reported attitudes or opinions, however. Moreover, it is hard to imagine how distraction or inattention would lead to different reports of attitudes or opinions between U.S.-born respondents and foreign-born Latinos.

These findings also help to shed further light on the heterogeneity in racial attitudes in the United States. Mode effects appear to exert the strongest effects on Whites and Blacks, whereas for Latinos, it seems to be less of an issue, particularly among Latinos who were born outside of the United States. It may be with time that Latinos will behave more like Whites and Blacks, and perceive more pressure to provide socially desirable survey responses on racial matters. On the contrary, due to the unique political socialization process that Latinos have experienced, along with the large foreign-born population, mode effects may be less of a concern in their reporting of socially undesirable attitudes. The lack of a mode effect among foreign-born Latinos, we contend, can be attributed to the fact that their socialization process is still in flux. As a result, becoming acquainted with the societal norms and expectations with regard to racial attitudes, in particular, is still being shaped and formed. It is also important to recognize that the patterns we uncover with respect to foreign-born Latinos may be biased in a conservative fashion, given that the ANES only surveys U.S. citizens. Undoubtedly, future work would be wise to assess mode effects on both citizens and noncitizens. Future research will also need to track the evolution of potential social desirability bias in surveys about race and ethnicity among Latinos in the United States.

This study also helps to reaffirm the powerful and enduring role that the racial hierarchy plays in structuring racial attitudes in the United States (Masuoka & Junn, 2013). One consistent finding that emerged from our analysis is that the public viewed Blacks and Latinos most negatively, with no negative perceptions directed toward Asians or Whites. These differential assessments held true for the entire sample of respondents, as well as when we disaggregated out sample by a respondent’s ethnic/racial identity. These findings underscore just how strongly embedded racial attitudes are in our society, and how difficult it is to alter racial stereotypes and perceptions, even when individuals have the opportunity to express themselves in a nearly anonymous fashion.

The existence of mode effects in the responses to questions about race in both the 2012 and 2016 ANES also offer some guidance for future studies. Given the widespread use of these studies, researchers should be aware of these mode effects and account for them in any studies that deal with highly sensitive question items. Despite the best efforts by the ANES to use tablets (CASI) to capture responses to sensitive survey items such as racial sentiment, our results indicate that the reporting of these attitudes still varied by survey mode. These findings suggest that interviewer effects are still present even when the interviewer is not asking the question; the mere presence of an interviewer is enough to pressure survey respondents to provide a response that is perceived as more socially acceptable and desirable.

Does this mean, then, that we should only ask highly sensitive questions on the Internet, using other types of self-completion survey modes? Our findings indeed suggest that surveys using FTF interviewers produce substantively different estimates of racial animus, as well as underestimating the effect of racial resentment on favorability toward Obama and the likelihood of voting for him. However, this is just a first step in an area ripe for future research. Subsequent studies would be well-served to further examine mode effects and the reporting of socially undesirable attitudes, and explore methodologies that allow applied researchers to eliminate the potential issues that may be introduced by survey mode (e.g., Blair & Imai, 2012; Rosenfeld, Imai, & Shapiro, 2016).

Supplemental Material

Appendix – Supplemental material for Answering Questions About Race: How Racial and Ethnic Identities Influence Survey Response

Supplemental material, Appendix for Answering Questions About Race: How Racial and Ethnic Identities Influence Survey Response by Marisa Abrajano and R. Michael Alvarez in American Politics Research

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Supplemental Material

Supplemental material for this article is available online.

Notes

Author Biographies

Marisa Abrajano is a professor of Political Science at UC San Diego.

R. Michael Alvarez is a professor of Political Science at California Institute of Technology.

References

Abrajano

Alvarez

R. M.

(2010). New faces, new voices: The Hispanic electorate in America. Princeton, NJ: Princeton University Press.

Abrajano

Elmendorf

Quinn

(2018). Labels vs. pictures: Treatment-mode effects in experiments about discrimination. Political Analysis, 26, 20-33.

Adorno

T. W.

Frenkel-Brunswick

Levinson

D. J.

Sanford

R. N.

(1950). The authoritarian personality. New York, NY: Harper & Row.

Allport

(1954). The nature of prejudice. Reading, MA: Addison-Wesley. http://projects.iq.harvard.edu/cces/data?dvn_subpage./faces/study/StudyPage.xhtml?globalId.hdl:1902.1/14003

Aquilino

W. S.

(1994). Interview mode effects in surveys of drug and alcohol use. Public Opinion Quarterly, 58, 210-240.

Atkeson

L. R.

Adams

A. N.

Alvarez

R. M.

(2014). Nonresponse and mode effects in self- and interviewer-administered surveys. Political Analysis, 22, 1-11.

Berinsky

A. J.

(1999). The two faces of public opinion. American Journal of Political Science, 43, 1209-1230.

Berinsky

A. J.

(2002). Political context and the survey response: The dynamics of racial policy opinion. Journal of Politics, 64, 567-584.

Blair

Imai

(2012). Statistical analysis of list experiments. Political Analysis, 20, 47-77.

10.

Bobo

(1983). Whites’ opposition to busing: Symbolic racism or realistic group conflict? Journal of Personality and Social Psychology, 45, 1196-1210.

11.

Bobo

Gilliam

. (1990). Race, Sociopolitical Participation, and Black Empowerment. American Political Science Review, 84(2), 377-393. doi:10.2307/1963525

12.

Campbell

Angus

Converse

Philip

Miller

Warren

Stokes

Donald

. (1960). The American Voter. Chicago: The University of Chicago Press.

13.

Crowne

D.P.

Marlowe

. (1960) . A new scale of social desirability independent of psychopathology. Journal of Consulting Psychology, 24(4), 349-354.

14.

Crowne

D.P.

Marlowe

. (1964). The Approval Motive. New York: John Wiley & Sons.

15.

Dawson

M.C

. (1994). Behind the Mule: Race and Class in African-American Politics. Princeton: Princeton University Press.

16.

DeSipio

Louis

. (1996). Making Citizens or Good Citizens? Naturalization as a Predictor of Organizational and Electoral Behavior among Latino Immigrants. Hispanic Journal of Behavioral Sciences, 18(2), 194-213. https://doi.org/10.1177/07399863960182007

17.

Garcia Bedolla

Lisa

. (2009). Latino politics. Cambridge, England: Polity.

18.

Gilens

Sniderman

P. M.

Kuklinski

J. H.

(1998). Affirmative action and the politics of realignment. British Journal of Political Science, 28, 159-183.

19.

Hainmueller

Hangartner

Yamamoto

(2015). Validating vignette and conjoint survey experiments against real-world behavior. Proceedings of the National Academy of Sciences of the United States of America, 112, 2395-2400.

20.

Homola

Jackson

Gill

(2016). A measure of survey mode differences. Electoral Studies, 44, 255-274.

21.

Huddy

Feldman

(2009). On assessing the political effects of racial prejudice. Annual Review of Political Science, 12, 423-447.

22.

Johnson

T. P.

Van de Vijver

F. J. R.

(2003). Social desirability in cross-cultural research. Cross-Cultural Survey Methods, 325, 195-204.

23.

Kim

(2003). Bitter fruit: The politics of Black-Korean conflict in New York City. New Haven, CT: Yale University Press.

24.

Kinder

D. R.

Sanders

L. M.

(1996). Divided by color: Racial politics and democratic ideals. Chicago, IL: University of Chicago Press.

25.

Kinder

D. R.

Sears

D. O.

(1981). Prejudice and politics: Symbolic racism versus racial threats to the good life. Journal of Personality and Social Psychology, 40, 414-431.

26.

Kreuter

Presser

Tourangeau

. (2008). Social desirability bias in CATI, IVR, and Web Surveys: The Effects of Mode and Question Sensitivity. Public Opinion Quarterly, 72(5), 847-865.

27.

Krysan

Couper

Mick P

. (2003). Race in the Live and the Virtual Interview: Racial Deference, Social Desirability, and Activation Effects in Attitude Surveys. Social Psychology Quarterly, 66(4): 364-383.

28.

Kuklinski

J. H.

Sniderman

P. M.

Knight

Piazza

Tetlock

P. E.

Lawrence

G. R.

Mellers

(1997). Racial prejudice and attitudes toward affirmative action. American Journal of Political Science, 41, 402-419.

29.

Masuoka

Junn

(2013). The politics of belonging: Race, immigration, and public opinion. Chicago, IL: University of Chicago Press.

30.

Mendelberg

. (2001). The Race Card: Campaign Strategy, Implicit Messages, and the Norm of Equality. Princeton, NJ: Princeton University Press.

31.

Okazaki

(2000). Asian American and White American differences on affective distress symptoms. Journal of Cross-Cultural Psychology, 31, 603-625.

32.

Piston

(2010). How explicit racial prejudice hurt Obama in the 2008 election. Political Behavior, 32, 431-451.

33.

Rosenfeld

Imai

Shapiro

(2016). An empirical validation study of popular survey methodologies for sensitive questions. American Journal of Political Science, 60, 783-802.

34.

Sanchez

Gabriel

Chris Garcia

. (2008). Hispanics and the U.S. Political System: Moving into the Mainstream. New York: Routledge.

35.

Sears

D. O.

Sidanius

Bobo

L. D.

(2000). Racialized politics: The debate about racism in America. Chicago, IL: University of Chicago Press.

36.

Tesler

(2012). The spillover of racialization into health care: How President Obama polarized public opinion by racial attitudes and race. American Journal of Political Science, 56, 690-704.

37.

Tesler

Sears

(2010). Obama’s race: The 2008 election and the dream of a post-racial America. Chicago, IL: University of Chicago Press.

38.

Tourangeau

Smith

T. W.

(1996). Asking sensitive questions: The impact of data collection mode, question format and question context. Public Opinion Quarterly, 60, 275-304.

39.

Tourangeau

Yan

. (2007). Sensitive questions in surveys. Psychological Bulletin, 133(5), 859-883.

40.

Vislavanich

(2016). Asian candidates in America: The surprising effects of positive racial stereotyping. Political Research Quarterly, 70, 68-81. doi:10.1177/1065912916674273

41.

Warnecke

R. B.

Johnson

T. P.

Chávez

Sudman

O’Rourke

D. P.

Lacey

Horm

(1997). Improving question wording in surveys of culturally diverse populations. Annals of Epidemiology, 7, 334-342.

42.

White

Laird

Allen

. (2014). Selling Out?: The Politics of Navigating Conflicts between Racial Group Interest and Self-interest. American Political Science Review, 108(4), 783-800. doi:10.1017/S000305541400046X

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.15 MB