Cognitive Reflection and the 2016 U.S. Presidential Election

Abstract

We present a large exploratory study (N = 15,001) investigating the relationship between cognitive reflection and political affiliation, ideology, and voting in the 2016 Presidential Election. We find that Trump voters are less reflective than Clinton voters or third-party voters. However, much (although not all) of this difference was driven by Democrats who chose Trump. Among Republicans, conversely, Clinton and Trump voters were similar, whereas third-party voters were more reflective. Furthermore, although Democrats/liberals were somewhat more reflective than Republicans/conservatives overall, political moderates and nonvoters were least reflective, whereas libertarians were most reflective. Thus, beyond the previously theorized correlation between analytic thinking and liberalism, these data suggest three additional consequences of reflectiveness (or lack thereof) for political cognition: (a) facilitating political apathy versus engagement, (b) supporting the adoption of orthodoxy versus heterodoxy, and (c) drawing individuals toward candidates who share their cognitive style and toward policy proposals that are intuitively compelling.

Keywords

political ideology 2016 election cognitive reflection intuition dual process theory

That experience taught me a few things. One is to listen to your gut, no matter how good something sounds on paper.

—Donald Trump (1987, p. 58)

Many have claimed that one of the core cognitive differences between conservatives and liberals—at least in the Western context—is that conservatives tend to rely more on their intuitions and gut feelings than liberals (Deppe et al., 2015; Eidelman, Crandall, Goodman, & Blanchar, 2012; Haidt, 2012; Jost, 2017; Jost, Glaser, Kruglanski, & Sulloway, 2003; Talhelm et al., 2015). However, support for this claim is often indirect, and data showing a negative correlation between conservative political ideology and behavioral measures of analytic thinking are equivocal (e.g., Kahan, 2013). As a consequence, the impact of individual differences in reliance on intuitive versus analytic thinking for political behavior in any particular context or at any particular time is unclear. Here, we shed new light on political cognition by focusing on the 2016 U.S. Presidential Election and by systematically investigating the correlation between individual differences in analytic thinking and political behavior (voting), political party affiliation, and political ideology in a large scale exploratory analysis. Our findings indicate that the distinction between intuitive and analytic thinking has complex implications for political behavior and ideology.

Dual Process Theory

A core claim about human cognitive architecture is that we are capable of generating two different types of cognitive outputs (Evans, 2008; Evans & Stanovich, 2013; Kahneman, 2011; Stanovich, 2005): one that emerges from automatic intuitive responses (“Type 1”) and one that emerges from deliberative or analytic thinking processes (“Type 2”). This “dual process” distinction has had wide-reaching implications for psychological science. Dual process theories have been applied to reasoning (Evans, 1989; Sloman, 1996; Stanovich & West, 2000), decision making (Barbey & Sloman, 2007; Kahneman & Frederick, 2005; Rand, 2016), social cognition (Chaiken & Trope, 1999; Epstein, Pacini, Denes-Raj, & Heier, 1996), cognitive development (Barrouillet, 2011; Klaczynski, 2001), evolutionary game theory (Bear & Rand, 2016; Jagau & van Veelen, 2017; Rand, Tomlin, Bear, Ludvig, & Cohen, 2017), and clinical disorder (Beevers, 2005; Pyszczynski, Greenberg, & Solomon, 1999), among others (see Evans, 2008 for a review).

An important consequence of the distinction between intuitive and deliberative processes is that, at least to some extent, analytic reasoning is discretionary. That is, some responses come to mind automatically (which serve as defaults), and one may or may not reason analytically about these initial intuitive outputs (Pennycook, 2017; Pennycook, Fugelsang, & Koehler, 2015b). Consider, for example, the following (now famous) bat and ball problem from the cognitive reflection test (CRT; Frederick, 2005):

A bat and ball cost $1.10 in total. The bat costs $1.00 more than the ball.

How much does the ball cost?

The response that comes to mind intuitively for most people on this problem is “10 cents,” which is the modal response (Campitelli & Gerrans, 2014; Pennycook, Cheyne, Koehler, & Fugelsang, 2016). Naturally, however, “10 cents” is not the correct answer (if the ball costed 10 cents, the bat would have to cost $1.10 and they would cost $1.20 in total).

The bat and ball problem is of particular interest because accuracy tends to be quite low (usually around 30%, depending on the sample; Frederick, 2005), even though only basic arithmetic is required to recognize that 10 cents is incorrect. Typically, to answer the problem correctly, one must reflect on an intuitively appealing response—an analytic process that is evidently not particularly common (Stanovich, 2005). It is for this reason that problems of this nature—specifically, those that cue an incorrect intuitive response—are thought to reflect (to some important degree) the propensity or willingness to engage analytic thinking (Pennycook, Fugelsang, & Koehler, 2015a; Pennycook & Ross, 2016; Toplak, West, & Stanovich, 2011). This propensity to think analytically—sometimes referred to as analytic cognitive style (Pennycook, Cheyne, Seli, Koehler, & Fugelsang, 2012)—is distinct from (although functionally related to) the capacity to think analytically (i.e., intelligence or cognitive ability; Stanovich, 2009, 2012; Stanovich & West, 2000), although both factors are important for the analytic thinking that is required to overcome intuitions.¹ Indeed, recent research has shown that variation in analytic thinking, as measured by performance on the CRT, correlates with a wide range of psychological factors, such as religious belief (Pennycook et al., 2012; Pennycook, Ross, Koehler, & Fugelsang, 2016; Shenhav, Rand, & Greene, 2012), various epistemically suspect beliefs (Browne, Thomson, Rockloff, & Pennycook, 2015; Gervais, 2015; Pennycook, Cheyne, Barr, Koehler, & Fugelsang, 2015; Shtulman & Mccallum, 2014; Swami, Voracek, Stieger, Tran, & Furnham, 2014), and moral judgments, values, and behavior (Arechar, Kraft-Todd, & Rand, 2017; Paxton, Ungar, & Greene, 2012; Pennycook, Cheyne, Barr, Koehler, & Fugelsang, 2014; Royzman, Landy, & Goodwin, 2014; Royzman, Landy, & Leeman, 2015), among others (see Pennycook, Fugelsang, & Koehler, 2015a for a review). Thus, individual differences in analytic cognitive style bridges together a wide range of psychological factors and evidences a broad dual process view of human cognition.

Political Ideology and Analytic Thinking

Political ideology represents an area of particular contention in the context of individual differences in analytic thinking, both theoretically and empirically. As a strong contrast to the perspective offered above, Kahan (2013) has argued that the primary role of analytic thinking is not to inform beliefs, behaviors, and ideologies but rather to reinforce them (see also, Haidt, 2001, 2012). That is, individuals typically reason more like lawyers (who use reasoning to convince others—and themselves—that they are correct) than philosophers (who use reason to get closer to the truth). Under this account, one should not expect analytic thinking to have a directional effect on political ideology such that one group is more analytic than the other. Rather, analytic thinking is used to engage in motivated reasoning and to protect one’s identity when challenged, such that more analytic individuals are expected to be more polarized (Kahan, Peters, Dawson, & Slovic, 2017; Kahan et al., 2012).

There are, in contrast, a suite of theories that do predict ideological differences in analytic thinking. Talhelm et al. (2015), for example, argue that liberals should be more analytic because they come from a more individualistic culture that is less focused on social bonds (which are facilitated by intuitive or holistic thinking). Jost (2017) argues that conservatism emerges from a need to manage threat and, in support of this contention, provides evidence from a series of meta-analyses that find liberals are more tolerant of uncertainty, less dogmatic, less cognitively rigid, have less need for order, and (more generally) are more disposed toward reflective thought (see also, Hibbing, Smith, & Alford, 2014; Jost et al., 2003). Finally, Eidelman et al. (2012) argue that the conservative emphasis on personal responsibility, acceptance of hierarchy, and preference for the status quo are facilitated by reliance on intuition instead of reason. Support for these theories would be undermined by a lack of correlation between political ideology and CRT performance. This is particularly the case given that the CRT is a behavioral measure of analytic thinking and is therefore more externally valid than the self-report measures that are typically used in this research (e.g., individuals who are intuitive often claim that they are analytic; Pennycook, Ross, Koehler, & Fugelsang, 2017)

The empirical evidence for a negative association between analytic thinking and conservative political ideology is just as contentious as the theories surrounding the association. Indeed, the first study that reported a small negative correlation (r = −.16) between conservative political ideology and CRT performance (among Americans) failed to replicate this finding in a second study using a more international sample (Pennycook et al., 2012). Subsequently, Iyer, Koleva, Graham, Ditto, and Haidt (2012) found, based on a single self-identification item, that liberals scored higher on the CRT than conservatives (but that libertarians scored the highest) in a large sample of American individuals (N = 9721) who signed up to participate in a psychology study on YourMorals.org. However, Kahan (2013) found that Republicans actually scored higher than Democrats using a large (N = 1,750) representative panel of Americans from YouGov but did not find a significant correlation with overall conservatism (using a Likert-type scale). Piazza and Sousa (2013) also failed to find a significant correlation between cognitive reflection and conservative political ideology. Moreover, a recent meta-analysis found that conservatives and liberals were just as prone to partisan bias in motivated reasoning experiments (Ditto et al., 2018).

More recent studies suggest that the distinction between social and economic political ideology is crucial: Whereas social conservatism is defined by opposition toward issues that pertain to social change (e.g., abortion, gay marriage, etc.), economic conservatism pertains to support for the free market and capitalism. More recent studies found that analytic thinking often correlates with social but not economic conservatism (Deppe et al., 2015; Pennycook et al., 2014; Saribay & Yilmaz, 2017; but see Sterling, Jost, & Pennycook, 2016; Saribay & Yilmaz, 2017c), including in a Turkish sample (Yilmaz & Saribay, 2016). Moreover, reliance on intuition is particularly strongly associated with conservative moral values (Deppe et al., 2015; Pennycook et al., 2014; Yilmaz & Saribay, 2017a), which pertain to social conservative issues.

Current Study

As summarized above, there is a great deal of contention surrounding the common claim that conservatives are more intuitive and less analytic than liberals in the United States. Moreover, the overwhelming majority of past work has focused on political ideology and attitudes, whereas to our knowledge, only one study (Kahan, 2013) reported differences based on party affiliation (showing the opposite result as would be expected, with Republicans relying less on intuition than Democrats). Even more importantly, no previous work has investigated the potential role of analytic thinking in political behavior. To this end, we report a large aggregate analysis of 15,001 participants from 19 studies completed since the 2016 U.S. Presidential Election (specifically, between December 2016 and November 2017; all on Mechanical Turk).² Along with the CRT, participants in every study completed a suite of political measures, including party affiliation, political ideology, and an identification of who they voted for (or if they voted) in the 2016 election. This large sample allowed us not only to compare liberals and conservatives on various measures but also to investigate the interaction between political party affiliation and political behavior.

Method

Participants

Across the 19 studies, we only retained participants for which CRT and voting behavior data were available. This left us with 16,650 participants. However, there were 1,619 participants who completed more than one study³ (based on their MTurk ID), and we only retained the first instance. A further 30 participants were removed because they did not enter a valid MTurk ID. The final sample therefore consisted of 15,001 participants (56.3% females; M_age = 35.3, SD_age = 11.3).

Materials

There were a variety of measures included across the 19 studies; here, we focus solely on the measures of interest (and which were present in all 19 studies). The original purpose of each study was to investigate various factors relating to fake and real news (Pennycook, Cannon, & Rand, 2018; Pennycook & Rand, 2017; 2018a; 2018b)—as a consequence, participants always read and rated the accuracy of (and/or willingness to share) news article headlines (which varied from study to study) prior to completing the measures of present interest. Measures were administered via Qualtrics’ survey software and in the order that they are outlined here.

Cognitive reflection test

We used a seven-item CRT: the original three-item CRT (Frederick, 2005), but reworded slightly (the mathematical structure was maintained; Shenhav et al., 2012; Shenhav, Rand, & Greene, 2017), and a less math-focused version from Thomson and Oppenheimer (2016). The CRT has been shown to predict a number of factors even after taking numeracy (Pennycook, Fugelsang, & Koehler, 2015a; Pennycook & Ross, 2016) or cognitive ability (Shenhav et al., 2012; Toplak et al., 2011; Toplak, West, & Stanovich, 2014) into account, although performance reflects both cognitive ability and cognitive style (i.e., “analytic thinking,” broadly). Recent research indicates that prior exposure to the CRT does not undermine its predictive validity (Bialek & Pennycook, 2017; Meyer, Zhao, & Frederick, 2018; Stagnaro, Pennycook, & Rand, 2018). The two versions were strongly correlated, r(14999) = .50, and the full seven-item CRT had acceptable reliability, Cronbach’s alpha = .75. The results were highly similar when analyzing the two versions of the CRT separately. We scored the CRT based on the number of correct answers as opposed to the number of incorrect intuitive answers (Pennycook, Cheyne, et al., 2016), but the results were highly similar regardless of the scoring strategy.

Demographics and political questions

The political questions of interest were presented following (and on the same page as) standard demographic questions (namely age, gender, education level, and English proficiency). For education level, participants were asked, “What is the highest level of school you have completed or the highest degree you have received?” and given the following options: less than high school degree, high school graduate (high school diploma or equivalent including GED), some college but no degree, associate degree in college (2 years), bachelor’s degree in college (4 years), master’s degree, doctoral degree, professional degree (JD, MD). For analysis purposes, we created dummy variables for no college degree (less than high school, high school, and some college but no degree) and college degree (associate or bachelor’s).

Participants were then asked, “Which of the following best describes your political position?” and given the following options: Democrat, Republican, Independent, Other (specify). This was followed with two political ideology measures (one study did not include these questions): (a) “On social issues I am:” and (b) “On economic issues I am:” Both were followed by a 5-point Likert-type scale with the following options: strongly liberal, somewhat liberal, moderate, somewhat conservative, and strongly conservative. Voting behavior was then measured using the following question: “Who did you vote for in the 2016 Presidential Election? Reminder: This survey is anonymous.” The following response options were provided: Hillary Clinton, Donald Trump, Other candidate (such as Jill Stein or Gary Johnson), I did not vote for reasons outside of my control, I did not vote but I could have, and I did not vote out of protest. Finally, participants were asked to choose between Clinton and Trump: “If you absolutely had to choose between only Clinton and Trump, who would you prefer to be the next [current] President of the United States?” (Hillary Clinton or Donald Trump).

Although we did not collect data on ethnicity or income in any of the studies, we were able to obtain this information for a subset of the sample (N = 8,226) by matching MTurk IDs of our participants with ethnicity and income data collected in other large studies.⁴ For ethnicity, individuals self-selected from nine options: American Indian/Alaska Native, Black/African American, East Asian American (e.g., China, Japan, Korea), European American/White, Hispanic/Latino, Middle Eastern (e.g., Saudi Arabia, Iran, Persian Gulf), Native Hawaiian or other Pacific Islander, South Asian American (e.g., India, Pakistan, Bangladesh), and Other. For analysis purposes, we created dummy variables for Caucasian, African American, and Hispanic. Household income data (before taxes) were curated from two sources with different response options, so we combined the questions into four categories: less than US$10,000, US$10,001-US$50,000, US$50,001-US$100,000, and greater than US$100,000. For analyses purposes, dummy variables for the first three income levels were used.

Results

A demographic breakdown for the full range of voting responses can be found in Table 1. Means and standard deviations can be found in the appendix. Data are available at the following link: https://osf.io/kshu7/.

Table 1.

Demographic Information and Sample Size for 2016 U.S. Presidential Election Voting Responses.

2016 U.S. Presidential Election vote	N (%)	M_age (SD)	% of females	College/postgraduate degree	< $50k income
Hillary Clinton	5,938 (39.6)	35 (11)	61.5	71.8%	46.5%
Donald Trump	3,757 (25)	49 (12)	55.3	62.9%	45%
Other candidate	1,832 (12.2)	34 (10)	48.2	67%	45.7%
DNV: reasons outside of control	925 (6.2)	32 (10)	60.3	55%	50.9%
DNV: but could have	1,764 (11.8)	32 (10)	51.5	49%	52.8%
DNV: out of protest	785 (5.2)	33 (10)	46.4	53.1%	54.4%

Note. DNV = did not vote.

Political Behavior

We first compared mean CRT accuracy across the full range of voting responses (see Figure 1), using a one-way ANOVA. This revealed significant variation in CRT scores, F(5, 14995) = 40.62, mean square error (MSE) = .08, p < .001, η² = .013. This was also true after entering age, gender, and education as covariates, F(5, 14825⁵) = 36.78, MSE = .08, p < .001, η² = .012. We also entered income and ethnicity as covariates—using the smaller subset of the data for which these measures were available—alongside age, gender, and education; the overall difference remained significant, F(5, 7863) = 22.61, MSE = .08, p < .001, η² = .014. A post hoc Tukey’s honest significant difference (HSD) test comparing CRT scores across the different levels of voting responses (on the full data set without covariates) isolated two homogeneous subsets (p < .05) and indicated that CRT scores were equivalent among individuals who either voted for Trump or did not vote (for any reason) and that these scores were lower than scores for those who voted for either Clinton or a third-party candidate (who were equivalent). The effect size for this difference (Trump/nonvote: M = .46, SD = .28; Clinton/third party: M = .52, SD = .29) was small, d = .23, r = .11, but significant given our large sample size, p < .001.⁶ As a point of reference, effect sizes of r = .11, .19, and .29 correspond to the 25th, 50th, and 75th percentiles in terms of average effect sizes in individual differences research in psychology (Gignac & Szodorai, 2016).

Figure 1.

Mean CRT accuracy (0-1) as a function of post hoc report of 2016 U.S. Presidential Election vote.

Political Affiliation

In a parallel analysis, we compared CRT scores as a function of party affiliation (Figure 2). This, too, revealed significant variation in CRT scores, F(3, 14953) = 20.81, MSE = .08, p < .001, η² = .004. Again, this maintained after entering age, gender, and education as covariates, F(3, 14787) = 19.41, MSE = .08, p < .001, η² = .004, as well as in the smaller sample with income and ethnicity entered as additional covariates, F(3, 7843) = 21.67, MSE = .08, p < .001, η² = .008. A Tukey’s HSD test found that Republicans scored the lowest but that the Democrats, Independents, or “Others” did not differ. As with voting behavior, this effect size was small (Republicans: M = .46, SD = .28; everyone else: M = .50, SD = .29), d = .15, r = .08, but significant given our large sample size, p < .001.⁷

Figure 2.

Mean CRT accuracy (0-1) as a function of party affiliation. Error bars represent 95% confidence intervals.

Interaction Between Behavior and Affiliation

As evident from Table 2, these analyses are not entirely redundant: Although a strong majority voted along party lines (74.9% of Democrats and 72.3% of Republicans), some Democrats voted for Trump (3.8%) and some Republicans voted for Clinton (5.3%). There are also a considerable number of Independents in the sample. Fortunately, due to our large sample, there are enough individuals in each of these cells to compare CRT scores across the full range of voting behavior and party affiliation (for ease of exposition, we drop those who did not affiliate as Democrat, Republican, or Independent and collapse across the three categories of nonvoters). For this analysis, we entered mean CRT accuracy as a dependent variable in a 3 × 4 univariate ANOVA with party affiliation (Democrat, Republican, and Independent) and voting behavior (Clinton, Trump, Other, or did not vote) as separate factors. This revealed a significant interaction between party affiliation and voting behavior in CRT performance (see, Figure 3) F(6, 14298) = 8.16, MSE = .08, p < .001, η² = .003. The interaction continued to be significant after entering age, gender, and education as covariates, F(6, 14143) = 9.46, MSE = .08, p < .001, η² = .004, as well as with income and ethnicity in the smaller sample, F(6, 7551) = 3.64, MSE = .08, p = .001, η² = .003.

Table 2.

Number of Individuals in the Data Set Who Voted for Clinton, Trump, or Other or Who Did Not Vote as a Function of Party Affiliation.

	Party affiliation
	Democrat	Republican	Independent	Total
Hillary Clinton	4,311	179	1,312	5,802
Donald Trump	216	2,455	997	3,668
Other candidate	306	249	1,091	1,646
Did not vote	919	513	1,762	3,194
Total	5,752	3,396	5,162	14,310

Figure 3.

Mean CRT accuracy (0-1) as a function of the interaction between voting behavior and party affiliation.

To further explore this interaction, we compared CRT scores across the four levels of voting behavior separately for each party. In all three cases, there was significant variability in CRT performance as a function of voting behavior, all Fs > 11, ps < .001. However, different homogeneous subsets (based on post hoc Tukey’s HSD tests, p < .05) emerged in each case.

For Democrats, there were no differences in CRT performance between those who voted for Clinton, voted for a third-party candidate, or did not vote, whereas those who voted for Trump scored significantly lower than the other three groups. Comparing Trump voters with all other voter categories (among Democrats) in a regression with age, gender, and education as controls also produced a significant difference, r = −.10, β = −.10, p < .001 (and with ethnicity and income as additional controls in the smaller sample, r = −.08, β = −.08, p < .001).

Among Republicans, the pattern was much different. A Tukey’s HSD test revealed that Republicans who voted for a third-party candidate scored higher on the CRT than any other group. There were no differences between Clinton voters, Trump voters, and nonvoters. Comparing third-party voters with all other voter categories (among Republicans) in a regression with age, gender, and education as controls also produced a significant difference, r = .10, β = .10, p < .001 (and with ethnicity and income as additional controls in the smaller sample, r = .12, β = .11, p < .001).

Finally, among Independents, there were two homogeneous subsets: (a) nonvoters and Trump voters, who had lower scores, and (b) Clinton and third-party voters, who had higher scores. Comparing Trump and nonvoters with Clinton and third-party voters (among Independents) in a regression with age, gender, and education as controls also produced a significant difference, r = −.12, β = −.11, p < .001 (and with ethnicity and income as additional controls in the smaller sample, r = −.09, β = −.08, p < .001).

To give a sense of the magnitude of these differences, the distribution of CRT scores for each distinct group indicated by the Tukey’s HSD tests is shown in Figure 4. As can be seen, Democrats who voted for Trump stand out relative to the other groups. For example, Democrats who voted for Trump were 2 to 3 times more likely than the other groups to answer all seven CRT questions incorrectly and half as likely as other Democrats to answer five, six, or seven questions correctly. Thus, much of the overall lower CRT scores for Trump voters relative to Clinton voters can be attributed to the Democrats who voted for Trump. Nonetheless, it should be noted that Democratic affiliated Clinton voters do score higher on the CRT (M = .51) than Republican affiliated Trump voters (M = .45), t(6764) = 9.08, SE = .01, p < .001, d = .23.

Figure 4.

Distribution of CRT scores for each subgroup identified by Tukey’s HSD tests.

The finding that CRT performance is markedly worse among Democrats who voted for Trump is further emphasized by Figure 5, which compares Democrats who voted for Trump to the aggregation of all other people. As with the other analyses, the differences between Democrats who voted for Trump and everyone else cannot be explained by differences in age, gender, and education—including those variables as covariates in a regression did not appreciably change the estimated difference in mean CRT sores between groups (13.5 percentage points lower mean CRT accuracy without controls, 13.7 percentage points lower mean CRT accuracy including controls). Furthermore, it seems unlikely that Democrats who indicted voting for Trump were merely careless and had meant to select Clinton instead—in addition to voting for Trump, they were also significantly more conservative than those who indicated voting for Clinton, social conservatism: t(3596) = 14.24, p < .001; economic conservatism: t(3692) = 10.25, p < .001, and significantly less educated, χ²(7, N = 4521) = 37.33, p < .001 (although as noted above, these demographic differences do not account for the difference in CRT scores observed).

Figure 5.

Distribution of CRT scores for Democrats who voted for Trump (red) compared to all other participants (purple).

Political Ideology

Next, we turn to associations with political ideology, as measured by separate single-item social and economic conservatism Likert-type scales. Individuals who had missing data for one or both of the political ideology questions (N = 2,959) were removed from the data, leaving a sample of N = 12,042. First, replicating findings from previous research (Pennycook et al., 2014; Yilmaz & Saribay, 2017b), CRT performance was significantly negatively correlated with social conservatism, r(12042) = −.15, p < .001, but only trivially correlated with economic conservatism (albeit significantly, due to our large sample size), r(12042) = −.02, p = .031.

However, these analyses average across important differences in American political ideology—most notably, libertarians who endorse economic conservatism but social liberalism. Indeed, when entering both social and economic conservatism into a multiple regression, the correlation between CRT performance and economic conservatism is significantly positive, β = .16, p < .001, and the correlation between social conservatism and CRT performance becomes more strongly and significantly negative, β = −.26, p < .001.

To illustrate the underlying source of these relationships, we computed novel analyses of CRT differences as a function of political ideology on social and economic issues. For this, we created four groups of interest (representing 81.5% of the sample): (a) classic liberals who identify as “somewhat” or “strongly” liberal for both social and economic issues (N = 4,020), (b) classic conservatives who identify as “somewhat” or “strongly” conservatives for both social and economic issues (N = 2,457), (c) libertarians who identify as (somewhat/strongly) liberal on social issues but (somewhat/strongly) conservative on economic issues (N = 1,221), and (d) Individuals who identified as moderate on both social and fiscal issues (N = 2,154). The remaining group, individuals who identify as fiscally liberal but socially conservative, were not represented in sufficient numbers (just 1.2% of the sample) to justify inclusion as a clear political category (this matches nationally representative data from a 2012 Gallup Poll in which only 1% of the American population identified as such; Jones, 2012). This analysis also excludes difficult-to-classify individuals who selected moderate on one but not both types of issues (17% of the sample).

Following the same analysis plan as above, there was significant variability in CRT scores across the four key political categories, F(4, 9852) = 160.44, MSE = .08, p < .001, η² = .05. As evident from Figure 6, consistent moderates scored the lowest and libertarians scored the highest.⁸ In parallel with the overall voting behavior and party affiliation analyses presented above, liberals scored higher than conservatives. A Tukey’s HSD test revealed that all four groups were significantly different from each other, ps < .001.

Figure 6.

Mean CRT accuracy (0-1) as a function of ideological category.

In addition to these results on the directional relationship between CRT and political ideology, we also consider the correlation between CRT and ideological extremity. This investigation is motivated in part by previous work showing that individuals who are more analytic tend to be more politically polarized on specific issues (e.g., climate change risk; Kahan et al., 2012)—although it should be noted that this line of work does not make clear predictions regarding the relationship between CRT and political extremity as the theory typically takes one’s ideology as a given and then stipulates that analytic thinking is used to justify that ideology. As can be seen in Figure 6, we find that politically engaged individuals (be they liberal or conservative) are more analytic than political moderates. To gain further insight into political extremity, we recoded the social and economic conservatism measures to reflect extremity of position: that is, those who indicated being “strongly” conservative/liberal were given a 2, those who indicated being “somewhat” conservative/liberal were given a 1, and those who indicated being “moderate” were given a 0. Using this measure, CRT performance was modestly positively correlated with extremity for both social issues, r(12042) = .14, p < .001, and economic issues, r(12042) = .08, p < .001. However, as is evident from Figure 7, these correlations with extremity are driven almost entirely by political moderates, scoring lower than either those with somewhat or strong ideological commitments rather than an increase in CRT moving from somewhat liberal/conservative to strongly liberal/conservative. The only increase in CRT performance as a function of political extremity beyond the “moderate” category is the difference between “somewhat” liberals (M = .50, SD = .28) and “strong” liberals (M = .53, SD = .29) on social issues, t(9081) = 5.11, SE = .006, p < .001. Thus, on balance, CRT performance appears to be mostly linked to an absence of partisanship (i.e., political indifference) rather than to a presence of political polarization or strong partisanship in political ideology.

Figure 7.

Mean CRT accuracy (0-1) as a function of strength of political ideology on both social and economic issues.

Interaction Between Behavior and Ideology

Finally, as a robustness check on the interaction between party affiliation and voting behavior, we completed a parallel analysis to that reported above, but using the four political ideology categories in Figure 6 instead of political party. This revealed a significant interaction between political ideology and voting behavior (Figure 8), F(9, 9836) = 6.62, MSE = .08, p < .001, η² = .006. The interaction remained significant after entering age, gender, and education as covariates, F(9, 9761) = 7.51, MSE = .07, p < .001, η² = .007, and after also entering income and ethnicity in the smaller sample, F(9, 4723) = 2.54, MSE = .08, p = .007, η² = .005. We compared CRT scores between the four levels of voting behavior separately for each ideological category. In all four cases, there was significant variability in CRT performance as a function of voting behavior, all Fs > 4.9, ps < .003. Again, however, different homogeneous subsets (based on post hoc Tukey’s HSD tests, p < .05) emerged in each case. The pattern of results based on the Tukey’s HSD tests was identical for liberals as it was for Democrats (i.e., there were no differences in CRT performance between those who voted for Clinton, who voted a third-party candidate, or who did not vote, whereas those who voted for Trump scored significantly lower than the other three groups) and for conservatives as it was for Republicans (i.e., those who voted for a third-party candidate scored higher on the CRT than any other group, which did not differ). The pattern for moderates was the same as it was for Republicans and conservatives: Those who voted for a third-party candidate scored higher than all other groups (which did not differ). Finally, among libertarians, those who did not vote scored the lowest (but they did not significantly differ from Trump voters). Clinton voters scored higher than nonvoters but not higher than Trump voters, and third-party candidate voters scored the highest but not significantly higher than Clinton voters.

Figure 8.

Mean CRT accuracy (0-1) as a function of the interaction between voting behavior and ideological category.

General Discussion

In a sample of 15,001 participants gathered in the year following the 2016 U.S. Presidential Election, we found strong evidence that reliance on intuition is correlated with political affiliation, ideology, and behavior. The pattern of results paints a complex picture which goes beyond the common claim that conservatives are more intuitive and less analytic than liberals.

The largest differences in performance on the CRT emerged when investigating the interaction between political opinions and political behavior. Most notably, Trump voters were less analytic overall—and this was particularly true for Democrats who voted for Trump. Although this only represented a small minority (3.8% of Democrats voted for Trump in our sample), these individuals scored substantially lower than the remainder of the sample (see Figure 5). Among Republicans, there was no difference between Clinton and Trump voters, but those who voted for a third-party candidate were the most analytic. As a consequence, although liberals tended to be more analytic than conservatives overall (see Figure 6), Republicans who voted for a third-party candidate scored 20% higher than Democrats who voted for Trump (d = .71; this was the largest difference in the sample based on party affiliation, see Figure 3). Furthermore, individuals who did not vote and/or who are politically “moderate” tended to be particularly intuitive, whereas those who hold less mainstream positions—either by identifying as libertarian or voting for a third-party candidate—tended to be particularly analytic.

Theoretical Implications

Liberalism versus conservatism

The present results are consistent with dominant accounts of political cognition—at least as they pertain to the role of analytic thinking in the formation and retention of political attitudes and behavior—but also demonstrate the limitations and incompleteness of these accounts. In particular, we do observe an overall negative correlation between the propensity to think analytically and conservative political ideology: CRT scores were (a) higher among individuals who voted for Clinton relative to those who voted for Trump, (b) higher among Democrats than Republicans (in contrast to Kahan, 2013), and (c) negatively correlated with social (but not economic) conservatism. Moreover, with respect to social issues, “strong” liberals scored higher on the CRT than “somewhat” liberals, suggesting a positive association between liberalism and analytic thinking even among liberals. These associations align with the broad claim that conservatism arises from reliance on intuitive thinking (Eidelman et al., 2012; Jost, 2017; Talhelm et al., 2015).

However, we also observe various exceptions to the overall tendency of liberals to think more analytically than conservatives. For example, the highest CRT subgroup in our affiliation-based analyses were Republicans who voted for third-party candidates and in our ideology-based analyses were libertarians. Thus, it is clearly not the case that being conservative necessitates relying on intuitive thinking—the intuitive conservatism account is not the full story. Instead, we propose that there are three additional ways in which analytic thinking may affect political attitudes and behavior: (a) apathy versus engagement, (b) orthodoxy versus heterodoxy, and (c) cognitive match versus mismatch with candidate/platform.

Apathy versus engagement

Individuals who did not vote (for any reason) scored lower on the CRT than people who voted. Moreover, individuals who identified consistently as politically moderate (i.e., they did not identify as liberal or conservative for either social or economic issues) scored lower than liberals, conservatives, and libertarians. Finally, although political extremity (in the context of social and economic political ideology) was positively correlated with CRT performance, this correlation was primarily driven by low CRT scores among political moderates. These results suggest that one way in which analytic thinking affects political attitudes and behavior is that thinking analytically undermines political apathy (and facilitates interest and engagement in political issues). This observation resonates somewhat with the claim that highly analytic individuals are more politically polarized because they are better able to reason in a motivated way (Kahan, 2013; Kahan et al., 2017). However, our results suggest that when it comes to overall ideology (as opposed to positions on specific issues), analytic thinking’s role may largely be in overcoming political indifference rather than facilitating extreme political partisanship (see also Sidanius & Lau, 1989). This observation is also consistent with recent research in which CRT performance was associated with the ability to discern between fake and real news, regardless of whether the news headlines aligned with one’s political ideology (Pennycook & Rand, 2018b). Future research should investigate this issue using measures of political partisanship that can distinguish between those with strong opinions and those who are intensely partisan.

Orthodoxy versus heterodoxy

Libertarians and individuals who voted for third-party candidates tended to score higher on the CRT than other groups. These individuals hold what can be viewed as heterodox positions: They have eschewed the two-party dichotomy and took up an alternative position. Thus, akin to accounts of the role of analytic thinking in religious disbelief (Pennycook et al., 2012; Pennycook, Tranel, Warner, & Asp, 2018), one possibility is that individuals who are more analytic are more likely to shift away from whatever political position they emerged with from childhood and adolescence (as has been shown for religious belief, Shenhav et al., 2012). Given that being a “Democrat” or “Republican” (or voting for a primary party candidate) is more often the default position (and eschewing these positions presumably relies on analytic thinking), those who hold an alternative stance (or vote in an alternative way) are on average more analytic. Nonetheless, it should be noted that we have no information in the present sample about familial or communal political ideology—rather, we are making inferences assuming a binary Democrat/Republican political default. Longitudinal studies that track changes in political attitudes over time are necessary to firmly evidence this account.

Cognitive match versus mismatch with candidate/platform

Although the heterodoxy mechanism explains some of our data, it does not explain perhaps our most striking findings: The particularly low CRT scores among Democrats (and liberals) who voted for Trump. However, it should be noted that this observation was in some sense mirrored by the particularly high CRT scores for Republicans (and conservatives) who voted for third parties instead of Trump. One possibility, then, is that Trump, and the campaign that he ran, may have been particularly attractive for relatively intuitive individuals and repellent for relatively analytic individuals. We speculate that this may be because one of the most salient features of Trump himself was his reliance on intuition and impulse (as noted in the epigraph) along with an informal communication style (Ahmadian, Azarshahi, & Paulhus, 2017; Jordan & Pennebaker, 2017; Oliver & Rahn, 2016). For example, using text analytic methods, Jordan and Pennebaker (2017) found that Trump uses language that is much more in-the-moment, informal, and narrative (as opposed to formal, logical, and analytical) relative to other presidents and presidential candidates. Previous work has shown that persuasive appeals are more effective when they are constructed to correspond with the target’s personality traits (Hirsh, Kang, & Bodenhausen, 2012; Matz, Kosinski, Nave, & Stillwell, 2017)—a finding that may extend to correspondence in cognitive style between political candidates and voters.

Trump may also have attracted intuitive thinkers (and repelled analytic thinkers) because of his specific policy proposals, many of which had a particularly intuitively or emotionally compelling appeal (as opposed to being built around detail and careful analysis). For example, his proposal to build a several stories high physical wall along the Mexican border to reduce illegal immigration evokes much more intuitively compelling mental imagery than the border fence favored by homeland security experts (Nixon, 2017). Similarly, his proposal of using tough trade policies to bring manufacturing jobs back to the United States in large numbers likely resonated more at an intuitive level than Clinton’s proposal to retrain individuals formerly employed in manufacturing. Future work should examine these speculative possibilities experimentally.

Limitations

There are a number of limitations of the present work that should be made clear. First, our sample is from Mechanical Turk which is not representative of the broader U.S. population. Individuals self-selected into the studies (although none were advertised as being about analytic thinking or political ideology) and presumably are comfortable with online surveys (which may not be true for pockets of the general population). Furthermore, as our data indicate, conservatives are underrepresented on Mechanical Turk, and it is possible that conservatives on Mechanical Turk differ from other conservatives in ways that could affect our results. Thus, although previous work has shown Mechanical Turk to be a reliable resource for research on political ideology (Berinsky, Huber, & Lenz, 2012; Buhrmester, Kwang, & Gosling, 2011; Clifford, Jewell, & Waggoner, 2015; Coppock, 2016; Krupnikov & Levine, 2014; Mullinix, Leeper, Druckman, & Freese, 2015), the present results should be replicated using a nationally representative sample, and it should not be assumed that our results generalize to the nation as a whole. Relatedly, our results speak only to the various theories of political ideology in the American context. It is unclear how or whether aspects of the present results generalize to other countries. Unfortunately, this is a limitation of the broad literature on CRT and political cognition (with one exception where a Turkish sample was used: Yilmaz & Saribay, 2016).

Although analytic thinking was measured using a seven-item behavioral (rather than self-report) measure, we did not include a direct measure of just cognitive ability (e.g., numeracy) as a control. As such, it is unclear whether the propensity to think analytically per se (as opposed to the ability) explains the present results. Indeed, it may be that cognitive ability is a strong predictor of political ideology (Hodson & Busseri, 2012; Onraet et al., 2015; Saribay & Yilmaz, 2017). Further research is necessary to more precisely identify the source of the associations reported here. Relatedly, we indexed political ideology with single-item measures—full scales may produce a different pattern of results (or, at least, different effect sizes).

A separate issue is that our participants were asked about voting behavior well after the U.S. Presidential Election (for some, close to a year after). Although it is unlikely that one would forget who they voted for, it is possible that some individuals might misreport their voting decision as a result of events that occurred afterward (Stocké & Stark, 2007) or report that they voted when they actually did not (Anderson & Silver, 1986). In addition, one’s performance on the CRT at the time that they completed the study may not be the same as it would have been on the day that they voted (although there is evidence that CRT performance is quite stable, including for time periods exceeding a year; Meyer, Zhou, & Frederick, 2018; Stagnaro, Pennycook, & Rand, 2018).

The present research takes a correlational as opposed to an experimental approach. There is some (albeit limited) experimental evidence that analytic thinking is linked to liberalism. Yilmaz and Saribay (2017b) found that analytic thought training increased liberal consistent responses to politicized news articles, but had no effect on political opinions on specific issues or general political ideology (see Deppe et al., 2015; Yilmaz & Saribay, 2016 for additional failures to induce an experimental effect). Further experimental work is required to understand the potential interaction between analytic training, political behavior, and political ideology.

Finally, the present data set emerged from studies with a different focus (namely, fake news: Pennycook, Cannon, & Rand, 2018; Pennycook & Rand, 2017; 2018a; 2018b). As such, this should be considered an exploratory study—albeit with a large sample size and substantial statistical power. Given the constantly changing political landscape, confirmatory research undertaken during future elections would help establish both the validity and generalizability of our results.

Conclusion

It has often been argued that liberals are more analytic and less intuitive than conservatives (Deppe et al., 2015; Eidelman et al., 2012; Jost, 2017; Jost et al., 2003; Talhelm et al., 2015). In a sample of 15,001 participants, we did find support for this claim. However, our results also provide evidence for three additional mechanisms by which analytic thinking may affect American political attitudes and behavior. First, analytic thinking may be important for encouraging political engagement (and discouraging political apathy). Second, analytic thinking may support the adoption of heterodox political positions and behavior. Third, analytic thinkers may be drawn toward political candidates who (at least by appearance) share a similar cognitive style, and intuitive thinkers might be drawn to policy proposals that have particular intuitive appeal. These observations illuminate new directions for future research on the psychological underpinnings of political ideology, affiliation, and behavior.

Supplemental Material

pennycook_online_appendix – Supplemental material for Cognitive Reflection and the 2016 U.S. Presidential Election

Supplemental material, pennycook_online_appendix for Cognitive Reflection and the 2016 U.S. Presidential Election by Gordon Pennycook and David G. Rand in Personality and Social Psychology Bulletin

Footnotes

Appendix

Table A8.

Sample sizes (N), means (M), and standard deviations (SD) for continuous political ideology.

	Strongly liberal	Somewhat liberal	Moderate	Somewhat conservative	Strongly conservative
Social issues	.556 (.284)	.522 (.279)	.418 (.281)	.461 (.272)	.444 (.290)
Economic issues	.513 (.290)	.514 (.283)	.441 (.282)	.505 (.281)	.494 (.287)

Note. CRT means and, in brackets, SD. CRT = cognitive reflection test.

Acknowledgements

We thank John Jost and Dan Kahan for comments on an earlier version of this manuscript.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by a Social Sciences and Humanities Council of Canada Banting Postdoctoral Fellowship (to G.P.) and grants from the Templeton World Charity Foundation and DARPA (to D.G.R.).

Notes

Supplemental Material

Supplementary material is available online with this article.

References

Ahmadian

Azarshahi

Paulhus

D. L.

(2017). Explaining Donald Trump via communication style: Grandiosity, informality, and dynamism. Personality and Individual Differences, 107, 49-53. doi:10.1016/J.PAID.2016.11.018

Anderson

Silver

(1986). Measurement and mismeasurement of the validity of the self-reported vote. American Journal of Political Science, 30, 771-785.

Arechar

Kraft-Todd

Rand

(2017). Turking overtime: How participant characteristics and behavior vary over time and day on Amazon Mechanical Turk. Journal of the Economic Science. Retrieved from https://link-springer-com-443.web.bisu.edu.cn/article/10.1007/s40881-017-0035-0

Barbey

A. K.

Sloman

S. A.

(2007). Base-rate respect: From ecological rationality to dual processes. Behavioral and Brain Sciences, 30, 241-297.

Barrouillet

(2011). Dual-process theories and cognitive development: Advances and challenges. Developmental Review, 31, 79-85. doi:10.1016/j.dr.2011.07.002

Bear

Rand

D. G.

(2016). Intuition, deliberation, and the evolution of cooperation. Proceedings of the National Academy of Sciences, 113, 936-941. doi:10.1073/pnas.1517780113

Beevers

C. C. G.

(2005). Cognitive vulnerability to depression: A dual process model. Clinical Psychology Review, 25, 975-1002. doi:10.1016/j.cpr.2005.03.003

Berinsky

A. J.

Huber

G. A.

Lenz

G. S.

(2012). Evaluating online labor markets for experimental research: Amazon.com’s Mechanical Turk. Political Analysis, 20, 351-368. doi:10.2307/23260322

Bialek

Pennycook

(2017). The cognitive reflection test is robust to multiple exposures. Behavior Research Methods. Advance online publication. doi:10.3758/s13428-017-0963-x

10.

Browne

Thomson

Rockloff

M. J.

Pennycook

(2015). Going against the herd: Psychological and cultural factors underlying the “vaccination confidence gap.” PLoS ONE, 10(9), 1-14. doi:10.1371/journal.pone.0132562

11.

Buhrmester

Kwang

Gosling

S. D.

(2011). Amazon’s Mechanical Turk: A new source of inexpensive, yet high-quality, data? Perspectives on Psychological Science, 6, 3-5. doi:10.1177/1745691610393980

12.

Campitelli

Gerrans

(2014). Does the cognitive reflection test measure cognitive reflection? A mathematical modeling approach. Memory & Cognition, 42, 434-447. doi:10.3758/s13421-013-0367-9

13.

Chaiken

Trope

(1999). Dual-process theories in social psychology. New York, NY: Guilford Press.

14.

Clifford

Jewell

R. M.

Waggoner

P. D.

(2015). Are samples drawn from Mechanical Turk valid for research on political ideology? Research & Politics, 2(4). doi:10.1177/2053168015622072

15.

Coppock

(2016). Generalizing from survey experiments conducted on Mechanical Turk: A replication approach. Retrieved from https://alexandercoppock.files.wordpress.com/2016/02/coppock_generalizability2.pdf

16.

Deppe

Gonzalez

Neiman

Jacobs

Pahlke

Smith

K. B.

Hibbing

J. R.

(2015). Reflective liberals and intuitive conservatives: A look at the cognitive reflection test and ideology. Judgment and Decision Making, 10, 314-331.

17.

Ditto

P. H.

Liu

B. S.

Clark

C. J.

Wojcik

S. P.

Chen

E. E.

Grady

R. H.

Zinger

J. F.

(2018). At least bias is bipartisan: A meta-analytic comparison of partisan bias in liberals and conservatives. Perspectives on Psychological Science. Advance online publication.

18.

Eidelman

Crandall

C. S.

Goodman

J. A.

Blanchar

J. C.

(2012). Low-effort thought promotes political conservatism. Personality and Social Psychology Bulletin, 38, 808-820. doi:10.1177/0146167212439213

19.

Epstein

Pacini

Denes-Raj

Heier

(1996). Individual differences in intuitive-experiential and analytical-rational thinking styles. Journal of Personality and Social Psychology, 71, 390-405. doi:10.1037/0022-3514.71.2.390

20.

Evans

J. S. B. T

. (1989). Bias in human reasoning: Causes and consequences. Hillsdale, NY: Lawrence Erlbaum.

21.

Evans

J. S. B. T

. (2008). Dual-processing accounts of reasoning, judgment, and social cognition. Annual Review of Psychology, 59, 255-278. doi:10.1146/annurev.psych.59.103006.093629

22.

Evans

J. S. B. T.

Stanovich

K. E.

(2013). Dual-process theories of higher cognition: Advancing the debate. Perspectives on Psychological Science, 8, 223-241. doi:10.1177/1745691612460685

23.

Frederick

(2005). Cognitive reflection and decision making. Journal of Economic Perspectives, 19(4), 25-42. doi:10.1257/089533005775196732

24.

Gervais

W. M.

(2015). Override the controversy: Analytic thinking predicts endorsement of evolution. Cognition, 142, 312-321. doi:10.1016/j.cognition.2015.05.011

25.

Gignac

G. E.

Szodorai

E. T.

(2016). Effect size guidelines for individual differences researchers. Personality and Individual Differences, 102, 74-78. doi:10.1016/j.paid.2016.06.069

26.

Haidt

(2001). The emotional dog and its rational tail: A social intuitionist approach to moral judgement. Psychological Review, 108, 814-834.

27.

Haidt

(2012). The righteous mind: Why good people are divided by politics and religion. New York, NY: Paragon.

28.

Hibbing

J. R.

Smith

K. B.

Alford

J. R.

(2014). Differences in negativity bias underlie variations in political ideology. The Behavioral and Brain Sciences, 37, 297-307. doi:10.1017/S0140525X13001192

29.

Hirsh

J. B.

Kang

S. K.

Bodenhausen

G. V.

(2012). Personalized persuasion. Psychological Science, 23, 578-581. doi:10.1177/0956797611436349

30.

Hodson

Busseri

M. A.

(2012). Bright minds and dark attitudes: Lower cognitive ability predicts greater prejudice through right-wing ideology and low intergroup contact. Psychological Science, 23, 187-195. doi:10.1177/0956797611421206

31.

Iyer

Koleva

Graham

Ditto

Haidt

(2012). Understanding libertarian morality: The psychological dispositions of self-identified libertarians. PLoS ONE, 7(8), e42366. doi:10.1371/journal.pone.0042366

32.

Jagau

van Veelen

(2017). A general evolutionary framework for the role of intuition and deliberation in cooperation. Nature Human Behaviour, 1(8), 152. doi:10.1038/s41562-017-0152

33.

Jones

J. M.

(2012). In U.S., nearly half identify as economically conservative. Retrieved from http://news.gallup.com/poll/154889/nearly-half-identify-economically-conservative.aspx

34.

Jordan

Pennebaker

(2017). The exception or the rule: Using words to assess analytic thinking, Donald Trump, and the American presidency. Translational Issues in Psychological Science, 3, 312-316. Retrieved from http://psycnet.apa.org/fulltext/2017-41186-008.html

35.

Jost

J. T.

(2017). Ideological asymmetries and the essence of political psychology. Political Psychology, 38, 167-208. doi:10.1111/pops.12407

36.

Jost

J. T.

Glaser

Kruglanski

A. W.

Sulloway

F. J.

(2003). Political conservatism as motivated social cognition. Psychological Bulletin, 129, 339-375. doi:10.1037/0033-2909.129.3.339

37.

Kahan

D. M.

(2013). Ideology, motivated reasoning, and cognitive reflection. Judgment and Decision Making, 8, 407-424. doi:10.2139/ssrn.2182588

38.

Kahan

D. M.

Peters

Dawson

Slovic

(2017). Motivated numeracy and enlightened self-government. Behavioural Public Policy, 1, 54-86.

39.

Kahan

D. M.

Peters

Wittlin

Slovic

Ouellette

L. L.

Braman

Mandel

(2012). The polarizing impact of science literacy and numeracy on perceived climate change risks. Nature Climate Change, 2, 732-735. doi:10.1038/nclimate1547

40.

Kahneman

(2011). Thinking, fast and slow. New York, NY: Farrar, Straus and Giroux.

41.

Kahneman

Frederick

(2005). A model of heuristic judgment. The Cambridge Handbook of Thinking and Reasoning, 267-294. doi:10.1111/cogs.12119

42.

Klaczynski

(2001). Analytic and heuristic processing influences on adolescent reasoning and decision-making. Child Development, 72, 844-861. Retrieved from http://onlinelibrary.wiley.com/doi/10.1111/1467-8624.00319/full

43.

Krupnikov

Levine

(2014). Cross-sample comparisons and external validity. Journal of Experimental Political Science, 1, 59-80.

44.

Matz

S. C.

Kosinski

Nave

Stillwell

D. J.

(2017). Psychological targeting as an effective approach to digital mass persuasion. Proceedings of the National Academy of Sciences of the United States of America, 114, 12714-12719. doi:10.1073/pnas.1710966114

45.

Meyer

Zhou

Frederick

(2018). The non-effects of repeated exposure to the Cognitive Reflection Test. Judgment and Decision Making, 13(3), 246-259.

46.

Mullinix

Leeper

Druckman

Freese

(2015). The generalizability of survey experiments. Journal of Experimental Political Science, 2, 109-138.

47.

Nixon

(2017, January). Homeland security secretary has said border wall alone will not work. The New York Times. Retrieved from https://www.nytimes.com/2017/01/25/us/politics/homeland-security-john-kelly-border-wall.html

48.

Oliver

J. E.

Rahn

W. M.

(2016). Rise of the Trumpenvolk: Populism in the 2016 election. The Annals of the American Academy, 667, 189-206. doi:10.1177/0002716216662639

49.

Onraet

Van Hiel

Dhont

Hodson

Schittekatte

De Pauw

(2015). The association of cognitive ability with right-wing ideological attitudes and prejudice: A meta-analytic review. European Journal of Personality, 29, 599-621. doi:10.1002/per.2027

50.

Paxton

J. M.

Ungar

Greene

J. D.

(2012). Reflection and reasoning in moral judgment. Cognitive Science, 36, 163-177. doi:10.1111/j.1551-6709.2011.01210.x

51.

Pennycook

(2017). A perspective on the theoretical foundation of dual-process models. In De Neys

(Ed.), Dual Process Theory 2.0 (pp. 5-39). London, England: Routledge. doi:10.4324/9781315204550.

52.

Pennycook

Cannon

T. D.

Rand

D. G.

(2018). Prior Exposure Increases Perceived Accuracy of Fake News. Journal of Experimental Psychology: General. Retrieved from https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2958246

53.

Pennycook

Cheyne

J. A.

Barr

Koehler

D. J.

Fugelsang

J. A.

(2014). The role of analytic thinking in moral judgements and values. Thinking & Reasoning, 20, 188-214. doi:10.1080/13546783.2013.865000

54.

Pennycook

Cheyne

J. A.

Barr

Koehler

D. J.

Fugelsang

J. A.

(2015). On the reception and detection of pseudo-profound bullshit. Judgment and Decision Making, 10, 549-563. doi:10.3389/fpsyg.2013.00279

55.

Pennycook

Cheyne

J. A.

Koehler

D. J.

Fugelsang

J. A.

(2016). Is the cognitive reflection test a measure of both reflection and intuition? Behavior Research Methods, 48, 341-348. doi:10.3758/s13428-015-0576-1

56.

Pennycook

Cheyne

J. A.

Seli

Koehler

D. J.

Fugelsang

J. A.

(2012). Analytic cognitive style predicts religious and paranormal belief. Cognition, 123, 335-346. doi:10.1016/j.cognition.2012.03.003

57.

Pennycook

Fugelsang

J. A.

Koehler

D. J.

(2015a). Everyday consequences of analytic thinking. Current Directions in Psychological Science, 24, 425-432. doi:10.1177/0963721415604610

58.

Pennycook

Fugelsang

J. A.

Koehler

D. J.

(2015b). What makes us think? A three-stage dual-process model of analytic engagement. Cognitive Psychology, 80, 34-72. doi:10.1016/j.cogpsych.2015.05.001

59.

Pennycook

Rand

(2017, December 8). The implied truth effect: Attaching warnings to a subset of fake news stories increases perceived accuracy of stories without warnings. Retrieved from https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3035384

60.

Pennycook

Rand

D. G.

(2018a, June 10). Who falls for fake news? The roles of bullshit receptivity, overclaiming, familiarity, and analytic thinking. Retrieved from https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3023545

61.

Pennycook

Rand

D. G.

(2018b). Lazy, not biased: Susceptibility to partisan fake news is better explained by lack of reasoning than by motivated reasoning. Cognition. Retrieved from https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3165567

62.

Pennycook

Ross

R. M.

(2016). Commentary on: Cognitive reflection vs. calculation in decision making. Frontiers in Psychology, 7, 9. doi:10.3389/fpsyg.2015.00532

63.

Pennycook

Ross

R. M.

Koehler

D. J.

Fugelsang

J. A.

(2016). Atheists and agnostics are more reflective than religious believers: Four empirical studies and a meta-analysis. PLoS ONE, 11(4), e0153039. doi:10.1371/journal.pone.0153039

64.

Pennycook

Ross

R. M.

Koehler

D. J.

Fugelsang

J. A.

(2017). Dunning–Kruger effects in reasoning: Theoretical implications of the failure to recognize incompetence. Psychonomic Bulletin & Review, 24, 1774-1784. doi:10.3758/s13423-017-1242-7

65.

Pennycook

Tranel

Warner

Asp

E. W.

(2018). Beyond reasonable doubt: Cognitive and neuropsychological implications for religious disbelief. In Coles

(Ed.), Neurology of religion (p. 20). Cambridge, UK: Cambridge University Press.

66.

Piazza

Sousa

(2013). Religiosity, political orientation, and consequentialist moral thinking. Social Psychological and Personality Science, 5, 334-342. doi:10.1177/1948550613492826

67.

Pyszczynski

Greenberg

Solomon

(1999). A dual-process model of defense against conscious and unconscious death-related thoughts: An extension of terror management theory. Psychological Review, 106, 835-845. Retrieved from http://psycnet.apa.org/journals/rev/106/4/835/

68.

Rand

D. G.

(2016). Cooperation, fast and slow. Psychological Science, 27, 1192-1206. doi:10.1177/0956797616654455

69.

Rand

D. G.

Tomlin

Bear

Ludvig

Cohen

(2017). Cyclical population dynamics of automatic versus controlled processing: An evolutionary pendulum. Psychological Review, 124, 626-642. Retrieved from https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2972420

70.

Royzman

E. B.

Landy

J. F.

Goodwin

G. P.

(2014). Are good reasoners more incest-friendly? Trait cognitive reflection predicts selective moralization in a sample of American adults. Judgment and Decision Making, 9, 176-190.

71.

Royzman

E. B.

Landy

J. F.

Leeman

R. F.

(2015). Are thoughtful people more utilitarian? CRT as a unique predictor of moral minimalism in the dilemmatic context. Cognitive Science, 39, 325-352. doi:10.1111/cogs.12136

72.

Saribay

Yilmaz

(2017). Analytic cognitive style and cognitive ability differentially predict religiosity and social conservatism. Personality and Individual Differences, 114, 24-29. Retrieved from https://www-sciencedirect-com.web.bisu.edu.cn/science/article/pii/S019188691730226X

73.

Shenhav

Rand

D. G.

Greene

J. D.

(2012). Divine intuition: Cognitive style influences belief in God. Journal of Experimental Psychology: General, 141, 423-428. doi:10.1037/a0025391

74.

Shenhav

Rand

D. G.

Greene

J. D.

(2017). The relationship between intertemporal choice and following the path of least resistance across choices, preferences, and beliefs. Judgment and Decision Making, 12, 1-18. doi:10.2139/ssrn.2724547

75.

Shtulman

McCallum

(2014). Cognitive reflection predicts science understanding. In Bello

Guarini

McShane

Scassellati

(Eds.), Proceedings of the 36th Annual Conference of the Cognitive Science Society (pp. 2937–2942). Austin, TX: Cognitive Science Society.

76.

Sidanius

Lau

R. R.

(1989). Political sophistication and political deviance: A matter of context. Political Psychology, 10, 85-109. Retrieved from https://www-jstor-org.web.bisu.edu.cn/stable/3791589

77.

Sloman

S. A.

(1996). The empirical case for two systems of reasoning. Psychological Bulletin, 119, 3-22. doi:10.1037/0033-2909.119.1.3

78.

Stagnaro

M. N.

Pennycook

Rand

D. G.

(2018). Cognitive reflection is a stable trait. Judgment and Decision making, 13(3), 260–267.

79.

Stanovich

K. E.

(2005). The robot’s rebellion: Finding meaning in the age of Darwin. Chicago, IL: The Chicago University Press.

80.

Stanovich

K. E.

(2009). Distinguishing the reflective, algorithmic, and autonomous minds: Is it time for a tri-process theory. In Evans

J. S. B. T.

Frankish

(Eds.), In two minds: Dual processes and beyond (pp. 55-88). Oxford, UK: Oxford University Press.

81.

Stanovich

K. E.

(2012). On the distinction between rationality and intelligence: Implications for understanding individual differences in reasoning. In Holyoak

K. J.

Morrison

(Eds.), The Oxford handbook of thinking and reasoning (pp. 433-455). New York, NY: Oxford University Press.

82.

Stanovich

K. E.

West

R. F.

(2000). Individual differences in reasoning: Implications for the rationality debate? Behavioral and Brain Sciences, 23, 645-665. doi:10.1017/S0140525X00003435

83.

Sterling

Jost

J. T.

Pennycook

(2016). Are neoliberals more susceptible to bullshit? Judgment and Decision Making, 11, 352-360.

84.

Stocké

Stark

(2007). Political involvement and memory failure as interdependent determinants of vote overreporting. Applied Cognitive Psychology, 21, 239-257. doi:10.1002/acp.1339

85.

Swami

Voracek

Stieger

Tran

U. S.

Furnham

(2014). Analytic thinking reduces belief in conspiracy theories. Cognition, 133, 572-585. doi:10.1016/j.cognition.2014.08.006

86.

Talhelm

Haidt

Oishi

Zhang

Miao

F. F.

Chen

(2015). Liberals think more analytically (more “WEIRD”) than conservatives. Personality and Social Psychology Bulletin, 41, 250-267. doi:10.1177/0146167214563672

87.

Thomson

K. S.

Oppenheimer

D. M.

(2016). Investigating an alternate form of the cognitive reflection test. Judgment and Decision Making, 11, 99-113.

88.

Toplak

M. E.

West

R. F.

Stanovich

K. E.

(2011). The Cognitive Reflection Test as a predictor of performance on heuristics-and-biases tasks. Memory & Cognition, 39, 1275-1289. doi:10.3758/s13421-011-0104-1

89.

Toplak

M. E.

West

R. F.

Stanovich

K. E.

(2014). Assessing miserly information processing: An expansion of the Cognitive Reflection Test. Thinking & Reasoning, 20, 147-168. doi:10.1080/13546783.2013.844729

90.

Trump

D. J.

(1987, December). Trump on Trump: How I do my deals. New York Magazine, pp. 50-66.

91.

Yilmaz

Saribay

S. A.

(2016). An attempt to clarify the link between cognitive style and political ideology: A non-western replication and extension. Judgment and Decision Making, 11, 287-300. doi:10.1017/CBO9781107415324.004

92.

Yilmaz

Saribay

S. A.

(2017a). Activating analytic thinking enhances the value given to individualizing moral foundations. Cognition, 165, 88-96. doi:10.1016/j.cognition.2017.05.009

93.

Yilmaz

Saribay

S. A.

(2017b). Analytic thought training promotes liberalism on contextualized (but not stable) political opinions. Social Psychological & Personality Science, 8, 789-795. doi:10.1177/1948550616687092

94.

Yilmaz

Saribay

S. A.

(2017c). The relationship between cognitive style and political orientation depends on the measures used. Judgment and Decision Making, 12, 140-147. Retrieved from http://search.proquest.com/docview/1883834797?pq-origsite=gscholar

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.05 MB