Complex Thinking as a Result of Incongruent Information Exposure

Abstract

In this article, we explore whether incongruent information influences what people recall to mind about a presidential candidate’s policy statements. We investigate whether the volume of people’s political thoughts, their ability to produce arguments, the affective valence of these thoughts, and their integrative complexity are influenced by the congruency between new political information and prior political convictions. We conducted an experiment via MTurk manipulating the congruency of information with respect to ideology. Our results show that incongruency significantly alters how people think about politics. Incongruent information increases integrative complexity of the opposing thoughts, becomes more voluminous, and includes more rationales. Moreover, these defensive thoughts are significantly more negative and less positive about the incongruent information. Parallel to what studies on motivated reasoning demonstrated, we also find that politically knowledgeable people in particular seem to strengthen their thoughts’ cognitive structure while defending their priors against information counter to their political views. We further discuss the general effects of these results and the importance of challenges to existing beliefs in generating complex thought systems.

Keywords

political thinking motivated reasoning integrative complexity experiments

How people think about politics, process information, and the complexity of their political cognitions have been recurring topics in mainstream studies of information processing (Bartels, 2002; Redlawsk, 2001), political learning and updating (Hill, 2017), attitude formation (Erisen, Lodge, & Taber, 2014), and political communication (Huckfeldt & Sprague, 1995). As these studies explore the central elements of the democratic citizenry, the conundrum of biased processing and motivated skepticism, in a polarized environment, overwhelms the nature of how people evaluate, assess, and contemplate about new information (Huber, Hill, & Lenz, 2012; Redlawsk, 2002; Taber & Lodge, 2006).

So far, information processing literature gives us enough reason to suspect a significant impact of information acquisition on depth and complexity of thoughts. Research has shown that how individuals react to new information strongly depends on its congruency with one’s prior convictions. Information is instantly evaluated, and different modes of processing follow congruent and incongruent pieces of information (Kunda, 1987, 1990). When new information is consistent with expectations, processing is generally fast and takes place with little active effort (Kahneman, 2011). But when information is unanticipated in some way, processing becomes more effortful. One outcome of this deeper processing can be motivated reasoning, the human tendency to defend the prior attitude from incongruent information through various means. Earlier research has accumulated significant evidence on how motivated reasoning influences attitude formation (Bolson, Druckman, & Cook, 2014; Erisen et al., 2014; Leeper & Slothuus, 2014; Lodge & Taber, 2013; Taber & Lodge, 2006), candidate evaluations (Lebo & Cassino, 2007; Redlawsk, 2002; Redlawsk, Civettini, & Emmerson, 2010), party preferences (Rudolph, 2006; Slothuus & de Vreese, 2010), issue preferences (Meffert, Chung, Joiner, Waks, & Garst, 2006), misperceptions (Nyhan & Reifler, 2010), and political learning (Hill, 2017). The basic premise of (directional) motivated reasoning is that motivated individuals strive to defend and maintain their extant values, identity, and attitudes against incongruent information challenging these priors. Hence, to the extent that the motivation to defend priors directs reasoning, it may also affect cognitive style, producing either integratively complex or simple and unidimensional thoughts. In this research what remains missing is the mechanism through which motivated reasoning operates, which we focus. Much of the earlier literature is clear about the outcomes of motivated reasoning where scholars have measured attitudes, issue preferences, or evaluations. What is lacking is the exploration of the process, the mechanism that shows how incongruent information influences the thoughts which are reflected on evaluations and preferences.

This gap in the literature is important as it results in conceptual ambiguity in thinking about motivated reasoning. Although many scholars theorize that challenges to one’s prior beliefs encourage increased cognitive effort in their defense, there is plenty of room for debate over what that cognitive effort might look like. Moreover, while motivated reasoning appears to be more common among individuals with high levels of political knowledge, few take a look into the “black box” explaining why these moderation effects exist. This article is thus an in-depth assessment of the mechanism of self-defensive behavior in motivated reasoning.

A second reason of why this research is relevant stems from current status of polarized American politics (Iyengar & Westwood, 2015; Mason, 2015), which also reflects major similarities with the populist surge across the globe (Norris, 2017). Today’s American politics presents an environment where the two political camps have never been so divided. The dislike of political parties and candidates are so large that issue positions taken by the parties show major obstacles as to how a common ground can be found in politics. In that regard, in this era, it is easier than ever to avoid alternative viewpoints. As motivated reasoners, people tend to defend their ground against incongruent information challenging their prior preferences. This study aims to enhance our understanding of both populism and affective polarization.

To that end, we explore this gap in the literature, examine a set of premises with respect to the mechanism of motivated reasoning, focusing on thought complexity, and contribute to the underlying associations of motivated reasoning within the models of information processing. First, congruency or incongruency of newly acquired information with priors may affect the extent to which people engage in effortful thinking about politics, which may alter the content of the thinking process and the integrative complexity of their thoughts. Second, as the motivated reasoning literature has shown, people often draw information from memory opposing the new incongruent evidence in an effort to defend their priors (Redlawsk, 2006). Hence, they may put more effort into thinking about arguments opposing the attitudinally incongruent evidence, which in turn may increase the integrative complexity of their opposing thoughts. Third, this effect could particularly be a function of political sophistication, including factual knowledge acquisition, interest in politics, and educational attainment. When challenged by incongruent information those with greater political sophistication may engage in significantly more effortful processing compared with low sophisticates.

In this article, we investigate the impact of incongruent information on the quality of people’s thoughts. We explore whether the volume of people’s political thoughts, their ability to produce rationales, and their integrative complexity are affected by the congruency between new political information and prior political convictions. To investigate how incongruent information affects the quality of thoughts, we designed an experiment in which we manipulated the ideological content of information about a mock presidential candidate to either be in line with or challenge the participants’ political ideologies, that is, to be either congruent or incongruent with their priors. Participants received congruent or incongruent information in three consecutive rounds, and they listed their considerations about the new information in each round of information receipt.

Our research goals require that we explore the quality of reasoning and its content by analyzing the rationalization process from basic indicators to the extensive limits of complexity. To explore the process of reasoning, we rely on related research that previously looked at political reasoning and rationalization (Sniderman, Brody, & Tetlock, 1991; Suedfeld & Tetlock, 1977, 2014) and try to capture the volume of thoughts, the content of the cognitive reasoning, the affective valence of the considerations, and the overall quality of the cognitive structure of the thoughts indicated by integrative complexity. Although earlier studies in this strand of research take motivated reasoning as a presumed process, we take a different approach, modeling thought quality as the mechanism through which motivated reasoning operates.

In the following pages, we first discuss the theoretical underpinnings of our approach on the role thought processes play in motivated reasoning theory and then discuss our expectations regarding the effects of thought quality. Next, we present our research design and data followed by the discussion of our findings. We conclude with the implications of our findings.

Theory

Much of the recent literature in political behavior on information processing and the updating of evaluations draws from basic studies on motivated reasoning ( Kunda, 1987, 1990; Lodge & Taber, 2000; Lord, Ross, & Lepper, 1979; Meffert et al., 2006; Redlawsk, 2002; Taber & Lodge, 2006). The core finding from these studies—replicated many times—is that citizens engaged in politics are, by and large, motivated reasoners who do not approach new evidence evenhandedly (Bartels, 2002). Instead, their prior political attitude significantly affects how new evidence is processed. Confirming evidence is considered stronger and compelling whereas disconfirming evidence is approached with skepticism, counter-argued, and its credibility downgraded (Kunda, 1987; Redlawsk et al., 2010; Taber & Lodge, 2006). Such directionally motivated citizens try to find ways to lower the value of the information incongruent with their existing attitudes or political ideology and overvalue congruent information’s effect on their prior attitudes when faced with such information.

Although earlier studies have shown the attitudinal aspect of this process (Redlawsk, 2002; Taber & Lodge, 2006), we still do not know how in fact the mechanism of counter-arguing and defensive behavior of the priors really operates. We argue that in the face of incongruent information, people try to seek out rationales confirming their priors, engaging in effortful cognitive processing after the initial affective judgment is made. When people are presented with a piece of information that does not conform to their prior political beliefs, the initial affective reaction created by incongruent information leads to more effortful cognitive processing as people actively seek out information from memory that confirm their point of view. Hence, incongruent information that challenges priors leads to more effortful information processing which might result in more integratively complex and causally associated rich thoughts.

Studies on political thinking generally focus on the cognitive rules used to process and to analyze information (McGuire & McGuire, 1991; Nisbett & Ross, 1980; Tetlock, 1985). First, to differentiate people on the basis of the cognitive rules they use to make sense of a political issue, people must, at a minimum, be able to produce some thoughts about the issue. Second, these thoughts may include certain antecedent phenomena that may or may not lead to certain consequences. In other words, people may engage in cause-and-effect-based thinking about competing policy proposals or other political decisions. Hence, the two basic indicators of thought quality would be the number of thoughts and their causal content.

On the contrary, the primary characteristic that represents the quality of political thought is the individual’s ability to engage in multidimensional and relational thinking in approaching a potential political problem, representing the degree of the thoughts’ integrative complexity. Integrative complexity, a measure of the cognitive structure and thought processes, has been used to predict substantively significant political variables such as decisions to go to war and the successes and failures of political leaders (Suedfeld, 2010). Although research has shown that contextual or individual variables such as stress, ideology, and social roles affect the complexity of thoughts, we know very little about how the processing of information itself affects the quality of how we think about politics.

Integrative complexity is probably the best measure of thought quality confirmed across many studies (Jost, Glaser, Kruglanski, & Sulloway, 2003; Suedfeld & Tetlock, 1977; Tetlock, 1983, 1984). As defined by Tetlock (1985), integrative complexity includes two dimensions capturing the content and depth of reasoning: differentiation and integration. Although differentiation refers to the number of different issue dimensions used in thinking about an issue, integration refers to how the individual discusses the associations among these dimensions. Because of the integration dimension’s emphasis on associations, argument quality goes hand in hand with the degree of causal thinking. By design, integrative complexity offers us one of the best tools to explore the extent of reasoning and rumination not only for the laypeople but also for elite decision makers.

As an example of measuring thought quality through integrative complexity, Conway et al. (2008) asked university students to write essays on political topics either reflecting their actual opinions or the opposite. Deceptive essays were higher in integrative complexity than truthful ones; but interestingly, in another part of the study that asked students to write on unimportant, nonpolitical issues, the opposite pattern was found. In the domain of politics, higher effort seems to be associated with greater integrative complexity, sometimes emphasized by one’s party alignment (Conway et al., 2015). Studies on rationalization and motivated reasoning show similar findings for cause-and-effect-based thinking, a mode of thinking which closely parallels integrative complexity.

Following this proposition, we question whether information challenging one’s priors increases thought quality. In line with earlier research, we know that incongruent information takes significantly more time to process as opposed to congruent information (Meffert et al., 2006; Redlawsk, 2006, 2002; Taber & Lodge, 2006). The more people are challenged by information that is incongruent with the priors, the more they try to defend their ground in favor of their prior attitudes and preferences. As a result, incongruency leads to more time spent recalling information that can downgrade the counter-attitudinal information or support the prior attitude. Hence, more time and effort put into defending the priors may result in increases in the thoughts’ integrative complexity. In fact, when motivated reasoning is activated by incongruent information, people tend to talk, speak, or write more about a topic, reason more about a topic challenging the priors, and generate affectively congruent thoughts to criticize or to defend their priors. The argument is that motivated reasoning itself requires defensive and effortful processing that overall strengthens the quality of the argumentation against incongruent information. But to our knowledge, this argument has not been tested.

Considering the content of thoughts, we are also interested in observing the affective nature of the thoughts associated with the thinking process. Although the complexity of thoughts could relate to detecting motivated reasoning, the affective valence of the thoughts would differentiate between the content of the thoughts as well. Prior research has shown that what one recalls to mind about a political object also triggers the affective system that makes affectively congruent information more accessible and affectively incongruent information less accessible (Erisen et al., 2014; Lodge & Taber, 2005, 2013; Redlawsk et al., 2010). Integrating this foundation to testing the level of motivated reasoning, we posit that as people defend the priors against incongruent information, they are more likely to produce negatively valenced thoughts about the incongruent position of an issue. In contrast, when encountered with congruent information people are more likely to generate positively valenced thoughts that match and bolster the priors. As individuals encounter information matching with their priors (or not), the direction of motivated reasoning resonates itself with the complexity of thoughts that also possess certain affective valence.

In addition to considering cognitive complexity and affective valence, we also account for the potential effects of political sophistication, as identified by earlier research (Meffert et al., 2006; Redlawsk, 2006; Taber & Lodge, 2006), on political thinking. Political sophisticates appear to have greater likelihood of engaging in motivated reasoning as a result of their accumulated knowledge on politics as opposed to those who do not have as much interest and inclination to follow and learn about politics. Our assumptions square with the possibility that more politically knowledgeable people could in fact recall more information about the challenging issue or a policy. This might lead to an increase in the ability of defending priors against incongruent information.

Hypotheses

As described above, we put thoughts and considerations at the center of our theory and examine the mechanism through which incongruent information operates on downstream evaluations. We use various indicators of thought quality such as thoughts’ frequency, their causal structure, affective content, and integrative complexity. By frequency, we refer to the volume of thoughts raised by the participants in reasoning about the policy statements delivered by a presidential candidate. By causal structure, we refer to thoughts’ inclusion of antecedents and consequences regarding policy statements. By affective content, we refer to the positivity and negativity used in the thoughts listed. Finally, we refer to integrative complexity (Suedfeld & Tetlock, 2014) to measure the strength of the arguments raised in the face of information ideologically congruent or incongruent with one’s priors. In line with the reasoning laid out above, we consider the following hypothesis:

Hypothesis 1: Incongruent information increases the integrative complexity of thoughts.

Following this core hypothesis, we establish hypotheses on the mechanisms behind and outcomes of this expectation. We anticipate the ideological incongruence of the information to trigger a counter-arguing mechanism. In receiving an incongruent piece of information, one should be more likely to recall considerations that support prior preferences, evidence of an effort to counteract new information that challenges them (Redlawsk, 2002). In the context of our study, we should see the effects of this process in a thought-listing task, where more thoughts should be in line with a participant’s initial issue preferences, while fewer statements should be evident in support of the incongruent information. Thus,

Hypothesis 2: Incongruent information increases the volume of thoughts opposing the incongruent information (henceforth referred to as opposing thoughts).

Receiving multiple ideologically incongruent policy statements as expressed by a political candidate should first generate higher quality reasoning on the opposing thoughts but at the same time this would lower quality reasoning for the supporting thoughts. Here, our expectation stems from the premise that while congruent information may be quickly and easily processed with little thought, incongruency triggers deeper processing. Challenging information motivates one to spare greater effort and time to refute the evidence which makes the cognitive engagement deeper, and perhaps, sophisticated. In contrast, congruent evidence is accepted at first sight favoring one’s prior evaluations and preferences leading to confirmation bias. In turn, we propose the following hypotheses taking into account two aspects of this premise:

Hypothesis 3: Incongruent information increases the integrative complexity of opposing thoughts whereas it reduces the integrative complexity of supporting thoughts.

Hypothesis 4: Incongruent information increases the number of arguments opposing the incongruent information while reducing the number of supporting arguments.

Moreover, this mechanism triggered by incongruent information may promote negativity in the reasoning process. Upon receiving an incongruent piece of information, thoughts may be mostly negatively valenced in content as opposed to being positively valenced. In other words, incongruent information may motivate the recalling of negative thoughts while suppressing the recalling of positive thoughts.¹ We therefore propose a hypothesis testing the valence-associated process:

Hypothesis 5: Incongruent information, as opposed to congruent information, triggers a negative thinking process in which more negatively valenced thoughts would be recalled about the policy statement as opposed to positively valenced thoughts.

Taken together, these hypotheses complement each other supporting the expectation that incongruent information motivates complex thought processing in favor and defense of one’s prior judgments. However, covariates that could alter this theoretical setup should be accounted for. Among these, political knowledge is the most potential leverage against the experimental effects of incongruent information on complex thinking. Those with greater knowledge of politics could retrieve more information bolstering their priors against challenging information. We thus propose the following hypothesis:

Hypothesis 6: Political knowledge interacts with incongruent information bolstering the level of engagement to complex thought processes.

Method and Data

Experimental Procedure

We use a between-subject experimental design with for a mock presidential election to raise the salience of political thinking. In line with our theoretical discussion, we designed an experiment where we varied the direction of the information (incongruent vs. congruent) shared about the candidate’s policy positions. We define incongruent information as issue positions ascribed to the candidate that contrast with the participant’s own political ideology, whereas congruent information comports with the participant’s ideology.² Every individual was able to receive up to three pieces of information about the candidate’s political positions, either congruent or incongruent with the participant’s political ideology. All participants received the first two pieces of information; they then had a choice whether or not to receive the third piece of information.

Our study was composed of three sections. After the consent procedure, in the first section, participants reported their political ideology according to which we assigned participants to congruent or incongruent information conditions. All of the participants in our study self-identified as conservative, liberal, or moderate, and all moderates were randomly assigned to one of the two political ideology groups for the information congruency manipulation.³ In the second section, each participant received the instructions about the procedure and the candidate biography, with information about the candidate. Then, all participants received policy information about the political candidate.⁴ Immediately after the receipt of the information, each participant was asked a sequence of questions on consequent screens. These included a feeling thermometer rating on the candidate, a vote likelihood for the candidate, interest in receiving more information about the policy, a battery measuring the emotional reactions toward the candidate, and a thought-listing procedure on the candidate’s policy statements.

To collect the thoughts supporting the policy statement, we used the following instructions: “Now, we would like you to list your thoughts on the policy statement on [immigration reform\economic crisis\Iranian nuclear pursuits] delivered by James Palmer. In the response box below, please list your thoughts that support this candidate’s policy statements.” There was a single response box right below the instructions (located in the middle of the screen) where the participants typed in their considerations. The “Next” button appeared on the lower left part of the screen, which immediately took the participants to the next stage where we asked them to report their thoughts opposing the policy statement. The instructions for that task were “Now, please list your thoughts that oppose this candidate’s policy statements on the same issue.” The response box was once again below the instructions, located in the middle of the screen.

The first policy information piece was on immigration reform, the second on economic crisis, and the third (optional for those interested in receiving an additional information) on a foreign policy issue, Iranian nuclear pursuits. After all these sections are completed, in the third section, participants responded to a battery of items on political knowledge, political interest, and demographics.

Sample

Our sample included 541 Amazon Mechanical Turk (MTurk) workers who participated in an online survey experiment providing a convenience sample closer to the general population repeatedly used in previous research (Berinsky, Huber, & Lenz, 2012; Krupnikov & Levine, 2014; Mullinix, Leeper, Druckman, & Freese, 2015). We fielded the study in April 2014 at a time of low-campaign and low-information environment to control for any potential confounding effects of the upcoming 2016 presidential elections.⁵

Participants who agreed to participate in the study by clicking on the MTurk link were taken to an external website to complete it. Upon completion of the survey they were given a completion code that they could use in the MTurk website to get paid. We advertised the study as a simple political candidate evaluation survey that would take about 15 min. We paid each respondent US$1 for participation. Considering the tasks in our study, we do not expect that people have significantly more incentives to work harder in this study context than they would in a typical political environment. In particular, since MTurk participants are highly conscious of their time, especially for the tasks that require “buy-in,” we do not expect that they would be overly enthusiastic to engage in the thought-listing procedure that is used for the primary measures of interest in this article.⁶

Our study included adults who were at least 18 years of age and resided in the United States at the time of the study. The average participant in the sample was 36 years old, White, with a bachelor’s degree and a mean income in the US$35.000 to US$60.000 bracket.⁷

Dependent Variables

Our primary dependent variables focus on the content of the thoughts listed by the participants throughout the study. The thought-listing procedure was a directed method on two domains: supporting thoughts and opposing thoughts on the policy statement delivered by the candidate. We asked participants to list their thoughts that first supported and then opposed the policy statement on the issue domain (immigration reform, economic crisis, and Iranian nuclear pursuits). There was a single response box for each category where participants listed their thoughts and considerations. We calculated four variables from the supporting or opposing thoughts: the integrative complexity of the thoughts, the volume of thoughts, the frequency of arguments associated with the policy, and the valence (positive or negative) of the thoughts listed. Each of these measures serves as a dependent variable in the following analyses. We also have another integrative complexity score for all thoughts, supporting or opposing, and used this variable to see the overall impact of incongruent information on thoughts’ integrative complexity.

All thoughts were coded by two research assistants who were unaware of the research hypotheses.⁸ The coding scheme was as follows for each measure:

Integrative complexity

The primary coding procedure refers to the integrative complexity of the thoughts (Suedfeld & Tetlock, 1977, 2014) We used Tetlock’s (1985) integrative complexity codebook to train the coders. The key in this coding scheme is to capture how distinct the participant’s thoughts and arguments are from each other in terms of the various issue dimensions of the statement (differentiation) and how the participant discusses the relationships among the different dimensions of the statement in the listed thoughts and arguments (integration).

Following earlier research and considering the higher validity of human coding for integrative complexity (Suedfeld & Tetlock, 2014; Tetlock, Metz, Scott, & Suedfeld, 2014), our coders read all the statements and rated them on a scale of 1 to 7. A score of 1 stands for low differentiation and low integration where the individual is found to create good-bad categories in his or her evaluation of the candidate’s policy statement. For instance, a score of 1 was given to a statement such as “I am completely opposed to immigration reform” stated by a participant on the liberal policy of immigration reform. A score of 3 stands for moderate to high differentiation, but low integration where we see that the participant recognizes alternative points of view, but does not perceive the relations among them. For instance, the following statement received a score of 3 on the same policy: “It seems unrealistic. I don’t think that it would be fathomable to round up all illegal immigrants. It would cost more to do this than to support various social services that they might use.”

A score of 5 stands for moderate to high differentiation and moderate integration where the participant develops an explicit comparison rule to contrast the alternative perspective on the issue. An example for this score is as follows:

He is absolutely right that America was built by immigrants. Just because they aren’t coming over on a boat through Ellis Island anymore, doesn’t make them any less valuable citizens. And if they can become citizens and get legal jobs, then they can get healthcare, their children will be healthier and they will be able to pay their medical bills.

Finally, a score of 7 stands for high differentiation and high integration where the participant uses complex rules to compare and contrast alternative perspectives on the issue. Although there were just a few entries that received a score of 7, one of those was the following:

I have very few thoughts that support this policy. He wants to allow those who came here illegally to get citizenship. This would be a good step because then they would pay income taxes and state taxes as most other people do. This would work to their benefit because the immigrants are usually hard workers just trying to make money to support their families and this would give them the rights and benefits of other citizens. They would then be covered by unemployment insurance as well as worker’s compensation. It would also allow them to go home to visit family and not have to worry about being allowed back into the US.

These examples were all for one of the three incongruent (or congruent) policy positions the participants have received. Intercoder reliability for the integrative complexity coding of the thoughts across the three policy statements, on average, was .91 (p < .01).⁹ We thus combined the scores by taking arithmetic average of the coders and next across three policy scores for a particular policy domain. The integrative complexity of the supporting thoughts ranged between 1 and 4.5 (mean: 1.64; SD: 0.65) and for opposing thoughts ranged between 1 and 4.67 (mean: 1.50; SD: 0.82). The overall average for the integrative scores of all types of thoughts was 1.57.¹⁰

Volume of words

We determined the volume of words by using the CountWords function in R. This function accounts for all possible issues with word counting in addition to a number of inquiries. The counting excludes definite and indefinite article indicators and the propositions to have a better capture of the extent of reasoning through the volume of words. As much as this is a simple count measure, the method captures the volume of words used to take a position or defend an argument. On average, the participants listed approximately 10 words for each policy statement. The volume for the supporting thoughts ranged up to 155 words (mean: 30; SD: 19) and for opposing thoughts ranged up to 164 (mean: 28; SD: 21) across three policies.

Frequency of arguments

Although the volume of words refer to the extent of the reasoning, in terms of the volume of words used in the process, frequency of arguments relates to the counting of the arguments such as a agree–disagree statement regarding the policy or causal associations (such as an antecedent or a consequence) that one generated as a response to the policy statement delivered by the candidate. We rely on earlier research by Moussaïd et al.’s (2015) conceptual definition of “unit of information” as an indicator of an argument. This approach also corresponds to the definition of a causal association in thoughts where a cause and/or an effect related to the policy statement are listed. Because this is a daunting coding process, we primarily rely on the coders to define the arguments and causal associations in line with the guidelines and categorize these thoughts accordingly. Intercoder reliability for the coding of arguments for all thoughts listed across three policy statements, on average, was .72 (p < .01). Given sufficient intercoder reliability, we combined the codings in a single measure by taking the average across the coders and then the arithmetic sum across thought categories. The number of arguments for supporting thoughts ranged between 0.5 and 13 (mean: 3.71; SD: 1.59) and for opposing thoughts ranged between 0.5 and 12 (mean: 3.45; SD: 1.67) across three policies.¹¹

Valence of thoughts

Finally, the valence of the thoughts refers to the positivity or negativity of a thought’s generic affect. This measure captures the affective content of the thoughts listed by the participants. Different from the previous three categories as discussed above, affective content refers to the negative or positive evaluations made in a statement. Although this assessment could be a single negative or a single positive evaluation, there could be more than one affective evaluation in a statement. We follow the basic definition of affective valence in previous research (Erisen et al., 2014; Redlawsk, 2001, 2002) and conceptualize the coding scheme accordingly. Our aim with this coding is to capture the generic affective nature of the thoughts, considerations, or arguments made in response to the policy statement of the candidate.

The coding scheme accounted for the statements in entirety. The coders were asked to count the number of positive and negative evaluations, statements, or arguments provided in the response. So, someone could have received scores for positive thoughts alone, or for negative thoughts alone. Although unlikely, it was also possible that someone received scores for both negative and positive statements. As a result, we created count measures of affective valence for the statements made by the participants in the thought-listing procedure.

The following thoughts were coded as positive on the three issues studied: “Immigrants make our country stronger,” “I like the extension of control over wall street and the banks,” and “I feel if we can peacefully resolve the situation, that should be our first choice”; and the following were coded as negative on the same issues: “All of those illegal youth join street gangs and ruin the safety of our neighborhoods,” “Having more government oversight on banks and financial institutions will hurt the economy. Banks will be more reluctant to take risks and approve loans,” and “There’s a chance it can backfire by making them very hostile to us and they find ways other than nukes to bring us down.”¹² If a thought could not be coded as either positive or negative, it was coded as neutral.

Intercoder reliability was .80 (p < .01) on average for the valence coding of all thoughts listed across three policy statements. We thus combined the positive and negative valence thoughts by first taking the average of the coders and then the arithmetic sum across supporting and opposing thoughts, independently creating two separate measures. The positively valenced thoughts measure ranged between 0 and 47.5 (mean: 3.07; SD: 2.67) and the negatively valenced thoughts measure ranged between 0 and 19.5 (mean: 4.52; SD: 2.92).

In Table 1, we summarize each dependent variable’s coding source, coding process, and measurement, including the distributions.

Table 1.

Conceptualization and Measurement of the Dependent Variables.

Dependent variable	Coding source	Coding process	Measure used in analysis	Minimum	Maximum	M (SD)
Integrative complexity	Tetlock (1985)	IC codebook	Supporting (average)	1	4.5	1.64 (0.65)
Integrative complexity	Tetlock (1985)	IC codebook	Opposing (average)	1	4.67	1.50 (0.82)
Volume of words	Software	Software	Supporting (sum)	1	155	30 (19)
Volume of words	Software	Software	Opposing (sum)	1	164	28 (21)
Frequency of arguments	Redlawsk (2001), Lau and Redlawsk (2006), Erisen, Lodge, and Taber (2014), Moussaïd et al. (2015)	Trained coders	Supporting (sum)	0.5	13	3.71 (1.59)
Frequency of arguments		Trained coders	Opposing (sum)	0.5	12	3.45 (1.67)
Positively valenced		Trained coders	Summed across valence type	0	47.5	3.07 (2.67)
Negatively valenced		Trained coders	Summed across valence type	0	19.5	4.52 (2.92)

Results

Experimental Effects on Thoughts and Considerations

In this section, we test the effects of incongruent information on thoughts and considerations listed about the candidate’s policy statements. As noted, we have four distinct dependent measures (integrative complexity of thoughts, volume of thoughts, frequency of arguments, and valence of thoughts) that capture specific aspects of the reasoning procedure about the presidential candidate’s policy statements, collected in two directed ways, with participants being asked to give thoughts in support of the candidate’s delivered statement and against the statement.

To test our first hypothesis, we begin with predicting the overall integrative complexity of thoughts listed for the three policy statements by incongruent information. We use ANOVA to test the experimental effects. We find that encountering information that does not match with a participant’s ideological priors increases the integrative complexity of thoughts in general, F(1, 539) = 6.53, p < .01, Cohen’s f = 0.10. Standing alone, this finding gives us the first evidence that information that challenges preexisting preferences triggers a higher quality thinking process, supporting our first hypothesis.

Second, we analyze the supporting and opposing thoughts listed across the policy statements separately. Because participants had the chance to list their thoughts across three policies (if they opted for receiving the third piece of information), we combined the integrative complexity scores of thoughts written for the three policies and created two measures¹³: integrative complexity score for the supporting thoughts and integrative complexity score for the opposing thoughts. We again see that incongruent information decreases the quality of supporting thoughts listed on the policy statements, F(1, 537) = 8.66, p < .003, Cohen’s f = 0.12, while increasing the quality of opposing thoughts, F(1, 536) = 38.54, p < .001, Cohen’s f = 0.26, compared with the receipt of congruent information. This finding illustrates that people are motivated to produce higher quality thoughts when opposing information that is counter to their views. On the contrary, people cannot produce cognitively complex thoughts supporting a policy against their views even when they are told to do so. This also suggests that the processing of information is different when encountering incongruency than when the information received fits prior expectations. As a result, these findings strongly support our third hypothesis and suggest that the congruency of the information alters the quality of the reasoning process.¹⁴

We next continue with the volume of thoughts listed for three policy statements, supporting and opposing thoughts separated. We find that those who received incongruent pieces of information wrote significantly more thoughts opposing the candidate’s policy statement, F(1, 541) = 4.05, p < .04, Cohen’s f = 0.08, than for those receiving congruent information, which supports our second hypothesis. Also, they write significantly fewer thoughts supporting it, F(1, 541) = 11.68, p < .000, Cohen’s f = 0.14, across the three policy statements. Those who received incongruent information wrote more opposing thoughts (approximately four words) and fewer supporting thoughts (approximately six words) about the candidate’s policy statements as opposed to those who received congruent information. Essentially, our participants focused on the congruency of the information with respect to the priors even when they are prompted to consider both sides of the issue.

We next tested the same models predicting the number of arguments listed with respect to the policy statements. The models confirm our previous findings with a significant effect of the incongruent information. Participants reported significantly fewer supporting arguments, F(1, 541) = 22.39, p < .001, Cohen’s f = 0.20, and more opposing arguments, F(1, 541) = 19.30, p < .001, Cohen’s f = 0.18, given incongruent information. Participants were more likely to be affected by the type of information matching with one’s priors, which taken together with earlier findings supports our fourth hypothesis.

Finally, we tested the same expectations on the valence of thoughts listed for the policy statements. We maintained the experimental treatment as our main independent variable and separately predicted the total frequency of all positive thoughts and the total frequency of all negative thoughts listed across the three policy statements. We find that incongruent information triggers a negatively valenced thinking process. The information treatment significantly and consistently decreased the volume of positive thoughts, F(1, 541) = 6.94, p < .008, Cohen’s f = 0.10, and increased the volume of negative thoughts, F(1, 541) = 10.91, p < .001, Cohen’s f = 0.14, one would list about the policy statements. This finding also supports our fifth hypothesis.

We find consistent and significant effects of incongruent information altering the frequency and affective content of what is recalled and thought about the policy statements that the presidential candidates were said to have delivered. In comparison to the congruent information, incongruent information bolstered the extent and depth of motivated reasoning. Figure 1 reports the mean distributions across the dependent variables. We see that ideologically incongruent information motivated a negative thinking process by assisting the participants to write more complex opposing thoughts while decreasing the likelihood of listing simple positive supporting thoughts.¹⁵ Equally relevant, congruent information bolsters the ability to generate more supporting thoughts, rationales, and integratively complex congruent thoughts that defend prior preferences and attitudes. We thus find consistent evidence both for the disconfirmation bias that one engages in when encountered incongruent information as well as the defense of prior dispositions when faced with confirming evidence.

Figure 1.

Effect of incongruent and congruent information on volume of thoughts, depth of rationalization, thought valence, and integrative complexity.

Putting these findings in a substantive perspective, a liberal receiving a conservative policy statement from a presidential candidate on (let’s say) immigration reform engages in a reasoning process that is more complex, writes significantly more considerations opposing the position and fewer supporting considerations, forms causally associated reasons that oppose the statement, and lists more negative thoughts and fewer positive thoughts. As a result, we see that incongruent information makes people oppose the policy statement with greater negativity and higher quality of reasoning rather than simply making a rudimentary defense. Although triggering a negative-laden thinking process, ideologically incongruent information also decreases the possibility of any thoughts positive or supportive of the position. In turn, people defend their position in favor of their priors while negating the incongruent piece of information instead of updating their judgments with it.

Overall, those who were assigned to the incongruent information condition consistently and significantly defended their priors by generating integratively complex opposing arguments and integratively weak supporting arguments, by writing fewer supporting thoughts and more opposing thoughts, by writing fewer supporting arguments and more opposing arguments, and, finally, by writing fewer positive thoughts and more negative thoughts.

Individual Determinants of Complex Thinking: The Role of Political Knowledge

Our findings so far consistently show that complex thinking is a function of the congruency of information. Taking this result as a strong support of our empirical expectations, we next account for the potential variables that could explain differences of motivated reasoning across individuals. To that end, we include political knowledge, interest in politics, and a number of demographics (age, gender, education, and income) in the following analysis.

Research on motivated reasoning suggests that political knowledge plays a significant role in making people more biased (Meffert et al., 2006; Miller, Saunders, & Farhart, 2016; Redlawsk, 2002; Taber & Lodge, 2006). To explore the potentially relevant individual differences in our findings, we respecified our initial model, adding political knowledge as a control variable. This measure asks participants to give the correct answer to six factual knowledge questions about politics.¹⁶

We also included political interest measured in two separate domains (attention to politics and the frequency of following politics). The first domain was captured by the self-reports of how interested the participant is in politics (1 = extremely interested, 5 = not interested) and how much attention the participant pays attention to news about national politics (1 = a great deal, 5 = not at all). Frequency of news consumption is a measure ranging from 0 to 7 based on the item asking the participants the number of days she or he watches, reads, or listens to news on TV/Internet/print media. We used a categorical measure to capture the participant’s level of income (1 = less than US$20,000, 8 = US$150,000 or more) and education (1 = no official schooling received, 14 = doctorate [PhD, EdD, ScD]).

We conduct the same analyses as reported above where the dependent variables capture the degree of complex thinking and the independent variables are the experimental treatment in addition to the covariates we introduced in the previous paragraph. Table 2 reports the results: First, in all of the following models, we maintain the significant effect of incongruent information as reported in the initial models. Second, we find that there is a significant political knowledge effect on the content of the thoughts opposing the policy statements. Those who scored high on political knowledge are in fact more likely to engage in cognitively complex thinking for the opposing statements, in writing a higher number of opposing thoughts, and in listing more negatively valenced statements, and in generating causal associations among these opposing negatively valenced thoughts. In other words, those who were able to produce cognitively complex thoughts were in fact using their general knowledge about politics when it comes to defending their prior preferences against challenging information. We do not see this same main effect of political knowledge on thought quality for supporting statements.

Table 2.

Predicting the effects of incongruent information on thought quality and reasoning depth.

	Integrative complexity score			Volume of words		Frequency of arguments		Affective valence
	Combined	Support	Oppose	Support	Oppose	Support	Oppose	Positive	Negative
Incongruent information	0.10^† (0.05)	−0.18*** (0.06)	0.37*** (0.07)	−10.88*** (3.50)	13.80*** (3.83)	−0.65*** (0.13)	0.58*** (0.14)	−0.57** (0.22)	0.76** (0.25)
Political knowledge	0.09*** (0.02)	0.05^† (0.02)	0.12*** (0.03)	1.92 (1.49)	5.22*** (1.53)	0.05 (0.05)	0.16** (0.05)	0.02 (0.08)	0.22* (0.11)
Attention to politics	0.04 (0.03)	0.04 (0.03)	0.03 (0.04)	1.25 (2.19)	1.33 (2.23)	0.01 (0.09)	−0.06 (0.08)	0.10 (0.10)	−0.06 (0.15)
Follow politics	0.01 (0.02)	0.01 (0.02)	0.01 (0.02)	0.61 (0.93)	1.22 (0.98)	0.05 (0.04)	0.04 (0.04)	0.02 (0.05)	0.10 (0.06)
Female	−0.01 (0.06)	−0.02 (0.06)	−0.01 (0.07)	3.57 (3.72)	4.39 (3.93)	0.34* (0.14)	0.12 (0.15)	0.56* (0.28)	0.10 (0.28)
Income	−0.03 (0.02)	−0.01 (0.02)	−0.05* (0.02)	−0.84 (1.15)	−2.42 (1.31)	−0.05 (0.04)	−0.13** (0.05)	−0.09 (0.07)	−0.12 (0.09)
Education	0.05* (0.02)	0.05* (0.02)	0.07* (0.03)	2.96* (1.32)	3.81** (1.41)	0.08 (0.05)	0.17*** (0.05)	0.06 (0.09)	0.27** (0.10)
Age	−0.00 (0.00)	−0.00 (0.00)	−0.00 (0.00)	0.37* (0.17)	0.02 (0.19)	0.02** (0.01)	0.01 (0.01)	0.01 (0.01)	0.00 (0.01)
Constant	0.62** (0.26)	0.92*** (0.27)	0.28 (0.33)	7.13 (16.60)	−18.35 (17.82)	2.18*** (0.65)	0.86 (0.66)	2.10 (1.27)	0.25 (1.16)
N	508	506	506	510	510	510	510	510	510
Prob. > F	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00	0.00
R ²	.06	.04	.12	.06	.08	.09	.11	.03	.07

Note. Unstandardized robust regression coefficients are reported.

†

p < .10. *p < .05. **p < .01. ***p < .001.

Considering the conditional effect of political knowledge, we added the interaction term between information incongruency and political knowledge to our models to predict the opposing thoughts only. As shown in Figure 2 (width of the lines corresponds to 95% confidence intervals), we find that politically knowledgeable individuals are more likely to defend their ground by producing opposing thoughts in particular when they receive an incongruent piece of information challenging their priors. Especially when challenged with incongruent information, sophisticates become much more likely to engage in cognitively complex thinking and thus more likely to become motivated reasoners. This interaction results in a significantly higher (p < .05) volume of words (a) and arguments (b) defending their priors. Also, the degree of negativity in these opposing considerations is higher (c) for those who scored high on political knowledge and who received incongruent information with their prior preferences. Integrative complexity of these considerations (d), produced by a politically knowledgeable individual who received incongruent pieces of information, is also significantly higher (p < .01) compared with someone with low political knowledge and who received congruent pieces of information. In opposition to these results, we do not find a significant interaction effect for the supporting thoughts where incongruency is the primary factor making people to become motivated reasoners rather than knowledge. With these results, we conclude that there is significant support for our final hypothesis.

Figure 2.

Interaction effect between incongruent information and political knowledge on opposing thoughts.

Conclusion and the Implications

In this article, we show that citizens are capable of engaging in complex thought structures about politics. We argue that, despite limited amount of time and incentives, people are likely to think carefully about the statements of political candidates (Sniderman, Brody, & Tetlock, 1991). Incongruency of the information with one’s political priors promotes distinct tendencies of motivated reasoning.

Our investigation details how incongruent information influences complex political thinking by examining the mechanism of motivated reasoning. Overall, we find that incongruent information increases thought quality as measured by thoughts’ integrative complexity, volume of thoughts, and frequency of arguments. Specifically, we find that people’s thoughts become more voluminous and more complex while opposing an ideologically counter policy statement. Parallel to what studies on motivated reasoning demonstrate, people seem to strengthen their thoughts’ cognitive structure while defending their priors against information counter to their political views. Moreover, political knowledge appears to bolster these effects.

Politically informed individuals are significantly more likely to engage in cognitively complex reasoning to defend their priors when challenged by incongruent information (Meffert et al., 2006; Miller et al., 2016; Taber & Lodge, 2006). However, even when told to write opposing thoughts about an ideologically congruent policy statement, our subjects cannot produce thoughts of similar quality. Their thoughts are much simpler and limited in detail.

This finding may have important substantive implications suggesting that ideological conflict can encourage pursuing quality, but ultimately self-serving, political cognition. In line with the polarization literature, we see these findings with major implications in an era where it is easier than ever to avoid alternative viewpoints. As the political and social cleavages that divide the publics further, people stick to their ideological preferences with the assistance of their political cognition to produce a process defending its own existence. Accepting alternative views and considerations not only become a threat to self-preservation of political values, but it also triggers a motivation to rely on cognition to generate self-confirming thoughts. Exposure to incongruent information thus includes serious consequences both for the public and for the political arena. Among various effects, political clashes across party lines and social cleavages are more likely to occur (Jamieson & Cappella, 2008); people would self-select to ideologically congruent media sources to receive information in line with their biases (Iyengar & Hahn, 2009); intolerance against different points of views, especially incivility in social media and in other platforms, would be more present (Suhay, Blackwell, Roche, & Bruggeman, 2015); and eventually, social networks will get disconnected and, over time, form dense homogeneous social bubbles (Erisen & Erisen, 2012).

This article explores the nuts and bolts of this particular mechanism. Our robust findings, although they are coming from a single experiment, conducted on a convenience sample, suggest that resisting a counter view or supporting one’s own ideological viewpoint triggers deeper and more effortful information processing, leading to recall from memory of more thoughts and rationales and recognition of different dimensions of the issue. This might be telling us something about the nature of motivated reasoning: people resist other political views not by few narrow-minded utterances blaming or downgrading the “other,” they put effort into constructing opposing thoughts that are rich in content and volume. We observe the same tendency when we look at the results on the integrative complexity of supporting thoughts that people tend to support ideologically congruent information with thoughts of better quality whereas thoughts opposing congruent information are lower in quality. In brief, whether they are opposing the counter ideological statement or supporting a statement in line with their own ideology, people produce thoughts of better quality when they defend their views.

Theoretical contribution of this article thus stands on its ability to explain the mechanism through which directional motivated reasoning functions. Previous studies’ strong reliance on the assumptions of how motivated reasoning takes place, as we propose and test, is a function of processing of the challenging (or confirming) evidence and thought quality that follows the initial reaction of the recipient. In that regard, our results shed light on this unexplored, yet postulated, process and extend the discussion on how public evaluates ideological information in the context of polarized politics.

Footnotes

Acknowledgements

Earlier versions of this paper have been presented in the American Political Science Association and Midwest Political Science Association annaul meetings. We thank Bethany Albertson, Yanna Krupnikov, Milton Lodge, and Emily Thorson for their invaluable comments on the earlier versions of the paper. We are also indebted to the extremely useful suggestions of the reviewers and the Editor.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Supplemental Material

Supplementary material is available for this article online.

Notes

Author Biographies

Cengiz Erisen is a research associate in the Center on Democratic Performance at Binghamton University and an associate professor, Department of Political Science and International Relations at Yeditepe University. His current research interests include comparative political behavior, emotions, and populism. His most recent book is “Political Behavior and the Emotional Citizen: Participation and Reaction in Turkey”.

Dave Redlawsk is James R. Soles professor and chair, Department of Political Science and International Relations at the University of Delaware. His most recent book is “The Positive Case for Negative Campaigning” with Kyle Mattes. He does research on voter decision making, emotions, and campaigns.

Elif Erisen is an associate professor, Department of Political Science and Public Administration at Hacettepe University. Her research interests include public opinion, political discussion, and more recently, immigration.

References

Bartels

L. M.

(2002). Beyond the running tally: Partisan bias in political perceptions. Political Behavior, 24, 117–150.

Berinsky

Huber

Lenz

(2012). Evaluating online labor markets for experimental research: Amazon.com’s Mechanical Turk. Political Analysis, 20, 351-368.

Bolson

Druckman

J. N.

Cook

(2014). The Influence of partisan motivated reasoning on public opinion. Political Behavior, 36, 235-262.

Conway

L. G.

III Gornick

L. J.

Houck

S. C.

Anderson

Stockert

Sessoms

. . . McCue

(2015). Are conservatives really more simple-minded than liberals? The domain specificity of complex thinking. Political Psychology, 37, 777-798.

Conway

L. G.

III Thoemmes

Allison

A. M.

Towgood

K. H.

Wagner

M. J.

Davey

. . . McCue

(2008). Two ways to be complex and why they matter: Implications for attitude strength and lying. Journal of Personality and Social Psychology, 95, 1029-1044.

Erisen

Lodge

Taber

(2014). Affective contagion in effortful political thinking. Political Psychology, 35, 187-206.

Erisen

(2012). The effect of social networks on the quality of political thinking. Political Psychology, 33, 839-865.

Hill

S. J.

(2017). Learning together slowly: Bayesian learning about political facts. Journal of Politics, 79.

Huber

G.A.

Hill

S. J.

Lenz

G.S.

(2012). Sources of bias in retrospective decision making: Experimental evidence on voters limitations in controlling incumbents. American Political Science Review, 106, 720-741.

10.

Huckfeldt

Sprague

(1995). Citizens, politics, and social communication. New York, NY: Cambridge University Press.

11.

Iyengar

Hahn

K. S.

(2009). Red media, blue media: Evidence of ideological selectivity in media use. Journal of Communication, 59, 19-39.

12.

Iyengar

Westwood

S. J.

(2015). Fear and loathing across party lines: New evidence on group polarization. American Journal of Political Science, 59, 690-707.

13.

Jamieson

K. H.

Cappella

J. N.

(2008). Echo chamber: Rush Limbaugh and the conservative media establishment. New York, NY: Oxford University Press.

14.

Jost

J. T.

Glaser

Kruglanski

A. W.

Sulloway

F. J.

(2003). Political conservatism as motivated social cognition. Psychological Bulletin, 129, 339-375.

15.

Kahneman

(2011). Thinking, fast and slow. New York, NY: Farrar, Straus and Giroux.

16.

Krupnikov

Levine

A.S.

(2014). Cross-sample comparisons and external validity. Journal of Experimental Political Science, 1, 59-80.

17.

Kunda

(1987). Motivation and inference: Self-serving generation and evaluation of evidence. Journal of Personality and Social Psychology, 53, 636-647.

18.

Kunda

(1990). The case of motivated reasoning. Psychological Bulletin, 108, 480-498.

19.

Lau

Redlawsk

(2006). How voters decide: Information processing in election campaigns. New York, NY: Cambridge University Press.

20.

Lebo

Cassino

(2007). The aggregated consequences of motivated reasoning and the dynamics of partisan presidential approval. Political Psychology, 28, 719-746.

21.

Leeper

Slothuus

(2014). Political parties, motivated reasoning, and public opinion formation. Advances in Political Psychology, 35, 129-156.

22.

Lodge

Taber

C. S.

(2000). Three steps toward a theory of motivated political reasoning. In Lupia

McCubbins

Popkin

(Eds.), Elements of reason: Cognition, choice, and the bounds of rationality (pp. 182-213). London, England: Cambridge University Press.

23.

Lodge

Taber

C. S.

(2005). The automaticity of affect for political leaders, groups, and issues: An experimental test of the hot cognition hypothesis. Political Psychology, 26, 455-482.

24.

Lodge

Taber

C. S.

(2013). The rationalizing voter: Unconscious thought in political information processing. New York, NY: Cambridge University Press.

25.

Lord

Ross

Lepper

(1979). Biased assimilation and attitude polarization: The effect of prior theories on subsequently considered evidence. Journal of Personality and Social Psychology, 37, 2098-2109.

26.

Mason

(2015). “I disrespectfully agree”: The differential effects of partisan sorting on social and issue polarization. American Journal of Political Science, 59, 128-145.

27.

McGuire

W. J.

McGuire

C. V.

(1991). The content, structure, and operation of thought systems. In Wyer Jr.

R. S.

Srull

T. K.

(Eds.), Advances in social cognition (Vol. IV, pp. 1-79). Hillsdale, NJ: Lawrence Erlbaum.

28.

Meffert

Chung

Joiner

A. J.

Waks

Garst

(2006). The effects of negativity and motivated information processing during a political campaign. The Journal of Communication, 56, 27-51.

29.

Miller

J. M.

Saunders

K. L.

Farhart

C. E.

(2016). Conspiracy endorsement as motivated reasoning: The moderating roles of political knowledge and trust. American Journal of Political Science, 60, 824-844.

30.

Moussaïd

Brighton

Gaissmaier

(2015). The amplification of risk in experimental diffusion chains. Proceedings of the National Academic of Sciences of the United States of America, 112, 5631-5636.

31.

Mullinix

J. K.

Leeper

T. J.

Druckman

J. N.

Freese

(2015). The generalizability of survey experiments. Journal of Experimental Political Science, 2, 109-138.

32.

Nisbett

R. E.

Ross

(1980). Human inference: Strategies and shortcoming of social judgment. Englewood Cliffs, NJ: Prentice-Hall.

33.

Norris

(2017, May 17). So is the wave of populist nationalism finished? Hardly. The Washington Post. Retrieved from https://www.washingtonpost.com/news/monkey-cage/wp/2017/05/17/so-is-the-wave-of-populist-nationalism-finished-hardly/?utm_term=.07b8782ee1cc

34.

Nyhan

Reifler

(2010). When corrections fail: The persistence of political misperceptions. Political Behavior, 32, 303-330.

35.

Redlawsk

D. P.

(2001). You must remember this: A test of the on-line model of voting. Journal of Politics, 63, 29-58.

36.

Redlawsk

D. P.

(2002). Hot cognition or cool consideration? Testing the effects of motivated reasoning on political decision making. Journal of Politics, 64, 1021-1044.

37.

Redlawsk

. (2006). Motivated reasoning, affect, and the role of memory in voter decision making. In Redlawsk

(Ed.), Feeling politics: Emotion in political information processing (pp. 87-108). New York, NY: Palgrave Macmillan.

38.

Redlawsk

D. P.

Civettini

Emmerson

(2010). The affective tipping point: Do motivated reasoners ever “Get it”? Political Psychology, 31, 563-593.

39.

Rudolph

T. J.

(2006). Triangulating political responsibility: The motivated formation of responsibility judgments. Political Psychology, 27, 99-122.

40.

Slothuus

de Vreese

C. H.

(2010). Political parties, motivated reasoning, and issue framing effects. The Journal of Politics, 72, 630-645.

41.

Sniderman

P. M.

Brody

R. A.

Tetlock

P. E.

(1991). Reasoning and choice: Explorations in political psychology. New York, NY: Cambridge University Press.

42.

Suedfeld

(2010). The cognitive processing of politics and politicians: Archival studies of conceptual and integrative complexity. Journal of Personality, 78, 1669-1702.

43.

Suedfeld

Tetlock

P. E.

(1977). Integrative complexity of communications in international crises. Journal of Conflict Resolution, 21, 169-184.

44.

Suedfeld

Tetlock

P. E.

(2014). Integrative complexity at forty: Steps toward resolving the scoring dilemma. Political Psychology, 35, 597-601.

45.

Suhay

Blackwell

Roche

Bruggeman

(2015). Forging Bonds and Burning Bridges: Polarization and Incivility in Blog Discussions About Occupy Wall Street. American Politics Research, 43, 643-679.

46.

Taber

C. S.

Lodge

(2006). Motivated skepticism in the evaluation of political beliefs. American Journal of Political Science, 50, 755-769.

47.

Tausczik

Y. R.

Pennebaker

J. W.

(2010). The psychological meaning of words: LIWC and computerized text analysis methods. Journal of Language and Social Psychology, 29, 24-54.

48.

Tetlock

(1984). Cognitive style and political belief systems in the British House of Commons. Journal of Personality and Social Psychology, 46, 365-375.

49.

Tetlock

(1985). Integrative complexity of American and Soviet foreign policy rhetoric: A time-series analysis. Journal of Personality and Social Psychology, 49, 1565-1585.

50.

Tetlock

Metz

S. E.

Scott

S. E.

Suedfeld

(2014). Integrative complexity coding raises integratively complex issues. Political Psychology, 35, 625-634.

51.

Tetlock

P. E.

(1983). Cognitive style and political ideology. Journal of Personality and Social Psychology, 45, 118-126.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.20 MB