The Quality of Political Information

Abstract

The article conceptualizes the quality of political information and shows how the concept can be used for empirical research. I distinguish three aspects of quality (intelligibility, relevance, and validity) and use them to judge the constituent foundations of political information, that is, component claims (statements of alleged facts) and connection claims (argumentative statements created by causally linking two component claims). The resulting conceptual map thus entails six manifestations of information quality (component claim intelligibility, connection claim intelligibility, component claim relevance, connection claim relevance, component claim validity, and connection claim validity). I explain how the conceptual map can be used to make sense of the eclectic variety of existing research, and how it can advance new empirical research, as a guide for determining variation in information quality, as a conceptual template for the analysis of different types of political messages and their common quality deficiencies, and as a generator of new research questions and theoretical expectations.

Keywords

political information concept formation typology information quality

Introduction

Information is the principal political currency. It shapes what and how we think of politics; political actors acquire, manipulate, and disseminate it to achieve their objectives; and the institutionally prescribed relations between people and policy-makers in both democratic and autocratic regimes cannot function without it. Even the less cerebral forms of political interaction, such as emotional appeals, hot cognitions, and group identities, need to be sustained by information. But when is information good enough to serve its purpose, and what happens when the quality of information is too low? What is the quality of political information, and how can different aspects of information quality be distinguished?

The article offers answers to these questions in order to advance theory-guided empirical research about the causes and consequences of variation in the quality of political information. This is particularly important during a time in which disinformation, misinformation, and fake news have become serious concerns in both mass politics and international affairs, while traditional quality claims based on evidence, established procedures of news reporting, and scientific methodology are under attack from populist politicians and movements. However, despite the great significance of political information, and despite the fact that many fields of scholarship are concerned with it, political science and cognate disciplines have not yet engaged in systematic attempts to make sense of the quality of political information and its manifestations.

Existing studies use a wide variety of partially overlapping and less than perfectly demarcated terms to identify different aspects and deficiencies of information quality, such as diagnostic value (Kuklinski et al., 2001), political clarity (Dalton, 1985), argument strength (Areni and Lutz, 1988), argument quality (Clark and Wegener, 2009), information accuracy (Shikano et al., 2017), misleading statements (Jerit and Barabas, 2006), fake news (Lazer et al., 2018), and imperfect information (Weyland, 2014). The eclectic terminology used in prior research reflects the absence of a commonly accepted and well-defined concept of the meaning and manifestations of the quality of political information. Empirical studies relying on the existing concepts frequently fail to pinpoint the particular manifestation of information quality that affects certain political outcomes. The purpose of this article is to remedy that problem, by developing a concept of the quality of political information, and by showing how the concept can be used not only to make sense of existing studies but also to advance new empirical research.

In the first part of the article, I develop a simple model of political information based on the principles of argumentation by Toulmin (2003). The model identifies component claims (statements of alleged facts) as the atoms of political information and connection claims (argumentative statements created by causally linking two component claims) as the most basic molecules. In the second part of the article, I conceptualize quality as an intrinsic feature of the content of political information. In the beginning, I invoke Grice’s (1975) conversation rules to develop a concept of the meaning of quality that distinguishes three aspects of quality, namely, intelligibility, relevance, and validity. After that, I establish a conceptual map of the quality of political information by applying the three quality judgments to the two previously introduced building blocks of information. The quality of political information thus entails six manifestations, that is, component claim intelligibility, connection claim intelligibility, component claim relevance, connection claim relevance, component claim validity, and connection claim validity. I use the conceptual map to classify existing studies of information quality from a wide range of thematic fields, including political behavior, party politics, public administration, political communication, political development, and international relations. In the third part of the article, I show how my model can be applied to empirical research. I discuss, in turn, how the model can be used to determine variation in information quality, how it can serve as a conceptual foundation for studying the quality of different kinds of political messages, and how it can help generate new research questions and theoretical expectations. The fourth part of the article offers a brief conclusion and discussion.

A Simple Model of Political Information

My concept of information quality rests on a simple model of political information, which I develop from the bottom up beginning with the constituent rhetorical foundations. I conceive of the foundational units of political information as component claims and connection claims. These two building blocks of political information are political statements, which can be used to compose, political messages, which in turn might be nested in political information packages. For instance, political messages such as a policy justification, a ministerial memo, or a campaign slogan represent a certain amount of text that might contain just one, several, or even many political statements. The election manifestos of Christian democratic parties between 1980 and 2010, the New York Times published on 1 September 1977, and the collected speeches of the members of the European Parliament are political information packages, meaning larger composite containers of a certain number of political statements, distinguished by some shared characteristic, and frequently composed of separate political messages. Political information distributors are channels through which political information is disseminated, such as newspapers, twitter, or election manifestos. Judgments of information quality can be made for individual component and connection claims, and they can be summarized (or identified separately) at any level of aggregation, including certain combinations of claims, political messages, political information packages, or entire political information distributors.

My model disassembles larger units of political information into the foundational items of which they are composed by transferring the micro-perspectives of linguistics and argumentation studies to the unique domain of politics. I rely specifically on the work of Stephen Toulmin (2003), who proposes a conception of the structure of arguments that I am taking as the starting point for my own model. His most elementary and unconstrained form of argumentation, which he calls a “skeleton of a pattern for analyzing arguments” consists of “data,” “warrant,” and “conclusion” (Toulmin, 2003: 92). The relation between these three components is such that a piece of data, or in other words a fact, is initially stated and then used to draw a conclusion through a warrant. For instance, in Toulmin’s own example, the data “Harry was born in Bermuda” leads to the conclusion “Harry is a British subject” because of the mediating warrant that “a man born in Bermuda will be a British subject.”

The distinction between “data” and “conclusion” is not inherent to the content of these statements. The difference in nomenclature stems exclusively from the function the statements perform in an argument. After all, both “data” and “conclusions” are by their very nature single units of information. The Toulminian “data” of one argument can easily become a “conclusion” in another argument, and vice versa. For instance, the statement “Harry is a British subject,” which performs the argumentative function of “conclusion” in the previous example, becomes “data” in another argument with the conclusion that “Harry has the right to vote in Britain.” This is why I propose that all political messages rest on one type of foundational unit, which consists of a single statement of an alleged fact. This foundational unit, which I call component claim, constitutes the atom of political information (see Figure 1).

Figure 1.

Political Information.

Why speak of “component claims” instead of “facts” or “data”? First, as explained above, the choice of terminology serves to highlight that these are as of yet functionally unspecified items of information, which can perform the functions of both “data” and “conclusion.” Second, I am using the term component claims to emphasize that these items are at the same time indivisible and constitutive of larger units of political information. Third, speaking of component claims rather than “facts” or “data” signals openness to different kinds of information, including not only facts in the narrow sense (“Britain has a plurality electoral system”) but also demands (“The West should ally with Saudi-Arabia to fight Iran”) and suppositions (“Britain will continue to have a two-party system”). Fourth, the choice of terminology underscores that component claims can be based on more than one data point, for example, in statements that involve comparisons (“Iran is more dangerous than Saudi-Arabia”) or analogies (“the refusal of the United States to sign the Paris Climate Change Convention is equivalent to the decision of North Korea to not sign the Nuclear Proliferation Treaty”).

Fifth, I am speaking of component claims, because these atoms of political information are subject to variation in judgments of truth. The truth value of a factual statement such as “Britain has a plurality electoral system” is typically less disputed than that of suppositions and demands. But even in the field of seemingly straightforward factual statements, perceived truth values can vary quite strongly between statements and recipients. The claim “Russia has more than 100 million inhabitants,” for example, will be much less controversial than the claim “Russia has violated international treaties 78 times in the past 3 years,” and one can also easily imagine that representatives of the Russian government will be less willing to accept the latter statement as true than the average Ukrainian foreign policy official. In the end, my specifications and modifications of the basic “skeleton” suggested by Toulmin reflect his own emphasis on the field-variant nature of argumentation. I develop a model of political information, and the field of politics is more than other social systems subject to the widespread use of demands and suppositions as well as controversy over many of even the simplest statements of fact.

Connection claims are the second foundational unit of political information (see Figure 1). They consist of two (atomic) component claims that are bound together (into the most basic molecule) by a term indicating causality (such as “because,” “which is why,” or “given that”). For instance, the component claims “Harry was born in Bermuda” and “Harry is a British citizen” can be turned into a connection claim by stating that “Harry is a British citizen because he was born in Bermuda.” In the unique domain of politics, connection claims include not only analytical statements (“Britain will continue to have a two-party system, because Britain has a plurality electoral system”), but also prescriptive statements (“Iran is more dangerous than Saudi-Arabia, which is why the West should ally with Saudi-Arabia to fight Iran”).¹

Connection claims represent a streamlined version of what Toulmin (2003) and argumentation research in general would refer to as an “argument.” So why call them “connection claims” instead of “arguments”? First, speaking of connections serves to emphasize that connection claims are the (most basic) molecules of political information, created by establishing a link between two atomic component claims. Second, the term connection claim is purposefully open to cover both analytical and prescriptive types of statements. Third, the reference to connection claims underscores the contested nature of political statements. It underlines the unique character of political information compared to information in other domains. Fourth, speaking of connection claims highlights that this type of political statement can be used for different objectives other than “argumentation,” which is traditionally defined as “a verbal, social, and rational activity aimed at convincing a reasonable critic of the acceptability of a standpoint” (Van Eemeren and Grootendorst, 2004: 1). Clearly, the process of intentional persuasion identified in this definition is one way in which connection claims are used in politics, but they can also be used for other objectives, including information provision, priming, agenda setting, framing, mobilization, conflict resolution, and demand making.²

I distinguish component and connection claims from political evidence claims. Component and connection claims are the primary foundational units of political information in my model, whereas political evidence claims are auxiliary units that exist exclusively to lend support to component and connection claims. Political evidence claims encompass not only the “warrants” of Toulmin’s (2003: 92) “skeleton” of argumentation but also what he calls “backing,” meaning “other assurances, without which the warrants themselves would possess neither authority nor currency” (Toulmin, 2003: 96). In the previously introduced example, the statement “Harry was born in Bermuda” constitutes the “data” that facilitates the conclusion “Harry is a British subject,” through the mediating warrant that “a man born in Bermuda will be a British subject.” The warrant used in this argument, according to Toulmin (2003), can be further solidified through additional “backing,” for example, by stating the warrant’s legal basis.

My model of political information merges “warrant” and “backing” into the category of political evidence claims and does not require connection claims to have a “warrant.” This is done to better reflect the specific character of political information. I would argue that connection claims in politics do not necessarily contain a supporting reason (i.e. a warrant). Arguments without giving a reason for the alleged existence of a connection between two component claims might have lower quality, but they are entirely conceivable and quite common. By not requiring a warrant in the Toulminian sense, the model of political information I develop allows for that possibility, which then makes it possible to estimate the effect that the existence, respectively the absence of a warrant or other political evidence (i.e. “backing”), might have on the quality of political statements.

Political evidence claims come in many forms, including not only references to research findings but also additional data points, legal provisions, and generalized rules that sustain claims about specific cases. They can support a single component claim, component claims that are part of a connection claim, as well as the connection claim itself. The classification of a unit of political information as a political evidence claim is justifiable as long as the unit exists exclusively in an auxiliary function to support some primary political statement, that is, a connection claim or a component claim. Whenever a unit of information constitutes a political statement in its own right, it should also be treated as a separate component claim or connection claim. Judgments about whether some statement is auxiliary or primary in specific analyses depend on the statement’s relations to other statements and a range of additional factors including the motive of the speaker and the communication context.

The Quality of Political Information: Conceptual Map and Classification of Existing Studies

I am now proposing a conception of the meaning of quality, which delineates the aspects of quality that can be used to judge political information. To that end, I turn to the field of pragmatics, which studies the meaning of language and communication in its real-world context. From this point of departure, Grice (1975) has developed widely accepted normative standards for the quality of the content of communication. These standards can also be applied to judge the quality of political information, given that political information constitutes a type of communicated content. Grice (1975: 46) summarizes his standards for the content of communication in three “supermaxims”: “try to make your contribution one that is true,” “be relevant,” and “be perspicuous.” The quality of the content of communication in general and of political information in particular can thus be conceived of as having three dimensions, namely, validity, relevance, and intelligibility.

My concept of the quality of political information relies on the three pertinent supermaxims formulated by Grice (1975). The normative standards delineated by these supermaxims describe what quality is all about, while my previously outlined model of political information designates the constituent components of political information to which the quality judgments can be applied. Applying the three normative standards for high-quality communication (intelligibility, relevance, and validity) to the two foundational units of political information (component claims and connection claims) yields a typology that designates six manifestations of the quality of political information. As shown in Table 1, the typology outlines a comprehensive map of the various types of information quality in politics. According to this conceptual map, the quality of political information encompasses component claim intelligibility, connection claim intelligibility, component claim relevance, connection claim relevance, component claim validity, and connection claim validity. I am now discussing each of the aspects of quality in turn, and I show how the conceptual map can be used to classify a representative selection of existing empirical studies from different thematic fields.³

Table 1.

Conceptual Map of the Quality of Political Information.

Object of quality judgment	Type of quality judgment
Object of quality judgment	Intelligibility	Relevance	Validity
Component claim	Component claim intelligibility	Component claim relevance	Component claim validity
Connection claim	Connection claim intelligibility	Connection claim relevance	Connection claim validity

Intelligibility

Intelligibility identifies the extent to which the content and meaning of political statements are comprehensible. Grice (1975: 46) substantiates his supermaxim advocating intelligibility (“be perspicuous”) through a list of more specific rules, namely, “avoid obscurity of expression,” “avoid ambiguity,” “be orderly,” and “be brief.” One way of violating the first two precepts has been identified by Schellens and de Jong (2004). They suggest that leaving “implicit” (Schellens and de Jong, 2004: 302) the connections between actions and their consequences reduces the intelligibility of these connection claims. I believe that a negative effect of such implicitness on intelligibility is entirely possible, but I would argue that failing to make connection claims explicit will only reduce their intelligibility when recipients require an explicit statement to understand the presence of the causal connection and the meaning of its constitutive component claims. In politics, suggestive argumentation constitutes a ubiquitous case of implicit yet intelligible connection claims. For instance, a right-wing politician saying “Maybe foreigners are not more prone to be criminals, but many of them live in the neighborhood where yesterday’s murder occurred” is suggesting implicitly that a foreigner is responsible for the murder. His coded language is still fully intelligible, and maybe even more appealing to his core audience.

Suggestive argumentation can be crafted so proficiently that its meaning becomes completely concealed to outsiders. For instance, Albertson (2015) finds that the religious connotation of “multi-vocal” political messages is recognized only by believers. Politicians can use these messages to reach insiders without alienating others. Haney Lopez (2014) describes the same mechanism for coded racial appeals. These scenarios raise important questions about the identification of intelligibility and the underlying meaning of information. They highlight the distinction between the motive of the speaker (specifically his intention to produce a message that is understood in different ways by different groups) and the outcome of his communication (the actual meaning as well as the quality of the statement). Intelligibility, just like relevance and validity, is a feature of the content of information. It should be judged on its own terms, because the political consequences of quality depend on its (objective or subjectively perceived) nature rather than the speaker’s motives. The distinction between objective and subjective measurement, explained in more detail below, is critical in this context. It allows us to determine whether differential meanings are effectively transported to different groups, how successfully this has been done (based on subjective judgments of intelligibility in these groups), and how the subjective judgments compare to objective standards of political discourse.

That meaning can be communicated in a highly intelligible fashion without making an entire claim fully explicit is true not only for connection claims but also for component claims. For instance, a member of parliament might say in a speech that “introducing the Euro-tax is good for social justice.” The two component claims bound together in this connection claim are not made fully explicit, as the politician says neither “the Euro-tax is a good policy” nor “social justice is a desirable goal.” A fully explicit rendition of her statement would be “The Euro-tax is a good policy, and it advances the desirable goal of social justice.” Making her statement fully explicit would not increase intelligibility, since it is already sufficiently clear from context that she likes both the policy she defends and the goal allegedly advanced by the policy. Grice’s (1975) precepts “be brief” and “be orderly” are in conflict here with his precepts “avoid obscurity” and “avoid ambiguity.” But in this particular example, being brief and therefore more orderly is arguably better for intelligibility than the alternative of a lengthy statement that would fully expunge obscurity and ambiguity.

The concept of “connections left implicit” studied by Schellens and de Jong (2004) is exclusively about connection claim intelligibility. It is contained by a single cell in the six-cell typology and thus clearly demarcated from other manifestations of the quality of political information. The same is true for Shikano et al. (2017: 230), who conceive of “information accuracy” in bureaucratic decision-making as a form of component claim intelligibility that depends on the amount of noise included with pertinent information. Other studies use less clearly demarcated terms that cover a wider conceptual space extending into at least one additional manifestation of information quality. To begin with, some studies investigate intelligibility without making a clear-cut distinction between connection and component claims. For instance, research on party politics has a long tradition of identifying the “clarity of party positions” (Dalton, 1985) or more broadly the “ideological clarity” of a party (Lo et al., 2014), both of which entail component and connection claims. Jerit (2009: 442) studies the style of news coverage (“how an issue is covered”), which also involves component and connection claim intelligibility. In the field of political behavior, “diagnostic value” (Kuklinski et al., 2001) is one of the most frequently invoked concepts related to the quality of political information. It represents a composite syndrome of information quality that encompasses the intelligibility and relevance of connection and component claims. Research on political development uses terms such as “imperfect” (Weyland, 2014), “inadequate” (Vössing, 2017), and “poor” (Schedler, 2013) information to identify the same syndrome. Elaborating on existing research, my conceptual map makes it possible to spell out the relative causal effects of specific manifestations of information quality that are currently merged into broad syndromic concepts.

Intelligibility refers to the quality of the content of political information. It is distinct from the quality of people’s information endowment. The former is understudied, while the latter is one of the most frequently studied topics in the field of political behavior. For instance, Delli Carpini and Keeter (1996) investigate the status and ramifications of variation in political knowledge, which is determined by the amount of information people accumulate. Converse (1964) observes a dearth of systematic belief systems in the American mass public, which also stems from limited exposure to political information. On this background, a wide range of existing studies investigate the modalities and consequences of people’s (limited) engagement with information, such as whether they use information effectively to cast correct votes (Lau and Redlawsk, 1997), make good political judgments (Kuklinski et al., 2001), avoid partisan bias (Bartels, 2002), and hold accurate views of political conditions (Holbrook and Weinschenk, 2020). It is important to distinguish the accumulation and use of information at the individual level from the quality of information supply, which is the concern of this article, because it makes a difference, for instance, whether non-rational political choices are individual fallacies rooted in limited exposure to information or the result of poor information quality.

Relevance

The relevance of political information describes the extent to which the information occurs in a meaningful relation with the context in which it is communicated. Grice (1975) simply recommends “be relevant” without detailing more specific precepts. Both connection claims and component claims can be subjected to judgments of relevance. For instance, the component claim “Britain uses plurality voting in its electoral system” and the connection claim “People vote for the two major parties in plurality electoral systems because they do not wish to waste their votes” are arguably valid and intelligible. But it depends very much on the context in which they exist whether they are also relevant. It is fair to suggest that they are relevant when used in a debate about the failure of the Green party to secure seats in the UK parliament, but they are irrelevant to a wide range of unrelated topics and conversations. Just like the other aspects of information quality, relevance refers to the content of information rather than the motive of the speaker. Some politicians, for instance, might honestly try to offer relevant information, but when they fail in their efforts, we would have to conclude that their statements suffer from a lack of relevance.

In existing research, relevance is rarely separated from other aspects of information quality. A notable exception is Austen-Smith (1992), who distinguishes the validity of connection claims about the consequences of certain actions (“validity of action-outcome claim”) from the relevance of such statements to the decision-maker (“relevance of action-outcome claim”). Most existing research treats relevance as an integral part of a larger syndrome of information quality. Several studies introduced above (Kuklinski et al., 2001; Schedler, 2013; Vössing, 2017; Weyland, 2014) conceive of quality as a combination of relevance and intelligibility. Some studies of political persuasion in the tradition of the Elaboration Likelihood Model (ELM) employ the commonly used terms “argument quality” (Clark and Wegener, 2009) and “argument strength” (Zhao et al., 2011) to designate syndromes of information quality involving validity and relevance. Explicitly distinguishing relevance from validity and intelligibility allows us to determine the relative impact of these separate manifestations of information quality on political outcomes.

Validity

Validity identifies the truth value of political statements. It might be affected by variation in intent, but it constitutes a judgment about the outcome (specifically the content) rather than the motive of communication. That being said, existing studies of the validity of political information can be conveniently distinguished based on their varying assumptions about the intentions of the communicator. First, the most prominent strand of research investigates the malicious production and dissemination of objectively false information, using terms such as “fake news” (Lazer et al., 2018), “misinformation” (Nyhan and Reifler, 2015), “disinformation” (Ferrara, 2017; Richey, 2017), “misleading statements” (Jerit and Barabas, 2006; Nyhan and Reifler, 2010), and “objectively false rumors” (Berinsky, 2017). Succinctly summarizing the dominant understanding of “fake news” and related terms, Lazer et al. (2018: 1094) define it as “fabricated information that mimics news media content in form but not in organizational process or intent.” The definition illustrates that the focus of this scholarship lies on the intention and process of producing and distributing “fabricated information.” The features of the outcome of fake news production, that is, fake news itself, are not directly identified. They are inferred from the attributes of the producers: “(f)ake-news outlets [. . .] lack the news media’s editorial norms and processes for ensuring the accuracy and credibility of information.”

I believe that it is helpful to keep the quality of information (the outcome) conceptually distinct from the process of information production, and to define both phenomena on their own terms. This reflects the argument that Jamieson (2018) made about the distinction between fake news and fake news dissemination. She also suggests a fitting term, “viral deception,” to designate the process of spreading fake news. Disentangling process and outcome based on my model of information quality increases not only conceptual clarity. It also offers analytical leverage. For example, existing research suggests that “misleading statements” (outcome) are produced by “misinformation” (process), but they could also stem from “disinformation” (process), “spreading of rumors” (process), “viral deception” (process), or a range of other processes unrelated to malicious motives and intentional fabrication, such as acting with limited political sophistication or carelessness, which are currently not being investigated.

Second, the concept of “falsified preferences” pioneered by Kuran (1997) puts an entirely different twist on the motives behind the production of invalid information, which highlights once again the importance of keeping outcome and process conceptually separate. The literature on fake news and similar phenomena suggests that the production and dissemination of invalid information results from nefarious intentions of powerful people and organizations. Kuran (1997), by contrast, demonstrates that producing and disseminating political lies can also be a necessary protective shield wielded by the weak. He suggests that oppositional politicians in repressive regimes falsify their preferences to avoid persecution, which shows that low-quality political information can come from the best of intentions. Falsifying preferences constitutes a case of invalid component claims, but it is prompted by survival instinct and a position of weakness rather than power and malicious intent.

Third, in contrast to the other two strands of research, studies about the impact of information quality on political attitudes remain silent on the intentions of message communicators. In the framework of the ELM, these studies propose that “argument quality” (Areni and Lutz, 1988; Petty and Wegener, 1991) entails the validity of a connection claim about the consequences of an action (“Allowing shale gas fracking will make us all richer”) and the validity of a component claim about the desirability of that consequence (“being rich is a good thing”). The distinction was pioneered by Areni and Lutz (1988). They use the term “argument strength” to designate the validity of the connection claim and “argument valence” for the validity of the component claim. Petty and Wegener (1991) use the terms “likelihood of consequence” for the former and “desirability of consequence” for the latter. In existing empirical research, Hoeken et al. (2014) investigate both aspects of “argument quality” using the terms coined by Petty and Wegener (1991), while other studies focus on the quality of the connection claim using terms such as “validity of action-outcome claim” (Austen-Smith, 1992) and “probability of consequence following from action” (Schellens and de Jong, 2004).

The validity of information is distinct from the persuasive effectiveness of information. As a matter of fact, in contrast to the literature on political persuasion discussed so far, the majority of contributions in the tradition of the ELM investigating “argument quality” or the interchangeably used “argument strength” (Eagly and Chaiken, 1993; Petty and Cacioppo, 1981, 1986; Rydell and McConnell, 2005) as well as studies of “frame strength” (Aarøe, 2011; Chong and Druckman, 2007; Druckman et al., 2013) are not about the quality of information but rather the capacity of information to change opinions, or in other words, the information’s persuasive effectiveness. However, information quality and its various aspects are inherent features of a political statement, whereas the capacity of a political statement to change opinions is a consequence of the statement and a range of other factors associated with it, such as the speaker and the channel.

The focus of research about argument strength on the persuasive effectiveness of information reflects the original formulations of the ELM (Petty and Cacioppo, 1981, 1986), which distinguishes weak from strong arguments based on how compelling and impactful people find them. Research on frame strength applies the same principle to the analysis of frames, defining “a frame’s strength as increasing with its perceived persuasiveness” (Chong and Druckman, 2007: 638), so that “(w)eak frames are typically seen as unpersuasive, whereas strong frames are compelling.” In order to advance research on political persuasion, Chong and Druckman (2007) and Aarøe (2011) highlight the critical distinction between the nature (including the quality) of political information and its capacity to change opinions, and they call for more research about the impact of quality on persuasiveness. By clarifying this distinction, and by identifying relevance, intelligibility, and validity as separate features of information quality, my model helps to delineate more clearly what information quality is about, and it provides the conceptual foundation for studying the impact of quality on persuasive effectiveness. The proposed model also facilitates analyses about the effects of information quality on other processes besides persuasion. For instance, the model makes it possible to study whether information quality changes the “affective tipping point” at which voters stop ignoring negative information about a preferred candidate (Redlawsk et al., 2010), whether affective reactions to information depend on variation in quality (Erisen et al., 2014), and how information quality influences the likelihood of making informed vote choices (Lau and Redlawsk, 1997).

Advancing New Empirical Research

My model of the quality of political information can be used in empirical research to determine variation in information quality, to identify different political messages and their common quality deficiencies, and to generate new research questions and theoretical expectations. I am now discussing each of these three applications in turn. First, the model can be used to determine the quality of political information at all levels of aggregation, including the foundational political statements as well as political messages, information packages, and information distributors. The six manifestations of information quality established by the model can be assessed using different approaches to measurement, which vary in their nature (subjective or objective) and unit of analysis (message or recipient). Besides epistemology and operationalization, using one or the other approach also has important substantive implications. For instance, if objective validity deviates from subjective validity in the eyes of some politician’s key constituency, it makes a huge difference for the nature of political communication, democratic standards, and public policy whether the politician aspires to tell the objective truth or the subjectively perceived truth of his median voter.

The objective-recipient approach involves objective measurement of how individuals who receive information judge its quality. This is clearly the most demanding measurement strategy. It could be implemented, for instance, by recording physiological responses to information. The subjective-recipient approach relies on the subjective judgments made by information recipients. It can be implemented through thought listing (Cacioppo et al., 1981) as well as direct questions about the content of information (Zhao et al., 2011). Following a subjective-message approach means to aggregate individual responses to the message level, for example, by calculating an average value of intelligibility of a political message from a survey of individual judgments.

In research on political persuasion, the dominant subjective-recipient and subjective-message approaches (Cacioppo et al., 1981; Chong and Druckman, 2007) have been countered by calls for an objective-message approach (O’Keefe and Jackson, 1995; Petty and Wegener, 1991). The objective-message approach applies normative standards, transparent procedures, and expertise to judge quality. Fact-checking websites, for example, rely on standardized coding rules as well as expert coders to determine the validity of politicians’ component claims. Studies of “readability” (Kayam, 2017) and “language complexity” (Schoonvelde et al., 2019) employ linguistic tools to assess important facets of intelligibility. In addition, a wide range of other techniques from the bourgeoning field of text analysis can be applied to study the quality of political information using the objective-message approach. This includes hand-coding (Kriesi et al., 2019) and automated procedures such as linguistic inquiry and word count (LIWC) (Windsor et al., 2018).

Second, my model can be used to identify different types of political messages and their common deficiencies in information quality. The model is intentionally unconstrained, so that it can be applied as a conceptual template for the analysis of all categories of political messages. Public service announcements, campaign slogans, ministerial memos, and diplomatic declarations all consist of component and connection claims. But they differ in terms of purpose as well as the typical number and interrelations of claims, so that they feature specific ways of exhibiting variation in intelligibility, relevance, and validity. My model can be used to identify the structure and composition of different political messages as well as their typical expressions of variation in quality.

The political message that I discuss now to illustrate the usefulness of my model as a conceptual template, the policy justification, constitutes a kind of connection claim that serves the purpose of justifying a policy. It consists of two component claims (“this is a desirable policy,” “that is a worthy norm”) and a causal link between them (“this desirable policy advances that worthy norm”). The causal connection between the two component claims indicates a positive effect of the desirable policy on the worthy norm, which then justifies support for the policy (see Figure 2). The statements “I support shale gas fracking, because it will increase our prosperity” and “I support shale gas fracking, because my party, to which I am loyal, supports it” represent typical examples of policy justifications.

Figure 2.

Policy Justifications.

My model of the quality of political information works as a conceptual template that allows me to identify common deficiencies in the quality of policy justifications. Table 2 shows specifically how judgments of low quality suggested by the model (not intelligible, not valid, not relevant) can be applied to the three components of which policy justifications are composed. This yields nine quality deficiencies of policy justifications, whose causes and consequences can be empirically investigated. Such research could focus on the ability of citizens to detect low-quality policy justifications, the impact of variation in quality on public opinion, or the conditions under which politicians use low-quality statements.

Table 2.

Quality Deficiencies in Policy Justifications.

	Not intelligible	Not valid	Not relevant
Desirable policy (component claim)	Purpose obfuscation	Fit overestimation	Need misreading
Worthy norm (component claim)	Justification blurring	Value misjudgment	Priority miscalculation
Plausible policy–norm effect (causal link)	Issue clouding	Issue stretching	Issue squeezing

To begin with, issue stretching constitutes an invalid claim about the connection between a policy and a norm that does not amount to an outright lie. It stretches the bounds of validity without breaking them. For instance, the statement “encouraging children to smoke is good for public health” constitutes a severe form of invalidity, while the statement “European integration has been good for social justice” is probably issue-stretched, but not a lie. By contrast, issue squeezing entails a causal connection that is valid yet irrelevant because it is too obvious. For instance, politicians justifying their support for policies extending the authority of the European Union frequently argue that such policies are “good for European integration.” The two component claims contain virtually the same statement. They are squeezed together so tightly that the connection claim they establish becomes irrelevant.

Low relevance can also affect individual component claims. To begin with, need misreading identifies a justification in which an explainer advocates a policy that is irrelevant (to the subjective views of an audience or the objective requirements of a problem). For instance, endorsing a policy of reforming decision-making in the European Council as a specific solution to job losses resulting from digitalization arguably constitutes a case of (objective) need misreading. In the case of priority miscalculation, irrelevance refers to the norm that is invoked to justify a policy. For instance, justifying a policy of European integration by pointing out that it helps French farmers might be considered a case of (subjective) priority miscalculation when the justification is used in an assembly of workers from a decaying industrial town in Italy.

Fit overestimation occurs when a justification contains an invalid claim about the desirability of the justified policy. The policy can be undesirable for lacking objective fit (when it fails to solve problems) or subjective fit (when an audience dislikes it). For instance, advocating the abolition of formal employee consultation in a meeting of trade union leaders constitutes a case of (subjective) fit overestimation, because the audience arguably holds pre-existing negative opinions about the policy. By contrast, value misjudgment describes a policy justification that invokes an unworthy norm to defend a policy. For instance, a justification for European integration which claims that integration will foster Christian values suffers from (subjective) value misjudgment when it is targeted at a group of atheists.

Issue clouding constitutes a form of “connections left implicit” (Schellens and de Jong, 2004) in which an explicit mentioning of the causal link between two component claims would be required to make the justification intelligible. Purpose obfuscation means to make a claim about the desirability of a policy that is not intelligible, and justification blurring does the same for the invoked norm. All three deficiencies of intelligibility can be the result of statements that are genuinely unclear. For instance, a politician who uses pretentious vocabulary to state a norm that is allegedly achieved by a policy will not be understood by his voters (which constitutes a case of justification blurring). In addition, each of the three intelligibility deficiencies can also occur when otherwise clear claims about policies, norms, or policy–norm connections are overwhelmed by too much extraneous information.

Third, my model of the quality of political information can also generate new research questions and theoretical expectations about the causes and consequences of information quality. To begin with, the model can prompt research about the trade-offs that arise when political actors attempt to maximize more than one aspect of information quality at the same time. For instance, greater validity of information communicated by political parties should be negatively related to intelligibility, especially for complex issues, and I would expect that parties will maximize intelligibility above all else to effectively communicate with citizens (while remaining merely in the realm of plausible deniability with respect to the validity of their statements).

Moreover, my distinction between (primary) political statements and (auxiliary) political evidence claims makes it possible to study systematically the way in which evidence affects the quality of political information. We should expect that both the presence and a higher quality of evidence increase subjective and objective validity judgments. But it is instructive to evaluate the boundaries of this effect. Does the accumulation of evidence negatively affect intelligibility and relevance, and are there conditions under which more evidence becomes detrimental even for judgments of validity?

Finally, the model can also facilitate systematic reflection and analysis about the causes of information quality. For instance, why do political actors sometimes make valid, intelligible, and relevant statements and sometimes not? How much of their low-quality political statements can be attributed to cognitive factors, how much to motivation, and how much to variation in motive? In other words, do politicians lack the cognitive and political resources, maybe increasingly overwhelmed by the demands of their vocation, do they get sloppy and careless over time, or are they driven by varying rationales, such as habit, conviction, or strategic intent, that have differential effects on information quality?

Conclusion and Discussion

The article outlined a model of the structure and quality of political information. It showed how different aspects of quality can be applied to judge the building blocks of political information, as well as the political messages that are constructed by using these blocks. The article also outlined a range of empirical applications of the model, including the classification of existing studies, the assessment of variation in information quality, the identification of different classes of political messages and their typical quality deficiencies, as well as the production of new research questions and theoretical expectations.

In addition, empirical applications of the model in different policy fields can also be useful for practitioners of political discourse. Findings about the quality of political statements would allow them to reflect on their rhetoric and facilitate more effective communication with their partners. The model can also contribute to the debate about the future of democracy. Reasoned communication based on high-quality political information constitutes a critical foundation of democratic discourse and accountability. The model helps to strengthen this deliberative core of democracy by offering concepts and tools that allow us to identify variation in quality and detect the propagators of good and bad political information.

Footnotes

Acknowledgements

I am grateful for the opportunity to present earlier versions of this research at the Annual MPSA conference; the APSA political communication pre-conference “Facts, Fiction, and Free Media: The Fourth Estate and Its Discontents”; the Berlin Summer School of Social Sciences; the International Politics Research Forum at City University London; the seminar series of the Department of Political Economy at King’s College London; the Comparative Politics Colloquium at Humboldt University Berlin; the seminar series of the Department of Democracy and Democratization at the Social Science Center Berlin (WZB); and the Berlin–Brandenburg Workshop for Research on Political Behavior. For very helpful comments and feedback, I would like to express my sincere gratitude to the anonymous reviewers and the editors of this journal as well as Diana Burlacu, Taylor Carlson, Johannes Gerschewski, Ellen Immergut, Morgan Marietta, Theres Matthieß, Stefano Pagliari, Till Weber, and Bernhard Wessels. Special thanks go to Robert Luskin for encouraging me to develop this research, and to Olivia Maky for introducing me to the work of Paul Grice. I would like to thank Lalit Chennamaneni, Tim Friedrich, Thomas Maruhn, Marcel Skaun, Katharina Storms, Friederike Talbot, and Dominik Vent for excellent research assistance and feedback. I gratefully acknowledge support from the Deutsche Forschungsgemeinschaft (German research foundation), grant number VO 1990/1-1.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The author gratefully acknowledges support from the Deutsche Forschungsgemeinschaft (German research foundation), grant number VO 1990/1-1.

ORCID iD

Konstantin Vössing

Notes

Author Biography

Konstantin Vössing is currently a lecturer in Comparative Politics at City University of London. He has had previous appointments at Ohio State University, Humboldt University Berlin, Harvard University, and the European University Institute. His research is concerned with European integration, public opinion, political leadership, and political mobilization.

References

Aarøe

(2011) Investigating Frame Strength: The Case of Episodic and Thematic Frames. Political Communication 28 (2): 207–226.

Albertson

(2015) Dog-Whistle Politics: Multivocal Communication and Religious Appeals. Political Behavior 37: 3–26.

Areni

Lutz

(1988) The Role of Argument Quality in the Elaboration Likelihood Model. Advances in Consumer Research 15: 197–203.

Austen-Smith

(1992) Strategic Models of Talk in Political Decision Making. International Political Science Review 13 (1): 45–58.

Bartels

(2002) Beyond the Running Tally: Partisan Bias in Political Perceptions. Political Behavior 24 (2): 117–150.

Berinsky

(2017) Rumors and Health Care Reform: Experiments in Political Misinformation. British Journal of Political Science 47: 241–262.

Cacioppo

Harkins

Petty

(1981) The Nature of Attitudes and Cognitive Responses and Their Relationships to Behavior. In: Petty

Ostrom

Brock

(eds) Cognitive Responses in Persuasion. Hillsdale: Erlbaum, pp.31–54.

Chong

Druckman

(2007) Framing Public Opinion in Competitive Democracies. American Political Science Review 101 (4): 637–645.

Clark

Wegener

(2009) Source Entitativity and the Elaboration of Persuasive Messages: The Roles of Perceived Efficacy and Message Discrepancy. Journal of Personality and Social Psychology 97 (1): 42–57.

10.

Converse

(1964) The Nature of Belief Systems in Mass Publics. In: Apter

(ed.) Ideology and Its Discontents. New York: Free Press, pp.206–261.

11.

Dalton

(1985) Political Parties and Political Representation: Party Supporters and Party Elites in Nine Nations. Comparative Political Studies 18 (3): 267–299.

12.

Delli Carpini

Keeter

(1996) What Americans Know about Politics and Why It Matters. New Haven, CT: Yale University Press.

13.

Druckman

Peterson

Slothuus

(2013) How Elite Partisan Polarization Affects Public Opinion Formation. American Political Science Review 107 (1): 57–79.

14.

Eagly

Chaiken

(1993) The Psychology of Attitudes. Orlando, FL: Harcourt Brace Jovanovich.

15.

Erisen

Lodge

Taber

(2014) Affective Contagion in Effortful Political Thinking. Political Psychology 35 (2): 187–206.

16.

Ferrara

(2017) Disinformation and Social Bot Operations in the Run Up to the 2017 French Presidential Election. First Monday 22 (8): 1–33.

17.

Grice

(1975) Logic and Conversation. In: Cole

Morgan

(eds) Syntax and Semantics, Vol. 3: Speech Acts. New York: Academic Press, pp.41–58.

18.

Haney Lopez

(2014) Dog-Whistle Politics: How Coded Racial Appeals Have Reinvented Racism and Wrecked the Middle Class. New York: Oxford University Press.

19.

Hochschild

Eisenstein

(2015) Do Facts Matter? Information and Misinformation in American Politics. Political Science Quarterly 130 (4): 585–624.

20.

Hoeken

Šorm

Schellens

(2014) Arguing about the Likelihood of Consequences: Laypeople’s Criteria to Distinguish Strong Arguments from Weak Ones. Thinking & Reasoning 20 (1): 77–98.

21.

Holbrook

Weinschenk

(2020) Information, Political Bias, and Public Perceptions of Local Conditions in U.S. Cities. Political Research Quarterly 73 (1): 221–236.

22.

Jamieson

(2018) Recognizing and Debunking Political Deception and Viral Deception (“Fake News”). Patten Lecture at Indiana University, February 27. Available at: https://patten.indiana.edu/lecturers2/Jamieson-Kathleen-Hall.html (accessed 15 April 2020).

23.

Jerit

(2009) Understanding the Knowledge Gap: The Role of Experts and Journalists. Journal of Politics 71 (2): 442–456.

24.

Jerit

Barabas

(2006) Bankrupt Rhetoric: How Misleading Information Affects Knowledge about Social Security. Public Opinion Quarterly 70 (3): 278–303.

25.

Kayam

(2017) The Readability and Simplicity of Donald Trump’s Language. Political Studies Review 16 (1): 73–88.

26.

Kriesi

Hutter

Bojar

(2019) Contentious Episode Analysis. Mobilization 24 (3): 251–273.

27.

Kuklinski

Quirk

Jerit

, et al. (2001) The Political Environment and Citizen Competence. American Journal of Political Science 45 (2): 410–424.

28.

Kuran

(1997) Private Truths, Public Lies: The Social Consequences of Preference Falsification. Cambridge, MA: Harvard University Press.

29.

Lau

Redlawsk

(1997) Voting Correctly. American Political Science Review 91 (3): 585–598.

30.

Lazer

DMJ

Baum

Benkler

, et al. (2018) The Science of Fake News. Science 359 (6380): 1094–1096.

31.

Proksch

S-O

Slapin

(2014) Ideological Clarity in Multiparty Competition: A New Measure and Test Using Election Manifestos. British Journal of Political Science 46: 591–610.

32.

Nyhan

Reifler

(2010) When Corrections Fail: The Persistence of Political Misperceptions. Political Behavior 32: 303–330.

33.

Nyhan

Reifler

(2015) Displacing Misinformation about Events: An Experimental Test of Causal Corrections. Journal of Experimental Political Science 2: 81–93.

34.

O’Keefe

Jackson

(1995) Argument Quality and Persuasive Effects: A Review of Current Approaches. In: Jackson

(ed.) Argumentation and Values. Annandale, VA: Speech Communication Association, pp.88–92.

35.

Petty

Cacioppo

(1981) Attitudes and Persuasion: Classic and Contemporary Approaches. Dubuque, IA: William C. Brown.

36.

Petty

Cacioppo

(1986) The Elaboration Likelihood Model of Persuasion. Advances in Experimental Social Psychology 19: 123–205.

37.

Petty

Wegener

(1991) Thought Systems, Argument Quality, and Persuasion. In: Wyer

Srull

(eds) The Content, Structure, and Operation of Thought Systems. Hillsdale, NJ: Lawrence Erlbaum, pp.147–161.

38.

Redlawsk

Civettini

Emmerson

(2010) The Affective Tipping Point: Do Motivated Reasoners Ever “Get It”? Political Psychology 31 (4): 563–593.

39.

Richey

(2017) Contemporary Russian Revisionism: Understanding the Kremlin’s Hybrid Warfare and the Strategic and Tactical Deployment of Disinformation. Asia-Europe Journal 16: 101–113.

40.

Rydell

McConnell

(2005) Perceptions of Entitativity and Attitude Change. Personality and Social Psychology Bulletin 31 (1): 99–110.

41.

Schedler

(2013) The Politics of Uncertainty: Sustaining and Subverting Electoral Authoritarianism. New York: Oxford University Press.

42.

Schellens

de Jong

(2004) Argumentation Schemes in Persuasive Brochures. Argumentation 18: 295–323.

43.

Schoonvelde

Brosius

Schumacher

, et al. (2019) Liberals Lecture, Conservatives Communicate: Analyzing Complexity and Ideology in 381.609 Political Speeches. PLoS ONE 14 (2): e0208450.

44.

Shikano

Stoffel

Tepe

(2017) Information Accuracy in Legislative Oversight: Theoretical Implications and Experimental Evidence. Rationality and Society 29 (2): 226–254.

45.

Toulmin

(2003) The Uses of Argument. New York: Cambridge University Press.

46.

Van Eemeren

Grootendorst

(2004) A Systematic Theory of Argumentation: The Pragma-Dialectical Approach. Cambridge: Cambridge University Press.

47.

Vössing

(2017) How Leaders Mobilize Workers: Social Democracy, Revolution, and Moderate Syndicalism. New York: Cambridge University Press.

48.

Vössing

Weber

(2019) Information Behavior and Political Preferences. British Journal of Political Science 49 (2): 533–556.

49.

Weyland

(2014) Making Waves: Democratic Contention in Europe and Latin America since the Revolutions of 1848. New York: Cambridge University Press.

50.

Windsor

Dowell

Windsor

, et al. (2018) Leader Language and Political Survival Strategies. International Interactions 44 (2): 321–336.

51.

Zhao

Strasser

Cappella

, et al. (2011) A Measure of Perceived Argument Strength: Reliability and Validity. Communication Methods and Measures 5 (1): 48–75.