From fallacies to semi-fake news: Improving the identification of misinformation triggers across digital media

An open-ended definition

It is widely recognised that the term fake news is an umbrella term rather than a consistently defined notion (Levi, 2018; Tandoc et al., 2018). The common ground among these definitions is that ‘fake news’ looks like real news without being so. Tandoc et al. (2018) point out that these definitions can be positioned on a continuum along the two dimensions of facticity and intention. Verstrate et al. (2017) also adopt a binary set of features to define a typology of fake news, putting next to intention to deceive, the presence of a financial motivation to create fake news.

Acknowledging the complexity of mis- and dis-information which happen to be shaped by national information environments (Humprecht, 2019), Wardle (2017) strays away from a holistic definition of fake news, proposing a clustered approach to characterise the information eco-system encompassing seven different types. What distinguishes the latter taxonomy from the others is that both the types of content created and their (un)intentional dissemination play a crucial role. As remarked by Gelfert (2018), however, in media studies the phenomenon of fake news is framed as an epistemological issue, where different criteria to evaluate news trustworthiness rely on the analysis of the news source or the news content, without accounting for the recipients’ ability to critically questions the news, especially when not felt dubious according to their judgement (Tandoc et al., 2018). Regardless of the gap between reported facts and reality, according to Gelfert (2018: 108), news is fake news if it manages to deceive at least a part of the audience: ‘Fake news is the deliberate presentation of (typically) false or misleading claims as news, where the claims are misleading by design [. . .] either because it aims to instil falsehoods in its target audience [. . .] or because the way it is deliberately operated is objectively likely to mislead its target audience’. As explained by Schwarzenegger (2020), personal epistemologies based on selective criticality, pragmatic trust and competence-confidence make it difficult to predict what target audience might be reached by fake news.

In line with Gelfert (2018) we believe that fake news constitutes a co-constructed concept where the audience’s reception is as important as the authors’ intention: a fake news recognised as such by readers stops to fulfil its ultimate goal of deception. But this awareness does not bring us to the end of the conundrum since the context of digital media makes it difficult to identify what counts as deliberation in the first place. It is, for example, true that in social media such as Facebook (mis)information is tightly correlated with filter bubbles, but users’ online conviviality plays a crucial role too in spreading fake news (Seargeant and Tagg, 2019), impacting at the same time interpersonal relationships when shared news turn out to be fake (Duffy et al., 2020).

Newsrooms feature a variety of actors involved in the deliberation process of news construction: an editorial move to change the original title of a news to make it more ‘clickable’ and increase visibility over a crucial matter might, for example, result in unintentional misleading information. This is especially the case when a crisis situation, such as the COVID-19 pandemic, brings us to a post-truth scenario (Collins, 2018): the radical uncertainty underlying preventive measures and future developments goes hand in hand with scientific advancement rendering objectivity a chimaera. Taking the BBC editorial guidelines as a benchmark (https://tinyurl.com/y6awgk3y) it is evident that the COVID-19 pandemic ticks all the boxes to be deemed not only a ‘controversial subject’, but even a ‘major matter’. However, advocated measures to deal with such a complexity such as giving ‘due weight to various opinions or distinguish opinions from fact’ are challenging: the results of a scientific study over the efficacy of a drug/vaccine are inherently provisional and might be discarded by larger trials. Thus, in such an unprecedented situation, to understand what counts as mis- or dis-information and goes against the public interest, we start from the analysis of those news that have been flagged as ‘fake’ by fact-checkers and have so far caused harm to citizens. Fact-checked news constitute a relevant sample of analysis since they have been selected by fact-checkers on the basis of their newsworthiness, measured by looking at analytics of their popularity on social media.

Fact-checking fake news

One of the challenges in using a notion of truth as the yardstick against which to measure fakery is that it assumes that language can somehow directly reflect truth, and that objectively measurable truth detection is a commonplace. This, indeed, is the assumption that underlies the very institution of the fact-checker: the idea that some stretch of text that can be identified as a fact can then be checked off against truth in the world. With such a conceptually clean picture being presupposed by the activity of fact-checkers, and even by their very name, it is no wonder that the computational sciences have latched on to it with such vigour. Large scale competitions or shared tasks such as FEVER (Thorne et al., 2018) set out to pit research labs against one another in developing AI algorithms (typically based in statistical models of language use) that can either identify which passages of text constitute checkable facts, or, more typically, can identify whether or not a given passage of text constitutes something that is true or not. This conception, however, belies the difficulty of identifying fake news, and belies what it is that fact checkers do.

In their computational account, Hanselowski et al. (2019) compare the major fact checking organisations and report that the only characteristic that all fact checkers have in common is textual commentary rather than a systematic methodology to identify factuality. The reason is that fact checkers understand full well that objective truth is hard to come by, and, furthermore, that by far the most common category of fakery from Wardle’s list is Misleading Content. We can see this even in fact checker datasets. Snopes, for example, collected into a machine-processable dataset in Hanselowski et al. (2019), comprises 2943 examples marked as false, 659 marked as true and 2890 marked with ratings pointing to not true and not false content. The acknowledgement of fake news as a continuum is reflected in the fact-checker’s truth barometer labels which contain many more values than the mere false and true, at different degrees of granularity: Healthfeedback.org, for example, features Inappropriate sources next to the vague Misleading, while Glenn Kessler’s fact checker puts together the Upside down Pinocchio (‘a statement that represents a clear but unacknowledged flip-flop from a previously held position’) and the Four Pinocchio (‘whoppers’). This lack of consistency in ratings shows that, despite general agreement that some news are more fake than others, a systematic methodology to name and scaffold misinformation is currently missing.

From (semi) fake news to fallacies

Fake news is news that has the appearance of proper news but is not. Fallacies are generally defined as arguments that seem to be valid but are not (Hamblin, 1970: 12). Thus, fake news pieces are likely hosts for fallacious arguments. The ability to identify fallacious arguments would constitute an asset for the journalist to avoid spreading potentially misleading information and for citizens to interpret them cum grano salis. The advent of digital and social media has, in fact, brought to an end the era of the ipse dixit, where few news media agencies were responsible for informing the public in a top down manner, in favour of what Sunstein (2018) calls a ‘divided democracy’. In such a scenario of news democratisation, social media platforms work as a contemporary agora that is subdued by Rhetorics at different levels: what news (fake or true) becomes viral is, for example, ultimately, a matter of rhetoric. Even the way we select our news feed can be critically interpreted through rhetorical lenses. What Negroponte et al. (1997) have long ago coined as the ‘Daily Me’, our personalised design of the information package we choose to receive, is an instantiation of the cherry picking fallacy: we decide to reinforce the world-view that we like exposing ourselves only to certain topics, certain sources etc. In designing our news architecture we are prompted by the algorithms which propose news liked by our friends, according to the ancient principle of homophilia, the tendency to appreciate what our peers appreciate. As underlined by Aristotle, Rhetoric is useful since it allows understanding reasons behind incorrect decision making processes (Rhet. 1355a22-25). In line with this view, if citizens make wrong decisions, it is the ecosystem of (mis)information to be blamed and strengthening their judgement criteria promises to make them better judges.

From fallacies to semi-fake news

In 1847, Augustus De Morgan begins the chapter ‘On Fallacies’ in his Formal Logic, with the statement, ‘There is no such thing as a classification of the ways in which men may arrive at an error: it is much to be doubted whether there ever can be’ (de Morgan, 1847: 237; emphasis in the original). This attitude that the study and attempt to taxonomise fallacies was inherently doomed was an attitude that prevailed for more than a century. It was not until the Australian philosopher Hamblin (1970) addressed the issue in Fallacies that the field started upon its road to academic rehabilitation. Compendious approaches to fallacies vary widely, but perhaps a common core can be identified that have been dubbed by Woods (2004) the ‘gang of eighteen’: equivocation, amphiboly, composition, division, petition, complex question, post hoc ergo propter hoc, ignoratio elenchi, ad verecundiam, ad populum, ad baculum, ad misericordiam, ad hominem, faulty analogy, slippery slope, affirming the consequent and denying the antecedent. The final two are representative of the class of formal fallacies, and reflect deep cognitive biases that have occupied psychologists for more than half a century (Wason, 1968). It is a subset of the remaining gang members that are of interest to us here: the informal fallacies. What is intriguing about informal fallacies is the way in which their invalidity is fuzzy. Whilst from a classical, logical perspective, a piece of reasoning may be incorrect or faulty, that same reasoning might (with appropriate context) be a perfectly sensible way of proceeding. Philosophers such as Walton (1996) have explored in detail the ways in which instances of what might classically be regarded as fallacies can be used justifiably, rationally and effectively – with more recent linguistic exploration uncovering the extraordinarily wide extent to which these patterns are used (Visser et al., 2020a). The fuzziness of the delineation of fallacious reasoning is precisely what underpins the notion of semi-fake news and what makes recognising such news so demanding. We found our approach on insights from one of the most extensive and incisive writers on fallacies. Bentham (2015), writing over 200 years ago, offers a scaffold that can help us. Talking in the context of fallacies in parliament, he distinguishes the social-discursive roles involved in fallacy (Bentham, 2015: 38), viz., the fabricator, utterer and acceptor. His explication rests on an analogy with counterfeit currency: the fabricator creates and first employs the deceit and may trigger a chain of subsequent acceptance and re-use, with acceptors receiving the fakery either cognisantly or not, and then re-using the fake similarly. The parallel with fake news in social media is striking. Fakery is first fabricated, then uttered (or tweeted) and then re-uttered (or re-tweeted) to acceptors (or followers). Bentham (2015) also issues an engagingly modern call in his demand for annotation of fallacies (in his case, in Hansard, the parliamentary record): ‘in each instance in which the use of any such [fallacy] is discoverable, let him at the bottom of the page. . .give intimation of it’ (p. 74). Bentham (2015), of course, was imagining a manual process of such intimation, yet his eloquent call to arms (which here is referred to as, ‘the faculty which Detection has of stripping Deception of her power’ (p. 79)) is one to which, two centuries later, we can respond with a combination of manual and automatic means.

The first challenge for any empirical study of informal fallacies is a reliable taxonomy that can serve as the foundation for analysis and annotation. Whilst Aristotle, for example, in his Sophistical Refutations, distinguishes fallacies dependent on language (in dictione) from those not dependent on language (extra dictionem), Whately (1875) cleaves logical fallacies from semi-logical ones which require world knowledge to be interpreted. Pragmatic frameworks classify fallacies as infringements of the rules of an ideal critical discussion (Van Eemeren and Grootendorst, 2004). Regardless of the taxonomic approach, the key issue at stake though is the Fallacy Fork (Boudry et al., 2015): cut-and-dry compendia of fallacies are unlikely to be reflected in real life discourse. As a result, we have adopted a bottom up approach, seeded by a contemporary taxonomic account.

Dataset and annotation procedure

As a case study to investigate interrelations between fallacies and fake news we have considered the fact-checker Snopes (https://www.snopes.com/about-snopes/). We have chosen to focus on this fact-checker to showcase the relation between fallacies and misinformation during the pandemic for two main reasons. First, Snopes unlike other fact checkers, does not focus on political issues (e.g. governmental responses), but attempts to cover a variety of topics depending on readers’ demand emerging from digital media. Second, it leverages a spectrum of ratings which acknowledges the presence and the importance of nuances in the information ecosystem (https://tinyurl.com/y96ahae6). Next to ratings such as true, false, mostly true, scam, it also includes labels such as unproven or misattribution that point to the reasons behind (un)trustworthiness of the news. When it comes to COVID-19, Snopes has assembled a coronavirus collection currently spread across 18 topics. The number of news considered for each topic reflects readers’ interest determined on the basis of reader email submissions and posts on the fact-checker’s social media accounts as well as general popularity on Google and other social media venues.

As a dataset, we have collected through web scraping all the fact-checked news appearing in the collection till the end of June 2020 distributed across all different topics at that time for an overall number of 220 articles (Table 1):

OPEN IN VIEWER

Table 1.

Composition of the Snopes dataset.

Collection topics	Number of articles
Business	11
Conspiracy	19
Entertainment	8
Gates-Foundation	11
History	5
International response	14
Memes	24
Origins	18
Pandemic	2
Predictions	7
Prevention	21
Protests	10
Trump	41
USA response	38
TOT	220

Even though the numbers are too low to allow for any significant correlation between attested fallacies/types of misinformation and subject matters, the range of topics reduces the risk of contextual bias. For each article we have collected (i) claims (ii) text of the source article (iii) weblink to the source (iv) factchecker’s comment (v) rating associated by the fact-checker (vi) collection type. The data have been archived in CSV files. It has to be noted that this dataset does not allow us to check for privileged associations of fallacies with cases of misinformation rather than disinformation or information. We leave such endeavour for future work, while shedding light on the relevance of certain types of fallacies to flag misinformation news.

Annotation guidelines and output

To investigate in a systematic manner correlations between fallacies and types of misinformation we have conducted an annotation study on the entire dataset. The annotation has been carried out by two students in Communication and Media without any previous knowledge in Argumentation Theory or Informal Logic. We, in fact, wanted to verify whether the task necessarily entails an institutional training and whether our guidelines and intersubjective. They have been introduced to the main notions of fallacy theory during a training session of 1.5 hours. They have been given the same set of 220 CSV files and asked to (i) read both the origin (comment from the fact-checker) and the source text (the text of the fact-checked source) (ii) identify the type of source according to a predefined set of categories (iii) identify the portion of text from the fact-checked source that triggers the critique according to the fact-checker (target critique) (iv) annotate (if any) the fallacy at stake in the original text according to the guidelines.

The categories that we provided for the classification of sources are the following: Broadcast news available only through a digital channel; Broadcast news multi-channel; Government and official political sources; Social media; Blog posts (personal or official); Scientific Papers. The rationale in adopting such categories is centred around the presence/absence and type of gatekeeping process at stake.

The guidelines for the annotation of fallacies contain a taxonomy of 10 fallacies. We have developed this typology using a bottom up approach: our starting point is Tindale (2007), which gathers the most common fallacies discussed in the informal logic tradition. Using this as a basis, we have commissioned expert analysis of 40 fact-checking commentaries and their source articles randomly picked from the dataset in order to identify which fallacies have been called out by the fact checker. We then summarise the most common fallacies identified.

The resulting annotation schema includes 10 types of fallacies organised into four classes related to the distribution of arguments and supported claims: evading the burden of proof (EBP); the (un)intentional diversion of the attention from the issue at hand: strawman (ST), false authority (FAUT), red herring (RH) and cherry picking (CP); the argument schemes at play: false analogy (FA), hasty generalisation (HG), post hoc (PH) and false cause (FC); and the language used: vagueness (VAG). As a verification step, we have analysed the definitions of the different verdicts and labels employed by main fact-checkers in English (snopes.com; healthfeedback.org; politifact.com; fullfact.org; theferret.scot;) to see whether critiques might point to fallacious moves different from the ones identified. Our set covers the fact-checkers ratings completely: even if not exhaustively representing the universe of fallacies, our sub-selection covers the most frequent fallacious moves executed in online news. To offer a degree of systematicity, the fallacies have been arranged starting from those having to do with the quantity of information provided (structural fallacies), followed by those related to aspects external to the issue discussed (fallacies from diversion); logical fallacies come into place after the other two classes are excluded. This order echoes the one provided by the pragma-dialectic rules for a critical discussion (Van Eemeren and Snoeck Henkemans, 2016), a series of principles which are meant to guarantee the reasonableness of an ideal discussion: the violations of rule 8 (Argument Scheme Rule) follow the violations of rule 2 (Burden- of- Proof Rule), rule 3 (Standpoint Rule) and rule 4 (Relevance Rule). It is, in fact, not worth looking at the reasoning at play if the information conveyed in the arguments is irrelevant for the conclusion. It has to be remarked that the PragmaDialectic definition of fallacies differs for the standard one of Informal Logic: fallacies are speech acts which hinder the resolution of a dispute due to the violation of the rules of a critical discussion (Van Eemeren and Grootendorst, 1987). Even though the validity of an argument goes beyond those rules, the normativity of the framework offers a rationale to prioritise certain critical questions over others.

The broad fallacy class encompassing vagueness, equivocation and ambiguity occupies the final position in the list, as a catch-all category for when the other options are excluded.

In our guidelines, each fallacy is first defined and then associated with an example and accompanied by one or more identification questions, which have turned out to be useful means to evaluate arguments (Song et al., 2014). In this way, the annotator can go through the identification questions in a binary way stopping when one of the identification questions applies. The pattern here is to exploit the notion of a dichotomous key, developed in taxonomic biology (Dallwitz, 1980) and applied successfully in the notoriously demanding task of identification of argumentation schemes (Visser et al., 2020b). The annotators were also given the option to use the label ‘none’ when no identification would apply, but they still thought that some type of misinformation would be in place. Let’s consider a random instance of news claim from our dataset accompanied by the fact-checker comment:

Annotators, after having read both the news claims and the comment, are asked to decide whether and which fallacy is at stake browsing the heuristics in Figure 1:

In order to evaluate the reliability of the annotations we have first calculated the inter-annotator agreement (IAA) using Cohen’s (1960) coefficient. For the type of source, the kappa value amounts to 0.988 which corresponds to perfect agreement according to Landis and Koch (1977). The annotation both of the type of source and of fallacies has been carried out on top of the entire dataset (220 news). The annotation of fallacy types has been carried out on top of 73/220 news: the excluded news are those labelled by the fact checkers as false (94 cases), true (36 cases), correct attribution (10 cases), scam (3 cases) and satire (4 cases).

OPEN IN VIEWER

Figure 1.

Heuristic identification key for fallacies.

For fallacy types, we have achieved a 0.41 kappa, which corresponds to moderate agreement. Such a result outperforms those obtained in tasks of similar complexity such as the annotation of argument schemes (Musi et al., 2016). All cases of disagreement have been checked by an expert, achieving an accurate gold annotation. Our dataset is the first annotated as to fallacies in the misinformation environment.

Distribution of fallacious news

To test our hypothesis that fallacies work as indicators of misinformation, we have first looked at the distribution of annotated fallacies across ratings pointing to misinformation (misattribution, mostly true, unproven, mixture, miscaptioned, labelled satire and mostly false):

As shown in Figure 2, according to the gold annotation, for each of the news articles tagged with a misinformation rating, a fallacy from our taxonomy has been identified, suggesting that our taxonomy exhaustively accounts for cases of misinformation in our sample. Overall, while the structural fallacy evading the burden of proof is the most frequent (15%); fallacies from diversion (strawman, 14%; false authority, 4%; cherry picking, 11%; red herring, 8%) and language (vagueness, 8%) also occupy a prominent position, leaving logical fallacies to a lower frequency on average (post hoc, 2%; false cause, 4%; hasty generalisation, 7%; false analogy, 4%).

OPEN IN VIEWER

Figure 2.

Distribution of fallacies across the misinformation ecosystem.

To check whether certain news venues tend to be associated with specific fallacious news types, we have looked at the types of sources that host them, keeping cases where no fallacies is at stake out of the picture (Figure 3):

OPEN IN VIEWER

Figure 3.

Distribution of fallacy per type of source.

The size of our dataset is too small to surmise any significant correlation between the two variables and call for verification over a larger dataset. However, major encountered trends can be explained through the design of different types of media.

The fallacies of hasty generalisation, cherry picking, red herring and evading the burden of proof are contained in at least 80% of cases in social media. This tendency well resonates with the presence of constraints in shaping and distributing news imposed by the gatekeeping process of official news media venues, regardless their (inter)national scope. Most codes of practices have in common as a core principle both the accuracy and the reliability of information. The Global Charter of Ethics for Journalists, for example, lists a set of rules that would be flouted when one of the above-mentioned fallacies is at stake: rule 2 stresses the importance of accounting for the difference between facts, which do not call for supplementary evidence, and opinions that have to be backed up by arguments (‘He/she will make sure to clearly distinguish factual information from commentary and criticism’); rule 3 calls out the importance of picking relevant information and avoiding cherry-picking behaviours (‘The journalist shall report only in accordance with facts of which he/ she knows the origin. The journalist shall not suppress essential information [. . .]’). Rule 5 reiterates these points underlying that ‘The notion of urgency or immediacy in the dissemination of information shall not take precedence over the verification of facts, sources and/or the offer of a reply’, while social media typically respond to users’ urgency of spreading information of more or less personal and emotional relevance.

It is thus not surprising that news such as ‘COVID-19 contact-tracing apps like Healthy Together and ABTraceTogether are tracking you and also the people in your phone contacts and Facebook friends lists’ have been spread through Facebook posts rather than news editorials (https://tinyurl.com/y3x8zy8z0). The suppression of the crucial information that contact numbers, Facebook lists or users’ locations are collected only under voluntary provision would have been deemed against the journalist’s ethics in an official news outlet. Similarly, official news media outlets have specific rules on how to select images which do not breach editorial standards. As a result, a video that portrays anti-surveillance protesters tearing down a ‘smart’ lamppost in Hong Kong (https://youtu.be/u1Ji7wonUhE) could have not been used to support the claim that 5G played a role in the pandemic, as had been suggested on the Instagram page of the actor Woody Harrelson (https://archive.vn/MzYv9), leading to a red herring fallacy. Instead, the video was correctly captioned by The Guardian on 2019 as evidence for protesters in Hong Kong in August 2019 tearing down ‘smart’ lamp posts used for surveillance (https://tinyurl.com/56ahuv5a).

It still might, of course, be the case that journalists happen to un-intentionally foreground certain types of evidence or draw illegitimate generalisations; however, the presence of standards to be ideally followed is likely to work as a nudging force in avoid such fallacious moves.

When it comes to flawed ways of reasoning in shaping a piece of news given a set of factual evidence, no guidance is formally supplied to or by news media agencies. As a result, professional journalists and social media users have the same epistemological starting points when trying to make sense of scientific studies about the symptoms of the virus or the readiness of a vaccine as well as when reporting about situations with uncertain causes. This might explain why broadcast media constitute almost 50% of the sources containing fallacies related to causal reasoning (false cause fallacy and post hoc fallacy). The headline ‘Teen Who Died of COVID-19 Was Denied Treatment Because He Didn’t Have Health Insurance’ published by Gizmodo (https://tinyurl.com/yw6fsjjw) was based on the initial claim advanced by the mayor of Lancaster in a video posted on YouTube and, thus, publicly available to journalists as well as to the larger public. The story quickly became viral since the teenager was the first to be considered killed by the complications of COVID-19 in the country.

However, as clarified by Snopes (https://www.snopes.com/fact-check/teen-insurance-coronavirus/?collection-id=244110), the mayor then retracted his claim explaining that there was a misunderstanding probably due to language barriers, but the family had health insurance and the teenager received emergency treatment. In such a confused scenario, it is difficult to blame Gizmodo journalists for having committed a ‘false cause’ fallacy: given that urgent care clinics in USA can legally deny treatment to uninsured people and that 27 millions of Americans do not own insurance, the causal relation could have easily been true. Although news outlets, to help the public making sense of reality, cannot avoid adopting the abductive way of reasoning of finding the best possible explanations for state of affairs in view of the available evidence, extra caution should be devoted to avoiding fearmongering when reporting situations that are likely to be felt close and emblematic by the audience. A possible strategy, even if not optimal, is that of modulating the degree of certainty through the use of modal verbs, especially when correlations might be, but are not necessarily, causations. An example is offered by the Mirror which originally published an article entitled ‘Netflix’s Tiger King star Joe Exotic hospitalised after contracting coronavirus in prison’ (https://tinyurl.com/kvwu83kw), but then had then to update its headline specifying that Exotic ‘could’ have contracted the virus but was still not officially diagnosed with it.

The fallacy of ambiguity and vagueness is, in our dataset, the one associated with the most varied set of sources. As to the former (ambiguity) our fact-checked cases show that frequently social media have tightened the tone of recommendations provided by governments and institutions, circulating panic. This mostly happens when the original source provides vague indications about the scope of applicability of the proposed measures. This is the case, for example, of a graphic containing recommendations from the U.S. Centers for Disease Control and Prevention (CDC) for reopening schools amid the COVID-19 coronavirus pandemic entitled ‘here we go’ (https://tinyurl.com/y4oh2uqh) that went viral on Facebook. While the graphic would suggest that the proposed set of rules is stringent, the CDS guidance stated that schools should determine whether and how it is feasible to implement such recommendations (https://tinyurl.com/y7k7ofnw). In our dataset, vagueness, when identified in broadcast media, often resides in the titles of the news, which allows for twisted interpretations. An example is offered by the Raw Story report entitled ‘Here’s how the Kushner family is cashing in on the coronavirus’ (https://tinyurl.com/y3t5ba5p), where the phrasal verb ‘cashing in on’ could be interpreted both as merely getting financial revenue from a situation or taking advantage of a situation in an unfair way. As underlined by Snopes, while it is true that the Kushner brothers are co-founders of the health insurance start-up Oscar, which released an online tool to locate COVID-19 testing centres in some areas, there is no evidence that the startup is linked to any public damage. Therefore, the use of polysemous terms in news titles shall be avoided since potentially misleading for the majority of readers who are used to getting their daily news feed scrolling through news titles.

Towards a fallacy truth barometer

In order to investigate how fallacies entangle the information ecosystem we have looked at the correlations between types of rating and fallacies types (Figure 4):

OPEN IN VIEWER

Figure 4.

Fallacy types and Snopes ratings.

The categories mostly false and mixture are compatible with a wide array of fallacies from all four classes: the two ratings point to a degree of veridicality rather than the trigger of potentially misleading content. The rating mostly true, defined as a counterpart to mostly false, is attested in correspondence of three types of fallacies only, but it could, in principle, be compatible with almost any kind. The other three ratings correspond, instead, to a more constrained set of fallacies. When the rating miscaptioned is marked, what the fact checkers are pointing at is mostly the presence of a red herring fallacy: multi-modal content (images, videos) is generally used in news to provide arguments for a claim easy to process; the presence of a miscaption that falsely describes the origin, the context or the meaning of an image invalidates its relevance as a source of evidence. A common example is a photograph of empty food shelves except for vegan products used to back up claims such as ‘Vegan Foods are Left Unsold During the COVID-19 Pandemic’ (https://tinyurl.com/yy7h4wo3): the pictures were not taken during the pandemic, being available online years before. While representing vegan food products, such pictures are irrelevant to conclude anything related to food consumption habits during the pandemic.

The rating misattribution includes fallacies from false authority where an authority is attributed a claim that has been either tweaked or never uttered. A typical example is the letter, spread on social media, attributed to Johns Hopkins University containing an ‘excellent summary’ of advice on how to avoid catching COVID-19 (https://tinyurl.com/y6cz6mbd), while the content did not originate within the university.

Finally, the rating Unproven co-occurs with the fallacy of evading the burden of proof since no or poor evidence for the claim is offered apart from an individual’s guarantee. An example is Dr. Vladimir Zelenko’s claim about the treatment of COVID-19 coronavirus patients with a cocktail of hydroxychloroquine, azithromycin and zinc, which rests solely on his words (https://tinyurl.com/y2k86xg4).

The identification of fallacies could be used as a backbone to build more informative fact-checking ratings. Let’s compare two examples both tagged by Snopes as Mixture (Table 2):

OPEN IN VIEWER

Table 2.

Examples rated as Mixture by Snopes..

Claims	What is true	What is false	Source
1. Dr. Anthony Fauci said there was ‘nothing to worry about’ in late February 2020 in regards to COVID-19 and it was ‘safe’ to do things like go to the movies and the gym. (https://tinyurl.com/yy7b5j85)	During a Feb. 29, 2020, interview, Dr. Fauci said that at that time and under the circumstances pertaining to that date, Americans didn’t need to change their behaviour patterns.	However, Fauci did not say there was ‘nothing to worry about’, and although he stated that Americans did not yet need to change their behaviours, he noted that what was then classified as the COVID-19 outbreak could require that to change	Social media
2. Amid a nationwide COVID-19 lockdown, Italians reported seeing wildlife such as swans and dolphins ‘returning’ to newly tranquil waterways, ports and canals. (https://tinyurl.com/y4dlxqg8)	Dolphins and swans were indeed spotted in some of Italy’s waterways after the nationwide lockdown was imposed.	Dolphins and swans swimming in Italy’s waterways were not necessarily new phenomena related to reduced human activity during the COVID-19 lockdown.	Social media

As correctly flagged by Snopes both claims contain elements of truth and of falsity. However, the origin and the nature of the false information differs wildly. Claim 1 reports a quote wrongly attributed to Dr Fauci: the sentence ‘there is nothing to worry about’ is a misleading rephrase of his statement uttered on February 29th during the NBC morning talk show Today that, at that moment in time, ‘the risk is still low, but this could change’. By neglecting the provisional tone of Dr Fauci’s assertion explicitly bound to the circumstances, social media posts from Trump’s supporters have misrepresented Dr Fauci’s position. The final goal of the critique is that of blaming the National Institute of Allergy and Infectious Diseases rather than Trump’s administration for mishandling the response to the COVID-19 pandemic. This claim constitutes a clear instance of strawman fallacy since an opponent’s point of view (Dr Fauci for Trump supporters) is distorted to make it easily attackable.

Claim 2, by contrast, gives voice to a non-legitimate causal inference between the lockdown and a revitalised animal wildlife in Italy. Probably eager to come up with good news, social media users have reframed a simple correlation as a causation, falling into a post hoc fallacy. There is, in fact, no scientific evidence that reduced human activity has caused dolphins and swans to go back to Venice, while it is clear that people had more time to notice their presence during the lockdown. In other words, in Claim 2 what is defeasible is the inference linking two states of affairs (‘lockdown with reduced human activity’ and ‘wildlife around’), while in Claim 2 the defeasibility lies in the attack towards Dr Fauci, since based on inaccurate data (a statement that he literally never uttered). Thus, even if both claims contain propositions involving both truth and falsity, the mechanisms underlying the misinformation are inherently different and can be distinguished to provide readers with an appropriate warning and cultivate their critical skills. In this respect, we believe that the broad classes of fallacies identified (structural fallacies, fallacies from diversion, logical fallacies and language fallacies) can serve as clusters for types of misinformation: the questions proposed in our heuristic identification key can serve as systematic criteria for the identification of the misinformation triggers. Going through this set of questions fact-checkers would have the means to target the roots of misinformation, enabling citizens to strengthen their critical skills.