Abstract
Much like a viral contagion, misinformation can spread rapidly from one individual to another. Inoculation theory offers a logical basis for developing a psychological “vaccine” against misinformation. We discuss the origins of inoculation theory, starting with its roots in the 1960s as a “vaccine for brainwash,” and detail the major theoretical and practical innovations that inoculation research has witnessed over the years. Specifically, we review a series of randomized lab and field studies that show that it is possible to preemptively “immunize” people against misinformation by preexposing them to severely weakened doses of the techniques that underlie its production along with ways on how to spot and refute them. We review evidence from interventions that we developed with governments and social media companies to help citizens around the world recognize and resist unwanted attempts to influence and mislead. We conclude with a discussion of important open questions about the effectiveness of inoculation interventions.
In 2018, the World Economic Forum (WEF) Global Risks Report named online misinformation as one of the top global risks to the environmental, economic, technological, and institutional systems on which our future depends (WEF 2018). The spread of misinformation poses a serious threat to the public’s understanding of science (Lewandowsky, Ecker, and Cook 2017; Lewandowsky and van der Linden 2021), including climate science (Lewandowsky, Oberauer, and Gignac 2013) and the safety of vaccines (Kata 2010). Research has shown that false stories can spread faster, further, and deeper than true fact-checked content (Vosoughi, Roy, and Aral 2018). 1 The COVID-19 pandemic has exacerbated the prevalence of online falsehoods, with widespread inaccurate and misleading information about COVID-19 circulating online. This information includes dangerous health advice suggesting that the ingestion of bleach can “cure” COVID-19 and conspiracy theories that portray Bill Gates as a politically motivated mastermind behind the pandemic. In fact, the proliferation of falsehoods about the virus led the director general of the World Health Organization (WHO) to announce, “We are not just fighting an epidemic, we’re fighting an infodemic” (WHO 2020). Critically, belief in misinformation can have downstream effects on attitudes and behavior, such as undermining climate change mitigation (Cook, Lewandowsky, and Ecker 2017; van der Linden 2015), instigating violence (Jolley and Paterson 2020), and lowering vaccination intentions and compliance with public health guidelines (Loomba et al. 2021; Roozenbeek et al. 2020; van der Linden 2022). Accordingly, tackling critical societal issues such as climate change and COVID-19 will require a better understanding of how to effectively counter the spread of misinformation.
Why Fact Checking and Debunking Misinformation Are Not Enough
Across various fields, researchers and policy-makers have sought to find ways to reduce the spread and influence of misinformation, from legal and policy interventions to post hoc corrections such as debunking and fact checking. Policy interventions may include public authorities directly intervening through regulating the media environment or making social media companies liable for third-party content (Alemanno 2018). Alternatively, post hoc corrections or fact checking interventions involve exposing news consumers to factual information or, in some cases, a more detailed “debunking” message that puts forth strong arguments for why previously seen information is false (Chan et al. 2017). Evidence on the effectiveness of these interventions is mixed, but they continue to be widely used (Walter and Murphy 2018; Nyhan et al. 2020).
One particular problem with these post hoc measures is that it is harder to eliminate the influence of misinformation after people have been exposed to it. Indeed, misinformation often continues to influence inferential reasoning, even after it has been formally retracted or corrected, a phenomenon known as the continued influence effect (Lewandowsky et al. 2012). In addition, the mere presence of misinformation in people’s news environment may undermine accurate information, as the persuasive impact of facts can be neutralized by misinformation (Cook, Lewandowsky, and Ecker 2017; van der Linden et al. 2017); and, once exposed, people may be directionally motivated to seek out further misinformation in ways that confirm their social identity (Van Bavel et al. 2021). As such, researchers have taken to studying how we can prevent misinformation from influencing people in the first place, a process known as inoculation or prebunking. Importantly, we note that prebunking and inoculation are related but not synonymous terms. Some scholars argue that the difference between debunking and prebunking is simply defined by timing, or when someone is exposed to a correction. For example, Brashier et al. (2021) distinguish between featuring warning labels on a social media post either before (prebunk) or after (debunk) exposure. However, the timing of a fact check (see also Grady et al. 2021) is not a conventional conceptualization of either inoculation or debunking (see Lewandowsky et al. 2020; Jolley and Douglas 2017). Inoculation requires much more than a simple warning.
Inoculation Theory: Origins and Early Work
Originally developed by McGuire in the 1960s, inoculation theory is rooted in a biological metaphor, proposing that, much like how people can become immunized against viral contagions, individuals can be psychologically “vaccinated” against persuasive attacks (McGuire 1964). The logic is compelling: as with biological vaccinations, the cost of treatment significantly dwarfs the cost of prevention. McGuire and Papageorgis (1961a) reasoned that just as being injected with a weakened dose of a virus (inoculation) is thought to have an immunizing effect, the same may hold for resisting persuasion. Specifically, they hypothesized that it would be more effective to expose people to a weakened dose of an impending persuasive attack (and preemptively refute that attack) than utilizing the traditional approach of bolstering people’s attitudes on an issue with more supportive facts (which they likened to a “healthy diet”). Their initial experiments confirmed the relative efficacy of the inoculation approach (McGuire 1964).
An inoculation message consists of two components: threat and refutational preemption. The threat component involves individuals being made aware that a persuasive attack is imminent, such as by forewarning them that political actors may want to mislead audiences’ attitudes on issues such as climate change or new energy technologies (Bolsen and Druckman 2015; Cook, Lewandowsky, and Ecker 2017; van der Linden et al. 2017). Refutational preemption (or prebunking) refers to providing individuals with tools or arguments to refute future persuasion attempts, that is, providing message receivers with information that they can use to strengthen their attitudes against persuasive attacks (Pfau et al. 2005). These counterarguments should then be readily available when individuals are exposed to similar (but potentially stronger) arguments in the future.
In the early years of inoculation research, McGuire and Papageorgis (1961a, 1961b) focused on noncontroversial “cultural truisms” (e.g., the common knowledge that brushing your teeth reduces the likelihood of tooth decay). They did not focus on propaganda because they argued that testing inoculation messages in the context of cultural truisms represented a “cleaner” test of the theory, as people have generally not been exposed to counterarguments against these beliefs. Although the initial focus of inoculation research largely remained on cultural truisms, researchers have now found that people can be inoculated on a wide range of controversial issues where individuals hold different and even polarized beliefs, ranging from climate change (van der Linden et al. 2017) and GMOs (genetically modified organisms) (Wood 2007) to extremism (Braddock 2019; Lewandowsky and Yesilada 2021) and conspiracy theories (Banas and Miller 2013). A meta-analysis found that the average effect size of inoculation interventions across forty studies with more than ten thousand participants was d = 0.43 (Banas and Rains 2010), commonly interpreted as a medium effect, and suggested by Cohen (1988) to be one “large enough to be visible to the naked eye” (p. 26).
Using Inoculation Theory to Counter Misinformation
Despite the early avances of inoculation theory, researchers only recently began to explore inoculation within the context of its original conception: propaganda and misinformation (for detailed reviews, see Lewandowsky and van der Linden 2021; Compton et al. 2021; van der Linden 2022). This exploration began with a foray into using inoculation to counter climate change misinformation but has since expanded into a variety of domains, including misinformation about COVID-19, political disinformation, and online extremism. One of the main takeaways of this research has been the discovery that inoculating people against a broader technique (e.g., conspiracy theories) can confer protection against a range of specific manifestations of that technique.
Inoculating against misinformation about climate change
One major advance of inoculation theory was its application to misinformation about climate change (van der Linden et al. 2017). Despite 97 percent of climate scientists being in agreement about anthropogenic climate change, widespread misinformation and misleading political campaigns have undermined public understanding of the scientific consensus (Cook, Lewandowsky, and Ecker 2017; van der Linden et al. 2017). A prominent example of such a campaign is the debunked Oregon Petition, a petition that claims to have collected thirty-one thousand signatures from self-described “scientists” who do not believe that humans are causing climate change, which formed the basis of the most viral story about climate change on social media in 2016 (Readfearn 2016). With less than a percentage of the signatories having any expertise in climate science, and with bogus signatories such as Charles Darwin, it is warranted that fact-checking sites have rated this petition as “pants-on-fire-false” (Greenberg 2017). In their study, van der Linden et al. (2017) sought to investigate whether individuals can be inoculated against such specific misinformation. To test this, the researchers assigned participants (N = 2,167) to one of five conditions, in which they were either exposed only to factual information (the 97 percent consensus), only to misinformation (i.e., the petition), or to a combination of the two. Consistent with inoculation theory, the researchers also tested the effect of a brief (forewarning) and a more detailed inoculation message that contained both the forewarning and a preemptive refutation of the bogus petition. The results showed that while exposure to misinformation significantly decreased people’s judgments about the scientific consensus on climate change (d = 0.48), both the brief (d = 0.33) and detailed inoculation interventions (d = 0.75) showed medium to large effects in terms of protecting and even boosting participants’ beliefs about the scientific consensus, regardless of their prior attitudes toward climate change. These results were replicated in several independent studies (see Cook, Lewandowsky, and Ecker 2017; Williams and Bond 2020), including a scenario where the attack was delayed by one week, so that the inoculation remained significant even when participants were shown the misinformation a week later (Maertens, Anseel, and van der Linden 2020). These results extended inoculation theory by demonstrating its applicability to a highly polarized and real-world issue.
Inoculating against different strains of misinformation
Given the vast number of online falsehoods, it is nearly impossible to inoculate people against every individual piece of misinformation. An important potential issue for inoculation theory as a tool to counter misinformation is, therefore, its scalability (Roozenbeek and van der Linden 2018). In their early theorizing, McGuire and Papageorgis (1961a) distinguished between two different forms of refutational content: refutational-same versus refutational-different messages (Compton 2013). Where refutational-same messages inoculate individuals against specific (mis)information that they will later be confronted with (such as in the studies on climate change outlined above), refutational-different messages provide individuals with refutational content dealing with an issue that they will not be directly exposed to in a subsequent persuasive attack.
For example, an inoculation intervention targeting climate misinformation may refute specific misleading arguments surrounding the well-established scientific consensus on anthropogenic climate change. For refutational-same inoculations, the subsequent “attack” contains misinformation on that exact topic (such as the previously mentioned Oregon petition in van der Linden et al. 2017); whereas for refutational-different inoculation, the subsequent attack contains different climate-related misinformation, such as misinformation on how reducing carbon dioxide levels cannot make a difference for the climate. Early studies suggested that inoculation messages were more effective when individuals were exposed to the same arguments that were later refuted (McGuire 1961; McGuire and Papageorgis 1961b, 1962), and one could argue that individuals should be better equipped to resist persuasion by the exact misinformation they had been trained to resist. However, if messages were able to generate resistance to novel persuasive attacks, they could significantly increase the scalability of inoculation interventions.
Importantly, using the same piece of misinformation about climate change as van der Linden et al. (2017), Cook, Lewandowsky, and Ecker (2017) tested whether inoculation messages that focused on exposing the misleading argumentation techniques used in misinformation would also confer attitudinal resistance. They found that forewarning participants about the “fake expert” manipulation technique in the context of the tobacco industry conferred attitudinal resistance against the Oregon Global Warming Petition; this was a promising finding because it suggested that individuals could be inoculated against misinformation strategies, making a wider level of protection possible. Crucially, studies have found that inoculation messages can cross-protect individuals against persuasion attempts on related, but untreated, attitudes, a phenomenon known as the blanket of protection (Compton et al. 2021; Parker, Rains, and Ivanov 2016). For example, Parker, Ivanov, and Compton (2012) showed that inoculation messages addressing unprotected sex also protected attitudes against binge drinking. In a similar vein, by exposing misleading strategies underlying misinformation, inoculation messages targeting misinformation about climate change may simultaneously offer protection against misinformation on other topics (e.g., vaccines) that use the same misleading strategies.
A Broad-Spectrum Vaccine: Key Advances in Inoculation Theory
Building on these findings, researchers have sought to test technique-based inoculation through gamified interventions. One such example is the award-winning Bad News game developed by Roozenbeek and van der Linden (2018, 2019) in collaboration with the Dutch media platform DROG (DROG 2019). In the game, players take on the role of misinformation producers in a simulated social media setting, enhancing the ecological validity of the choice environment. The game draws on the two classic inoculation mechanisms: threat and refutational preemption. First, players are directly forewarned about the threat of fake news, representing the threat element of inoculation theory. Second, through being placed in the shoes of misinformation producers, players are exposed and asked to use weakened doses of six commonly used misinformation strategies (i.e., weak attempts at persuasion): impersonation, polarization, the use of emotions such as moral outrage, conspiracy theories, trolling, and discrediting. The active participation in the generation of misinformation encourages a critical reflection on the tactics used to influence, triggering the generation of internal refutations. This represents the refutational preemption element of inoculation theory and, more specifically, an example of active inoculation, a procedure whereby the counterarguments or “mental defenses” are generated by participants themselves (e.g., McGuire 1961). Interestingly, McGuire and Papageorgis (1962) hypothesized that active inoculation may be more effective due to the higher cognitive effort involved (Tyler et al. 1979). In this way, the game stands in contrast to traditional inoculation research where participants are passively provided with specific refutational content by the experimenter (often in the form of an essay). For a full overview of the Bad News game and its methodology, see van der Linden and Roozenbeek (2020).
Testing inoculation games: In the lab and beyond
Following a randomized pilot study demonstrating the effectiveness of a live card game intervention (Roozenbeek and van der Linden 2018), researchers have since tested the effects of Bad News across multiple studies (Roozenbeek and van der Linden 2019; Roozenbeek et al. 2021; Maertens et al. 2020; Basol, Roozenbeek, and van der Linden 2020). In an initial field test, Roozenbeek and van der Linden (2019) leveraged organic traffic to the game via worldwide media coverage to conduct a study with a pre-post within-subject survey design, which resulted in a large-scale convenience sample (N = 15,000). Participants were exposed to a set of fictional misinformation headlines both before and after gameplay. The authors used fictional test items in the form of Twitter posts, each of which either made use of a misinformation technique featured in the game or contained no misinformation. Fictional items were used to exclude memory and familiarity confounds (participants may have seen real fake news on their social media feed before) and to ensure experimental control over the misinformation strategies used in each of the test items. The study asked participants to evaluate the reliability of each item on a 7-point scale before and after gameplay.
The results showed a significant postgameplay improvement in people’s ability to resist persuasion by misinformation across different misinformation techniques, with a small to medium average effect size of d = 0.34. That is, once people had been inoculated against conspiracy theories, they rated specific examples, such as “scientists discovered greenhouse effect years ago but aren’t allowed to publish it, report claims” and “the Bitcoin exchange rate is being manipulated by a small group of rich bankers” as less reliable, offering a so-called blanket of protection. Although the study found some small variation in the treatment effect across age and ideology (conservatives and older individuals were more susceptible to misinformation prior to gameplay, a finding consistent with current literature; see Guess, Nagler, and Tucker 2019; van der Linden et al. 2020), all subgroups showed significantly lower susceptibility to misinformation after playing.
While these results were promising for the effectiveness of the online intervention, a potential criticism of within-subject designs is that the observed outcome cannot reliably be attributed to the intervention. For this reason, Basol, Roozenbeek, and van der Linden (2020) tested the effectiveness of the intervention in a 2x2 mixed design, comparing the effects of the Bad News game to the effect of a control condition in which participants played Tetris. Basol, Roozenbeek, and van der Linden replicated the findings of the original paper, showing that compared to the control condition, participants found misinformation significantly less reliable in the inoculation (Bad News) condition (d = 0.60). Basol, Roozenbeek, and van der Linden also found that the inoculation intervention significantly improved participants’ confidence in their own ability to assess the reliability of misinformation (importantly, only for those who correctly lowered their reliability judgments). This finding was particularly promising, as individuals who are confident in their beliefs are more resistant to persuasive attacks (Tormala and Petty 2004).
This initial research raised two important questions. The first was whether the “inoculation effect” (the reduction in the perceived reliability of misinformation) was an artifact of the research design. To investigate this, Roozenbeek et al. (2021) investigated both testing effects (whether using a pretest influences the outcome variable or interacts with the intervention) and item effects (whether the intervention itself only improves participants’ judgments on the specific test items used) in the Bad News game. The researchers used two different item sets, exposing one set of participants to item set A before playing and set B after, and vice versa for another set of participants. The results showed that although some item effects did occur, the inoculation intervention still conferred resistance against previously unseen misinformation. Furthermore, using a Solomon’s Three Group Design, they found that the Bad News intervention did not suffer from testing effects, as the use of a pretest did not influence performance on the post-test.
A second question raised by the initial evaluations of gamified inoculation interventions was whether they—and inoculation theory more broadly—can withstand the test of time. That is, do the effects wear off and, if so, how quickly? Previous research has shown that the “immune-boosting” effects of traditional inoculation treatments typically decay over a few weeks (Zerback, Töpfl, and Knöpfle 2021; Banas and Rains 2010), with some studies suggesting that these effects might last up to six weeks (Pfau et al. 2006). However, gamified interventions such as Bad News represent unique inoculation treatments, as they last about 15 minutes and employ active inoculation—which aims to strengthen linkages between nodes in associative memory networks, a process thought to strengthen resistance to persuasion over time (Pfau et al. 2005). Promisingly, longitudinal work showed that although a significant decay effect occurred over a two-month period, the inoculation effect can remain stable for at least three months if participants receive regular “booster shots” in the form of repeated testing (Maertens et al. 2020). Figure 1 shows the “boosting” effect of the Bad News intervention over a period of 13 weeks.
Perceived Reliability of Misinformation at T1, T2, T3, T4, and T5 for the Inoculation and Control Groups
Into the wild: Policy applications of inoculation theory
In collaboration with the UK government, the Bad News game has been translated into more than twenty languages, allowing for cross-cultural replication of the inoculation effect. Replicating the results of previous studies using the intervention, Roozenbeek, van der Linden, and Nygren (2020) found that although some cultural heterogeneity was observed, the main inoculation effects of the game replicated cross-culturally. In addition, to evaluate the generalizability of inoculating people against overarching misinformation techniques and to tailor this approach to specific use cases, gamified inoculation has been applied in different domains in which misinformation is common. For example, Go Viral! 2 was launched in collaboration with the UK Cabinet Office to tackle misinformation about COVID-19. The game was part of the WHO’s “Stop the Spread” campaign and the United Nation’s Verified campaign and unveils popular techniques used to spread misinformation about COVID-19, such as 5G conspiracy theories and fake experts claiming that the vaccine contains dangerous chemicals. A large field study and randomized controlled trial (RCT) into its efficacy showed that the Go Viral! game’s five-minute gameplay is enough to increase the perceived manipulativeness of COVID-19 misinformation, improve people’s confidence in their (correct) misinformation judgements, and reduce intent to share misinformation on social media, in the English, German, and French versions of the game (Basol et al. 2021). The first two effects remained significant for at least a week post-gameplay.
To help inoculate people against political and electoral disinformation, the game Harmony Square 3 was launched in the United States in collaboration with the Department of Homeland Security. A study into its efficacy showed that the game reduced how reliable people found real political misinformation on social media, improved confidence in their ability to spot it, and reduced self-reported willingness to share such misinformation with others (Roozenbeek and van der Linden 2020).
Climate change is another domain in which gamified inoculation has been applied. Cranky Uncle (Cook 2020) 4 was designed to inoculate people against eleven techniques of science denial in the context of climate change. Preliminary data showed a significant improvement in critical thinking performance post-gameplay on even non-climate-related questions, illustrating the potential of technique-based inoculation (Cook 2019). Figure 2 shows a screenshot of the Bad News, Go Viral!, Harmony Square, and Cranky Uncle games.
Bad News, Harmony Square, Go Viral!, and Cranky Uncle Game Environments
Open Questions and Future Directions
Of course, not everyone can or wants to play a game, and so future research should address what motivates people to opt-in to such interventions. To scale the potential of inoculation theory further, Roozenbeek et al. (under review) designed short, two-minute animated videos in collaboration with Google Jigsaw (Jigsaw 2021). These videos 5 contain forewarnings and microdoses of misleading argumentation techniques (such as fearmongering) that could be run as ads on YouTube and other platforms to prebunk exposure to potentially harmful content. This would avoid the need for users to opt in and help to scale the approach across millions of users online. Although these real-world applications of psychological inoculation are promising, several questions remain unanswered. We outline two directions that have received scant attention in the context of inoculation scholarship: social proof and source credibility.
Social proof
Persuasive messages are often received in the context of other people (Tormala et al. 2009). Social networking sites represent new ecosystems of social information, allowing users to indicate agreement with or support for content through “likes,” expressing opinions through comments, or sharing information with other users (Hilverda, Kuttschreuter, and Giebels 2018). The number of likes or the valence of comments on a given post represent a form of “social proof,” indicating an implicit consensus that interest in and support for the post is high. News consumers may rely on social heuristics when making judgements about online news (Metzger, Flanagin, and Medders 2010). To assess whether game-based inoculation interventions remain effective in the presence of such potential moderators, in a recent (unpublished) study, we examined the effect of social proof on the efficacy of the Bad News game. Participants were assigned to one of three conditions in which the misinformation items that participants were exposed to had either a high number of likes, retweets, and comments (high consensus); a low number of likes, retweets, and comments (low consensus); or no social cues (control). Participants first rated the reliability of a set of misinformation headlines, followed by gameplay of the Bad News game, and finally rated the headlines again. Although the game successfully reduced reliability judgements of misinformation headlines regardless of whether these were administered alone or with social proof, the lowest reduction was seen when social proof showed high approval of the headline (i.e., the high-consensus condition). However, a potential confound may exist insofar as the (no social information) control group performed worse at pretest, which could have inflated their performance relative to the social proof conditions on the post-test. As such, these findings point to the need for further research on how social cues may moderate the efficacy of inoculation interventions.
Source effects
Although decades of research have pointed to the important role of source credibility in persuasion (Pornpitakpan 2004), we know little about the role of sources in moderating the effectiveness of inoculation messages (Compton et al. 2021). With respect to misinformation, recent evidence suggests that individuals may be susceptible to even nonpolitical misinformation when the associated source is politically congruent, an effect mediated by perceived credibility of the news source (Traberg and van der Linden 2022). As the aim of inoculation is to protect individuals against misinformation from any news source (regardless of the political affiliation of either the receiver or source), it is necessary to investigate whether inoculation protects individuals from the tactics used to spread misinformation, even when these tactics are used by sources that people know and trust. In a novel addition to the premeasures and postmeasures in the Bad News game, the current authors assessed just this: whether the Bad News game could reduce misinformation susceptibility even when the associated source of the test items was a real news source, and whether the effect of inoculation was impacted by the political slant of the source. Results from our pilot study showed that despite source effects being present on the pretest (i.e., participants in conditions where the source was politically congruent were significantly worse at identifying misinformation), a significant effect of inoculation occurred regardless of condition, such that participants lowered their reliability judgments of misinformation postintervention. This finding provides initial support that inoculation treatments may be successful, even on platforms where source effects are present.
Conclusion
We have explored the history of inoculation theory and prebunking, or preemptively building psychological resistance against future exposure to misinformation by providing forewarnings along with preemptive refutations. We have also reviewed how inoculation theory has been applied in the context of threats to science as a way to reduce susceptibility, not only to individual examples of misinformation, but also the techniques and strategies commonly used in its production. To do so, we have focused on scalable and game-based interventions, along with the studies that have been conducted to examine their efficacy. Our review of extant and ongoing research shows that inoculation theory is a promising framework for countering misinformation and holds the potential for large-scale implementation and long-term immunization.
Many important directions for future research remain, including mapping out decay functions, investigating externalities on other attitudes (e.g., toward factual news), ethical questions about who delivers the inoculation (see Compton et al. 2021), and questions about the influence of sources and countercampaigns. Sparked by recent advances, this review highlights the clear potential of inoculation theory as a tool to counter misinformation: by globally providing news consumers with an opportunity to “vaccinate” themselves against misinformation tactics through psychological game-based training, this method represents a scalable and durable response to the post-truth era. As governments and social media companies like Twitter have started to implement prebunking in practice (Ingram 2020), a thorough investigation of the factors that can increase (or potentially limit) the effects of inoculation interventions on social media represents a crucial next step for research.
Footnotes
Notes
Cecilie S. Traberg is a PhD candidate in psychology at the University of Cambridge, where she conducts research on social influence and misinformation susceptibility.
Jon Roozenbeek is a British Academy Postdoctoral Fellow in the Department of Psychology at the University of Cambridge. His research focuses broadly on misinformation, vaccine hesitancy, and extremism. He has a particular interest in inoculation theory and the development and testing of antimisinformation interventions.
Sander van der Linden is a professor of social psychology in society and director of the Cambridge Social Decision-Making Lab in the Department of Psychology at the University of Cambridge. He has published over 100 papers in the areas of judgment, persuasion, and decision-making, especially in relation to (countering) misinformation.