Misinformation and Literacies in the Era of Generative Artificial Intelligence: A Brief Overview and a Call for Future Research

Characterizing misinformation and related research

To avoid the blur of definitions and conceptualizations of misinformation-related terms, we in this essay primarily adopt the term misinformation to cover a wide array of information that is not accurate or false. Disinformation can be considered a specific type of misinformation that is intentionally false (Scheufele & Krause, 2019), and fake news is also deemed one type of misinformation presented in the form of news (Ha et al., 2021). Scholars may base their definitions of misinformation on different research contexts. For example, political science researchers define misinformation as false or misleading information that is believed to be true and related to one's political preferences (Nyhan & Reifler, 2010), or “incorrect, but confidently held, political beliefs” (Jerit & Zhao, 2020, p. 79). It is concerning that misinformed individuals may take political actions upon political misinformation (Jerit & Zhao, 2020). In the science and health domain, misinformation is defined as “information that is contrary to the epistemic consensus of the scientific community regarding a phenomenon” (Swire-Thompson & Lazer, 2019, p. 434). Studies on health misinformation mainly focus on misinformation related to vaccines and infectious diseases (Wang et al., 2019).

The orientations and topics of misinformation research can differ across regions. The authors conducted a brief literature review of articles related to “fake news” published in prestigious communication journals in mainland China, Taiwan, and the United States. The purpose was to explore differences between articles published in different regions in their research orientations. Considering the 2016 U.S. presidential election constitutes a catalyst for the spike of research on misinformation (Li, 2020), the time frame of this research was set between 2017 and 2021. The preliminary analyses (see Figure 1) showed that a large proportion of articles published in mainland China and Taiwan were (often non-empirical) review articles, case analyses, and essays, and they put great emphasis on the institutional-level factors centering around fake news and legislation against fake news. Comparatively, most articles published in the United States explored the influences of fake news on individuals’ attitudes, behaviors, and social interactions particularly.

OPEN IN VIEWER

Figure 1.

Research orientations in three regions between 2017 and 2021. Note. A-0 represents (often non-empirical) review articles; A-1, A-2, and A-3 are articles exploring conceptualization, the psychological mechanism underlying fake news consumption, and influences of fake news on individuals’ attitudes and behaviors, respectively; A-4 focuses on news fact-checking; A-5 predominantly utilizes cases analyses and elaborates on trending fake news cases related to social issues.

Misinformation, media bias, and psychological determinants

Research on the production of misinformation can be traced back to the investigation of media bias, which is anchored in the perspective of structural influences. As the intensity of mediated experiences, institutions, events, and processes beyond one's reach have increased, media has become a vital channel for information exchange and communication and breeds mediated politics (Strömbäck, 2008). This mediated communication often goes hand in hand with bias in media coverage, characterized by the lack of accuracy and realism, the blur between fact and opinion, and slant (Hackett, 1984; McQuail, 1996). In political communication, media bias is reflected in partisanship and biased reporting that favors one political actor/position/stance over another. More specifically, the bias in political media coverage may manifest itself in the visibility of political actors/parties (i.e. visibility bias), evaluation of political actors/parties (i.e. tonality bias), and the association between political actors/parties and certain issues (i.e. agenda bias) (Eberl et al., 2017). Nevertheless, it's noteworthy that misinformation in other social domains (e.g. health-related misinformation) is found to be created mostly by individuals with no official or institutional affiliations (Wang et al., 2019), which shifts a wide bunch of researchers’ attention from the production of misinformation to individuals’ reception and formation of misinformation at a micro-level.

Misinformation research at a micro-level emphasizes people's false and unsubstantiated belief formation process. According to Nyhan and Reifler (2010), misinformation refers to a situation where “people's beliefs about factual matters are not supported by clear evidence and expert opinion” (p. 305). This definition is open to discussion on the formation of such beliefs from psychological perspectives. Accordingly, a general pathway is that individuals who are exposed to misinformation will evaluate or assess the information first, during which rational thinking and heuristics may intertwine, and emotions enter into the decision process at the same time; consequently, attitudinal changes and decision bias will occur, and the behavioral decision will be made (see Figure 2) (Chu-Ke, 2022). Amazeen (2023) in a recent paper proposed a misinformation recognition and response model and treated the misinformation reception process in much detail. For instance, problem identification and issue motivation are important prerequisites of the activation of misinformation recognition, after which individuals may adopt different cognitive coping strategies and display ensuing responses.

OPEN IN VIEWER

Figure 2.

Misinformation and individual information processing.

Psychological decision theories on the role of heuristics may shed light on the individuals’ reception and formation of misinformation. As behavioral decision theory suggests, humans use heuristics to simplify their decision-making under uncertainty. Three heuristics that are often used to assess probabilities and values of uncertain information include representativeness heuristics (i.e. assessing the probability by considering the object or event to what extent is representative of existing knowledge or stereotypes), availability heuristics (i.e. assessing the probability by considering the instances or occurrences of an object or event can be brought to mind), and adjustment and anchoring heuristics (i.e. assessing the probabilities biased toward the initial values that are either provided by others or produced by insufficient computation) (Slovic et al., 1977; Tversky & Kahneman, 1974). Despite being very economical and beneficial sometimes (Bílek et al., 2018), these judgmental heuristics can lead to serious cognitive biases and facilitate the reception of political and health misinformation among citizens (Dancey & Sheagley, 2013; Hwang & Jeong, 2023).

Emotions may produce bias when individuals identify misinformation and motivate individuals to spread misinformation. According to a review of research on emotion and decision-making by Lerner et al. (2015), emotions, including incidental and integral ones, can “powerfully, predictably, and pervasively influence decision-making” (p. 802) such that they shape the content of thought, depth of thought, goal activation, etc. Anger, specifically, is associated with approach motivation (Harmon-Jones & Sigelman, 2001) and, together with counterarguments, goes against typically beneficial advocacies and drives unwise decisions (Brehm, 1966; Rosenberg & Siegel, 2018). A systematic review explored the association between emotions and misinformation, and it concluded that moderate levels of stress and short-term distress can decrease susceptibility to misinformation, while anger may increase susceptibility (Sharma et al., 2023).

Misinformation and vulnerable groups

Empirical research has suggested that senior citizens are susceptible to misinformation. Loos and Nijenhuis (2020) in their study on political fake news articles in the form of advertisements on Facebook revealed that older adults, compared with younger adults, are more likely to be exposed to political fake news articles. Yousuf et al. (2021) also found that senior citizens in the Netherlands generally believed misinformation related to COVID-19 vaccines. Some reasons behind the positive relationship between age and susceptibility to misinformation have been summarized by Brashier and Schacter (2020): they pointed out that senior citizens may have greater misplaced trust, more difficulty in detecting misinformation, and less emphasis on information accuracy when consuming information, which facilitates the reception of misinformation among senior citizens. With a cross-national survey, Jo et al. (2022) found that senior citizens are less likely to check on suspicious content. These may also be related to the digital literacy of senior citizens. Some of the senior citizens may be newcomers to the Internet and have insufficient ability to detect sponsored or manipulated content. Besides, the reception of misinformation among senior citizens can be attributed to both media exposure and social influence. For instance, family members and friends who are exposed to misinformation can actively relay it to senior citizens (Chia et al., 2023).

In addition to senior citizens, it is important to recognize that adolescents are also at risk of falling victim to misinformation. Baumel et al. (2021) suggest that adolescents may be frequently exposed to misinformation and “at a substantial risk of misinformation” because of their deep engagement in social media (p. 1021). Individuals with certain political stances can also be vulnerable to misinformation. For instance, conservative political orientation was found to be associated with higher acceptance of COVID-19 misinformation compared with liberal political orientation (Xiao, 2022). When communicating information about stigmatized health topics, those stigma-aware individuals are found to display information avoidance, which encourages the belief in misinformation (Dong et al., 2024). Besides, misinformation acceptance may depend on various information-processing styles and states. Those who experienced message fatigue and information overload were found to have a greater acceptance of COVID-19 misinformation via increased information avoidance (Hwang et al., 2022; Tandoc & Kim, 2022). For those who engaged in heuristic processing, the effect of COVID-19 misinformation exposure on COVID-19 misinformation acceptance was greater (Hwang & Jeong, 2023).

A case analysis of misinformation: pesticides into the air via aerial spraying

The emission of pesticides into the air via aerial spraying used to play an important role in preventing the spread of diseases and insect attacks in many parts of the world. In May 2016, a piece of misinformation about “the emission of pesticides into the air via aerial spraying for the control of fall webworms” started to circulate in group chats on WeChat, the most popular social media in China. The misinformation informed the citizens that the local Forestry and Park Bureau planned to use aircraft to spray pesticides to prevent and control the fall webworms and suggested citizens avoid outdoor activities during the spraying period. The misinformation spread from the east of the Chinese mainland to the whole Chinese mainland and Taiwan area at a very fast speed, which drew the attention of mainstream media and local government. The media coverage followed very soon after the virality of misinformation and confirmed that the local government would not release pesticides into the air via aerial spraying. Every year social media in China witnesses the viral spread of similar misinformation on “the emission of pesticides into the air via aerial spraying” and the correction to it as follows.

First, it's noteworthy that this misinformation heavily relies on authority heuristics. In the misinformation, either the Forestry and Park Bureau or the Department of Agriculture as the unfounded information source attracted public attention and significantly increased the information's credibility via heuristics. Second, as time went by, the content of misinformation evolved and was inextricably interwoven with trending social events. In 2019, the misinformation on “the emission of pesticides into the air via aerial spraying for the control of fall webworms” included more details than in 2016, such as associating the outbreaks of fall webworms with swine fever. Since 2020, the misinformation combined pesticide spraying rumors with COVID-19 information.

Senior citizens are found to play an important role in spreading and accepting this misinformation. In eight counties and cities in Taiwan area where the misinformation about “the emission of pesticides into the air via aerial spraying” gained the greatest popularity, senior citizens occupy over 14% of the local population. As pesticide misinformation is tightly associated with peoples’ health and environment, it may draw much attention from senior citizens and raise their rumor anxiety, which consequently leads to a greater likelihood of spreading misinformation (He et al., 2019).

Generative AI and misinformation

The debate about AI flooding the public sphere can be traced back to social bots and deepfakes. Social bots, “a computer algorithm that automatically produces content and interacts with humans on social media, trying to emulate and possibly alter their behavior” (Ferrara et al., 2016, p. 96), have been criticized for posing great risks to the digital environment as well as our society. They amplify interactions with low-credibility content and help circulate it to influential users so that low-credibility content can go viral (Shao et al., 2018; Shu et al., 2020). The social networks consisting of bots help diffuse misinformation and fake news (Miró-Llinares and Aguerri, 2023). Deepfake is another debatable generative AI technique that has been exploited for creating fake media. In a study on the discussion about deepfakes in Reddit communities, Gamage et al. (2022) revealed that niche social media platforms such as Reddit provide an incubator for deepfake generators/creators, and especially for expert users to support the creation of customized pornography.

Generative AI is defined as “AI based on advanced neural networks with the ability to produce highly realistic synthetic text, images, video and audio (including fictional stories, poems, and programming code)” (Galaz et al., 2023, p. 5). They are characterized by increased accessibility (i.e. being easily available), sophistication (i.e. the capacity to create longer pieces of synthetic text, video, and voice with specific prompts), and capabilities for persuasion (i.e. the capacity to elaborate the content persuasively). With these characteristics, generative AI can now help create and spread false, seemingly credible information quickly and effortlessly (Zhou et al., 2023). For instance, ChatGPT, a chatbot developed by OpenAI and launched on 30 November 2022, has been found to produce convincing text that comprises conspiracy theories and misleading narratives (Hsu & Thompson, 2023) and manufacture fake news articles that claim trusted news organizations and journalists as the source (Moran, 2023). Meanwhile, due to the easy access with minimal programmatic knowledge, human users who deploy generative AI originally as assistive technologies can unintentionally become creators and spreaders of misinformation.

Given the capabilities of generative AI and its potential impact on misinformation, it's necessary to consider the following three dimensions for the development of ethical AI and ethical uses of AI in counteracting misinformation, the first two of which concern AI practitioners and legislation, and the rest at the user end will be discussed with a focus on its implications for the literacy program and communication studies.

Making ethical AI: regulations From inside and outside

Ethical AI for combating misinformation necessitates efforts from AI practitioners, especially those who design and develop models, platforms, and APIs. First, transparent AI development practices should be encouraged. It is critical that AI practitioners, platforms, and organizations come together to create a culture that nurtures values of collaboration and openness. Specifically, AI algorithms, data, and methodologies should be openly accessible while maintaining ethical bounds to promote transparency and collective understanding. Moreover, it's important to deploy interpretable models and explainable user interfaces. With enhanced interpretability and explainability of generative AI, users can better understand how generative AI arrives at specific decisions and predictions and detect potential falsehoods. Second, there's a need for user guidance and monitoring of misuse (Zhou et al., 2023). Despite that the current ChatGPT notifies the users that “ChatGPT can make mistakes. Consider checking important information” at the bottom of its interface, such a warning cue may not be salient enough to raise users’ awareness of the potentially flawed information. Cues appearing right after copy-and-paste action may also assist in reminding the users to do fact-checking if necessary. To help users verify information, it can provide useful links to fact-checking websites, accredited news organizations, and relevant governmental agencies.

Ethical AI for combating misinformation also entails the intervention of laws. On December 8, 2023, European Parliament and Council negotiators successfully reached a provisional agreement on the Artificial Intelligence Act (i.e. the “AI Act”). This is the inaugural comprehensive law on AI enacted by a major regulatory body, aiming to “ensure that fundamental rights, democracy, the rule of law and environmental sustainability are protected from high risk AI, while boosting innovation and making Europe a leader in the field” (Artificial Intelligence Act, 2023). While the AI Act does not directly target misinformation, its provisions regarding prohibited AI practices and requirements for high-risk AI can contribute to the development and deployment of more reliable AI systems for combatting misinformation. For instance, in line with the interpretability and explainability, we previously mentioned, the AI act emphasizes the rights of individuals who may be subject to decisions produced by high-risk AI or adversely impacting their health, safety, etc., to request from the AI deployer clear and meaningful explanations about the role of the AI system plays in the decision-making process. Some general-purpose models that are often deployed to produce misinformation and disinformation, such as generative models that create images, code, and video, have also been regulated accordingly (Gibney, 2024). Enforcement mechanisms and penalties for non-compliance to provisions regarding data governance, transparency obligations, monitoring, and surveillance are established.

Promoting ethical use of AI: constructing AI literacies

Faced with the changing media landscape and pervasive misinformation, a variety of literacy has been conceptualized and investigated. For instance, Khan and Idris (2019) contribute to the conceptualization of information literacy by specifying information-seeking, sharing, and verification skills. Specifically, information verification skills refer to the capacity to verify online information using an online tool and to use a search engine. Carmi et al. (2020) argued about the significance of data literacies in an increasingly datafied society. They emphasized what data literacies encompass in addition to the previous literacies, e.g. networked literacies, critical thinking about the online ecosystem, and literacies assisting in active citizenship, to help citizens identify and combat the misinformation crisis. In light of the development of generative AI, Long and Magerko (2020) further devised a conceptual framework for AI literacy composed of five overarching themes: What is AI?; What can AI do?; How does AI work?; How should AI be used?; and How do people perceive AI?. They also outlined competencies in each theme, which includes the aforementioned data literacy as a competency needed for the understanding of AI's working mechanism.

Here, we’d like to engage in a dialogue with the conceptualization of literacy by Livingstone (2004) and explore a few points that may offer directions for future literacy education and research regarding AI-fueled misinformation. According to Livingstone (2004), literacy is rooted in the historically and culturally conditioned relationship among three processes:

The symbolic and material representation of knowledge, values, and cultures. AI literacy can be considered the ability to understand and critically evaluate knowledge, values, and cultures in which AI systems operate and their impacts on AI-generated content. Specifically, AI literacy requires an understanding of specific cultural, social, and political contexts in which AI systems operate, and the potentially prejudiced decisions based on demographic features (e.g. race, sex, etc.) (e.g. Ntoutsi et al., 2020). This can help individuals identify how AI-fueled misinformation may exploit or perpetuate societal biases and narratives. This perspective in education programs can incorporate this understanding into curricula to encourage critical reflection on the societal implications of AI. Social science research may delve into AI as a cultural artifact in different tasks and its socio-psychological implications for misinformation detection and correction.

The dissemination of interpretative skills and abilities among the population. As Livingstone (2004) emphasized, literacy should be conceived as “a coproduction of the interactive engagement between technology and user” (p. 11). The skilled and strategic interpretation of AI-generated content in countering misinformation should include hands-on experience with AI tools and platforms and exploration of the underlying principles and mechanisms of AI systems. During the information-seeking phase, users can utilize source verification prompts, fact-checking suggestions, and debunking resources. When using AI-generated content, individuals should realize that this content is not inherently original. When available, individuals should acknowledge the source of the content. The education program should target vulnerable groups (e.g. senior citizens and adolescents), using accessible, easy-to-understand, and tailored materials. Research should be dedicated to the effectiveness of cues and prompt suggestions in encouraging proper use of AI-generated content and verification.

The institutional use and management of the power that grants skills and knowledge to people. Literacy is often institutionalized through educational systems, libraries, and other organizations that disseminate knowledge and skills. Similarly, AI literacy can be institutionalized through the cooperation between aforementioned AI practitioners, deployers, and legislation that promotes responsible use of AI technologies. From the users’ end, users should develop an awareness of and skills in the feedback mechanisms of the AI systems and report potentially misleading or false information encountered through AI use. This feedback can help improve the system's accuracy and effectiveness in countering misinformation over time and empower users to facilitate institutional management of the AI power.