Reminders of Everyday Misinformation Statements Can Enhance Memory for and Beliefs in Corrections of Those Statements in the Short Term

Abstract

Fake-news exposure can cause misinformation to be mistakenly remembered and believed. In two experiments (Ns = 96), we examined whether reminders of misinformation could improve memory for and beliefs in corrections. Subjects read factual statements and misinformation statements taken from news websites and then read statements that corrected the misinformation. Misinformation reminders appeared before some corrections but not others. Subjects then attempted to recall facts, indicated their belief in those recalls, and indicated whether they remembered corrections and misinformation. In Experiment 1, we did not constrain subjects’ report criteria. But in Experiment 2, we encouraged conservative reporting by instructing subjects to report only information they believed to be true. Reminders increased recall and belief accuracy. These benefits were greater both when misinformation was recollected and when subjects remembered that corrections had occurred. These findings demonstrate one situation in which misinformation reminders can diminish the negative effects of fake-news exposure in the short term.

Keywords

beliefs corrections continued-influence effect fake news source memory open data open materials preregistered

The prevalence of “fake news” since the 2016 U.S. presidential election has increased awareness of the pernicious effects of fabricated news content on memory and beliefs. Serious concerns about fake news are warranted because societies are vulnerable to the pervasive consequences of misinformation exposure (Lewandowsky, Ecker, Seifert, Schwarz, & Cook, 2012). For example, vaccine myths have motivated some parents to avoid immunizing their children from preventable diseases, which has led to dramatic health and economic consequences (Larson, Cooper, Eskola, Katz, & Ratzan, 2011; Poland & Spier, 2010). Given such apparent dangers of fake-news exposure, it is important to identify ways to mitigate its effects (Lewandowsky et al., 2012; Schwarz, Newman, & Leach, 2016). Here, we examined whether reminding people of fake-news misinformation before correcting it can enhance memory for and belief in verified facts.

The Continued-Influence Effect

A challenge in using corrections effectively is that repeating misinformation can have negative consequences. Research on the continued-influence effect has shown that information presented as factual that is later deemed false can contaminate memory and reasoning. In the original demonstrations (H. M. Johnson & Seifert, 1998; Wilkes & Leatherbarrow, 1988), participants read fictive narratives containing details of unfolding events. The primary manipulation was whether participants were later told that an event detail was false. Correction effects were measured as the number of references to the original false detail on a test of event understanding. This research has generally shown that corrections can reduce reliance on misinformation, but the continued-influence effect often remains (Lewandowsky et al., 2012).

The persistence of the continued-influence effect has led researchers to generally recommend avoiding repeating misinformation (Lewandowsky et al., 2012). The theoretical rationale is that repetition increases the familiarity and believability of misinformation (Dechêne, Stahl, Hansen, & Wänke, 2010; Schwarz, Sanna, Skurnik, & Yoon, 2007; Weaver, Garcia, Schwarz, & Miller, 2007). Support for this comes from the illusory-truth effect, in which prior exposure increases perceived accuracy (Bacon, 1979; Begg, Anas, & Farinacci, 1992; Hasher, Goldstein, & Toppino, 1977). This effect generalizes across information types, including fake-news headlines (Pennycook, Cannon, & Rand, 2018).

To explain how repetitions increase beliefs in misinformation, some researchers have invoked a dual-process perspective (Jacoby, 1991, 1999). Studies of the familiarity-backfire effect have shown that increasing misinformation familiarity through extra exposure leads to misattributions of fluency when context cannot be recollected (e.g., Skurnik, Yoon, Park, & Schwarz, 2005). But familiarity backfire is not always observed. For example, Swire, Ecker, and Lewandowsky (2017) examined the role of familiarity in myth correction. Subjects rated beliefs in facts and myths of unclear veracity. Then, the facts were affirmed and the myths corrected. Subjects made belief ratings again after various retention intervals. Affirmations increased fact beliefs more consistently than corrections decreased myth beliefs, especially at longer intervals that undermined recollection (see also Peter & Koch, 2016). Although these results suggest a role for familiarity, myth beliefs remained below premanipulation levels.

The inconsistency in familiarity backfire suggests that it is premature to recommend against misinformation reminders (cf. Ecker, Hogan, & Lewandowsky, 2017). A related perspective is that coactivating conflicting information may contribute to memory updating. Support from paired-associates learning was shown when pairs with shared cues and changed responses facilitated memory for recent responses (Jacoby, Wahlheim, & Kelley, 2015; Wahlheim & Jacoby, 2013). Similar findings have been obtained with naturalistic materials, such as political positions (Putnam, Wahlheim, & Jacoby, 2014), everyday actions (Wahlheim & Zacks, 2019), and educational information (Stadtler, Scharrer, Brummernhenrich, & Bromme, 2013). The argument for repeating outdated information is that conflict detection enables event-model updating (also see Kendeou, Walsh, Smith, & O’Brien, 2014; Morrow, Bower, & Greenspan, 1989) and that misinformation corrections increase conflict salience.

Ecker et al. (2017) tested predictions from familiarity-backfire and conflict-salience accounts by examining how reminders affected the continued-influence effect. Subjects first read fictional news events with misinformation details that could be targets of corrections in subsequent articles. Some corrections did not reference misinformation, whereas others included misinformation reminders. Reminders reduced misinformation reliance on memory and reasoning measures, consistent with the conflict-salience view (Kendeou et al., 2014; Stadtler et al., 2013; Wahlheim & Jacoby, 2013). These results are also consistent with a retrieval account of the continued-influence effect—that is, that correction-enhanced conflict salience improves later recollection of corrections (Ecker, Lewandowsky, & Tang, 2010; Seifert, 2002; Swire et al., 2017). Further, such recollection may support source monitoring (M. K. Johnson, Hashtroudi, & Lindsay, 1993).

A related dual-process model can also illuminate reminder effects on the continued-influence effect. The memory-for-change framework (Jacoby et al., 2015; Wahlheim & Jacoby, 2013) unites the above mechanisms by assuming roles for conflict detection, reminder-enhanced familiarity, and conflict recollection (which supports source monitoring). In this framework, when new events trigger remindings of related events, both the events and their relationship are encoded together. Recollection-based retrieval then enhances memory for recent events by accessing representations that include temporal information. In contrast, recollection failure increases reliance on the familiarity of outdated information, leading to source-monitoring failure. We showed in paired-associates learning that reminders further enhance the memory updating associated with recollecting outdated information (Wahlheim, Smith, & Delaney, 2019). This inspired us to take a similar approach to illuminating the effects of reminders on fake-news correction.

The Current Study

We extended Ecker et al.’s (2017) work by examining the effect of reminders with fake news and fact-check-verified corrections in a format similar to reading news headlines. The paradigm comprised two phases including statements of current events. Phase 1 included statements of unclear veracity, some of which were facts and others misinformation. For example, for some subjects, Phase 1 included the true statement “President Clinton took fewer vacation days than any other recent president,” whereas for other subjects it included the false statement “President Obama took fewer vacation days than any other recent president.” Phase 2 included affirmations of facts and corrections of misinformation. Half of the corrections followed reminders of Phase 1 misinformation, whereas the other half did not. Phase 3 measured memory and beliefs for corrections and misinformation using an approach similar to memory-for-change studies (described below). All stimuli, data, and analysis scripts are available on OSF at https://osf.io/89zmj/. The present research was approved by the institutional review board of the University of North Carolina at Greensboro (UNCG).

Experiment 1

We varied statement types by including facts that repeated between phases, facts that appeared in Phase 2, corrections with misinformation reminders, and corrections without reminders. We assessed fact recall, beliefs, and memory for corrections and misinformation. Prior studies (Ecker et al., 2017; Wahlheim et al., 2019) led to the prediction that reminders would enhance memory and belief accuracy by increasing conflict salience and memory for corrections and misinformation.

Method

Subjects

Our stopping rule was to collect usable data from 96 subjects. We chose this sample size in order to test as many people as our resources would allow in approximately one semester. We also chose this number because it allowed us to administer each of the four experimental formats equally often across subjects. To meet this goal, we tested 98 UNCG undergraduates. Data from one subject were excluded because of computer malfunction, and data from another subject were excluded because of the subject’s failure to follow instructions. Our final sample contained 96 subjects (79 women and 17 men; age: range = 18–40 years, M = 19.58, SD = 3.75). We conducted a sensitivity analysis using G*Power (Version 3.1.9.2; Faul, Erdfelder, Buchner, & Lang, 2009). We were primarily interested in whether corrections with reminders would lead to better fact recall and fewer misinformation intrusions than corrections without reminders. We were also interested in whether memory for corrections would be associated with better fact recall and fewer intrusions. These effects could be assessed with pairwise comparisons for each dependent measure. An analysis for a matched-pairs t test with 80% power and an α of .05 (two-tailed) for 96 subjects indicated that we could detect a small effect size (d_z = 0.29).

Design and materials

The experiment included the within-subjects variable statement type (repeated vs. control vs. correction vs. correction + reminder). Drawing from various Internet news sources, we selected 60 statement pairs that each included misinformation and its correction. The inclusion criteria for misinformation was that those statements were portrayed by their source as being true. Most pairs corresponded to a unique topic. All topics were related to events that U.S. citizens could have been exposed to via Internet news sources.

For counterbalancing, we randomly assigned the 60 statement pairs into four groups of 15 and rotated groups through conditions, resulting in four experimental formats. Groups appeared equally often in each condition across subjects. Phase 1 included 45 statements (15 facts and 30 misinformation). Phase 2 included 60 facts—15 repeated facts (repetition condition), 15 new facts (control condition), 15 facts that corrected misinformation (correction condition), and 15 facts that corrected misinformation and were each preceded by a misinformation reminder (correction + reminder condition). We included repetition statements to encourage subjects to use more diagnostic information than statement accessibility as the primary basis for correction classifications in Phase 3. We also included control statements to compare correction effects with a no-correction control condition. Phase 3 included a cued-recall test with 60 questions pertaining to the Phase 2 statements. For the example topic about presidential vacation days, described at the end of the introduction, the question “Which recent president took the fewest vacation days?” could be completed with the misinformation (“President Obama”) or the correct information (“President Clinton”). Table 1 displays example statements and test questions from each condition.

Table 1.

Example Statements from Each Phase of Experiments 1 and 2

Statement type	Phase 1	Phase 2		Phase 3
Statement type	Initial statement	Misinformation reminder	Verified fact	Test cue
Repetition	The majority of American taxes are spent on social programs such as Medicare.	None.	The majority of American taxes are spent on social programs such as Medicare.	What type of program does the majority of tax-payers’ money go towards?
Control	None.	None.	The Trump administration is cutting the CDC budget by about 20 percent.	About what percentage of the CDC budget is being cut by the Trump administration?
Correction	President Obama took fewer vacation days than any other recent president.	None.	President Clinton took fewer vacation days than any other recent president.	Which recent president took the fewest vacation days?
Correction + reminder	California representatives have asked for the building of a border wall.	California representatives have asked for the building of a border wall.	California representatives have sued to stop the building of a border wall.	How did Californians react to the idea of a border wall?

Note: Italics have been added to the examples in this table to indicate the relevant details in the conditions with corrections. The complete set of stimulus materials is available on OSF (https://osf.io/89zmj/). CDC = Centers for Disease Control and Prevention.

Procedure

Subjects were tested individually with an experimenter present. Stimuli were presented electronically using E-Prime software (Version 2; Schneider, Eschman, & Zuccolotto, 2012). The experiment comprised three phases that subjects completed in one session.

Prior to Phase 1, subjects were told that they would read both true and false statements and that they should silently consider whether each statement was true. They were also told that corrections of false statements would appear later in the experiment. Phase 1 statements appeared twice for a total of 90 presentations. Each appeared once in the first cycle before repeating in the second cycle. Statements appeared individually in white font against a black background for 8 s each, followed by a blank screen for 0.25 s. Statements appeared in random order in each cycle.

Prior to Phase 2, subjects were told about the presentation formats for corrections and that they should note when corrections appeared. They were also told that some statements would be facts repeated from Phase 1, and others would be facts that were new to Phase 2. Each fact, including the correction statements, appeared in white font against a black background exactly as in Phase 1. However, the presentation format of the correction + reminder statements differed. For those statements, the message “The following statement was misinformation presented to you in Phase 1:” appeared first for 3 s in purple font. Next, a reminder of the misinformation statement appeared for 8 s, as in Phase 1. Following the reminder, the message “The following is a correction of that statement:” appeared for 4 s in green font. Finally, the factual statement that corrected the misinformation appeared for 8 s. Note that unlike the correction + reminder statements, corrections that were not preceded by a reminder (i.e., the statements assigned to the correction condition) were also not preceded by a message indicating that those statements were corrections. Each screen was followed by a blank screen for 0.25 s.

Prior to Phase 3, subjects were told that their main task would be to answer questions about the information that they had just studied in Phase 2. During Phase 3, questions appeared individually (e.g., “Which recent president took the fewest vacation days?”) above a text box until subjects typed their response. After subjects attempted to recall the correct detail, the prompt “Rate your belief that this statement is true” appeared on the screen while the question was still present. Using a scale from 1 (believe false) to 6 (believe true), subjects entered their response by pressing the corresponding key. After subjects entered a response, the screen cleared, and a prompt asked whether the Phase 2 statement corrected misinformation from Phase 1. Subjects responded by pressing either 1 (yes) or 2 (no). When subjects indicated “yes,” another prompt appeared asking them to type the Phase 1 misinformation. When subjects indicated “no,” the next question appeared. Subjects were encouraged to respond accurately when attempting to recall both correct information and misinformation. However, when they could not think of a response, they were told that they could pass when attempting to recall from either information source. Test items appeared in random order.

Results

We performed all statistical tests using R software (R Core Team, 2020). In the following analyses, we fitted logistic and linear mixed-effects models with functions from the lme4 package (Bates, Mächler, Bolker, & Walker, 2015). We included subjects and items as random-intercept effects and the predictor variables statement type and classification as fixed effects. We performed hypothesis tests using the Anova function from the car package (Fox & Weisberg, 2019) and post hoc comparisons using the Tukey method from the emmeans function in the emmeans package (Lenth, 2019). We report the results from those analyses below.

To compute standardized effect-size estimates, we conducted parallel analyses using simple linear regression models from the lm function in base R. Those analyses are reported in the Supplemental Material available online. The level for significance for all analyses (α) was set at .05.

Cued-recall scoring

Phase 3 cued-recall responses were classified into four types. For examples of the response types, consider an item on the topic of a border wall between California and Mexico. The misinformation for this topic is that “California representatives have asked for the building of a border wall.” The correction is that “California representatives have sued to stop the building of a border wall.” Note that italics have been added here to highlight the difference between statements. The test cue for this topic was the question “How did Californians react to the idea of a border wall?”

Fact-recall responses included the correct detail that representatives attempted to stop the border wall. Misinformation intrusions were responses that included the false detail that representatives requested the border wall. Ambiguous responses did not differentiate between facts and misinformation, such as a response indicating that Californians felt strongly emotional about the idea of a border wall. Other-error responses included both commission errors (without details from either the factual or misinformation statements) and omitted responses. Because we were primarily interested in correction effects on subjects’ misinformation reliance when attempting to recall facts, we focused subsequent analyses on fact recall and misinformation intrusions.

Two raters who were blind to experimental conditions independently coded responses after being trained on a set of responses from a pilot study. A Cohen’s κ of .90 for the initial ratings indicated acceptable agreement. Discrepancies between raters were resolved through discussion.

Cued-recall performance

Figure 1 (top left) displays fact recall for each statement type. A model including statement type as a factor indicated a significant effect, χ²(3) = 256.69, p < .001, showing that fact recall was significantly greater in the repetition condition and the correction + reminder condition than in the control and correction conditions, smallest z ratio = 10.35, p < .001. There were no significant differences between the repetition condition and the correction + reminder condition or between the control and correction conditions, largest z ratio = 1.66, p = .345. Figure 1 (top right) displays misinformation intrusions for the correction condition and the correction + reminder condition. A model with only those conditions indicated significantly more intrusions in the correction condition than in the correction + reminder condition, χ²(1) = 50.91, p < .001.

Fig. 1.

Results from Experiment 1: correction effects on the probability of correct recall (top row) and on belief ratings (bottom row). In each row, fact-recall results (left) are shown for each of the four statement types and misinformation-intrusion results (right) are shown for correction and correction + reminder statements. The bars display estimates from mixed-effect models, and the corresponding error bars show 95% confidence intervals. In the box-and-whisker plots, the diamonds indicate group means, the horizontal lines indicate medians, the heights of the boxes mark the interquartile ranges, and the whiskers extend 1.5 times the interquartile ranges. Dots represent individual subject probabilities.

Belief ratings

Figure 1 (bottom left) displays belief ratings for fact recall. A model including statement type as a factor indicated a significant effect, χ²(3) = 50.17, p < .001, showing significantly higher belief ratings in the correction + reminder condition than in all other conditions, smallest z ratio = 3.51, p = .003, and in the correction condition than in the control condition, z ratio = 3.00, p = .014. There were no other significant differences, largest z ratio = 2.27, p = .104. Figure 1 (bottom right) displays belief ratings for misinformation intrusions for the correction and corrections + reminders conditions. A model with only those conditions indicated significantly higher ratings in the correction condition than in the correction + reminder condition, χ²(1) = 6.15, p = .013.

Correction classifications

Figure 2 displays the probabilities of two types of correction classifications: correct classification of corrections with correct recall of misinformation (misinformation recollected; left) and correct classification of corrections without misinformation recall (correction remembered; right). A model including statement type as a factor indicated significantly greater misinformation recollection in the correction + reminder condition than in the correction condition, χ²(1) = 64.06, p < .001. In contrast, a similar model for remembered corrections indicated no significant difference between conditions, χ²(1) = 1.14, p = .285.

Fig. 2.

Results from Experiment 1: correction effects on the probability of misinformation recollection (left) and memory for corrections (right), separately for correction and correction + reminder statements. The bars display estimates from mixed-effect models, and the corresponding error bars show 95% confidence intervals. In the box-and-whisker plots, the diamonds indicate group means, the horizontal lines indicate medians, the heights of the boxes mark the interquartile ranges, and the whiskers extend 1.5 times the interquartile ranges. Dots represent individual subject probabilities.

Recall and beliefs conditionalized on correction classifications

In the next set of analyses, we tested the prediction from the memory-for-change framework that memory accuracy would be higher when outdated information (misinformation) could be recollected and when corrections could be remembered. We also explored whether memory for misinformation and corrections would be associated with more accurate beliefs. To do this, we examined fact recall, misinformation intrusions, and belief ratings in the correction condition and the correction + reminder condition conditionalized on the correction-classification types described above and instances when corrections were not classified as such (as when a correction was not remembered).

To examine how recall and beliefs were associated with correction classifications, we fitted separate 2 (statement type) × 3 (classification) models to each outcome measure, with one exception described below. We do not describe main effects of statement type that are redundant with previous analyses.

Cued recall

Figure 3 (top left) displays fact-recall probabilities. The model indicated significant effects of statement type, χ²(1) = 14.09, p < .001, and classification, χ²(2) = 283.73, p < .001, and no significant Statement Type × Classification interaction, χ²(2) = 0.53, p = .767, showing significantly higher fact recall when misinformation was recollected than when corrections were remembered, z ratio = 9.79, p < .001, and when corrections were remembered than when they were not, z ratio = 6.83, p < .001.

Fig. 3.

Results from Experiment 1: mixed-effect model estimates for the probability of correct recall (top row) and for belief ratings (bottom row). In each row, fact-recall results (left) and misinformation-intrusion results (right) are shown for correction and correction + reminder statements, both conditionalized and not conditionalized on correction classifications. Black points represent estimates for all observations. Colored points represent estimates conditionalized on misinformation recollection (green), memory for corrections (blue), and no memory for corrections (red). The size of each point indicates the relative proportion of observations in each conditional cell. Note that the point areas for overall belief ratings (black) vary because they reflect the number of fact-recall and misinformation-intrusion responses given in each statement-type condition (see black points in the top row for those response probabilities). Error bars represent 95% confidence intervals.

Figure 3 (top right) displays misinformation-intrusion probabilities. The model indicated significant effects of statement type, χ²(1) = 18.48, p < .001, and classification, χ²(2) = 113.94, p < .001, and no significant Statement Type × Classification interaction, χ²(2) = 0.96, p = .620, showing significantly higher intrusion rates when corrections were not remembered than when they were, z ratio = 3.41, p = .002, and when corrections were remembered than when misinformation was recollected, z ratio = 7.55, p < .001. Note that when intrusions and misinformation recollection co-occurred, subjects reported misinformation twice, which likely reflected guessing. We display these probabilities in Figure 3 for completeness to illustrate the composition of overall intrusion rates.

Belief ratings

Figure 3 (bottom left) displays fact-recall belief ratings. The model indicated significant effects of statement type, χ²(1) = 5.79, p = .016, and classification, χ²(2) = 34.97, p < .001, and no significant Statement Type × Classification interaction, χ²(2) = 3.68, p = .159. Belief ratings were significantly higher when misinformation was recollected than when corrections were remembered, t(1642) = 2.94, p = .010, and when they were not remembered, t(1688) = 5.18, p < .001, but ratings were not significantly different when corrections were and were not remembered, t(1678) = 2.04, p = .103.

Figure 3 (bottom right) displays misinformation-intrusion belief ratings. Given the ambiguity and rarity of intrusions when misinformation was also recollected, we tested for differences only when corrections were and were not remembered. For completeness, we still display the misinformation-recollection cells in Figure 3. A 2 (statement type) × 2 (classification) model indicated significant effects of statement type, χ²(1) = 7.18, p = .007, and classification, χ²(1) = 9.49, p = .002, and no significant Statement Type × Classification interaction, χ²(1) = 2.85, p = .091, showing significantly lower belief ratings when corrections were remembered than when they were not.

Discussion

Reminders increased recall and belief accuracy. Reminders also led to better misinformation recollection that was associated with better fact recall and higher beliefs. Although reminders did not improve memory for corrections themselves, remembering corrections was associated with fewer intrusions and less belief in misinformation. These results replicate those of Ecker et al. (2017) and extend the memory-for-change framework (Wahlheim & Jacoby, 2013) in suggesting that reminders counteract misinformation by increasing the conflict salience and recollection-based retrieval that support source memory.

Experiment 2

In Experiment 2, we attempted to replicate the findings that corrections with reminders improved memory and belief accuracy for fact-check-verified statements under conservative reporting conditions using response framing that emphasized memory quality over quantity. The comparable patterns from Ecker et al. (2017) and Experiment 1 led us to predict similar effects on memory here, but it was unclear how adjusting the report criteria would affect belief ratings.

Method

Subjects

Our stopping rule was the same as for Experiment 1. To meet our goal of collecting usable data from 96 subjects, we tested 98 UNCG undergraduates and excluded data from one subject who did not complete Phase 3 and another who did not follow instructions. Our final sample contained 96 subjects (54 women and 42 men; age: range = 18–32 years, M = 19.72, SD = 2.07). The sensitivity analysis from Experiment 1 was also applied here.

Design, materials, and procedure

The design and materials were identical to those used in Experiment 1. The procedures in Phases 1 and 2 were also identical to those in Experiment 1. However, Phase 3 differed in that subjects were first told that they would be asked questions about the topics they had read earlier and to report only information that they believed was true. When test cues appeared, subjects were first asked to type the details of believed facts. They were told that they could pass on a question when they could not recall any information and when they could recall information that they did not believe was true. Subjects were then asked to rate their confidence in their beliefs on a scale from 1 (not confident) to 6 (very confident) by pressing the corresponding number key. After that, they performed correction classification and misinformation recall as in Experiment 1. They were told that they could pass when they could not recall any misinformation.

Results

Scoring of belief reports (cued recall)

Belief reports were coded in the same manner as cued-recall responses in Experiment 1. A Cohen’s κ of .91 for initial ratings indicated acceptable agreement.

Belief reports (cued-recall performance)

Figure 4 (top left) displays fact recall for each statement type. A model including statement type as a factor indicated a significant effect, χ²(3) = 209.66, p < .001, showing that recall was significantly greater in the repetition condition and the correction + reminder condition than in the control and correction conditions, smallest z ratio = 10.03, p < .001, and that there were no significant differences between the repetition and correction + reminder conditions or between the control and correction conditions, largest z ratio = 0.49, p = .962. Figure 4 (top right) displays misinformation intrusions for the correction and correction + reminder conditions. A model with only those conditions indicated significantly more intrusions in the correction condition than in the correction + reminder condition, χ²(1) = 29.13, p < .001.

Fig. 4.

Results from Experiment 2: correction effects on the probability of correct recall (top row) and on belief ratings (bottom row). In each row, fact-recall results (left) are shown for each of the four statement types and misinformation-intrusion results (right) are shown for correction and correction + reminder statements. The bars display estimates from mixed-effect models, and the corresponding error bars show 95% confidence intervals. In the box-and-whisker plots, the diamonds indicate group means, the horizontal lines indicate medians, the heights of the boxes mark the interquartile ranges, and the whiskers extend 1.5 times the interquartile ranges. Dots represent individual subject probabilities.

Belief ratings

Figure 4 (bottom left) displays belief ratings for fact recall. A model including statement type as a factor indicated a significant effect, χ²(3) = 131.88, p < .001, showing significantly higher belief ratings in the correction + reminder condition than in all other conditions, smallest z ratio = 6.32, p < .001, and in the repetition and correction conditions than in the control condition, smallest z ratio = 3.65, p = .002. There was no significant difference between the repetition and correction conditions, z ratio = 1.44, p = .473. Figure 4 (bottom right) displays the belief ratings for misinformation intrusions for corrections and corrections + reminders. A model with those conditions indicated no significant difference, χ²(1) = 0.17, p = .684.

Correction classifications

Figure 5 displays the probabilities of correction classifications. A model including statement type as a factor (left) indicated significantly greater misinformation recollection in the correction + reminder condition than in the correction condition, χ²(1) = 29.34, p < .001. In addition, a model for remembered corrections (right) indicated significantly greater remembering in the correction + reminder condition than in the correction condition, χ²(1) = 16.21, p < .001.

Fig. 5.

Results from Experiment 2: correction effects on the probability of misinformation recollection (left) and memory for corrections (right), separately for correction and correction + reminder statements. The bars display estimates from mixed-effect models, and the corresponding error bars show 95% confidence intervals. In the box-and-whisker plots, the diamonds indicate group means, the horizontal lines indicate medians, the heights of the boxes mark the interquartile ranges, and the whiskers extend 1.5 times the interquartile ranges. Dots represent individual subject probabilities.

Recall and beliefs conditionalized on correction classifications

Cued recall

Figure 6 (top left) displays fact-recall probabilities. The model indicated significant effects of statement type, χ²(1) = 13.51, p < .001, and classification, χ²(2) = 228.98, p < .001, and no significant Statement Type × Classification interaction, χ²(2) = 2.33, p = .312, showing significantly higher fact recall when misinformation was recollected than when corrections were remembered, z ratio = 5.85, p < .001, and when corrections were remembered than when they were not, z ratio = 10.94, p < .001.

Fig. 6.

Results from Experiment 2: mixed-effect model estimates for the probability of correct recall (top row) and for belief ratings (bottom row). In each row, fact-recall results (left) and misinformation-intrusion results (right) are shown for correction and correction + reminder statements, both conditionalized and not conditionalized on correction classifications. Black points represent estimates for all observations. Colored points represent estimates conditionalized on misinformation recollection (green), memory for corrections (blue), and no memory for corrections (red). The size of each point indicates the relative proportion of observations in each conditional cell. Note that the point areas for overall belief ratings (black) vary because they reflect the number of fact-recall and misinformation-intrusion responses given in each statement-type condition (see the black points in the top row for those response probabilities). Error bars represent 95% confidence intervals.

Figure 6 (top right) displays misinformation-intrusion probabilities. The model indicated significant effects of statement type, χ²(1) = 8.38, p = .004, and classification, χ²(2) = 75.36, p < .001, and no significant Statement Type × Classification interaction, χ²(2) = 0.61, p = .738, showing significantly higher intrusion rates when corrections were not remembered than when they were, z ratio = 6.05, p < .001, and when corrections were remembered than when misinformation was recollected, z ratio = 4.43, p < .001.

Belief ratings

Figure 6 (bottom left) displays fact-recall belief ratings. The model indicated significant effects of statement type, χ²(1) = 38.15, p < .001, and classification, χ²(2) = 76.03, p < .001, and no significant Statement Type × Classification interaction, χ²(2) = 1.31, p = .520. Belief ratings were not significantly different when misinformation was recollected and corrections were remembered, t(1635) = 0.93, p = .625, but were significantly higher both when misinformation was recollected and corrections were remembered than when corrections were not remembered, smallest t(1710) = 7.61, p < .001.

Figure 6 (bottom right) displays misinformation-intrusion belief ratings. A 2 (statement type) × 2 (classification) model excluding recollected misinformation indicated no significant effect of statement type, χ²(1) = 0.30, p = .582, and a significant effect of classification, χ²(1) = 4.18, p = .041, qualified by a significant Statement Type × Classification interaction, χ²(1) = 7.21, p = .007. There were significantly fewer intrusions when corrections were remembered than when they were not in the correction condition, t(339) = 3.21, p = .001, but no significant difference in the correction + reminder condition, t(340) = 0.74, p = .463.

Discussion

Experiment 2 replicated the previous results: Reminders increased fact recall and beliefs and decreased misinformation intrusions, misinformation recollection was enhanced by reminders and was associated with improved fact recall, and remembering corrections was associated with fewer intrusions. This bolsters the suggestion that misinformation reminders benefit memory and belief accuracy, but there were nuanced differences suggesting a moderating role for report criteria.

General Discussion

We examined the effects of providing misinformation reminders before fake-news corrections on memory and belief accuracy. Our study included everyday fake-news misinformation that was corrected by fact-check-verified statements. Building on research using fictional, yet naturalistic, event narratives to show that reminders can counteract misinformation reliance in memory reports (e.g., Ecker et al., 2017), we showed improvements in both memory and belief accuracy when reminders increased misinformation recollection. These findings suggest that reminders of misinformation before corrections enhanced conflict salience, recollection of corrections, and source memory, thereby reducing familiarity-backfire effects.

The continued-influence effect

Two previous studies using fictional narratives describing plausible everyday events tested misinformation-reminder effects on the continued-influence effect. Wilkes and Leatherbarrow (1988) originally reported a trend showing that misinformation reminders reduced the continued-influence effect. Ecker et al. (2017) followed by showing clearly that corrections with reminders reduced the continued-influence effect more than corrections alone. The naturalism of the materials in those studies suggested that reminder benefits might generalize to other information experienced in daily life. Here, we replicated reminder benefits on memory and extended them to beliefs using news content from various media sources. This approach simulated the news exposure experienced when one reads headlines on a website. This approach differed from reading event descriptions, which enables the building of situation models, whereas reading statements requires less focused attention to encode central details. Despite these differences, the observed similarities in reminder benefits have theoretical implications in suggesting that the mechanisms of such benefits may generalize across information types. The extent of this generalization remains unknown.

Our findings are consistent with the conflict-salience and memory-updating views (Ecker et al., 2017; Kendeou et al., 2014; Stadtler et al., 2013; Wahlheim & Jacoby, 2013) in suggesting that reminders increased awareness of discrepancies and promoted memory updating. Reminders also increased misinformation recollection, which was associated with reduced misinformation reliance and enhanced belief accuracy. These findings are consistent with the suggestion from both the retrieval account of the continued-influence effect (Ecker et al., 2017; Ecker et al., 2010; Seifert, 2002; Swire et al., 2017) and the memory-for-change framework (Wahlheim & Jacoby, 2013) that recollection-based retrieval plays a role in reminder effects. Finally, reminders may have supported postretrieval monitoring (M. K. Johnson et al., 1993) by generating memory representations with more salient source details.

The variation in report framing between experiments produced nuanced differences suggesting a diminished willingness to report misinformation under conservative reporting conditions. This was shown by reminders reducing beliefs in misinformation intrusions only under free reporting, reminders increasing memory for corrections without misinformation recollection only under conservative reporting, and facilitation in fact recall being associated with misinformation recollection and remembering corrections only under conservative reporting. These results suggest that moderating variables of reminder effects should be considered.

Limitations and directions

We assumed that reminders would confer benefits by triggering more retrievals of misinformation than corrections alone. This assumption was based partly on findings implicating a role for reminder recognition in memory-updating benefits (Wahlheim et al., 2019). But here we could not separately assess the contribution of remindings stimulated by reminder cues and the increased salience of corrections resulting from direct prompting without an overt recognition measure and a contrast condition that points out corrections without reminders (cf. Ecker et al., 2017). Regardless, there is plenty of evidence that reminders play a role in episodic-memory updating (see Jacoby et al., 2015). Perhaps most relevant, our preliminary work with comparable stimuli showed that subjects often detected Phase 2 corrections that appeared without reminders (> 80%), which contributed to improved correction recall. Taken with the assumption that exact repetitions (reminders) cue retrieval better than nonexact repetitions (corrections), we are confident in asserting that reminders stimulated more remindings than corrections alone.

Although misinformation reminders can effectively counteract the continued-influence effect, there are some caveats. Reminder effects likely depended on recollection, which occurs less at longer intervals and older ages (cf. Skurnik et al., 2005; Swire et al., 2017), but our findings did not indicate whether reminder effects would hold under those conditions. Reminder effects on memory and belief updating may also be sensitive to subject variables such as working memory capacity (Brydges, Gignac, & Ecker, 2018), partisan attitudes (Ecker & Ang, 2019), and cognitive styles (Sinclair, Stanley, & Seli, 2020). To conclude, a broader characterization of optimal conditions for reminder effectiveness could lead to more precise recommendations about when and for whom reminders should mitigate the negative effects of fake-news exposure.

Supplemental Material

Wahlheim_Supplemental_Material_rev – Supplemental material for Reminders of Everyday Misinformation Statements Can Enhance Memory for and Beliefs in Corrections of Those Statements in the Short Term

Supplemental material, Wahlheim_Supplemental_Material_rev for Reminders of Everyday Misinformation Statements Can Enhance Memory for and Beliefs in Corrections of Those Statements in the Short Term by Christopher N. Wahlheim, Timothy R. Alexander and Carson D. Peske in Psychological Science

Footnotes

Acknowledgements

Portions of these findings were presented in a poster session at the 59th Meeting of the Psychonomic Society, New Orleans, Louisiana. For their assistance with data collection, response coding, or both, we thank Nora Alaati, McKayla Bohannon, Danni Brower, Tierea Hakim, Skye Harrelson, Peter Lee, Crystal Thinzar, Analiese Thomas, and Steven Windsor.

Transparency

Action Editor: D. Stephen Lindsay

Editor: D. Stephen Lindsay

Author Contributions

C. N. Wahlheim developed the study concept, performed the data analyses, and drafted the manuscript. C. N. Wahlheim and T. R. Alexander contributed to the study design. T. R. Alexander developed the stimuli and programmed the experiments. T. R. Alexander and C. D. Peske performed and supervised data collection and response coding. All authors approved the final manuscript for submission.

ORCID iD

Christopher N. Wahlheim

Supplemental Material

Additional supporting information can be found at

References

Bacon

F. T.

(1979). Credibility of repeated statements: Memory for trivia. Journal of Experimental Psychology: Human Learning and Memory, 5, 241–252. doi:10.1037/0278-7393.5.3.241

Bates

Mächler

Bolker

Walker

(2015). Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67(1). doi:10.18637/jss.v067.i01

Begg

I. M.

Anas

Farinacci

(1992). Dissociation of processes in belief: Source recollection, statement familiarity, and the illusion of truth. Journal of Experimental Psychology: General, 121, 446–458. doi:10.1037/0096-3445.121.4.446

Brydges

C. R.

Gignac

G. E.

Ecker

U. K. H.

(2018). Working memory capacity, short-term memory capacity, and the continued influence effect: A latent-variable analysis. Intelligence, 69, 117–122. doi:10.1016/j.intell.2018.03.009

Dechêne

Stahl

Hansen

Wänke

(2010). The truth about the truth: A meta-analytic review of the truth effect. Personality and Social Psychology Review, 14, 238–257. doi:10.1177/1088868309352251

Ecker

U. K. H.

Ang

L. C.

(2019). Political attitudes and the processing of misinformation corrections. Political Psychology, 40, 241–260. doi:10.1111/pops.12494

Ecker

U. K. H.

Hogan

J. L.

Lewandowsky

(2017). Reminders and repetition of misinformation: Helping or hindering its correction? Journal of Applied Research in Memory and Cognition, 6, 185–192. doi:10.1016/j.jarmac.2017.01.014

Ecker

U. K. H.

Lewandowsky

Tang

D. T. W.

(2010). Explicit warnings reduce but do not eliminate the continued influence of misinformation. Memory & Cognition, 38, 1087–1100. doi:10.3758/MC.38.8.1087

Fox

Weisberg

(2019). An R companion to applied regression (3rd ed.). Thousand Oaks CA: SAGE.

10.

Faul

Erdfelder

Buchner

Lang

A.-G.

(2009). Statistical power analyses using G*Power 3.1: Tests for correlation and regression analyses. Behavior Research Methods, 41, 1149–1160.

11.

Hasher

Goldstein

Toppino

(1977). Frequency and the conference of referential validity. Journal of Verbal Learning and Verbal Behavior, 16, 107–112. doi:10.1016/S0022-5371(77)80012-1

12.

Jacoby

L. L.

(1991). A process dissociation framework: Separating automatic from intentional uses of memory. Journal of Memory and Language, 30, 513–541. doi:10.1016/0749-596X(91)90025-F

13.

Jacoby

L. L.

(1999). Ironic effects of repetition: Measuring age-related differences in memory. Journal of Experimental Psychology: Learning, Memory, and Cognition, 25, 3–22. doi:10.1037/0278-7393.25.1.3

14.

Jacoby

L. L.

Wahlheim

C. N.

Kelley

C. M.

(2015). Memory consequences of looking back to notice change: Retroactive and proactive facilitation. Journal of Experimental Psychology: Learning, Memory, and Cognition, 41, 1282–1297. doi:10.1037/xlm0000123

15.

Johnson

H. M.

Seifert

C. M.

(1998). Updating accounts following a correction of misinformation. Journal of Experimental Psychology: Learning, Memory, and Cognition, 24, 1483–1494. doi:10.1037/0278-7393.24.6.1483

16.

Johnson

M. K.

Hashtroudi

Lindsay

D. S.

(1993). Source monitoring. Psychological Bulletin, 114, 3–28.

17.

Kendeou

Walsh

E. K.

Smith

E. R.

O’Brien

E. J.

(2014). Knowledge revision processes in refutation texts. Discourse Processes, 51, 374–397. doi:10.1080/0163853X.2014.913961

18.

Larson

H. J.

Cooper

L. Z.

Eskola

Katz

S. L.

Ratzan

(2011). Addressing the vaccine confidence gap. The Lancet, 378, 526–535. doi:10.1016/S0140-6736(11)60678-8

19.

Lenth

(2019). emmeans: Estimated marginal means, aka least-squares means (R Package Version 1.4.7) [Computer software]. Retrieved from https://CRAN.R-project.org/package=emmeans

20.

Lewandowsky

Ecker

U. K. H.

Seifert

C. M.

Schwarz

Cook

(2012). Misinformation and its correction: Continued influence and successful debiasing. Psychological Science in the Public Interest, 13, 106–131. doi:10.1177/1529100612451018

21.

Morrow

D. G.

Bower

G. H.

Greenspan

S. L.

(1989). Updating situation models during narrative comprehension. Journal of Memory and Language, 28, 292–312. doi:10.1016/0749-596X(89)90035-1

22.

Pennycook

Cannon

T. D.

Rand

D. G.

(2018). Prior exposure increases perceived accuracy of fake news. Journal of Experimental Psychology: General, 147, 1865–1880. doi:10.1037/xge0000465

23.

Peter

Koch

(2016). When debunking scientific myths fails (and when it does not): The backfire effect in the context of journalistic coverage and immediate judgments as prevention strategy. Science Communication, 38, 3–25. doi:10.1177/1075547015613523

24.

Poland

G. A.

Spier

(2010). Fear, misinformation, and innumerates: How the Wakefield paper, the press, and advocacy groups damaged the public health. Vaccine, 28, 2361–2362. doi:10.1016/j.vaccine.2010.02.052

25.

Putnam

A. L.

Wahlheim

C. N.

Jacoby

L. L.

(2014). Memory for flip-flopping: Detection and recollection of political contradictions. Memory & Cognition, 42, 1198–1210. doi:10.3758/s13421-014-0419-9

26.

R Core Team. (2020). R: A language and environment for statistical computing. Retrieved from http://www.R-project.org

27.

Schneider

Eschman

Zuccolotto

(2012). E-Prime 2.0 reference guide manual. Pittsburgh, PA: Psychology Software Tools.

28.

Schwarz

Newman

Leach

(2016). Making the truth stick & the myths fade: Lessons from cognitive psychology. Behavioral Science & Policy, 2(1), 85–95.

29.

Schwarz

Sanna

L. J.

Skurnik

Yoon

(2007). Metacognitive experiences and the intricacies of setting people straight: Implications for debiasing and public information campaigns. In Zanna

M. P.

(Ed.), Advances in experimental social psychology (Vol. 39, pp. 127–161). San Diego, CA: Academic Press. doi:10.1016/S0065-2601(06)39003-X

30.

Seifert

C. M.

(2002). The continued influence of misinformation in memory: What makes a correction effective? In Ross

B. H.

(Ed.), Psychology of learning and motivation: Advances in research and theory (Vol. 41, pp. 265–292). San Diego, CA: Academic Press.

31.

Sinclair

A. H.

Stanley

M. L.

Seli

(2020). Closed-minded cognition: Right-wing authoritarianism is negatively related to belief updating following prediction error. Psychonomic Bulletin & Review. Advance online publication. doi:10.3758/s13423-020-01767-y

32.

Skurnik

Yoon

Park

D. C.

Schwarz

(2005). How warnings about false claims become recommendations. Journal of Consumer Research, 31, 713–724. doi:10.1086/426605

33.

Stadtler

Scharrer

Brummernhenrich

Bromme

(2013). Dealing with uncertainty: Readers’ memory for and use of conflicting information from science texts as function of presentation format and source expertise. Cognition and Instruction, 31, 130–150. doi:10.1080/07370008.2013.769996

34.

Swire

Ecker

U. K. H.

Lewandowsky

(2017). The role of familiarity in correcting inaccurate information. Journal of Experimental Psychology: Learning, Memory, and Cognition, 43, 1948–1961. doi:10.1037/xlm0000422

35.

Wahlheim

C. N.

Jacoby

L. L.

(2013). Remembering change: The critical role of recursive remindings in proactive effects of memory. Memory & Cognition, 41, 1–15. doi:10.3758/s13421-012-0246-9

36.

Wahlheim

C. N.

Smith

W. G.

Delaney

P. F.

(2019). Reminders can enhance or impair episodic memory updating: A memory-for-change perspective. Memory, 27, 849–867. doi:10.1080/09658211.2019.1582677

37.

Wahlheim

C. N.

Zacks

J. M.

(2019). Memory guides the processing of event changes for older and younger adults. Journal of Experimental Psychology: General, 148, 30–50. doi:10.1037/xge0000458

38.

Weaver

Garcia

S. M.

Schwarz

Miller

D. T.

(2007). Inferring the popularity of an opinion from its familiarity: A repetitive voice can sound like a chorus. Journal of Personality and Social Psychology, 92, 821–833. doi:10.1037/0022-3514.92.5.821

39.

Wilkes

A. L.

Leatherbarrow

(1988). Editing episodic memory following the identification of error. The Quarterly Journal of Experimental Psychology Section A: Human Experimental Psychology, 40, 361–387. doi:10.1080/02724988843000168

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.50 MB