Exposure to Countering Messages Online: Alleviating or Strengthening False Belief?

Abstract

Posting countermessages is commonly used as a strategy to combat false rumors spreading online. The effectiveness of countermessage exposure has been investigated in past studies, but little is known about its repercussions. The aim of this study was to contribute to the understanding of rumor control by investigating the factors impacting the effectiveness of countermessage exposure. A total of 164 participants were asked to judge the believability of rumor and factual tweets before and after countermessage exposure in a web-based experiment. Two forms of countermessage were compared to examine the effects of countermessages on belief change in the target tweets. One was subjective countermessages based on personal experiences, and the other was objective countermessages based on evidence. The results showed that objective countermessages reduced belief in rumor tweets, whereas subjective countermessages strengthened false beliefs. In addition, the half of the participants who were exposed to objective countermessages randomly mixed with subjective countermessages formed negative attitudes not only toward the rumor tweets but also toward the factual tweets. The results also showed gender differences in response to countermessage exposure; women tended to be more susceptible to countermessages and changed their beliefs regarding the target tweets negatively after the exposure. We discuss the practical implications of the results associated with the adverse effects of countermessage exposure.

Introduction

The recent exponential growth in communication technologies has affected how rumors spread. Rumors are defined as “public communications that are infused with private hypotheses about how the world works”¹; they differ from gossip, which involves evaluative statements about someone's private life.² People are more likely to transmit rumors than accurate news on social media.³ The Internet serves as a platform to disseminate different kinds of rumor, such as those pertaining to faulty scientific knowledge,^4,5 business reputations,⁶ political biases,⁷ and disasters.⁸

As a strategy to combat rumors, past studies^9,10 focused on the role of counterrumor messages that deny, criticize, or question rumors to mitigate belief in them.¹¹ Perceived accuracy in rumors was reduced by exposure to countermessages, quelling the intention to transmit false rumors.¹² In reality, there is a move toward using such messages to detect rumors automatically.^13,14

Despite the promise of countermessage exposure, little is known of the risks. Previous psychological experiments^9,15 operationally set a questionable rumor as a target to test the effects of countermessage exposure; another study¹⁶ called attention to counters as a type of fake news. Countermessaging is also used to manipulate information because it can create a sense of legitimacy by finding fault with an opposing argument/message and labeling it false. In our rapidly changing information society, we need to carefully consider the prospect of countermessages turning out to be false.

This study examined the effect of countermessage exposure on beliefs and the associated risk. To display messages, we used a layout mimicking a “tweet” on Twitter. A countermessage was operationally defined as a message that questions the credibility of a tweet by referring to contradictory information in reference to a target message.

The primary focus was countermessage quality. A past study revealed that rumor belief tended to be mitigated by persuasive countermessages with strong arguments.¹¹ Another study⁹ demonstrated that refutations with stronger arguments reduced belief in rumors among people with a negative attitude toward the rumor. Thus, we hypothesized that the quality of countermessages will influence belief in false tweets. In this study, the quality of countermessages was manipulated in terms of subjectivity and objectivity. We hypothesized that exposure to objective counters would cause greater belief reduction in target rumor tweets compared with subjective counters (Hypothesis 1).

The secondary focus was on the potential repercussions of countermessage exposure, especially when the countermessage is untrustworthy and the target is true. We hypothesized that exposure to untrustworthy counters would cause belief reduction in factual targets (Hypothesis 2). If this were true, it would imply that exposure to countermessages can not only be helpful in reducing belief in rumors but also adverse in terms of distorting belief in facts. To test this, we compared two types of target: false rumor tweets and factual tweets.

The third focus was individual factors that predict belief change after countermessage exposure. A related study¹⁷ demonstrated a negative relationship between critical thinking and the tendency to believe misconceptions regarding psychological knowledge. In addition to critical thinking ability, age and gender were used as potential factors associated with individual differences in the effects of counter exposure.

Materials and Methods

Participants

An a priori power analysis using G*Power 3.1¹⁸ determined the required sample size as 64 for the 2 groups to have a power of 0.08 and to detect an effect size (f) of 0.25, using an alpha of 0.05. Once additional participants were added in case of attrition, 242 (115 women, 127 men) Japanese adults (M_age = 41.4 years, SD_age = 13.75, range: 18–75 years old) were recruited by an online research service (Cross Marketing, Inc., Japan) and voluntarily participated through the Internet after providing informed consent.

Stimuli

Three rumor tweets and three factual tweets were the targets (Table 1). False rumors were selected from a book on popular psychology,¹⁹ whereas factual topics were chosen from psychology textbooks. For each rumor, subjective and objective versions were developed. Subjective countermessages were operationally defined as critiques based on personal experience. Conversely, objective countermessages were defined as critiques based on objective reasons, citing evidence that contradicts or is inconsistent with rumors. For each factual message, only a subjective counter was developed for ethical reasons; an objective counter would have required fabricating evidence, and that could have unnecessarily distorted the participants' beliefs.

Table 1.

Stimuli: Target Tweets and Countering Messages

R1:	The weight of the brain is ∼1.5 kg and apparently, humans, approximately, use only 10% of their brains.
	S:	This is wrong. It is incredible how some can state that people use only 10% of their brains. When I am anxious, I feel mentally exhausted because I am using various parts of my brain.
	O:	This is wrong. The weight of the brain is only 2–3% of our bodyweight but it consumes >20% of the oxygen we get from breathing. If only 10% of the brain were being used, it would not require this much oxygen.
R2:	If people are presented information below a conscious threshold (subliminal) over a short span of time, then they can be persuaded or be pushed to buy products.
	S:	This is wrong. When I buy something, I always buy what I want to buy based on my own volition. In other words, if you have a strong will, you will not be affected by subliminal messages.
	S:	Hypnosis can be used to make a witness remember the details of a crime.
	O:	This is wrong. An analysis of research from a Canadian television station revealed that the subliminal message “please telephone us right away” was aired 352 times. However, there was no increase in incoming telephone calls. Likewise, people cannot be made to buy things in this manner.
R3:	Hypnosis can be used to make a witness remember the details of a crime.
	S:	This is wrong. I have attempted hypnosis in the past, but it did not work on me. My friends who had participated with me were also unable to recount details that they had long forgotten. Therefore, in my experience, people do not remember things accurately while under hypnosis.
	O:	This is wrong. People's memories are not always accurate and they forget things as time passes. Hypnosis creates a mixture of images in the mind that are drawn from facts. Hence, it is probable that hypnosis actually increases the number of erroneous memories.
F1:	Apparently, people become more prone to committing acts of violence by watching movies or playing violent video games.
	S:	I frequently watch action movies, and I also play fighting games a lot. However, I have never hit anyone in actuality. Therefore, it is not entirely feasible to simply state that watching violence makes a person become violent.
F2:	Doing something else after forming a memory apparently disturbs the memory. Therefore, memorization should be performed before sleeping.
	S:	People usually go to sleep after a tiring day. My friends, for instance, study in the morning as they say that this is when their mind seems to be the clearest. My friends' grades are actually quite good, so I think that it is better to memorize things in the morning than before going to bed.
F3:	People apparently acquire a “sense of helplessness” when they have repeated experiences in which their situation does not improve, despite their efforts and resistance.
	S:	I have repeatedly had experiences that didn't go well, even though I tried very hard. However, I am still taking on new challenges. It is a strange assertion to say that a sense of helplessness is an acquired ability and that a person's situation does not improve despite making serious efforts.

[R], rumor tweet; [F], factual tweet; [O], objective counter; [S], subjective counter.

Each message was converted into a Twitter PNG image. To avoid the influence of perceived source credibility,¹¹ usernames were created by randomly ordering letters. User images were an egg shape against a colored background. The countering tweet was created by showing the target rumor tweet below the countermessage.

Procedure

Each participant accessed and completed all procedures online. After some demographic questions, participants proceeded through four phases: 1.

Preexposure belief measurement: The rumor and factual tweets were presented one at a time. The presentation order was randomized. Participants were asked to answer the following three questions about each tweet: (1) Familiarity (Yes: I have heard, No: I have never heard); (2) Accuracy (1: Not at all, 7: Highly accurate); (3) Importance (1: Not at all, 7: Highly important); and (4) Interest (1: Not at all, 7: Highly interested). Participants were not informed that some stimuli were false.

Countermessage exposure: Participants were randomly allocated to either the subjective counter group (SG) or mixed counter group (MG). For the rumor tweets, SG members were presented with subjective counters, whereas MG members were presented with objective counters. All participants were presented with subjective counters for the factual tweets.

Postexposure belief measurement: The same set of rumor and factual tweets from the prebelief session were presented again, excluding the familiarity question.

Critical thinking test: We used a subset of the Watson Glaser Critical Thinking Appraisal²⁰ that specifically focuses on critical thinking inferences.²¹ After receiving directions and practice questions, participants were allowed 12 minutes to complete the test. The test was automatically terminated when the time expired.

Participants were debriefed on the study's purpose after completion. A timestamp was recorded every time a participant proceeded to the next phase. Participants completed the tasks at their own pace (except for the critical thinking test).

Results

A problem with online studies is the potential for invalid data.²² To reduce the effect of unreliable responses, we planned and eliminated 1.2% of participants whose item response times were too long or short (under 1 second or over 60 seconds). A further 31.4% of participants who completed the critical thinking test phase too quickly (under 4 minutes) were also eliminated.

In total, 164 participants (53% female; M_age = 42.60, SD_age = 14.21, range: 18–75 years old) had valid response data. A post hoc power analysis using G*Power 3.1¹⁸ revealed that there was adequate power >0.80 at the medium to large effect size levels based on the recommended effect sizes used for analysis of variance (ANOVA) (f) and multiple regression (f²).²³

As a manipulation check, we conducted t- and χ² tests to ensure that the two groups differed only with respect to counter quality. The results revealed no significant differences between the SG and MG in terms of age, proportion of women, or critical thinking scores. The average critical thinking score was 7.30 (SD = 2.94) out of 20. Table 2 shows the mean and standard deviations for pre- and postexposure beliefs. The results of a group by target ANOVA on prebelief revealed no significant main effects of group or target on accuracy, importance, or interest.

Table 2.

Means (Standard Deviations) Obtained from Subjective Counter Group and Mixed Counter Group, in Both Pre- and Postexposure to Denials

		Preexposure		Postexposure
Target	Belief	SG	MG	SG	MG
Rumor tweets	Accuracy	3.80 (1.49)	3.84 (1.35)	3.95 (1.51)	3.84 (1.16)
	Importance	3.66 (1.67)	3.65 (1.53)	3.87 (1.69)	3.56 (1.41)
	Interest	3.69 (1.79)	3.66 (1.68)	3.93 (1.76)	3.65 (1.56)
Factual tweets	Accuracy	3.78 (1.58)	3.94 (1.34)	3.91 (1.60)	3.86 (1.31)
	Importance	3.77 (1.76)	4.01 (1.56)	3.88 (1.74)	3.78 (1.47)
	Interest	3.76 (1.78)	3.78 (1.78)	3.96 (1.87)	3.59 (1.59)

MG, mixed denial group (objective denials for rumor tweets and subjective denials for factual tweets); SG, subjective denial group (subjective denials for both rumor and factual tweets).

To test the effects of countermessage type, belief change before and after countermessage exposure was compared between groups. A group by target mixed ANOVA revealed a significant main effect of group: F(1, 162) = 4.86, p = 0.029, η²_G = 0.02. Figure 1 shows the mean and standard errors for belief change after countermessage exposure. In the SG, beliefs changed positively after countermessage exposure for both rumor and factual tweets, whereas the beliefs of the MG changed negatively after countermessage exposure for factual tweets but not for rumor tweets. These results supported Hypothesis 1. For importance and interest, the same patterns emerged; there was a significant main effect of group: F's(1, 162) = 9.96 and 7.59, p = 0.002 and 0.007, η²_G = 0.04 and 0.03, respectively. There was no significant effect of target or interaction. For both importance and interest, beliefs were changed positively in the SG but negatively in the MG, regardless of target type. The main effect of the target did not reach statistical significance in any of the three beliefs, nor did the interaction effect. These results were in line with Hypothesis 2.

FIG. 1.

Means and standard errors for belief changes in perceived accuracy, importance, and interest (pre–post). MG, mixed denial group (objective denials for rumor tweets and subjective denials for factual tweets); SG, subjective denial group (subjective denials for both rumor and factual tweets).

Next, we turned to individual factors and belief change (pre–post) after countermessage exposure. Using the “lme4” package²⁴ in R,²⁵ we constructed a generalized linear mixed model of belief change, entering group and gender as fixed effects and subjects and multiple stimuli as random effects. Gender (men as a reference category) predicted belief change in accuracy (χ²(1) = 10.12, p = 0.001), decreasing it by about −0.27 ± 0.08 (standard errors) [95% confidence interval (CI): −0.44 to −0.11]. Gender also predicted both importance [χ²(1) = 4.59, p = 0.03] and interest (χ²(1) = 6.87, p = 0.008); females showed more negative belief change after counter exposure (−0.21 ± 0.10 [standard errors] [95% CI: −0.40 to −0.02], −0.25 ± 0.09 [standard errors] [95% CI: −0.43 to −0.07], respectively). Critical thinking ability and the other individual factors did not predict belief change.

Discussion

This study investigated whether countermessage quality interferes with belief change in rumor and factual tweets. As predicted, exposure to objective counters tended to alleviate belief in false rumors, especially those associated with perceived importance and interest. This was consistent with previous studies^12,15,26 that observed that countermessages reduced rumor belief. However, the positive effects of exposure were moderated by countermessage quality. When participants were exposed to subjective counters, belief in rumor tweets was strengthened afterward. While supporting Hypothesis 1, the results were inconsistent with past studies on countermessages' effectiveness. This suggests that countermessage quality matters in determining whether they alleviate false beliefs or strengthen them. Subjective counters exemplify the latter case.

Supporting Hypothesis 2, exposure to objective counters exerted both positive and negative effects on belief change. Despite all participants being presented with subjective counters for factual tweets, belief in factual tweets was weakened in the MG but strengthened in the SG. Why did the MG develop negative attitudes toward factual tweets despite them never being denied objectively? Perhaps, while exposed to subjective counters randomly mixed with objective counters, participants overgeneralized the strength of arguments in objective counters to subjective counters. This opposite direction can be interpreted as an adverse effect of objective counter exposure.

As for individual factors, critical thinking ability did not moderate the effect of counter exposure. However, gender was an individual factor in predicting belief change. Compared with men, women perceived the target tweet to be less believable after countermessage exposure, suggesting they are susceptible to countermessages. This susceptibility would work well when a countermessage was true and the target was false; however, it would be counterproductive when a countermessage was false and the target was true.

A study limitation is that the results refer to immediate effects after one-time counter exposure. For instance, we were unable to assess the illusory truth effect,²⁷ which refers to people gaining confidence in their response following repetitive exposure to information. This effect appears even when people acquire contradictory information.²⁸ Further research is needed to understand whether the effects of counter exposure can be strengthened by repetition and how long this will last.

Although recent rumor psychology research has clarified the pros of countermessages, this study demonstrated their cons in the context of evaluating target tweets. Our findings contribute to the understanding of rumor control by elucidating the repercussions of false countermessages on beliefs and the importance of examining countermessages' quality. One practical implication is that it might be too early to implement an automated system based on counters to detect rumors. Furthermore, the gender differences in responses to countermessages suggest that men and women might benefit from different types of intervention for preventing the diffusion of falsity online.

Footnotes

Acknowledgments

We would like to thank anonymous reviewers for their helpful comments and suggestions.

Author Disclosure Statement

No competing financial interests exist.

Funding Information

This work was supported by KAKENHI Grant Numbers 26780376 and 18K12010. The sponsors have no involvement in deciding the study design, the collection, analysis, and interpretation of data, the writing of the report, and the decision to submit the paper for publication.

References

Rosnow

. Inside rumor: A personal journey. American Psychologist, 1991; 46:484–496.

Rosnow

, Foster

. Rumor and gossip research. APA Online: Psychological Science Agenda, 2005; 19:1–4.

Vosoughi

, Roy

, Aral

. The spread of true and false news online. Science, 2018; 359:1146–1151.

DiFonzo

, Robinson

, Suls

, et al. Rumors about cancer: Content, sources, coping, transmission, and belief. Journal of Health Communication, 2012; 17:1099–1115.

Horne

, Powell

, Hummel

, et al. Countering antivaccination attitudes. Proceedings of the National Academy of Sciences, 2015; 112:10321–10324.

DiFonzo

, Bordia

. How top PR Professionals handle hearsay: Corporate rumors, their effects, and strategies to manage them. Public Relations Review, 2000; 26:173–190.

Garrett

. Troubling consequences of online political rumoring. Human Communication Research, 2011; 37:255–274.

Spiro

, Sutton

, Greczek

, et al. (2012) Rumoring during extreme events: A case study of deepwater horizon 2010. In Proceedings of the ACM Web Science 2012 Conference. New York, NY: ACM, pp. 275–283.

Einwiller

, Kamins

. Rumor has it: The moderating effect of identification on rumor impact and the effectiveness of rumor refutation. Journal of Applied Social Psychology, 2008; 38:2248–2272.

10.

Iyer

, Debevec

. Origin of rumor and tone of message in rumor quelling strategies. Psychology and Marketing, 1991; 8:161–175.

11.

Bordia

, DiFonzo

, Haines

, et al. Rumors denials as persuasive messages: Effects of personal relevance, source, and message characteristics. Journal of Applied Social Psychology, 2005; 35:1301–1331.

12.

Tanaka

, Sakamoto

, Matsuka

. (2013) Toward a social-technological system that inactivates false rumors through the critical thinking of crowds. In Proceedings of the 46th Hawaii International Conference on System Sciences. Washington, DC: IEEE Computer Society, pp. 649–658.

13.

Mendoza

, Poblete

. (2014) Twitter under crisis: Can we trust what we RT? In Proceedings of the First Workshop on Social Media Analytics. New York, NY: ACM, pp. 71–79.

14.

Zhao

, Resnick

, Mei

. Enquiring minds: Early detection of rumors in social media from enquiry posts. In WWW’15 Proceedings of the 24th International Conference on World Wide Web. Geneva, Switzerland: International World Wide Web Conferences Steering Committee, pp. 1395–, 1405.

15.

Bordia

, DiFonzo

, Schulz

. Source characteristics in denying rumors of organizational closure: Honesty is the best policy. Journal of Applied Social Psychology, 2000; 30:2309–2321.

16.

Caplan

, Hanson

, Donovan

. (2018) Dead and reckoning: Navigating content moderation after “fake news.” Data & Society Research Institute. https://datasociety.net/pubs/oh/DataAndSociety_Dead_Reckoning_2018.pdf (accessed March 22, 2019).

17.

Bensley

, Lilienfeld

, Powell

. A new measure of psychologial misconceptios: Relations with academic background, critical thinking, and acceptance of paranormal and pseudoscientific claims. Learning and Individual Differences, 2014; 36:9–18.

18.

Faul

, Erdfelder

, Lang

A-G

, et al. G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 2007; 39:175–191.

19.

Lilienfeld

, Lynn

, Ruscio

, et al. (2010) 50 Great myths of popular psychology: Shattering widespread misconceptions about human behavior. Oxford, UK: Wiley-Blackwell.

20.

Watson

, Glaser

. (1964) Watson-Glaser critical thinking appraisal. New York: Harcourt Brace World.

21.

Kuhara

, Inoue

, Hatano

. Construction and validation of a test assessing critical thinking ability. The Science of Reading, 1983; 27:131–142.

22.

Gosling

, Mason

. Internet research in psychology. Annual Review of Psychology, 2015; 66:877–902.

23.

Cohen

. (1988) Statistical power analysis for the behavioral sciences. Hillsdale, NJ: Lawrence Erlbaum Associates.

24.

Bates

, Maechler

, Bolker

, et al. Fitting linear mixed-effects models using lme. Journal of Statistical Software, 2015; 67:1–48.

25.

R Core Team. (2013) R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing.

26.

Koller

. Rebutting accusations: When does it work, when does it fail?. European Journal of Social Psychology, 1993; 23:373–389.

27.

Hasher

, Goldstein

, Toppino

. Frequency and the conference of referential validity. Journal of Verbal Learning and Verbal Behavior, 1977; 16:107–112.

28.

Fazio

, Brashier

, Payne

, et al. Knowledge does not protect against illusory truth. Journal of Experimental Psychology: General, 2015; 144:993–1002.