Negative Feedback Fuels Hate Speech: A Deep Learning Analysis of 25 Million News Comments

Abstract

This study investigates the dynamics of hate speech, looking at feedback on comments and subsequent commenting. We examine the relationship between feedback on comments, hate speech presence, and commenter types, with analysis of news comments during the 2022 South Korean presidential election campaigns. The data include 25 million comments, analyzed with a deep learning hate speech detection model. It was found that positive feedback encourages more commenting for non-hateful content, and negative feedback reduces subsequent non-hateful comments. However, surprisingly, negative feedback was found to rather increase the frequency of hateful comments, particularly among light commenters. Implications of the findings are discussed.

Keywords

hate speech feedback news comment user engagement deep learning automated detection model

Get full access to this article

View all access options for this article.

References

Abrams

Wetherell

Cochrane

Hogg

M. A.

Turner

J. C.

(1990). Knowing what to think by knowing who you are: Self-categorization and the nature of norm formation, conformity and group polarization. British Journal of Social Psychology, 29(2), 97–119. https://doi.org/10.1111/j.2044-8309.1990.tb00892.x

Allan

(2023). The pleasure and pain of swearing. https://users.monash.edu.au/~kallan/homepage.html

Allport

G. W.

(1954). The nature of prejudice. Addison-Wesley.

Álvarez-Benjumea

Winter

(2018). Normative change and culture of hate: An experiment in online environments. European Sociological Review, 34(3), 223–237. https://doi.org/10.1093/esr/jcy005

Antonioni

(1998). Relationship between the big five personality factors and conflict management styles. International Journal of Conflict Management, 9, 336–355. https://doi.org/10.1108/eb022814

Baldassare

Katz

(1996). Measures of attitude strength as predictors of willingness to speak to the media. Journalism & Mass Communication Quarterly, 73(1), 147–158. https://doi.org/10.1177/107769909607300113

Brady

W. J.

McLoughlin

Doan

T. N.

Crockett

M. J.

(2021). How social learning amplifies moral outrage expression in online social networks. Science Advances, 7(33), eabe5641. https://doi.org/10.1126/sciadv.abe5641

Branscombe

N. R.

Ellemers

Spears

Doosje

(1999). The context and content of social identity threat. In Ellemers

Spears

(Eds.), Social identity: Context, commitment, content (pp. 35–58). Blackwell Science.

Buckley

K. E.

Winkel

R. E.

Leary

M. R.

(2004). Reactions to acceptance and rejection: Effects of level and sequence of relational evaluation. Journal of Experimental Social Psychology, 40(1), 14–28. https://doi.org/10.1016/S0022-1031(03)00064-7

10.

Carr

C. T.

Hayes

R. A.

Sumner

E. M.

(2018). Predicting a threshold of perceived Facebook post success via likes and reactions: A test of explanatory mechanisms. Communication Research Reports, 35(2), 141–151. https://doi.org/10.1080/08824096.2017.1409618

11.

Carr

C. T.

Wohn

D. Y.

Hayes

R. A.

(2016). As social support: Relational closeness, automaticity, and interpreting social support from paralinguistic digital affordances in social media. Computers in Human Behavior, 62, 385–393. https://doi.org/10.1016/j.chb.2016.03.087

12.

Chen

G. M.

(2015). Losing face on social media: Threats to positive face lead to an indirect effect on retaliatory aggression through negative affect. Communication Research, 42(6), 819–838. https://doi.org/10.1177/0093650213510937

13.

Cheng

Danescu-Niculescu-Mizil

Leskovec

(2014, June 1–4). How community feedback shapes user behavior [Paper presentation]. Eighth International AAAI Conference on Web and Social Media, Ann Arbor, MI, United States.

14.

Cialdini

R. B.

Goldstein

N. J.

(2004). Social influence: Compliance and conformity. Annual Review of Psychology, 55, 591–621. https://doi.org/10.1146/annurev.psych.55.090902.142015

15.

Cialdini

R. B.

Trost

M. R.

(1998). Social influence: Social norms, conformity, and compliance. In Gilbert

D. T.

Fiske

S. T.

Lindzey

(Eds.), The handbook of social psychology (Vol. 4, pp. 151–192). McGraw-Hill.

16.

Correa

Jeong

S. H.

(2011). Race and online content creation: Why minorities are actively participating in the Web. Information, Communication & Society, 14(5), 638–659. https://doi.org/10.1080/1369118X.2010.514355

17.

Crandall

C. S.

Miller

J. M.

White

M. H.

(2018). Changing norms following the 2016 US presidential election: The Trump effect on prejudice. Social Psychological and Personality Science, 9(2), 186–192. https://doi.org/10.1177/1948550617750735

18.

Crawford

Gillespie

(2016). What is a flag for? Social media reporting tools and the vocabulary of complaint. New Media & Society, 18(3), 410–428. https://doi.org/10.1177/1461444814543163

19.

Davidson

Warmsley

Macy

Weber

(2017, May 15–18). Automated hate speech detection and the problem of offensive language [Paper presentation]. 11th International AAAI Conference on Web and Social Media, Montreal, Quebec, Canada.

20.

Deutsch

Gerard

H. B.

(1955). A study of normative and informational social influences upon individual judgment. The Journal of Abnormal and Social Psychology, 51(3), 629–636. https://doi.org/10.1037/h0046408

21.

Diggle

P. J.

Heagerty

Liang

K.-Y.

Zeger

(2002). Analysis of longitudinal data (2nd ed.). Oxford University Press.

22.

Dynel

(2012). Swearing methodologically: The (im) politeness of expletives in anonymous commentaries on YouTube. Journal of English Studies, 10, 25–50.

23.

Erjavec

Kovačič

M. P.

(2012). “You don’t understand, This is a new war!” Analysis of hate speech in news web sites’ comments. Mass Communication and Society, 15(6), 899–920. https://doi.org/10.1080/15205436.2011.619679

24.

European Commission Against Racism and Intolerance. (2011). ECRI report on Spain. https://rm.coe.int/fourth-report-on-spain/16808b56c6

25.

Fisher

R. A.

(1915). Frequency distribution of the values of the correlation coefficient in samples from an indefinitely large population. Biometrika, 10(4), 507–521. https://doi.org/10.2307/2331838

26.

Freelon

(2015). Discourse architecture, ideology, and democratic norms in online political discussion. New Media & Society, 17(5), 772–791. https://doi.org/10.1177/1461444813513259

27.

Freeman

E. A.

Moisen

G. G.

(2008). A comparison of the performance of threshold criteria for binary classification in terms of predicted prevalence and kappa. Ecological Modelling, 217(1), 48–58. https://doi.org/10.1016/j.ecolmodel.2008.05.015

28.

Glynn

C. J.

Hayes

A. F.

Shanahan

(1997). Perceived support for one’s opinions and willingness to speak out: A meta-analysis of survey studies on the “spiral of silence”. Public Opinion Quarterly, 61(3), 452–463. http://www.jstor.org/stable/2749581

29.

Glynn

C. J.

McLeod

J. M.

(1984). Public opinion du Jour: An examination of the spiral of silence. Public Opinion Quarterly, 48(4), 731–740. https://doi.org/10.1086/268879

30.

Golder

S. A.

Donath

(2004, September 19–22). Social roles in electronic communities. [Paper presentation]. Association of Internet Researchers (AoIR) Conference Internet Research 5.0, Brighton, England.

31.

Golf-Papez

Veer

(2017). Don’t feed the trolling: Rethinking how online trolling is being defined and combated. Journal of Marketing Management, 33(15–16), 1336–1354. https://doi.org/10.1080/0267257X.2017.1383298

32.

Hammond

Watson-Nortey

N. K. L.

Osei-Mensah

(2022). “Why aren’t you married? The same reason you are not dead, it’s not my time”: Social media use of hate speech on female celebrities in Ghana. Research Journal of Mass Communication and Information Technology, 8(1), 109031. https://doi.org/10.56201/rjmcit.v8.no1.2022.pg109.131

33.

Hayes

R. A.

Wesselmann

E. D.

Carr

C. T.

(2018). When nobody “likes” you: Perceived ostracism through paralinguistic digital affordances within social media. Social Media + Society, 4(3), 1–12. https://doi.org/10.1177/205630511880030

34.

Herring

Job-Sluder

Scheckler

Barab

(2002). Searching for safety online: Managing “Trolling” in a feminist forum. The Information Society, 18(5), 371–384. https://doi.org/10.1080/01972240290108186

35.

Hogg

M. A.

(2010). Influence and leadership. In Fiske

S. T.

Gilbert

D. T.

Lindzey

(Eds.), Handbook of social psychology (pp. 1166–1207). John Wiley & Sons. https://doi.org/10.1002/9780470561119.socpsy002031

36.

Hong

Chen

(2017). “Liking” and being “liked”: How are personality traits and demographics associated with giving and receiving “likes” on Facebook? Computers in Human Behavior, 68, 292–299. https://doi.org/10.1016/j.chb.2016.11.048

37.

Hong

Park

H. S.

(2011). User ratings and willingness to express opinions online. International Journal of Marketing Studies, 3(2), 2–15. https://doi.org/10.5539/ijms.v3n2p2

38.

Hsueh

Yogeeswaran

Malinen

(2015). “Leave your comment below”: Can biased online comments influence our own prejudicial attitudes and behaviors? Human Communication Research, 41(4), 557–576. https://doi.org/10.1111/hcre.12059

39.

Huberman

B. A.

Romero

D. M.

(2009). Crowdsourcing, attention and productivity. Journal of Information Science, 35(6), 758–765. https://doi.org/10.1177/0165551509346786

40.

Ives

A. R.

(2015). For testing the significance of regression coefficients, go ahead and log-transform count data. Methods in Ecology and Evolution, 6(7), 828–835. https://doi.org/10.1111/2041-210X.12386

41.

Izuma

Saito

D. N.

Sadato

(2008). Processing of social and monetary rewards in the human striatum. Neuron, 58(2), 284–294. https://doi.org/10.1016/j.neuron.2008.03.020

42.

Jost

Ziegele

(2022). How to get on top—The effect of rationality and incivility of user comments on their visibility in political online discussions on Facebook. Communication Research Reports, 39(4), 224–235. https://doi.org/10.1080/08824096.2022.2120861

43.

Karapin

(1999). The politics of immigration control in Britain and Germany: Subnational politicians and social movements. Comparative Politics, 31(4), 423–444. https://doi.org/10.2307/422238

44.

Klein

Spears

Reicher

(2007). Social identity performance: Extending the strategic side of SIDE. Personality and Social Psychology Review, 11(1), 28–45. https://doi.org/10.1177/1088868306294588

45.

Knobloch-Westerwick

Sharma

Hansen

D. L.

Alter

(2005). Impact of popularity indications on readers’ selective exposure to online news. Journal of Broadcasting & Electronic Media, 49(3), 296–313. https://doi.org/10.1207/s15506878jobem4903_3

46.

Korostelina

(2014). Intergroup identity insults: A social identity theory perspective. Identity, 14(3), 214–229. https://doi.org/10.1080/15283488.2014.921170

47.

Ksiazek

T. B.

(2015). Civil interactivity: How news organizations’ commenting policies explain civility and hostility in user comments. Journal of Broadcasting & Electronic Media, 59(4), 556–573. https://doi.org/10.1080/08838151.2015.1093487

48.

Lee

E. J.

Jang

Y. J.

(2010). What do others’ reactions to news on Internet portal sites tell us? Effects of presentation format and readers’ need for cognition on reality perception. Communication Research, 37(6), 825–846. https://doi.org/10.1177/0093650210376189

49.

Lee

S. M.

Thomer

A. K.

Lampe

(2022). The use of negative interface cues to change perceptions of online retributive harassment [Paper presentation]. ACM on Human-Computer Interaction, New York, NY, United States.

50.

Levendusky

(2013). How partisan media polarize America. University of Chicago Press.

51.

Levine

J. M.

(1980). Reaction to opinion deviance in small groups. In Paulus (Ed.), Psychology of group influence (pp. 187–231). Erlbaum.

52.

Liang

K.-Y.

Zeger

S. L.

(1986). Longitudinal data analysis using generalized linear models. Biometrika, 73(1), 13–22. https://doi.org/10.1093/biomet/73.1.13

53.

Long

S. J.

Freese

(2006). Regression models for categorical dependent variables using Stata. Stata Press.

54.

Lutz

Schneider

F. M.

(2021). Is receiving dislikes in social media still better than being ignored? The effects of ostracism and rejection on need threat and coping responses online. Media Psychology, 24(6), 741–765. https://doi.org/10.1080/15213269.2020.1799409

55.

Mackie

D. M.

Devos

Smith

E. R.

(2000). Intergroup emotions: Explaining offensive action tendencies in an intergroup context. Journal of Personality and Social Psychology, 79(4), 602–616. https://doi.org/10.10371/0022-3514.79.4.602

56.

McDevitt

Kiousis

Wahl-Jorgensen

(2003). Spiral of moderation: Opinion expression in computer-mediated discussion. International Journal of Public Opinion Research, 15(4), 454–470.

57.

Molden

D. C.

Lucas

G. M.

Gardner

W. L.

Dean

Knowles

M. L.

(2009). Motivations for prevention or promotion following social exclusion: Being rejected versus being ignored. Journal of Personality and Social Psychology, 96(2), 415–431. https://doi.org/10.1037/a0012958

58.

Muchnik

Aral

Taylor

S. J.

(2013). Social influence bias: A randomized experiment. Science, 341(6146), 647–651. https://doi.org/10.1126/science.1240466

59.

Muddiman

Stroud

N. J.

(2017). News values, cognitive biases, and partisan incivility in comment sections. Journal of Communication, 67(4), 586–609. https://doi.org/10.1111/jcom.12312

60.

Munger

(2017). Tweetment effects on the tweeted: Experimentally reducing racist harassment. Political Behavior, 39, 629–649. https://doi.org/10.1007/s11109-016-9373-5

61.

Mutz

D. C.

(1994). The political effects of perceptions of mass opinion. Research in Micropolitics, 4, 143–167.

62.

Mutz

D. C.

(2006). Hearing the other side: Deliberative versus participatory democracy. Cambridge University Press.

63.

Naab

T. K.

Heinbach

Ziegele

Grasberger

M.-T.

(2020). Comments and credibility: How critical user comments decrease perceived news article credibility. Journalism Studies, 21(6), 783–801. https://doi.org/10.1080/1461670X.2020.1724181

64.

Nekmat

Gonzenbach

W. J.

(2013). Multiple opinion climates in online forums: Role of website source reference and within-forum opinion congruency. Journalism & Mass Communication Quarterly, 90(4), 736–756. https://doi.org/10.1177/1077699013503162

65.

Neuwirth

Frederick

Mayo

(2007). The spiral of silence and fear of isolation. Journal of Communication, 57(3), 450–468. https://doi.org/10.1111/j.1460-2466.2007.00352.x

66.

Newman

Fletcher

Schulz

Andi

Robertson

C. T.

Nielsen

R. K.

(2021). Reuters Institute digital news report 2021. https://reutersinstitute.politics.ox.ac.uk/digital-news-report/2021

67.

Noelle-Neumann

(1974). Spiral of silence: A theory of public opinion. Journal of Communication, 24(2), 43–51. https://doi.org/10.1111/j.1460-2466.1974.tb00367.x

68.

Noelle-Neumann

(1977). Turbulences in the climate of opinion: Methodological applications of the spiral of silence theory. Public Opinion Quarterly, 41(2), 143–158. https://doi.org/10.1086/268371

69.

Noelle-Neumann

(1993). The spiral of silence. Chicago University Press.

70.

Noelle-Neumann

(1995). Public opinion and rationality. In Glasser

T. L.

Salmon

C. T.

(Eds.), Public opinion and the communication of consent. Guilford Press.

71.

Okten

(2022). On commemorating Hrant Dink: Affective nationalism, hate speech, and digital news media users. Journalism & Mass Communication Quarterly. Advance online publication. https://doi.org/10.1177/107769902210939

72.

Oshagan

(1996). Reference group influence on opinion expression. International Journal of Public Opinion Research, 8(4), 335–354. https://doi.org/10.1093/ijpor/8.4.335

73.

Pan

(2001). Akaike’s information criterion in generalized estimating equations. Biometrics, 57(1), 120–125. https://doi.org/10.1111/j.0006-341X.2001.00120.x

74.

Pang

S. S.

Zhang

A. M.

J. S.

Low

Tan

K. S.

(2016). Can spiral of silence and civility predict click speech on Facebook? Computers in Human Behavior, 64, 898–905. https://doi.org/10.1016/j.chb.2016.07.066

75.

Peter

Rossmann

Keyling

(2014). Exemplification 2.0. Journal of Media Psychology: Theories, Methods, and Applications, 26(1), 19–28. https://doi.org/10.1027/1864-1105/a000103

76.

Porten-Cheé

Eilders

(2020). The effects of likes on public opinion perception and personal opinion. Communications, 45(2), 223–239. https://doi.org/10.1515/commun-2019-2030

77.

Postmes

Spears

(1998). Deindividuation and antinormative behavior: A meta-analysis. Psychological Bulletin, 123(3), 238–259.

78.

Price

Allen

(1990). Opinion spirals, silent and otherwise: Applying small-group research to public opinion phenomena. Communication Research, 17(3), 369–392. https://doi.org/10.1177/009365090017003005

79.

Rainie

Smith

(2012). Social networking sites and politics. https://www.pewresearch.org/internet/2012/03/12/social-networking-sites-and-politics/

80.

Rains

S. A.

Kenski

Coe

Harwood

(2017). Incivility and political identity on the Internet: Intergroup factors as predictors of incivility in discussions of news online. Journal of Computer-Mediated Communication, 22(4), 163–178. https://doi.org/10.1111/jcc4.12191

81.

Rau

P.-L. P.

Gao

Ding

(2008). Relationship between the level of intimacy and lurking in online social network services. Computers in Human Behavior, 24(6), 2757–2770. https://doi.org/10.1016/j.chb.2008.04.001

82.

Reader

(2012). Free press vs. free speech? The rhetoric of “civility” in regard to anonymous online comments. Journalism & Mass Communication Quarterly, 89(3), 495–513. https://doi.org/10.1177/1077699012447923

83.

Reich

Schneider

F. M.

Heling

(2018). Zero likes—Symbolic interactions and need satisfaction online. Computers in Human Behavior, 80, 97–102. https://doi.org/10.1016/j.chb.2017.10.043

84.

Reichelmann

Hawdon

Costello

Ryan

Blaya

Llorent

Oksanen

Räsänen

Zych

(2021). Hate knows no boundaries: Online hate in six nations. Deviant Behavior, 42(9), 1100–1111. https://doi.org/10.1080/01639625.2020.1722337

85.

Ruff

C. C.

Fehr

(2014). The neurobiology of rewards and values in social decision making. Nature Reviews Neuroscience, 15(8), 549–562.

86.

Sageman

(2011). Leaderless jihad: Terror networks in the twenty-first century. University of Pennsylvania Press.

87.

Sherrick

Hoewe

(2018). The effect of explicit online comment moderation on three spiral of silence outcomes. New Media & Society, 20(2), 453–474. https://doi.org/10.1177/1461444816662477

88.

Skinner

B. F.

(1953). Some contributions of an experimental analysis of behavior to psychology as a whole. American Psychologist, 8(2), 69–78. https://doi.org/10.1037/h0054118

89.

Smith

Morgan

Monks

(2017). Students’ perceptions of the effect of social media ostracism on wellbeing. Computers in Human Behavior, 68, 276–285. https://doi.org/10.1016/j.chb.2016.11.041

90.

Spears

Lea

Lee

(1990). De-individuation and group polarization in computer-mediated communication. British Journal of Social Psychology, 29(2), 121–134. https://doi.org/10.1111/j.2044-8309.1990.tb00893.x

91.

Stephens

Dowber

Barrie

Almeida

Atkins

(2023). Effect of swearing on strength: Disinhibition as a potential mediator. Quarterly Journal of Experimental Psychology, 76(2), 305–318. https://doi.org/10.1177/17470218221082657

92.

Stroud

N. J.

Scacco

J. M.

Muddiman

Curry

A. L.

(2015). Changing deliberative norms on news organizations’ Facebook sites. Journal of Computer-Mediated Communication, 20(2), 188–203. https://doi.org/10.1111/jcc4.12104

93.

Sun

Rau

P. P.-L.

(2014). Understanding lurkers in online communities: A literature review. Computers in Human Behavior, 38, 110–117. https://doi.org/10.1016/j.chb.2014.05.022

94.

Sunstein

C. R.

(2009). Going to extremes: How like minds unite and divide. Oxford University Press.

95.

Taylor

(2007). Disgust is a factor in extreme prejudice. British Journal of Social Psychology, 46(3), 597–617. https://doi.org/10.1348/014466606X156546

96.

Tormala

Z. L.

Rucker

D. D.

(2007). Attitude certainty: A review of past findings and emerging perspectives. Social and Personality Psychology Compass, 1(1), 469–492. https://doi.org/10.1111/j.1751-9004.2007.00025.x

97.

Valkenburg

P. M.

Peter

Schouten

A. P.

(2006). Friend networking sites and their relationship to adolescents’ well-being and social self-esteem. CyberPsychology & Behavior, 9(5), 584–590. https://doi.org/10.1089/cpb.2006.9.584

98.

Weaver

(1997). On-line democracy or on-line demagoguery? Public opinion “polls” on the Internet. Harvard International Journal of Press/Politics, 2(4), 71–86. https://doi.org/10.1177/1081180X97002004006

99.

Wu-Ouyang

(2025). Internet freedom and social media’s political consequences: Political nationalism and authoritarian orientation among six Asian societies. Journalism & Mass Communication Quarterly. Advance online publication. https://doi.org/10.1177/10776990241313183

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.32 MB