Evaluating the Adequacy of Emoji Use in Positive and Negative Messages from Close and Distant Senders

Abstract

Computer-mediated communication allows people to communicate across several contexts (e.g., friends, professional settings) by using video-based or text-based channels. In the latter case, communication lacks nonverbal cues (e.g., tone of voice) that are critical to message interpretation. Including emoji can help express emotion and reinforce or clarify the meaning of a message. However, the benefits of using emoji are likely to depend on the context (e.g., the relationship between interlocutors) and the messages' features (e.g., the valence of the message). To date, studies have not systematically and empirically examined how the use of emoji is perceived across different communication scenarios. In the current study, we asked 175 participants (49.5 percent women; M_age = 36.32, SD = 12.22) to imagine receiving either a negative or a positive message from 22 senders (e.g., friend; bank manager) and to indicate, for each case, how much they would like to receive an emoji and how useful and appropriate they considered the use of emoji. These ratings were combined into a single index of emoji use adequacy. Based on factor analysis, the 22 scenarios were aggregated in distant and close scenarios. Overall, results showed that participants considered emoji use more inadequate in distant (vs. close) scenarios and for negative (vs. positive) messages. These findings suggest that the perceived benefits of emoji use for text-based communication may not be generalized to all text-based communication contexts, such that relationship proximity and message valence should not be overlooked. Implications for the fields of consumer psychology and communication are discussed.

Introduction

Computer-Mediated Communication (CMC) allows asynchronous and fast communication, being a valuable tool to contact family, friends, work colleagues, but also service providers and brands.¹ Users often include emoji in CMC.² Emoji includes pictorial representations of different categories, including facial expressions,³ and may help in adding an emotional tone to text-based communication, softens the seriousness of messages, decreases their negativity, reduces discourse ambiguity, and strengthens senders' intent.^2,4,5 For example, research has shown that users include emoji to convey sarcasm, humor, and flirt with partners.⁶

Emoji use may not always be perceived as appropriate for all communicative situations (e.g., sending a “kissing-face” emoji in a message to a co-worker⁷). Valence, particularly, has been suggested as a potential moderator of emoji effects.^8,9 For example, participants reading a positive review of a hotel with (vs. without) emoji reported more positive attitudes toward the hotel, whereas the reverse was observed for a negative review.¹⁰

Moreover, the same emoji can positively impact messages addressing moderate conflicts while producing adverse effects for more severe conflicts.⁹ These differences arguably occur because emoji are perceived as fun and less serious,¹¹ and therefore inadequate to be used in more serious contexts. Indeed, people tend to use positive emoji more frequently,¹² especially when communicating positive information.¹³ Previous research also suggests that emoji are more frequently used with friends and romantic partners,⁶ which may mimic societal norms of emotional expression.¹⁴ However, emoji can be used beyond close social circles.

For instance, researchers^15,16 have argued that the communication between health care professionals and patients can be facilitated by using emoji. Research in educational settings showed that students evaluate an instructor as more caring but less competent when the instructor uses emoji.¹⁷ Likewise, research¹⁸ found that using emoji in work-related e-mails was seen as inappropriate, and senders were perceived as less competent. This goes in line with research suggesting that emoji use may be inappropriate in professional communication^5,7 and perceived as unprofessional.¹⁹

Brands also often rely on informal language (e.g., emoji) when communicating with consumers²⁰ to foster customer loyalty.²¹ Previous research²² showed that emoji use can positively affect consumers' perceptions of the brand, increase buying intention, and promote the experience of positive affect in consumers. Notably, other studies²³ also show that emoji use by brands may be inappropriate in some communication contexts.

In summary, the research presents mixed findings regarding emoji use across different communication contexts. To our knowledge, researchers have overlooked how varying contextual cues (e.g., the relationship between interlocutors) and messages' features (e.g., valence) impact the perceived adequacy of using emoji. An exception is a work by Völkel et al.,²⁴ who tested emoji use in different scenarios (varying in valence, situation, and interlocutor). However, this study focused on the senders' perspective and not on how a receiver perceives emoji as adequate or not in a given scenario.

To the best of our knowledge, our study is the first to systematically examine the perceived adequacy of emoji use across a comprehensive set of communication scenarios. These scenarios vary according to sender (e.g., receiving emoji from friends to share news) and valence (e.g., receiving emoji from friends sharing [good vs. bad] news). Because emoji are typically perceived as fun and informal,¹¹ their use may be deemed more adequate with close interlocutors. Lastly, previous studies have shown relevant individual differences in emoji use (e.g., women and younger people use emoji more often^11,25), shaping how people perceive and react to emoji. Hence, we will control for these variables in our analyses.

Methods

Participants and design

A sample of 175 Portuguese adults^a volunteered to participate in a web survey: 58.3 percent women, aged between 18 and 64 (M = 36.32, SD = 12.22), 79.3 percent had at least Bachelor's degree and were either workers (70.3 percent), students (20.6 percent), working students (5.1 percent), or unemployed (4 percent).

The design was a 22 (scenarios) × 2(valence of the message: positive, negative), with scenarios being a within-participants factor and valence of the message being a between-participants factor.

Materials

We developed scenarios with varied interlocutors (identified in previous research^3,17,22,26), in which emoji use was plausible and allowed messages to be framed positively and negatively. The positive and negative versions of the messages for the 22 scenarios are presented in Table 1.

Table 1.

Positive and Negative Version of the Emoji Use Scenarios

Sender	Valence
Sender	Positive	Negative
Service provider
Message with emoji from… a health care professional (e.g., doctor) communicating…	… positive information (e.g., favorable diagnosis)	… negative information (e.g., unfavorable diagnosis)
a personal accountant (e.g., bank, insurance) communicating…	(e.g., lower prices or increase of services offered)	(e.g., higher prices or decrease of services offered)
a telecommunications provider communicating…	(e.g., lower prices or increase of services offered)	(e.g., higher prices or decrease of services offered)
a gymnasium communicating…	(e.g., lower prices or increase of services offered)	(e.g., higher prices or decrease of services offered)
a public service provider (e.g., water, energy) communicating…	(e.g., lower prices or increase of services offered)	(e.g., higher prices or decrease of services offered)
Professional setting
Message with emoji from a potential employer to candidate communicating…	… positive information (e.g., refusing an interview; refusing to hire)	… negative information (e.g., scheduling interview; hiring proposition)
Message with emoji from a candidate to a potential employer communicating…	… positive information (e.g., accepting the offer)	… negative information (e.g., declining the offer)
Message with emoji from employees communicating to work colleagues…	…positive results (e.g., winning important client; approval of project)	…negative results (e.g., loss of an important client; refusal of project)
Message with emoji from employee communicating to supervisors…	…positive results (e.g., winning important client; approval of project)	…negative results (e.g., loss of an important client; refusal of project)
Message with emoji from supervisors communicating…	…positive results (e.g., increase of annual bonus)	…negative information (e.g., decrease of annual bonus)
Educational setting
Message with emoji from a professor communicating to student(s)…	… positive information (e.g., very high grades)	… negative information (e.g., very low grades)
Message with emoji from a student(s) communicating to a professor…	… positive information (e.g., very positive performance evaluation)	… negative information (e.g., very negative performance evaluation)
Message with emoji from a student(s) communicating to colleagues…	… positive information (e.g., good grade on group assignment)	… negative information (e.g., bad grade on group assignment)
Interpersonal setting
Message with emoji from friends communicating…	… good news (e.g., marriage, new job)	… bad news (e.g., divorce, unemployment)
Message with emoji from romantic partner communicating…	…happy news (e.g., scheduling vacation)	…unhappy news (e.g., canceling vacation)
Message with emoji from potential romantic partner communicating…	…positive information (e.g., acceptance of invitation to go on a date)	…negative information (e.g., refusal of the invitation to go on a date)
Message with emoji from family communicating…	…good news (e.g., marriage, new job)	…bad news (e.g., divorce, unemployment)
Brands/marketing
Message with emoji from company/brand (e.g., flyers, newsletters) communicating…	…positive information (e.g., disclosure of new product)	…negative information (e.g., product callback)
Message with emoji from company/brand replying…	…to positively comment on social media (e.g., amazing experience at restaurant/hotel)	…to negatively comment on social media (e.g., awful experience at restaurant/hotel)
Message with emoji from company/brand replying…	…to positively comment on private chat/e-mail (e.g., amazing experience at restaurant/hotel)	…to negatively comment on private chat/e-mail (e.g., awful experience at restaurant/hotel)
Message with emoji from company/brand…	…confirming customer request (e.g., acceptance of specific service)	…refusing customer request (e.g., cancel of specific service)
Message with emoji from company/brand…	…communicating success of operation (e.g., confirmation of online order)	…communicating failure of operation (e.g., cancelation of online order)

Procedure and measures

The study was conducted following the ethical guidelines of [Iscte-Instituto Universitário de Lisboa]. People were invited through social media platforms and e-mailing services to collaborate on a web survey regarding emoji use. Participants were informed about the general objectives and assured the confidentiality and voluntary nature of the study. After providing informed consent, participants were asked to indicate their opinions about emoji use across different contexts.

Participants were then randomly assigned to one of the valence conditions (positive vs. negative messages) and presented with the 22 scenarios in random order. Specifically, each participant was presented with either negative (e.g., a health professional [e.g., a doctor] communicating an unfavorable diagnosis) or positive messages (e.g., a health professional [e.g., a doctor] communicating a favorable diagnosis).

After each scenario, participants were asked to indicate how much they would like to receive an emoji in that scenario (1 = Dislike to 7 = Like), how appropriate was the use of emoji in that scenario (1 = Not appropriate at all to 7 = Very appropriate), and how useful was the inclusion of an emoji in that scenario (1 = Useless to 7 = Useful). These variables were averaged into a single score of emoji use adequacy (α = 0.92), with higher scores indicating perceptions of greater adequacy of emoji use.

After evaluating the 22 scenarios, participants were asked to indicate how frequently they use emoji in their daily text-based interactions (1 = Rarely to 7 = Frequently). Finally, participants answered demographic questions (gender, age, education level, and employment status) and they were thanked and debriefed.

Results

Preliminary analyses: frequency of emoji use, gender, and age

Participants reported using emoji frequently in their daily text-based interactions (M = 4.71, SD = 2.10), 95 percent confidence interval (CI) [4.39–5.02]. Women (M = 5.09, SD = 2.03) reported using emoji more often than men (M = 4.18, SD = 2.09), t(172) = −2.89, p = 0.004. We found a negative correlation between age and frequency of using emoji, r = −0.398, p < 0.001, with older individuals reporting using emoji less often.

Principal components analysis

We conducted a principal components analysis (Promax rotation) with the 22 scenarios regardless of the valence of the message (Table 2). Based on the eigenvalues, we extracted two components that explained 71.3 percent of the variance (Kaiser-Meyer-Olkin = 0.96), presenting high internal consistency (Table 2). The first component was designated “distant scenarios” and the second “close scenarios.”

Table 2.

Principal Component Analysis

	C1	C2	Corrected item-total correlations
Items
S5: Public service provider (e.g., water, energy) to customer	0.97	−0.18	0.74
S2: Personal accountant (e.g., bank, insurance company) to customer	0.95	−0.11	0.78
S3: Telecommunications provider to customer	0.92	−0.09	0.77
S7: Candidate to potential employer	0.90	−0.18	0.67
S6: Potential employer to candidate	0.89	−0.03	0.78
S21: Brand/company answering customer's request	0.85	−0.01	0.77
S11: Employee to supervisors	0.85	−0.04	0.75
S9: Student to professor	0.82	−0.04	0.72
S1: Health care professional to patient	0.79	0.01	0.72
S13: Supervisor to employee	0.76	0.14	0.80
S8: Professor to student	0.71	0.19	0.79
S22: Brand/company communicating about operation (e.g., online order)	0.69	0.15	0.74
S18: Brand/company communicating (e.g., flyers, newsletters)	0.69	0.14	0.73
S4: Gymnasium communicating to customer	0.68	0.21	0.78
S20: Brand/company replying to customer on private chat/e-mail	0.64	0.26	0.78
S19: Brand/company replying to customer's social media comment	0.57	0.29	0.74
S15: Romantic partner	−0.10	0.98	0.66
S16: Potential romantic partner	−0.06	0.96	0.67
S14: Friend	−0.02	0.93	0.69
S17: Family member	−0.05	0.93	0.66
S10: Student to colleague	−0.01	0.84	0.62
S12: Employee to work colleague	0.21	0.68	0.70
Eigenvalue	12.76	2.92
Explained variance	58.0	13.3
Cronbach's α	0.97	0.95

Note: Saturation values above 0.40 are indicated in boldface.

One-sample t-tests against the scale midpoint (i.e., 4) showed that participants rated the adequacy of emoji use for distant scenarios below the scale midpoint (M = 2.37, SD = 1.33), t(174) = −16.22, p < 0.001, whereas for close scenarios the rating were above this point (M = 4.47, SD = 1.73), t(174) = 3.60, p < 0.001.

Adequacy of emoji use

We computed a 2 (valence of the message: positive vs. negative) × 2 (gender: women vs. men) × 2 (type of scenario: close vs. distant) mixed analysis of variance. The latter variable was entered in the model as repeated measures, and age and frequency of using emoji as co-variates. We found a main effect of the scenario, F(1,157) = 12.29, p = 0.001, η_p² = 0.07, such that participants perceived greater adequacy of emoji use in close (M = 4.47, SD = 1.74) than in distant scenarios (M = 2.36, SD = 1.34).

We also found a main effect of valence, F(1,157) = 99.86, p < 0.001, η_p² = 0.39, such that participants perceived greater adequacy of emoji use when the message was positive (M = 4.42, SE = 0.130) than negative (M = 2.71, SE = 0.110). The interaction between valence and type of scenario was also significant, F(1,157) = 12.98, p < 0.001, η_p² = 0.08. Planned contrasts showed that although the impact of distance for negative messages was significant, t(99) = 13.13, p < 0.001, it was stronger for the positive messages, t(74) = 14.56, p < 0.001 (Fig. 1).

FIG. 1.

Relationship between valence and scenario on the perceived adequacy of using emoji. Error bars indicate standard deviations.

Finally, we found a main effect of frequency of emoji use, F(1,157) = 4.14, p = 0.043, such that participants who use emoji more frequently evaluated overall emoji use as more adequate. There were no main effects of gender or age on emoji use perceived adequacy, ps ≥ 0.476, nor any interactions with valence or scenario, ps ≥ 0.060.

Supplementary analyses: rating norms

Data were coded and analyzed according to each of the 22 scenarios with negative messages and the 22 scenarios with positive messages. We calculated means, standard deviations, standard errors, and CIs for each evaluative dimension (i.e., liking, appropriateness, and usefulness) and for the overall adequacy index. Scenarios were categorized as “low adequacy” when the upper bound of the CI was below the scale midpoint (n_negative = 20; n_positive = 12); “moderate adequacy” when the CI included the response scale midpoint (n_negative = 2; n_positive = 4); and “high adequacy” when the lower bound of the CI was above the scale midpoint (n_negative = 0; n_positive = 6). These ratings are available as supplementary data at Open Science Framework (https://osf.io/w49a5/?view_only=57650f5f41db4f8b81c36f497eb89802).

Discussion

This experimental study tested the effect of the type of scenario (distant or close) and the valence of the messages (negative vs. positive) on the perceived adequacy of emoji use. Results showed that emoji use was considered more adequate when communicating positive information and/or with closer interlocutors (e.g., friends vs. work supervisor). Noteworthy, participants rated emoji use in all distant scenarios as inadequate. Our findings align with past research showing that using emoji may be seen as more adequate in some situations than others (e.g., close interpersonal vs. professional settings^5,7,18).

This may be one of the reasons why people use emoji more frequently with their close ones (e.g., friends⁶). Moreover, similarly to other studies,^8,9 we found that the valence of a message determines how people evaluate the adequacy of emoji use. For example, Rodrigues et al.⁹ also found that emoji use was perceived less favorably for communication outcomes in negative (vs. positive) messages, particularly when addressing more (vs. less) severe conflicts. Our study did not account for within-valence differences (e.g., messages about “canceling a vacation” may not be as negative as messages sharing the news about a “divorce”), and future studies should seek to assess whether the intensity of different positive and negative scenarios modulates emoji use evaluations.

The scenarios used in our study included two relevant cues: valence and sender. Still, overall, the situation description was quite abstract (e.g., “sharing good/bad news”), and messages did not include actual emoji. Therefore, we did not control for the way participants construed each specific situation (e.g., expectations of emoji use according to sender⁸) nor the emoji they envisioned, which raises concerns for the generalizability of our findings. For example, identifying a specific emoji may be important as they vary in several characteristics, namely valence, familiarity, and even attributed meaning.³

Even controlling for this, emoji may interact with other contextual cues. For instance, both the “smiling face” and “red heart” emoji are highly positive and familiar but using the latter in communication with co-workers may be perceived as inappropriate.⁷ Future experimental studies could use more ecological scenarios (e.g., simulated text messages⁹) to extend our current findings and further explore whether participants would include emoji in their CMC with different interlocutors.²⁴

Besides closeness, patterns in CMC may depend on other features of the relationship between interlocutors (e.g., communicating with a professor/supervisor may be more frequent than communicating with other distant interlocutors such as a bank account manager). Therefore, future studies should also assess the frequency or likelihood of contact with each sender.

Moreover, in line with previous studies,¹¹ women and younger participants reported using emoji more often. However, frequency of using emoji, age, and gender did not moderate the evaluation of emoji use in the set of scenarios presented, suggesting that these individual differences may be secondary when contextual cues are available.

This study is one of the first to systematically evaluate how emoji use is perceived across communication scenarios. Our main contribution is the demonstration that contextual cues related to the valence of the message and the relationship between interlocutors influence how individuals perceive the adequacy of emoji use. By providing a scenario-level analysis, we also contribute to the field, allowing researchers to select specific scenarios in which emoji use is deemed more/less adequate.

Our findings can also have practical implications by helping professionals in different fields (e.g., marketing, health communication, education) to efficiently integrate emoji in their communication strategies.²⁶ For example, receivers may be more open to emoji use whenever the content of the message is positive. However, in negative messages, we recommend restricting emoji use to communication between closer interlocutors (e.g., a professor could only include an emoji in an e-mail sharing low grades after building rapport with students).

Footnotes

Authors' Contributions

B.P.C., M.P., D.L.R., D.L., and M.V.G. conceptualized and designed the study. B.P.C. and M.P. collected the data. Analysis and interpretation of data was carried out by B.P.C., M.P., D.L.R., and D.L. Drafting of the article was done by B.P.C., M.P., and D.L.R. Finally, all authors revised the article and gave the final approval.

Author Disclosure Statement

No competing financial interests exist.

Funding Information

Part of this work was funded by a grant from Fundação para a Ciência e a Tecnologia awarded to B.P.C. (Ref.: FRH/BD/146388/2019) and to D.L.R. (Ref.: 2020.00523.CEECIND).

Note

a. A sample of 174 of participants was determined by an a priori power analysis (G*Power; Faul et al., 2007)²⁷, using as reference a medium effect size (η_p² = 0.06) and a power 1-β = 0.85 to detect the interaction between valence (negative vs. positive) and measurement (liking, appropriateness, usefulness; within participants).

References

Hayes

, Britt

, Applequist

, et al. Leveraging textual paralanguage and consumer–brand relationships for more relatable online brand communication: a social presence approach. Journal of Interactive Advertising, 2019; 20:17–30.

Tang

, Hew

. Emoticon, emoji, and sticker use in computer-mediated communication: a review of theories and research findings. International Journal of Communication, 2019; 13:27.

Rodrigues

, Prada

, Gaspar

, et al. Lisbon Emoji and Emoticon Database (LEED): norms for emoji and emoticons in seven evaluative dimensions. Behavior Research Methods, 2018; 50:392–405.

Sampietro

. Emoji and rapport management in Spanish WhatsApp chats. Journal of Pragmatics, 2019; 143:109–120.

Kaye

, Wall

, Malone

. “Turn that frown upside-down”: a contextual account of emoticon usage on different virtual platforms. Computers in Human Behavior, 2016; 60:463–467.

Thomson

, Kluftinger

, Wentland

. Are you fluent in sexual emoji?: exploring the use of emoji in romantic and sexual contexts. The Canadian Journal of Human Sexuality, 2018; 27:226–234.

Butterworth

, Giuliano

, White

, et al. Sender gender influences emoji interpretation in text messages. Frontiers in Psychology, 2019; 10:1–5.

Coyle

, Carmichael

. Perceived responsiveness in text messaging: the role of emoji use. Computers in Human Behavior, 2019; 99:181–189.

Rodrigues

, Lopes

, Prada

, et al. A frown emoji can be worth a thousand words: perceptions of emoji use in text messages exchanged between romantic partners. Telematics and Informatics, 2017; 34:1532–1543.

10.

Manganari

, Dimara

. Enhancing the impact of online hotel reviews through the use of emoticons. Behaviour and Information Technology, 2017; 36:674–686.

11.

Prada

, Rodrigues

, Garrido

, et al. Motives, frequency and attitudes toward emoji and emoticon use. Telematics and Informatics, 2018; 35:1925–1934.

12.

Novak

, Smailović J, Sluban B, et al. Sentiment of emojis. PLoS One, 2015; 10:e0144296.

13.

Chen

, Siu

KWM

. Exploring user behaviour of emoticon use among chinese youth. Behaviour and Information Technology, 2016; 36:637–649.

14.

Dunlap

, Bose

, Lowenthal

, et al. (2016) What sunshine is to flowers: a literature review on the use of emoticons to support online learning. In Tettegah SY, Gartmeier M, eds. Emotions, technology, design, and learning. Elsevier Academic Press, pp. 163–182.

15.

Donovan

. Mood, emotions and emojis: conversations about health with young people. Mental Health Practice, 2016; 20:23–26.

16.

Skiba

. Face with tears of joy is word of the year: are emoji a sign of things to come in health care?. Nursing education perspectives, 2016; 37:56–57.

17.

Vareberg

, Westerman

. To: −) or to ☺, that is the question: a study of students' initial impressions of instructors' paralinguistic cues. Education and Information Technologies, 2020; 25:4501–4516.

18.

Glikson

, Cheshin

, van Kleef

. The dark side of a smiley: effects of smiling emoticons on virtual first impressions. Social Psychological and Personality Science, 2017; 9:614–625.

19.

Tigwell

, Flatla

. (2016) Oh that's what you meant!: reducing emoji misunderstanding. In Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct. Association for Computing Machinery, New York, NY, USA, pp. 859–866.

20.

Luangrath

, Peck

, Barger

. Textual paralanguage and its implications for marketing communications. Journal of Consumer Psychology, 2017; 27:98–107.

21.

Kwon

, Sung

. Follow Me! Global Marketers' Twitter Use. Journal of Interactive Advertising, 2011; 12:4–16.

22.

Das

, Wiener

HJD

, Kareklas

. To emoji or not to emoji?: examining the influence of emoji on consumer reactions to advertising. Journal of Business Research, 2019; 96:147–156.

23.

Li X (Shirley), Chan

KWA

, Kim

. Service with emoticons: how customers interpret employee use of emoticons in online service encounters. Journal of Consumer Research, 2018; 45:973–987.

24.

Völkel

, Buschek

, Pranjic

, et al. (2019) Understanding emoji interpretation through user personality and message context. In Proceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI '19). Association for Computing Machinery, New York, NY, USA, pp. 1–12.

25.

Oleszkiewicz

, Karwowski

, Pisanski

, et al. Who uses emoticons?: data from 86 702 Facebook users. Personality and Individual Differences, 2017; 119:289–295.

26.

Bai

, Dan

, Mu

, et al. A systematic review of emoji: current research and future perspectives. Frontiers in Psychology, 2019; 10:2221.

27.

Faul

, Erdfelder

, Lang

, Buchner

. G* Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior research methods, 2007; 39:175–191.