The perception of humanness in conversational journalism: An algorithmic information-processing perspective

Abstract

How much do anthropomorphisms influence the perception of users about whether they are conversing with a human or an algorithm in a chatbot environment? We develop a cognitive model using the constructs of anthropomorphism and explainability to explain user experiences with conversational journalism (CJ) in the context of chatbot news. We examine how users perceive anthropomorphic and explanatory cues, and how these stimuli influence user perception of and attitudes toward CJ. Anthropomorphic explanations of why and how certain items are recommended afford users a sense of humanness, which then affects trust and emotional assurance. Perceived humanness triggers a two-step flow of interaction by defining the baseline to make a judgment about the qualities of CJ and by affording the capacity to interact with chatbots concerning their intention to interact with chatbots. We develop practical implications relevant to chatbots and ascertain the significance of humanness as a social cue in CJ. We offer a theoretical lens through which to characterize humanness as a key mechanism of human–artificial intelligence (AI) interaction, of which the eventual goal is humans perceive AI as human beings. Our results help to better understand human–chatbot interaction in CJ by illustrating how humans interact with chatbots and explaining why humans accept the way of CJ.

Keywords

algorithmic information processing anthropomorphized chatbots conversational journalism explanatory cues perceived humanness social cues in AI two-step flow

Journalism and the way news are delivered are transforming from a lecture-based model to a conversation. Journalism-as-a-conversation, or conversational journalism (CJ), has become an ever-greater focus of attention due to recent progress in machine learning and artificial intelligence (AI; Jones and Jones, 2019). CJ delivers news through interactive chatbot conversations. The use of chatbots in journalism is increasing due to the development of increasingly innovative algorithmic tools for newsgathering, reporting, and dissemination (Ford and Hutchinson, 2019). Recently, AI-driven chatbots have established themselves in news media by offering useful alternatives for journalistic practices and news services (BBC News Labs, 2019; Wölker and Powell, 2018).

However, a number of obstacles remain regarding the widespread use and adoption of chatbots in journalism (Dörr and Hollnbuchner, 2017). While AIs can offer relevant suggestions, predictive content, and personalized news, journalists must be thoughtful and deliberate about preventing bias and harmful behaviors affecting automated processes (Guzman and Lewis, 2020). Such concerns are inherently ethical in that they deal with fairness, accountability, transparency, and explainability (FATE), which are even more critical in CJ than in traditional journalism (Graefe et al., 2018). Issues associated with FATE are inseparably connected to algorithmic performance (Helberger, 2019; Shin, 2020b). Questions about how to uphold best practices and the underlying processes of CJ (Diakopoulos and Koliska, 2016), about who should be held liable for harm caused by AI (Shin and Park, 2019), and whether AI is working in ways that humans consider are right and just remain unresolved and continue to pose a major challenge in CJ (Crain, 2018). These issues, including ethical concerns involving how one tackles and responds to these challenges, are pivotal to CJ sustainability and development (Dörr and Hollnbuchner, 2017; Park and Skoric, 2017).

Recently, these issues have been approached in terms of the opacity of black-box algorithm processes and how to give reasonable explanations of why they are receiving certain recommendations to users (Gunning et al., 2019). Shin (2021b) argued that explainability and anthropomorphism together play significant roles in personalized AI systems, proposing that explainability influences user trust and attitudes toward AI in news recommendation systems. Shin (2021b) further analyzed the effects of functional and anthropomorphic chatbot features on user experiences, finding that how users understand AI functions and perceive AI-based interactions are key factors to focus upon (Lewis et al., 2019). These issues will become even more vital in personalized CJ (Marchionni, 2013), because accuracy, fairness, and impartiality are considered critical journalistic values (Graefe et al., 2018; Tandoc et al., 2020). Due to increasing trends of algorithm-driven news consumption, growing attention is being paid to explanations of how such news works and why certain results are generated for users. The right to such explanations is derived from safeguards against automated decision-making (Shin, 2021), which is an integral part of CJ. Such transparency is a remedy to algorithmic harm, particularly against the backdrop of decreasing credibility in journalism.

Despite their widespread use of AIs, little is known about the roles and the impacts of anthropomorphism and explainability on the process by which users perceive and make sense of trust in algorithm-based journalisms (Shin, 2021a). Furthermore, there is a lack of research on the roles of anthropomorphic frames and the response from users (perceived humanness) in the CJ context (Verhagen et al., 2014). In light of this gap in the literature, we address the effects of anthropomorphism and explainability by highlighting the following research questions: How do explanatory and anthropomorphic features in CJ influence user perception of humanness in chatbots? Specifically, how does explainability affect user heuristics and quality evaluations of personalized and interactive CJ services? How do users perceive anthropomorphism in the context of CJ, and how does that perception influence user trust in and acceptance of algorithms? We intend to conceptualize humanness in chatbots by clarifying the role and function of anthropomorphic explainability in CJ.

Our findings reveal the heuristic role of anthropomorphic explainability in users’ two-step flow of interaction with CJ (Soffer, 2019) that algorithmic information flow from AI to users’ evaluative framework of non-functional qualities first, and then from there to functional qualities of algorithms. Trust intervenes between the two steps by linking the anthropomorphism and explainability to the quality evaluation. Users use anthropomorphic explainability as an information shortcut to judge the trust and quality of CJ and define their further behaviors. A peripheral-central route is used to make sense of algorithmic attributes and decide how and whether to continue to use chatbots to consume news media. Explainability facilitates users to understand non-functional algorithmic qualities, whereas anthropomorphic cues guide users to evaluate functional algorithmic performance as well as facilitate explanations. When users interact with chatbots, they make judgments as to what extent people perceive chatbots as humans not robots (Westernman et al., 2019), which is also related as to whether, how, and to what extent to believe algorithm-based news (Kemper and Kolkman, 2019; Thurman et al., 2019). Such decisions are based on heuristic user assessments of normative values, including fairness, transparency, and accountability (FAT), which are triggered by explainability (E; Shin, 2021b). In the process, anthropomorphic explanations serve as heuristic cues, inspiring user perception of humanness (Westernman et al., 2019), which then influences trust (Sah and Peng, 2015). As an extension of previous studies (e.g. Shin et al., 2020), which outlined the liaison role of trust in the algorithm processing, we advance the finding by linking the trust with perceived humanness in CJ. Humans trust AI more when they see AI as human-like entities. Perceived humanness leads users to allow a higher likelihood of sharing personal information with chatbots, which produce higher performance of CJ.

The relational implications of humanness and trust provide meaningful directions for theoretical formulations and practical considerations. Theoretically, explicating the role of anthropomorphic explainability in CJ makes worthwhile contributions to the ongoing debate and development of human–AI interactions (HAII; Sundar, 2020). This research direction is essential to make AIs that genuinely interact with humans, which is the ultimate goal of HAII; humans should perceive AIs as humans (Shin, 2021). Our findings contribute to the area of humanizing AI (Go and Sundar, 2019) by clarifying how humanness concepts are epistemologically realized, illustrating how they can be applied along with anthropomorphism, and exemplifying how the effects of anthropomorphic explainability are measured along with perceived humanness. As the AI industry has faced growing challenges in the context of transparency and skepticism over the trust, the roles of explainability and cognate issues afford insights into what users need to do with future interaction with the system. This study extends previous literature on anthropomorphism (e.g. Zarouali et al., 2020), humanness (Westernman et al., 2019), and two-step flow of communication (Soffer, 2019) by revealing a relevant mechanism by which anthropomorphic cues influence algorithmic contexts. Practically, the heuristic roles of explainability and anthropomorphism in AIs lend strategic direction that can inform how to theorize about and develop human-centered CJ and to enable algorithm adoption in mainstream journalism practices.

Literature review

CJ and anthropomorphism: how AI can realize the promise of CJ

Chatbots are AI-based programs that simulate human conversations to facilitate interactions between users and algorithms via texts or voice commands (Westernman et al., 2019). Recently, chatbot algorithms have become widely used in the news sector to help users find stories, to mine data, to transcribe interviews, to fact check, and to spot fake news and disinformation in articles (Jones and Jones, 2019). Chatbots make possible a new frontier of CJ, comprising conversational formats such as Quartz Bot (Lewis et al., 2019). Quartz and BBC use chatbots to send readers personalized news recommendations through mobile apps or Facebook Messenger (BBC News Labs, 2019). With the advancement of AI technologies, users can use chatbots to inquire about the latest news or information, and the agent can then reply appropriately. Chatbots can assist reporters by communicating news interactively or collect data from users (Go and Sundar, 2019). The systems can filter news and provide items that best serve user needs through personalization, so readers are served with news they are likely to be interested in (Rai, 2020; Soffer, 2019).

Recently, the concept of anthropomorphic conversational agents has emerged in AI (Verhagen et al., 2014; Zarouali et al., 2020). Anthropomorphism is defined as the individual tendency to perceive a nonhuman object as if it was human, which can be observed in individual reactions to an object (Kim and Sundar, 2012). The humanization of conversational agents has fulfilled practical needs in the industry. The progress toward making chatbots appear human-like is slow, but one current approach is to include social cues in conversations, such as making chatbots include emojis in conversations. Sah and Peng (2015) argue that human cues trigger responses like those of a real human regarding how people perceive social characteristics. Similarly, the inclusion of human-like cues on websites increases social presence (Kim and Sundar, 2012) and trust (De Visser et al., 2016).

The people’s anthropomorphic attitudes and the effects have been extensively discussed in the literature. However, how anthropomorphism permeates AI news processing precisely, and what the psychological consequences of this might have in the context of journalism received less attention. The question remains as to how the human tendency to anthropomorphize chatbots affects how we perceive and interact with them and whether anthropomorphic cues would enhance the perception of humanness and trust in CJ.

Right to explanations and explainable journalism

Conventional journalists hold the exclusive rights to decide what news is presented and to create stories and write articles in ways that affect reader understanding. There has been little opportunity for users to request explanations other than the Right of Reply. This lack of explanations raises both practical and ethical questions, particularly with the development of AIs. As journalism continues to embrace AI, explainability becomes important in CJ in which decisions are based on user input data (Shin, 2021). Explainability is paramount, as the primary goal in CJ is to establish credibility by explaining how and why journalistic practices work. Explainability has been interpreted in the context of the Right of Explanation established by the General Data Protection Regulation (GDPR). The legal notion of seeking meaningful explanations of the logic involved in automated decision-making is akin to the technical representation of explainability in AI and CJ. Explainability in journalism can be defined as the extent to which a user can understand how and why produce particular results (Shin, 2020a).

News readers, as a matter of curiosity, may wonder about the reasons why and how automated news is generated and distributed (Gunning et al., 2019). As the complexity of algorithms and AI systems drastically increase, readers increasingly consider them to be black boxes since specialized expertise and knowledge are needed to understand AI decisions and performance (Tandoc et al., 2020). These issues are significantly worsened when users are laypeople who lack technical knowledge but interact with AI systems. Explainability is pivotal to establishing faith, credibility, and rapport between an AI agent and users, especially when it comes to assessing fake news and misinformation (Helberger, 2019). Explainability in CJ offers users’ assurance and faith that chatbot systems work properly, helps designers understand why a system works in a particular way, and safeguards against partiality and bias.

Elaboration-likelihood model: algorithmic information processing

With the emergence of CJ services, how users interact with chatbots has become a logical topic of study in algorithm design and development. User heuristics concerning algorithmic qualities include questions like: How do users determine the qualities and attributes of CJ? How do people sense them and with what meanings? Since algorithm-based content entails numerous benefits and advantages, it is key to examine users’ a priori expectations and how these expectations are met. The elaboration-likelihood model (ELM) is a useful theoretical lens for this inquiry because the model posits that attitude and behavior changes occur through different routes of processing (Petty and Cacioppo, 1986). Using ELM, we can trace how users process the algorithmic information they receive, beyond merely responding to stimuli (anthropomorphism/explanations). We seek to explain how humans process different stimuli and the influences of these processes on attitudes and behavior. ELM is utilized as a theoretical basis to examine user sensemaking in CJ by explicating the role that algorithmic features play in shaping user perception of humanness and sensemaking of algorithms, as well as how user actions affect their sensemaking. ELM theory is suitable for this task because users may use a simplified form of quality judgment for FAT, as it is a highly hypothetical concept. When users make heuristic evaluations and thus establish trust, they then use systematic processing whereby they think carefully about chatbot functions to determine whether chatbots are personalized or accurate. User attitudes are then based on their conclusions drawn from the careful central evaluation (Figure 1).

Figure 1.

Conceptual frame.

A cognitive process of CJ

Our model is designed to explore the effects of anthropomorphic explainability on perceived humanness in chatbot news systems. Explainability is hypothesized to function as an antecedent of FAT, and anthropomorphic features are proposed as moderating effects.

FAT of conversational agents: are algorithms working ethically?

User perception of CJ content often depends on their understanding of how the algorithm works (Shin et al., 2020). How and why certain news items are generated and how user input contributes to the results are confirmed to be important to users (Rai, 2020). Transparency and visibility of relevant feedback enhance search performance and efficiency in recommendation systems. Providing explanations in recommendation systems can enhance positive attitudes and subsequent behaviors among users. Related studies have suggested a causal link between explainability and assurance in the context of algorithm journalism (e.g. Shin, 2021). Based on ongoing research, CJ should assist users in understanding the algorithmic process and thus increase user confidence. People are more likely to trust and accept explainable systems when they aware of how data are collected, analyzed, and presented (Shin, 2020a). When the mechanism is transparent, users are willing to increase their data input to advance recommendation outputs. CJ users are also able to understand the process of a recommendation system (Rai, 2020). Accountability in the CJ context refers that firms should be held responsible for the results of their programmed algorithms. As the product of humans, CJ can have issues resulting from human bias or simple oversight. In this light, explainability can be hypothesized as follows:

H1. Explainability positively influences user perception of CJ transparency.

H2. Explainability positively influences user perception of CJ fairness.

H3. Explainability positively influences user perception of CJ accountability.

Normative belief and trust: how do algorithms work?

Trust is a critical factor to consider when designing AI services in journalism (Tandoc et al., 2020). Trust plays a role across the range of human perception, behavior, and assessment of technology and can determine the way people interact with AI (Shin, 2020a). In CJ, trust is considered one of the central questions concerning transparency in journalism. In the news recommendation context, trust represents how much readers believe the news and news sources (credibility) and reflects the belief in the accuracy of news filtering and user willingness to accept the recommender system’s capacity. There are two types of trust in AI research: human-like trust constructs (e.g. emotion, satisfaction, and normative values) and system-like trust constructs (e.g. usefulness, personalization, and functionality). In this study, trust is proposed as a mediator between human-like and system-like construct (non-functional vs functional quality).

First, trust is related to non-functional qualities such as transparency, fairness, and accountability. People tend to accept trustworthy systems because they know how data are collected, processed, and how recommendations are produced. When there is a fair process, users increase their input data to improve recommendation outputs (Renijith et al., 2020). If users have notions of accountability including that CJ is supposed to operate in a manner that contributes to the public good, they feel reassured and trust is established (Diakopoulos and Koliska, 2016). Collectively, the three factors are considered paramount in the development of algorithm journalism as they help establish user trust in AIs (Dörr and Hollnbuchner, 2017). Previous research extensively supports the hypothesis that trust dimensions determine user decisions whether or not to engage with technology, but few studies have examined algorithms, particularly in the context of CJ news.

H4. Perceived transparency positively influences user trust in CJ.

H5. Perceived fairness positively influences user trust in CJ.

H6. Perceived accountability positively influences user trust in CJ.

Personalization and accuracy: does the news really reflect user needs?

Second, trust is related to functional qualities such as accuracy and personalization, both of which are central features of CJ systems (Shin et al., 2020; Soffer, 2019). Accuracy in the CJ context is defined as the degree to which the result of a recommended information conforms to the user value or user needs. Simply accuracy is about the algorithm returns relevant results and news. Chatbot users expect AI to distinguish real sources of news content from artificially generated sources. Since fake news and misinformation are prevalent threats, chatbots can aid news agencies in identifying and eliminating fake news.

An accurately personalized news service acts as an information gatekeeping filter because machine learning can predict user preferences and expectations based on user profile or browsing history (Rhee and Choi, 2020). When users perceive that news generation is individually tailored to their interests, they consider the service to be valuable and are willing to further engage with chatbots to improve algorithmic performance (Shin, 2021a). As users continue to provide more data to chatbots, CJ becomes ever more capable of delivering accurate and personalized content. Accurately personalized news recommendations keep users engaged. Recent studies have confirmed the existence of this circular relationship in a diverse sample of algorithm services (Kim & Sundar, 2012; Shin, 2020). It can be hypothesized that accuracy and personalization are greatly influenced by trust (Soffer, 2019).

H7. Trust significantly influences user perception of CJ accuracy.

H8. Trust significantly influences user perception of CJ personalization.

Emotion and AI: what do users really feel?

In recent HAII, efforts have been made to understand the role of emotion in AI, focusing on contexts such as emotional AI (Lee, 2018) and affective AI. Using chatbots, marketers provide emojis and emoticons to express a range of emotions while messaging. Previous studies reported that emotions affect user interactions with AI (Shin, 2020b). The goal of HAII is to ensure that users are not merely satisfied but are actively pleased to use AI and associate AI with positive emotional valence (Turing, 1950). Improving the ability to recommend news that adapts to user emotions is therefore important for the future of CJ. Strong positive emotions derived from assessment processes reassures users and gives them feelings of satisfaction and intent to use the algorithm.

It has been found that emotions in CJ users play key roles in interactions with algorithms (Araujo, 2018), particularly regarding the acceptance of highly complicated AI (Shin and Park, 2019). Emotion analyses of chatbots have revealed a connection between the deployment of anthropomorphic chatbots and users reporting more positive emotions toward the agent, compared with users who did not anthropomorphize the device. Prior findings, therefore, provide empirical evidence that anthropomorphism is an antecedent of emotional interactions with chatbots (Go and Sundar, 2019; Sah and Peng, 2015). When interacting with AI, multiple emotions make up attitudes, as attitude has a significant effect on human emotions. When users confirm the value of a system, their emotion toward the AI algorithm becomes more positive.

H9. Perceived accuracy significantly influences user emotions toward CJ.

H10. Perceived personalization significantly influences user emotions toward CJ.

Perception of humanness

Westernman et al. (2019) propose that users’ acceptance of chatbots depends on perceived humanness, which is defined as the degree to which users believe that chatbots might be human. Studies have suggested that human-like attributes positively influence the probability of speech-based chatbots (De Visser et al., 2016). Araujo (2018) argues that users have positive feelings if they perceive high humanness. Similarly, Shin (2021b) shows that a human-like chatbot with an explanation and humanized communication skills can increase user trust, which in turn leads to an improved user experience. Perceived humanness can increase user trust and assured feeling and improve user’s experience using chatbots (Westernman et al., 2019).

H11. Perceived humanness positively influences the user trust in CJ.

H12. Perceived humanness is positively influenced by explainability of CJ.

H13. Perceived humanness positively influences the user emotion of CJ.

Moderating effect of anthropomorphism

AI has been conceptualized and studied in anthropomorphic dimensions. The goal of anthropomorphism in AI is to imbue chatbots with social presence and human emotions that are sufficiently credible for human users to engage with in meaningful and potentially sustainable ways CJ. Examining the moderating effects of anthropomorphism produces critical insights into the user tendency to apply social attributes to AIs (Sah and Peng, 2015). In line with the social presence concept (Shin and Biocca, 2018), the relative influence of an agent’s characteristics on its ability to promote a personal relationship with users should be stronger when the agent is anthropomorphized. Previous studies confirmed that anthropomorphized agents with higher levels of realism induce more meaningful social interaction, especially when heuristic processes are used (Araujo, 2018). Hence, by utilizing affectionate human characteristics, the agent telegraphs the capabilities of chatbot interactions, which in turn enhances feelings of social interaction and trust (De Visser et al., 2016).

H14. Anthropomorphism in CJ plays a moderating role in chatbot interaction with users.

Methods: multigroup structural equation modeling

We conducted a survey to examine the effects of anthropomorphism on user algorithm processes in chatbot news services. The experiment was conducted during the first quarter of 2020 and a few months after the quarter for follow-up. Respondents were divided into two groups: a control group that did not receive anthropomorphic cues and a treatment group that received anthropomorphic cues in the experimental design. A group with explanatory cues was provided with semantics or words that explain the recommended articles. A group with anthropomorphic cues were given human-like conversational agents in the interaction.

Sampling and data collection

With an a priori power analysis by G*Power, we determined a sample size of at least 350 responses per group (effect size = .85, alpha = .05, and power = .95). We recruited a random sample of 760 respondents residing in the United States via Amazon’s Mechanical Turk service in exchange for monetary compensation between US$3 and US$5. The sample comprised 380 users per group (control and treatment). To ensure the reliability and validity of the sample, a series of confirmation procedures were included in the questionnaire. Among the submitted responses, 40 partial responses with missing data were dismissed, leaving a final sample of 760. Among the respondents, 39% were between the ages of 30 and 45, 41% were between the ages of 19 and 29, 13% were between the ages of 46 and 60, and 8% were older than 61. Fifty-one percent were female and 2% were either unknown or other. The average ages were 33.7 years for the control group (SD = 11.10) and 33.9 years for the treatment group (SD = 9.72).

Experimental design

Participants were presented with an experiment in which they were tasked to find information and read news articles with the help of a chatbot’s advice. They were asked to chat and interact with AI and to read AI-generated recommendations on chatbot-based apps for several hours (Figure 2). Participants were directed to search for information using personalized algorithms such as Netflix (hybrid recommender systems), Amazon (Amazon Personalize), and Hulu (streaming algorithm). They were requested to submit information about user characteristics, behaviors, demographics, and preferences and to evaluate the recommendations they received based on how well they were personalized and relevant to their needs. They searched using preferred personalized algorithms for items, such as movies, books, media content, and news. During the experiment, they were informed that the news recommendations were produced via automated algorithms and AI services.

Figure 2.

Two-step flow of algorithmic information processing in CJ.

The experiment took 1 hour to complete and consisted of two phases. In the first phase, participants were provided with explanations of FAT in the context of AI and journalism, since FAT concepts are so abstract and complex that casual users do not understand them. Participants were also instructed in specific procedures for how to access, interact with, and read the news. Participants were encouraged to go through the processes carefully and take note of any ambiguities. After the first phase, participants engaged in a 25-minute news-searching and news-reading task, in which they were asked to search for preferred news through interactions with chatbots. Once they completed the task, they were given surveys to complete. They were then debriefed and dismissed (Figure 3).

Figure 3.

Chatbots experiments.

Scales and measurements

The measurements (27) in this study were derived from previously developed and validated items. The FAT measurements were derived from Shin (2020) and Shin et al. (2020). The explainability measurements were modified from the works of Renijith et al. (2020) and Rai (2020). The measurements of personalization and accuracy were derived from the technology acceptance literature (Rhee and Choi, 2020; Soffer, 2019). The trust measurements were derived from Sah and Peng (2015) and Verhagen et al. (2014). The emotion measurements were derived and modified from the work of Lee (2018). The measurements of anthropomorphism for post hoc analysis were adapted from De Visser et al. (2016). The humanness measurements were modified from Westernman et al. (2019). Measurements were combinations of formerly used items and measurements adapted from other research. Some measurements required changes in order to reflect new traits of chatbots and AI services. Twenty college students with prior experience using chatbot services or a conversational AI assistant (Alexa/Siri) completed a pretest about a specific news topic.

As the variables of the FATE model are conceptually overlapped and interdependent each other, it is important to resolve the simultaneity bias issue. We used the multicollinearity test to resolve possible simultaneity problems. The results of the variance inflation factor through the multicollinearity test shows that the value of simultaneity is larger than 0.2 and lower than 10. Thus, multicollinearity is not a concern and the simultaneity bias is checked. Confirmatory factor analysis test shows the existence of the relationship between observed variables and their underlying latent constructs. Nine constructs were measured and specified as latent variables in the model. All factor loadings are greater than .70 and all alpha greater than .80 (Table 1).

Table 1.

Descriptive statistics.

	Group 1	Group 2
Age (mean/SD)	39.12/14.24	40.11/16.23
Gender (female rate)	52.30	51.11
College educated	27.11	29.34
CJ service experience	1.3 years	1.1 years

CJ: conversational journalism.

Confirmatory factor analysis.

Factor	Factor loading	Mean/SD	Cronbach’s alpha	CR
Explainability	.79	4.1/1.10	.725	0.713
	.72	4.5/0.49
	.86	4.6/0.15
Transparency	.82	4.1/1.51	.714	0.704
	.84	4.3/1.55
	.73	4.6/0.94
Fairness	.89	4.5/0.56	.785	0.841
	.82	4.6/1.52
	.71	4.9/1.94
Accountability	.84	4.1/2.41	.727	0.837
	.86	4.2/1.55
	.78	4.1/1.59
Personalization	.89	4.15/1.66	.796	0.982
	.88	4.13/0.98
	.71	4.19/1.81
Accuracy	.87	4.52/1.55	.802	0.827
	.89	4.13/1.50
	.81	4.62/4.69
Trust	.90	4.61/2.50	.905	0.789
	.91	4.56/0.99
	.84	4.62/0.15
Emotion	.88	4.62/1.59	.772	0.892
	.93	4.56/1.66
	.91	4.76/1.32
Humanness	.89	5.58/1.022	.845	0.874
	.87	5.66/1.095
	.90	5.47/1.085

	1	2	3	4	5	6	7	8	9
Explainability	.93	–	–	–	–	–	–	–	–
Transparency	.21	.85	–	–	–	–	–	–	–
Fairness	.34	.25	.82	–	–	–	–	–	–
Accountability	.42	.33	.33	.88	–	–	–	–	–
Personalization	.33	.04	.04	.02	.91	–	–	–	–
Accuracy	.48	.19	.39	.13	.22	.87	–	–	–
Trust	.23	.23	.23	.21	.09	.32	.87	–	–
Emotion	.43	.42	.12	.10	.42	.21	.12	.90	–
Humanness	.21	.25	.24	.25	.30	.30	.02	.14	.84

Model fit

The use of standard error of the mean (SEM) commonly involves using several indices to assess model fit, including χ². Because the χ² statistic is sensitive to outlier observations and sample size, normed fit index (NFI), root mean square error of approximation (RMSEA), and comparative fit index (CFI) were used to measure model fit (Hair et al., 2013). Results were acquired for IFI = 0.94; 0.93, RMSEA = 0.05/0.09, NFI = 0.98/0.97, and CFI = 0.92/0.94. Hoelter’s values showed acceptable results because both cases were within the suggested range (75 ⩽ value < 200). Although NFI and CFI values were lower than recommended, other fit indices were satisfactory. Considered together, these values constitute evidence of a reasonably good fit. Internal consistencies for the scales were good, with a mean score of approximately 0.86. The model fit is therefore considered satisfactory, and 10 structural relations were analyzed with the model. The fit indices indicated a satisfactory fit of the data (Table 2).

Table 2.

Model fit indices.

Fit statistics	Group 1	Group 2	Criteria
χ²	1489.68	1490.66	–
χ²/df	1489/310 = 4.80	1490/310 = 4.80	<5
p value	.000	.000	<.05
RMSEA	0.05	0.09	0.05–0.10
CFI	0.918	0.932	>0.90
RFI	0.786	0.749	0< x < 1
Hoelter	95 (0.01)	79 (0.01)	75 ⩽ x < 200
IFI	0.93	0.92	>0.90
GFI	0.965	0.965	>0.95
AGFI	0.915	0.922	>0.90

RMSEA: root mean square error of approximation; CFI: comparative fit index; GFI: goodness of fit index; AGFI: adjusted GFI; RFI: Relative Fit Index; IFI: Incremental Fit Index.

Results

Multigroup analysis: testing for multigroup invariance

We used a method described by Wu (2018) in multigroup analysis conducted in AMOS 26 software to assess the structural paths from the model across control and treatment groups (Table 3 and Figure 4). The two models indicated a good fit with the data, and distinct patterns were observed that supported the hypotheses. The results indicated notable distinctions in item composition and path formation, providing insights on the effect of anthropomorphism on CJ (Table 3). In group 1, the paths from explainability to transparency and accountability were rejected, and in group 2, the paths from explainability to FAT showed higher effect sizes. Comparing the two models, group 1 showed higher values for systematic processes whereas group 2 showed higher values for heuristic processes, signifying anthropomorphism. In addition, all paths from transparency, accountability, and fairness to trust were significant, with high coefficient values in group 2 (0.317; 1.086; 0.408), whereas counterpart values were all either low or rejected. For anthropomorphism, overall paths of group 2 showed greater effect sizes than in group 1. Anthropomorphic cues significantly influence heuristic processes as well as systematic processes, and the effects are greater on heuristic processes than systematic processes. The results of squared multiple correlations also supported the underlying role of anthropomorphism. The R² of trust in group 2 was .810, and the counterpart value was .553. The R² of emotion in group 2 was .691, and the counterpart value was .373. In addition, the R² of the FATE of group 2 was noticeably higher than that of group 1.

Table 3.

Results of hypothesis testing.

H	Group 1: Explanatory cue					Group 2: Anthropomorphic stimuli
H	B	SE	CR	p	Result	β	SE	CR	p	Result
H1	–0.022	0.070	–0.316	.111	No	1.021***	0.070	14.712	.000	Yes
H2	0.119	0.106	2.999	.007	No	0.065**	0.026	2.481	.013	Yes
H3	0.345***	0.076	4.556	.000	Yes	0.539***	0.055	9.746	.000	Yes
H4	0.432***	0.077	5.600	.000	Yes	0.317***	0.038	8.341	.000	Yes
H5	1.644	0.565	2.911	.006	No	1.086*	0.388	2.802	.005	Yes
H6	0.204***	0.053	3.843	.000	Yes	0.408***	0.053	7.696	.000	Yes
H7	0.561***	0.053	10.625	.000	Yes	0.648***	0.055	11.846	.000	Yes
H8	0.603***	0.060	10.024	.000	Yes	0.660***	0.063	10.465	.000	Yes
H9	0.473***	0.060	7.835	.000	Yes	0.373***	0.060	6.200	.000	Yes
H10	0.235*	0.076	3.096	.006	No	0.134	0.077	1.738	.082	No
H11	1.236	0.637	1.939	.053	No	.228***	0.038	6.076	.000	Yes
H12	0.134*	0.060	2.237	.025	Yes	1.021***	0.069	14.712	.000	Yes
H13	0.307*	0.254	0.816	.003	Yes	0.061*	0.045	2.360	.004	Yes

p < .05; **p < .01; ***p < .001.

Squared multiple correlation comparison.

	Group 1	Group 2
Fairness	.176	.436
Accountability	.148	.367
Transparency	.020	.579
Trust	.553	.810
Accuracy	.374	.450
Personalization	.214	.570
Emotion	.373	.691
Humanness	.274	.319

Note. The result of multicollinearity test shows no signs of a multicollinearity problem.

Figure 4.

Compared coefficients.

Post hoc analysis: do anthropomorphism and explainability interact with each other?

Due to the significant roles of explainability and anthropomorphism, possible interaction effects were tested. We conducted a post hoc analysis to see if there was a significant interaction effect of explainability and anthropomorphism on the perceived humanness. Two-way analysis of variance (ANOVA) revealed interactions between anthropomorphism and explainability on non-functional evaluation and trust (F[1, 31]= 4.65, p < .01, partial η² = 0.004). The interaction effects suggest that higher anthropomorphism resulted in high explainability and therefore produced a greater positive FAT value. This indicates more positive trust than in contexts with low anthropomorphism tendency and low explainability. Explainability and anthropomorphism have conjoined effects on humanness and trust. People with high anthropomorphic perception accept chatbots as more trustworthy than those with low anthropomorphic perception. High explainability users recognize more human-like and are happier with anthropomorphic chatbots, whereas low explainability users are less accepting of anthropomorphism and show lower trust and humanness values. The link of anthropomorphism and explainability is supported by the validated paths of perceived humanness and explainability (group 1: β = 0.134*; CR = 2.237; group 2: β = 1.021*; CR= 14.712).

Anthropomorphism and explainability are related and are key determiners of humanness and trust in interactions with chatbots. This explainability is facilitated by human-like anthropomorphic stimuli. The interaction effects suggest the complementary relations of explainability and anthropomorphism in AI. Users desire human-friendly and human-like explanations. The higher the explainability, the greater the anthropomorphism and vice versa. Anthropomorphic explainability together serves as heuristic cues suggesting better algorithmic quality than text explanation alone. In a chatbot interaction, this humanized explanatory anthropomorphism may manifest to such an extent that users believe they are interacting with a human—they experience high-perceived humanness of the system (Figure 5).

Figure 5.

Interaction effect of types of cues on trust.

Discussion

Our findings indicate that framing chatbots using anthropomorphic explanations can impact how users perceive and search news via chatbots. Perceived humanness has a significant impact on the users’ willingness to interact with chatbots and has notable consequences for user’s beliefs and behaviors. These findings further illustrate that interacting with CJ engages a cognitive information process in accessing chatbot news, wherein algorithmic qualities are utilized to inspire a heuristic that users use to determine the extent of humanness and attitudes toward CJ services. The results of our empirical study confirmed that anthropomorphized cues could trigger positive and persuasive reactions by initiating and sustaining positive user heuristics. Anthropomorphism arouses users’ heuristic processes and helps them to understand FATE features in CJ. The findings for perceived humanness are also in line with previous studies, which showed the significance of humanness in chatbots. The results substantiated the hypothesis that anthropomorphic cues are critical in creating positive user responses. The results of this study offer a meaningful contribution to the dynamics in heuristics, anthropomorphism, and trust in CJ.

First, using anthropomorphic cues as a stimulus, we examined how humanized explanations influence value, trust, and emotion through two-step flows. Relevant prior work does not explicitly characterize trust relations, and merely considers visual/technical rather than cognitive/behavioral anthropomorphism. Participants who interacted with chatbots using anthropomorphic cues reported higher perceived humanness, greater trust, and stronger emotional valence. These results parallel those of prior studies (e.g. Araujo, 2018; Go and Sundar, 2019) and reinforce the idea that users evaluate CJ with trust-based humanized explanations (Shin, 2021a). Results support existing literature (e.g. Westernman et al., 2019) highlighting that increased perceived humanness correlates with increased trust and emotion.

These results are intriguing for several reasons. In line with the results of previous research, the existence of anthropomorphized entities in CJ activated positive perceptions of trust, and how AI works. One factor in user trust in CJ is the degree to which a system is perceived as human-like or anthropomorphic, which also triggered positive valences of trust and emotion. This parallels the results of previous research on CJ finding that computers are more likely to be received as being human-like when they have a humanized cue/stimulus (Araujo, 2018; Verhagen et al., 2014). Moreover, this suggests that anthropomorphic cues not only affect/affected by explainability, but also enhance humanness, which further leads to trust. Anthropomorphic cues serve as not only function facilitators affecting FATE, but also influence perceived humanness, which then influences personalization and accuracy. This result shows that perceptions of anthropomorphism play a key role in stewarding positive relationships through trust, judgment, and attitude. Anthropomorphism and FATE are indeed related and are paramount sources of trust in interactions with CJ. This empirical study further supports the influence of perceived performance as a determinant of positive user valence, particularly service encounter emotion and satisfaction. Thus, our findings provide a baseline understanding of how users process anthropomorphism in algorithms, and from there, how CJ should be designed and what human-centered CJ are.

Second, the findings shed light on the users’ algorithmic information processes by incorporating trust into the two-step flow of interaction. Users’ cognitive development of chatbots is heuristic and nonlinear, not organized into structured ready-made automatic processes. How users perceive, analyze, understand, and consume the algorithmic information depend on how they view chatbots as human-like, with human-like capacities, which echoes the Turing test (1950). With the two-step flow mechanism, users actively process and proactively maneuver stimuli and process of the algorithmic choices/information they receive from their cognitions and evaluate them in terms of humanness and trust. Users are becoming increasingly skeptical of algorithmic processes, suspecting that chatbots may not be as objective as human journalists and may instead evince the racial, gender, and other social biases of their human coders. Thus, users naturally seek to know how algorithms work, how fake news and chatbots operate, and how to protect against disinformation. Based on previous studies (e.g. Lee, 2018), trust dynamics offer insight into how trust can be built and how it mediates the connection between the functional and non-functional quality dimension (Guzman and Lewis, 2020). Trust in CJ is cognitively constructed in such a way that processes are transparently explained and interpreted in a human way (Shin and Park, 2019). Established trust allows users to believe that the news is relevant and accurate, and the source is credible. Algorithmic trust mediates the influences of normative beliefs (non-functional algorithms) on CJ-related behaviors. Increasing user trust and emotion may warrant that personal data be handled in a lawful and fair manner with observable processes, thereby generating trust of the CJ algorithms, leading to positive emotion.

Third, the confirmed relationship between FATE and performance expectancy implies a mediating role of trust in stimulating the relationship between non-functional and functional algorithmic qualities. It can be argued that non-functional algorithmic qualities and performance values are positively associated with trust. That is, when users perceive algorithmic processes to be fair, transparent, and accountable, there tends to be a positive evaluation of performance (Shin, 2020a). Users evaluate the personalization and accuracy of AI via a two-step flow; one through the peripheral route process of FAT and the other via acceptance of the central route process of performance. Peripheral route evaluation occurs with a more heuristic assessment, simplifying FATE decision assessment to easily allow assessment of service quality as their technical knowledge is limited. Since the notions of FAT are abstract, users rely on human-understandable/interpretable explanations. Central route processing involves a deliberate evaluation of utility and performance. Users scrutinize news contents in terms of relevance, personalization, and accuracy. Trust links the two-step of peripheral and central evaluations. Trust constructed through peripheral processing is related to cognitive attributes that reflect a non-functional assessment, whereas that formed through central routes is more likely to affect the performance dimension.

Implications: perceived humanness and HAII

The implications of this study are twofold, with both theoretical and managerial components. Theoretically, the relationship between algorithmic information processes and anthropomorphism/trust has implications for user cognitive processes (two-step flow) in CJ that can be translated into practical guidelines for how to provide user-centered CJ. The findings have practical implications concerning what CJ newsrooms should do to forge effective content recommendations, particularly how to address and reflect FAT in CJ user interfaces and interaction.

Theoretical implications: rethinking Turing’s test for HAII

This study makes meaningful contributions to the extant literature about perceived humanness, trust, and algorithms in the context of CJ. First, our results reinforce the importance of explainability to users seeking to understand and trust the reasoning behind news generations and predictions. We characterize the antecedents of and associations among algorithmic attributes and highlight the algorithmic information processing of those antecedents and emotion. These relations are important because algorithm increasingly becomes a relational agent and thus how users relate with chatbots becomes a fundamental question to address (Westernman et al., 2020). These relationships contribute to existing theoretical knowledge of how perceived humanness and trust are formed in an algorithmic context and how anthropomorphic explanations can guide users to make sense of recommended news. These insights provide theoretical implications of how humanness can be conceptualized, measured, and theorized in CJ. Perceived humanness in CJ is related to explainability, which affords users with a cue frame that invokes the issues at hand. Not only does it provide an understandable base of CJ, but the presence of the explanation itself is trustworthy and humanizing, enhancing user confidence in assessing FAT (Go and Sundar, 2019). Explanations with human-like cues guide users to construe the results of CJ by demystifying its black box, while explainability itself serves as a guarantee of certain aspects of FAT. Explanations can keep CJ accountable for their services by warranting that their services meet ethical/legal standards, allow users to audit the system, and generate the best results (Park and Skoric, 2017). How to optimize the demand for algorithmic innovation and public interest while providing transparency and accountability for users are key practical concerns as well as theoretical issues to address. Our results contribute to ongoing literature on how to address such issues in CJ and how CJ can best serve the public by maintaining a high level of journalistic standards and practices.

Second, the elaboration-likelihood processes in the study expand upon ongoing research into user interaction with CJ and specifically the algorithmic information process literature, by illustrating how algorithmic processing has been applied to chatbots. Understanding the extent to which users actively process and proactively manage sequence and process of algorithmic choices has key implications for theories and design in HAII. The findings that the role/process of anthropomorphism and the relationships among its associated measures (Moller et al., 2018; Shin and Park, 2019) not only confirm the theory’s key proposition, that decision-making is guided by a dual process (Petty and Cacioppo, 1986), but also advance the theory by implicating the process to the two-step flow of communication (Soffer, 2019). Algorithmic stimuli are processed in two-step flow of interaction; first, users evaluate humanness based on heuristics and other non-functional quality information, and second, based on the first evaluation, users then evaluate functional quality of algorithms. Users process algorithmic stimuli in order to evaluate whether they are conversing with a human or machine. When provided with human explanations, users feel more confident when assessing algorithmic features. Anthropomorphic cues help users to assess FAT and further establish trust, which in turn links and facilitates the processes between peripheral and central routes, enabling user heuristic activity, evaluations, and attitudes. Users process anthropomorphic cues and explanations by following a heuristic process in assessing perceived humanness with all available information influencing perceptions, which in turn influences the way information is used, as well as the effectiveness of persuasive explanations. Explicating the interplay between non-functional and functional processes in AI news storytelling illustrates that two-step flow theory is relevant for AI research.

Another contribution of the study is that our model clarifies the role of user emotion in CJ, which is becoming an important issue as firms attempt to integrate AI into psychological contexts (Ananny and Finn, 2020). HAII involves sentiments, as shown by the growing realization that robots need to include a human emotional state in their operations (Sundar, 2020). As our findings imply, human emotion in CJ is largely affected by perceived humanness through heuristic and analytical processes. The functional and non-functional qualities of algorithms are sorted through user cognitive processes of FAT. Users experience positive emotional states when their evaluations via peripheral and central routes are pertinent and empathetic. Emotion is substantially influenced by user trust, rather than performance (H10, the insignificant effect of personalization on emotion).

Managerial implications: anthropomorphizing in CJ

The results of the present study have implications for the design of intuitive CJ interfaces that could better enable human-centered chatbot media. This study has practical implications for CJ because it provides interface design guidelines. Without a doubt, providing human-like conversational delivery is interesting for news organizations because AI may fundamentally change news delivery in the future. To design AI-driven systems for journalism that readers can trust and use in good faith, the industry should understand the users’ algorithmic information processes, that is, how AI qualities impact trust placed in these services by users. Newsrooms should be mindful of what is enhancing the perceived humanness of their chatbot agents in user interactions. The two-step flow framework provides heuristic guidelines for the design of human-centered AI journalism. Since AIs fundamentally transform the way people read and search for news items, how to design a fair algorithm, transparent interactions, and how to apply explainability in the journalism context are urgent questions to solve.

Our results are meaningful in the context of journalism and chatbots. Topics of explainability and fairness are current in algorithmic journalism (Shin et al., 2020), and users increasingly expect assurances regarding such concerns when adopting CJ. Trust is a linchpin in determining user evaluations, intentions to use, follow, and interact with CJ, and trust performs a pivotal role in building user trust and credibility in journalism. The more transparent the processes of CJ are, the stronger the trust between AI and users. Trust serves as a key mediator between AI systems and users, creating positive feedback cycles in the algorithm processing model. The journalism industry should gain insights regarding how to realize FAT and integrate relevant concerns with other factors — for example, how to maximize user data and/or implicit feedback while enhancing user trust and media credibility. Our clarification of explainability provides insights regarding how to address the GDPR’s right to explanation in CJ. Framing CJ design in such a way that users feel like interacting with humans by incorporating both explanation and anthropomorphism is an important follow-up topic for further research.

Conclusion and future studies

Our results bring us to the open problems and we propose a paradigm shift in the understanding of AI in journalism (Guzman and Lewis, 2020). CJ is a paradigm of the innovative AI platform within the current media ecology (Park et al., 2018). The lack of trust in conventional journalism and the limited time that readers should dedicate to reading news has led journalists to develop new and creative approaches that make journalistic content more relevant and interesting. The best course to a sustainable future for CJ is to best optimize trust, explainability, and humanization in news storytelling. Our results show that anthropomorphism plays a pivotal role in chatbot interactions and should be reflected in chatbot form (interface), performance (news recommendation), and interactions (modality). The combined influence of anthropomorphized explanations can facilitate two-step flow of interaction in the news consumption process.

Finally, a few propositions can be suggested for future studies. Methodologically, future studies may consider using a 2 × 2 factorial design (group 1 without any cues, group 2 with two types of cues, group 3 with explanation, and group 4 with anthropomorphic cues). Theoretically, future studies may further examine the two-step flow of communication, particularly in the theoretical justification of the influence of emotion on the relationships between normative belief and functional features. Hypothesis-wise, no significant effects were found between personalization and emotions. This is somewhat unexpected, since customized personalization typically influences user emotion (Sah and Peng, 2015). Past research has confirmed that accurate personalization increases the positive human emotion (Rhee and Choi, 2020; Soffer, 2019). Maybe the experiment situation and the controlled manipulation may have misled findings. Future research can examine this hypothesis in different AI context.

Footnotes

Appendix

Appendix 1.

Measurement items.

Variables	Measures
Perceived humanness	1. I feel like to converse with real human when I interact with chatbots 2. It makes me feel that chatbots are human-like entities 3. This chatbot system has human properties. I thought chatbots might be human
Fairness	1. The system has no favoritism and does not discriminate against people 2. The source of data throughout an algorithm and its data sources should be identified, logged, and checked 3. The system follows due process of impartiality with no prejudice
Accountability	1. The system requires a person in charge who should be accountable for its adverse individual or societal effects in a timely fashion 2. Algorithms should be designed to enable third parties to examine and review the behavior of an algorithm 3. Algorithms should have the ability to control a system in its entire configuration using only certain manipulations
Transparency	1. The evaluation and the criteria of algorithms used should be publicly released and understandable and interpretable to people 2. Any outputs produced by an algorithmic system should be explainable to the people affected by those outputs 3. Algorithms should enable people to understand how well internal states of algorithms can be understood from knowledge of its external outputs
Explainability	1. I found algorithm are easily understandable 2. The algorithm services are interpretable 3. I can figure out the internal mechanics of a machine learning. I hope that algorithm can be clearly explainable
Accuracy	1. The contents produced by algorithms are without errors 2. Recommended items by algorithm systems are in general precise 3. Algorithm-enabled recommendations are exact and correct
Personalization	1. The recommended items reflect my personalized preferences 2. I found the recommended items are a great match to my needs 3. The algorithm-based service is customized to me
Trust	1. I trust the recommendations by algorithms-driven services 2. Recommended items through algorithmic processes are trustworthy 3. I consider the algorithm service results are reliable
Emotion	1. Using AI was much better than what I expected 2. I feel good with AI. The service provided by AI was better than what I expected 3. I feel comfortable with the services provided by AI

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This project has been funded by the Research Office of Zayed University, Research Incentive Fund (R20082/2020). This project has been also supported by the Center for Educational Innovation of ZU, Teaching Innovation Research Fund (TIRF-S19-01: B19053) and Start-Up Grant (R19032).

ORCID iD

Donghee Shin

Author biography

Donghee Shin (Ph.D./M. Phil. Syracuse University) has taught various universities in the US and Korea over 20 years. Prior to Zayed University, he was a Professor at Sungkyunkwan University, Seoul, Korea. He also was founding Director of the Department of Interaction Science, an interdisciplinary research initiative sponsored by Ministry of Education and Samsung Foundation. As a Head and Director of Interaction Science Research Center, he also served as a Principal Investigator of BK21 Plus. His research interests include digital journalism, human-computer interaction, and algorithmic media. His recent research in computer/algorithm as media addresses: embodied cognition, the subjective value of algorithms, interaction and interactivity.

References

Ananny

Finn

(2020) Anticipatory news infrastructures. New Media & Society 22(9): 1600–1618.

Araujo

(2018) Living up to the chatbot hype. Computers in Human Behavior 85: 183–189.

BBC News Labs (2019) Bots. Available at: https://bbcnewslabs.co.uk/projects/bots

Crain

(2018) The limits of transparency. New Media & Society 20(1): 88–104.

De Visser

Monfort

McKendrick

, et al. (2016) Almost human. Journal of Experimental Psychology 22: 331–349.

Diakopoulos

Koliska

(2016) Algorithmic transparency in the news media. Digital Journalism 5(7): 809–828.

Dörr

Hollnbuchner

(2017) Ethical challenges of algorithmic journalism. Digital Journalism 5(4): 404–419.

Ford

Hutchinson

(2019) Newsbots that mediate journalist and audience relationships. Digital Journalism 7: 1013–1031.

Sundar

(2019) Humanizing chatbots: the effects of visual, identity and conversational cues on humanness perceptions. Computers in Human Behavior 97: 304–316.

10.

Graefe

Haim

Haarmann

, et al. (2018) Readers’ perception of computer-generated news. Journalism 19(5): 595–610.

11.

Gunning

Stefik

Choi

, et al. (2019) XAI: explainable artificial intelligence. Science Robotics 4(37): 7120.

12.

Guzman

Lewis

(2020) Artificial intelligence and communication. New Media & Society 22(1): 70–86.

13.

Hair

Hult

Ringle

, et al. (2013) A Primer on Partial Least Squares Structural Equation Modeling. New York: SAGE Publications, Inc.

14.

Helberger

(2019) On the democratic role of news recommenders. Digital Journalism 7: 993–1012.

15.

Jones

(2019) Public service chatbots. Digital Journalism 7(8): 1032–1053.

16.

Kemper

Kolkman

(2019) Transparent to whom? Information, Communication & Society 22: 2081–2096.

17.

Kim

Sundar

(2012) Anthropomorphism of computers. Computers in Human Behavior 28: 241–250.

18.

Lee

(2018) Understanding perception of algorithmic decisions. Big Data & Society 5(1): 1–16.

19.

Lewis

Guzmna

Schmidt

(2019) Automation, journalism, and human-machine communication. Digital Journalism 7(4): 409–427.

20.

Marchionni

(2013) Conversational journalism in practice. Digital Journalism 1: 252–269.

21.

Moller

Trilling

Helberger

, et al. (2018) Do not blame it on the algorithm. Information, Communication & Society 21(7): 959–977.

22.

Park

Chung

Shin

(2018) The structuration of digital ecosystem, privacy, and big data intelligence. American Behavioral Scientist 62(10): 1319–1337.

23.

Park

Skoric

(2017) Personalized ad in your Google glass? Journal of Business Ethics 142(1): 71–82.

24.

Petty

Cacioppo

(1986) Communication and Persuasion: Central and Peripheral Routes to Attitude Change. Berlin: Springer-Verlag.

25.

Rai

(2020) Explainable AI. Journal of the Academy of Marketing Science 48: 137–141.

26.

Renijith

Sreekumar

Jathavedan

(2020) An extensive study on the evolution of context-aware personalized travel recommender systems. Information Processing & Management 57(1): 102078.

27.

Rhee

Choi

(2020) Effects of personalization and social role in voice shopping: an experimental study on product recommendation by a conversational voice agent. Computers in Human Behavior 109: 106359.

28.

Sah

Peng

(2015) Effects of visual and linguistic anthropomorphic cues on social perception, self-awareness, and information disclosure in a health website. Computers in Human Behavior 45: 392–401.

29.

Shin

(2020a) How do users interact with algorithm recommender systems? Computers in Human Behavior 109: 106344.

30.

Shin

(2020b) User perceptions of algorithmic decisions in the personalized AI system: perceptual evaluation of fairness, accountability, transparency, and explainability. Journal of Broadcasting & Electronic Media. 64(4), 541–565.

31.

Shin

(2021a) Expanding the role of trust in the experience of algorithmic journalism: user sensemaking of algorithmic heuristics in Korean users. Journalism Practice. Epub ahead of print 4 December. DOI: 10.1080/17512786.2020.1841018

32.

Shin

(2021b) The effects of explainability and causability on perception, trust, and acceptance. International Journal of Human-Computer Studies 146: 102551.

33.

Shin

Biocca

(2018) Exploring immersive experience in journalism. New Media & Society 20(8): 2800–2823.

34.

Shin

Park

(2019)) Role of fairness, accountability, and transparency in algorithmic affordance. Computers in Human Behavior 98: 277–284.

35.

Shin

Zhong

Biocca

(2020) Beyond user experience. International Journal of Information Management 52: 1–11.

36.

Soffer

(2019) Algorithmic personalization and the two-step flow of communication. Communication Theory. Epub ahead of print 30 April. DOI: 10.1093/ct/qtz008.

37.

Sundar

(2020) Rise of machine agency. Journal of Computer-Mediated Communication 25(1): 74–88.

38.

Tandoc

Yao

(2020) Man vs. machine? Digital Journalism 8(4): 548–562.

39.

Thurman

Moeller

Helberger

, et al. (2019) My friends, editors, algorithms, and I. Digital Journalism 7(4): 447–469.

40.

Turing

(1950) Computing machinery and intelligence. Mind 49: 433–460.

41.

Verhagen

Nes

Feldberg

(2014) Virtual customer service agents. Journal of Computer-Mediated Communication 19: 529–545.

42.

Westernman

Aaron

Lindmark

(2019) I believe in a thing called bot. Communication Studies 70(3): 295–312.

43.

Wölker

Powell

(2018) Algorithms in the newsroom? News readers’ perceived credibility and selection of automated journalism. Journalism. Epub ahead of print 18 February. DOI: 10.1177/1464884918757072

44.

(2018) How age affects journalists’ adoption of social media as an innovation. Journalism Practice 13(5): 537–557.

45.

Zarouali

Makhortykh

Bastian

, et al. (2020) Overcoming polarization with chatbot news? Investigating the impact of news content containing opposing views on agreement and credibility. European Journal of Communication. Epub ahead of print 18 July. DOI: 10.1177/0267323120940908.