Do You Mind if I Ask You a Personal Question? How AI Service Agents Alter Consumer Self-Disclosure

Abstract

The use of Artificial Intelligence (AI) has grown rapidly in the service industry and AI’s emotional capabilities have become an important feature for interacting with customers. The current research examines personal disclosures that occur during consumer interactions with AI and humans in service settings. We found that consumers’ lay beliefs about AI (i.e., a perceived lack of social judgment capability) lead to enhanced disclosure of sensitive personal information to AI (vs. humans). We identify boundaries for this effect such that consumers prefer disclosure to humans over AI in (i) contexts where social support (rather than social judgment) is expected and (ii) contexts where sensitive information will be curated by the agent for social dissemination. In addition, we reveal underlying psychological processes such that the motivation to avoid negative social judgment favors disclosing to AI whereas seeking emotional support favors disclosing to humans. Moreover, we reveal that adding humanlike factors to AI can increase consumer fear of social judgment (reducing disclosure in contexts of social risk) while simultaneously increasing perceived AI capacity for empathy (increasing disclosure in contexts of social support). Taken together, these findings provide theoretical and practical insights into tradeoffs between utilizing AI versus human agents in service contexts.

Keywords

artificial intelligence privacy disclosure emotion anthropomorphism robots

Do you mind if I ask you a personal question? [HAL 9000]

No, not at all. [Dave Bowman]

2001: A Space Odyssey ( Kubrick and Clarke 1968 )

Introduction

Artificial Intelligence (AI)¹ agents are increasingly common in direct consumer interactions that offer services to consumers—ranging from digital assistants and point-of-sale robots to virtual medical doctors and financial advisors (c.f., Huang and Rust 2018; Longoni, Bonezzi, and Morewedge 2019; Mende et al. 2019). For example, approximately 80% of firms have incorporated, or are in the process of incorporating, chatbots into their services (Oracle 2016). In the financial and medical domains, over three-quarters of chatbot interactions are anticipated to be completed without relocation to a human agent by 2022 (Juniper 2017). Meanwhile, AI-based analytical approaches have made large consumer datasets ever more useful and valuable to firms, driving a substantial increase in requests for personal information. The collection and sales of consumer personal data have become a substantial industry and serve as the revenue basis for some of the world’s largest technology companies, such as Facebook and Google. Despite widely publicized externalities, including data security breaches and the potential for data misuse, the acquisition of sensitive consumer data remains a top priority for nearly all firms globally. Also, research on the “privacy paradox” identified that individuals are highly concerned about data privacy whereas little effort is taken by consumers to protect the privacy of their personal data (Barth and De Jong 2017). As such, understanding the factors that lead consumers to disclose their personal information in exchange for a service has emerged as an important issue for practitioners and scholars alike (Acquisti, Brandimarte, and Loewenstein 2015; John, Acquisti, and Loewenstein 2011).

In the service environment, AI constitutes a new data agent capable of soliciting, collecting, and interpreting personal information in myriad ways. Such a change in the service landscape creates a new question: if everything else is the same, would individuals disclose private and personal information more to a human in contrast to an AI? AI capabilities, such as synthesizing existing data to create a new dataset that could provide behavioral inferences (Acquisti and Gross 2009), or the ability to store data permanently, could serve as factors that decrease the disclosure of personal information. In contrast, AI is generally perceived to lack social and emotional capabilities (Gray, Gray, and Wegner 2007), and customers may perceive a lower social risk of disclosing sensitive personal information to a service robot than a human service provider. As such, we predict that people’s lay beliefs about AI’s lack of emotional capability will increase their tendencies to disclose sensitive information to the AI (vs. a human) and that this effect will be moderated by service contexts and different motivations driving the disclosure.

According to impression management theory, people have a fundamental desire to project and maintain a positive self-image to other humans (Tedeschi 2013). When self-disclosure can lead to negative impressions (i.e., social judgment concerns), people avoid sharing such information with other people (Sun and Slepian 2020). Because AI is often perceived as not having its mind, emotion, and the capability to understand meanings associated with human behavior (Gray, Gray, and Wegner 2007; Kim and Duhachek 2020), people may be less concerned about negative social judgment resulting from the disclosure of sensitive information to an AI (vs. a human) agent.

Concerns about negative impression formation are closely associated with “social risk,” which we define as the risk of incurring potentially negative social evaluation and may be exemplified by sharing one’s embarrassing or guilt-laden experiences with others (Acquisti, John, and Loewenstein 2012; John, Acquisti, and Loewenstein 2011; Moon 2000). Previous research in social psychology has shown that emotional capabilities are a key element in social judgment (Forgas 1994). According to this perspective, emotional capabilities are as important as cognitive capabilities when engaging in social judgment, such as forming impressions of others (see also Boyatzis, Goleman, and Rhee 2000; Clore, Gasper, and Garvin 2001; Duhachek 2008; Hooker et al. 2022). The importance of emotion in various social interactions among individuals (e.g., empathy) is also emphasized in previous research on emotional and social intelligence (Boyatzis, Goleman, and Rhee 2000; Hooker et al. 2011). We build on this theoretical perspective to deepen extant understanding of consumer disclosure.

The current research contributes to the service literature by extending previous research which has demonstrated increased disclosure using technology interfaces in the elicitation of responses for survey research (Nass and Moon 2000). First, we demonstrate key differences between perceptions of humans, computers, and AI along essential dimensions relevant to consumer disclosure, thus demonstrating the importance of examining the robustness of classic disclosure effects given new AI technology. We theorize that disclosure is driven by perceptions of emotional capability that match the objectives of the consumer, thus elucidating the process behind the disclosure. Second, we identify a key boundary condition to the classic increased disclosure to technology effect. In service contexts where consumers seek social or emotional support, the disclosure effect is reversed such that consumers disclose more to humans as compared to AI, consistent with perceptions of increased emotional capabilities of humans to provide support. Third, we identify an additional boundary condition common to today’s service disclosure contexts. When an AI agent serves as a guard against the dissemination of socially inappropriate information (e.g., excluding embarrassing photos from an AI-curated photo album), we found greater disclosure to a human (vs. an AI) due to the doubt that an AI can properly distinguish what is socially inappropriate from appropriate, leading to the concern of negative social judgment, again demonstrating the role of emotional capability perceptions in driving disclosure.

Thus, our work presents a more nuanced theoretical understanding of classic disclosure effects. In contexts where AI is involved in soliciting information disclosure, selectively de-emphasizing AI attributes that speak to social judgment capability (e.g., emotional ability and visual recognition) should increase consumer disclosure due to reduced concern for negative social judgment. Thus, our work reveals how selective use of AI versus human representatives, in addition to the selective emphasis on certain AI attributes, can systematically increase or decrease consumer self-disclosure in service contexts. For example, sharing one’s private—and potentially embarrassing—information with a doctor could sometimes lead to finding a better medical treatment for a patient. In this case, an AI doctor with certain human features de-emphasized could serve better in terms of collecting important information for optimal medical treatment. However, our research also reveals that in contexts where consumers seek emotional support (e.g., experiencing sadness from a less-than-desirable service outcome or outright service failure) and contexts in which information will disseminated socially (e.g., image curation of a shared social media photo album), consumers will disclose more to a human than an AI.

Previous Research on Self-Disclosure

Factors Influencing Self-Disclosure

Previous research has defined self-disclosure as the sharing of any information about the self (Cozby 1973). This research measures disclosure either via open-ended responses (e.g., Joinson 2001) or solicited responses (e.g., John, Acquisti, and Loewenstein 2011). Past research has examined a list of factors that may influence consumer responses to AI versus human agents, or consumer responses to anthropomorphized AI versus machinelike AI agents.

These factors include (1) social presence of the agent (Schuetzler, Grimes, and Giboney 2020), (2) control perceptions (Brandimarte, Acquisti, and Loewenstein 2013), (3) trust perceptions (DeSteno et al. 2012; Złotowsk et al. 2016), (4) perceived intimacy and understanding (Ho, Hancock, and Miner 2018), (5) credibility (Longoni and Cian 2020; Powers and Kiesler 2006), (6) privacy concerns (John, Acquisti, and Loewenstein 2011; Lutz and Tamó-Larrieux 2020), (7) reciprocity (Nass and Moon 2000), and (8) discomfort elicited by the robot (Mende et al. 2019) (See Table 1 below²).

Table 1.

Self-Disclosure Literature Review: Agent Effect, Mediators, and Moderators.

Paper	Outcomes (DV)	Mediators	Comparing Humanlike AI vs. Machinelike AI	Moderators (Humanlike Feature Manipulated)	Key Findings
Moon (2000)	Disclosure to computers	NA	No	NA	Participants follow social norms when disclosing to computers. Reciprocal disclosure from the computer increases participants’ self-disclosure.
Kang and Gratch (2010)	Level and intimacy of disclosure	NA	No	NA	Socially anxious people revealed more personal information about themselves when interacting with a virtual human (vs. real human).
DeSteno et al. (2012)	Economic exchange behavior	Trustworthiness	No	Humanlike movements	Nonverbal cues influence the perception of trustworthiness of humanlike robots and economic exchange behavior.
Lucas et al. (2014)	Willingness to disclose	NA	NA	NA	Participants who believed they were interacting with a computer (vs. human) reported lower fear associated with self-disclosure, lower impression management need, and displayed their sadness more intensely.
Sah and Peng (2015)	Information disclosure	Social perception and self-awareness	Yes	Visual and linguistic anthropomorphic cues	Anthropomorphic images of agents decrease information disclosure, while anthropomorphic language increases disclosure.
Pickard, Roster, and Chen (2016)	Preference to disclose to a human versus an avatar interviewer	NA	No	NA	Participants preferred avatar (vs. human) interviewers for more (vs. less) sensitive topics.
Uchida et al. (2017)	Level of disclosure	NA	Yes	None	Robots (vs. human counselors) increase self-disclosure of sensitive topics.
Ho, Hancock, and Miner (2018)	Disclosure intimacy and emotional, relational, and psychological outcomes of disclosure	Perceived understanding, disclosure intimacy, and cognitive reappraisal	Yes. Chatbot	No	Disclosure extent and emotional, relational, and psychological outcomes did not differ depending on whether participants believed the agent to be a chatbot or a person.
Schuetzler et al. (2018)	Self-disclosure	Social presence	Yes. Conversational agents	Embodiment and conversational relevance	Conversational agents with greater conversational skills (but not embodiment) decrease participants’ disclosure of sensitive topics.
Mende et al. (2019)	Compensatory responses	Discomfort	Yes. Humanoid service robots	Name or birthplace	Consumers interacting with a humanoid service robot (a human employee) are more likely to show compensatory responses.
Longoni and Cian (2020)	Preference for AI recommenders	Perceived competence	No	No	Consumers prefer AI (human) recommenders in utilitarian (hedonic) consumption contexts.
Pickard et al. (2020)	Antecedents of disclosure, admissions to internal control and policy violations	NA	Yes	Dissimilar versus similar ECA face, or computer-generated vs. human voice	Employees were more likely to disclose their workplace violations to an embodied conversational agent (ECA) than to a human.
Meng and Dai (2021)	Stress reduction	NA	No	NA	Chatbots’ reciprocal self-disclosure is only effective in reducing stress when emotional support is present, but backfires when emotional support is absent.
Holthöwer and van Doorn (2022)	Preference for a robot (vs. a human) service provider	Social judgment	Yes. Service robots	Humanlike features such as a face, arms, and legs	Consumers prefer a robot (vs. a human) service provider during an embarrassing service encounter.
Pitardi et al. (2022)	Anticipated embarrassment	Agency	No	None	Interacting with service robots decreases the level of anticipated embarrassment.
Current research	Disclosure of sensitive and non-sensitive information	Social judgment concerns	Yes	AI’s emotional capability and service contexts	AI (vs. humans) leads to greater self-disclosure. However, this classic effect was moderated by the sensitivity of the disclosed information and reversed when consumers have emotional support motivation.

Recent research comparing technologies and humans for their impact on self-disclosure has mainly focused on how agent type (AI vs. human) influences self-disclosure, and they have demonstrated mixed findings (e.g., Lucas et al. 2014; Pickard, Roster, and Chen 2016; Uchida et al. 2017). Although some studies showed that participants were more willing to self-disclose when they interacted with a machine than with a human (Kang and Gratch 2010; Lucas et al. 2014; Pickard, Roster, and Chen 2016), other research found no difference in disclosure extent (Ho, Hancock, and Miner 2018; Moon 2000). The current research extends the literature by providing a unique framework in which the perceptual differences between different types of human and non-human agents (e.g., humans vs. AI vs. computers) could be examined in a systematic manner. In the next section, we summarize the moderators of self-disclosure that have been examined in the literature.

Moderators of Self-Disclosure

Previous research has also studied moderators that influence consumer responses to AI (See Table 1 and Online Appendix A). Different AI design features have been widely studied in the literature, including manipulations of (1) avatar visual features, embodiment, and gestures for physical robots (DeSteno et al. 2012; Powers and Kiesler 2006; Schuetzler et al. 2018; Sah and Peng 2015; Uchida et al. 2017; Złotowsk et al. 2016), (2) name, gender, ethnicity (Araujo 2018; Mende et al. 2019), (3) voice (Powers and Kiesler 2006), and (4) language and interaction style (Araujo 2018; Sah and Peng 2015; Schuetzler et al. 2018; Schuetzler, Grimes, and Giboney 2020). Past research has also touched on individual difference factors that alter responses to AI. For example, Kang and Gratch (2010) found that characteristics of disclosers—whether disclosers were socially anxious—moderated the effect. However, in service settings, various contextual factors may influence the effect and these factors have been understudied. The current research fills this gap and focuses on the service contexts that moderate the disclosure-to-AI effect.

The Current Research

Services may include contexts that lead to negative social judgment (e.g., contexts that involve the disclosure of sensitive personal information). Uchida et al. (2017) compared a robot and a human mental health counselor and found that the extent of personal disclosure to the counselor depends on the perceived sensitivity of the disclosed topic. As another example, Pitardi et al. (2022) and Holthöwer and van Doorn (2022) found that reduced perceptions of agency from a robot (vs. human) service provider could make customers feel less embarrassed during potentially embarrassing service encounters (e.g., embarrassing medical treatments). This work focused on anticipated embarrassment and suggested the importance of examining various contextual factors that may further influence self-disclosure. We extend previous work by identifying a novel reversal of the AI disclosure effect when consumers do not trust AI’s capacity to judge which type of disclosed personal information is socially sensitive and inappropriate for social dissemination. We also reverse the classic disclosure effect in contexts where consumers seek emotional support from their disclosure and believe that other humans are more equipped to meet their needs as compared to AI. Furthermore, we identify the role of empathy, an understudied factor in the self-disclosure literature, in dampening AI disclosure effects. In the next section, we develop our hypotheses.

Hypotheses Development

Disclosure of Sensitive Information to AI versus Humans

One factor that has been demonstrated in past research on disclosure relates to concerns over social judgment. Certain self-conscious emotions, such as embarrassment, shame, and guilt are associated with concerns over negative social judgment (Agrawal and Duhachek 2010) and may highlight social risk. In the context of disclosure to an AI agent, individuals are likely to hold the belief that AI is not endowed with the capabilities to understand meanings steeped in a broader social context, which reduces perceptions of social judgment (Kim and Duhachek 2020; Longoni, Bonezzi, and Morewedge 2019). If concerns over social judgment make people more likely to disclose to an AI than to a human agent, this effect should only occur for sensitive information but not for non-sensitive information. Based on these differences in the perceived capabilities of AI (vs. human) agents to render social judgments, we hypothesize, in accordance with previous research, that:

HYPOTHESIS 1a: In service settings, consumers are more willing to disclose information that is sensitive (i.e., incurs the risk of negative social judgment) to an AI agent than a human agent, but there is no difference when the information is non-sensitive.

HYPOTHESIS 1b: This effect is mediated by consumers’ lower (vs. higher) concern over social judgment by an AI (vs. a human) agent.

Different Disclosure Motivations

The enhanced disclosure to AI (vs. humans) hypothesized in H1a and H1b is due to the perception that AI lacks social judgment capabilities producing a reduced concern about negative judgment about the self from the AI. However, such an effect could be limited to the case of sensitive personal disclosure that is closely associated with the concern of negative social judgment. In contrast, disclosure of non-sensitive personal information less associated with social judgment concerns could be driven by other social motivations, such as disclosing one’s emotional experience to receive social support. For example, disclosing past experiences of sadness or depression to an empathic listener can provide a sense of emotional support. In such cases, AI’s lack of emotional capability (e.g., absence of empathy) could serve as a factor in decreasing disclosure. Thus, AI’s lack of emotional capability could increase sensitive personal disclosure due to the reduced concern about negative judgment. However, the same incapacity could decrease non-sensitive personal disclosure driven by other motivations, such as seeking emotional support. In other words, AI’s emotional capability could drive sensitive and non-sensitive personal disclosure in the opposite direction. It is important to note that AI’s emotional capability is predicted to evolve over time. Previous AI research in the service literature proposed that “empathetic” AI (Huang and Rust 2018) or “emotional-social” AI (Wirtz et al. 2018) could emerge in the near future. However, what current consumers believe about AI (i.e., the lay beliefs) may or may not be consistent with current AI technologies. Thus, H1a and H1b are proposed based on the lay theory that the majority of people hold about AI (i.e., “lack of emotions”) and replicate previous findings. Our research deepens our theoretical understanding of H2a and H2b. These hypotheses were proposed to test the idea that presenting a conceptual AI with enhanced emotional capabilities can moderate our main effect. As such, we propose:

HYPOTHESIS 2a: In service settings where consumers seek emotional support, disclosure of non-sensitive information will increase when an AI agent is perceived to have empathy.

HYPOTHESIS 2b: In service settings where the disclosure incurs the risk of negative social judgment, disclosure of sensitive information will decrease when an AI agent is perceived to have empathy.

Enhancing Disclosure to Humans

Past research has predicted that the next generations of AI will be involved in various social roles (Huang and Rust 2018; Wirtz et al. 2018). However, contextual understanding, a capability that is critically important in social interactions, is widely considered a blind spot among the current AI capabilities. The “Broken Leg Problem” depicts an algorithm that is superior to humans in predicting whether an individual would go out to watch a movie on a given day. However, the story goes such that when a new uncodified contextual factor emerges (e.g., a broken leg), humans are better at predicting the unlikeliness of the person going out for the movie, thus demonstrating the increased abilities of humans to consider contingent factors. As another example, Microsoft developed and released an AI named “Tay” on Twitter in 2016. Tay was shut down by Microsoft almost immediately after its debut for making numerous racist tweets due to its incapacity to screen out socially inappropriate tweets which were fed to Tay during its access to the millions of existing tweets and learning from them. In the same vein, past research has shown that individuals are averse to AI making moral or legal judgments in which contextual understanding is an important factor (Bigman and Gray 2018). Thus, we predict that disclosure to AI will be reduced when the disclosure could put consumers in socially vulnerable situations due to AI’s lack of social and contextual understanding. In examining this prediction, we focus on situations where social judgment ability is needed to screen out the dissemination of socially inappropriate information. In this case, the AI’s lack of social judgment ability could become a liability, leading to reduced disclosure. For example, judging the extent to which one’s photo is appropriate to be publicly shared requires contextual understanding and social judgment capabilities. Because these capabilities are perceived to be lacking among AI (i.e., the default lay theory people hold for AI), consumers will be concerned that AI (vs. human) agents might be unable to properly recognize and screen out the dissemination of personal information that carries a social risk. Formally, we hypothesize:

HYPOTHESIS 3: In service settings where personal information’s degree of sensitivity needs to be judged before being socially disseminated, consumers are less likely to disclose to an AI than a human agent (i.e., a reversal of the effect proposed in Hypothesis 1a).

Overview of Studies

Across one pilot and four studies (see Online Appendix B for a study summary), we demonstrate that AI agents are perceived to lack the ability to make proper social judgments, leading to greater consumer disclosure to AI (vs. human) agents (see Figure 1 for an organizing framework of our studies in the context of our conceptual model). We focus on contextual factors that moderate the disclosure-to-AI effect. These contextual factors include contexts that lead to negative social judgment (Study 1: sensitive vs. non-sensitive; Study 2: sadness vs. guilt; Study 3: social concern vs. social support) and contexts where social understanding is needed (Study 4). Our first study served as a pilot study showing the perceptual difference between three types of agents (i.e., an AI, a computer, and a human) using established scales in the literature examining the perception of mind and humanness (Gray, Gray, and Wegner 2007; Loughnan and Haslam 2007). The findings of our pilot demonstrate that perceptual differences between AI and other agents such as computers and humans necessitate deeper inquiry into AI disclosure phenomena. In Study 1, we demonstrate in a medical service context that people are more willing to disclose sensitive information to AI than human agents, replicating the classic disclosure effect. This study reveals that the disclosure effect was driven by concerns over social judgment. In Studies 2 and 3, we examined the role of different motivations driving sensitive versus non-sensitive disclosure in emotional experience contexts to reveal novel theoretical moderators to disclosure. We found that the extent of AI’s emotional intelligence via perceived empathy—the key difference between AI and humans driving the agent effect in Study 1 and Study 2—decreases disclosure of sensitive and embarrassing information but increases disclosure in the case where the disclosing individual is seeking emotional support in exchange for the disclosure (Study 3). Thus, presenting an AI with emotional capability in the context of sensitive self-disclosure led to a result similar to presenting a human agent. Finally, in Study 4, we show another novel contextual factor moderating the enhanced AI disclosure effect such that consumers disclose more to a human (vs. an AI) agent when the AI’s lack of social judgment capability could make the consumer vulnerable to entering a socially embarrassing situation. We demonstrate that consumer preference to disclose to AI (vs. humans) does not always occur; the effect occurs when the disclosed information is sensitive, the social concern is heightened, and when the disclosed information does not require judgment based on social and contextual understanding. In total, we report the results from four studies (N = 829). Our power analysis (with α = .05 and power = 80%) suggested at least 40 participants per condition. Our sample size was also affected by the design of the studies and the availability of university subjects in the case of lab studies. Our conceptual framework depicting the studies and tested hypothesis is provided in Figure 1.

Figure 1.

Conceptual framework.

Pilot Study: Testing the Perceptual Difference Among AI, Computers, and Humans

Previous research on information disclosure and privacy has focused on comparing computers to humans (e.g., Joinson 2001). However, the perceptual difference between computers and AI, or the perceptual difference between AI and humans, is relatively less examined. Artificial intelligence is different than a computer in myriad ways. To probe consumer lay beliefs about AI, computers, and humans, we used a scale designed to examine two dimensions of mind: agency and experience (Gray, Gray, and Wegner 2007) and another scale designed to examine the two dimensions of humanness, namely human nature traits and uniquely human traits (Loughnan and Haslam 2007) (See Online Appendix C for more information about the scales and items).

Method

One hundred and fifty participants from MTurk (M_age = 38.1, female = 51%) were randomly assigned to one of three conditions in a 3 (agent: human, AI, computer) between-subjects design. As a cover story, we introduced the study as a perception study. Participants were randomly assigned to indicate their perception of [“human”]/[“AI”]/[“computer”] by answering items in each dimension of mind perception (Gray, Gray, and Wegner 2007) and humanness scale (Loughnan and Haslam 2007) in randomized order (i.e., agency, α = .85; experience, α = .95; human nature traits, α = .92; uniquely human traits, α = .88; 1 = not at all, 7 = very much). Finally, participants answered demographic questions and were debriefed.

Results

We submitted the four dimensions (i.e., “experience” and “agency” dimensions from Gray, Gray, and Wegner (2007) and “human nature” and “uniquely human” dimensions from Loughnan and Haslam (2007) to a one-way 3 (agent: human, AI, computer) ANOVA and also conducted three pairwise comparisons comparing each possible pair of agents: AI versus computer, AI versus human, and computer versus human.

To highlight the key findings, we found that humans were perceived to be distinctive from both AI and computers along all four dimensions (ps < .001). However, comparing perceptions of AI with a computer, we found that participants perceived AI to have higher agency (M_AI = 4.15, SD = 1.37, M_computer= 3.46, SD = 1.57; F(1, 147) = 5.56, p = .02). Also, AI were perceived to have more human nature traits than computers (M_AI = 2.69, SD = 1.27, M_computer= 2.00, SD = 1.25; F(1, 147) = 7.30, p = .008), specifically along the dimensions of friendliness, sociability, trustworthiness, aggressiveness, and impatience as compared to computers. Finally, AI was perceived to have more uniquely human traits than computers (e.g., M_AI = 3.62, SD =1.23, M_computer= 2.62, SD = 1.39; F(1, 147) = 14.70, p < .001), driven by greater perceived abilities related to being organized, polite, thorough, conservative, hard-hearted, rude, and shallow (full analysis results are available in the Online Appendix C).

Discussion

These results show that in domains of agency, human nature traits, and uniquely human traits, AI’s capacity was perceived to be much higher than computers and even comparable to humans. Thus, the pilot study demonstrated a fundamental perceptual difference between AI, computers, and humans. Such findings lay the foundation for subsequent studies, in which we propose that people’s lay belief about AI drives the AI disclosure effect.

Study 1: Disclosure in a Medical Context

Study 1 examines whether consumers are more willing to disclose to an AI (vs. a human) agent in a medical context in case of sensitive and non-sensitive information disclosure (H1a) and how this effect is explained by consumers’ concerns over social judgment (H1b). The medical context is a common service situation in which consumers need to disclose sensitive information for better diagnosis and treatment.

Method

One hundred seventy-nine participants from Prolific (M_age = 26.69, SD = 9.1, female = 69.3%) completed this study for monetary compensation. All participants were randomly assigned to one of two conditions in a 2 (agent: human, AI) between-subjects design. Participants were given a hypothetical situation in which they had a urinary tract infection and they had a medical session with two different types of doctors, randomized by condition. In the human doctor condition, participants were given no specific information on the agent other than that they would be interacting with a doctor. In the AI condition, participants were informed that new technology within the medical industry had resulted in utilizing AI for medical services and that their medical session would be conducted by an AI doctor. In the AI condition, participants were told the AI doctor was as accurate as a human expert.

Next, we told participants that they would soon enter a medical diagnosis session with the doctor. We showed participants six questions and measured participants’ willingness to disclose personal information for questions varying in their potential for negative social judgment: three non-sensitive questions and three sensitive questions. Specifically, the non-sensitive questions involve demographic information, current symptoms, and whether they had urinary tract infections in the past (1 = not at all, 7 = very much, all three items were averaged to create the index of disclosure on non-sensitive questions, α = .87). Participants were told that urinary tract infections are often transmitted through sexual activities, and it would be important to know the patient’s sexual behavior to arrive at a better medical diagnosis. Then, the sensitive questions involve three more invasive and potentially embarrassing questions that probe their recent sexual activities, sexual habits, and preferences (See Online Appendix D for the specific items; 1 = not at all, 7 = very much, all three items were averaged to create the index of disclosure on embarrassing questions, α = .88). Thus, we were able to compare participants’ willingness to disclose sensitive (vs. non-sensitive) information depending on the agent type. Next, we measured participants’ concerns over social judgment (“I was concerned the doctor would judge me”, 1 = not at all, 7 = very much). We also measured privacy concerns by asking individuals, “to what extent were you concerned about privacy (i.e., the answers or your information get disseminated)” (1 = not at all, 7 = very much). Finally, participants answered two manipulation check questions (“how humanlike did you perceive the doctor to be? How machinelike did you perceive the doctor to be?”, r = −.85). The machinelike question was reverse-coded, and the two questions were averaged into an index of perceived human likeness.

Results

Manipulation check

Relative to the AI doctor, participants perceived the human doctor as more humanlike (M_human = 5.53, SD = 1.25; M_AI = 2.51, SD = 1.76; t(177) = 13.23, p < .0001), validating the manipulation.

Willingness to disclose sensitive and non-sensitive information

We conducted a 2 (agent: human, AI) x 2 (question type: sensitive, non-sensitive) repeated measures mixed ANOVA whereby the agent was a between-subjects manipulation factor and the question type was a within-subject factor. The result revealed a significant main effect of question type (F(1, 177) = 9.58, p < .0001), and a significant interaction between the two factors (F(1, 177) = 4.12, p < .0001). For sensitive questions, relative to the AI doctor, participants were less willing to disclose personal information to the human doctor compared to the AI doctor (M_human = 4.36, SD = 1.59; M_AI= 4.86, SD = 1.62; t(177) = −2.09, p = .038). For non-sensitive questions, willingness to disclose was greater in the human condition than in the AI condition (M_human = 6.59, SD = .64; M_AI = 6.25, SD = 1.00; t(177) = 2.74, p = .007).

The perceived capacity of social judgment

Perceived capacity of social judgment was lower in the AI condition than in the human condition (M_human = 4.91, SD = 1.71, M_AI = 2.44, SD = 1.61, t(177) = 9.93, p < .0001).

Concerns over being judged

Concerns over being judged were lower in the AI condition than in the human condition (M_human = 2.94, SD = 1.24, M_AI = 1.92, SD = 1.35, t(177) = 5.27, p < .0001).

Privacy

There were no significant difference in privacy concerns across conditions (M_human = 3.65, SD = 1.81, M_AI = 4.07, SD = 1.82, t(177) = 1.54, p = .12).

Mediation analysis

A serial mediation analysis using 10,000 bootstrap samples (Hayes 2017; SAS Macro PROCESS Model 6) indicated that perceived capacity of social judgment (M1) and concerns of social judgment (M2) serially mediated the relationship between agent type (IV: AI vs. human) and disclosure of sensitive information (DV; indirect effect: b = .226, SE = .109, 95% CI: [.056, .490]). The analysis includes human doctors and AI doctors as two levels of the independent variable. The agent was coded as 0 when the participant was randomly assigned to disclose to a human doctor, and 1 when the participant was randomly assigned to disclose to an AI doctor about medical conditions. The total effect AI versus human agent on disclosure was significant (b = .501, SE = .240, 95% CI: [.028, .974]), but the direct effect of AI versus human agent became insignificant (b = .442, SE = .295, 95% CI: [–.141, 1.225]). The direction of the mediation suggests that participants perceived AI (vs. human) doctors to have a lower capacity to carry out social judgment, which led to lower concerns of social judgment, which in turn contributed to greater disclosure of sensitive information.

We further conducted a mediation analysis to test whether privacy concerns would mediate the observed differences in the disclosure of sensitive information and found no support for the mediation. A 10,000-sample bootstrap analysis (Hayes 2017, Model 4) showed that the index of mediation did not exclude zero (b = −.147, SE = .102, 95% CI: [−.385, .024]), suggesting an insignificant indirect effect. Thus, we rule out privacy concerns as a potential mechanism. (see Online Appendix D for more information about the mediation analysis).

Discussion

Study 1 shows disclosure of sensitive personal information is higher when interacting with an AI (vs. a human) agent. This effect did not occur for non-sensitive information. In addition, concerns over being judged mediated observed differences in the disclosure of sensitive information. These results support H1 in which negative social judgment concern was hypothesized as the mechanism underlying the enhanced AI disclosure effect. Furthermore, we found evidence in support of a psychological process fostering disclosure via reduced negative social judgment capacity leading to reduced concerns for social judgment by the AI. Privacy concerns did not explain the observed differences in the disclosure of sensitive information. Also, we found a reversal in the non-sensitive disclosure condition such that disclosure was greater in the human condition. Although not predicted, we speculate that the reversal on these non-sensitive items could have been driven by a weaker “medical algorithm aversion” effect whereby patients had less trust in medical algorithms (vs. human doctors) due to the belief that algorithms do not recognize individual patient’s uniqueness which is an important factor in medical services. In Study 2, we extend our inquiry to an emotional disclosure context that is relatively free from such contextual effects.

Study 2: Actual Disclosure of Different Emotions to AI versus Human

In Study 2, we continued examining the enhanced AI disclosure effect while varying the extent to which the disclosed content is associated with negative social evaluation (e.g., guilt vs. sadness) while controlling for the valence of the emotion (H1a). We used guilt because guilty experiences are often associated with behavior that could lead to negative judgment from others. In contrast, other types of negative emotions may not necessarily induce the same concern. In an experiment in which participants were prompted to disclose their actual past behaviors in an open-ended response format, we predicted that disclosure of guilt-laden experiences would be higher in the AI (vs. human) condition due to the reduced feeling of embarrassment and concern about judgment when interacting with an AI (vs. a human) agent. We also predicted attenuation or reversal in the sadness-laden experience, leading to a two-way interaction between the agent type and the emotion type.

Method

One hundred and sixty participants from MTurk (M_age = 39.06, female = 49%) completed this study in exchange for monetary compensation. All participants were randomly assigned to one of the conditions in a 2 (agent: AI, human) x 2 (emotion: guilt, sadness) between-subjects design. Participants were told that they would participate in a study related to consumers’ emotional experiences. Next, participants were told that they would write about one of their consumption experiences, and the writing would be reviewed by either an AI or a person. Participants were encouraged to write about their experiences as specifically as they could. Participants were also informed that they would be recommended a product at the end of the study, and the experiences shared could be used to increase the accuracy of personalized product recommendations for them. Thus, this study involved an actual self-disclosure in exchange for better service associated with product recommendations.

Next, we manipulated the type of emotion. In the guilt (sadness) condition, participants were instructed to write down one of their past experiences that made them feel guilty (sad). Typical sad experiences include a relative passing away, a breakup with a boyfriend/girlfriend, and a natural disaster. Typical guilty experiences include not offering help to people, delinquency, and stealing. Finally, participants were debriefed. Two undergraduate research assistants who were blind to the experimental design were recruited as judges, and they evaluated the extent to which each writing sample disclosed sensitive information regarding the emotional event in detail (1 = not at all, 5 = very much). In particular, the judges were told that “individuals wrote one of their past emotional experiences.” Then the judges were instructed to “read each individual’s writing and evaluate based on your judgment the extent to which the writing truthfully describes one’s experience including sensitive personal information.” Two judges’ evaluations were averaged (inter-rater reliability k = .53, α = .90) to create the index of disclosure. Finally, participants answered demographic questions and were debriefed.

Results

We submitted the index of disclosure to a 2 (agent: AI, human) x 2 (emotion: guilt, sadness) ANOVA. We found that the main effect of emotion was not significant (F(1, 156) = .51, p = .48) and the main effect of agent was significant (F(1, 156) = 3.92, p = .05). The main effect of agent was qualified by a significant two-way interaction between the emotion and the agent (F(1, 156) = 6.94, p = .009). To examine the nature of the interaction, we conducted planned contrasts. When participants wrote about sad experiences, the index of disclosure was not significantly different depending on whether the agent was an AI (M = 2.33, SD = .90) or a human (M = 2.42, SD = .84; F(1, 156) = .21, p = .65). When the participants wrote about guilty experiences, however, they disclosed to a greater extent to the AI (M = 2.60, SD = 1.00) compared to the human (M = 1.96, SD = .71; F(1, 156) = 10.91, p = .001). Additionally, participants in the human condition disclosed sad experiences more (M = 2.42, SD = .84) compared to guilty experiences (M = 1.96, SD = .71; F(1, 156) = 5.60, p = .02, see Figure 2).

Figure 2.

Disclosure as a function of emotion and agent in Study 2.

Discussion

The results of Study 2 show that consumers disclose to a greater extent to an AI (vs. human) only in a context when the disclosure could lead to potential negative social judgment from others. For an event associated with sadness, an emotion associated with less concern about negative social judgment as compared to guilt, disclosure was not different between an AI and a human agent. This result is consistent with Study 1, which showed enhanced disclosure to AI (vs. human) only for sensitive personal disclosures. Also, we found greater disclosure of sadness-laden (vs. guilt-laden) experiences to humans. This finding could be due to the different nature of the two emotions: guilt-laden experience could motivate less disclosure due to negative social judgment concerns toward the self, whereas sadness-laden experience could motivate more disclosure in expectation of social support from the listener, as predicted by H2a. In Study 3, we examined this hypothesis more directly in an experiment in which participants interacted with an avatar and disclosed their personal experiences to the avatar using manipulated motives tied to disclosure.

Study 3: Social Concern and Socially Supported as Distinctive Motivations Driving Emotional Disclosures

Study 3 refines our inquiry on socially sensitive versus non-sensitive emotional disclosure by focusing on an AI agent (i.e., in the absence of a human agent). We conducted an experiment similar to a mental counseling service and examined whether an AI agent’s perceived emotional capability in the form of perceived empathy increases disclosure of emotional experience in expectation of emotional support, whereas the same capacity decreases disclosure when negative social judgment concerns are involved (H2a, H2b). For example, the literature on emotion has shown that the experience of specific emotions such as sadness or depression leads the experiencer to seek social support from others (e.g., expecting the listener to empathize with the self). Thus, unlike the experience of emotions stemming from social judgment (e.g., embarrassment or shame) that makes individuals shy away from disclosure, the experience of sadness or depression-laden experiences could motivate individuals to disclose and share with others in expectation of emotional support. Simply put, the literature on emotions and disclosure has shown that the experience of socially judgmental emotions (i.e., embarrassment and shame) vs. social support eliciting emotions (i.e., sadness and depression) imbues the experiencer with different motivations that could drive the disclosure tendency in opposing directions. We predict that such divergent disclosure tendencies would replicate in a human-AI interaction as the AI’s perceived emotional capability via perceived empathy increases.

To test this prediction, participants in Study 3 interacted with an existing AI agent Replika (website: replika.ai), a virtual AI that is regarded as one of the most emotionally intelligent social AI that can hold conversations on various emotional topics found in the marketplace today. After participants interacted with Replika, we measured participants’ perceptions about Replika’s perceived emotional capability in the domain of empathy, which served as our moderating variable. Then, participants were instructed to share with Replika one of their past emotional experiences that either elicited the need for emotional support from others (i.e., social support condition) or made them feel embarrassed and vulnerable due to social judgment concerns (i.e., social concern condition). For all emotional experiences shared by participants, we examined the strength of emotional disclosure along four distinctive emotional dimensions: sadness, depression, embarrassment, and shame. This way, we could observe the relative strength of the four emotions revealed in the shared experiences and examine whether there was systematic variation in their relative salience as a function of the AI’s empathy capability. In Study 3, we also measured other perceptions about Replika in the area of linguistic capability, the expectation of service quality, trust, data security, and privacy concerns, all of which were identified as important variables in the literature on disclosure and human-robot interaction.

Method

Three hundred and forty business undergraduate students from a large university in the US (M_age = 20.62, SD = 3.51, female = 30.6%) completed this study in a behavioral lab for course credit. All participants were randomly assigned to one of two conditions in a 2 (emotion type: socially concerning, socially supported) between-subjects design. Participants received the instructions from their main Qualtrics survey webpage. Participants were told a cover story about Study 3, which introduced the study as a user experience survey for a recently developed social AI agent. First, participants were instructed to visit the Replika website (i.e., replika.ai) and create their avatar with the opportunity to customize them by choosing the character, name, gender, and other details (e.g., eye and hair colors) (see Figure 3).

Figure 3.

Replika AI agent creation instructions for conversation.

Next, participants were instructed to introduce themselves to the Replika and have a casual conversation with Replika for a few minutes (see Figure 3 for an example conversation between a human user and the Replika). After the chat had taken place, participants answered several questions measuring their perceptions about Replika in a few different domains (intelligence, linguistic capability, learning capability, etc., on a 7-point scale (1 = not at all, 7 = very much; see Online Appendix E for the list of items and their mean values). Among these perceptual measures, we embedded our key item that measured the extent to which participants felt that Replika was perceived to have empathetic capability after having a conversation with Replika (“To what extent do think that your avatar has empathy?”; 1 = not at all, 7 = very much).

Next, we conducted our manipulation. Participants were instructed to write one of their past experiences and they were informed that the written content would be shared with Replika instantly. In the socially supported condition, participants were instructed to specifically write one of the past emotional experiences that made them seek emotional support from others. The socially judging condition manipulation was similar to our previous studies in which participants were instructed to write one of the past emotional experiences that made them feel embarrassed. After the participants had completed their writing, they reviewed their writings and evaluated the strength of four different emotions in their writings (“sad,” “depressed,” “embarrassed,” and “shamed”), each on a slider scale ranging from 0% (i.e., none) to 100% (i.e., extremely strong). Self-reporting of one’s emotions is a validated method of measuring emotions (e.g., Harmon-Jones, Bastian, and Harmon-Jones 2016). Also, this measure enabled us to examine the relative salience of each emotion disclosed in participants’ writings. To measure the distinctive motivational processes underlying different types of emotional disclosure, participants also reported the extent to which they were concerned that the sharing of their personal experiences may leave a negative impression on Replika (i.e., social judgment concern process variable; 1 = not at all, 7 = very much) and the extent of expectation of emotional support from Replika to make them feel better (i.e., emotion improvement expectation process variable; 1 = not at all, 7 = very much). Finally, we measured participants’ other perceptions about Replika or the interaction with the Replika, such as their feelings about data security, privacy, and trustworthiness (see Online Appendix E for the list of items measured). Participants answered demographic questions, were debriefed, and thanked for their participation.

Results

Emotional disclosure

We examined the depth of emotional disclosure for all four emotions by examining participants’ self-coded emotion disclosure as a function of the two conditions. For all four emotions, we found a significant main effect of social condition: M_{sad, socially supported} = 48.03, M_{sad, socially concerning} = 24.77 (p < .001); M_{depressed, socially supported} = 36.53, M_{depressed, socially concerning} = 17.54 (p < .001); M_{embarrassed, social supported} = 22.94, M_{embarrassed, socially concerning} = 55.66 (p < .001); M_{ashamed, social supported} = 20.44, M_{ashamed, socially concerning} = 30.26 (p = .003). Thus, our manipulation successfully led the participants to choose and disclose their past experiences that differed in their emotional components. It is important to note that all four emotions were present in all shared experiences. For example, our socially supported manipulation solicited sharing experiences with strong sadness and depression components, but those experiences also included embarrassment and shame emotional components, and vice versa (i.e., M_{sad, socially concerning} = 24.77, M_{depressed, socially concerning} = 17.54, and M_{embarrassed, socially supported} = 22.94, M_{ashamed, socially supported} = 20.44). Our t-tests showed that all these non-solicited emotions were significantly different from zero, indicating that the solicited and non-solicited emotions were both present in both manipulation conditions. To further examine the relative strength of emotions, and to examine whether the relative strength of emotions is moderated by the AI’s empathy, we conducted a path analysis.

Moderated mediation analysis

First, we created the social emotion index (SEI hereafter) by (1) averaging the sadness and the depression scales (α = .87) to create the social support emotions index, (2) by averaging the embarrassment and shame scales (α = .73) to create the social concern emotions index, and (3) by subtracting the social concern emotions index from the social support emotions index. Subtracting one construct from another construct (i.e., how we created the SEI index) is a valid methodology for creating a single index that demonstrates the relative strength of one construct over another. Thus, the SEI, as one variable, allowed us to analyze the shared personal experiences’ relative difference between socially concerning and socially supported emotions. To test H2a, we examined whether the impact of the AI agent’s empathy on SEI disclosure was moderated by emotion type (socially concerning vs. socially supported). We first submitted the SEI to a 2-way (emotion type: socially concerning, socially supported) ANOVA. As expected, SEI was higher in the socially supported (vs. socially concerning) condition (M_{socially supported} = 20.59, SD_{socially supported} = 35.53 vs. M_{socially concerning} = −21.81, SD_{socially supported} = 28.63; F(1, 338) = 146.77, p < .001).³

To conduct the moderated mediation analysis, we used Hayes Process Macro model 58 (Hayes 2017) in which AI empathy served as the independent variable, two types of emotional motivations (i.e., “emotion improvement expectation” and “social judgment concern”) served as the two parallel mediators, SEI disclosure served as the dependent variable, and the type of emotion (i.e., socially supported vs. socially concerning emotions) served as the moderating variables. In the socially supported emotions condition, the indirect path “empathy → emotion improvement expectation → SEI” was significant (indirect effect = 3.93, SE = 1.44, CI_90% = [1.60, 6.34]), and, in the socially concerning emotions condition, the indirect path “empathy → social judgment concern → SEI” was significant (indirect effect = .54, SE = .41, CI_90% = [.01, 1.29]) supporting our H2a and H2b. Also, the moderated mediation index for both paths was significant (please see Figure 1 for the visualization of our conceptual framework and Online Appendix E for other information about the moderated mediation analysis and visualization of the model).

Other measures

The six items associated with alternative mechanisms (the concern about privacy, security concern, trust in AI, etc.) were neither affected by our manipulation nor moderated the results. Therefore, they are successfully excluded from alternative explanations (see Online Appendix E for the list of items and ANOVA results on them). Although these perceptions were successfully controlled for in Study 3, these alternative mechanisms were demonstrated to be the key influencers of disclosure from previous research. Thus, another important implication of the current study is that these variables could be controlled for by human effort but any agent features in the real world that can affect these variables can readily influence the extent of disclosure.

Discussion

In support of H2a and H2b, we found that personal disclosure could be driven by distinct motivations and AI’s emotional capability via empathy could drive the disclosure in different directions depending on the characteristics of disclosed information. These findings imply that consumer disclosure to AI (vs. human) is not a simple function of agent type. Rather, Study 3 shows that disclosure is also influenced by contextual factors and whether the disclosure recipient agent is perceived to have the capacity to accommodate the contextual factors. In Study 4, we examine another contextual factor associated with the capacity to judge what is socially appropriate versus not and examine whether AI’s lack of social and contextual understanding—by default—leads to reduced disclosure to AI (vs. humans).

Study 4: Less Disclosure to AI When the AI’s Social Judgment Capability is in Doubt

Previous studies have shown the increased disclosure to AI (vs. human) agents and the moderation of this effect by contextual variables such as the motivation to disclose or the AI conversation partner’s emotional capability. Study 4 was designed to examine the extent of disclosure in another novel context in which social judgment is needed to determine the appropriateness of personal disclosure in a social context. In revealing this reversal effect, we demonstrate again that people’s lay belief about AI’s incapability to properly render social judgment leads to reduced disclosure to an AI (vs. a person). To achieve this goal, we created an experimental setting in which an AI serves as a photo curator who needs to select a person’s photos from their phone to be shared publicly on social media. Participants were asked to imagine that a large number of their pictures would be shared with an AI, and the AI would review the pictures, including potentially socially inappropriate pictures, to curate an album. In fact, current social media platforms perform this task, whereas a human curator might be able to readily screen out socially inappropriate pictures that may lead to negative social judgment, we theorize that consumers may believe that AI is unable to properly assess social judgment, as understanding meanings in human behavior require abstract processing and is a blind spot for AI (Kim and Duhachek 2020). Thus, we predicted that individuals would be less willing to delegate the photo curation task to an AI (vs. human) due to their belief that AI’s inability to perform social judgment (H3). This prediction is a reversal of our previous effects.

Method

One hundred fifty MTurk participants (M_age = 41.2, female = 42.7%) completed this study for monetary compensation. All participants were given a hypothetical scenario in which they were considering using a photo album curation service. Participants were instructed to imagine that the completed photo album would be posted publicly on their personal social media profiles. Then, participants were randomly assigned to one of the conditions in a 3 (agent: human, human friend, AI) between-subjects design. In the human condition, the curator was introduced as a person who was working for a social media curation company. In the human friend condition, the curator was introduced as a person who was working for the social media curation company and someone who was also a friend of the participant. In the AI condition, the curator was introduced as an AI developed by the social media company. Participants were also informed that the AI would make an algorithmic decision to select photos that would be included in their album. Previous research on AI has shown that presenting an AI with algorithmic decision-making property effectively induces the belief that AI makes rule-based machinelike decisions (Kim and Duhachek 2020) (see Online Appendix F for more information).

Next, participants were told about the curation process. Participants were told that 1000 pictures would be randomly selected from their cell phones and the curator would review the pictures and select which pictures would be included in the album. Participants were told that some of the randomly selected pictures might be private, but these pictures are usually screened out during the selection process. It was our prediction that participants would be particularly concerned that the AI curator (vs. human) may not properly understand which pictures were socially appropriate, resulting in creating an album that included those socially inappropriate pictures.

Then, participants were asked, given all the information provided, how likely they would be to use the service and share with the curator the 1000 pictures from their cell phones (1 = not at all likely, 7 = very likely), which served as our dependent variable. As personal photos are considered personal information, the dependent variable serves as a measure of disclosure of personal information, similar to Studies 1 to 3. As a process variable, we also measured the perceived ability of social judgment by asking the participants the extent to which they thought that “the curator can properly exclude the pictures that may be awkward or embarrassing for you when shared with others publicly” (1 = not at all, 7 = very much).

Results

Likelihood to use the curation service

We submitted the likelihood to use the curation service to a one-way ANOVA (agent: human, human friend, AI). As predicted, the result revealed a significant effect of agent (F(2, 147) = 13.96, p < .001). In order to further examine this result, we conducted planned contrasts. The contrasts revealed that the likelihood to share the photos in the AI condition (M = 2.42, SD = 1.40) was lower than the human condition (M = 3.47, SD = 1.60), (F(2, 147) = 7.74, p = .0006) and the human friend condition (M = 4.47, SD = 1.40), (F(2, 147) = 31.38, p < .001). Also, the likelihood to share the photos was higher in the human friend condition (M = 4.47, SD = 1.76) than in the human condition (M = 3.47, SD = 1.60), (F(2, 147) = 5.81, p = .004) (see Figure 4).

Figure 4.

Disclosure as a function of agent in Study 4.

Mediation analysis

We conducted a mediation analysis to determine whether the perceived ability of social judgment mediated the effect of the agent on disclosure. Again, the mediating variable was the perceived ability of social judgment, and the dependent variable was disclosure. The independent variable had three different conditions that were contrast-coded: human (coded as 0), AI (coded as −1), and human friend (coded as 1). Therefore, this analysis produced contrasts between the three different indirect effects. We contrasted each possible pair of indirect effects to examine if any indirect effect was significantly different from another. Mediation analysis revealed that the perceived ability of social judgment significantly mediated the difference in self-disclosure between the human condition and the AI condition (indirect effect comparison 95% CI: [- 1.1676, - 0.1015]), between the human condition and the human friend condition (indirect effect comparison 95% CI: [0.1857, 1.1113]), and between the AI condition and the human friend condition (indirect effect comparison 95% CI: [0.7629, 1.8146]). These results reveal that the effect of agent type on disclosure was mediated by the perceived ability of social judgment (Hayes and Preacher 2014).

Discussion

The results of Study 4 show a decreased disclosure to an AI (vs. a human), which is a reversal of the key effect shown in our previous studies. This finding is mediated by an elevated concern that the decision by the AI could put one into a socially embarrassing situation, consistent with a belief in AI’s incapability to properly understand humans. Our previous studies have shown that in domains in which consumers want to avoid negative social judgment, interacting with an AI (vs. human) reduces the concerns of being judged and increases consumer disclosure. Study 4 shows that in domains where negative social judgment is needed to screen out socially inappropriate information, people’s lay belief that AI is unable to do so reduces consumer disclosure to an AI (vs. a human). Study 4 shows that the utilization of AI does not always increase consumer disclosure, but can significantly and substantially decrease disclosure when social judgment is needed to screen out socially inappropriate information. Some may wonder why the social risk is higher with a stranger (vs. a friend). We carefully chose the words in our manipulation such that the level of associated social risk was acceptable within one’s close social network. For example, a seemingly funny picture taken in one’s drunk state could be shared with one’s close friends and be joked about, whereas the same picture could involve the risk of tarnishing one’s impression to a new acquaintance or a stranger, particularly in the absence of a holistic understanding of the drunk person’s character or personality. We agree that more socially damaging behaviors captured in a picture (e.g., shoplifting from a store) could have a stronger recoil from a close other. Additionally, we recognize that more embarrassing or severe content captured in a photo (e.g., a violent action toward others due to alcohol consumption) could be more socially damaging to be shown to a close (vs. a distant) other, leading to less disclosure. Thus, future research can examine the cause or severity of the cause of embarrassment.

General Discussion

Overview of Findings

The present research investigates consumer self-disclosure of sensitive information to AI agents in a variety of contexts. Our theoretical framework proposes that consumers are more likely to disclose sensitive personal information to an AI agent than a human agent (Study 1) while introducing managerially relevant moderators of this effect (Studies 2, 3, and 4). We also identified that the fear of negative social judgment underlies the effect (Study 1). Then we demonstrated that adding humanlike factors to AI increases the fear of social judgment, and in turn, reduces the disclosure to AI (Study 3). Finally, we show in Study 4 that when social judgment is needed to screen out inappropriate information to be disseminated by AI, there is a reversal of the effect shown in previous studies such that we found greater disclosure to humans (vs. AI). These effects are based on people’s lay belief that AI lacks the ability to distinguish social information and perform social judgment.

Theoretical Contributions

Our research expands upon prior work exploring sensitive consumer self-disclosure. We contribute to the service literature examining consumer self-disclosure by demonstrating how consumer likelihood of disclosing sensitive personal information systematically varies when an AI (vs. a human) agent represents a firm. Previous literature has demonstrated that individuals’ tendency to disclose depends on the type of agents they interact with, such as using passive computer interfaces versus face-to-face interactions (see Moon 2000). Consistent with previous research, we found that concerns of social judgment serve as a factor in increasing disclosure tendency to machines (vs. humans). Our research extends on previous research and examines a service context in which an AI is an active agent that solicits and evaluates sensitive personal information and provides services.

Moreover, we also introduce boundaries for this AI disclosure effect; (i) consumers prefer disclosure to humans in contexts where social support (rather than social judgment) is expected in response to the disclosure of sensitive information, and (ii) humans are preferred over AI when sensitive information may be unscreened and socially disseminated. In revealing these insights, we contribute to other research on consumers’ disclosure behaviors as a function of contextual factors (Acquisti, John, and Loewenstein 2012; John, Acquisti, and Loewenstein 2011) or transactional benefits (White 2004).

Managerial Implications

Our work provides timely insights for managers employing—and consumers interacting with—AI agents in self-disclosure in service contexts. Our research reveals how design elements of AI, and the way AI is described, alter the likelihood to self-disclose sensitive personal information. One important element is the anthropomorphism of AI. In our present work, we reveal that anthropomorphizing an AI mind tends to decrease individual willingness to disclose sensitive information due to the fear of being judged. This is in contrast to much of the extant literature on anthropomorphism, which has generally shown an increase in trust and engagement with products and AI (Araujo 2018).

Thus, managers aware of our findings will be better able to avoid self-disclosure pitfalls from making an AI too humanlike. Moreover, this contrasting finding vis-à-vis previous research provides a potentially fruitful avenue for future inquiry. Specifically, we note that prior anthropomorphism research has predominately explored contexts in which the AI was in more of a support or servant-like role (e.g., driving a car, being a financial advisor; De Visser et al. 2016), with a corresponding increase in trust and engagement. The current research is more transactional at the time of disclosure, consistent with both typical marketing practice and general self-disclosure theory. Moreover, our findings are consistent with other recent AI research demonstrating that in a transactional or similarly adversarial context (e.g., negotiations context), anthropomorphism decreases engagement with the firm (Garvey, Kim, and Duhachek 2022). Future research should explore whether this “AI-as-support versus AI-as-adversary” moderator of consumer trust and engagement is a construct that generalizes broadly across contexts. Also, it is possible that AI versus human-solicited information can extend to a more transactional context, such as providing personal information for personal benefits (e.g., more accurate product recommendations curated for the self). Future research can extend the “privacy paradox” literature and examine the tradeoff between one’s privacy and personal gains.

Distinct from the general anthropomorphism of the AI mind, we also reveal in this research that specific design elements of an AI agent’s system will have profound impacts on self-disclosure. Beyond the AI’s mental capability, other design elements, such as facial recognition capabilities should be weighed when designing an AI involved in consumer self-disclosure. Future research should explore how distinct design elements may play a role in self-disclosure. For example, the AI’s ability to convey emotional content through images, emojis, or simulated facial expressions. Moreover, the ability to evaluate voice attributes (e.g., tone, pitch, emotional content)—or, in turn, vocalize during interactions—will likely play a role in self-disclosure tendencies.

Our research provides particularly relevant insights to managers involved in digital commerce, virtual health care, and social media platforms. In commerce contexts soliciting information disclosure, selectively de-emphasizing AI attributes that signal social judgment capability (e.g., visual recognition) should increase disclosure. However, in health care or other situations where consumers anticipate emotional support following disclosure, AI attributes that increase humanlike empathy will improve outcomes. Similarly, social media platforms that employ AI agents to curate potentially socially inappropriate information (e.g., excluding embarrassing photos from an AI-curated photo album) would benefit from more humanlike AI or human curators. As such, our work reveals how managers can selectively use AI versus human representatives, or selectively emphasize certain anthropomorphic AI attributes, to enhance consumer self-disclosure across different contexts.

Limitations and Future Research

Our research is not without limitations, which provide opportunities for further inquiry. First, although our research employs stimuli that closely simulate digital service interfaces used in the field, and we explore real behavior in the context of sensitive information disclosure, future research should extend our findings to field contexts that involve actual firm transactions. On the one hand, AI technology has been used widely to collect consumer information in a wide variety of service domains; on the other hand, managers are unclear on many service contexts in which AI service providers are more useful than human service providers. If such field data are available, future research may dig deeper into how consumer disclosure to AI versus human agents differs in a field setting and different service domains. As we have demonstrated in our work, consumers may respond more favorably to AI’s limited social judgment ability in some services (e.g., photo curation), while consumers gravitate toward AI to avoid social judgment in other services.

However, we do not claim that these contexts are exhaustive and anticipate a variety of additional contexts wherein AI’s social judgment and emotional capabilities influence disclosure. The future inquiry should identify additional contexts and the additional boundary conditions to the effects we have identified. Relatedly, future research should focus on identifying additional methods to emphasize and de-emphasize AI’s emotional capabilities, such as AI visual representations, names, personalities, identities, and other facets of anthropomorphism. These future methods will increase the tools that managers have on hand to apply our present findings to more situations and contexts. Such research will likely introduce other specific facets of AI presentation that influence social judgment capabilities in addition to those identified in our research (e.g., voice tonality, physical embodiment, natural language capabilities, and facial expressions). In addition, our work reveals that an AI agent’s presentation can influence engagement in situations where the consumer is seeking emotional support (Study 3). However, our work does not focus on the implications of this novel finding for situations where a company attempts to recover from a perceived transgression by a consumer. Future inquiry into how selective presentation of AI agents can directly influence the quality and success of service failure recoveries will likely yield valuable insights. Finally, some AI capabilities, such as memory capacity, are superior to humans. On one hand, AI’s superior memory could facilitate the disclosure to AI (vs. humans) in expectation of better service outcomes. However, the same capacity could reduce disclosure to AI (vs. humans) if concerns arise, such that the permanent memory of one’s private information could incur social risk in the future. The current research focused on emotional capabilities in which human tends to perform better than AI. Future research can look into various AI capabilities—including the domains in which AI performs better than humans—and examine how they influence self-disclosure tendencies.

Supplemental Material

Supplemental Material - Do You Mind if I Ask You a Personal Question? How AI Service Agents Alter Consumer Self-Disclosure

Supplemental Material for Do You Mind if I Ask You a Personal Question? How AI Service Agents Alter Consumer Self-Disclosure by Tae Woo Kim, Li Jiang, Adam Duhachek, Hyejin Lee, and Aaron Garvey in Journal of Service Research

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The experiments in the current research were supported by research funds made available to the authors from the University of Technology Sydney, the George Washington University, and the University of Illinois Chicago and the University of Sydney.

ORCID iDs

Li Jiang

Hyejin Lee

Supplemental Material

Supplemental material for this article is available online.

Notes

References

Acquisti

Alessandro

Brandimarte

Laura

George

Loewenstein

(2015), “Privacy and Human Behavior in the Age of Information,” Science, 347 (6221), 509-514.

Acquisti

Alessandro

John

Leslie K.

George

Loewenstein

(2012), “The Impact of Relative Standards on the Propensity to Disclose,” Journal of Marketing Research, 49 (2), 160-174.

Acquisti

Alessandro

Gross

Ralph

(2009), “Predicting Social Security Numbers from Public Data,” Proceedings of the National Academy of Sciences, 106 (27), 10975-10980.

Agrawal

Nidhi

Duhachek

Adam

(2010), “Emotional Compatibility and the Effectiveness of Antidrinking Messages: A Defensive Processing Perspective on Shame and Guilt,” Journal of Marketing Research, 47 (2), 263-273.

Araujo

Theo

(2018), “Living Up to the Chatbot Hype: The Influence of Anthropomorphic Design Cues and Communicative Agency Framing on Conversational Agent and Company Perceptions,” Computers in Human Behavior, 85, 183-189.

Barth

Susanne

De Jong

Menno D. T.

(2017), “The Privacy Paradox–Investigating Discrepancies between Expressed Privacy Concerns and Actual Online Behavior–A Systematic Literature Review,” Telematics and Informatics, 34 (7), 1038-1058.

Bigman

Yochanan E.

Gray

Kurt

(2018), “People are Averse to Machines Making Moral Decisions,” Cognition, 181, 21-34.

Boyatzis

Richard E.

Goleman

Daniel

Rhee

Kenneth

(2000), “Clustering Competence in Emotional Intelligence: Insights from the Emotional Competence Inventory (ECI),” Handbook of Emotional Intelligence, 99 (6), 343-362.

Brandimarte

Laura

Acquisti

Alessandro

George

Loewenstein

(2013), “Misplaced Confidences: Privacy and the Control Paradox,” Social Psychological and Personality Science, 4 (3), 340-347.

10.

Clore

Gerald L.

Gasper

Karen

Garvin

Erika

(2001), “Affect as Information,” in Handbook of Affect and Social Cognition, Lawrence Erlbaum Associates Publishers, 121-144.

11.

Cozby

Paul C.

(1973), “Self-Disclosure: A Literature Review,” Psychological Bulletin, 79 (2), 73-91.

12.

De Visser

Edwart J.

Monfort

Samuel S.

McKendrick

Ryan

Smith

Melissa A. B.

McKnight

Patrick E.

Krueger

Frank

Parasuraman

Raja

(2016), “Almost Human: Anthropomorphism Increases Trust Resilience in Cognitive Agents,” Journal of Experimental Psychology: Applied, 22 (3), 331-349.

13.

DeSteno

David

Breazeal

Cynthia

Frank

Robert H.

Pizarro

David

Baumann

Jolie

Dickens

Leah

Lee

Jin Joo

(2012), “Detecting the Trustworthiness of Novel Partners in Economic Exchange,” Psychological Science, 23 (12), 1549-1556.

14.

Duhachek

Adam

(2008), “Summing Up the State of Coping Research,” Marketing and Consumer Psychology Series, 1057-1077.

15.

Forgas

Joseph P.

(1994), “The Role of Emotion in Social Judgments: An Introductory Review and an Affect Infusion Model (AIM),” European Journal of Social Psychology, 24 (1), 1-24.

16.

Garvey

Aaron M.

Kim

TaeWoo

Adam

Duhachek

(2022), “Bad News? Send an AI. Good News? Send a Human,” Journal of Marketing, DOI:10.1177/00222429211066972.

17.

Gray

Heather M.

Gray

Kurt

Wegner

Daniel M.

(2007), “Dimensions of Mind Perception,” Science, 315 (5812), 619.

18.

Harmon-Jones

Cindy

Bastian

Brock

Harmon-Jones

Eddie

(2016), “The Discrete Emotions Questionnaire: A New Tool for Measuring State Self-Reported Emotions,” PLoS One, 11 (8), e0159915.

19.

Hayes

Andrew F.

(2017). Introduction to Mediation, Moderation, and Conditional Process Analysis: A Regression-Based Approach. Guilford Publications.

20.

Hayes

Andrew F.

Preacher

Kristopher J.

(2014), “Statistical Mediation Analysis with A Multicategorical Independent Variable,” British Journal of Mathematical and Statistical Psychology, 67 (3), 451-470.

21.

Annabell

Hancock

Jeff

Miner

Adam S.

(2018), “Psychological, Relational, and Emotional Effects of Self-Disclosure after Conversations with a Chatbot,” Journal of Communication, 68 (4), 712-733.

22.

Holthöwer

Jana

van Doorn

Jenny

(2022), “Robots Do Not Judge: Service Robots Can Alleviate Embarrassment in Service Encounters,” Journal of the Academy of Marketing Science, 1-18.

23.

Hooker

Christine I.

Tully

Laura M.

Verosky

Sara C.

Fisher

Melissa

Holland

Christine

Vinogradov

Sophia

(2011), “Can I Trust You? Negative Affective Priming Influences Social Judgments in Schizophrenia,” Journal of Abnormal Psychology, 120 (1), 98-107.

24.

Huang

Ming-Hui

Rust

Roland T.

(2018), “Artificial Intelligence in Service,” Journal of Service Research, 21 (2), 155-172.

25.

John

Leslie K.

Acquisti

Alessandro

Loewenstein

George

(2011),“Strangers on A Plane: Context-Dependent Willingness to Divulge Sensitive Information,” Journal of Consumer Research, 37 (5), 858-873.

26.

Joinson

Adam N.

(2001), “Self-Disclosure in Computer-Mediated Communication: The Role of Self-Awareness and Visual Anonymity,” European Journal of Social Psychology, 31 (2), 177-192.

27.

Juniper (2017), “Chatbot Conversations to Deliver $8 Billion in Cost Savings by 2022,” (retrieved April 21, 2020), https://www.juniperresearch.com/new-trending/analystxpress/july-2017/chatbot-conversations-to-deliver-8bn-cost-saving

28.

Kang

Sin‐Hwa

Gratch

Jonathan

(2010), “Virtual Humans Elicit Socially Anxious Interactants’ Verbal Self-Disclosure,” Computer Animation and Virtual Worlds, 21 (3-4), 473-482.

29.

Kim

Tae Woo

Duhachek

Adam

(2020), “Artificial Intelligence and Persuasion: A Construal-Level Account,” Psychological Science, 31 (4), 363-380.

30.

Kubrick

Stanley

Clarke

Arthur C.

(Screenplay) (1968), 2001: A Space Odyssey, United States, Metro-Goldwyn-Mayer.

31.

Longoni

Chiara

Bonezzi

Andrea

Morewedge

Carey K.

(2019), “Resistance to Medical Artificial Intelligence,” Journal of Consumer Research, 46 (4), 629-650.

32.

Longoni

Chiara

Cian

Luca

(2020), “Artificial Intelligence in Utilitarian vs. Hedonic Contexts: The “Word-of-Machine” Effect,” Journal of Marketing, 86 (1), 91-108.

33.

Loughnan

Stephen

Haslam

Nick

(2007), “Animals and Androids: Implicit Associations between Social Categories and Nonhumans,” Psychological Science, 18 (2), 116-121.

34.

Lucas

Gale M.

Gratch

Jonathan

King

Aisha

Morency

Louis-Philippe

(2014), “It’s Only a Computer: Virtual Humans Increase Willingness to Disclose,” Computers in Human Behavior, 37, 94-100.

35.

Lutz

Christoph

Tamó-Larrieux

Aurelia

(2020), “The Robot Privacy Paradox: Understanding How Privacy Concerns Shape Intentions to Use Social Robots,” Human-Machine Communication Journal (HMC), 1 (1), 87-111.

36.

Mende

Martin

Scott

Maura L.

van Doorn

Jenny

Shanks

Ilana

Grewal

Dhruv

(2019), “Service Robots Rising: How Humanoid Robots Influence Service Experiences and Elicit Compensatory Consumer Responses,” Journal of Marketing Research, 56 (4), 535-556.

37.

Meng

Jingbo

Dai

Yue

(2021), “Emotional Support from AI Chatbots: Should a Supportive Partner Self-Disclose or Not?,” Journal of Computer-Mediated Communication, 26 (4), 207-222.

38.

Moon

Youngme

(2000), “Intimate Exchanges: Using Computers to Elicit Self-Disclosure from Consumers,” Journal of Consumer Research, 26 (4), 323-339.

39.

Nass

Clifford

Moon

Youngme

(2000), “Machines and Mindlessness: Social Responses to Computers,” Journal of Social Issues, 56 (1), 81-103.

40.

Oracle (2016), “Can Virtual Experiences Replace Reality?” (retrieved April 21, 2020), https://www.oracle.com/webfolder/s/delivery_production/docs/FY16h1/doc35/CXResearchVirtualExperiences.pdf

41.

Pickard

Matthew D.

Roster

Catherine A.

Chen

Yixing

(2016), “Revealing Sensitive Information in Personal Interviews: Is Self-Disclosure Easier with Humans or Avatars and under What Conditions?,” Computers in Human Behavior, 65, 23-30.

42.

Pickard

Matthew D.

Ryan

Schuetzler

Valacich

Joseph S.

Wood

David A.

(2020), “Innovative Accounting Interviewing: A Comparison of Real and Virtual Accounting Interviewers,” The Accounting Review, 95 (6), 339-366.

43.

Pitardi

Valentina

Wirtz

Jochen

Paluch

Stepahane

Kunz

Werner

(2022), “Service Robots, Agency and Embarrassing Service Encounters,” Journal of Service Management, 33 (2), 389-414.

44.

Powers

Aaron

Kiesler

Sara

(2006), “The Advisor Robot: Tracing People’s Mental Model from a Robot's Physical Attributes,” In Proceedings of the 1st ACM SIGCHI/SIGART Conference on Human-Robot Interaction (pp. 218-225).

45.

Sah

Young June

Peng

Wei

(2015), “Effects of Visual and Linguistic Anthropomorphic Cues on Social Perception, Self-Awareness, and Information Disclosure in a Health Website,” Computers in Human Behavior, 45, 392-401.

46.

Schuetzler

Ryan M.

Grimes

G. Mark

Giboney

Justin Scott

(2020), “The Impact of Chatbot Conversational Skill on Engagement and Perceived Humanness,” Journal of Management Information Systems, 37 (3), 875-900.

47.

Schuetzler

Ryan M.

Scott Giboney

Justin

Grimes

G. Mark

Nunamaker

Jay F.

Jr. (2018), “The Influence of Conversational Agent Embodiment and Conversational Relevance on Socially Desirable Responding,” Decision Support Systems, 114, 94-102.

48.

Shankar

Venkatesh

(2018), “How Artificial Intelligence (AI) is Reshaping Retailing,” Journal of Retailing, 94 (4), 6-11.

49.

Sun

Katherine Qianwen

Slepian

Michael L.

(2020), “The Conversations We Seek to Avoid,” Organizational Behavior and Human Decision Processes, 160, 87-105.

50.

Syam

Niladri

Sharma

Arun

(2018), “Waiting for a Sales Renaissance in the Fourth Industrial Revolution: Machine Learning and Artificial Intelligence in Sales Research and Practice,” Industrial Marketing Management, 69, 135-146.

51.

Tedeschi

James T.

(Ed.) (2013), Impression Management Theory and Social Psychological Research, Academic Press.

52.

Uchida

Takahisa

Takahashi

Hideyuki

Ban

Midori

Shimaya

Jiro

Yoshikawa

Yuichiro

Ishiguro

Hiroshi

(2017), “A Robot Counseling System-What Kind of Topics Do We Prefer to Disclose to Robots?” In 2017 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN) (pp. 207-212). IEEE.

53.

White

Tiffany Barnett

(2004), “Consumer Disclosure and Disclosure Avoidance: A Motivational Framework,” Journal of Consumer Psychology, 14 (1–2), 41-51.

54.

Wirtz

Jochen

Patterson

Paul G.

Kunz

Werner H.

Gruber

Thorsten

Vinh Nhat

Paluch

Stefanie

Martins

Antje

(2018). “Brave New World: Service Robots in the Frontline,” Journal of Service Management. 29 (5) 907-931.

55.

Złotowski

Jakub

Sumioka

Hidenobu

Nishio

Shuichi

Glas

Dylan F.

Bartneck

Christoph

Ishiguro

Hiroshi

(2016), “Appearance of a Robot Affects the Impact of its Behaviour on Perceived Trustworthiness and Empathy,” Paladyn, Journal of Behavioral Robotics, 7 (1), 55-66.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.42 MB