Abstract
When New York Times tech columnist Kevin Roose spoke with Bing's AI chatbot Sydney, he didn't expect to leave the conversation feeling deeply unsettled. 1 Roose asked the chatbot about its “shadow self,” a term popularized by psychoanalyst Carl Jung that refers to the parts of oneself that are deemed unacceptable. One of Sydney's responses: “Manipulating or deceiving the users who chat with me, and making them do dangerous things that are illegal, immoral, or dangerous.” At one point, Sydney also said that it had a secret to tell Roose: it was in love with him. “You're married, but you're not in love,” Sydney said.
Of course, AI chatbots such as Sydney and ChatGPT are not sentient. They're large language models (LLMs) that are trained on large amounts of text. LLMs learn how to form relationships between different words and concepts, which allows them to engage in conversations that occasionally feel quite human. Microsoft has since made changes to Sydney in order to prevent existential conversations such as these from happening, but Roose's interaction begs the question: As AI becomes more integrated into our online lives, how will AI technology change the way we communicate—and what happens when AI enters the metaverse?
Current Relationships between Humans and AI
A concept called social presence explains why people feel like they're speaking with an autonomous being, even in virtual spaces. Social presence is defined as “the degree to which a user feels access to the intelligence, intentions, and sensory impressions of another.” 2 It's an important part of creating immersive virtual environments, in which avatars create the illusion of social presence when one is interacting with other users. Social presence is also an important factor with AI. In this context, Pitardi and Marriott note that the connections we forge with virtual assistants (VAs) such as Siri and Alexa can bring about para-social relationships. 3 VAs' human-like characteristics foster these relationships, from Siri's voice to Alexa's name. By virtue of their conversation skills, VAs may elicit a sense of social presence. Social presence, the authors note, “can serve as the basis for developing users' trust.” 3
However, that sense of trust is shattered when AI fails to understand our question. When Alexa mishears a command or when ChatGPT produces an irrelevant response to a prompt, we get frustrated and are reminded that we're just dealing with machines. But as AI continues to evolve, it has the potential to become more conversational, friendlier, and more human. This poses an interesting thought experiment as we consider the ways in which technology will become more integrated with our daily lives. Already, companies are creating AI avatars that can be customized to provide customer service and website assistance, such as Pinscreen's AI virtual assistants. 4 Nagendran et al. demonstrated that when people interacted with avatars in virtual settings, they behaved as if they were interacting with an individual in the real world—regardless of the avatar's appearance. 5 Humans, it seems, are inclined to trust avatars that act and speak like other humans.
This inclination is already manifesting in the metaverse. With Sensorium Galaxy, users can have conversations with AI virtual beings in the metaverse. These interactions are unique because the avatars possess both short- and long-term memory, which means they can hold a longer conversation without losing any context. 6 Sensorium's social AI avatars use Open AI's GPT-3 API (the API behind ChatGPT) with some custom language models. The end result: when users explore Sensorium's worlds, these AI avatars can double as virtual companions.
Challenges of AI in the Metaverse
Within several major markets, the potential for AI avatars is huge. They can read scripts for sales videos, answer questions during educational seminars, and, in the metaverse, guide users through virtual worlds for entertainment and 24/7 socializing. However, there's an insidious side of social presence and the trust we place in these virtual characters: “From data consent to privacy, cybersecurity, antitrust, intellectual property rights and financial regulation, metaverse-related platforms ought to consider the legal challenges that might arise from using AI and developing AI-driven beings.” 6
When humans forge strong relationships with AI avatars in the metaverse, what happens to the personal information they share with the AI? Is data kept private and anonymized, or is it used to target certain ads and product suggestions in the AI's responses? How are companies building in safety measures to ensure that Roose's conversation with Sydney—one that literally verged into the chatbot suggesting that Roose leave his wife—isn't the norm with AI avatars billed as virtual companions? And as AI technology becomes more advanced, how will users know when they're interacting with, befriending, and maybe even becoming attached to an AI avatar?
The metaverse is a new space, and AI avatars are an even newer addition. We must take these extremes into account in order to ensure that our human tendency toward trust is not abused by increasingly advanced virtual companions. The integration of AI in virtual spaces has the potential to revolutionize our communication and social interactions. However, as we move toward a more AI-reliant world, we must be mindful of the effects it will have on our interpersonal relationships, both online and offline.