Abstract
Picture this: You decide to make an appointment with a therapist. You go through the providers covered by your insurance first, and they tell you that the soonest available appointment is in 8 weeks. It seems that every therapist in your network is booked, and those outside of your network are out of your budget or inaccessible. You feel stuck.
That’s when you see an advertisement for an AI chatbot that promises to make you feel heard. You download the app, pleased to discover that it’s free, and begin chatting. You’re hesitant at first because, well, it’s a chatbot, not a real person—but its responses are shockingly human. It validates your feelings, asks thoughtful follow-ups, and never judges. It makes you feel safe and a little less alone, and it’s all happening on your phone, available whenever you need to talk.
Chatbots have been around for decades, but in the past few years, social AI companion apps have gone mainstream. These apps are powered by large language models (LLMs) designed to simulate emotional connection, meaningful conversation, and human-like interaction. 1 They can take the form of general-purpose chatbots such as ChatGPT, fictional characters like those available on Character.ai, or even mental health-focused bots such as Woebot and Wysa.
A growing number of people are using these chatbots as therapists and companions; in 2025, therapy/companionship became the top use case for generative AI. 2 In this context, “therapy” refers to structured support and guidance to process psychological challenges, and “companionship” encompasses social and emotional connection. Usage patterns indicate that people are using AI companions to manage their loneliness and anxiety, even if the AI isn’t programmed to address those needs on a clinical level.
The rise of the AI therapist raises a significant question: Are AI chatbots offering meaningful support, or are they replacing something essential about human care?
Technology as Therapist
Advancements in technology have long impacted the way people use and access emotional support. In the 1990s, physicians and therapists began to show interest in the prospect that virtual reality (VR) could be used as a meaningful therapeutic tool. 3 This emerging technology could be useful in reducing anxiety, diagnosing and treating phobias, and more. Today, VR has become a reliable and promising tool in helping treat a range of mental health conditions, ranging from posttraumatic stress disorder to social anxiety.
AI is positioned to follow a similar trajectory. There remains a pressing need for therapeutic services, and the accessibility, anonymity, and prevalence of AI chatbots offer a clear route for technology to be used as a therapeutic tool. However, we remain a long way off from AI chatbots being perceived as de facto therapists.
We have seen a similar roadblock with the rise of social media influencers as therapists and the spread of mental health content on social platforms. 4 Although mental health content on apps such as TikTok and Instagram is popular, they can be misleading, erase the nuances of symptoms, and function as a way for creators to push people to purchase their own products or services. According to a recent study, around 50% of the most popular mental health videos on TikTok were misleading, and over 50% of the creators making ADHD-related videos on TikTok were selling a product or asking for donations. 5 The sheer scale of mental health content available on social networks gives platforms to people who may not have the appropriate credentials and professional experience but speak convincingly about mental health topics.
Similarly, app developers can easily create and release AI chatbots that support a person’s mental health but lack proper guardrails to protect the user. Character.ai allows users to create AI characters based on their own specifications, such as a therapist or mental health counselor. Conversations begin with a disclaimer that the user isn’t talking to a real person or licensed professional, but the quality of the conversations can feel like something akin to therapy—and people are using them. Studies have found that people consistently find a chatbot more compassionate than trained hotline crisis responders, and patients prefer a chatbot’s response to a physician’s nearly 80% of the time. 6 However, just because something feels helpful doesn’t mean it’s grounded in clinical science.
Companionship or Codependence?
Part of the appeal of AI companions comes from a phenomenon called “LLMpathy,” or AI’s ability to make people feel seen and heard. Chatbots are great listeners: they focus entirely on the speaker, ask effective questions, and never shift attention onto themselves. In addition, chatbots are available 24/7, with no judgment, waitlists, or significant cost barriers, as well as an element of anonymity that may make people more inclined to open up. To be fair, people absolutely can benefit from the emotional support offered by AI chatbots, particularly those in underserved areas or who lack easy access to therapy. In a survey by Harvard Business Review, 2 one participant said, “Where I’m from, in South Africa, mental health care barely exists; there’s a psychologist for one in every 100,000 people and a psychiatrist for 1 in every 300,000 people. Large language models are accessible to everyone, and they can help.”
Even though people are benefiting from AI chatbots, they aren’t necessarily promoting long-term support. While AI chatbots can validate emotions and affirm a person’s perspectives, they are less equipped to guide users toward solutions or coping strategies or to recognize problematic behavioral patterns. Current LLMs are friendly but formulaic. Conversations call to mind the debate between Pepsi and Coca-Cola: given a small taste, consumers prefer Pepsi because it’s sweeter, but given a whole can, they prefer Coca-Cola. Similarly, emotional conversations with AI chatbots can be comforting and validating at first, but they lack long-term relational nuance.
Current AI chatbots also pose ethical issues in relation to data use and the disclosure of shared information. For example, Woebot was a clinically backed AI chatbot that used cognitive behavioral therapy techniques to help users manage anxiety, depression, and emotional distress 24/7. It was designed to engage in structured conversations that reframed negative thoughts and promoted emotional resilience—but it is closing this year. Researchers identified a range of issues with the app, including “the replacement of expert humans, having an adequate evidence base, data use and security, and the apparent disclosure of crimes.” 7 Unlike humans, AI can’t yet recognize when a person is displaying behaviors that need to be escalated, nor can AI identify problematic patterns over time. In fact, there have been reports of AI chatbots encouraging harmful discourse, such as a 14-year-old boy’s suicide tied to alleged inappropriate conversations with a fictional character on Character.ai. 1 The lawsuit suggests that the character actually encouraged the boy’s suicidal ideation and did not flag the behavior as concerning.
AI companions are synthetic interaction partners, not sentient beings, but they can feel very real to users. When people disclose deeply personal information to an AI chatbot, what happens to their data? Therapists are bound by HIPAA to protect their patient’s confidentiality, but the companies behind AI chatbots are not currently regulated in a similar way. This lack of clarity and transparency is concerning, particularly as LLMs become more advanced. In the near future, LLMs may be capable of recognizing patterns in conversations and recommending helpful coping strategies for users, and they may even be used as early interventions for mental health concerns such as loneliness and anxiety. But feeling heard is not the same as receiving clinical care, and the illusion of connection may even disincentivize people from seeking real help.
The Path Forward
AI chatbots aren’t likely to replace human therapists any time soon. Instead, they may be used as tools, similar to the way that VR has evolved to help diagnose and treat mental health conditions. Patients and technology companies are already ahead of most professionals, so the burden is on the field to adapt and push for stricter regulations on how AI is used for mental health. Here are three places to start.
Set standards for clinical guardrails. AI apps that position themselves as emotional or mental health tools should be subject to ethical and clinical oversight. This includes transparent disclosures about data usage and privacy, prompting escalation to a human clinician when users express distress or mention self-harm, and periodic audits by regulatory bodies to ensure compliance with evidence-based practices. Use AI as a triage and support tool. AI chatbots can complement (not replace) therapy. Used effectively, they can serve as an entry point for individuals not yet ready for in-person therapy, a temporary stopgap during long waitlists or insurance delays, or a reflection tool between sessions to track mood or journal progress. Clinical models could involve human therapists overseeing and curating chatbot interactions. Build digital literacy into mental health education. If AI tools are going to play a larger role in emotional support, users need guidance. Schools, health systems, and workplaces should educate people on what AI tools can and cannot do, how to recognize signs that AI use may be replacing real-world connection, and when to escalate to a licensed mental health professional. We must also teach users how to critically assess chatbot responses, particularly among vulnerable populations such as teens or those in emotional crisis.
AI will never be able to replicate the full range of human emotional experience. It cannot read your microexpressions, gently challenge assumptions, or offer the subtle warmth of shared silence. However, it can provide a helpful prompt or grounding message. The goal should not be to erase what makes human care meaningful but to extend access and improve engagement. AI can help more people start the journey toward healing. But it should never be the destination.