“You Are Not Alone”: The Allure and Limitations of Artificial Intelligence in Serious Illness Communication

Introduction

Health care providers routinely have difficult conversations when caring for patients with serious illnesses. These conversations include breaking bad news, advance care planning, and setting goals of care. In these conversations, clinicians need to discuss poor prognosis, explore goals and priorities, and make appropriate recommendations in a clear and compassionate manner. Skilled communication can lead to various positive outcomes, such as decreased anxiety, higher patient and family satisfaction, and increased adherence to medical care. Yet, many clinicians feel uncomfortable having these conversations, in part because they do not receive formal communication skills training, or fear that frank conversations will negatively impact patients and families. ¹

Palliative care specialists, considered experts in the area of serious illness communication, often get consulted to lead these conversations. However, there is a paucity of palliative care specialists in the United States relative to the need. Generalists and other nonpalliative care specialists will be expected to facilitate the vast majority of serious illness conversations, and thus need the appropriate training to do so. Specialized communication training typically involves extensive practice (and failure) to develop precise language, tone, and cadence. The right combination of clarity and empathy needs to be demonstrated for clinicians to show mastery in this skill. Trainees often find this to be uncomfortable and feel ill-prepared initially. ² One solution to mitigate the shortage of highly trained palliative care specialists would be to employ artificial intelligence (AI) tools to assist with serious illness communication. However, this potential benefit must be balanced against the practical and ethical risks that AI poses in sensitive situations.

The Growing Role of AI in Medicine

AI is not the future of medicine; it is already here and showing promising outcomes. ³ Advances in computing—deep learning, natural language processing—have enabled AI to streamline patient intake, improve medication education, and even assist in radiology interpretation. ⁴ While using AI to assist in analyzing a chest X-ray may seem intuitive, communication-focused specialties such as psychiatry would seem more immune to AI incursions. However, chatbots have been successfully deployed as virtual therapists. AI chatbots have been shown to reduce depression, and in some cases may be preferable to patients who would otherwise be too ashamed of human judgment. ⁵ As language models in AI chatbots become more sophisticated—less rote and more nuanced—can an AI system provide clinicians with the “right words” to use in high-stakes, serious illness conversations?

The Allure of AI in Serious Illness Communication

A recent study by Ayers et al. ⁶ showed that AI chatbots are already capable of producing empathetic responses to patient questions, in some cases even more effectively than their physician counterparts. Does this represent a sea change in the way we should approach serious illness communication?

ChatGPT (Generative Pretrained Transformer) is a groundbreaking AI tool built on a technology known as a “large language model.” It works by sifting through a vast database of text to understand patterns in natural human language. Those patterns help ChatGPT predict sequences of words to create responses that are relevant and meaningful in countless scenarios. Current iterations of ChatGPT have received additional fine-tuning and reinforcement from human trainers. ⁷ We gave ChatGPT the following prompt (version 3.5, queried on February 4, 2023): “share bad prognosis in the style of a palliative care doctor.” This was its response:

I'm sorry to inform you that based on the results of your tests and assessments, it appears that your illness has progressed to a point where it is no longer responding to treatments. The prognosis is not favorable, and I want to emphasize the importance of focusing on comfort and quality of life in these remaining days. My team and I are here to support you and your loved ones through this difficult time, and to ensure that you are as comfortable and pain-free as possible. Please know that you are not alone, and we are committed to being with you every step of the way.

Overall, this is a reasonably good construction of the words one might begin to use when sharing difficult information with prognostic significance. If chatbots are already this eloquent, it is easy to imagine a push to develop patient-facing tools that would assist clinicians with difficult conversations.

Limitations of AI in Serious Illness Communication

Words matter. Communication training emphasizes the importance of avoiding jargon, euphemisms, and clichés. In the ChatGPT's response to the prompt, one might identify the phrase “the prognosis is not favorable” as too jargony and amorphous. However, words are not everything. Even if the language were more exact, that would not be sufficient to communicate bad news with empathy. The flow of “The prognosis is not favorable, and I want to emphasize the importance of focusing on comfort and quality of life” can be unsettling; moving onto a plan to focus on comfort immediately after delivering a poor prognosis can be jarring for patients and does not give space for the emotional weight of hearing bad news.

Similar to poetry, the spaces between and around the words are immensely important in serious illness conversations. Nonverbal communication and the use of silence are essential components in these conversations, as is attending to the emotional content that is expressed by patients, family, and caregivers. Palliative care clinicians are trained to recognize emotional cues and respond empathically, thereby deepening trust and advancing difficult conversations. These are interactions that would be missed in a simple text exchange. It is conceivable that AI systems could learn the right time to pause, or interject with an empathic response in patient-facing care situations. Yet there remains an understandable concern among providers that delivering bad news via technology would be seen by patients as dehumanizing, and potentially negatively impact patient–provider relationships. ⁸

AI-generated text poses other risks. AI does a remarkable job of producing relevant phrases by sifting through billions of data points, but even statistical accuracy approaching perfection can sometimes miss the mark. The responses are not vetted, and a chatbot might generate an inaccurate or insensitive comment. Providers should be wary of using AI to write a script that can be read verbatim. Again, this misses the point of serious illness communication—an over-reliance on getting the “perfect words” should not supplant the importance of developing a therapeutic presence with patients and families.

Lastly, the notion of outsourcing empathetic communication in the setting of serious illness could be considered emotional abandonment. The relationship between provider and patient is never more important than in a moment of fear and uncertainty. These moments require increased presence and awareness by providers; even fine-tuned technology is liable to create an emotional gulf at a sensitive time.

A Potential Role for AI in Communication Skills Training

The rapid rise of AI poses a dilemma for medicine: it is unrealistic to pretend that AI will not play an integral role, but we should be cautious about how quickly applications are embraced and integrated into clinical care without significant assurances that it will not violate primum non nocere—first, do no harm. Furthermore, to make a prediction about the future role of a technology that is evolving so quickly is a form of hubris we should be careful of. For now, it is possible to harness the linguistic power of AI without abandoning clinicians' emotional and moral responsibility to their patients.

ChatGPT cannot replace skilled communicators in palliative medicine, but it can be a useful educational tool, especially for novice learners or those not familiar or comfortable with serious illness communication. A clinician might feel comfortable with active listening and empathic responses, but might not know which words to use. They may feel more confident initiating a difficult conversation with a few suggested phrases and a logical structure to guide the conversation. AI systems are already able to supply such phrases and more in an infinite range of scenarios and thus can be useful to enhance communication skills, at the very least enhance providers' level of confidence before initiating such conversations. Furthermore, AI-powered simulators could help learners hone their communication skills with nuanced, fluid, real-time feedback.

The most striking phrase from ChatGPT's response is “please know that you are not alone.” This is an important phrase to use after delivering bad news to signal support and our commitment to nonabandonment. The irony is that if this phrase is delivered by a chatbot instead of a human provider, then maybe you are alone. This further underscores the risk for patient abandonment posed by technological intermediaries.

Before developing patient-facing tools, AI systems can serve as communication aids in assisting clinicians to improve their comfort in initiating difficult conversations. Nonpalliative care specialists and novice communicators may benefit from having structured language, simulations, and guidance generated from AI systems. While such tools can help clinicians struggling to find the right words, additional training and mentoring will continue to be necessary for human providers to master effective and compassionate communication practices in serious illness.

Funding Information

No funding was received for this article.