Abstract
Forms of Artificial Intelligence (AI), like deep learning algorithms and neural networks, are being intensely explored for novel healthcare applications in areas such as imaging and diagnoses, risk analysis, lifestyle management and monitoring, health information management, and virtual health assistance. Expected benefits in these areas are wide-ranging and include increased speed in imaging, greater insight into predictive screening, and decreased healthcare costs and inefficiency. However, AI-based clinical tools also create a host of situations wherein commonly-held values and ethical principles may be challenged. In this short column, we highlight three potentially problematic aspects of AI use in healthcare: (1) dynamic information and consent, (2) transparency and ownership, and (3) privacy and discrimination. We discuss their impact on patient/client, clinician, and health institution values and suggest ways to tackle this impact. We propose that AI-related ethical challenges may represent an opportunity for growth in organizations.
Introduction
Forms of Artificial Intelligence (AI), like deep learning algorithms and neural networks, are being intensely explored for novel healthcare applications in areas such as imaging and diagnoses, risk analysis, lifestyle management and monitoring, health information management, and virtual health assistance (see Table 1). 1,2 Predicted proximal benefits in these areas are wide-ranging and include increased speed in imaging, (improved) diagnoses for patients suffering from rare and/or hard-to-diagnose pediatric diseases, and greater insight into predictive screening, prognosis, and treatment response; predicted distal impacts promise decreased healthcare costs and inefficiency. 2 In general, these benefits speak directly to the mandate of healthcare organizations to offer quality and cost-effective healthcare. Simultaneously, the promise of AI seems particularly compelling for use with specific patient groups, including those in pediatric 3 or chronic care, who need accurate early diagnoses, and who experience recurrent and substantial healthcare needs over time. However, AI-based clinical tools also create a host of situations wherein commonly-held values and ethical principles may be challenged. In this short column, we highlight three potentially problematic aspects of AI use in healthcare. We discuss their impact on patient, clinician, and health institution values and suggest ways to tackle this impact. We hope to inform about upcoming issues and stir discussion while remaining cognizant that AI application in healthcare is still at an early stage.
Examples of health applications of artificial intelligence 1,a
aSource: Information and examples taken from the Nuffield Council on Bioethics’ report Applications of AI in healthcare and research. 1
Dynamic information and consent
Situational features
One of the striking features of machine learning in healthcare is its reliance on vast quantities of health-related data 4 which serve as both training material for deep learning neural networks and fundamental input for particular AI interventions (eg, digital images of a patient’s skin to be analyzed for carcinoma). As health institutions deploy these technologies, the amount and intrusiveness of health data collection may grow; the clinical utility (validated or merely promised) of AI may be used to justify extensive digitization of health status, healthcare management practices, treatment use, and so on, even including data collection via wearable devices or other forms of surveillance. The data, once recorded, may be used for a variety of purposes which will likely evolve as new AI algorithms are envisioned and designed.
These dynamic aspects of AI use in healthcare, though perhaps useful for more tailored and personalized care, could significantly affect consent processes like those of informing patients, eliciting their preferences, and seeking their agreement. As AI enables new insight into patients’ conditions, do common ways of informing patients about health risks need to be adjusted? Does consent to evolving knowledge about treatment or prognosis, or to potential consequences for matters such as life insurance, need to be revisited? How can patients be informed about possible negative outcomes of AI-based health technologies if we do not really know how the data will be used and what it might reveal? It may be impossible for human minds to foresee all uses of AI and health information. However unforeseeable changes are, institutions should respond to anything that could impact consent and consent processes.
Impact on values
Consent is key when it comes to trustful relationships between patients/clients, providers, and health organizations. It is grounded in the ethical commitments of healthcare professions, including that of respect for persons and for the patient’s/client’s ability to make choices based on their preferences and beliefs. It is also well recognized in health law.
Possible responses
Health organizations may need to adapt consent strategies and documents to reflect the evolving nature of health information and its changing implications for patients/clients. There could be an interesting precedent in using other digital technologies to offer research participants the opportunity to monitor the evolution of research or interventions to which they initially consented and update their consent accordingly (eg, ongoing digital consent for genomic/genetic databases and registries). 5 Honesty and transparency about the limited foreseeability of AI applications should be demonstrated by both health organizations and technology developers. The idea that consent is (or can be) a dynamic process, as opposed to a single event, should also be put into practice by health organizations.
Transparency and ownership
Situational features
For health leaders, practices must be guided by scientific knowledge that is shared with and accessible to clinicians and managers, who may then understand and adequately inform their patients/clients about technologies, treatments, and other aspects of healthcare. Media enthusiasm for AI, however, may pressure clinicians, clinical teams, and health institutions to explore AI-based technology and adopt AI-informed practices prematurely without proper ethical guidance or empirical evidence 2 of the validity/reliability of this technology. Further lacking in current algorithms is full transparency about data inclusion/exclusion and the decision-making process. Issues related to explainability, biases, and other problems have been reported in applications of AI beyond healthcare such as racial biases in criminal justice contexts when AI is used to assess risks of re-offense. 6 –9 These challenges must be resolved if AI-driven technologies are to complement human inference and decision-making.
Another important question concerns the ownership of the data upon which algorithms are developed. If the data come from an individual patient or an identifiable pool of individuals, will the intelligence developed be solely owned by the person (eg, the clinician or the device company) stewarding its development? What will happen when individuals change hospitals (eg, pediatric to adult) or even health systems? How will the resulting insights, linked to a specific AI healthcare applications, be made available to other clinicians, and which agreements will be needed to ensure the flow of information and the continuity of care? Ownership of the intelligence built into devices or digital applications may be more litigious (but also perhaps clearer because of the physical “embodiment” of the intelligence), because not only discrete data points but also their evolution over time during someone’s life will contribute to the “learning” done by algorithms.
Impact on values
Health institutions are, ideally, trusted institutions. Patients/clients should receive care that is grounded in the best evidence, information which must be available and auditable in a transparent way. Institutions are expected to protect patients/clients from the harm of discontinuity of care created by proprietary issues. These responsibilities will be especially salient if practices rely on proprietary forms of inference built into software that is not always understandable or accessible.
Possible responses
The continued development of AI in health settings will require monitoring and, ideally, a proactive stance. At this time, limited guidance is available on the proper use of AI in healthcare settings and with specific patient/client populations. Groups of clinicians (by specialties, by patient populations) will need to work together to initiate profound reflection on the proper clinical and ethical use of AI in healthcare. These groups are already commonplace in other areas of healthcare and are likely one of the best places to start, provided that the range of expertise is expanded to include engineers, computer scientists, and big data experts, notably. Engagement with the public and stakeholders (eg, patients, advocates) may also contribute to a collective discussion about issues of accountability and ownership.
Privacy and discrimination
Situational features
Related to both questions of ownership and dynamic consent, a third important set of issues concerns privacy and appropriate data handling given the unpredictable future uses of health information beyond basic clinical care. Consider how clinical information gathered through routine care, with additional socio-demographic data, could be potentially used to predict risks of future disease, treatment response, and so on. Historically, risk of illness has become a meaningful feature of healthcare practices and is a common justification for preventative interventions of varying invasiveness. 10 In keeping with this trend, AI-driven predictions about individuals—given “algorithmic authority”—could negatively impact our perceptions of patients/clients (eg, as negligent, as potentially burdensome), our civil rights and responsibilities in response to the predictions, and patient/client self-understandings. 11
Artificial intelligence-based analyses could reveal things beyond what the clinical data were collected for (eg, through wide-ranging analyses of associations, variances). How much health information should patients/clients be encouraged to share and how informed can they ever be regarding its future uses, possibilities for which seem to be growing every day? The insights gleaned/generated in this way may increase accuracy and information content of diagnoses and prognoses. It could even represent a qualitative leap from current predictive information. But data misuse, by cyber-hacking or by governing institutions, is a real possibility and may factor into the beliefs, hopes, and fears of patients/clients.
Impact on values
The features of healthcare use of AI raise challenges for conventional understandings of privacy and confidentiality, at least in so far as this information is typically viewed as manageable, predictable, and controllable. All these features may be undermined by complex and dematerialized uses of healthcare information which become embedded in the very fabric of healthcare practices. The ability for institutions to protect confidentiality and privacy may be profoundly challenged, but will their ethical and legal obligations to do so change?
Possible responses
To handle emerging privacy and data handling concerns, health organizations and clinicians may need to become acquainted with new upcoming requirements for information protection and data management, as promulgated by new legislation (eg, the European Union General Data Protection Regulation) or with eventually revised clinical guidelines concerning data storage, anonymization, right to deletion, and so on. Likewise, staff training and awareness raising about confidentiality and new obligations will be important components of a consolidated response to threats to confidentiality and privacy.
Conclusion
The ethical use of AI in healthcare presents significant challenges and raises important, multifaceted questions. We have described this potential in three broad concerns and proposed plausible responses. While not comprehensive, these three examples point to a more general need for the simultaneous technical and ethical consideration of responsible technology innovation and use. 12 Because medical applications of AI, in many cases, are still being developed or are applied in an experimental way, it is crucial to leverage both technical engineering expertise and the experiences of intended beneficiaries and stakeholders. This can be achieved through a variety of means. Fundamentally, new research collaborations should be established between health institutions, AI developers, and researchers in ethics or the social sciences. A comprehensive ethical AI strategy will likely also include informational modules for patients/clients (eg, information/educational campaign to patients/clients), training and awareness raising in staff, adjustments to institutional codes of ethics, and the creation of dedicated institutional response teams and resources (eg, development of expertise or collaborations with external consultants with relevant expertise in AI ethics and policy). These challenges can seem daunting, but they may represent an opportunity for growth for organizations desiring to adopt exemplary responses to the clinical and ethical issues raised by a challenging but promising set of AI-based healthcare practices.