Regulation of Artificial Intelligence in Drug Discovery and Health Care

2.1. Drug discovery

The area of machine learning has brought about an essential change in the research and development sectors of the pharmaceutical industry. It can be used to develop prediction tools which can learn properties related to different structures. Ineffective drug targets are a significant reason for the failure of late-stage clinical trials, due to which few drugs make it to the drug approval process finally. However, AI is becoming a powerful tool to expand the drug discovery process and pathway. As a result, the development of AI platforms and processes has fuelled AI-based drug development.

BioXcel Therapeutics uses novel AI technology to identify and develop new medicines in the field of immune-oncology and neuroscience. ³ Berg, a clinical-stage biotechnology company, uses AI to map diseases to accelerate the discovery and development of breakthrough medicines. ⁴ XtalPi is increasingly utilizing the AI-based Intelligent Digital Drug Discovery and Development (ID4) Platform for reinventing drugs. The ID4 platform provides accurate predictions on the physicochemical and pharmaceutical properties of small-molecule candidates for drug design, solid-form selection, and other critical aspects of drug development. ⁵ Based on convolutional neural networks, the AI technology of Atomwise makes it convenient for the researchers to pursue hit discovery, lead optimization, and toxicity predictions with unparalleled precision and accuracy. ⁶ Deep Genomics uses AI technology to find drug candidates with desirable properties. Through the use of the Project Saturn platform, Deep Genomics evaluates, in silico, several billion molecules against one million targets. ⁷ With the help of biomedical data, BenevolentAI provides an end-to-end drug discovery process which covers early discovery to late-stage clinical development. ⁸ These examples are representative of how companies and start-ups are focusing AI solutions on the drug discovery and development pathway. In addition to the growing use of AI in drug discovery, there has also been a remarkable growth in AI's other health care applications.

Artificial intelligence in medicine was introduced during the early 1960s. Expert systems such as the MYCIN (developed by Stanford University), INTERNIST (developed by University of Pittsburgh), CASNET (by Rutgers Research Resource), MUNIN, Pathology Expert Interpretative Reporting System, and many others represent the development of AI systems for medical-related work (Table 1). Medical AI technologies have been growing over the years. With an increase in the growth as AI as a discipline, newer systems are being created with the help of fuzzy logic, neural networks, expert systems with integrated intelligent decision support systems (DSS) to machine learning (ML), deep learning, and computer vision.

Although the area of medicine was well established, the use of AI technology in the earlier days was considered unconventional. Clancey and Shortliffe defined its use in medicine:

Medical artificial intelligence are primarily concerned with the construction of AI programs that perform diagnosis and make therapy recommendations. Unlike medical applications based on other programming methods, such as purely statistical and probabilistic methods, medical AI programs are based on symbolic models of disease entities and their relationship to patient factors and clinical manifestations.” ⁹

Various subsets of AI have brought notable achievements to the health care sector in the last few years. Its efficiency has gone far beyond mere bioactivity predictions, as it is now also being used provide solutions to problems related to drug development and discovery process. The flexibility of the neural networks enables deep learning technology to distinguish itself from other machine learning methods. Companies such as IBM, TwoXAR, Insilico Medicine, Atomwise, Berg, GE, Google, and many others are using deep learning and machine learning technologies to achieve various objectives in the health care sector. The current use of artificial intelligence can be seen in various use in the health care sector, like telemedicine, diagnostics, and AI-assisted surgeries (Fig. 1).

OPEN IN VIEWER

FIG. 1.

Application areas of AI in health care.

2.2. Diagnostics

The predominant use of AI in diagnostics has been with radiology. Capitol Health Limited, an Australian radiology service company, and Enlitic, a medical start-up, announced their first collaboration on the end-to-end transformation of medical diagnostics using deep learning technologies for radiologists and health care providers in 2015. ¹⁰ In 2017, a team at Karolinska Institutet launched Stockholm3 test, a blood-based prostate cancer diagnostic test which is currently being used in clinical practice in Sweden, Norway, Finland, and Denmark. In early January 2020, a team of researchers in Sweden discovered that an AI platform could be used to diagnose prostate cancer in tissue samples. ¹¹ PathAI and Bristol-Myers Squibb are together working on the predictive promise of AI for a reliable and reproducible AI-powered pathology interpretation in immuno-oncology. Assessing PD-L1 expression on immune cells by traditional approaches can be challenging, and it is expected that the AI application would be useful. ¹² AI genomics company Freenome uses molecular biology and machine learning to detect early-stage cancer and precision oncology with the use of its multiomics platform. ¹³ Researchers at Beth Israel Deaconess Medical Center (BIDMC) utilize artificial intelligence-enhanced microscopes for scanning for harmful bacteria (such as E.coli and Staphylococcus) in blood samples at a faster rate than is possible using manual scanning. ¹⁴

While global solutions are being developed, the use of AI in health care specifically in India is also growing. Microsoft India has several initiatives in health care, including the Microsoft Intelligent Network for Eyecare (MINE) project working with the government of Telangana and its Rashtriya Bal Swasthya Karyakram. This AI platform has been adopted to reduce avoidable blindness. ¹⁵

Artelus, a start-up, uses deep learning algorithms for detecting diabetic retinopathy (DR). Niramai, a Bangalore-based start-up, began in 2016 and focused on providing ML-based software to detect breast cancer at a much earlier stage. Qure.ai uses deep learning algorithms for radiology, CT scans, and MRIs. Sigtuple, a Bangalore-based start-up founded in 2015, uses AI, deep learning, and image processing tools for various uses, such as analysis of peripheral blood smears, urine microscopy, semen and fundus screening, OCT scans, and chest X-rays. ¹⁶ Google Health has developed a deep learning–based model diagnosing anaemia based on noninvasive retinal screening, rather than a traditional blood test. ¹⁷

2.3. Precision medicine

It is interesting to note several health care AI collaborative frameworks in recent times. The ability to determine outcomes accurately in the health care area is mainly dependent on expert evaluation. Due to the extensive data that needs to be evaluated, not only is it tedious, but timely health care also cannot be ensured. Several products have been developed for several diseases, including those of the lifestyle disease segment to increase speed of delivery.

Both Phytel and Explorys (since 2015) have added IBM Watson for Genomics and Oncology respectively to the supercomputer's portfolio. Microsoft in 2017 collaborated with University of Pittsburgh Medical Center to advance AI and released new products including HealthVault Insights, Microsoft Genomics, a chatbot, and Project InnerEye for radiotherapy. Google AI has released DeepVariant, a deep-learning technology for improving the accuracy of genome sequencing, as an open-source product in 2017. The open-source DeepVariant is more accurate than its original version for single nucleotide polymorphisms (SNPs) prediction. ¹⁸ Healthi, a Bangalore-based digital health and wellness start-up, uses machine learning, personalization algorithms, and predictive analytics for personalized health suggestions. ¹⁹ Oncostem Diagnostics, a start-up founded in 2011, aims to provide personalized treatment for cancer based on machine learning algorithms. ²⁰ These are some of the representative precision medicine applications of AI.

2.4. Electronic health records

Documentation and information retrieval through an electronic health record (EHR) has made health care data available for use in different settings. However, there have been concerns about the extended time which physicians must dedicate to record keeping. AI applications have been developed to expedite and enhance clinical documentation and the retrieval of EHR data. Health institutions, as well as companies, are working together to develop AI applications in this area. EarlySign collaborated with Geisinger's Steele Institute for Health Innovation to develop and deploy machine learning-based digital technologies to identify patients at risk of high-burden diseases. ²¹ Researchers at Kaiser Permanente developed a machine-learning algorithm that could help prevent HIV transmission using electronic health records. ²² Cleveland Clinic researchers developed an artificial neural network that analyzes lung cancer patients' EHRs and medical scans to determine the most effective radiation dosage for therapy. ²³ In order to address physician fatigue concerning EHRs, Google Cloud has partnered with an AI assistant, Suki, for efficient use of time as well as enhance value related to EHR information.

3.1. Evolution of AI regulation

The basic proposition is that we should develop only “Beneficial AI,” which is created safely and for the benefit of the society. Two types of measures to build Beneficial AI have been suggested. Extrinsic measures necessitate AI designers to adopt beneficial designs. Such designs may be in the form of bringing in certain constraints, incentives, and compliances. Intrinsic measures include the cultivation of social norms and the framing of communications. The effect of intrinsic measures may impact extrinsic measures. ³² Developers of AI algorithms face the risks, including dataset shift, casual fitting of confounders, and unintended discriminatory bias. Further, challenges including generalization to new populations and the potential unintended negative consequences of new algorithms on health outcomes are other aspects which need thoughtful consideration and regulation. ³³

Another vital area in the regulatory pathway is understanding AI, or more accurately, AI-related, liability. The possible grounds for liability can be conceptualized in several ways. First is respondeat superior liability where the master is liable on behalf of his/her/its servants—and in this context, the AI would be the servant. Second is vicarious liability, where liability for an AI's behavior will be vicariously imposed on a person on whose behalf the AI acts. The third is a strict liability, which absent a statute cannot be imposed except in relation to ultrahazardous activities. ³⁴ Responsible development involves ensuring that AI systems are safe, secure, and socially beneficial. To reduce the potential harm from an AI system, it needs to be constructed responsibly. Regulatory compliance can also increase cost considerations for companies.

Market forces alone may not always incentivize companies to invest the appropriate amount into ensuring their products are safe. In real-world scenarios in which markets may not function correctly and information asymmetries exist, incentives for companies to invest sufficiently in product safety typically come from three sources: market forces, liability law, and the industry or government regulation. ³⁵ In case of the role of AI in health care, there is a need for effective regulation. AI technologies rely on various algorithms—programs which are created with the help of health care data, which assist in making recommendations and predictions. However, the algorithms themselves are often too complicated for their reasoning to be understood or even stated explicitly.” ³⁶ Algorithms in precision medicine guide care by predicting patient risks, making accurate diagnoses, selecting drugs, and even prioritizing patients to preserve or assign limited health resources. Patients may recover compensatory and punitive damages from physicians, health care organizations, pharmaceutical companies, and medical device manufacturers if they are injured as a result of the party's failure to meet judicially accepted standards. ³⁷ It is quite settled now that physicians (and their employers: e.g., hospitals or clinics) may be held liable for any harm caused by faulty diagnosis. Machine learning software-based products and processes will need to be closely scrutinized to identify the scope of regulatory compliance. In a product liability scenario, the claimant goes uncompensated unless it can be shown that the defendant was at fault: that is, they acted unreasonably. ³⁸

3.2. Comparative perspectives on the regulation of AI

According to Article 12 of the United Nations Convention on the Use of Electronic Communications in International Contracts, a person (natural or an entity) on behalf of whom a program was created must, ultimately, be liable for any action generated by the machine. This reasoning is based on the notion that a tool has no will of its own. ³⁹ Hence, the AI tool cannot be held legally liable for the damage it has caused. In the U.S., the privacy of patient information is mandated under the Health Insurance Portability and Accountability Act's (HIPAA's) Privacy Rule. HIPAA creates a relatively complex set of permitted and restricted uses of protected health information. FDA requires manufacturers to report only a limited number of risks their devices present and actions taken to minimize the vulnerability. The autonomous behavior of AI technology complicates the issue of accountability. When AI devices decide, their decision is based on the collected data, the algorithms they are based on, and what they have learned since their creation. The 21st Century Cures Act clarified the scope of FDA regulatory jurisdiction over software used in health care, specifying that a medical device is an instrument or other tool intended for use in the diagnosis of disease or other conditions, or in the cure, mitigation, treatment, or prevention of disease, in man or other animals, or intended to affect the structure or any function of the body of man or other animals. FDA categorizes medical devices into three classes, according to their uses and risks, and regulates them accordingly. The higher the risk, the stricter the control. Class III is the category which includes the devices involving the highest risk.

Compared to computer-aided detection systems, FDA clearance is even harder to obtain for an AI system that does not need a radiologist's supervision and which cannot be compared to previous medical devices used as replacements for radiologists. FDA has categorically differentiated the algorithms designed to automate the performance of clinical tasks as medical devices, by either being embodied in traditional medical devices or under Software as a Medical Device (SaMD). FDA has already approved some machine vision imaging applications. ⁴⁰ The U.S. report Preparing for the Future of Artificial Intelligence, released by the White House Office of Science and Technology Policy (OSTP) in 2016, focused on innovation and research on AI. Later, the FUTURE of AI Act of 2017 established the Federal Advisory Committee on Development and Implementation of AI. In 2019, the U.S. joined the Organisation for Economic Co-operation and Development (OECD) AI Recommendation, the first intergovernmental standards for AI, which list five complementary value-based principles and recommendations. ⁴¹

In the case of the EU, the General Data Protection Regulation (GDPR) will be applicable in each member state and will also apply to organizations outside the European Union that process data about EU citizens. This recognizes four areas of compliance: 1.

Checklist interpretation: automating a compliance checklist and offering guidance on critical sections.

The EU countries signed a Declaration of Cooperation on AI in 2018, even though some of the countries have their own AI policy. The European Commission established a High-Level Expert Group to gather expert input and develop guidelines for AI ethics, which will build upon a statement by the European Group on Ethics in Science and New Technologies. The Group published ethical guidelines for trustworthy AI in April 2019. The Joint Declaration on the EU's legislative priorities for 2018–2019 additionally named data protection, digital rights, and ethical standards in artificial intelligence and robotics as a priority. ⁴³

The term “medical device” is applied to any instrument or other tool, including any software, intended by the manufacturer to be used on human beings for the purpose, among others, of diagnosis, prevention, monitoring, treatment, or alleviation of disease (as per Article 1(2) of Directive 93/42/EEC). The regulatory framework has been reformed by the new Medical Devices Regulation (MDR) and the new In Vitro Diagnostic Medical Device Regulation (IVDR). Both came into force on May 25, 2017; however, the MDR will apply beginning May 26, 2020 while the IVDR will apply beginning May 26, 2022. Unlike the directives, these regulations take effect immediately in member states. ⁴⁴ The Massachusetts Institute of Technology (MIT) Media Lab and FDA signed a memorandum of understanding, “Health 0.0,” to foster AI and ML research for computational medicine and clinical development as well as to develop an accompanying regulatory framework to improve health outcomes for patients. Life sciences, biotechnology, foundations, universities, and patient advocacy groups are parts of this effort. ⁴⁵

In the case of the UK, the UK House of Commons Science and Technology Committee released a report on AI in 2016. The report provides details of the capabilities, areas of focus, problems and consequences of AI. ⁴⁶ In 2018, five principles relating to ethical aspects of AI were enunciated:

The Committee has suggested building checks so that there are approaches developed to audit AI data. ⁴⁷

In the case of India, a task force on AI for India's Economic Transformation was constituted in 2017. An interministerial National AI Mission has been recommended to act as a nodal agency for coordination of all AI-related developments. Four committees have been identified by the Ministry of Electronics and Information Technology (MeitY) for creating a policy framework for AI. These committees are the Committee on Platforms and Data for AI; Committee on Leveraging AI for identifying National Missions in Key Sectors; Committee on Mapping Technological Capabilities, Key Policy Enablers, Skilling, Re-skilling, R&D; and the Committee on Cybersecurity, Safety, Legal, and Ethical Issues. ⁴⁸ Several ministries and governmental agencies are working on the use of AI in various applications. For instance, the Department of Agriculture Cooperation and Farmer's Welfare is working with private sectors for monitoring crops, identifying pests, and systematic management of the weather-based crop. The Department of Revenue is using AI and data analytics for the administration of taxes. ⁴⁹ NITI Aayog, a governmental think tank, in its 2018 report emphasized on the use of AI in health care. ⁵⁰ The state of Kerala, through the “e-Health Project” has collected and stored electronic health records of its population, enabling access to patient information and access to health care at government hospitals. Several domestic, as well as international, health care players are working on AI solutions for patient care. ⁵¹ In 2018, Wadhwani AI, India's first research institute for development of artificial intelligence solutions for the social good, was set up with the focus in health care, education, agriculture, and infrastructure in order to accelerate social development. ⁵²

3.3. Regulatory approval of AI-based tools—current perspectives

The growth of AI can be seen mostly in diagnostic prediction such as radiology, ophthalmology, pathology, etc. One of the first FDA clearances for such AI was related to ophthalmology. ⁵³ A device called IDx-DR, which is produced by IDx LLC, was first of its kind to get approval from FDA in 2018. It provides a screening decision without the need for a clinician also to interpret the image or results. IDx-DR will be used to detect diabetic retinopathy. This device was reviewed under new FDA regulations designed to fast-track some devices, which are considered as low-to-moderate risk. ⁵⁴ Since then, there have been many approvals by FDA (Table 2).

Recently, in January 2020, a drug created with the help of AI for treating obsessive-compulsive disorder is in the clinical trials phase. This was developed by UK-based AI start-up Exscientia, in partnership with Japanese pharmaceutical company, Sumitomo Dainippon Pharma, within one year. ⁵⁵

All the approvals by FDA are under the Digital Healthcare Innovation Action Plan, which consists of an innovative Software Precertification (Pre-Cert) Pilot Program under which many AI-based digital health projects are already completed. For AI-based tools, a manufacturer application has to go through Section 510(k) clearance and Premarket Approval (PMA) (under the Food, Drug, and Cosmetics Act) or De Novo classification. As yet there is no directory to find out the time it takes for approval of AI algorithms as a medical device.