Abstract
For over 100 years pathologists have been using microscopes to study disease at a cellular level and despite improvements in the objective lens and light sources, there have been few changes to the techniques during this time.
In the 1990's, laboratories began attaching cameras to microscopes and taking still images of small fields of use for publication and training purposes. Then in the late 90's whole slide imaging began to emerge and the term ‘digital pathology' was coined.
“Initially it took almost a day to scan one slide because of how slow the technology was, but nowadays, it can take just one to two minutes to get the whole slide scanned,” said Vipul Baxi, Scientific Senior Director of Digital Pathology at Bristol Myers Squibb (BMS).
He explained that digital pathology broadly falls into two categories. The first involves using computer-based virtual slides that allow a pathologist to review on a screen exactly what they would normally look at under a microscope.
This method has been particularly valuable during the COVID-19 pandemic, allowing pathologists to examine slides remotely. Indeed, in April 2020 the FDA relaxed their regulations1, which normally prevent pathologists from making primary diagnoses outside of the laboratory. The regulatory body is temporarily allowing pathologists to access images of patient tissue remotely through a secure VPN high-speed internet connection and make definitive diagnoses via a web-based browser using a monitor that meets specified minimum requirements.
Artificial intelligence
The second category of digital pathology takes the technology further by adding computer vision and image analysis algorithms on top. The algorithms are developed using state-of-the-art artificial intelligence (AI) techniques, including various machine learning (ML) and deep learning (DL) models.
This technology can “mimic what the pathologist is doing [then also] go a step beyond and uncover things that the pathologist may see but may not be able to quantify as robustly, or as consistently,” Baxi explained.
Baxi joined BMS 6 years ago to build the internal digital pathology infrastructure and establish external strategic partnerships that the global biopharmaceutical company is now deploying within its translational research for novel biomarker discovery. His team began with routine digitalization of tissue specimen slides collected during the trials and they are now developing algorithms to enhance the analysis of biomarkers in a quantitative manner.
The next step, he said, will be to develop advanced AI techniques to uncover computationally derived biomarkers that go beyond what a human pathologist could visually quantify, incorporating contextual features that can better characterize the disease biology and potentially determine the appropriate treatment for patients.
Research from 2016 published in Nature Reviews Drug Discovery2 suggests that around just 10% of drug development projects reach the approval stage, so any technology that boosts this proportion by improving clinical trial success rates will be valuable to the pharmaceutical industry.
A 2021 survey3 by the analytics firm GlobalData compounds this viewpoint, showing that AI was expected to be the emerging technology that would have the greatest impact on the pharmaceutical industry in that year.
Geographic contextualization
One particular use of AI that is gaining momentum in digital pathology is the geographic contextualization of data using spatial algorithms. Baxi explained that geographic contextualization is “not just quantifying the biomarker on a patient, it's understanding where the biomarker expression is located in the tumor microenvironment, how the cells are spatially oriented, and how they're interacting with each other.”
This information cannot be determined using traditional pathology methods and will help researchers to understand how a disease is progressing or why a patient might respond to one particular treatment over another.
Multiplexed staining
Another area of progress that could help improve the success rate of clinical trials is within staining technology. It is now possible to stain slides for multiple markers and look at different cell types on a single tissue specimen. Analysis of these digitized multiplexed slides can then be carried out by a digital algorithm that is capable of processing the large volumes of information available all at once. This is advantageous because tissue samples can be limited, and it is not always possible to stain for every marker of interest using standard techniques.
Streamlining drug discovery
Taken together, these new methods can help streamline the drug discovery process in a number of ways. Baxi sets them out in a review article recently published in Modern Pathology4 beginning with target identification, followed by indication selection, where digital pathology can assist with selecting which biomarkers to target. Tumor profiling can then help to understand the mechanism of action, while comparing pre- and post-treatment samples can shed light on pharmacodynamics.
Digital pathology can also be used to stratify patients within trials and then finally as a companion diagnostic to identify patients most likely to benefit from a particular treatment.
Global healthcare company GSK started building AI models and computational pathology around 2 years ago and now have the largest AI group, with around 110 members of staff, in the pharmaceutical industry.
Kim Branson, Senior Vice President and Global Head, AI and ML, GSK, believes that digital pathology “is going to be absolutely transformative” for clinical trials and drug discovery because it “allows us to do different things we couldn't do before.”
One such thing is spatial transcriptomics, he said, which involves looking at mRNA expression in various cell types on the same slide. This can then be taken up a level and applied to 3D images of whole tumor samples, such as those produced by Alpenglow Biosciences.
3D digital pathology
3D images can better capture the complexity of human tumors, including influences from the surrounding microenvironment, than cells cultured in 2D. GSK are now working on a 5-year project with King's College London5 that will use these tumor models alongside digital pathology and AI to develop personalized immuno-oncology treatments for a number of solid cancers, including lung, gastrointestinal, and women's cancers.
The study will use tumor samples to culture and grow a 3D “biological twin” that will then be digitalized and allow researchers to test multiple drugs at multiple doses and timepoints.
These images depict a stylized neural net and the DNA sequence the model predicts. The photos show a saliency map superimposed onto a histopathology image of breast cancer and indicate the relative importance of the tissue regions when predicting the homologous recombination deficiency status of this patient by a deep learning model.
Another collaboration that GSK have recently announced is with PathAI, who provide various AI-powered algorithms to 80% of the top global pharmaceutical companies for use in translational research, clinical trials, and clinical diagnostic development.
Non-alcoholic steatohepatitis
One of those algorithms, which Branson describes as “world class” is for non-alcoholic steato-hepatitis (NASH), which is an advanced form of non-alcoholic fatty liver disease characterized by liver inflammation that can lead to fibrosis, cirrhosis, and ultimately liver cancer.
GSK has a drug candidate for NASH, so they will be using PathAI's model to support their clinical trials. They will also be deploying some of their other computational pathology algorithms on PathAI's platform for clinical trials in other disease areas such as oncology.
Andy Beck, CEO and co-founder of PathAI, says that “there is a growing need within the pharmaceutical industry to get more accurate and reliable data from pathology samples in a way that is repeatable, scalable, and quantitative. Digital image analysis via AI can help reduce the variability that is associated with manual pathology and improve the accuracy of biomarker measurement.”
He adds that AI also allows for more complex analysis of large amounts of data and has “created a new pathway for biomarker discovery, as it can improve the assessment of predictive bio-markers in tissue biopsies leading to new, impactful discoveries with therapeutic indications.”
Breast cancer cells viewed under a microscope.
Commercial pathology
Aside from biomarker and drug discovery, the use of digital pathology is also expanding in clinical laboratories with companies such as PreciseDx offering proprietary AI algorithms that can be used to guide patient management.
Their Morphology Feature Array “is unique in its ability to mine millions of data points to identify and quantify the key cellular characteristics, enabling a new level of disease characterization and therefore personalization in treatment,” said the company's CEO Wayne Brinster.
At present, PreciseDx is in the process of releasing a commercial breast cancer test that looks for 8–12 disease-specific morphologic characteristics that are associated with current and future progression. Once they receive a digital scan or a glass slide sample they can provide clinicians with a detailed report within 2 to 3 days, which can be used to inform treatment plans.
A test for prostate cancer is now in development but the company has also used its connections with Mount Sinai, where it spun out from, to apply the technology to Parkinson's Disease (PD), having previously had some success characterizing TAU protein in Alzheimer's disease.
Parkinson's disease
PreciseDx applied the Morphology Feature Array™ to immunohistochemistry detection of α-synuclein – a protein that is linked to PD pathogenesis – within peripheral nerves of salivary glands and showed that they could use morphology features to accurately detect peripheral Lewy type synucleinopathy (LTS), the pathological hallmark of PD, in early-stage Parkinson's disease.
John Crary, a Professor in the Departments of Pathology, Neuro-science, and Artificial Intelligence & Human Health at the Icahn School of Medicine at Mount Sinai, said that the “industry-changing study,” published in Acta Neuropathologica Communications6, “has shown that we need to revitalize the way we think about pathology and lean into using AI to detect diseases more accurately, such as PD. This enlightens the industry to a direct case study into how computational pathology can truly advance medicine in terms of accurately identifying and detecting diseases.”
And Brinster noted that although PreciseDx's commercial focus will remain with oncology for now, “the success with Parkinson's and other unpublished work allows us to have a broad menu of potential diseases in which to address unmet clinical needs.”
He added: “Our mission however will always remain the same, to provide otherwise unavailable knowledge to drive better treatment decisions and patient outcomes.”
Limitations of digital pathology
Despite the advances that digital pathology can bring, there are still some limitations to the technology, particularly the wide amount of data that is needed to develop a ML model in a way that is unbiased. For example, if models are trained using biomarkers from just one cohort they may learn differences between cases and controls that are specific to that cohort, but are not actually disease-related.
Naeblys / Getty Images
Expanding the size of the training data set can also help to diminish bias that may arise, as a result of racial or ethnic differences. If the training data is not diverse it can affect the model's performance in minority populations.
To ensure that such bias doesn't arise, access to “consistent, compliant, and consented data is needed across points in the patient journey,” said Beck. "Partnerships with healthcare leaders, such as the one [PathAI] have with Cleveland Clinic, are crucial to help build the volume and depth of data needed across a diverse set of patient types to properly train and validate our models.”
He also noted that a lack of standardization across formats, protocols, systemic quality control and workflows within industry “can create interoperability issues and can even hinder regulatory progress” but guidelines from CAP, CLIA, the FDA, and DICOM “should help progress in this area and future novel approvals will set the additional precedent needed.”
And in spite of the limitations, Beck, like Branson, believes that now “is a transformative time in the field of pathology,” commenting that “AI will allow pathology, and specifically digital pathology, to keep pace with the rapid advancements in precision medicine and therapeutic development.”