Artificial Intelligence (AI)-Based Computer-Assisted Detection and Diagnosis for Mammography: An Evidence-Based Review of Food and Drug Administration (FDA)-Cleared Tools for Screening Digital Breast Tomosynthesis (DBT)

Introduction

The development of artificial intelligence (AI) applications in breast imaging has expanded at a rapid pace. As of this writing in July 2024, the U.S. Food and Drug Administration (FDA) has cleared more than 20 AI products for mammography, which include tools for computer-assisted detection (CADe) and diagnosis (CADx), triage (CADt), breast density assessment, and breast arterial calcification detection and quantification.^1–3 Despite the recent heightened interest in CAD due to AI, it has a long-standing history in breast imaging, with the first CADe tool approved by the FDA in 1998. ⁴ Following reimbursement approval by the Centers for Medicare & Medicaid Services in 2002, CADe was rapidly implemented into clinical settings across the United States, with its use increasing from less than 5% in 2001 to 74% in 2008 and 83% in 2012.^5–7

Based on early reader and single-center retrospective studies, CADe was initially thought to be a promising tool that would assist radiologists and improve interpretation accuracy. ⁸ Subsequent large-scale studies evaluating CADe in routine clinical practice showed that it fails to improve any screening performance metric.^6,7 In particular, CADe has been criticized for generating multiple annotations per case, resulting in high rates of false-positives, which has led to radiologists disregarding most of its marks. ⁹ A postimplementation Breast Cancer Surveillance Consortium study, including 495,818 mammograms interpreted with CADe and 129,807 without CADe by 271 radiologists from 66 U.S. facilities, found that cancer detection rates were the same with and without CADe (4.1 per 1000). ⁷ Further research showed that use of CADe increased interpretation times by 19%. ¹⁰ Thus, despite early promise and enthusiasm, there have been no improvements in radiologist performance nor efficiency gains in clinical practice using traditional CADe tools applied to screening mammography.

The development of new-generation deep learning-based AI tools that learn from large datasets and automatically analyze mammographic images to extract features associated with breast cancer has reignited enthusiasm surrounding the potential impact of CAD on screening mammography^8,11,12 (Fig. 1). AI, a computer science subdiscipline, encompasses both machine learning and deep learning.^13,14 Machine learning refers to computers learning directly from inputted data without explicit instructions or programming. This field encompasses various techniques such as supervised, unsupervised, and semisupervised learning.^14,15 Deep learning is a subdiscipline of machine learning relying on multilayer neural networks, such as the convolutional neural network, to learn complex patterns and representations from data.^13,16,17 The FDA-cleared AI tools for mammography have predominantly been trained using supervised learning techniques and are based on deep learning methodologies. ¹⁸

OPEN IN VIEWER

Fig. 1.

Comparison of traditional computer-assisted detection (CAD) with machine learning and new-generation artificial intelligence (AI)-based CAD with deep learning. Traditional CAD relies on expert knowledge for feature extraction and machine learning algorithms for classification, whereas new-generation AI-based CAD uses convolutional neural networks for both feature extraction and classification.

Digital breast tomosynthesis (DBT), approved by the FDA for clinical use in 2011, was rapidly adopted in the United States and is becoming the standard of care in mammography. ¹⁹ As of July 2024, 90% of breast imaging practices in the United States utilize DBT. ²⁰ DBT consists of low-dose X-rays acquired along an arc and reconstructed to produce a volume rendering of the breast consisting of a stack of thin sections. ²¹ Radiologists interpret DBT images in combination with either digital 2D mammographic (DM) images, which are separately acquired, or synthetic 2D mammographic images, which are reconstructed from the DBT stack. With use of synthetic images, patients avoid the additional radiation exposure from a separate acquisition. Research studies conducted in the United States have demonstrated that combined DBT/DM results in higher detection rates of cancers and fewer false-positives than DM alone.^22–24 DBT reduces the effects of superimposed breast parenchyma, which enables improved visualization of real lesions and reduces shadow summation that can prompt recall. ²¹ However, DBT has led to increased interpretation times due to the additional time needed by the reader to scroll through the reconstructed DBT stack of images. ²⁵

The longer reading times required for DBT, workforce shortages, and double-reading protocols common in European countries have fueled interest in developing AI approaches that optimize reading efficiency while maintaining or improving performance.^25,26 AI-based CADe/x devices have been developed that flag lesions within the reconstructed DBT image stack to detect and localize clinically significant lesions. This approach is intended to direct radiologists toward significant findings, thereby improving cancer detection rates and concurrently reducing interpretation times. This article reviews published evidence supporting the use of commercial FDA-cleared AI-based CADe/x algorithms for screening DBT.

AI Tools for Lesion Detection and Diagnosis on DBT

AI-based CADe/x systems encompass both computer-assisted detection (CADe) for lesion identification and diagnosis (CADx) for lesion characterization as likely benign or malignant. ²⁷ As of this writing in July 2024, there are six FDA-cleared CADe/x applications for screening DBT (Table 1).^1,2 These AI tools analyze the DBT stacks with or without the accompanying DM and/or synthetic images. The AI algorithms identify the presence of soft tissue lesions (architectural distortions, asymmetries, and masses) and calcifications that may indicate malignancy (Fig. 2). The systems also provide bounding boxes for suspicious lesions and assign a suspicion level to the finding reflecting the likelihood of malignancy (e.g., minimal/low/intermediate/high versus a score ranging from 0 to 10 or 0 to 100). Some models will not display markings under a certain threshold (e.g., if a lesion is scored on a scale of 0 to 100, the model will not display markings assigned a value of 10 or less). The higher the score, the more likely the detected finding is malignant. Some models then assign a likelihood of cancer score to each breast, and all assign a likelihood of cancer score to the examination based on the degree of suspicion of the findings.^28–33

OPEN IN VIEWER

Fig. 2.

Examples of various display markings and scores generated by artificial intelligence algorithms. These simulated images are based on the display markings and scores generated by the algorithms listed in Table 1.

Some models will mark the suspicious lesions with different shapes depending on the finding type. For example, masses, distortions, and asymmetries will be marked with a star, and calcifications will be marked with a triangle. ²⁸ One model offers the option of displaying the lesion localization with a color heatmap or contour map. ²⁹ The outputs are intended to be used as a concurrent reading aid for radiologists interpreting screening mammograms, where the radiologist confirms or dismisses the findings during interpretation.

For purposes of this review article, we searched PubMed for validation studies of the FDA-cleared AI-based CADe/x tools for DBT. Search terms included the names of the FDA-cleared AI algorithms (Table 1), “artificial intelligence,” and “digital breast tomosynthesis.” To the best of our knowledge, there are 15 published studies supporting the use of the FDA-cleared AI-based CADe/x tools for DBT (Table 2).^34–48 Four of these studies are recent multireader studies with 240–260 cancer-enriched cases.^34,38,43,44 One additional study was also a multireader study, but utilized single-view DBT images (bilateral mediolateral oblique images) rather than standard-of-care two-view DBT images (bilateral mediolateral oblique and craniocaudal images). ³⁶ Two reader studies utilizing an older version of an FDA-cleared AI-based CADe/x tool for DBT are not reviewed herewith;^45,46 the most recent reader study of that product is described in Table 2 and in the text. ³⁴

One study is a retrospective simulation focused on the potential workload reduction with use of AI, ³⁷ one study using the same mammography dataset as the prior study is a retrospective evaluation of screening DBT examinations focused on the stand-alone performance of AI-based CADe/x, ³⁹ one study compares double-reading of DM images of biopsy-proven cancers to stand-alone AI applied to the corresponding DBT images, ³⁵ and one study retrospectively evaluates different screening workflows using AI, double-read single-view DBT examinations, and double-read DM examinations. ⁴⁰ To the best of our knowledge, only two studies have assessed the utility of AI-based CADe/x for DBT in the “real-world” setting.^41,42 Two studies, one of which details an annotation-efficient deep learning approach, ⁴⁷ and one that investigates the impact of patient characteristics on the performance of an AI algorithm interpreting negative screening DBT examinations, ⁴⁸ do not report standard metrics and are therefore not reviewed herewith.

AI Tools for Lesion Detection and Diagnosis on DM

DBT in conjunction with native or synthetic DM images is widely used for screening mammography throughout the United States; however, DM alone continues to be used in many parts of the world, including Europe. In Europe, mammograms are typically double-read, with disagreements arbitrated in a consensus setting. Although there are no AI-based CADe/x tools cleared for stand-alone interpretation at this time, the possibility of using AI to replace the second reader in European screening programs is an area of active research. Several of the key studies supporting the use of FDA-cleared AI-based CAD e/x tools for screening DM are highlighted in this section.

The Mammography Screening with Artificial Intelligence (MASAI) trial is a randomized controlled trial at four screening sites in Sweden. ⁵² Participants are randomized to two arms as follows: (a) standard-of-care double-reading without AI and (b) AI-supported screening, in which the AI system score determines whether the mammogram is single-read or double-read with AI output available to all readers. This trial seeks to address two important questions as follows: one, whether AI can safely reduce screening workload and, two, the impact of AI on screening outcomes (with interval cancer rate being a primary outcome). In the first report of the MASAI trial in August 2023, which was published after 80,000 women were randomized to one of the two arms, the cancer detection rate was found to be 5.1 cancers per 1000 screened participants in the standard-of-care arm versus 6.1 per 1000 in the AI-supported screening arm (p = 0.052). ⁵² Recall rates were similar in both arms (2.0% in the standard-of-care arm versus 2.2% in the AI-supported screening arm). Importantly, with use of AI, workload was reduced by 44.3%. The trial results thus far offer strong evidence supporting the use of AI in double-reading programs; it is continuing with plans to assess interval cancer rates after 100,000 participants are enrolled and have two years of follow-up.

In ScreenTrustCAD, a prospective paired-reader study in Sweden, 55,581 women underwent screening mammograms, which were interpreted by two radiologists as per routine protocol. ⁵³ For purposes of the study, the AI tool (to which the radiologists did not have access) acted as an independent reader. If any of the three readers (two radiologists or AI) interpreted the examination as abnormal, the examination was discussed in consensus, at which time the radiologists could review AI output. The primary analysis consisted of comparing cancer detection rates between double-reading by two radiologists and double-reading by one radiologist and AI (with the read of the first and not the second reader considered). Double-reading by one radiologist and AI led to a 4% higher cancer detection rate. Both this prospective study ⁵³ and the aforementioned MASAI trial ⁵² report higher cancer detection rates with AI, suggesting that AI could lead to a reduction in interval cancer rates; however, this desirable outcome may not occur if more in situ cancers relative to invasive cancers are detected, as reported in the MASAI trial. ⁵⁴ Both of the aforementioned prospective studies are based in Sweden.^52,53 A third prospective study, titled Artificial Intelligence for Breast Cancer Screening in Mammography (AI-STREAM), is underway at multiple centers in Korea to compare radiologist accuracy with and without AI-based CADe/x. ⁵⁵

In the only head-to-head comparison, to the best of our knowledge, of commercial AI-based CADe/x tools for DM, in which three tools independently analyzed the same test set of 8805 mammograms (including 739 cancers) from women who underwent screening at an academic hospital in Sweden, the best-performing tool attained an AUC of 0.956 and a sensitivity of 81.9% (at the specificity of the radiologists). ⁵⁶ Although this tool attained and even exceeded the performance of radiologists in some comparisons, the best sensitivity (88.6%) was achieved by pairing the best-performing tool with first-reader radiologists (which was higher than the sensitivity achieved by combining the first and second radiologists). The tool with the best performance had been trained with more mammograms than the other two tools (although it had not been trained on diverse datasets nor with different vendors), had used pixel-level (rather than image-level) annotations for training, and had used a convolutional neural network architecture with a high-capacity background; these features were hypothesized to contribute to its superior performance.

Clinical Implementation Challenges and Future Directions

To avoid the low impact of older CAD products on mammography interpretation accuracy, attention to best practices in clinical implementation of new AI products is encouraged. AI-based CADe/x algorithms may be integrated into clinical workflows based on local need. ³ In double-reading practices, the AI algorithm could supplant the second reader. In batch reading practices, the algorithm could triage cases more likely to need additional imaging for immediate read before the patient leaves the center or triage more challenging cases to more experienced radiologists. There are challenges with various scenarios, which will likely be best addressed based on local practices and preferences. AI algorithms are costly, which may pose a limitation for practices with limited resources, and the integration of AI algorithms into clinical practice must lead to measured gains in quality and/or efficiency to justify the significant financial investment. ⁵⁷ At this time, in the United States, payment for computer-assisted mammography interpretation is bundled with payment for the mammogram with no additional payment made for use of the AI algorithm itself.

As AI algorithms continue to improve, stand-alone AI interpretation tools will develop, allowing for transition from assisted to autonomous reading of mammograms by computers. This transition will require extensive regulatory work as AI-based CADe/x is not cleared by the FDA for stand-alone interpretation at this time, and the medicolegal and billing implications of stand-alone interpretation of imaging studies with AI remain unresolved. ⁵⁸ Future directions include the incorporation of temporal analysis to improve AI algorithm performance (i.e., AI algorithm considers prior mammographic examinations in its interpretation), the use of customized neural network architectures for DBT, and validation of algorithms across diverse patient populations and mammography vendors.^3,59 Research studies on AI-based CADe/x for DBT are relatively limited compared with AI-based CADe/x for DM. Specifically, prospective randomized studies and rigorous postimplementation clinical evaluation are needed to determine the impact of AI-based CADe/x for DBT on screening mammography performance metrics and patient outcomes.

Conclusion

Early studies show that AI-based CADe/x holds potential to improve not only the accuracy but also the efficiency of screening DBT. Multireader, multicase studies show that AI leads to improvements in AUC and sensitivity without compromising specificity. Interpretation times may also be reduced with use of AI, but this metric has been investigated only in the context of reader studies, which may not translate to real-world clinical practice. There are insufficient studies to assess the stand-alone performance of AI in screening DBT interpretation. Only two studies report radiologist performance with AI-based CADe/x in real-world settings, which suggest that cancer detection rates could be increased with use of AI; however, both studies are relatively small with methodology limitations. Prospective randomized trials, as are being done for DM, in addition to careful postimplementation clinical evaluation, are essential to comprehensively understand the impact of AI-based CADe/x on screening DBT outcomes.