A machine-learning approach based on multiparametric MRI to identify the risk of non-sentinel lymph node metastasis in patients with early-stage breast cancer

Abstract

Background

It has been reported that patients with early breast cancer with 1–2 positive sentinel lymph nodes have a lower risk of non-sentinel lymph node (NSLN) metastasis and cannot benefit from axillary lymph node dissection.

Purpose

To develop the potential of machine learning based on multiparametric magnetic resonance imaging (MRI) and clinical factors for predicting the risk of NSLN metastasis in breast cancer.

Material and Methods

This retrospective study included 144 patients with 1–2 positive sentinel lymph node breast cancer. Multiparametric MRI morphologic findings and the detailed demographical characteristics of the primary tumor and axillary lymph node were extracted. The logistic regression, support vector classification, extreme gradient boosting, and random forest algorithm models were established to predict the risk of NSLN metastasis. The prediction efficiency of a machine-learning–based model was evaluated. Finally, the relative importance of each input variable was analyzed for the best model.

Results

Of the 144 patients, 80 (55.6%) developed NSLN metastasis. A total of 24 imaging features and 14 clinicopathological features were analyzed. The extreme gradient boosting algorithm had the strongest prediction efficiency with an area under curve of 0.881 and 0.781 in the training set and test set, respectively. Five main factors for the metastasis of NSLN were found, including histological grade, cortical thickness, fatty hilum, short axis of lymph node, and age.

Conclusion

The machine-learning model incorporating multiparametric MRI features and clinical factors can predict NSLN metastasis with high accuracy for breast cancer and provide predictive information for clinical protocol.

Keywords

Multiparametric magnetic resonance imaging machine learning breast cancer sentinel lymph node non-sentinel lymph node metastasis

Introduction

As the most common cancer among women, breast cancer poses a significant threat to public health worldwide (1). Axillary lymph node (ALN) status is very important for the clinical staging, prognosis assessment, and treatment choice for patients with breast cancer (2). The sentinel lymph node (SLN) is the first organ that metastatic cancer cells reach (3,4). Currently, the sentinel lymph node biopsy (SLNB) procedure is used to assess regional lymph node involvement of patients with breast cancer. The standard treatment for patients with breast cancer with SLN metastasis is a thorough axillary lymph node dissection (ALND) (5,6). However, ALND could cause many complications, such as lymphedema, limited arm motion, and neuropathic pain (7). Furthermore, approximately 50% of patients with positive SLN did not have additional nodal metastases after ALND (8). The National Comprehensive Cancer Network's (NCCN) Panel has recommended that a complete ALND could be avoided for patients with a low risk (LR) of residual non-SLN (NSLN) metastatic disease even though the SLN is positive (9) . The American College of Surgeons Oncology Group Z0011 (ACOSOG Z0011) trial and International Breast Cancer Study Group (IBCSG) 23-01 revealed that ALND for one or two SLN metastases in early breast cancer did not confer a survival benefit (10,11). A long-term follow-up of the Z0011 trial confirmed these results (12). Therefore, it is controversial whether complete ALND is required in patients with SLN metastasis and LR of lymph node metastasis.

In addition to the assessment of axillary nodal disease burden, several models have been developed to predict metastasis involving NSLN. The most widely used models included nomograms (Memorial Sloan Kettering Cancer Center and Stanford) and scoring systems (Tenon and Saidi) (13 –16). These models used clinical-pathologic information including patient age, lymphovascular invasion, multifocality, and number of positive SLNs. None of these models used results from imaging studies.

Machine learning (ML) is a new type of artificial intelligence. Compared with traditional statistical approaches, which usually only consider and evaluate a limited set of hypotheses, ML approaches have been used in predictive mode and decision-making in biomedicine (17 –19). For multiparametric magnetic resonance imaging (mpMRI) of the breast, it combines morphological and functional sequences to not only provide the morphological characteristics of the tumor, but also reflect the pathological changes associated with the lesion; however, the potential of mpMRI in this respect has not been fully explored (20 –22).

Therefore, the aim of the present study was to assess the feasibility of ML with mpMRI using T1-weighted (T1W) imaging, T2-weighted (T2W) imaging, dynamic contrast-enhanced MRI (DCE-MRI), and diffusion-weighted imaging (DW) for predicting the risk of NSLN metastasis in patients with breast cancer, which may help some patients with breast cancer with one or two positive SLNs but consistently negative NSLN to avoid overtreatment and promote personalized axillary management. The study ultimately developed a personalized predictive model that could guide clinicians in making better cancer treatment choices for different patients.

Material and Methods

Patients

The present study was a retrospective review of the medical records of patients with early breast cancer diagnosed and treated at our institution between September 2017 and November 2022. The protocol of this observational study was approved by the institutional review board of our hospital (2021-02).

The inclusion criteria included the following: (i) primary cT1–2 stage invasive breast cancer; (ii) preoperative breast MRI performed within one week before breast surgery; (iii) patients with primary breast cancer who may receive SLNB and/or ALND; (iv) patients with only 1–2 positive SLNs; and (v) a visible tumour on MRI. The exclusion criteria included the following: (i) a previous history of breast cancer; (ii) non-invasive breast cancer; (iii) no breast MRI examination; (iv) inadequate quality of MRI; (v) neoadjuvant chemotherapy or radiation therapy before breast surgery; and (vi) patients who did not undergo SLNB and/or ALND. A total of 144 women met the criteria and were included in the analysis.

MRI examinations

MRI was performed on a 3.0-T MR scanner (GE Signa HDxt, Boston, USA) with 16-channel phased-array breast coil. The patient had an advanced head, prone position, natural breast ptosis, and centered nipple. The MRI sequences included axial T1W (repetition time [TR]/echo time [TE] = 725/7.18 ms, slice thickness/gap = 5/1 mm), axial T2W (TR/TE = 5200/90 ms, slice thickness/gap = 5/1 mm), and axial DWI (TR/TE = 8300/63 ms, slice thickness/gap = 5/1 mm, b value = 800 s/mm²). DCE-MRI was performed with a volume acceleration sequence in the axial plane (TR/TE = 6.2/2.3 ms, slice thickness/gap = 2/0 mm, field of view = 360 × 360 mm, matrix = 512 × 512, and flip angle = 12°). The contrast agent (Gd-DTPA, 0.1 mmol/kg; Omniscan, GE Healthcare) was injected intravenously at a rate of 2 mL/s. In total, eight phases were scanned without intervals.

Image analysis

mpMRI data were anonymized by two experienced breast radiologists (with 12 and 5 years of experienced, respectively) for evaluation, who were blinded to the clinicopathologic details. Any disagreement was resolved by consultation. Imaging features were extracted from mpMRI images. These features are used as properties of machine learning classifiers.

Imaging features of the primary tumor are as follows: (i) the characteristics of lesion enhancement: mass enhancement, non-mass enhancement; (ii) multiple lesions; (iii) fibroglanular type (FGT) was as follows: (a) almost, (b) scattered, (c) heterogeneous, (d) dense; (iv) background parenchymal enhancement (BPE) includes minimal, mild, moderate, and marked; (v) tumor location: inner upper quadrant, inner lower quadrant, outer upper quadrant, outer lower quadrant, central glandular region and other regions; (vi) the shape of the mass: oval, round, irregular; (vii) the margin of the mass/non-mass: smooth, irregular, spiculated; (viii) mass enhancement features: homogeneous, heterogeneous, rim; (ix) non-mass enhancement (NME) features: distribution (linear, segmental, focal, regional, multiple regional) and internal enhancement features (homogeneous, heterogeneous, clumped, cluster ring); (x) peritumoral edema (yes/no); (xi) thickening of adjacent skin; and (xii) inversion of the nipple.

Quantitative indicators are as follows: (i) the maximum diameter of tumor was measured; and (ii) tumoral apparent diffusion coefficent (ADC). The minimum ADC value (ADC_min) was taken after multiple measurements on different parts of the same lesion. The time-signal intensity curve (TIC) was divided into three types: persistent (I), plateau (Ⅱ), and washout (Ⅲ).

Predefined imaging features of axillary lymph nodes were assessed for the most suspicious lymph node in each participant. Morphologic features for the assessment of lymph node were as follows: (i) short axis (in mm); (ii) long axis (in mm); (iii) aspect ratio; (iv) margin (smooth/irregular); (v) hilum (absent/present); (vi) cortical thickness (in mm); (vii) perifocal edema (yes/no); and (viii) axillary LNs symmetry (yes/no). Lymph nodes were considered suspect malignancies if they had at least one of the following characteristics (23,24): (i) absence of fatty hilum, when fat signal intensity was not visualized in the lymph nodes; (ii) irregular cortex, when there was a focal increase in cortical thickness associated with the fatty hilum, but not centrally located in the node, and located on one side; (iii) lobulated margins, when nodules have irregular external contours; and (iv) when there was edema around the lymph nodes. Fig. 1a–d shows the morphological features of lymph node metastasis.

Fig. 1.

Example of morphologic features for the assessment of axillary lymph nodes in axial T1W, T2W, and DCE-MRI: (a) lymph node with absense of fatty hilum; (b) lymph node with an inhomogeneous cortex; (c) lymph node with perifocal edema; and (d) lymph node with irregular margin. DCE-MRI, dynamic contrast-enhanced magnetic resonance imaging; T1W, T1-weighted; T2W, T2-weighted.

Data processing and feature screening

Dataset

All patients were randomly divided into a training set (n = 100 (70%) and a test set (n = 44, 30%) with a ratio of 7:3. The training set was used to train the model, and the test set was used to validate the model independently.

Feature engineering

Before model training, the values of all features were rescaled to range [0, 1] to reduce the excessive reliance on a certain feature. To avoid the model overfitting, reduce the redundancy, and find the optimal feature subset, the analysis of variance and the pearson correlation analysis were applied to filter the features.

Candidate ML algorithms

Four types of ML algorithms were applied in this study, including logistic regression (LR), extreme bradient boosting (XGBoost), random forest (RF), and support vector classification (SVC).

Evaluation metrics

This experiment obtained several parameters of the model confusion matrix, including true positive (TP), true negative (TN), false positive (FP), and false negative (FN). Accuracy, sensitivity, specificity, F1 score, and area under the receiver operating characteristic (ROC) curve (AUC) were used to evaluate the ML models.

Model tuning process

In the process of training the model, fivefold cross-validation was performed to gauge the stability of the model and to overcome the overfitting. Due to limited sample size, the learning curve was combined to analyze the effect of the model in order to ensure the generalization and stability, and to improve the prediction performance of the model on new samples. The AUCs were compared using the DeLong test. The model with the highest AUC was chosen as the final model. In order to verify the effect of the model on unknown samples, all indicators of each model were predicted and calculated on the test set to further confirm that the selected model was the optimal mode.

Statistical analysis

Categorical variables were reported as integers and proportions. The continuous variables were described as means ± standard deviation (SD) if the variable follows normal distribution and with medians and lower and upper quartiles (Q1, Q3) otherwise. Cohen's Kappa was used to calculate the interrater reliability between the two readers. Based on the Python language (version 3.7.4) and Jupyter lab software (version 3.0), we used the Scikit-learn library (version 1.0.2), and introduced four ML classification algorithms to predict NSLN metastasis, including LR, SVM, RF, and XGboost. A P value <0.05 was considered to indicate a significant difference.

Results

Patient characteristics

In total, 144 patients with SLN-positive breast cancer were included (mean age = 51.8 ± 11.0 years; age range = 23–78 years). The clinicopathologic characteristics of 80 patients with metastatic NSLN and 64 patients without metastatic NSLN are summarized in Table 1. The rates of NSLN(+) in the training and test sets were 52.0% and 83%, respectively. Table 2 shows the MRI features of tumors and lymph nodes. There was also no variability between the observers. The unweighted values of Cohen's k were 0.694–0.931.

Table 1.

Patient demographics and tumor characteristics.

Characteristic	Value	Characteristic	Value
No. of patients	144	Upper-inner quadrant	25 (17.4)
Age (years)	51.8 ± 11.0 (23–78)	Lower-inner quadrant	3 (2.1)
Menopause		Center	11 (7.6)
Yes	85 (59.0)	Other	25 (17.4)
No	59 (41.0)	Ki-67
		High (≥14%)	114 (79.2)
		Low (<14%)
BC laterality		Molecular subtype	30 (20.8)
Right	61 (42.4)	Luminal	107 (74.3)
Left	83 (57.6)	HER2 enriched	21 (14.6)
Histologic type		Triple negative	16 (11.1)
IDC	120 (83.3)	LVI
Other	24 (16.7)	Yes	118 (81.9)
ER		No	26 (18.1)
Positive	106 (73.6)	Clinical T stage
Negative	38 (26.4)	T1 (≤2 cm)	46 (31.9)
PR		T2 (2.1–5 cm)	98 (68.1)
Positive	94 (65.3)	Histological grade
Negative	50 (34.7)	Ⅰ	4 (2.8)
HER2		Ⅱ	100 (69.4)
1+	34 (23.6)	Ⅲ	40 (27.8)
2+	28 (19.4)	Number of positive SLN
3+	30 (20.8)	1	85 (59.0)
−	52 (36.1)	2	59 (41.0)
Tumor location		NSLN status (SLN positive)
Upper-outer quadrant	77 (53.5)	Yes	80 (55.6)
Lower-outer quadrant	3 (2.1)	No	64 (44.4)

Values are given as n (%) or mean ± SD (range).

BC, breast cancer; ER, estrogen receptor; HER2, human epidermal growth factor receptor 2; IDC, invasive ductal carcinoma; LVI, lymphovascular invasion; NSLN, non-sentinel lymph node; PR, progesterone receptor; SLN, sentinel lymph node.

Table 2.

MRI features of tumors and lymph nodes.

Characteristic	Value	Characteristic	Value
No. of patients	144	Heterogeneous	81 (56.2)
MRI_BI_RADS		Rim	7 (4.9)
4a	10 (6.9)	NME distribution
4b	15 (10.4)	Linear	1 (0.7)
4c	66 (45.8)	Focal	5 (3.5)
5	53 (36.8)	Segmental	30 (20.8)
Tumor size(cm)		Regional	8 (5.6)
≤2	41 (28.5)	Multiple regional	1 (0.7)
2-4	86 (59.7)	NME enhancement
＞4	17 (11.8)	Heterogeneous	81 (56.2)
FGT		Clumped	17 (11.8)
Almost	3 (2.1)	Cluster ring	12 (8.3)
Scattered	55 (38.2)	Thickening of skin
Heterogeneous	66 (45.8)	Yes	11 (7.6)
Dense	20 (13.9)	No	133 (92.4)
BPE		Inversion of the nipple
Minimal	5 (3.5)	Yes	13 (9.0)
Mild	106 (73.6)	No	131 (91.0)
Moderate	24 (16.7)	Kinetic curve types
Marked	9 (6.2)	Persistent（Ⅰ）	6 (4.2)
Multifocality		Plateau（Ⅱ）	47 (32.6)
Yes	9 (6.2)	Washout（Ⅲ）	91 (63.2)
No	135 (93.8)	Tumoral ADC^† (10-3mm2/s）	1.14 (1.03-1.29)
Peritumoral edema		LN margin
Yes	62 (43.1)	Smooth	55 (38.2)
No	82 (56.9)	Irregular	89 (61.8)
Mass enhancement		LN hilum
Mass	99 (68.8)	Present	40 (27.8)
NME	45 (31.2)	Absent	104 (72.2)
Mass/NME margin		LN perifocal edema
Smooth	23 (16.0)	Yes	68 (47.2)
Irregular	92 (63.9)	No	76 (52.8)
Spiculated	29 (20.1)	LN cortical thickness ^† (mm)	4.13 (3.41-5.98)
Mass shape		LN short axis^† (mm)	7 (5-8.58)
Round	2 (1.4)	LN long axis^† (mm）	10.16 (7.97-14）
Oval	53 (36.8)	LN aspect ratio^†	1.43(1.26-1.71）
Irregular	44 (30.6)	Axillary LNs symmetry
Mass enhancement		Yes	100 (69.4)
Homogeneous	11 (7.6)	No	44 (30.6)

Data in parentheses were percentages unless otherwise noted. ^† , Values refer to median (interquartile range). NME, non-mass enhancement; FGT, fibroglanular type; BPE, background parenchymal enhancemen; LN, lymph node; ADC, apparent diffusion coefficent

Feature screening results

Delete features with low variance

In this experiment, the variance information of the features that can be quantified is shown in Fig. 2.

Fig. 2.

Descriptors variance values: the variance in low variability represents the difference between the different samples on this feature, which means that the samples cannot be classified. ADC, apparent diffusion coefficient; ER, oestrogen receptor; LN, lymph node; LVI, lymphovascular invasion; PR, progesterone receptor.

Optimize strong correlation features

This paper draws the Pearson linear correlation coefficient map (Fig. 3) among the remaining features to analyze and identify highly correlated features, and then eliminate multicollinearity between features.

Fig. 3.

Correlation of variables: each value in the left part of the figure represents the Pearson correlation coefficient value between the features marked on the horizontal axis and the vertical axis. Significant cross-correlation (Pearson correlation coefficient >0.8) was not observed among most of the features. This proves that the collinearity analysis worked well. ADC, apparent diffusion coefficient; ER, oestrogen receptor; LN, lymph node; LVI, lymphovascular invasion; PR, progesterone receptor.

Performance of ML models

Combining the confusion matrix in Fig. 4a, the learning curve in Fig. 4b, the ROC curve in Fig. 4c, Fig. 4d, and Table 3, we comprehensively observe the performance of the prediction model under each algorithm. For the training set, AUC values of the XGBoost, LR, SVC, and RF models were 0.881, 0.866, 0.843, and 0.853, respectively. In the test set, AUC values of the XGBoost, LR, SVC, and RF models were 0.781, 0.692, 0.680, and 0.705, respectively. DeLong test shows that there are significant differences between the LR and SVC models (P = 0.009) in the training set. In the test set, the AUC values of the XGBoost and LR models were significantly different (P = 0.04). We applied the optimal model obtained on the training set to the test dataset to obtain the approximate performance evaluation metrics of the model on the new sample.

Fig. 4.

Confusion matrices, obtained by different models when predicting NSLN metastasis. (a) The XGboost model predicts the most correct positive and negative samples and has the highest overall accuracy rate. (b) The learning curve for fivefold cross-validation in the training set. (c, d) ROC curves of predictive models based on machine learning algorithms for the training set and test set, respectively. AUC, the area under the ROC curves; NSLN, non-sentinel lymph node; ROC, receiver operating characteristic; SVC, support vector classification; XGBoost, extreme gradient boosting.

Table 3.

Comparison and prediction performances of diferent models for NSLN metastasis.

Models	AUC	Accuracy	Precision	Sensitivity	Specificity	F1
Training set
XGBoost	0.881	0.840	0.812	0.928	0.727	0.866
LR	0.866	0.740	0.714	0.893	0.545	0.793
SVM	0.843	0.740	0.714	0.893	0.545	0.793
RF	0.853	0.630	0.913	0.375	0.954	0.532
Test set
XGBoost	0.781	0.750	0.741	0.833	0.650	0.784
LR	0.692	0.614	0.613	0.792	0.400	0.691
SVM	0.680	0.523	0.542	0.792	0.200	0.644
RF	0.705	0.636	0.643	0.750	0.600	0.692

AUC, area under the curve; LR, logistic regression; SVM, support vector machine; RF, random forest; XGBoost, extreme gradient boosting.

Relative importance of variables in ML models

The evaluation results in order of feature importance obtained using the optimal model XGBoost in this experiment are shown in Fig. 5.

Fig. 5.

Variable importance of features included in the XGBoost algorithm for the prediction of NSLN status. LN, lymph node; NME,non-mass enhancement; NSLN, non-sentinel lymph node; PR, progesterone receptor; SLN, sentinel lymph node.

Discussion

Our study explored the value of ML models based on mpMRI and clinicopathological features in predicting the NSLN metastasis of clinical T1 or T2 breast cancer patients. The feature “tumoral ADC” with the lowest variance value in the training dataset was deleted. For multiple features with strong correlation, only one valid feature is reserved generally. The correlation between “short axis” and “long axis” is relatively high (up to 0.8) (Fig. 3); however the “long axis” variance value is low (Fig. 2), so the “long axis” feature is deleted. Among the four models, XGBoost displayed a better performance for predicting NLSN, with four performance metrics outperforming the other models in the training set (AUC = 0.881, accuracy = 0.840, sensitivity = 0.928, F1 score = 0.866), and similar results were observed in the test set. Therefore, we propose a novel non-invasive method based on preoperative mpMRI images to predict NSLN status, which may be helpful for guiding the individualized treatment of patients with positive SLNB.

The present study further evaluated the importance of different imaging features of the lymph nodes in the model. Nodal cortical thickness, fatty hilum, short axis, margin, and aspect ratio ranked top 10 in the model, which had a great influence on the prediction results of NSLN. Previous studies have shown that lymph node cortical thickness is the most discriminating objective criterion for predicting axillary positive status (25). Our results were similar, finding that cortical thickness was highly positively correlated with NSLN metastasis. In short, cortical morphologic change is a well-established sign of metastases as metastatic cells reside in the nodal cortex (26).The size of lymph nodes characterized by the diameter of the short axis is a recognized standard for evaluating the status of lymph node metastasis (27). Unlike previous studies (28), the characteristic importance of our XGboost classifier shows that in addition to the short axis of lymph nodes, aspect ratio is also an important predictor. Benign lymph nodes are kidney shaped, and the longitudinal diameter is significantly larger than the transverse diameter, while malignant lymph nodes become rounded and the aspect ratio is reduced. It can be seen from the model that nodal hilum and margin are also independent predictors of lymph node metastasis, which is consistent with previous research results (29) . Clinical studies have confirmed that cancer cells lead to abnormal hyperplasia of marginal sinus through the input of lymphatic vessels, and then invade the cortex and medulla. Excessive proliferation of cancer cells leads to thickening of nodal cortex, abnormal morphology of lymphatic hilum, enhanced lateral growth, and reduced aspect ratio (26,30). Therefore, the predictive factors included in this study model have high universal application value.

The ML model has been used to differentiate benign and malignant breast nodules in some studies (31 –33). However, few studies used traditional ML methods combined with imaging features to predict NSLN metastasis in patients with breast cancer. In our study, four ML methods combined with mpMRI and clinical features were constructed to predict NSLN metastasis, and the AUC value of the optinal model was 0.759 (34). Compared to previous studies, our XGBoost model performed better than most clinical models, but not as well as in some radiomics studies (13,19,35 –37). There could be several reasons for this. A previous study reported that the XGBoost algorithm showed better performance in predicting the response to neoadjuvant chemotherapy in breast cancer than traditional supervised ML algorithms such as LR and SVC (18). Our results also show that the XGBoost model is superior to the SVC, RF, and LR models in assessing the NSLN status. Radiomics is the study of transforming image analysis into quantifiable features by extracting higher-level high-throughput features from images, which may provide more useful information for NSLN evaluation, such as texture features. However, previous radiomics studies have not yet entered the clinic, and their complex and abstract analytical process has limited clinical practice (35,38). In our study, although subjective imaging assessments may vary depending on the radiologist's experience, recognition performance was confirmed in the test dataset. In addition, the mpMRI features used in our model are relatively easy to obtain in routine clinical practice, compared to radiomic features, and may help optimize clinical workflow. At present, the clinical characteristics and treatment mode of some patients do not meet the requirements of Z0011 and IBCSG 23-01 trial, but ALND or axillary radiotherapy is still needed, and about two-thirds of the patients have not developed NSLN metastasis. If axillary treatment is performed in all patients, there is the problem of axillary overtreatment. Therefore, assessing the risk of NSLN metastasis will help to develop a rational axillary management strategy.

The present study has some limitations. First, we used morphologic features that can be routinely extracted from mpMRI, still relying on subjective assessment of radiologists. Such inter- or intra-observer variability may affect the extracted imaging features, and in turn, reproducibility may be a limitation of this study. This potential effect should be the subject of future research. Second, we did not compare suspicious nodes using MRI with its pathological findings on a pathologic basis. Therefore, the accuracy of NSLN prediction could be affected for the constructed predictive models. Finally, a limited number of cases were analyzed at an institution with a small number of patients. The XGBoost model established in this study still needs to be used in multicenter datasets with larger sample sizes to obtain high-level evidence for clinical application.

In conclusion, ML with mpMRI of the breast enables the early prediction of the risk of NSLN metastasis in patients with breast cancer with 1–2 positive SLNs with high accuracy. Thus is a pivotal step for the realization of precision medicine in breast cancer.

Footnotes

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Natural Science Foundation of Shandong Province ZR202103060229.

ORCID iD

Haitong Yu

References

Siegel

Miller

Jemal

. Cancer statistics, 2018. CA Cancer J Clin 2018;68:7–30.

Kim

. Breast cancer metastasis. Adv Exp Med Biol 2021;1187:183–204.

Berg

. The significance of axillary node levels in the study of breast carcinoma. Cancer 1955;8:776–778.

Okur

Sagiroglu

Kir

, et al. Diagnostic accuracy of sentinel lymph node biopsy in determining the axillary lymph node metastasis. J Cancer Res Ther 2020;16:1265–1268.

Veronesi

Paganelli

Viale

, et al. A randomized comparison of sentinel-node biopsy with routine axillary dissection in breast cancer. N Engl J Med 2003;349:546–553.

Rao

Euhus

Mayo

, et al. Axillary node interventions in breast cancer: a systematic review. JAMA 2013;310:1385–1394.

Brackstone

Baldassarre

Perera

, et al. Management of the axilla in early-stage breast cancer: Ontario Health (Cancer Care Ontario) and ASCO guideline. J Clin Oncol 2021;39:3056–3082.

Abdessalam

Zervos

Prasad

, et al. Predictors of positive axillary lymph nodes after sentinel lymph node biopsy in breast cancer. Am J Surg 2001;182:316–320.

Gradishar

Moran

Abraham

, et al. Breast cancer, version 3.2022, NCCN clinical practice guidelines in oncology. J Natl Compr Canc Netw 2022; 20: 691–722.

10.

Lucci

McCall

Beitsch

, et al. Surgical complications associated with sentinel lymph node dissection (SLND) plus axillary lymph node dissection compared with SLND alone in the American College of Surgeons Oncology Group Trial Z0011. J Clin Oncol 2007;25:3657–3663.

11.

Galimberti

Cole

Zurrida

, et al. Axillary dissection versus no axillary dissection in patients with sentinel-node micrometastases (IBCSG 23-01): a phase 3 randomised controlled trial. Lancet Oncol 2013;14:297–305.

12.

Giuliano

Ballman

McCall

, et al. Effect of axillary dissection vs no axillary dissection on 10-year overall survival among women with invasive breast cancer and sentinel node metastasis: the ACOSOG Z0011 (Alliance) randomized clinical trial. JAMA 2017;318:918–926.

13.

Van Zee

Manasseh

D-ME

Bevilacqua

JLB

, et al. A nomogram for predicting the likelihood of additional nodal metastases in breast cancer patients with a positive sentinel node biopsy. Ann Surg Oncol 2003;10:1140–1151.

14.

Coutant

Rouzier

Fondrinier

, et al. Validation of the Tenon breast cancer score for predicting non-sentinel lymph node status in breast cancer patients with sentinel lymph node metastasis: a prospective multicenter study. Breast Cancer Res Treat 2009;113:537–543.

15.

Kohrt

Olshen

Bermas

, et al. New models and online calculator for predicting non-sentinel lymph node status in sentinel lymph node positive breast cancer patients. BMC Cancer 2008;8:66.

16.

Saidi

Dudrick

Remine

, et al. Nonsentinel lymph node status after positive sentinel lymph node biopsy in early breast cancer. Am Surg 2004;70:101–105.

17.

Nattkemper

Arnrich

Lichte

, et al. Evaluation of radiological features for breast tumour classification in clinical screening with machine learning methods. Artif Intell Med 2005;34:129–139.

18.

Tahmassebi

Wengert

Helbich

, et al. Impact of machine learning with multiparametric magnetic resonance imaging of the breast for early prediction of response to neoadjuvant chemotherapy and survival outcomes in breast cancer patients. Invest Radiol 2019;54:110–117.

19.

Madekivi

Bostrom

Karlsson

, et al.

Can a machine-learning model improve the prediction of nodal stage after a positive sentinel lymph node biopsy in breast cancer?

Acta Oncol 2020;59:689–695.

20.

Liu

Ding

Spuhler

, et al. Preoperative prediction of sentinel lymph node metastasis in breast cancer by radiomic signatures from dynamic contrast-enhanced MRI. J Magn Reson Imaging 2019;49:131–140.

21.

Choi

Youk

Choi

, et al. Dynamic contrast-enhanced and diffusion-weighted MRI of invasive breast cancer for the prediction of sentinel lymph node status. J Magn Reson Imaging 2020;51:615–626.

22.

Liu

Feng

, et al. Preoperative prediction of lymphovascular invasion in invasive breast cancer with dynamic contrast-enhanced-MRI-based radiomics. J Magn Reson Imaging 2019;50:847–857.

23.

Mortellaro

Marshall

Singer

, et al. Magnetic resonance imaging for axillary staging in patients with breast cancer. J Magn Reson Imaging 2009;30:309–312.

24.

Mann

Cho

Moy

. Breast MRI: state of the art. Radiology 2019;292:520–536.

25.

Scaranelo

Eiada

Jacks

, et al. Accuracy of unenhanced MR imaging in the detection of axillary lymph node metastasis: study of reproducibility and reliability. Radiology 2012;262:425–434.

26.

Bedi

Krishnamurthy

, et al. Cortical morphologic features of axillary lymph nodes as a predictor of metastasis in breast cancer: in vitro sonographic study. AJR Am J Roentgenol 2008;191:646–652.

27.

Elsholtz

FHJ

Asbach

Haas

, et al. Introducing the node reporting and data system 1.0 (node-RADS): a concept for standardized assessment of lymph nodes in cancer. Eur Radiol 2021;31:6116–6124.

28.

Morawitz

Sigl

Rubbert

, et al. Clinical decision support for axillary lymph node staging in newly diagnosed breast cancer patients based on (18)F-FDG PET/MRI and machine learning. J Nucl Med 2023;64:304–311.

29.

Fusco

Sansone

Granata

, et al. Use of quantitative morphological and functional features for assessment of axillary lymph node in breast dynamic contrast-enhanced magnetic resonance imaging. Biomed Res Int 2018;2018:2610801.

30.

Ecanow

Abe

Newstead

, et al. Axillary staging of breast cancer: what the radiologist should know. Radiographics 2013;33:1589–1612.

31.

Safia

. Prediction of breast cancer through random forest. Current Medical Imaging 2022;19:1144–1155.

32.

Bhardwaj

Sakalle

, et al. Tree-based and machine learning algorithm analysis for breast cancer classification. Comput Intell Neurosci 2022;2022:6715406.

33.

Zhao

Chen

Zhang

, et al. MRI-based machine learning in differentiation between benign and malignant breast lesions. Front Oncol 2021;11:552634.

34.

Deng

Jiang

, et al. Application of the machine-learning model to improve prediction of non-sentinel lymph node metastasis Status among breast cancer patients. Front Surg 2022;9:797377.

35.

Qiu

Zhang

, et al. MRI-based radiomics nomogram: Prediction of axillary non-sentinel lymph node metastasis in patients With sentinel lymph node-positive breast cancer. Front Oncol 2022;12:811347.

36.

Meng

Zheng

Wang

, et al. Development of a prediction model based on LASSO regression to evaluate the risk of non-sentinel lymph node metastasis in Chinese breast cancer patients with 1-2 positive sentinel lymph nodes. Sci Rep 2021;11:19972.

37.

Guo

Liu

Sun

, et al. Deep learning radiomics of ultrasonography: identifying the risk of axillary non-sentinel lymph node involvement in primary breast cancer. EBioMedicine 2020;60:103018.

38.

Lambin

Rios-Velazquez

Leijenaar

, et al. Radiomics: extracting more information from medical images using advanced feature analysis. Eur J Cancer 2012;48:441–446.