Radiomics nomogram for predicting axillary lymph node metastasis in breast cancer based on DCE-MRI: A multicenter study

Abstract

OBJECTIVES:

This study aims to develop and validate a radiomics nomogram based on dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) to noninvasively predict axillary lymph node (ALN) metastasis in breast cancer.

METHODS:

This retrospective study included 263 patients with histologically proven invasive breast cancer and who underwent DCE-MRI examination before surgery in two hospitals. All patients had a defined ALN status based on pathological examination results. Regions of interest (ROIs) of the primary tumor and ipsilateral ALN were manually drawn. A total of 1,409 radiomics features were initially computed from each ROI. Next, the low variance threshold, SelectKBest, and least absolute shrinkage and selection operator (LASSO) algorithms were used to extract the radiomics features. The selected radiomics features were used to establish the radiomics signature of the primary tumor and ALN. A radiomics nomogram model, including the radiomics signature and the independent clinical risk factors, was then constructed. The predictive performance was evaluated by the receiver operating characteristic (ROC) curves, calibration curve, and decision curve analysis (DCA) by using the training and testing sets.

RESULTS:

ALNM rates of the training, internal testing, and external testing sets were 43.6%, 44.3% and 32.3%, respectively. The nomogram, including clinical risk factors (tumor diameter) and radiomics signature of the primary tumor and ALN, showed good calibration and discrimination with areas under the ROC curves of 0.884, 0.822, and 0.813 in the training, internal and external testing sets, respectively. DCA also showed that radiomics nomogram displayed better clinical predictive usefulness than the clinical or radiomics signature alone.

CONCLUSIONS:

The radiomics nomogram combined with clinical risk factors and DCE-MRI-based radiomics signature may be used to predict ALN metastasis in a noninvasive manner.

Keywords

Breast cancer Lymph node Radiomics Nomogram Magnetic resonance imaging.

1 Introduction

The incidence of breast cancer, which is the most common malignant tumor in women worldwide, has been increasing [1]. Preoperative knowledge of the status of the axillary lymph node (ALN) can potentially help in the clinical treatment for breast cancer [2]. The ALN dissection (ALND) with the histologic examination of ALN had been the most reliable staging procedure [3]. At present, sentinel lymph node (SLN) biopsy (SLNB) has replaced complete ALND as the standard procedure for ALN staging in patients with clinically diagnosed node-negative breast cancer [4, 5]. Neoadjuvant chemotherapy (NAC) has become a widely accepted initial treatment modality for breast cancer patients with ALNM identified by SLNB, while the early stage ALN negative breast cancer offers a good prognosis and breast-conserving surgery or mastectomy is preferred. However, methods, such as biopsy and lymphadenectomy, for assessing the ALN status are invasive and have potential complications [6 –8]. Majority of patients with breast cancer have node-negative disease, and SLNB could be avoided if a reliable preoperative diagnostic evaluation of ALN is performed [9]. In addition, the potentially high false-negative rate of SLNB (5.5% –16.7%) is a point of concern [10]. Given the limitations of existing methods, a noninvasive method for predicting preoperative ALN status of breast cancer is highly attractive.

Dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) is a promising diagnostic tool [11] because of its good soft tissue contrast and its superior sensitivity to breast cancer to investigate the tissue microvasculature [12, 13]. Among breast MRI sequences, DCE-MRI is considered the best for primary tumor identification and characterization, although the diagnosis of ALNM by it is still not ideal. At the time of writing, diagnostic criteria for ALNM on DCE-MRI are based mainly on morphology [14]. However, these remain controversial.

The concept of radiomics was first proposed by Lambin in 2012 [15]. Radiomics has shown great potential in providing crucial information about tumor heterogeneity, status of pathologic types, and efficacy of neoadjuvant chemotherapy by converting the medical images into high-dimensional and exploitable data [16 –19], which are difficult to recognize using the naked eye [20]. The potential of radiomics in oncology decision support is enhanced with the development of analytical techniques. [21, 22]. Nomogram can transform complex regression equation into visual graphs. The existent studies indicates promising results in terms of predicting ALNM using MRI radiomics nomogram in breast cancer, particularly using DCE sequences [23]. Thus, radiomics nomogram models can be used by clinicians to facilitate clinical decision-making and evaluate the outcome of treatment.

Various radiomics models have been proposed to predict ALN metastasis (ALNM). Liu et al. [24] and Cui et al. [25] had verified that the radiomics model can predict the presence of ALNM and thereby prevent morbid ALND in patients with node-negative breast cancer. However, current studies on the prediction of ALN metastasis have focused only on primary tumor lesions [2 , 26]. Shan et al. [27] validated a nomogram model to detect ALNM in patients with invasive breast cancer, which based on the DCE-MRI-based radiomics features extracted from ALN with a high AUC of 0.86. This suggests that the ALN region may yield useful information, so combining primary tumor and ALN images to evaluate the lymph node status could help better assess ALNM. Therefore, an ALNM predictive nomogram, combined with clinical risk factors and DCE-MRI-based radiomics features of primary tumor and ALN, is needed.

This study was performed to develop and validate a noninvasive method of great clinical significance to predict the preoperative ALN status of breast cancer. The value of a radiomics nomogram model combined with clinical risk factors and DCE-MRI-based radiomics features of primary tumor and ALN was also investigated for the preoperative prediction of ALNM in patients with breast cancer.

2 Materials and methods

2.1 Patients

The patients in this retrospective analysis was recruited from the Yantai Yuhuangding Hospital (Center 1) from July 2018 to April 2020 and Fudan University Shanghai Cancer Center (Center 2) from March 2019 to August 2020. This multicenter study was approved by the Institutional Review Board of both participating hospitals. The need for individual informed consent was waived. Inclusion criteria were as follows: (1) patients with invasive breast cancer breast cancer who received SLND/ALND; (2) patients who did not receive any preoperative therapy and neoadjuvant chemotherapy before the DCE-MRI inspection; and (3) patients diagnosed by complete histopathological examination. Exclusion criteria were as follows: (1) patients with nonmass lesions without delineated boundaries or other malignancies; (2) patients who had surgery, neoadjuvant chemotherapy, radiotherapy, and/or SLND before the DCE-MRI examination; (3) poor DCE-MRI image quality; and (4) unavailable pathological information. Finally, a total of 263 patients were included, consisting of 108 patients with ALNM and 155 patients without ALNM. All patients from center 1 were randomly classified into the training set with 140 patients (61 with ALNM and 79 without ALNM) and the internal testing set with 61 patients (27 with ALNM and 34 without ALNM) for a ratio of 7:3. The 62 patients (20 with ALNM and 42 without ALNM) from center 2 were enrolled in the external testing set.

Patients with complete clinicopathologic data on age, tumor diameter, molecular subtype, estrogen receptor (ER), progesterone receptor (PR), human epidermal growth factor receptor (HER-2, express ≥2), Ki-67 proliferation index, and T stage were included. ER and PR were identified as positive when the threshold values ≥1%, and ≥20% was the threshold for Ki-67 [28]. Pathological classification of breast cancer was based on the hormone receptor expression and proliferation index [29]. Data were acquired from the patients’ medical records and pathology departments. The gold standard for evaluating ALNM is the pathological result.

All patients underwent ultrasound-guided SLNB or lymphadenectomy by surgery. Although SLNB is the current standard for determining the ALN stage, when the SLNB and ALND are inconsistent, the latter is the standard. Each patient’s ALN status was reconfirmed by two pathologists with 5 years of experience.

2.2 DCE-MRI acquisition

All MRI investigations were obtained on a 3.0T MRI system (GE Healthcare, USA, Discovery 750 W). During the scanning, the patients were examined in prone position with calm breathing by using a dedicated 8-channel breast coil, and the positioning line was parallel to the long axis of the breast in the sagittal plane. The scanning area included both mammary glands (naturally drooping and placed at the center of the coil) and both armpits.

The MRI sequences collected included axial T1WI, axial T2WI, DCE-MRI, axial diffusion weighted imaging, and sagittal contrast-enhanced imaging. The pertinent scanning parameters were as follows: (1) axial T1WI: TR = 460 ms, TE = 6.3 ms, slice thickness = 5 mm, and slice spacing = 1 mm; (2) T1 fat suppression was performed with DCE scan, and the contrast agent used was GD-DTPA. The filming of each patient was completed within 6 min, and a total of 8 phases were scanned with dose = 0.2 mmol/kg, TR = 5.7 ms, TE = 1.7 ms, slice thickness = 2 mm, slice spacing = 0 mm, FOV = 36 cm×36 cm, an matrix = 288×320; (3) axial fat suppression T2WI: TR = 5210 ms, TE = 84.7 ms, slice thickness = 5 mm, and slice spacing = 1 mm; (4) axial DWI (SE-EPI sequence): TR = 2496 ms, TE = 71.9 ms, slice thickness = 5 mm, slice spacing = 1 mm, B = 0/800 s/mm²; (5) sagittal contrast-enhanced imaging (performed after DCE): TR = 6.7 ms, TE = 1.7 ms, slice thickness = 2 mm, slice spacing = 0 mm, FOV = 28 cm×28 cm, and matrix = 200×256. The resulting MRI images were obtained using the Picture Archiving and Communication Systems.

2.3 Image segmentation and radiomics feature extraction

DCE-MRI was selected for radiomics analysis, and the peak enhanced phase of the multiphase contrast-enhanced MRI was selected for segmentation according to the time intensity curve. First, manual 3D segmentation of the primary tumor and ipsilateral ALN was performed using the ITK-SNAP software (Version 3.8.0) by a well-trained radiologist (reader 1) who had at least 10 years of breast imaging experience. The primary tumor and one of the most suspicious metastasis ALN of the ipsilateral axilla on the same phase were selected for segmentation. The radiologist was unaware of the ALN status and pathologic results. Then, the radiologist extracted the radiomics features from the region of interests (ROIs) of the primary tumors and ipsilateral lymph nodes separately for each case by using the “Radiomics Feature Extractor package” in the Pyradiomics software. Before the feature extraction, image preprocessing, including the standardization of the gray value of the ROI, discretization of gray level, and image resampling, was required by Pyradiomics software [30, 31].

2.4 Consistency assessment of the segmentation and feature extraction

Breast MRI images of 30 patients were randomly selected to evaluate the inter-observer and intra-observer agreement. Dice coefficient, also called the overlap index, was used to measure the segmentation performance (range of 0–1) between two radiologists. Inter- and intra-correlation coefficients (ICCs) were calculated to assess the reproducibility of the radiomics features. On the one hand, reader 1 repeated the step after two months to evaluate the intra-agreement. On the other hand, another radiologist (reader 2) used the same method to segment and extract the radiomics features to compare with reader 1 to evaluate the inter-agreement. Agreement was considered excellent for Dice coefficient > 0.75 and ICC > 0.80 [32].

2.5 Feature selection and radiomics signature construction

The radiomics features selected from the ROIs of primary tumors and ipsilateral lymph nodes were selected separately. A four-step method was used for feature selection. First, the radiomics features with both inter- and intra-correlation stability (with ICC values > 0.80) were selected through ICC analysis. Then, the variance threshold, SelectKBest, and least absolute shrinkage and selection operator (LASSO) methods were performed to reduce the redundant features and select optimal radiomics features [33]. The threshold in the variance threshold method was 0.8. Eigenvalues of the variance that were more than 0.8 in the variance threshold and features with P < 0.05 in SelectKBest method were included. LASSO-based feature selection determined the optimal LASSO alpha parameters through five-fold cross validation to avoid overfitting. Features with nonzero coefficients in the training set were ultimately selected. The selected features were weighted according to their respective coefficients. Radiomics signature (Radiomics signature score) predicting the risk of ALNM was calculated for each patient by using a linear combination of these selected features, which were weighted by their respective coefficients. Based on this procedure, rad-score-tumor and rad-score-LN were constructed from ROIs of primary tumor and ALN, respectively.

2.6 Development of the radiomics nomogram

The clinical factors included age, tumor diameter, ER, PR, HER2, Ki-67, molecular subtype, and T stage. First, the clinical risk factors, rad-score-tumor and rad-score-LN were determined using univariate logistic regression analysis. Then, the clinical risk factors, rad-score-tumor and rad-score-LN were integrated into the multivariate logistic regression analysis of backward-stepwise selection. During this procedure, collinearity was considered and the factor with variance inflation factor (VIF) >10 and P≥0.05 were eliminated. Akaike information criterion (AIC) was used to encourage model parsimony. $AIC = 2 p + n [log (SSE / n)]$ (1)

When the smallest AIC was reached, the stepwise procedure was stopped. Finally, the independent risk factors screened out by multivariate logistic regression were selected to construct the radiomics nomogram.

2.7 Assessment and validation of the radiomics nomogram

The receiver operating characteristic (ROC) of the area under the curve (AUC) was used to evaluate the predictive performance of the radiomics model, clinical model, and radiomics nomogram. Then, the accuracy, sensitivity, and specificity were calculated according to the optimal truncation value of these models as determined by the Youden index. The Hosmer-Lemeshow test and calibration curve were used to evaluate the consistency between the observed and predicted results. The predicted probability of the nomogram represented good agreement with the true state of ALN when the calibration curves were close fit to the diagonal line.

Decision curve analysis (DCA) was performed to assess the clinical utility and the net benefit of the nomogram in the testing set. The x-axis is the high risk threshold, and the y-axis is the net benefit obtained by subtracting the proportion of all patients who were false-positive from the proportion who were true-positive, weighed by the relative hazards of false-positive and false-negative results [34]. The larger the distance between the curve and the two extreme curves (all and no metastases), the higher was the clinical benefit.

2.8 Statistical analysis

All data obtained were statistically analyzed using Python (version 3.6), SPSS (version 26), and the R statistical software (version 3.4.1). The data set was divided into three sets. The training set was utilized to develop the radiomics models, while the internal and external testing sets were used to assess these models. The χ² test or Fisher’s exact test was utilized to compare the categorical variables (i.e., ER, PR, HER2, Ki-67, molecular subtype, and T stage). The two-sample t test was applied to compare the differences in clinical continuous variables (i.e., age and tumor diameter) between three groups. Furthermore, univariate and multivariate logistic regression were employed to screen risk factors. DCA, calibration curve, and ROC curve, which were conducted to evaluate the reliability of the nomogram, were evaluated using the R software with rmda, rms, and pROC packages, respectively. P < 0.05 was identified as a statistically significant difference.

3 Results

3.1 Clinical characteristics

The patients’ characteristics are presented in the Table 1. The ALNM rates of the training, internal testing, and external testing sets were 43.6%, 44.3% and 32.3%, respectively. The clinical factors, with the exception of tumor diameter, had no significant difference between the ALNM and nonmetastasis sets (P > 0.05), which justified their use as the training, internal testing, and external testing sets.

Table 1
Patient characteristics in three sets

Training set Internal testing set External testing set

(N = 140) (N = 61) (N = 62)

pN0 pN+ P pN0 pN+ P pN0 pN+ P

Age, years (mean±SD) 51.85±9.8 53.95±9.4 0.203 51.41±9.3 54.78±9.7 0.173 47.24±8.2 46.75±7.4 0.822

Diameter, cm (SD) 1.93±0.8 2.25±1.1 0.045* 1.73±0.9 2.81±1.1 0.000* 2.22±0.79 2.41±0.76 0.376

Molecular subtype 0.802 0.065 0.722

Luminal A 12 10 4 2 2 2

Luminal B 49 41 27 17 21 13

HER2 over-expression 6 4 0 5 4 1

Triple negative 12 6 3 3 12 4

ER status 0.430 0.084 0.308

Negative 20 12 4 9 16 5

Positive 59 49 30 18 26 15

PR status 0.255 0.087 0.303

Negative 25 14 5 10 17 7

Positive 54 47 29 17 25 13

Her-2 status 0.700 0.272 0.078

Negative 63 47 29 19 34 12

Positive 16 14 5 8 8 8

Ki-67 status 0.430 0.374 0.456

Negative 20 12 5 2 9 2

Positive 59 49 29 25 33 18

	Training set	Internal testing set	External testing set
Age, years (mean±SD)	51.85±9.8	53.95±9.4	0.203	51.41±9.3	54.78±9.7	0.173	47.24±8.2	46.75±7.4	0.822
Diameter, cm (SD)	1.93±0.8	2.25±1.1	0.045*	1.73±0.9	2.81±1.1	0.000*	2.22±0.79	2.41±0.76	0.376
Molecular subtype			0.802			0.065			0.722
Luminal A	12	10		4	2		2	2
Luminal B	49	41		27	17		21	13
HER2 over-expression	6	4		0	5		4	1
Triple negative	12	6		3	3		12	4
ER status			0.430			0.084			0.308
Negative	20	12		4	9		16	5
Positive	59	49		30	18		26	15
PR status			0.255			0.087			0.303
Negative	25	14		5	10		17	7
Positive	54	47		29	17		25	13
Her-2 status			0.700			0.272			0.078
Negative	63	47		29	19		34	12
Positive	16	14		5	8		8	8
Ki-67 status			0.430			0.374			0.456
Negative	20	12		5	2		9	2
Positive	59	49		29	25		33	18

Note: pN+, pathologically confirmed lymph node positive; pN0, pathologically confirmed lymph node negative. Chi-square test or Fisher’s exact test was used as appropriate to compare the differences in categorical variables, whereas the two-samplet test was utilized to compare the differences in age and tumor diameter. The ER and PR threshold values for each level were ≤1%, and the threshold value for Ki-67 was ≤20%. Asterisks indicate a statistically significant difference (^*P < 0.05).

3.2 Consistency assessment of segmentation and feature extraction

Dice coefficients ranged from 0.82 to 0.94 in the same reader 1 and from 0.83 to 0.93 in two different readers (readers 1 and 2), indicating satisfactory consistency in the manual ROI segmentation. The reader1 had longer had longer years of breast imaging experience than the reader 2, so the ROI selected was based on the segmentation of the reader1. The ICCs ranged from 0.783 to 0.945 in the same reader 1 and from 0.817 to 0.963 in two different readers (readers 1 and 2), indicating the good reproducibility of radiomics feature extraction.

3.3 Construction and performance of the radiomics signature

We extracted 1409 radiomics features from each ROI of ALN and tumor, respectively, and the features were divided into three broad categories: first-order, shape, and texture features. A total of 11 and 17 radiomics features were finally selected from the tumor and ALN ROIs (Fig. 1A–D) (Table 2). In the three sets, significant difference was noted in both ALN and tumor radiomics signature between the ALN-positive and ALN-negative patients (P < 0.01). The ALN radiomics signature yielded AUCs of 0.844 (95% confidence interval [CI], 0.790–0.892) in the training set, 0.766 (95% CI, 0.644–0.854) in the internal testing set, and 0.723 (95% CI, 0.581–0.845) in the external testing set, while the primary tumor radiomics signature was developed with the AUC of 0.773 (95% CI, 0.701–0.833) in the training set, 0.759 (95% CI, 0.674–0.885) in the internal testing set, and 0.723 (95% CI, 0.604–0.839) in the external testing set (Fig. 2A–C). The radiomics signature combined with ALN and primary tumor exhibited better than they predicted alone, producing AUCs of 0.889 (95% CI, 0.837–0.934), 0.817 (95% CI, 0.708–0.906), and 0.811 (95% CI, 0.708–0.905) in the training, internal testing, and external testing sets, respectively (Fig. 5A–C).

Fig. 1

Lasso algorithm for radiomics features selection. (A) Least absolute shrinkage and selection operator (LASSO) coefficient profiles of the 516 features extracted from the primary tumor. The y-axis represents the coefficient of each feature. The optimal value of alpha was 0.0027, and the optimal -log(alpha) was 2.57, where 11 features with nonzero coefficient were selected. (B) Mean square error path using five-fold cross-validation of primary tumor. (C) LASSO coefficient profiles of 491 features extracted from ALN. The y-axis represents the coefficient of each feature. The optimal value of alpha was 0.034, and the optimal -log(alpha) was 1.47, where 17 features with nonzero coefficient were selected. (D) Mean square error path using five-fold cross-validation of ALN.

Table 2

Least absolute shrinkage and selection operator (LASSO) coefficient profiles of the features

	Radiomics Features	Coefficients
Primary tumor	wavelet-HLH_glszm_ZoneEntropy	0.078366207
	wavelet-LHH_glszm_ZoneEntropy	0.056225618
	wavelet-HLL_glszm_ZoneEntropy	0.03187748
	wavelet-LLL_glszm_HighGrayLevelZoneEmphasis	0.116799495
	wavelet-HHH_glszm_ZoneEntropy	0.013458237
	wavelet-LLL_glszm_SmallAreaHighGrayLevelEmphasis	0.042440583
	wavelet-HHL_glszm_SmallAreaLowGrayLevelEmphasis	0.055142309
	wavelet-HHL_glcm_SumEntropy	0.041245032
	wavelet-LLL_glszm_ZoneEntropy	–0.12751972
	wavelet-LHH_glcm_SumSquares	0.045358583
	exponential_firstorder_InterquartileRange	0.079596884
ALN	wavelet-LLL_glrlm_LongRunLowGrayLevelEmphasis	0.049840493
	wavelet-HLH_glrlm_ShortRunLowGrayLevelEmphasis	–0.045451184
	original_shape_Flatness	0.084558616
	wavelet-HLH_firstorder_Mean	–0.024598437
	wavelet-HHH_glrlm_ShortRunLowGrayLevelEmphasis	–0.006005458
	wavelet-LLH_glrlm_ShortRunHighGrayLevelEmphasis	–0.001150125
	wavelet-HHL_firstorder_Skewness	–0.021560956
	wavelet-LLH_gldm_DependenceNonUniformityNormalized	0.026586166
	wavelet-HLH_gldm_DependenceVariance	–0.018734669
	wavelet-LHH_firstorder_Median	–0.025508322
	wavelet-HHL_gldm_LargeDependenceHighGrayLevelEmphasis	0.00292846
	wavelet-HHL_glszm_SizeZoneNonUniformityNormalized	–0.022267328
	wavelet-HHL_glszm_SizeZoneNonUniformity	0.009361118
	wavelet-LHL_gldm_DependenceNonUniformityNormalized	–0.003679422
	wavelet-HLH_glszm_SmallAreaLowGrayLevelEmphasis	0.004213029
	wavelet-LLL_glszm_SizeZoneNonUniformity	–0.00317445
	wavelet-LLH_glszm_SmallAreaLowGrayLevelEmphasis	0.011510372

Note: glszm, gray level size zone matrix; glcm, gray level co-occurrence matrix; glrlm, gray level run length matrix; gldm, gray level dependence matrix.

Fig. 2

ROC curves of the radiomics signature for predicting ALNM. ROC curves of the radiomics signature for ALNM prediction in the (A) training set, (B) internal testing set, and (C) external testing set.

Fig. 5

ROC curves of the radiomics signature score, clinical model, and radiomics nomogram in the three sets: ROC curves of three models for the ALNM prediction in the (A) training set, (B) internal testing set, and (C) external testing set.

3.4 Development and validation of the radiomics nomogram model

The univariate logistic regression showed that the rad-score-tumor, rad-score-LN and T stage were risk factors associated with ALNM. Among them, the tumor diameter and T stage were clinical risk factors. After multivariate logistic regression combined with backward stepwise regression, only the rad-score-tumor and rad-score-LN were the final risk factors, while no clinical risk factors of ALNM prediction were found. Tumor diameter has been reported to be a clinical risk factor for ALNM in breast tumors [35, 36]. Thus, tumor diameter was added in this study as the final clinical risk factor. Thereby, the radiomics nomogram model contained tumor diameter and DCE-MRI-based radiomics signature of the primary tumor and ALN (Fig. 3). The Hosmer-Lemeshow test and the calibration curve (the x-axis was the nomogram-predicted probability, and the y-axis was the actual outcome) (Fig. 4) were used to evaluate the consistency between the true state of ALN and the predicted probability of the nomogram model. Excellent results were obtained in the training (P = 0.877), internal testing (P = 1.00), and external testing (P = 1.00) sets.

Fig. 3

Radiomics nomogram for the prediction of ALNM.

Fig. 4

Calibration curves of radiomics nomogram in the three sets. Note: The diagonal line represents the perfect prediction of the radiomics nomogram. The red, blue, and black solid lines represent the calibration curve of radiomics nomogram in the training, internal testing, and external testing sets, respectively. Calibration curves indicate that the predicted probability has a good agreement with the actual state of ALN.

The sensitivity, accuracy, and specificity values of the nomogram model in the internal testing set were 0.815, 0.787, and 0.765, respectively. The corresponding values for the external testing set were 0.600, 0.790, and 0.881. The nomogram yielded AUCs of 0.822 (95% CI, 0.713–0.911) in the internal testing set and 0.813 (95% CI, 0.711–0.909) in the external testing set. As shown in the Fig. 5A–C and Table 3, the AUCs of the radiomics nomogram were higher than those of radiomics signature and clinical model. The DCA (Fig. 6A–C) in the three sets demonstrated that if the threshold probability ranged from 0.1 to 1.0, using the nomogram model to predict ALNM added more net benefit than either “treat all” or “treat none” scheme. The DCA also demonstrated that the radiomics nomogram displayed better clinical predictive usefulness than the clinical or radiomics signature alone.

Table 3

Predictive performance of three models in three sets

		Training set				Internal testing set			External testing set
	AUC	Sensitivity	Specificity	AUC	Sensitivity	Specificity	AUC	Sensitivity	Specificity
	(95% CI)	(95% CI)	(95% CI)	(95% CI)	(95% CI)	(95% CI)	(95% CI)	(95% CI)	(95% CI)
Rad-score	0.889	0.852	0.785	0.817	0.741	0.824	0.811	0.600	0.881
	(0.837– 0.934)	(0.733– 0.926)	(0.675– 0.866)	(0.708– 0.906)	(0.534– 0.881)	(0.648– 0.926)	(0.708– 0.905)	(0.364– 0.800)	(0.364– 0.800)
Clinical model	0.593	0.508	0.620	0.783	0.556	0.853	0.590	0.900	0.238
	(0.510– 0.671)	(0.378– 0.637)	(0.504– 0.725)	(0.703– 0.898)	(0.356– 0.740)	(0.682– 0.945)	(0.468– 0.713)	(0.669– 0.982)	(0.126– 0.398)
Radiomics nomogram	0.884	0.770	0.848	0.822	0.815	0.765	0.813	0.600	0.881
	(0.830– 0.930)	(0.642– 0.865)	(0.746– 0.916)	(0.713– 0.911)	(0.613– 0.930)	(0.584– 0.886)	(0.711– 0.909)	(0.364– 0.800)	(0.736– 0.955)

Fig. 6

Decision curve analysis for the prediction models in the three sets: (A) training set, (B) internal testing set, and (C) in the external testing set Note: The y-axis represents the net benefits, and the x-axis represents the threshold probability. The red and blue lines represent the Rad-scores of the primary tumor and ALN, respectively. The green line represents the clinical model. The black line represents the radiomics nomogram. The gray line represents the assumption that all patients were included in the benign group. The thin black line represents the assumption that all patients were included in malignant group. The decision curve shows that the radiomics nomogram can add more net benefit than “none” or “all” treatment with the threshold probability range from 0 to 1.0 in the three sets.

Hence, the above results indicated that the radiomics nomogram performed well in the calibration, discrimination, and clinical application.

4 Discussion

ALNM is considered as the key factor affecting the prognosis and treatment of patients with breast cancer [37]. However, the methods for assessing the ALN status from biopsy to lymphadenectomy are invasive. Therefore, noninvasive ALN preoperative assessment tools have great application prospects. This multicenter study explored the potential of radiomics based on DCE-MRI in predicting ALNM. The radiomics nomogram model, which combined clinical risk factor and radiomics features (primary tumor and ALN), showed the best predictive performance, with AUCs of 0.822 and 0.813 in the internal and external testing sets, respectively.

Breast cancer has many distinct subtypes and may exhibit radiographic heterogeneity. Traditional imaging methods, such as ultrasonography, MRI, and mammography, play a role in evaluating the ALN morphology [38]. High-resolution preoperative ultrasound is a primary method for the preoperative diagnosis of the evaluation of ALNM, but morphologic features showed poor discriminatory power for the diagnosis of ALNM [39, 40]. Although preoperative MRI imaging enhances the detection of potentially metastatic lymph nodes, enlarged lymph nodes are poorly distinguished from reactive inflammatory lymph nodes or normal lymph nodes by traditional MRI, resulting in the incomplete detection of ALNM [41, 42]. Nowadays, MRI images could reflect remarkably greater information [43], including intensity, shape, and texture [20, 44]. Compared with traditional MRI, the radiomics features improve the accuracy of preoperative ALNM prediction.

In previous radiomics studies, most of the studies on ALNM have merely focused on the primary radiomics features of breast tumor [6 , 45], and the ipsilateral ALN can also provide considerable amount of information. Liu et al. [24] established radiomics signature extracted from DCE-MRI to predict noninvasively whether ALN had metastasis and produced AUCs of 0.806. Similarly, Cui et al. [25] performed AUCs of 0.8658, 0.7041, and 0.6837 by the support vector machine, K nearest neighbors, and linear discriminant Analysis, respectively. Then, they established a nomogram that scored morphological and texture features to calculate the probability of ALNM. Moreover, Dong et al. [46] were the first to predict the status of SLN using radiomics based on T2-FS and DWI sequences, which yielded an AUC of 0.863 in the training set and 0.805 in the validation set. In addition, patients with invasive breast cancer were investigated by Shan et al. [27], they validated a nomogram model to detect ALNM that incorporated the kinetic curve model and radiomics features extracted from DCE-MRI.

The present radiomics signature combined with ALN radiomics and primary tumor outperformed with AUCs of 0.817 and 0.811 in the internal and external testing sets, respectively. These results confirmed the predictive value of the radiomics signature combined with the ALN radiomics and primary tumor was better than they predicted alone in ALNM in patients with breast cancer. This result demonstrated the feasibility of identifying predictive radiomics features from ALN regions.

Furthermore, many previous studies have predicted ALNM solely based on the clinical information or radiomics features [12 , 47–51]. Mao et al. [26] substantiated that MRI-based radiomics nomogram showed better performance to visualize the risk of ALNM by doctors. In addition, these studies established a radiomics nomogram based on contrast-enhanced spectral mammography (CESM) for prediction of ALNM in breast cancer, and the nomogram model, which included the radiomics signature and the CESM-reported ALN status, had areas under the ROC curves of 0.79 in the external validation set [47]. Yu et al. [48] developed and validated a clinical-radiomics nomogram that successfully stratified patients with early-stage breast cancer according to their risk of ALNM. Furthermore, a nomogram, that can be used to distinguish the number of metastatic LNs (less than two positive nodes/more than two positive nodes), was developed based on 12 LN status-related features by Han et al. [52].

Thus, in this study, the visualization nomogram model was used. The model merged radiomics signature (i.e., primary tumor and ALN) with clinical risk factors with AUCs of 0.822 and 0.813 in the internal and external testing sets, respectively. The final conclusion confirmed that the nomogram model had a higher efficiency than a single clinical or radiomics model. The radiomics nomogram provide a more exact multiomics method for predicting the ALNM.

This study offers some notable advantages on improving the reliability and usability of the findings. First, this work was a multicenter study that involved patients from different districts of China and provides additional radiomics evidence. The external testing set was used to verify the universality of the proposed model, and good prediction ability with AUC of 0.813 was shown. Second, the Dice coefficient was utilized to test the consistency of segmentation. Finally, ICCs were calculated to verify the reproducibility of the radiomics feature extraction.

The kinetic image features based on DCE-MRI, which can provide kinetic parameters that reflect the permeability and flow characteristics of the vessels within the lesion of breast cancer, is commonly used in clinical practice. The combination of them may improve the AUC in predicting the molecular typing of breast cancer, the efficacy of neoadjuvant chemotherapy and the prognosis [53 –56]. Liu et al. [56] established radiomics model, the pharmacokinetic parameters model, and the combined model based on DCE-MRI for predicting SLN metastasis in patients with breast cancer. The three models yielded AUC values of 0.74, 0.74 and 0.76, respectively, for the validation cohort. So, the performance of ALN prediction by the combination of radiomics and kinetic image features can be further studied.

The present study also has several potential limitations. First, this work is a retrospective study with a limited number of enrolled patients. The evaluation of generalization and robustness of the nomogram was lacking even though a multicenter sample was used. Thus, in the future, the authors should incorporate a double-blinded prospective design with more rigorous analyses of large-sample data to overcome these limitations. Second, the ROIs of the tumor and ALN were delineated manually by experienced radiologists, and this process may have increased variability between observers. Although the Dice coefficient exhibited good consistency of the manual segmentation in this study, an automatic or semi-automatic method for segmentation showed higher stability [57 –59]. In addition, recent evidence has shown that the combination of radiomics and genomics might better guide clinical decision-making [60, 61]. Thus, the authors will develop a combined radiomics and genomics model in the future to predict the ALNM accurately.

5 Conclusion

The radiomics nomogram, which combined with clinical risk factors and DCE-MRI-based radiomics signature of primary tumor and ALN, could be used to predict the ALNM. The nomogram performed well in the calibration, discrimination, and clinical application to avoid overtreatment for patients with primary breast cancer. Therefore, the model can help clinicians to improve individual treatment and generate optimal clinical decisions.

Statements and declarations

Competing interests: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Funding

The research fund was provided by General Project of Natural Science Foundation of Shanxi Province(201901D111347), Fund Program for the Scientific Activities of Selected Returned Overseas Professional in Shanxi Province (20200036) and Research Project Supported by Shanxi Scholarship Council of China (2021-157).

Author contributions

JZ wrote the manuscript. ZZ and NM contributed to segment and extract the radiomics features. WL and JG analyzed the statistical results. HJ and YW contributed to the conception of the study. HZ, BW, JR, XL, BZ and DT assisted with this research and collected the case data. All authors approved the final manuscript.

Footnotes

Acknowledgments

This study was supported by Shanxi Medical University.

Ethics

This multicenter study was approved by the Institutional Review Board of both participating hospitals. Informed consent from the study participants was exempted as it was a retrospective study.

References

Siegel

R.L.

, Miller

K.D.

, Fuchs

H.E.

and Jemal

, Cancer statistics, CA Cancer J Clin 71(1) (2021), 7–33.

Mao

, Yin

, Li

, et al., Radiomics nomogram of contrast-enhanced spectral mammography for prediction of axillary lymph node metastasis in breast cancer: A multicenter study, Eur Radiol 30(12) (2020), 6732–6739.

Solá

, Margelí

, Castellá

, et al., Prognosticvalue of hematogenous dissemination and biological profile of thetumor in early breast cancer patients: A prospective observational study, BMC Cancer 11 (2011), 252.

Majid

, Rydén

and Manjer

, Determinants for non-sentinel node metastases in primary invasive breast cancer: A population-based cohort study of 602 consecutive patients with sentinel node metastases, BMC Cancer 19(1) (2019), 626.

Yoon

K.H.

, Park

, Kim

J.Y.

, et al., Is the frozen section examination for sentinel lymph node necessary in early breast cancer patients? Ann Surg Treat Res 97(2) (2019), 49–57.

Santucci

, Faiella

, Cordelli

, et al., 3T MRI-radiomic approach to predict for lymph node status in breast cancer patients, Cancers (Basel) 13(9) (2021), 2228.

Dihge

, Bendahl

P.O.

and Rydén

, Nomograms for preoperative prediction of axillary nodal status in breast cancer, Br J Surg 104(11) (2017), 1494–1505.

, He

, Ouyang

, et al., Magnetic resonance imaging radiomics predicts preoperative axillary lymph node metastasis to support surgical decisions and is associated with tumor microenvironment in invasive breast cancer: A machine learning, multicenter study, EBioMedicine 69 (2021), 103460.

Moorman

A.M.

, Rutgers

E.J.T.

and Kouwenhoven

E.A.

, Omitting SLNB in breast cancer: Is a nomogram the answer? Ann Surg Oncol 29(4) (2022), 2210–2218.

10.

Hindié

, Groheux

, Brenot-Rossi

, et al., The sentinel node procedure in breast cancer: Nuclear medicine as the starting point, J Nucl Med 52(3) (2011), 405–414.

11.

Kang

S.R.

, Kim

H.W.

and Kim

H.S.

, Evaluating the relationship between dynamic contrast-enhanced MRI (DCE-MRI) parameters and pathological characteristics in breast cancer, J Magn Reson Imaging 52(5) (2020), 1360–1373.

12.

Kvistad

K.A.

, Rydland

, Smethurst

H.B.

, et al., Axillary lymph node metastases in breast cancer: Preoperative detection with dynamic contrast-enhanced MRI, Eur Radiol 10(9) (2000), 1464–1471.

13.

Zhao

, Zhang

, Zhu

Q.L.

, et al., The value of contrast-enhanced ultrasound for sentinel lymph node identification and characterisation in pre-operative breast cancer patients: A prospective study, Eur Radiol 28(4) (2018), 1654–1661.

14.

Fusco

, Sansone

, Granata

, et al., Use of quantitative morphological and functional features for assessment of axillary lymph node in breast dynamic contrast-enhanced magnetic resonance imaging, Biomed Res Int 2018 (2018), 2610801.

15.

Lambin

, Rios-Velazquez

, Leijenaar

, et al., Radiomics: Extracting more information from medical images using advanced feature analysis, Eur J Cancer 48(4) (2012), 441–446.

16.

Whitney

H.M.

, Taylor

N.S.

, Drukker

, et al., Additive benefit of radiomics over size alone in the distinction between benign lesions and luminal A cancers on a large clinical breast MRI dataset, Acad Radiol 26(2) (2019), 202–209.

17.

Agner

S.C.

, Rosen

M.A.

, Englander

, et al., Computerized image analysis for identifying triple-negative breast cancers and differentiating them from other molecular subtypes of breast cancer on dynamic contrast-enhanced MR images: A feasibility study, Radiology 272(1) (2014), 91–99.

18.

Sutton

E.J.

, Onishi

, Fehr

D.A.

, et al., A machine learning model that classifies breast cancer pathologic complete response on MRI post-neoadjuvant chemotherapy, Breast Cancer Res 22(1) (2020), 57.

19.

Xie

, Wang

, Zhao

, et al., Machine learning-based analysis of MR multiparametric radiomics for the subtype classification of breast cancer, Front Oncol 9 (2019), 505.

20.

Aerts

H.J.

, The potential of radiomic-based phenotyping in precision medicine: A review, JAMA Oncol 2(12) (2016), 1636–1642.

21.

Pinker

, Chin

, Melsaether

A.N.

, et al., Precision medicine and radiogenomics in breast cancer: New approaches toward diagnosis and treatment, Radiology 287(3) (2018), 732–747.

22.

Yamaguchi

, Abe

, Newstead

G.M.

, et al., Intratumoral heterogeneity of the distribution of kinetic parameters in breast cancer: Comparison based on the molecular subtypes of invasive breast cancer, Breast Cancer 22(5) (2015), 496–502.

23.

Calabrese

, Santucci

, Landi

, et al., Radiomics MRI for lymph node status prediction in breast cancer patients: The state of art, J Cancer Res Clin Oncol 147(6) (2021), 1587–1597.

24.

Liu

, Ding

, Spuhler

, et al., Preoperative prediction of sentinel lymph node metastasis in breast cancer by radiomic signatures from dynamic contrast-enhanced MRI, J Magn Reson Imaging 49(1) (2019), 131–140.

25.

Cui

, Wang

, Zhao

, et al., Preoperative prediction of axillary lymph node metastasis in breast cancer using radiomics features of DCE-MRI, Sci Rep 9(1) (2019), 2240.

26.

Mao

, Dai

, Lin

, et al., Radiomics nomogram of DCE-MRI for the prediction of axillary lymph node metastasis in breast cancer, Front Oncol 10 (2020), 541849.

27.

Shan

Y.N.

, Xu

, Wang

, et al., A nomogram combined radiomics and kinetic curve pattern as imaging biomarker for detecting metastatic axillary lymph node in invasive breast cancer, Front Oncol 10 (2020), 1463.

28.

Goldhirsch

, Winer

E.P.

, Coates

A.S.

, et al., Personalizing the treatment of women with early breast cancer: Highlights of the St Gallen International Expert Consensus on the Primary Therapy of Early Breast Cancer, Ann Oncol 24(9) (2013), 2206–2223.

29.

Gradishar

W.J.

, Moran

M.S.

, Abraham

, et al., NCCN guidelines^® insights: Breast cancer, version 4, J Natl Compr Canc Netw 19(5) (2021), 484–493.

30.

, Li

, Mou

, et al., Radiomics nomogram for identifying Sub-1cm benign and malignant thyroid lesions, Front Oncol 11 (2021), 580886.

31.

Zhang

, Hu

, Ma

, et al., Application of enhanced computed tomography-based radiomics nomogram analysis to differentiate metastatic ovarian tumors from epithelial ovarian tumors, J Xray Sci Technol 30(6) (2022), 1185–1199.

32.

Bilgili

, Saglam

and Göksan

S.B.

, et al., Treatment of Graf type IIa hip dysplasia: A cut-off value for decision making, Balkan Med J 35(6) (2018), 427–430.

33.

Lin

, Wang

, Zhang

, et al., Contrast-enhanced spectral mammography-based radiomics nomogram for identifying benign and malignant breast lesions of sub-1cm, Front Oncol 10 (2020), 573630.

34.

, Zhang

, Zhu

and Tang

, Application of a prediction model with laboratory indexes in the risk stratification of patients with COVID-19, Exp Ther Med 21(3) (2021), 182.

35.

Miguchi

, Takakura

, Egi

, et al., Malignant peripheral nerve sheath tumor arising from the greater omentum: Case report, World J Surg Oncol 9 (2011), 33.

36.

Wang

, Luo

, Li

, et al., Recurrence pattern of squamous cell carcinoma in the midthoracic esophagus: Implications for the clinical target volume design of postoperative radiotherapy, Onco Targets Ther 9 (2016), 6021–6027.

37.

Kong

, Ding

, Li

, et al., 53BP1 suppresses epithelial-mesenchymal transition by downregulating ZEB1 through microRNA-200b/429 in breast cancer, Cancer Sci 106(8) (2015), 982–989.

38.

Y.S.

, Lee

D.H.

, Yoon

J.K.

, et al., Diagnostic performance of 18F-FDG PET/CT, ultrasonography and MRI. Detection of axillary lymph node metastasis in breast cancer patients, Nuklearmedizin 53(3) (2014), 89–94.

39.

Thakur

S.B.

, Durando

, Milans

, et al., Apparent diffusion coefficient in estrogen receptor-positive and lymph node-negative invasive breast cancers at 3.0T DW-MRI: A potential predictor for an oncotype Dx test recurrence score, J Magn Reson Imaging 47(2) (2018), 401–409.

40.

Zhang

, Shi

, Xiao

, et al., Early SUVmax is the best predictor of axillary lymph node metastasis in stage III breast cancers, Quant Imaging Med Surg 11(5) (2021), 1680–1691.

41.

Zhang

, Zheng

, Yang

, et al., Axillary sentinel lymph nodes in breast cancer: Quantitative evaluation at dual-energy CT, Radiology 289(2) (2018), 337–346.

42.

Sun

, Zhang

, Li

, et al., Image-guided surgery of head and neck carcinoma in rabbit models by intra-operatively defining tumour-infiltrated margins and metastatic lymph nodes, EBioMedicine 50 (2019), 93–102.

43.

Gillies

R.J.

, Kinahan

P.E.

and Hricak

, Radiomics: Images are more than pictures, They are data, Radiology 278(2) (2016), 563–577.

44.

Marusyk

, Almendro

and Polyak

, Intra-tumour heterogeneity: A looking glass for cancer? Nat Rev Cancer 12(5) (2012), 323–334.

45.

Yun

S.J.

, Sohn

Y.M.

and Seo

, Re: Differentiation of benign and metastatic axillary lymph nodes in breast cancer: Additive value of MRI computer-aided evaluation, A Reply, Clin Radiol 71(7) (2016), 711.

46.

Dong

, Feng

, Yang

, et al., Preoperative prediction of sentinel lymph node metastasis in breast cancer based on radiomics of T2-weighted fat-suppression and diffusion-weighted MRI, Eur Radiol 28(2) (2018), 582–591.

47.

Mao

, Yin

, Li

, et al., Radiomics nomogram of contrast-enhanced spectral mammography for prediction of axillary lymph node metastasis in breast cancer: A multicenter study, Eur Radiol 30(12) (2020), 6732–6739.

48.

, Tan

, Xie

, et al., Development and validation of a preoperative magnetic resonance imaging radiomics-based signature to predict axillary lymph node metastasis and disease-free survival in patients with early-stage breast cancer, JAMA Netw Open 3(12) (2020), e2028086.

49.

Sodano

, Clauser

, Dietzel

, et al., Clinical relevance of total choline (tCho) quantification in suspicious lesions on multiparametric breast MRI, Eur Radiol 30(6) (2020), 3371–3382.

50.

, Yao

, Xiao

, et al., Contrast-enhanced CT radiomics for predicting lymph node metastasis in pancreatic ductal adenocarcinoma: A pilot study, Cancer Imaging 20(1) (2020), 12.

51.

Parmar

, Rios Velazquez

, Leijenaar

, et al., Robust Radiomics feature quantification using semiautomatic volumetric segmentation, PLoS One 9(7) (2014), e102107.

52.

Han

, Zhu

, Liu

, et al., Radiomic nomogram for prediction of axillary lymph node metastasis in breast cancer, Eur Radiol 29(7) (2019), 3820–3829.

53.

Jiang

, Xie

, Liu

, et al., Discrimination of malignant and benign breast masses using automatic segmentation and features extracted from dynamic contrast-enhanced and diffusion-weighted MRI, Oncol Lett 16(2) (2018), 1521–1528.

54.

, Ji

, Qi

, et al., Breast cancer Ki67 expression prediction by DCE-MRI radiomics features, Clin Radiol 73(10) (2018), 909.e1–909.e5.

55.

Whitney

H.M.

, Drukker

, Edwards

, et al., Effect of biopsy on the MRI radiomics classification of benign lesions and luminal A cancers, J Med Imaging (Bellingham) 6(3) (2019), 031408.

56.

Liu

, Mao

, Ma

, et al., Pharmacokinetic parameters and radiomics model based on dynamic contrast enhanced MRI for the preoperative prediction of sentinel lymph node metastasis in breast cancer, Cancer Imaging 20(1) (2020), 65.

57.

de Hoop

, Gietema

, van Ginneken

, et al., A comparison of six software packages for evaluation of solid lung nodules using semi-automated volumetry: What is the minimum increase in size to detect growth in repeated CT examinations, Eur Radiol 19(4) (2009), 800–808.

58.

Chai

, Ma

, Xu

, et al., Differentiating axillary lymph node metastasis in invasive breast cancer patients: A comparison of radiomic signatures from multiparametric breast MR sequences, J Magn Reson Imaging 50(4) (2019), 1125–1132.

59.

Yeh

A.C.

, Li

, Zhu

, et al., Radiogenomics of breast cancer using dynamic contrast enhanced MRI and gene expression profiling, Cancer Imaging 19(1) (2019), 48.

60.

Yamamoto

, Maki

D.D.

, Korn

R.L.

and Kuo

M.D.

, Radiogenomic analysis of breast cancer using MRI: A preliminary study to define the landscape, AJR Am J Roentgenol 199(3) (2012), 654–663.

61.

Bismeijer

, van der Velden

B.H.M.

, Canisius

, et al., Radiogenomic analysis of breast cancer by linking MRI phenotypes with tumor gene expression, Radiology 296(2) (2020), 277–287.