CT-based intratumoral and peritumoral deep transfer learning features prediction of lymph node metastasis in non-small cell lung cancer

Abstract

BACKGROUND:

The main metastatic route for lung cancer is lymph node metastasis, and studies have shown that non-small cell lung cancer (NSCLC) has a high risk of lymph node infiltration.

OBJECTIVE:

This study aimed to compare the performance of handcrafted radiomics (HR) features and deep transfer learning (DTL) features in Computed Tomography (CT) of intratumoral and peritumoral regions in predicting the metastatic status of NSCLC lymph nodes in different machine learning classifier models.

METHODS:

We retrospectively collected data of 199 patients with pathologically confirmed NSCLC. All patients were divided into training (n = 159) and validation (n = 40) cohorts, respectively. The best HR and DTL features in the intratumoral and peritumoral regions were extracted and selected, respectively. Support Vector Machine (SVM), k-Nearest Neighbors (KNN), Light Gradient Boosting Machine (Light GBM), Multilayer Perceptron (MLP), and Logistic Regression (LR) models were constructed, and the performance of the models was evaluated.

RESULTS:

Among the five models in the training and validation cohorts, the LR classifier model performed best in terms of HR and DTL features. The AUCs of the training cohort were 0.841 (95% CI: 0.776–0.907) and 0.955 (95% CI: 0.926–0.983), and the AUCs of the validation cohort were 0.812 (95% CI: 0.677–0.948) and 0.893 (95% CI: 0.795–0.991), respectively. The DTL signature was superior to the handcrafted radiomics signature.

CONCLUSIONS:

Compared with the radiomics signature, the DTL signature constructed based on intratumoral and peritumoral areas in CT can better predict NSCLC lymph node metastasis.

1 Background

Lung cancer is the second most prevalent malignancy worldwide, accounting for approximately 11.4% of new cancer cases in 2020, and the leading cause of cancer-related deaths, accounting for approximately 18% of cancer-related deaths due to cancer [1]. Non-small cell lung cancer (NSCLC) accounts for approximately 82% of all lung cancer cases [2]. The main metastatic route for lung cancer is lymph node metastasis, and studies have shown that NSCLC has a high risk of lymph node infiltration [3, 4]. It is the main cause of tumor recurrence and distant metastases in patients after surgery. The postoperative survival rates also differ significantly for NSCLC patients with different lymph node statuses [5]. The 5-year survival rate for patients with no mediastinal or hilar lymph node metastases (N0) is approximately 56%, and for those with lymph node metastases (N1, N2, and N3), it is approximately 38%, 22%, and 6%, respectively [6]; therefore, the higher the N-stage of lymph node metastasis, the lower the survival rate of patients. Thus, preoperative determination of whether lymph node metastasis has occurred in NSCLC is important for individualized treatment and prognosis.

Conventional imaging methods for the preoperative assessment of lymph node metastases are inadequate, with Computed Tomography (CT) and Magnetic Resonance (MR) relying on changes in lymph node size to determine the presence of lymph node metastases: a short lymph node diameter>10 mm commonly used as a criterion [7]. However, the accuracy of determining whether a lymph node is metastatic based solely on changes in imaging morphology is low, and it is easy to misdiagnose enlarged lymph nodes with inflammatory responses as lymph node metastases, resulting in patients without lymph node metastases being overly cleared intraoperatively. However, metastatic lymph nodes<10 mm are easily missed, with a sensitivity rate of 51% [5, 7], resulting in patients who should have had their lymph nodes cleared intraoperatively and instead did not receive this treatment. Positron Emission Tomography/ Computed Tomography (PET/CT) has a higher accuracy than CT and MR; however, it has a relatively high false-positive rate and is easily confused with inflammatory processes and granulomatous infections in the early stages of lung cancer [8, 9]; the sensitivity of detecting lymph node metastases in lymph nodes<10 mm is only 32.4% [10]. Therefore, conventional imaging methods are limited in their ability to accurately assess preoperative lymph node metastasis in NSCLC.

Solid NSCLC tumors can spread through the tracheal, vascular, and lymphatic systems into the surrounding lung parenchyma, and there are areas of subclinical infiltration around the solid tumor [11, 12]. Studies have shown that spreading of lung adenocarcinoma tumor through the airspace is a significant predictor of occult lymph node metastasis and that the smaller the extent of tumor resection, the higher the probability of lymph node metastasis [13]. This subclinical stage of microscopic invasion cannot be detected by physicians using conventional imaging methods; however, the tissue surrounding the tumor is infiltrated and may spread distantly via lymphatics and blood vessels. Therefore, there is a need for a new examination technique that can predict the areas of peritumoral subclinical infiltration at the microscopic level.

Recently, radiomics, a technique that can quantify the biology of a lesion and heterogeneity within a tumor at a microscopic level, has been widely studied [14 –16]. Radiomics breaks away from the traditional morphology-based medical imaging paradigm and allows access to biological behavioral diagnostic information that is not visible to the naked eye. This technical approach can be applied to the analysis of peritumoral microenvironmental alterations. Some researchers extracted CT imaging histological features from the peritumoral region of the lung and found that the inclusion of distal peritumoral imaging histological features could improve the predictive power of the model [17]. However, radiomics features, which can only manually extract the texture, size, volume, shape, and intensity features of tumors, also have limitations. Therefore, new methods, such as deep learning, are required to extract more complex and higher-dimensional features, potentially improving the predictive and generalization capabilities of the model [18 –20].

Deep convolutional neural networks (CNN) have recently achieved remarkable results in computer vision for tumor grading prediction, patient prognosis, pathological classification, and organ segmentation [21, 22]. Compared to handcrafted radiomics(HR) features, deep learning features reflect information in medical images from a different perspective and at a deeper level and may add predictive value to the status of lymph node metastasis in NSCLC. However, deep learning models require a large number of image sets for training, and medical image sets are often limited compared with natural image sets, making it difficult to train CNN models from scratch. Therefore, deep transfer learning (DTL) has been proposed to overcome this shortcoming [23, 24]. DTL uses a model pre-trained from images in other domains, applying what the model learns during training to perform a specific task and using it to perform another potentially irrelevant new dataset [25]. Currently, DTL is widely used in the field of deep learning to alleviate the limitations of small datasets. Therefore, this study aimed to investigate the prediction of NSCLC lymph node metastasis status based on HR features and DTL features on intratumoral and peri-tumoral preoperative CT images and compare the differences between them, thus providing a complementary aid to the radiological assessment of patients.

2 Image preprocessing related work

The CT scan parameters are presented in Supplementary Table. All patient images were retrieved from our Picture Archiving and Communication Systems (PACS)and exported in Digital Imaging and Communications in Medicine Format for enhanced CT images of the patients’ chest. First, to eliminate parameter interference from different machines, we resampled all patient CT images into 1 ×1 ×1 mm voxels. Two radiologists with over five years of experience outlined the regions of interest (ROIs) in the images, which were then combined into a volume of interest. A week later, CT images of a random sample of 30 patients were re-outlined to calculate the intraclass correlation coefficient (ICC), retaining features with an ICC>0.75. In cases where tumor boundaries were unclear, decisions were made by a more experienced radiologist (>15 years).

3 Methods

This retrospective study was conducted in accordance with the Declaration of Helsinki and approved by the Ethical Committee of the First Hospital of Jiaxing (NO. 2022-LY-474), which waived the requirement for informed consent from individual patients.

3.1 Intra-peritumoral segmentation

The peritumoral region was expanded 0–4 mm around the tumor using the “SimpleITK” package in Python 3.6. Radiomic features were extracted from both intratumoral and peritumoral areas after pre-processing. The extent of the peritumoral area was determined based on previous studies, and most studies were within 10 mm of peritumoral extension.

Before DTL feature extraction, images underwent pre-processing. The “largest tumor cross-section” was automatically selected on axial slices, with two levels above (+1,+2) and below (–1, –2) taken for a total of five cross-sectional images. These were resized to 224×224 pixels to fit the CNN models’ input requirements. A Resnet50-based CNN model was constructed for the intra-peritumoral ROI images.

3.2 Handcrafted radiomic feature extraction and deep learning feature integration

Firstly, radiomic features were extracted from manually delineated intratumoral regions and automatically segmented peritumoral regions. These features are categorized into three groups: (I) geometry, (II) intensity, and (III) texture. Texture features, in particular, were derived using various methods such as the gray-level co-occurrence matrix, gray-level run length matrix, gray-level size zone matrix, and neighborhood gray-tone difference matrix. Next, deep transfer learning was employed to extract deep tumor-related (DTL) features from CT images of the intra-peritumoral regions. The pre-trained ResNet50 model on the ImageNet dataset was utilized as the base model [26]. This model contains an input layer with a 224×224 image size and five convolutional layers that progressively enhance the feature map, culminating in an average pool of 2048 features. ResNet50 incorporates residual learning to prevent issues like gradient vanishing and accuracy degradation in deeper networks, thus improving efficiency, accuracy, and speed.

For HR features, features with ICC>0.75 were filtered first, and then the ICC-filtered intra-peritumoral HR features were fused for features. For DTL features, we fused features from DTL features extracted at five intra-peritumoral levels. Regularization of HR features and DTL feature data was then performed. Normalizing the data change to a 0 mean 1 variance, the regularization formula is $C = \frac{C - \bar{X}}{Std}$ . After normalizing all data, the features were filtered through three steps. First, we filtered all HR features using a t-test and retained only those with P < 0.05. Spearmans’ rank correlation coefficient was then used to calculate the correlation between features, and for features with high repeatability, only features with a correlation coefficient greater than 0.9 between any two features were retained. Finally, the remaining features were filtered using the least absolute shrinkage and selection operator (LASSO) regression model. Depending on the adjustment weights λ, LASSO reduces all regression coefficients to zero and sets the coefficients of many irrelevant features to zero.

3.3 Radiomics and DTL signature construction

After feature fusion and filtering, we constructed machine-learning classifier models using HR and DTL features. Provided by Python Scikit-learn (https://scikit-learn.org/stable/user_guide), including the Support Vector Machine (SVM), k-Nearest Neighbors (KNN), Light Gradient Boosting Machine (Light GBM), Multilayer Perceptron (MLP), and Logistic Regression (LR) models. To prevent overfitting, a five-fold cross-validation was performed to select the best parameters for the classifier models in the training cohort and compare the performance of different classifiers to finally select the best machine learning model. The discriminative power of the model was assessed using area under curve (AUC) and receiver operating characteristic (ROC) curve. Accuracy, precision, recall, specificity, and F1-score were also used as quantitative indicators.

3.4 Statistical analysis

We used independent sample t-tests, Mann–Whitney U tests, or chi-square tests to compare the clinical characteristics of the patients. ROC curve was used to evaluate the diagnostic effectiveness of the intra-peritumor HR features model and intra-peritumor DTL features model. In addition, we used precision (the ratio of the true value to the true value), recall (the ratio of true values predicted by the model to true values), F1-score (the summed average of accuracy and recall), and accuracy (proportion of true values out of the total number of true categories predicted) to quantitate indicators. The AUC of the best machine-learning models was compared using the Delong test. Calibration curves were used to compare the agreement between the predictions of the best machine-learning models and actual observations. Finally, the clinical utility of the best models was compared using decision curve analysis (DCA). P < 0.05 was considered statistically significant.

4 Results

4.1 Patients’ clinical characteristics

In total, 199 patients (59 patients with lymph node metastases and 140 patients without lymph node metastases) with pathologically confirmed NSCLC diagnosed at our hospital between November 2018 and July 2022 were enrolled. Approximately 138 adenocarcinomas, 53 squamous carcinomas, and 8 large-cell lung carcinomas were included. All patients underwent puncture biopsy for lung cancer and a CT-enhanced scan of the chest within 2 weeks preoperatively, with histological results obtained within 2 weeks postoperatively as the gold standard. The inclusion criteria were as follows: 1) surgically resected and pathologically confirmed NSCLC with definitive lymph node biopsy results and 2) complete CT imaging data recorded within 2 weeks preoperatively. The exclusion criteria were as follows: 1) history of preoperative treatment (e.g., radiotherapy), 2) a combination of other extra-pulmonary malignancies, 3) lack of CT data or histological findings, and 4) poor image quality. All patients were divided into training (n = 159, November 2018 to September 2019) and validation (n = 40, September 2019 to July 2022) cohorts. Clinicopathological information, age, sex, smoking history, maximum tumor diameter, tumor clinical stage, and pathological type were collected for each patient. A baseline table of the clinical characteristics of the lymph node metastatic status in patients with NSCLC is presented in Table 1.

Table 1
Baseline characteristics of patients in cohorts

Characteristic Training cohort Validation cohort

LN Metastasis (–) LN Metastasis (+) P LN Metastasis (-) LN Metastasis (+) P

Age 66.813±10.408 65.021±9.561 0.312 69.393±11.279 68.583±8.754 0.826

Tumor maximum 3.844±1.919 2.679±1.050 <0.001 3.682±1.527 3.225±1.495 0.388

diameter

Sex 0.549 0.075

Male 77(0.6875) 30(0.6383) 10(0.3571) 8(0.6667)

Female 35(0.3125) 17(0.3617) 18(0.6429) 4(0.3333)

Smoking 0.922 0.730

Yes 51(0.4554) 21(0.4468) 11(0.3929) 4(0.3333)

No 61(0.5446) 26(0.5532) 17(0.6071) 8(0.6667)

Stage <0.001 < 0.001

I 36(0.3214) 16(0.3404) 4(0.1429) 3(0.2500)

II 4(0.0357) 17(0.3617) 1(0.0357) 5(0.4167)

III 12(0.1071) 13(0.2766) 0(0) 3(0.2500)

IV 60(0.5357) 1(0.0213) 23(0.8214) 1(0.0833)

Pathological 0.067 0.719

Adenocarcinoma 71(0.6339) 37(0.7872) 21(0.7500) 9(0.7500)

Squamous Carcinoma 36(0.3214) 9(0.1915) 5(0.1786) 3(0.2500)

Large Cell Carcinoma 5(0.0446) 1(0.0213) 2(0.0714) 0(0)

Characteristic	Training cohort	Validation cohort
Age	66.813±10.408	65.021±9.561	0.312	69.393±11.279	68.583±8.754	0.826
Tumor maximum	3.844±1.919	2.679±1.050	<0.001	3.682±1.527	3.225±1.495	0.388
diameter
Sex			0.549			0.075
Male	77(0.6875)	30(0.6383)		10(0.3571)	8(0.6667)
Female	35(0.3125)	17(0.3617)		18(0.6429)	4(0.3333)
Smoking			0.922			0.730
Yes	51(0.4554)	21(0.4468)		11(0.3929)	4(0.3333)
No	61(0.5446)	26(0.5532)	17(0.6071)		8(0.6667)
Stage			<0.001			< 0.001
I	36(0.3214)	16(0.3404)		4(0.1429)	3(0.2500)
II	4(0.0357)	17(0.3617)		1(0.0357)	5(0.4167)
III	12(0.1071)	13(0.2766)		0(0)	3(0.2500)
IV	60(0.5357)	1(0.0213)		23(0.8214)	1(0.0833)
Pathological		0.067				0.719
Adenocarcinoma	71(0.6339)	37(0.7872)		21(0.7500)	9(0.7500)
Squamous Carcinoma	36(0.3214)	9(0.1915)		5(0.1786)	3(0.2500)
Large Cell Carcinoma	5(0.0446)	1(0.0213)		2(0.0714)	0(0)

LN, lymph node.

Statistically significant differences in maximum tumor diameter and tumor clinical stage were observed between the lymph node metastasis-positive and lymph node metastasis-negative groups in the training cohort. Statistically significant differences were observed in the clinical stage of the tumor between the groups with positive and negative lymph node metastases in the validation cohort. Conversely, the differences between groups for the remaining clinical features were not statistically significant.

4.2 Feature selection

In total, 1906 HR features were extracted for intratumoral and peritumoral (0–4 mm) regions, respectively, including 397 first-order features, 14 shape features, and 1495 texture features, the details of which can be found in Supplementary Fig. The original CT images were passed through wavelet, lbp-3D, log-sigma, square root, square, logarithm, exponential, and gradient filters. All features were extracted using PyRadiomics (http://pyradiomics.readthedocs.io) internal feature analysis program. A total of 3812 features were obtained by fusing the intratumoral and peritumoral 0–4 mm HR features. For the DTL features, 2048 DTL features were extracted for each of the five images, and all DTL features were fused to obtain 10240 features. After feature screening, the final 16 best HR features and 28 best DTL features remained.

Fig. 1

Flowcharts of radiomics and DTL features modeling process.

4.3 Radiomics and DTL signature construction

After obtaining the best HR features and DTL features, we constructed SVM, KNN, Light GBM, MLP, and LR models using five-fold cross-validation and compared the performances of the different models (Table 2). Of the five classifier models in the training and validation cohorts, the LR classifier model performed best. The AUC box plots of the best models for different classifier models after selection of the best parameters by five-fold cross-validation are shown in Fig. 2. Histograms and line charts of the accuracies of the different models are shown in Supplementary Fig. Plots of the LR classifier model feature weights are shown in Fig. 3.

Fig. 2

Comparison of AUC box plots of different best classifier models in the training cohort after selection of optimal parameters by five-fold cross-validation.

Fig. 3

Feature weighting for HR (A) and DTL (B) features.

Table 2

Performance of Radiomics and DTL Signatures in Training and Validation Cohorts.

Models	Methods	Task	AUC	95% CI	Accuracy	Sensitivity	Specificity	Precision	Recall	F1-score
SVM	Radiomics	Training	0.895	0.824–0.965	0.874	0.894	0.866	0.935	0.894	0.914
		Validation	0.753	0.601–0.905	0.750	0.917	0.630	0.667	0.917	0.772
	DTL	Training	0.987	0.975–1.000	0.937	1.000	0.946	0.951	1.000	0.975
		Validation	0.854	0.739–0.969	0.750	0.917	0.750	0.600	0.917	0.725
KNN	Radiomics	Training	0.869	0.814–0.924	0.830	0.809	0.777	0.833	0.809	0.821
		Validation	0.613	0.418–0.808	0.700	0.500	0.704	0.500	0.500	0.500
	DTL	Training	0.910	0.868–0.952	0.849	0.787	0.875	0.725	0.787	0.755
		Validation	0.847	0.700–0.993	0.800	0.833	0.857	0.750	0.833	0.789
Light GBM	Radiomics	Training	0.934	0.896–0.971	0.818	0.872	0.911	0.875	0.872	0.874
		Validation	0.661	0.482–0.839	0.700	1.000	0.370	0.500	1.000	0.667
	DTL	Training	0.980	0.965–0.996	0.862	1.000	0.902	0.963	1.000	0.981
		Validation	0.804	0.658–0.950	0.725	0.917	0.571	1.000	0.917	0.957
MLP	Radiomics	Training	0.853	0.793–0.914	0.792	0.745	0.786	0.850	0.745	0.794
		Validation	0.717	0.540–0.894	0.725	0.583	0.815	0.667	0.583	0.622
	DTL	Training	0.948	0.917–0.978	0.849	0.957	0.795	0.848	0.957	0.899
		Validation	0.884	0.777–0.991	0.750	0.833	0.926	0.667	0.833	0.741
LR	Radiomics	Training	0.841	0.776–0.907	0.811	0.660	0.875	0.743	0.660	0.699
		Validation	0.812	0.677–0.948	0.725	0.917	0.607	0.600	0.917	0.725
	DTL	Training	0.955	0.926–0.983	0.893	0.872	0.911	0.857	0.872	0.865
		Validation	0.893	0.795–0.991	0.775	1.000	0.643	0.600	1.000	0.750

AUC, area under the curve; CI, confidence interval; DTL, deep transfer learning.

4.4 Radiomics and DTL signature validation

The LR model for HR features and DTL features defined the HR and DTL signatures, respectively. The ROC curves for the HR and DTL signature are shown in Fig. 4, with AUCs of 0.841 (95% CI: 0.776–0.907) and 0.955 (95% CI: 0.926–0.983) for the training cohort and 0.812 (95% CI: 0.677–0.948) and 0.893 (95% CI: 0.795–0.991) for the validation cohort, respectively. The DTL signature outperformed the HR signature in terms of accuracy, sensitivity, specificity, precision, recall, and F1-score (Table 2), with the DTL signature showing better predictive performance.

Fig. 4

HR and DTL signature ROC comparison. (A) ROC curve of HR vs. DTL signature for the training cohort; (B) ROC curve of HR vs. DTL signature for the validation cohort.

The results showed a higher index of AUC in predicting distant metastases with the radiomics features of peritumoral tissues compared with regions of lung cancer lesions. The inclusion of peritumoral radiomics features in these studies can significantly improve the predictive power of the model. In this study, radiomics features were extracted by adding a 4-mm region around the tumor and fusing features with intratumoral radiomics features. After multiple machine learning classifier models were compared, the final LR classifier model showed the best predictive performance. This study not only incorporates peritumoral regional features in combination with intratumoral features but also different levels to extract the maximum amount of information. It was found that the 28 best-filtered DTL features contained features of different levels, further enhancing the predictive power of the model. Better results were achieved in the LR classifier model, and the results were better than those of the radiomics feature model.

The calibration curves for the radiomics and DTL signature showed good consistency between the actual results and the model predictions in the training and validation cohorts (Fig. 5). The Hosmer–Lemeshow tests for HR and DTL signature for the training cohort were P = 0.73 and 0.74, respectively. The Hosmer–Lemeshow tests for HR and DTL signature for the validation cohort were P = 0.44 and 0.35, respectively, indicating good consistency between the two models in the training and validation cohorts. Fig. 6 depicts the DCA curves for the HR and DTL signature in the training and validation cohorts. The results showed that the HR signature and DTL signature had better net gains within most threshold probabilities, and the range of the DTL signature was higher than that of the HR signature, indicating that the DTL signature has better clinical utility and higher utility than the HR signature in predicting lymph node metastasis status in NSCLC. The Delong test showed a statistically significant difference between the HR and DTL signatures (P < 0.001).

Fig. 5

HR and DTL signature calibration curves. (A) Calibration curve of HR vs. DTL signature for the training cohort; (B) Calibration curve of HR vs. DTL signature for the validation cohort.

Fig. 6

DCA of HR and DTL signature. (A) DCA for HR and DTL signature in the training cohort; (B) DCA for HR and DTL signature in the validation cohort.

5 Discussion

In this study, HR features and DTL features were constructed in combination with intra-peritumoral regions in CT images to predict the metastatic status of NSCLC lymph nodes in LR classifier models, which showed good predictive performance. The DTL signature was superior to the radiomics signature. Therefore, the DTL signature may be used as an adjunct to the radiological assessment of NSCLC. To the best of our knowledge, no studies have examined HR features compared with DTL features models for predicting lymph node metastasis status in NSCLC.

Radiomics is now widely used for the identification of benign and malignant lung nodules, pathological typing and staging of lung cancer, lung cancer gene expression, and sensitivity to immunotherapy in lung cancer. The correlation between radiomics features and biological behavior has been analyzed [27 –30]. However, most studies have performed radiomics analysis of the primary lesion, ignoring peritumoral tissue. Furthermore, the microenvironment surrounding the lesion is difficult to represent using conventional imaging techniques. The presentation of peritumoral radiomics is a good way to uncover and reflect this micro-environmental heterogeneity, as radiomics can provide a macroscopic view of the micro-environmental changes surrounding a tumor [31]. Wang et al. [32] predicted lymph node metastasis in clinical stage Tl lung adenocarcinoma by extracting radiomics features and tumor volume within 15 mm of the lung tissue surrounding the lung adenocarcinoma. The AUC increased to 0.869 with the addition of peritumoral radiomics compared to that with intratumoral radiomics alone (AUC = 0.825). Dou et al. [33] used peritumoral lung cancer tissue as the ROI to construct a radiomics model to predict the rate of distant metastasis in lung adenocarcinoma. Apart from conventional radiomics approaches, the ongoing advancements in computed tomography (CT) technology have enabled spectral CT imaging to furnish an array of contrastive data regarding lesions and tissues, which holds the potential to significantly augment our capability to anticipate lymph node metastasis in non-small cell lung cancer with heightened precision [34 –36].

Deep learning has received increasing attention in recent years and can help improve the performance of models [37, 38]. Relatively few studies have been conducted on deep learning for predicting lymph node metastasis in NSCLC. Tau et al. [39] evaluated the potential of deep learning using a CNN to predict newly diagnosed NSCLC lymph node metastases by analyzing the characteristics of Fluorine-18 Fluorodeoxyglucose (¹⁸F-FDG) PET primary lesions. The results showed an AUC of 0.80 for predicting lymph node metastasis using CNN analysis of PET images from patients with untreated NSCLC. This indicates that CNN has some value in predicting lymph node metastasis in NSCLC; however, the high cost of PET/CT examination makes it unsuitable for widespread dissemination. Previous research on deep learning has rarely included perineural imaging. Sun et al. [40] studied the performance of CNN in breast ultrasound prediction of axillary lymph node metastasis and compared it with radiomics. The results of the CNN model constructed in the combined intra-peritumoral region were significantly improved over those of the intratumoral model alone. The predictive performance of the combined intra-peritumoral CNN model was superior to that of the combined intra-peritumoral radiomic model. To compensate for the loss of information in deep learning feature extraction owing to single images, comprehensive Two-Dimensional (2D) information was extracted. In this study, the axial section with the largest tumor region was automatically selected as the “largest tumor image.” Additional DTL features were extracted from the upper two (+1,+2) and lower two (–1, –2) slices of the largest tumor image for a total of five images. Five levels of deep learning features were fused and added to the different classifier models. Li et al. [41] referred to this approach as a 2.5D CNN model to complement the information from a single-level CNN model. Choi et al. [42] also used this approach to predict IDH mutation status in gliomas using deep learning and radiomics, with an accuracy of up to 93.8% on the test set and an AUC of 0.96. The above results show that the inclusion of multilevel image information can significantly improve the prediction of deep learning models, complementing the lack of information at the individual level, although better results can also be achieved at the individual level.

Resnet50 is a deep learning model that learns increasingly advanced features from the input image through a series of successive linear and nonlinear layers [39]. Compared with traditional HR features, higher-order DTL features can provide further complementary information that can improve the performance of the model [43]. In this study, the DTL signature outperformed the radiomics signature in terms of accuracy, sensitivity, specificity, precision, recall, and F1 score.

This study has some limitations. First, our data were obtained from a relatively small dataset from a single institution and may not yet take full advantage of deep learning. Despite incorporating multiple layers of image information, we inevitably face challenges with data diversity. Second, the ROIs for this study were manually outlined by radiologists and were subject to human error. Prospective, multicenter, large-sample, multimodal imaging model associations are also needed for future research.

6 Conclusions

The DTL signature constructed based on intra-peritumoral regions in CT images can better predict NSCLC lymph node metastasis.

Footnotes

Acknowledgments

Thanks to Dr. Shuxing Wang for his assistance in data analysis.

The supplementary material is available in the electronic version of this article: .

References

Sung

, Ferlay

, Siegel

R.L.

, Laversanne

, Soerjomataram

, Jemal

, et al., Global Cancer Statistics: GLOBOCAN Estimates of Incidence and Mortality Worldwide for 36 Cancers in 185 Countries, CA Cancer J Clin 71 (2021), 209–249.

Miller

K.D.

, Nogueira

, Devasia

, Mariotto

A.B.

, Yabroff

K.R.

, Jemal

, et al., Cancer treatment and survivorship statistics, CA Cancer J Clin 72 (2022), 409–436.

Herbst

R.S.

, Morgensztern

, Boshoff

The biology and management of non-small cell lung cancer, Nature 553 (2018), 446–454.

Herbst

R.S.

, Morgensztern

, Boshoff

The biology and management of non-small cell lung cancer, Nature 553 (2018), 446–454.

Silvestri

G.A.

, Gonzalez

A.V.

, Jantz

M.A.

, Margolis

M.L.

, Gould

M.K.

, Tanoue

L.T.

, et al., Methods for staging non-small cell lung cancer: Diagnosis and management of lung cancer, 3rd ed: American College of Chest Physicians evidence-based clinical practice guidelines, Chest 143 (2013), e211S–e250S.

Rusch

V.W.

, Crowley

, Giroux

D.J.

, Goldstraw

, Im

J.-G.

, Tsuboi

, et al., The IASLC Lung Cancer Staging Project: proposals for the revision of the N descriptors in the forthcoming seventh edition of the TNM classification for lung cancer, J Thorac Oncol 2 (2007), 603–612.

Silvestri

G.A.

, Gould

M.K.

, Margolis

M.L.

, Tanoue

L.T.

, McCrory

, Toloza

, et al., Noninvasive staging of non-small cell lung cancer: ACCP evidenced-based clinical practice guidelines (2nd edition), Chest 132 (2007), 178S–201S.

Lin

W.-Y.

, Hsu

W.-H.

, Lin

K.-H.

, Wang

S.-J.

Role of preoperative PET-CT in assessing mediastinal and hilar lymph node status in early stage lung cancer, J Chin Med Assoc 75 (2012), 203–208.

Kanzaki

, Higashiyama

, Fujiwara

, Tokunaga

, Maeda

, Okami

, et al., Occult mediastinal lymph node metastasis in NSCLC patients diagnosed as clinical N0-1 by preoperative integrated FDG-PET/CT and CT: Risk factors, pattern, and histopathological study, Lung Cancer 71 (2011), 333–337.

10.

, Li

, Zhang

, Shi

, Wu

, Zhang

, et al., Diagnostic value of fluorine 18 fluorodeoxyglucose positron emission tomography/computed tomography for the detection of metastases in non-small-cell lung cancer patients, Int J Cancer 132 (2013), E37–47.

11.

Kadota

, Yeh

Y.-C.

, Villena-Vargas

, Cherkassky

, Drill

E.N.

, Sima

C.S.

, et al., Tumor Budding Correlates With the Protumor Immune Microenvironment and Is an Independent Prognostic Factor for Recurrence of Stage I Lung Adenocarcinoma, Chest 148 (2015), 711–721.

12.

Kadota

, Nitadori

J.-I.

, Sima

C.S.

, Ujiie

, Rizk

N.P.

, Jones

D.R.

, et al., Tumor Spread through Air Spaces is an Important Pattern of Invasion and Impacts the Frequency and Location of Recurrences after Limited Resection for Small Stage I Lung Adenocarcinomas, J Thorac Oncol 10 (2015), 806–814.

13.

Vaghjiani

R.G.

, Takahashi

, Eguchi

, Lu

, Kameda

, Tano

, et al., Tumor Spread Through Air Spaces Is a Predictor of Occult Lymph Node Metastasis in Clinical Stage IA Lung Adenocarcinoma, J Thorac Oncol 15 (2020), 792–802.

14.

Lambin

, Rios-Velazquez

, Leijenaar

, Carvalho

, van Stiphout

R.G.P.M.

, Granton

, et al., Radiomics: extracting more information from medical images using advanced feature analysis, Eur J Cancer 48 (2012), 441–446.

15.

Gillies

R.J.

, Kinahan

P.E.

, Hricak

Radiomics: Images Are More than Pictures, They Are Data, Radiology 278 (2016), 563–577.

16.

Liao

, Huang

, Wu

, Zhang

, Xie

, Yao

, et al., Preoperative CT-based peritumoral and tumoral radiomic features prediction for tumor spread through air spaces in clinical stage I lung adenocarcinoma, Lung Cancer 163 (2022), 87–95.

17.

Tunali

, Hall

L.O.

, Napel

, Cherezov

, Guvenis

, Gillies

R.J.

, et al., Stability and reproducibility of computed tomography radiomic features extracted from peritumoral regions of lung cancer lesions, Med Phys 46 (2019), 5075–5085.

18.

Han

, Qin

, Bay

, Chen

, Yu

K.-H.

, Miskin

, et al., Deep Transfer Learning and Radiomics Feature Prediction of Survival of Patients with High-Grade Gliomas, AJNR Am J Neuroradiol 41 (2020), 40–48.

19.

, Li

, Chen

, Huang

, He

, Zhao

, et al., Deep Learning Features Improve the Performance of a Radiomics Signature for Predicting KRAS Status in Patients with Colorectal Cancer, Acad Radiol 27 (2020), e254–e262.

20.

Lao

, Chen

, Li

Z.-C.

, Li

, Zhang

, Liu

, et al. A Deep Learning-Based Radiomics Model for Prediction of Survival in Glioblastoma Multiforme, Sci Rep 7 (2017), 10353.

21.

Litjens

, Kooi

, Bejnordi

B.E.

, Setio

A.A.A.

, Ciompi

, Ghafoorian

, et al. A survey on deep learning in medical image analysis, Med Image Anal 42 (2017), 60–88.

22.

Cicero

, Bilbily

, Colak

, Dowdell

, Gray

, Perampaladas

, et al., Training and Validating a Deep Convolutional Neural Network for Computer-Aided Detection and Classification of Abnormalities on Frontal Chest Radiographs, Invest Radiol 52 (2017), 281–287.

23.

Xue

L.-Y.

, Jiang

Z.-Y.

, Fu

T.-T.

, Wang

Q.-M.

, Zhu

Y.-L.

, Dai

, et al., Transfer learning radiomics based on multimodal ultrasound imaging for staging liver fibrosis, Eur Radiol 30 (2020), 2973–2983.

24.

, Xie

, Yang

, Ho

J.W.K.

, Wen

, Han

, et al., Computed tomography-based deep-learning prediction of neoadjuvant chemoradiotherapy treatment response in esophageal squamous cell carcinoma, Radiother Oncol 154 (2021), 6–13.

25.

Christodoulidis

, Anthimopoulos

, Ebner

, Christe

, Mougiakakou

Multisource Transfer Learning With Convolutional Neural Networks for Lung Pattern Analysis, IEEE J Biomed Health Inform 21 (2017), 76–84.

26.

, Jia

, Qiao

, Duan

Self-Taught Learning Based on Sparse Autoencoder for E-Nose in Wound Infection Detection, Sensors (Basel) 17 (2017), E2279.

27.

Zhu

, Dong

, Chen

, Fang

, Zhang

, Song

, et al., Radiomic signature as a diagnostic factor for histologic subtype classification of non-small cell lung cancer, Eur Radiol 28 (2018), 2772–2778.

28.

Yoon

H.J.

, Sohn

, Cho

J.H.

, Lee

H.Y.

, Kim

J.-H.

, Choi

Y.-L.

, et al., Decoding Tumor Phenotypes for ALK, ROS1, and RET Fusions in Lung Adenocarcinoma Using a Radiomics Approach, Medicine (Baltimore) 94 (2015), e1753.

29.

M K

, A G

, P

, Pd

, R

, et al. Changes in CT Radiomic Features Associated with Lymphocyte Distribution Predict Overall Survival and Response to Immunotherapy in Non-Small Cell Lung Cancer, Cancer Immunology Research (2020), 8.

30.

Vaidya

, Bera

, Patil

P.D.

, Gupta

, Jain

, Alilou

, et al., Novel, non-invasive imaging approach to identify patients with advanced non-small cell lung cancer at risk of hyperprogressive disease with immune checkpoint blockade, J Immunother Cancer 8 (2020), e001343.

31.

Dou

T.H.

, Coroller

T.P.

, van Griethuysen

J.J.M.

Mak

R.H.

, Aerts

H.J.W.L.

,Peritumoral radiomics features predict distant metastasis in locally advanced NSCLC, PLoS One 13 (2018), e0206108.

32.

Wang

, Zhao

, Li

, Xia

, Peng

, Zhang

, et al., Can peritumoral radiomics increase the efficiency of the prediction for lymph node metastasis in clinical stage T1 lung adenocarcinoma on CT? Eur Radiol 29 (2019), 6049–6058.

33.

Dou

T.H.

, Coroller

T.P.

, van Griethuysen

J.J.M.

Mak

R.H.

, Aerts

H.J.W.L.

, Peritumoral radiomics features predict distant metastasis in locally advanced NSCLC, PLoS One 13 (2018), e0206108.

34.

Wang

, Wu

, Feng

, Liu

, Yu

Low-dose spectral CT reconstruction based on image-gradient L0-norm and adaptive spectral PICCS, Phys Med Biol 65 (2020), 245005.

35.

Zhang

, Yu

, Xi

, Wang

, Liu

Spectral CT Image-Domain Material Decomposition via Sparsity Residual Prior and Dictionary Learning, IEEE Trans. Instrum. Meas 72 (2023), 1–13.

36.

Wang

, Yu

, Xi

, Gong

, Wu

, Liu

Spectral-Image Decomposition With Energy-Fusion Sensing for Spectral CT Reconstruction, IEEE Trans. Instrum. Meas. 70 (2021), 1–11.

37.

Zhang

, Lobo-Mueller

E.M.

, Karanicolas

, Gallinger

, Haider

M.A.

, Khalvati

Improving prognostic performance in resectable pancreatic ductal adenocarcinoma using radiomics and deep learning features fusion in CT images, Sci Rep 11 (2021), 1378.

38.

, Deng

, Liu

, Zhou

, Jia

, Xiao

, et al., Lymph node metastasis prediction of papillary thyroid carcinoma based on transfer learning radiomics, Nat Commun 11 (2020), 4807.

39.

Soffer

, Ben-Cohen

, Shimon

, Amitai

M.M.

, Greenspan

, Klang

Convolutional Neural Networks for Radiologic Images: A Radiologist’s Guide, Radiology 290 (2019), 590–606.

40.

Sun

, Lin

, Zhao

, Li

, Yan

, Liang

, et al., Deep Learning vs. Radiomics for Predicting Axillary Lymph Node Metastasis of Breast Cancer Using Ultrasound Images: Don’t Forget the Peritumoral Region, Front Oncol 10 (2020), 53.

41.

, Wei

, Liu

, Fan

, Wang

, Li

, et al., Molecular subtyping of diffuse gliomas using magnetic resonance imaging: comparison and correlation between radiomics and deep learning, Eur Radiol 32 (2022), 747–758.

42.

Choi

Y.S.

, Bae

, Chang

J.H.

, Kang

S.-G.

, Kim

S.H.

, Kim

, et al., Fully automated hybrid approach to predict themutation status of gliomas via deep learning and radiomics, Neuro-Oncology 23 (2021), 304–313.

43.

Zhu

, Man

, Gong

, Dong

, Yu

, Wang

, et al., A deep learning radiomics model for preoperative grading in meningioma, Eur J Radiol 116 (2019), 128–134.