A predictive model for early therapeutic efficacy of colorectal liver metastases using multimodal MRI data

Abstract

BACKGROUND:

Liver metastases is a pivotal factor of death in patients with colorectal cancer. The longitudinal data of colorectal liver metastases (CRLM) during treatment can monitor and reflect treatment efficacy and outcomes.

OBJECTIVE:

The objective of this study is to establish a radiomic model based on longitudinal magnetic resonance imaging (MRI) to predict chemotherapy response in patients with CRLM.

METHODS:

This study retrospectively enrolled longitudinal MRI data of five modalities on 100 patients. According to Response Evaluation Criteria in Solid Tumors (RECIST 1.1), 42 and 58 patients were identified as responders and non-responders, respectively. First, radiomic features were computed from different modalities of image data acquired pre-treatment and early-treatment, as well as their differences (Δ). Next, the features were screened by a two-sample t-test, max-relevance and min-redundancy (mRMR), and least absolute shrinkage and selection operator (LASSO). Then, several ensemble radiomic models that integrate support vector machine (SVM), k-nearest neighbor (KNN), gradient boost decision tree (GBDT) and multi-layer perceptron (MLP) were established based on voting method to predict chemotherapy response. Data samples were divided into training and verification queues using a ratio of 8:2. Finally, we used the area under ROC curve (AUC) to evaluate model performance.

RESULTS:

Using the ensemble model developed using featue differences (Δ) computed from the longitudinal apparent diffusion coefficient (ADC) images, AUC is 0.9007±0.0436 for the training cohort. Applying to the testing cohort, AUC is 0.8958 and overall accuracy is 0.9.

CONCLUSIONS:

Study results demonstrate advantages and high performance of the ensemble radiomic model based on the radiomics feature difference of the longitudinal ADC images in predicting chemotherapy response, which has potential to assist treatment decision-making and improve clinical outcome.

Keywords

Colorectal liver metastases chemotherapy radiomics Apparent diffusion coefficient (ADC) value longitudinal data

Abbreviations

ADC – apparent diffusion coefficient

AUC – area under the ROC

AdaBoost – adaptive boosting

CNN – convolutional neural networks

CRLM – colorectal liver metastases

DWI – diffusion-weighted imaging

D – pure molecular diffusion

D* – perfusion-related diffusion

f – perfusion fraction

GLCM – gray level co-occurrence matrix

GLRLM – gray level run length matrix

GLSZM – gray level size zone matrix

GLDM – gray level dependence matrix

GBDT – gradient boost decision tree

IVIM – intravoxel incoherent motion

KNN – k-nearest neighbor

LR – logistic regression

LASSO – least absolute shrinkage and selection operator

LGBM – light gradient boosting model

MRI – magnetic resonance imaging

mRMR – max-relevance and min-redundancy

MLP – multi-layer perceptron

NB – Naïve Bayes

NGTDM – neighbouring gray tone difference matrix

RECIST – response evaluation criteria in solid tumors

ROC – receiver operating characteristic curve

ROI – region of interest

RF – random forest

SVM – support vector machine

SMOTE – synthetic minority oversampling technique

T1_FLAIR – T1_fluidattenuated inversion recovery

VIBE – volume interpolated body examination

XGBoost – extreme gradient boosting

1 Introduction

The incidence and mortality of colorectal cancer rank third and second, respectively, among global cancers [1]. Liver metastases appear in approximately 50% of colorectal cancer patients and are the leading factor of death in these patients. Currently, the primary clinical treatment for liver metastases of colorectal cancer is still surgery, but many patients in CRLM do not have the conditions for surgical treatment [2]. Studies have demonstrated that patients on conversion treatment may be eligible for surgery [3, 4]. Nonetheless, Response Evaluation Criteria in Solid Tumors RECIST 1.1, which relies solely on measuring tumor size as a standard, cannot accurately assess the microcosmic transformations of tumors during treatment and accurately assign suitable patients to surgery. The emerging radiomics can use computers to mine microstructural and metabolic information in medical images to perform non-invasive analysis of tumor heterogeneity and assist better tumor diagnosis or prediction of treatment efficacy [5, 6]. Compared with RECIST 1.1, radiomic can reflect tumor size and analyze tumor metabolism and microstructure changes. Consequently, applying radiomics to predict the chemotherapy response in CRLM patients is particularly critical, assisting clinicians in selecting appropriate CRLM patients for surgery or conservative treatment. At present, studies [7 –10] have proved that CT, MRI (T2WI, DWI-MRI), and other images have the potential to predict CRLM chemotherapy response.

However, the missed diagnosis rate of CT is higher. In contrast, magnetic resonance imaging MRI has a higher accuracy rate of diagnosis, especially for tumors less than 10 mm [11]. Therefore, MRI data is worth further mining. With the high resolution, multi-parameter and other new imaging sequences are continuously included in the imaging examinations of patients with CRLM, and imaging marker parameters for the diagnosis of CRLM are provided [9 , 13]. The apparent diffusion coefficient ADC parametric images fitted by single exponential DWI imaging contain quantitative information related to pure molecular diffusion and capillary perfusion. The pure molecular diffusion D images fitted by bi-exponential IVIM can more accurately reflect the distribution of water molecules without being affected by capillary perfusion. That indicates the changes in diffusion and perfusion in IVIM imaging of CRLM patients may reflect the differences in cell structure and metabolism between the lesion area and normal tissues. However, the current research on parametric images is limited to ADC images and simply histogram statistics, revealing the necessity of radiomics analysis on MRI parametric images.

It is essential to monitor the differential response of patients longitudinally during the treatment of CRLM patients. Recently, some studies established radiomic models based on longitudinal medical imaging data to predict the chemotherapy response of tumor diseases [10, 14]. Nevertheless, to the best of our knowledge, there are no radiomic models based on multimodal MRI longitudinal data combined with the analysis of IVIM parameter images to predict CRLM chemotherapy response. Simultaneously, there is still a lack of reliable models to predict the chemotherapy response of CRLM.

The aim of the present study is to establish a machine learning-based ensemble radiomic model, and then predict the efficacy of CRLM treatment. The prediction performances of 10 machine learning classifiers on each MRI modal were compared. The best ones were selected to build an ensemble radiomics model to correct the decision error of a single classifier.Furthermore, an in-depth experiment was designed to explore the predictive capacity of IVIM parametric images, further complementing our conclusions. It is hoped that this study will provide a reference for further clinical research on CRLM based on radiomics.

2 Materials and methods

2.1 Data collection

This study enrolled 100 patients with CRLM treated at Fudan University Shanghai Cancer Center from June 2016 to August 2019. All patients were treated with FOLFOX (oxaliplatin, leucovorin plus fluorouracil), FOLFIRI (irinotecan, leucovorin plus fluorouracil), and XELOX (oxaliplatin plus capecitabine). The treatment duration is two to three weeks in two cycles. All patients included underwent MRI examinations before and after two or three weeks’ chemotherapy treatment and were pathologically confirmed to be colorectal cancer, and imaging examinations showed at least one liver metastases larger than 10 mm. In addition, the patient did not receive any other treatment prior to receiving the corresponding treatment for colorectal liver metastases. Our institutional review board approved the study and written informed consent was obtained from all participants.

MRI equipment is a 3T MRI scanner (MAGNETOM Skyra, Siemens Healthcare, Erlangen, Germany). Using this device, All patients underwent multimodal MRI images and fitted the corresponding parametric images, including T2 BLADE fat-suppressed MRI, T1_Fluidattenuated inversion recovery, venous phase of dynamic contrast enhanced magnetic resonance imaging DCE-MRI, and parametric images (D, D*, f) fitted by intravoxel incoherent motion IVIM, ADC fitted by diffusion weighted imaging DWI. The above MRI techniques are abbreviated as T2, T1, DCE-MRI, D, D*, f, ADC, respectively. DCE-MRI performed a 3D VIBE (volume interpolated body examination) sequence scanning, flip angle 12°, with one pre-contrast and twenty-five consecutive post-contrast dynamic series after a bolus injection of 0.1 mmol/L of Magnevist per kilogram of body weightinjected at a rate of 1.5 mL/s. IVIM scan parameter bandwidth was 2298 Hz/Px. Nine b values were used such as 0, 20, 40, 60, 100, 150, 200, 500 and 800 s/mm². The average is six for a b-value of 800 s/mm² and five for all other b-values. Other parameters are shown in Table 1.

Table 1
MRI acquisition parameters

Parameters IVIM T1_FLAIR BLADE(T2 Suppressed) VIBE

TR/ TE 5700/54 ms 120/1.4 ms 3500/84 ms 3.78/1.72 ms

FOV 380×308 mm² 380×380 mm² 380×380 mm² 380×380 mm²

Slice thickness 5 mm 6 mm 6 mm 3 mm

slice gap 20% 20% 20% 20%

scan matrix 256×208 352×290 256×256 256×256

Parameters	IVIM	T1_FLAIR	BLADE(T2 Suppressed)	VIBE
TR/ TE	5700/54 ms	120/1.4 ms	3500/84 ms	3.78/1.72 ms
FOV	380×308 mm²	380×380 mm²	380×380 mm²	380×380 mm²
Slice thickness	5 mm	6 mm	6 mm	3 mm
slice gap	20%	20%	20%	20%
scan matrix	256×208	352×290	256×256	256×256

2.2 Image preprocessing

Each 3D MRI slice was examined simultaneously by two radiologists with at least five years of experience using software (ITK-SNAP [15], version: 3.8.0; http://www.itksnap.org). And they manually segmented all regions of interest ROIs. Figure 1 shown the multiple MRI slices of one case.MRI images of the VIBE sequence were used as a reference for ROI segmentation, and all tumors with a tumor diameter greater than 10 mm were included in the study. The largest two lesions were selected as representative studies in patients with more than 10 liver metastases, and benign cysts were not included in the study. There were no cases of tumor disappearance during treatment in this study. We performed the Response Evaluation Criteria in Solid Tumors (RECIST version 1.1) for patients who received two cycles of chemotherapy. Patients with complete and partial responses were assigned to the treatment-effective group, and patients with stable and progressive disease were assigned to the treatment-ineffective group.

Fig. 1

One segmentation case of MRI slices. Each column in the figure represents three slices of 3D MRI for a patient.

The synthetic minority oversampling technique (SMOTE) [16] can generate new data through linear interpolation, which can expand the amount of data and avoid the situation where the classifier does not fully distinguish a small number of samples, but the classification accuracy is still high. The K-means SMOTE [17] method aims to improve SMOTE that may produce noise and cannot resolve the issue of intra-class imbalance. This technique makes it possible to identify and locate the best area for synthetic artificial samples, which can better solve the inequality of the dataset. K-means SMOTE is divided into three steps: ding172Using K-means clustering splits the samples into N clusters, ding173Selecting clusters with a low proportion of minority samples, ding174Applying SMOTE in each selected collection. In the process of longitudinal multimodal MRI data processing, 80% of the data samples are expanded and equalized by K-means SMOTE oversampling. The processed data is subjected to four-fold cross-validation to train the model. The remaining 20% of the data is used as validation input for the model.

2.3 Extraction and selection of radiomic features

Based on the Python (version: 3.8.5) algorithm package pyradiomics [18] (version:3.0.1) and referring to the Imaging Biomarker Standardization Initiative (IBSI) [19], we extracted radiomic features of 7 categories for each ROI: first-order statistics features, shape-based 3D or 2D features, gray level co-occurrence matrix GLCM features, gray level run length matrix GLRLM features, gray level size zone matrix GLSZM features, neighbouring gray tone difference matrix NGTDM features, gray level dependence matrix GLDM features. GLCM, GLRLM, GLSZM, GLDM, and NGTDM are also known as texture features. 1688 features were calculated on the whole 3D ROI instead of 2D slices, and they consisted of 107 features based on original images and 1581 features based on filtered images. The filtering features include 93 exponential filtering features, 93 gradient filtering features, 93 LocalBinaryPattern2D features, 93 Gaussian filtering features, 186 square, and square root filtering features. In addition, there are 744 three-dimensional wavelet filtering features in 8 directions, 279 LocalBinaryPattern3D features. Considering the changes in tumor characteristics during treatment to predict the performances of CRLM patients’ chemotherapy responses, we calculated 1688 pre-chemotherapy and post-chemotherapy features. We made the difference (ΔADC, ΔT2,ΔT1,ΔDCE-MRI) at the two-time points of baseline and early 2-3 weeks in treatment.

Radiomic generally includes the steps of delineation of the region of interest, feature extraction, feature selection, and model establishment and evaluation, as shown in Fig. 2 Feature selection is one of the critical steps of the radiomic model. To prevent the model from overfitting and improve the robustness and reproducibility, we sequentially carried out a three-step selection of the radiomic features. First, a two-sample t-test [20] was used to select the features in the training cohort with significant differences (P < 0.05) between the two groups of features that are chemotherapy effective and chemotherapy ineffective. Next, the max-relevance and min-redundancy mRMR and the least absolute shrinkage and selection operator LASSO [21] were used to select the optimal features in turn. Both mRMR and LASSO algorithms took into account the maximum dependence of features and class labels, but mRMR based on information theory selected too many features, and LASSO based on linear regression model can zero the unimportant feature coefficients and filter features more finely. In this study, the feature selection method combining the three algorithms considered both the relationship between features and labels and the correlation between features, eliminating redundant features and retaining significant features.

Fig. 2

Flow chart for predicting chemotherapy response of liver metastases. Up to down MRI images are ADC, T2, T1, DCE-MRI. In the original 3D MRI images, we delineated the pre-chemotherapy ROIs (such as Tumor(A), Tumor(B)) and the early-chemotherapy ROIs(such as Tumor(a), Tumor(b)), and then extracted radiomic features from 3D ROIs. After three-step feature selection, we obtained the optimal radiomic features and established radiomic models. We used box plots, receiver operating curve ROC, and AUC to evaluate the models. The models were based on Python (version:3.8.5).Finally, An ensemble radiomics model was established based on the optimal classifiers and the data of ADC and IVIM sequences.

2.4 Predicting CRLM chemotherapy response based on machine learning

Our experiment implemented different machine learning models to predict the chemotherapy response to colorectal liver metastases from a small dataset. We established ten machine learning models based on python [22]. K-Nearest Neighbor KNN, which measures the distance between different features, has a better classification performance on small datasets [23]. Logistic Regression LR is a widely used statistical direct probability model, leading to overfitting for small datasets. At the same time, Naïve Bayes NB is considered a simple and excellent method based on Bayes’ theorem, and it may appear better performance than LR sometimes [24]. Support Vector Machine SVM classifies samples by finding optimized decision boundaries and has good generalization ability, which may be comparable with convolutional neural networks CNN [25]. Multi-layer Perceptron MLP is a neural network with multiple neurons and hidden layers, often used to complete classification tasks [26]. The rest of the classifiers are all Bagging ensemble algorithms or Boosting ensemble algorithms with decision trees [27], including Random Forest RF, Adaptive Boosting AdaBoost [28], Gradient Boost Decision Tree GBDT [29], Extreme Gradient Boosting XGBoost [30], Light Gradient Boosting Model LGBM [31]. These classifiers have different advantages in classifying datasets of different sizes, distributions, and types. When the classification result of a single classifier reaches its limit, the ensemble method that combines several classifiers into a model may be considered, which may have better performance.

After obtaining the optimal features, we established ten models for the five MRI sequences, including AdaBoost (base_estimator = DecisionTreeClassifier (max_depth = 4), learning-rate = 0.8), GBDT (learning-rate = 0.8, max_depth = 4), KNN (K = 6, weights = uniform), LR (Penalty = l1, C = 0.88), MLP (Hidden-layer-sizes = (20,10,10), activation = relu, solver = lbfgs), NB (default), RF (n-estimators = 41, max-depth = 4), SVM (C = 0.8, kernel = sigmoid), Xgboost (Max-depth = 4, learning-rate = 0.8, n-estimators = 60), LGBM (loss = ’binary_crossentropy’, learning_rate = 0.7, max_iter = 50, max_leaf_nodes = 20, max_depth = 4).

In this process, the grid search of four-fold cross-validation and the learning curve were used to search for the best radiomic model. To visually analyze the radiomic model of the imbalanced dataset, we introduced receiver operating curve ROC and area under the curve AUC to evaluate the robustness of the model, and at the same time monitored the specificity, sensitivity, F1 value and accuracy, abbreviated as SP, SE, F1, ACC.

3 Results

3.1 Clinical characteristics

The clinical characteristics of the patients are shown in Table 2. There are 42 patients in the response group and 58 patients in the non-response group. Patient characteristics are expressed as mean±standard deviation or the number of corresponding characteristics. In this study, the chi-square test χ² of SPSSAU (Version 22.0) was used to conduct statistical tests on the clinical characteristics of the two groups of patients. All P values were greater than 0.05 and were not statistically significant.

Table 2
Patient characteristics

Characteristics Responding Nonresponding P-value

Gender, No. (%) 0.934

Male 25(41.67%) 35(58.33%)

Female 17(42.50%) 23(57.50%)

Age(years) 58.1±12.36 56.8±11.96 0.815

Location, No. (%) 0.406

Rectum 11 14

Colon+Rectum 0 1

Colon 31 43

Chemotherapy regimen, No. (%) 0.588

XELOX 23 32

FOLFOX 5 8

FOLFIRI 14 18

Diameter of the largest lesion(mm) 49±2.5^a 63±3.8^a 0.362

Characteristics	Responding	Nonresponding	P-value
Gender, No. (%)	0.934
Male	25(41.67%)	35(58.33%)
Female	17(42.50%)	23(57.50%)
Age(years)	58.1±12.36	56.8±11.96	0.815
Location, No. (%)			0.406
Rectum	11	14
Colon+Rectum	0	1
Colon	31	43
Chemotherapy regimen, No. (%)			0.588
XELOX	23	32
FOLFOX	5	8
FOLFIRI	14	18
Diameter of the largest lesion(mm)	49±2.5^a	63±3.8^a	0.362

Abbreviation: FOLFOX = oxaliplatin, leucovorin plus fluorouracil, FOLFIRI = irinotecan, leucovorin plus fluorouracil, XELOX = oxaliplatin plus capecitabine, a Values are expressed as mean±standard deviation.

3.2 Results of feature selection

In the ensemble model,we used t-test, mRMR, and LASSO to select the features, in turn, retained the optimal features, as shown in Table 3. Figure 3(A) and 3(B) show the histogram and heatmap to visualize features. Last, we obtained 10 optimal features. It can be seen from Fig. 3(A) that the correlation between most feature pairs is relatively low, which indicates that the redundancy of the feature set is low. Figure 3(B) indicts that the weight of texture features is high.

Table 3
Optimal features

Optimal features Weight

original_glcm_Idn 0.098206

exponential_gldm_HighGrayLevelEmphasis 0.049696

exponential_gldm_SmallDependenceEmphasis 0.000731

wavelet-LLH_ngtdm_Contrast 0.025677

exponential_glcm_DifferenceAverage 0.005526

original_gldm_LargeDependenceLowGrayLevelEmphasis –0.036610

lbp-3D-k_glcm_DifferenceVariance 0.019015

wavelet-HLL_glszm_HighGrayLevelZoneEmphasis –0.003778

wavelet-HHH_firstorder_TotalEnergy –0.009290

gradient_glrlm_GrayLevelNonUniformity 0.012928

Optimal features	Weight
original_glcm_Idn	0.098206
exponential_gldm_HighGrayLevelEmphasis	0.049696
exponential_gldm_SmallDependenceEmphasis	0.000731
wavelet-LLH_ngtdm_Contrast	0.025677
exponential_glcm_DifferenceAverage	0.005526
original_gldm_LargeDependenceLowGrayLevelEmphasis	–0.036610
lbp-3D-k_glcm_DifferenceVariance	0.019015
wavelet-HLL_glszm_HighGrayLevelZoneEmphasis	–0.003778
wavelet-HHH_firstorder_TotalEnergy	–0.009290
gradient_glrlm_GrayLevelNonUniformity	0.012928

Fig. 3

Figure 3(A) Heatmap of feature cluster implies the correlation between features. Figure 3 (B) feature weight histogram shows each feature’s weight in the model.

3.3 Predicting results using a single machine model

We used ten single-classifier radiomic models for each MRI modality to predict the efficacy of treatment of CRLM, and the corresponding testing cohort evaluation indicators were exhibited in Table 4. It can be seen from the table that the radiomic models built using the feature differences outperformed the models built using the untreated baseline features. The SVM and GBDT models based on baseline ADC features and the T1-based GBDT model achieved an accuracy of 0.8000, which same as the accuracy of the partial difference-based model. Combining accuracy and AUC, the radiomic model of differences based on ADC parameter images were most potential to predict chemotherapy response of CRLM patients(accuracy: 0.85, testing AUC 0.8800). Secondly, many models’ accuracy based on the DCE-MRI sequence reached 0.8000, and the AUC reached 0.8800. In contrast, the prediction ability of the models based on T2 and T1 sequences was weak. To better evaluate the stability of the models, we used four-fold cross-validation to draw a boxplot on the training cohort with AUC as the evaluation index.The x-axis of boxplot is the classifier, y-axis is the accuracy,as shown in Fig. 4. Generally speaking, the AUC of the ADC was relatively high and stable. Results revealed that ADC acquired best performance, so an ensemble model would be established only based on ADC in section 3.4.

Table 4
Evaluation of classifier predictve effect

Classifiers Pre-feature Δfeature

SE SP F1 ACC AUC SE SP F1 ACC AUC

ADC SVM 0.7778 0.8182 0.7980 0.8000 0.7374 0.8667 0.8000 0.8119 0.8500 0.8800

LR 0.7273 0.6667 0.6970 0.7000 0.6364 0.7692 0.8571 0.7917 0.8000 0.8022

KNN 0.8462 0.5714 0.7151 0.7500 0.6970 0.9091 0.7778 0.8465 0.8500 0.8030

NB 0.8182 0.5556 0.6875 0.7000 0.6667 0.8333 0.7500 0.7917 0.8000 0.7969

RF 0.7500 0.6250 0.6875 0.7000 0.6979 0.8571 0.6667 0.7619 0.8000 0.7976

AdaBoost 0.9286 0.3333 0.6416 0.7500 0.7143 0.8200 0.7800 0.8000 0.8000 0.8182

GBDT 0.7778 0.8182 0.7980 0.8000 0.7879 0.8333 0.8750 0.8465 0.8500 0.7812

MLP 0.7273 0.7778 0.7494 0.7500 0.6869 0.8000 0.9000 0.8496 0.8500 0.8700

Xgboost 0.6000 0.8000 0.6970 0.7000 0.7000 0.7000 0.9000 0.7980 0.8000 0.7300

LGBM 0.6923 0.7143 0.6875 0.7000 0.6484 0.7143 1.0000 0.7917 0.8000 0.8571

T2 SVM 0.5833 0.7500 0.6491 0.6500 0.7188 0.7273 0.7778 0.7494 0.7500 0.6869

LR 0.7778 0.6364 0.7000 0.7000 0.6162 0.7273 0.7778 0.7434 0.7500 0.8182

KNN 0.6429 0.6667 0.6267 0.6500 0.6548 0.7500 0.7500 0.7442 0.7500 0.7500

NB 0.6923 0.5714 0.6267 0.6500 0.6429 0.8182 0.6667 0.7442 0.7500 0.6263

RF 0.7500 0.7500 0.7442 0.7500 0.7812 0.7273 0.8889 0.8000 0.8000 0.8586

AdaBoost 0.6429 0.6667 0.6267 0.6500 0.6429 0.6667 0.8750 0.7494 0.7500 0.7708

GBDT 0.7000 0.7000 0.7000 0.7000 0.7200 0.8333 0.7500 0.7917 0.8000 0.8229

MLP 0.6000 0.7000 0.6491 0.6500 0.6800 0.6000 0.8000 0.6970 0.7000 0.7050

Xgboost 0.6667 0.6364 0.6491 0.6500 0.6970 0.7000 0.7000 0.7000 0.7000 0.8000

LGBM 0.5455 0.7778 0.6491 0.6500 0.7172 0.6154 0.8571 0.6970 0.7000 0.6154

DCE-MRI SVM 0.8182 0.6667 0.7442 0.7500 0.7273 0.8889 0.7273 0.8000 0.8000 0.8676

LR 0.6429 0.8333 0.6875 0.7000 0.7738 0.8333 0.7500 0.7917 0.8000 0.8646

KNN 0.7857 0.5000 0.6429 0.7000 0.5060 1.0000 0.4444 0.7151 0.7500 0.7121

NB 0.7500 0.7500 0.7442 0.7500 0.7812 0.7500 0.8750 0.7980 0.8000 0.7812

RF 0.7500 0.7500 0.7442 0.7500 0.8438 0.6667 1.0000 0.8000 0.8000 0.7292

AdaBoost 0.6667 0.8182 0.7442 0.7500 0.5859 0.9000 0.7000 0.7980 0.8000 0.8800

GBDT 0.6000 0.9000 0.7442 0.7500 0.7700 0.8333 0.7500 0.7917 0.8000 0.8125

MLP 0.9167 0.5000 0.7151 0.7500 0.6250 0.8462 0.7143 0.7802 0.8000 0.8132

Xgboost 0.7273 0.6667 0.6970 0.7000 0.6364 0.7692 0.8571 0.7917 0.8000 0.8352

LGBM 0.5833 0.8750 0.7000 0.7000 0.7917 0.8462 0.7143 0.7802 0.8000 0.8352

T1 SVM 0.7273 0.6667 0.6970 0.7000 0.6061 0.7143 0.8333 0.7333 0.7500 0.7857

LR 0.6000 0.7000 0.6491 0.6500 0.5800 0.6364 0.7778 0.7000 0.7000 0.7273

KNN 0.9000 0.5000 0.6875 0.7000 0.7050 0.6667 0.8750 0.7494 0.7500 0.8698

NB 0.7000 0.7000 0.7000 0.7000 0.7200 0.7500 0.7500 0.7442 0.7500 0.7292

RF 0.7273 0.6667 0.6970 0.7000 0.6768 0.8182 0.7778 0.7980 0.8000 0.7172

AdaBoost 0.8750 0.6667 0.7594 0.7500 0.7083 0.7000 0.8000 0.7494 0.7500 0.7100

GBDT 1.0000 0.5000 0.7619 0.8000 0.6146 0.7692 0.8571 0.7917 0.8000 0.7802

MLP 0.7000 0.7000 0.7000 0.7000 0.6500 0.6667 0.8182 0.7442 0.7500 0.7020

Xgboost 0.7333 0.6000 0.6492 0.7000 0.7067 0.8462 0.5714 0.7151 0.7500 0.7473

LGBM 0.7000 0.6000 0.6491 0.6500 0.5700 0.9000 0.7000 0.7980 0.8000 0.7900

Classifiers	Pre-feature	Δfeature
ADC	SVM	0.7778	0.8182	0.7980	0.8000	0.7374	0.8667	0.8000	0.8119	0.8500	0.8800
	LR	0.7273	0.6667	0.6970	0.7000	0.6364	0.7692	0.8571	0.7917	0.8000	0.8022
	KNN	0.8462	0.5714	0.7151	0.7500	0.6970	0.9091	0.7778	0.8465	0.8500	0.8030
	NB	0.8182	0.5556	0.6875	0.7000	0.6667	0.8333	0.7500	0.7917	0.8000	0.7969
	RF	0.7500	0.6250	0.6875	0.7000	0.6979	0.8571	0.6667	0.7619	0.8000	0.7976
	AdaBoost	0.9286	0.3333	0.6416	0.7500	0.7143	0.8200	0.7800	0.8000	0.8000	0.8182
	GBDT	0.7778	0.8182	0.7980	0.8000	0.7879	0.8333	0.8750	0.8465	0.8500	0.7812
	MLP	0.7273	0.7778	0.7494	0.7500	0.6869	0.8000	0.9000	0.8496	0.8500	0.8700
	Xgboost	0.6000	0.8000	0.6970	0.7000	0.7000	0.7000	0.9000	0.7980	0.8000	0.7300
	LGBM	0.6923	0.7143	0.6875	0.7000	0.6484	0.7143	1.0000	0.7917	0.8000	0.8571
T2	SVM	0.5833	0.7500	0.6491	0.6500	0.7188	0.7273	0.7778	0.7494	0.7500	0.6869
	LR	0.7778	0.6364	0.7000	0.7000	0.6162	0.7273	0.7778	0.7434	0.7500	0.8182
	KNN	0.6429	0.6667	0.6267	0.6500	0.6548	0.7500	0.7500	0.7442	0.7500	0.7500
	NB	0.6923	0.5714	0.6267	0.6500	0.6429	0.8182	0.6667	0.7442	0.7500	0.6263
	RF	0.7500	0.7500	0.7442	0.7500	0.7812	0.7273	0.8889	0.8000	0.8000	0.8586
	AdaBoost	0.6429	0.6667	0.6267	0.6500	0.6429	0.6667	0.8750	0.7494	0.7500	0.7708
	GBDT	0.7000	0.7000	0.7000	0.7000	0.7200	0.8333	0.7500	0.7917	0.8000	0.8229
	MLP	0.6000	0.7000	0.6491	0.6500	0.6800	0.6000	0.8000	0.6970	0.7000	0.7050
	Xgboost	0.6667	0.6364	0.6491	0.6500	0.6970	0.7000	0.7000	0.7000	0.7000	0.8000
	LGBM	0.5455	0.7778	0.6491	0.6500	0.7172	0.6154	0.8571	0.6970	0.7000	0.6154
DCE-MRI	SVM	0.8182	0.6667	0.7442	0.7500	0.7273	0.8889	0.7273	0.8000	0.8000	0.8676
	LR	0.6429	0.8333	0.6875	0.7000	0.7738	0.8333	0.7500	0.7917	0.8000	0.8646
	KNN	0.7857	0.5000	0.6429	0.7000	0.5060	1.0000	0.4444	0.7151	0.7500	0.7121
	NB	0.7500	0.7500	0.7442	0.7500	0.7812	0.7500	0.8750	0.7980	0.8000	0.7812
	RF	0.7500	0.7500	0.7442	0.7500	0.8438	0.6667	1.0000	0.8000	0.8000	0.7292
	AdaBoost	0.6667	0.8182	0.7442	0.7500	0.5859	0.9000	0.7000	0.7980	0.8000	0.8800
	GBDT	0.6000	0.9000	0.7442	0.7500	0.7700	0.8333	0.7500	0.7917	0.8000	0.8125
	MLP	0.9167	0.5000	0.7151	0.7500	0.6250	0.8462	0.7143	0.7802	0.8000	0.8132
	Xgboost	0.7273	0.6667	0.6970	0.7000	0.6364	0.7692	0.8571	0.7917	0.8000	0.8352
	LGBM	0.5833	0.8750	0.7000	0.7000	0.7917	0.8462	0.7143	0.7802	0.8000	0.8352
T1	SVM	0.7273	0.6667	0.6970	0.7000	0.6061	0.7143	0.8333	0.7333	0.7500	0.7857
	LR	0.6000	0.7000	0.6491	0.6500	0.5800	0.6364	0.7778	0.7000	0.7000	0.7273
	KNN	0.9000	0.5000	0.6875	0.7000	0.7050	0.6667	0.8750	0.7494	0.7500	0.8698
	NB	0.7000	0.7000	0.7000	0.7000	0.7200	0.7500	0.7500	0.7442	0.7500	0.7292
	RF	0.7273	0.6667	0.6970	0.7000	0.6768	0.8182	0.7778	0.7980	0.8000	0.7172
	AdaBoost	0.8750	0.6667	0.7594	0.7500	0.7083	0.7000	0.8000	0.7494	0.7500	0.7100
	GBDT	1.0000	0.5000	0.7619	0.8000	0.6146	0.7692	0.8571	0.7917	0.8000	0.7802
	MLP	0.7000	0.7000	0.7000	0.7000	0.6500	0.6667	0.8182	0.7442	0.7500	0.7020
	Xgboost	0.7333	0.6000	0.6492	0.7000	0.7067	0.8462	0.5714	0.7151	0.7500	0.7473
	LGBM	0.7000	0.6000	0.6491	0.6500	0.5700	0.9000	0.7000	0.7980	0.8000	0.7900

Fig. 4

Figure 4 box plot of classifier evaluation shows the AUCs of the four-fold cross-training for each ADC, T2, T1, and DCE-MRI image. The center of the box is the median of the data, representing the sample data’s average level. The upper and lower limits of the box are the upper quartile and the lower quartile of the AUC data containing 50% of the AUC data. Therefore, the boxes’ width reflects the fluctuation of the AUC to a certain extent. Generally speaking, the AUCs of the ADC sequence are relatively high and stable.

In order to analyze the significance of 10 classifiers and 4 MRI models in predicting efficacy of CRLM,correlation analysis of ACC and AUC of 10 classifiers are shown in Fig. 5. Two-factor rank ANOVA were involved to calculate the P-values of 10 classifiers and 4 MRI sequences (ADC, T1, T2 and DCE-MRI), as shown in Table 5. There is almost no significant difference between 10 classifiers based on two kinds of features. However, significant differences are existing between 4 MRI modes (P < 0.05). In particular, 4 MRI types based on Δfeatures showed significant differences in predicting efficacy of CRLM.

Fig. 5

Correlation analysis of ACC and AUC for 10 classifiers. It can be seen that ACC has a higher correlation. Among the 45 pairs, 19 are greater than 70%, and the probability of high positive correlation is 42.22%. The probability of high positive and negative correlation of AUC was 24.44% and 8.89%, respectively.

Table 5

P-values calculation of classifiers and MRI modes

Features\P	indicators	SE	SP	F1	ACC	AUC
pre-features	classifiers	0.4280	0.5128	0.0756	0.0216	0.5231
	MRI	0.0719	0.3443	0.0094	0.0002	0.4075
Δ features	classifiers	0.7948	0.9271	0.4650	0.2741	0.9227
	MRI	0.0162	0.4876	0.0002	0.0000	0.0476

3.4 Predicting result and evaluation of ensemble model

In response to the last part’s evaluation results of the machine learning classifiers, we trained an ensemble radiomic model based on the ADC images consisting of SVM, GBDT, KNN, and MLP four classifiers.The weight matrix of emsemble model is [1,1,2,1 , 1,1,2,1]. The four classifiers were analyzed in any combination of 2, 3, and 4, but only when four classifiers were used at the same time, the model worked best. The factors considered in the model combination are ding172Small difference in classifiers’ performance ding173Small homogeneity of classifiers (such as linear and tree models have little homogeneity). In the end, we selected the four classifiers with the best performance in accuracy and AUC to establish an ensemble model based on the voting method. The classification probability of voting model is the average classification probability of the four single classifiers, as shown in the Fig. 6(B). The following Fig. 6(A) provide an example. SVM, MLP and GBDT in Fig. 6(A) defined the sample as class 1 or treatment-effective, which is consistent with the accurate results. The experimental results show that the ensemble model can improve the prediction accuracy for CRLM patients. The accuracy of the ensemble model was 0.9000, the training AUC was 0.9007±0.0436, the test AUC was 0.8958.

Fig. 6

Integrated algorithm evaluation index. Figure 6(A) and 6(B) show examples of voting algorithms. Figure 6(C) shows that the training set has an AUC of 0.9007±0.0436 in the four-fold cross-training of the ensemble algorithm. Figure 6(D) is the test set AUC of each classifier and ensemble classifier.

3.5 IVIM parameter images

The experiments confirmed that the ADC sequence has the best accuracy and AUC. We further analyzed the pure molecular diffusion D, perfusion-related diffusion D*, and perfusion fraction f images from IVIM to predict the chemotherapy response of CRLM patients, as shown in Fig. 7. The ROIs involved in the three-parameter images all refer to the ADC images. This part of the experiment was carried out in the same steps as the previous experiment and relies on the ensemble classifier. The experiment revealed the AUCs in the training cohort were 0.8874±0.0200 [0.8750,0.9236], 0.8371±0.0623 [0.7727,0.9091], 0.9230±0.0582 [0.8403,1.0000], the verification cohort AUCs were 0.8229, 0.7374, 0.9200, accuracy were 0.8000,0.8500,0.8000, as shown in Table 6 and Fig. 8.

Fig. 7

IVIM parameter images D, D*, f are shown in the Fig. 7. Red arrows marked the tumors.

Table 6

Evaluation of radiomic models of parameters

Machine learning classifiers	SE	SP	F1	ACC	train AUC	test AUC
ensemble classifiers	D	0.7500	0.8750	0.7980	0.8000	0.8874±0.0200	0.8229
	D*	0.9091	0.7778	0.8465	0.8500	0.8371±0.0623	0.7374
	f	0.7333	1.0000	0.7802	0.8000	0.9230±0.0582	0.9200

Fig. 8

AUC evaluations of the three-parameter sequences in the integrated algorithm. The above figure (A), (B), (C) corresponds to the AUC evaluations of the three sequences of D, D*, and f in turn.

4 Discussion

In this study, we used the difference in radiomic characteristics including baseline examination and early treatment (2-3weeks) to build machine learning models to predict the treatment response of CRLM patients. The ensemble radiomic model based on longitudinal ADC data has the best predictive ability. The AUC of the training and validation cohorts are 0.9007±0.0436 and 0.8958, respectively, and the accuracy rate is 0.9000, which shows that the voting-based ensemble radiomic model proposed in this paper can predict the treatment response of CRLM patients and help clinical treatment decision-making.

In the treatment of CRLM patients, predicting the outcome of patients and assisting clinical decision-making are particularly important. Many scholars have done related research. Wei [7] combined deep learning and traditional machine learning and added clinical predictor(CEA level) to establish an analysis model. The experimental results revealed that MDCT images could provide more comprehensive tumor heterogeneous information than the single factor of increased CEA. MDCT images are more effective in predicting chemotherapy response, but his test AUC is only 0.830[95% CI, 0.688–0.973]. Nakanishi [8] verified that the pre-chemotherapy radiomic features of CT can predict the chemotherapy response of CRLM patients to oxaliplatin. The two studies by Wei and Nakanishi and the study of Ahn [32] based on baseline CT texture analysis to predict cytotoxic chemotherapy response are all relied on a single time point.

Nevertheless, for most patients receiving long-term chemotherapy, there is urgent to monitor the tumors’ radiological changes to reveal the treatment response. This article used the longitudinal imaging data of patients including baseline examination and early treatment (2-3weeks) to establish radiomic models and characterize the patient’s differential response through the difference in radiomic features, which improves the models’ accuracy and robustness and reflects the patients’ differential response.

Directing at the problems mentioned above, Boraschi [9] and Zhu [10] proposed a combination of pre-treatment and post-treatment imaging data to analyze the chemotherapy response of CRLM patients. Boraschi et al. [9] used the statistical analysis of pre-chemotherapy and post-chemotherapy ADC sequence from the single exponential model (DWI) to confirm that postADC and ΔADC can be reliable biomarkers for evaluating chemotherapy response. In comparison, they did not analyze the predictive ability of the radiomic features from the ADC parameter images. Although Zhu et al. [10] cascade the T2WI and DWI-ADC features including baseline examination and early treatment (2-3weeks) through the convolutional network CNN, they did not consider the difference. Compared to previous studies, Δfeatures from five MRI sequences were used to build an ensemble model to predict the chemotherapy response in early treatment rather than at the end of treatment. Radiomic features were deeply analyzed instead of a single statistical analysis. Radiomic features can provide specific predictors for doctors compared with deep learning. While analyzing conventional MRI sequence images, MRI parameter images were analyzed innovatively. Final ensemble radiomic model based on ADC can predict the chemotherapy response well, and achieve the best prediction accuracy.

The experiment first established ten machine learning models for each MRI modality based on the feature differences and baseline features, and the accuracy can reach 0.85. To further improve the prediction accuracy, we will perform the best four based on the idea of voting ensemble learning, and the accuracy is increased to 0.9. Through horizontal comparative analysis, the integrated model based on DWI-ADC performed best. We analyzed IVIM’s D, D*, and f parameter images and confirmed that they also have predictive potential with an accuracy rate of 0.85. In the process of data preprocessing, the number of positive and negative samples are 58 and 42. To build a better model, we use Kmeans SMOTE [16 , 34] oversampling to make the ratio of positive and negative samples close to 1:1. Balancing the training data distribution also expands the training data, avoiding the algorithm’s bias towards the majority.

This study has some limitations. Firstly, limited to the amount of data, we only prospectively analyzed a small number of samples from a single center. We will include many samples from different institutions to verify the model in the follow-up research. Secondly, the time intervals of the chemotherapy of the patients included in the study were inconsistent, between 6 to 8 weeks, so that irregular sampling may lead to deviations in experimental results. According to the experimental results in this article, longitudinal ADC data can predict the outcome of CRLM patients. Subsequent research will consider combining deep learning. RNN recurrent neural network jointly analyzes three or more treatment time points to establish a more automated analysis of the radiomic model. Since this study is a prospective experiment, we need to perform detailed histopathological analysis on the resected tumor specimens to verify and calibrate the model.

5 Conclusion

In this study, through the comparative analysis of ten machine learning classifiers, the voting method was used to construct an ensemble radiomic model to predict the chemotherapy response of CRLM patients. This model can effectively predict the preoperative chemotherapy response of tumors. Ensemble radiomic model based on ADC parameter images can provide personalized diagnostic information and assist treatment decisions.

Footnotes

Acknowledgments

This work was supported by Natural Science Foundation of Shanghai(18ZR1426900), Natural Science Foundation of Shanghai(20ZR1412700), National Natural Science Foundation of China (81971687).

Conflict of interest

The authors have no conflicts to disclose.

References

Sung

, Ferlay

, Siegel

R.L.

, et al., Global cancer statistics: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries, CA Cancer J Clin 71(3) (2021), 209–249.

Villard

, Habib

, Nordenvall

, et al., Conversion therapy in patients with colorectal liver metastases, Eur J Surg Oncol 47(8) (2021), 2038–2045.

Martin

, Petrillo

, Smyth

E.C.

, et al., Colorectal liver metastases: Current management and future perspectives, World J Clin Oncol 11(10) (2020), 761–808.

Birrer

D.L.

, Tschuor

, Reiner

, et al., Multimodal treatment strategies for colorectal liver metastases, Swiss Med Wkly 151 (2021), w20390.

Sun

, Jin

, Zhang

, et al., Preoperative prediction for Lauren type of gastric cancer: A radiomics nomogram analysis based on CT images and clinical features, J Xray Sci Technol 29(4) (2021), 675–686.

Gai

, Thai

, Jones

, et al., Applying a radiomics-based CAD scheme to classify between malignant and benign pancreatic tumors using CT images, J Xray Sci Technol 30(2) (2022), 377–388.

Wei

, Cheng

, Gu

, et al., Deep learning-based radiomics predicts response to chemotherapy in colorectal liver metastases, Med Phys 48(1) (2021), 513–522.

Nakanishi

, Oki

, Hasuda

, et al., Radiomics texture analysis for the identification of colorectal liver metastases sensitive to first-line oxaliplatin-based chemotherapy, Ann Surg Oncol 28(6) (2021), 2975–2985.

Boraschi

, Donati

, Cervelli

, et al., Colorectal liver metastases: ADC as an imaging biomarker of tumor behavior and therapeutic response, Eur J Radiol 137 (2021), 109–609.

10.

Zhu

H.B.

, Xu

, Ye

, et al., Deep learning-assisted magnetic resonance imaging prediction of tumor response to chemotherapy in patients with colorectal liver metastases, Int J Cancer 148(7) (2021), 1717–1730.

11.

Mainenti

P.P.

, Romano

, Pizzuti

, et al., Non-invasive diagnostic imaging of colorectal liver metastases, World J Radiol 7(7) (2015), 157–169.

12.

, Zhang

, Zheng

, et al., Preoperative histogram analysis of intravoxel incoherent motion (IVIM) for predicting microvascular invasion in patients with single hepatocellular carcinoma, Eur J Radiol 105 (2018), 65–71.

13.

Kim

J.H.

, Joo

, Kim

T.Y.

, et al., Diffusion-related MRI parameters for assessing early treatment response of liver metastases to cytotoxic therapy in colorectal cancer, AJR Am J Roentgenol 207(3) (2016), W26–W32.

14.

Jin

, Yu

, Ke

, et al. Predicting treatment response from longitudinal images using multi-task deep learning, Nat Commun 12(1) (2021), 1851.

15.

Yushkevich

P.A.

, Piven

, Hazlett

H.C.

, et al., User-guided 3D active contour segmentation of anatomical structures: Significantly improved efficiency and reliability, Neuroimage 31(3) (2006), 1116–1128.

16.

Douzas

, Bacao

and Last

, Improving imbalanced learning through a heuristic oversampling method based on k-means and SMOTE, Inf Sci 465 (2018), 1–20.

17.

Chawla

N.V.

, Bowyer

K.W.

, Hall

L.O.

and Kegelmeyer

W.P.

, Smote: Synthetic minority over-sampling technique, J Arti Intell Res 16(0) (2002), 321–357.

18.

Van Griethuysen

J.J.M.

, Fedorov

, Parmar

, et al., Computational radiomics system to decode the radiographic phenotype, Cancer Res 77(21) (2017), e104–e107.

19.

Zwanenburg

, Vallières

, Abdalah

M.A.

, et al., The image biomarker standardization initiative: Standardized quantitative radiomics for high-throughput image-based phenotyping, Radiology 295(2) (2020), 328–338.

20.

Zhang

, Huang

Z.X.

, Zhao

, et al., Development and validation of magnetic resonance imaging-based radiomics models for preoperative prediction of microsatellite instability in rectal cancer, Ann Transl Med 9(2) (2021), 134.

21.

Meng

, Dong

, Chen

, et al., 2D and 3D CT radiomic features performance comparison in characterization of gastric cancer: A multi-center study, IEEE J Biomed Health Inform 25(3) (2021), 755–763.

22.

Pedregosa

, Varoquaux

, Gramfort

, et al., Scikit-learn: Machine learning in Python, JMLR 12(85) (2011), 2825–2830.

23.

Gupta

S.C.

and Goel

, Performance enhancement of diabetes prediction by finding optimum K for KNN classifier with feature selection method, 2020 Third International Conference on Smart Systems and Inventive Technology (ICSSIT), 2020, pp. 980–986.

24.

Paraskevas

and Ioanna

, Comparison of a logistic regression and Naïve Bayes classifier in landslide susceptibility assessments: The influence of models complexity and training dataset size, Catena 145 (2016), 164–179.

25.

Wang

, Fan

and Wang

, Comparative analysis of image classification algorithms based on traditional machine learning and deep learning, Pattern Recognit Lett 141 (2021), 61–67.

26.

Massaoudi

, Refaat

S.S.

, Chihi

, et al., A novel stacked generalization ensemble-based hybrid LGBM-XGB-MLP model for Short-Term Load Forecasting, Energy 214 (2021), 1–14.

27.

Hancock

, Put

, Coomans

, et al., A performance comparison of modern statistical techniques for molecular descriptor selection and retention prediction in chromatographic QSRR studies, Chemometr Intell Lab Syst 76(2) (2005), 185–196.

28.

Freund

and Schapire

R.E.

, A decision-theoretic generalization of on-line learning and an application to boosting, J COMPUT SYST SCI 55(1) (1997), 119–139.

29.

Friedman

J.H.

, Greedy function approximation: A gradient boosting machine, Ann Statist 29(5) (2001), 1189–1232.

30.

Chen

T.Q.

and Guestrin

, XGBoost: A scalable tree boosting system. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD’16), 2016, pp. 785–794.

31.

G.L.

, Meng

, Finley

, et al., LightGBM: A highly efficient gradient boosting decision tree. In Proceedings of the 31st International Conference on Neural Information Processing Systems (NIPS’17), Vision, 2017, pp. 3149–3157.

32.

Ahn

S.J.

, Kim

J.H.

, Park

S.J.

and Han

J.K.

, Prediction of the therapeutic response after FOLFOX and FOLFIRI treatment for patients with liver metastasis from colorectal cancer using computerized CT texture analysis, Eur J Radiol 85(10) (2016), 1867–1874.

33.

Liu

, Cui

, Liu

, et al., Multi-subtype classification model for non-small cell lung cancer based on radiomics: SLS model, Med Phys 46(7) (2019), 3091–3100.

34.

, Jiang

, Yan

and Wang

, Imbalanced classification based on minority clustering synthetic minority oversampling technique with wind turbine fault detection application, IEEE Trans Industr Inform 17(9) (2021), 5867–5875.