Ischemic stroke subtyping method combining convolutional neural network and radiomics

2.1 Experimental data

Data used in this study are CTA images of 82 patients with IS from Shanghai Sixth People’s Hospital (36 cases of CE and 46 cases of LAA). The image bit depth is 16, and each patient’s CTA data contains about 350 images, all of which have a pixel size of 512×512 and a layer thickness of 0.8mm∼1 mm. All the included data were collected from 2016 to 2021. The conditions for data screening were: each patient underwent surgical treatment of thrombectomy, the patient’s IS type was obtained through pathological diagnosis and cases with other treatment methods such as thrombolysis were excluded; CTA are no artifacts in the image samples and the images are clear and not blurred; in order to control variables, only a single IS subtype patient is selected, the patient does not include LAA and CE at the same time; too small and blurred emboli are excluded unclear cases.

The experimental platform used in this stud is a 64-bit Windows 10 operating system, i5-8400-2.80 GHz processor, 16GB memory, and the software used is 3D Slicer 4.10.2 and SPSS 25.0.

2.2 Experimental framework

In this study, an IS subtyping model is constructed based on CNN and radiomics features. The specific process is as follows: Firstly, the cerebral embolism region was segmented from the CTA image of the patient’s head. The radiomics features of the extracted embolic regions include grayscale, morphological, and texture features; The extracted radiomics features are optimized for L2 norm, and the features are selected by combining random forest and sequence backward algorithms; at the same time, the CNN Cap-UNet is built to extract the deep learning features of the last layer of the network; finally, the selected radiomics features and deep learning features are combined, and 9 small-sample classifiers are used to train the combined features, respectively. The IS subtyping model is constructed, and the classifier with the best results is obtained to construct the IS subtyping model. The experimental process is shown in Fig. 1.

OPEN IN VIEWER

Fig. 1

Overview of the proposed method.

In order to highlight the information of brain emboli region, the method of window width and level adjustment was used to preprocess the image. The CT value of the cerebral embolus is about 45 HU, so in this paper, the window level of the CTA image is set to 45 HU, and the window width is set to 300 HU, the CT value ranges from –105 HU to 195 HU. After setting the window width and window level, the image can display the structural information of the brain tissue more clearly, which is conducive to the marking and feature extraction of the subsequent embolization part.

At present, there is no accurate automatic or semi-automatic segmentation method for IS embolism segmentation of CTA images, so manually delineating the region of interest (ROI) is still the standard segmentation method. The embolization area labeling in this paper was manually labelled by three experienced interventional radiologists from the Sixth People’s Hospital at the same time, and the labeling results are shown in Fig. 2. After the marking of the IS embolization part is completed, the consistency of the marking results of the three doctors needs to be judged. The intraclass correlation coefficient (ICC) [16] was used to test the three sets of labels, and the final ICC value was greater than 0.75, which shows that the same sample drawn by three doctors has high repeatability and good reliability.

OPEN IN VIEWER

Fig. 2

The result of marking the embolized area.

Feature extraction is used to extract large number of quantitative descriptions from medical image data and reflect the potential diseases of organs or tissues at the microscopic level from the quantitative image features. In this paper, the features of the emboli were extracted based on the embolic region after segmentation of the CTA image. The final extracted radiomics features were divided into three categories: grayscale features, morphological features, and texture features, for a total of 116 radiomics features.

Morphological features are mainly used to describe the geometric shape of the marked area, reflect the morphological structure of the embolus in two-dimensional and three-dimensional, and abstract the morphological structure of the embolus from the head CTA image, which is a very important feature. The morphological features have nothing to do with the gray intensity distribution of the ROI, and they are only calculated on the original image and mask during calculation, so the morphological structure of the obtained emboli will not be changed by the previous preprocessing method. The extraction of grayscale features is based on the properties of the first-order histogram, which reflects the grayscale distribution of pixels inside the emboli in three dimensions, which is the most intuitive feature of IS emboli images, including the mean, median, maximum, and minimum values of voxel intensity values. The texture feature is a second-order statistical feature that reflects the relationship between voxels in the three-dimensional emboli. Generally, this relationship is represented in structure by a repetitive or slowly changing arrangement relationship. Texture features are obtained through the gray level co-occurrence matrix (GLCM) [17], the gray level run length matrix (GLRLM) [18], the gray level size zone matrix (GLSZM) [19], the neighboring gray tone difference matrix (NGTDM) [20] and the gray level dependence matrix (GLDM) [21] five ways to calculate these. In the feature extraction process, the image types used are all unfiltered original images, the bin width is set to 25, the label is set to 1, the interpolator is set to ‘sitkBSpline’, and the resampled pixel spacing and weighting norm are both set to ‘None.’

There are often outliers and noise in the extracted features, so that it is necessary to normalize the features to optimize the features. In this study, L2 norm normalization is used to process the data. L2 norm normalization is to divide each data in the vector by the norm of the vector to obtain a new vector.

In order to solve the high-dimensional problem caused by too many features, the feature selection method needs to be used to reduce the dimension of the radiomics feature matrix. In this study, the random forest algorithm is used to rank the importance of the features, and the sequence backward search method is added on this basis. The lowest feature is removed, resulting in a new feature subset. Specific steps are as follows:

There is an original data set with D samples, and d samples are randomly selected from the original samples to construct a new data subset. Denote the number of features in the new data subset as H, extract h (h <  H) features from the features to form a new feature matrix, select an optimal feature from the feature matrix for splitting, and finally generate a decision tree.

The ten-fold cross-validation method was used to split the data into training dataset and test dataset for experiments. The feature selection step is performed only on the training dataset. The value of n is set to 45, the procedure is aborted when there are only 45 radiomics features left. At the same time, the feature VIM value obtained by the last random forest calculation is saved, and the features are arranged in descending order according to the VIM value to prepare for the subsequent construction of the subtyping model. The 45 features are screened, and their corresponding VIM values are shown in Fig. 3.

OPEN IN VIEWER

Fig. 3

Filtered Features.

In this study, Cap-UNet was built to extract CNN features, and the network structure is shown in Fig. 4. Firstly, according to the position of the embolus on the CTA image, a 64×64×64×1 pixel block that can contain all the information of the embolus is cropped, and then the cropped data samples are input into Cap-UNet for training. The overall design of the network refers to the symmetrical design of UNet, divides the overall network into an encoding part and a decoding part, and draws on the idea of Inception, replaces the convolutional part of the encoder with a capsule layer consisting of four different convolutional layers and a concatenated layer, and the SENet module [22] is embedded in the previous layer of the output feature, so that the model can pay more attention to the relationship between the channel features, the feature map after the SENet module, and the global maximum After the pooling layer, 128 CNN features are finally extracted.

OPEN IN VIEWER

Fig. 4

Cap-UNet structure.

The structure of the capsule layer is shown in Fig. 5. Each capsule layer contains four convolutional layers with convolution kernel sizes of 1×1×1, 3×3×3, 5×5×5 and 7×7×7, respectively. This study selects four convolutional layers with different scales to simulate the receptive fields of different sizes, respectively. In the figure, N is the number of channels of the convolutional layer. After four different convolutions, a connection layer will be used to obtain the features. The graph is spliced, and finally, the result of the connection layer splicing is added to the input to obtain the output.

OPEN IN VIEWER

Fig. 5

Capsule layer structure.

In IS subtyping, the construction of the classifier is the most important part, and the typing model directly determines the typing performance. This study uses a variety of machine learning classifiers to build models, namely support vector machine (SVM), K-nearest neighbor (KNN), decision tree (DT), extremely randomized trees (ET), random forest (RF), logistic regression (LR), adaptive boosting (AdaBoost), gradient boosting (GBoost), and the extreme gradient boosting (XGBoost) classifier. The above 9 classifiers have their own advantages and disadvantages, and their algorithm complexity, computational cost, sensitivity to noise, and interpretability have different characteristics, as well as different requirements for the size of the sample, object attributes, and sample balance problems. In this study, the selected classifier was used to train the extracted features, and the classifier with the best subtyping results was selected according to the experimental results to realize the prediction of IS subtypes.

The hyperparameters of each classifier are optimized by grid search technique, and the hyperparameter with the highest score on the training set was taken. In this study, the SVM classifier was based on RBF kernel and the regularization parameter (C) and the kernel coefficient (gamma) are respectively 0.01 and 0.001. For KNN classifier, the number of n-neighbors is 5. The min-leaf for DT is 3. The trees of ET and RF are 300, and other parameters are default. The regularization parameter of LR classifier with L2 regularization is 0.01. For AdaBoost, the base estimator used in this experiment is DT, and the algorithm is set to ‘SAMME.R’. For GBoost and XGBoost, the number of estimators is 100, and the learning rate is set as 0.001. In addition, the L2 regularization coefficient in XGBoost is set to 0.01 to avoid overfitting.

The experiment evaluates the performance of the IS subtyping model from two aspects, namely the evaluation index and verification method of the model. The evaluation indicators of the experiment are accuracy (ACC), sensitivity (SEN), specificity (SPE), receiver operating characteristic curve (ROC), area under the ROC curve (AUC), and 95% confidence interval (CI). ACC is the ratio of the number of IS subtypes correctly predicted by the model to the total number of samples, SEN represents the ratio of the number of correctly predicted cardiac types to the total number of CE types, and SPE represents the correct number of LAA types predicted by the model Ratio of the number of sclerosing types to total LAA types.

The experimental verification method is ten-fold cross-validation, which can eliminate the bias effect caused by data imbalance. In the verification process, the samples of each fold are randomly selected, and after the verification is completed, the mean value of the evaluation indicators is taken to reduce the difference caused by different sample divisions.

3.1 Influence of the number of radiomics features on the model

In the experiment, 5 different numbers of radiomics features were selected for comparative experiments, and the top 5, top 15, top 25, top 35 and top 45 features were selected, respectively, and 9 subtyping models were used to train the selected features. For the experimental evaluation indicators of different feature number models, the average value of 9 subtyping models is taken.

The experimental results are shown in Table 1. When the number of radiomics features is 25, the model obtains the best subtyping results, the AUC value is 0.8092, and the ACC value is 0.7683. The smaller or larger the number of features used, the worse the classification result of the model. Using too many features for typing does not achieve the expected effect. It may be that there are some redundant features in the feature set, which leads to a decrease in the accuracy rate, which also proves the necessity of feature selection before building a model. Using too few features for typing achieves a good typing effect, but too few features cannot reflect all the information of the data, and the loss of some data information leads to a lower accuracy than that of using 25 features. Using too many features for typing does not achieve the expected effect, and there may be some redundant features in the feature set, resulting in a decrease in the accuracy rate. This also proves the necessity of feature selection before building the model, and the optimal feature subset required for typing the model is obtained experimentally. The 25 features finally selected through experiments and the corresponding VIM values of the features are shown in Fig. 6, sorted in ascending order according to their importance.

OPEN IN VIEWER

Fig. 6

Feature Selection Sort.

In order to explore the influence of different features on the subtyping effect, 5 different feature combination configurations were selected in this experiment, namely single radiomics feature, single UNet feature, single Cap-UNet feature, combination of radiomics features with UNet and Cap-UNet features, respectively. the radiomics features are the 25 features selected in the previous experiment. The comparison of the results is shown in Table 2. The subtyping accuracy obtained by using the UNet feature and Cap-UNet feature alone is poor, and the ACC values are only 0.5976 and 0.622. After combining radiomics with UNet and Cap-UNet features, respectively, each indicator has a certain improvement compared with single features. The experimental results show that the combination of traditional radiomics features and deep learning features can more effectively reflect the information of the embolic region and improve the accuracy of IS classification. In addition, the features learned by the proposed Cap-UNet in this paper achieves better evaluation results than that of based on UNet model, which derived from that the addition of the residual module, SENet module and Capsule layer makes the network more focus on the target embolism region and effectively extract feature the information of the embolism, thus realizing the higher IS typing accuracy.

In order to further evaluate the effectiveness of the Cap-UNet features proposed in this paper, three classic CNNs, Vgg16, AlexNet, and DenseNet, were selected in this experiment to extract their deep learning features, and these three sets of features were compared with the obtained 25 radiomics features. After the combination, 9 subtyping algorithms are used for training, and the average of the evaluation results is obtained. The experimental results are shown in Table 3. The Cap-UNet feature subtyping performance proposed in this paper is better than other CNN features, and combined with the radiomics features, it can classify IS more accurately.

It can be seen from the above experiments that the combination of radiomics features and Cap-UNet features achieves the best results in the comparative experiments. In this experiment, the combination of 25 radiomics features and 128 Cap-UNet features obtained is used as the input of the model, and 9 different IS subtyping models are constructed using the 9 classifiers selected. The experimental results are shown in Table 4, Fig. 7 shows the ROC curves for each subtyping model.

OPEN IN VIEWER

Fig. 7

ROC curve of the subtyping model.

The experimental results show that the subtyping effect of the model based on the Boosting method is significantly better than other classifiers, and the subtyping model based on the Adaboost algorithm has achieved the best results among the 9 machine learning models. The AUC value and ACC value of the model are 0.9018 and 0.8929, the SEN value and SPE value were also higher than other typing methods, indicating that the typing model based on Adaboost has better typing ability and can more stably predict IS subtypes. The Adaboost algorithm trains different weak classifiers for the same training set, and then aggregates these weak classifiers to form a strong classifier to obtain higher prediction accuracy. For this classifier, the predicted probability of the sample set classification as the weight of the weak learner to obtain faster iteration speed and improved the generalization ability of the model.