Colon cancer prediction on histological images using deep learning features and Bayesian optimized SVM

Abstract

Colon cancer is one of the highest cancer diagnosis mortality rates worldwide. However, relying on the expertise of pathologists is a demanding and time-consuming process for histopathological analysis. The automated diagnosis of colon cancer from biopsy examination played an important role for patients and prognosis. As conventional handcrafted feature extraction requires specialized experience to select realistic features, deep learning processes have been chosen as abstract high-level features may be extracted automatically. This paper presents the colon cancer detection system using transfer learning architectures to automatically extract high-level features from colon biopsy images for automated diagnosis of patients and prognosis. In this study, the image features are extracted from a pre-trained convolutional neural network (CNN) and used to train the Bayesian optimized Support Vector Machine classifier. Moreover, Alexnet, VGG-16, and Inception-V3 pre-trained neural networks were used to analyze the best network for colon cancer detection. Furthermore, the proposed framework is evaluated using four datasets: two are collected from Indian hospitals (with different magnifications 4X, 10X, 20X, and 40X) and the other two are public colon image datasets. Compared with the existing classifiers and methods using public datasets, the test results evaluated the Inception-V3 network with the accuracy range from 96.5% - 99% as best suited for the proposed framework.

Keywords

Transfer learning features CNN colon cancer classification

1 Introduction

Colorectal cancer is common, and the fourth-leading cause of cancer-related deaths worldwide. In 2018, 10% of the worldwide cancer cases were recorded and it was evaluated that colorectal cancer was the second most common in both men and women [9]. The heterogeneity of biological tissue structures poses a challenge for both manual and automated histopathological study of slides [19]. Precise tumor detection is crucial for the survival of the patient and can be effectively achieved by analyzing the stained histological sections collected through biopsy or surgery. However, the significant role of virtual microscopy in pathology departments is undoubtedly used but the intra- and inter-observer variability remains a challenge because of the qualitative slide inspection. Even experienced pathologists are still not in agreement with tissue classification, and this leads to the conclusion that expert assessment alone is not sufficient to be the gold standard for histopathological evaluation [37]. Therefore, the market for computerized automation to boost diagnostic reproducibility is in high demand [19] across different magnifications. Traditional approaches seek to incorporate pattern recognition systems for rapid and automated cancer diagnostics with a range of hand-made features (texture and morphological). These are extracted from histology images and trained over a classifier to identify cancerous cells. Paradigm has recently moved to deep learning techniques in which the extraction of features and classifications are integrated within a single body.

Deep CNN consists of feed-forward artificial neural networks of several hidden layers and has become common for several applications, such as object tracking, detection, image classification, computer vision, etc., as they have shown significant performance [18, 20]. However, a large data set is typically needed with ground truth labels to train these deep CNNs from scratch. This is particularly unlikely in cases where the labeled data are scarce as it is expensive and tedious and typically restricted due to privacy issues for biomedical datasets. These factors prohibit deep CNNs from being implemented in many realistic circumstances.

Transfer learning has become popular among researchers as a viable alternative to profound CNNs as the original structure of the pre-trained model is preserved and used either to extract features or can be further tuned using the data available. Although the natural and biomedical data differ, it has been shown that deep CNN models trained with large-scale natural image datasets can be refined effectively to produce consistent and reliable results for biomedical imaging analyses [28]. With the BreaKHis dataset [33], transfer learning was employed with AlexNet and VGG16 models and with the deep learned features provided improved results using Support Vector Machine (SVM) classifier [12]. Pre-trained network models VGG16, VGG19, and ResNet50 were later analyzed with the same dataset for magnification independent scenario, and VGG16 with logistic regression classifier performed the best [27]. ResNet-50 and DenseNet-161 models were tested for the classification of histopathological images, and the former performed better [36]. Thus, different pre-trained CNN models are used for particular types of histopathological images as their efficiency varies. Consequently, the performance of the transfer learning model depends on the choice of the pre-trained models, the dataset images, their magnification, and the classifier. Hence, the choice of the existing CNN model best suited for a particular type of histopathological image is difficult.

In light of the above facts, a magnification-independent feature learning framework for colon cancer detection with a BO-SVM classifier is proposed in this paper. Pre-trained networks AlexNet, VGG16, and Inception-V3 are analyzed by extracting the deep learned features and the best network is compared with the effect of traditional SVM and softmax layer classifier. This is done for the particular task of colorectal cancer screening using four datasets of various magnifications. Specifically, the contributions are as follows:

Which pre-trained neural network is best suited for colon cancer detection in multiple datasets with various magnifications? The proposed model performance evaluation measures are compared for the pre-trained neural networks including the Alexnet, VGG 16, and Inception-V3 which were trained with colon images.

Does the BO-SVM classifier perform better than traditional classifiers? To verify this, the proposed framework was compared with traditional SVM and softmax layer classifier with the high-level features derived from different pre-trained networks.

How robust is the proposed framework? In this case, the model was evaluated using four datasets where two were collected from Indian hospitals with different magnifications, and the other two from public colon image datasets available. The model was compared with the existing techniques on the public datasets for generalization.

The following sections are structured as: Section 2 presents a brief overview of the related work. The proposed methodology is given in Section 3. Section 4 presents the experimental results and discussion. Section 5, concludes the paper.

2 Related works

Traditional supervised cancer screening approaches typically include the extraction of features and the training of a classifier for these features. For the colon cancer detection problem, a series of colon images were tested for different combinations of textural and morphological features, and an SVM classifier was trained to detect cancer [3, 5]. Various features including architectural, texture, and wavelet were extracted from the dataset images of colon histology for grade prediction [4 , 26]. In [25], the colon biopsy samples are segmented into three clusters using Adaptive Pillar K-means, and, based on Lumen Circularity (LUC), measure images are categorized as normal and malignant with the decision tree. The colorectal gland tissues are segmented using an adjustable threshold procedure in [7] and geometrical features are extracted to classify them as cancerous and non-cancerous using an SVM classifier with an efficiency of 93.74%. A predictive precision of 98% is achieved for colorectal cancer detection with 174 images by a combination of traditional texture and morphology with some modern geometric features [23, 24]. Only low-level and non-representational features of histopathological images are the final features extracted. This can then contribute to a bad classification result for the final model.

The recent development of deep neural networks that can combine automated feature extraction and classification in a single framework has taken over the traditional framework due to tuned complex segmentation and hand-crafted features. [31] proposed a new deep learning method of locality-sensitive approaches for detecting and identifying nuclei on hematoxylin and eosin (H&E) stained histopathology images of colorectal adenocarcinoma based on CNNs. Segmentation and classification based on the deep neural network were proposed to classify the benign and malignant colon images with 95% accuracy [15]. To obtain a square patch, each image is cropped and applied with affinity transformations to acquire more training samples. These samples are used to train a customized CNN architecture consisting of three convolution blocks, three completely connected layers, and a softmax classification layer [8]. Though this technique surpassed handcrafted strategies and amounted to about 83.25% accuracy, a large data set with the ground-truth mark is always required to train these deep CNNs from scratch. However, the use of deep CNNs in many practical biomedical problems is paralyzed due to limited datasets and being computationally expensive.

Moreover, transfer learning has been identified by researchers as an alternative to deep CNN training from scratch and restricted data availability. [38] suggested using deep-neural CNN activation features from ImageNet in large-scale images of the brain tissue and colon histopathology for segmentation and classification with an SVM classifier. This proved CNN features are better than hand-crafted features. [2] proposed a shape feature - Best Alignment Metric (BAM) - extracted from the region of interest obtained after CNN segmentation and tabulated an accuracy of 97% with SVM classifier for colon histological images. The abstract features extracted from Alexnet were used with a tandem of classifiers for the breast and colon histological image classification with an accuracy of 86% and 98.2% for four-class classification [17]. An analysis of feature learning with the CNN models SqueezeNetv1.1, MobileNet-v2, ResNet-18, and DenseNet-201, and classification with an SVM classifier lead to higher accuracy on four publically available datasets when compared to the fine-tuned models [1].

Due to the complex structure, scarce annotated data, and time-consuming nature, training CNN from scratch is less preferred. Transfer learning with finetuning or feature learning with classification is considered a solution to this. SVM classifier is usually considered for binary classification where the performance depends on the hyperparameters, and hence, optimum values must be chosen for better performance. Depending on the type of histopathological images, various pre-trained models have been experimented with, and the efficiency of the framework varies. Thus, the performance of transfer learning for the detection of cancer in the colon is highly dependent on the CNN model and the classifier. This paper aims to analyze the CNN models best suited for colon cancer detection across various datasets where hyperparameters of SVM are optimized with Bayesian optimization.

3 Proposed methodology

In this paper, the proposed colon cancer detection model identifies the cancer images with the deep neural features derived from the existing pre-trained neural network with the BO-SVM classifier. The model comprises three stages as shown in Fig. 1; (i) Data Augmentation; (ii) Feature Extraction with Transfer Learning; (iii) Classification. Thus, the proposed model is analyzed for three pre-trained networks, Alexnet, VGG 16, and Inception-V3, and the best network is chosen for the colon cancer detection framework. The following section describes these stages.

Fig. 1

Proposed methodology.

3.1 Data augmentation

Herein the data augmentation process, the quality and degree of variance in the data training are enhanced. It is used to classify images for stronger and simplified models that are invariant in image transformations and quality [22]. To specify additional incremental operations on the training images: flip the training images randomly around the vertical axis and convert them randomly up to 30-pixels, and scale up to 10% vertically and horizontally. Moreover, the augmentation in data helps to prevent overfitting the network and store the exact information of training images. After the augmentation, all images were sampled down to the network input size.

3.2 Feature extraction with transfer learning

Specifically, the Alexnet, Inception-V3, and VGG- 16 networks were used for the transfer and feature learning experiments for the colon biopsy image classification. Such architectures are similar because they comprise of sequentially ordered, convolutionary, pooling, and completely connected layer that creates a classification at the imaging level. However, the basic structure and number of these layers differ in each architecture (Fig. 1). A brief description of these architectures is given below.

Alexnet [16]: It is the name of Alex Krizhevskys CNN. It is based on the ImageNet database on over one million images. The network is 8 layers thick and categorizes images into a thousand categories including keyboards, mouse, crayons, animals, and so on. Therefore, the network has represented numerous images, and the input size of the network is 227 × 227. Thus, the features are extracted from the FC8 layer for classification.

VGG-16 [29]: It is a neural network developed on over one million images of ImageNet. The network consists of 16 layers and recognizes pictures in 1000 object categories such as keyboard, mouse, crayon, and animals. Hence, it represents the variety of images in the network. The network has a 224 × 224 image feed, as well as extracted features from the FC8 layer after the transfer learning.

Inception-V3 [35]: It is the CNN equipped with over a million ImageNet pictures. This is a wide network with 48 layers and can group images into 1000 categories of objects, including keyboards, mouse, pencils, and animals. Thus, the network has a rich depiction of a wide variety of images with a picture resolution of 299 × 299.

Though the architectural network model is initialized randomly in the final and completely connected stage, each architecture is considered by initializing its model weights based on pre-training in the image dataset in the transfer learning version. These pictures are converted into numeric feature vectors using the pretrained CNN, optimized weights, and without the need for fine-tuning. Further, these extracted features are used for the classification.

3.3 Classification

From the pre-trained networks, the high-level features are fed into the BO-SVM classifier to categorize the images into normal and malignant ones. Moreover, the performance of the SVM classifier was enhanced by optimizing the parameters with Bayesian optimization and tabulating the optimal hyperparameters [10, 32].

The SVM-based predictor model is created using sets parameters ω to a specific hyperparameters λ (such as the width of Radial Basis Function (RBF) kernel γ and C strength). Thus the objective function S is maximized with f (λ) as in Equation 1. $\underset{ω, λ}{maximize} S (ω, λ | {(x_{i}, y_{i})}_{i = 1}^{N})$ (1) where x_i represents the features and y_i denotes the label for N observations. The value for f is very expensive to calculate since a classifier needs to be trained to solve this problem using a simple, derivative-free optimization technique. Thus, we optimize objective functions that take a long time for evaluation(minutes or hours) using Bayesian optimization. S is the accuracy for the function under consideration of the corresponding SVM classifier with the RBF kernel, and λ = (γ, C)) is the set of two hyperparameters adjusted to improve SVM efficiency to predict cancer by f.

The data is assumed to be constrained from a subordinate joint distribution over samples with their binary labels μ as $D = {(x_{i}, y_{i})_{i = 1}^{N}} \sim μ (X \times {- 1, + 1})$ (2) where X represents the features of the samples being investigated. Specifically, we want to maximize the desired accuracy over all possible μ samples. $f (λ) = \int [svm (x_{i} | D, λ) = y_{i}] d μ$ (3) where svm (x_i|D, λ) is a prediction of the SVM x_i’s label trained on training set D with hyperparameters λ. An unknown probability distribution cannot be implemented, but this value can be approximated using internal cross-validation. In other words, a stochastic approximation is used. $\bar{f} (λ) = {CV}_{D} (svm (X_{test} | D_{train}, λ))$ (4) where CV_D is the mean precision of the predictions model p, in comparison with true labels y oversplits of D in D_Train and D_Test (comprising of X_test data and the respective Y_test labels). Thus $\bar{f} (λ) = f (λ) + ɛ$ , where ɛ represents the noise. Random optimization is specified as a strategy for the distribution of a probability over hyperparameters P (L). In other terms, we sample P (L), ignoring all previous samples and their effects in any iteration. Ultimately, we return the maximum obtained values. The optimization method would be much easier if the exact form of f were known. Unfortunately f is a black-box function, which is very complex, even costly to test. Nevertheless, certain simplifying assumptions for f could resolve a problem. Assume that a probability distribution sample can be interpreted as f over functions f ∼ P (f) , fεF. The assumption about the loss function δ_n is expressed as in Equation 5. $\arg \max_{α} E_{P (f)} [δ_{n} (α)] = \arg \min_{α} \int_{F} δ_{n} (α) dP (f)$ (5) Thus, by applying Bayes rule $\begin{matrix} \arg \max_{α} E_{P (f | x_{1 : n - 1}, {\bar{f}}_{1 : n - 1})} [δ_{n} (α)] = \\ \arg \min_{α} \int_{F} δ_{n} (α) dP (f | x_{1 : n - 1}, {\bar{f}}_{1 : n - 1}) \end{matrix}$ (6) it is a very simple equation for Bayesian general techniques of optimization. With additional assumptions concerning the prior distribution of P (f),), very successful solutions can be given for the entire cycle. To exploit the features, the Gaussian process is considered, and the target function f is the stochastic process. Expected improvement is utilized to select the loss function because δ_n (α) needs to know the optimal solution.

Thus, the obtained optimized parameters of SVM are used to classify the samples as normal and malignant.

4 Results and discussion

This section discusses the datasets used for evaluation and performance measures. Later, the results are analyzed.

4.1 Data sources and performance evaluation measures

The proposed model is analyzed using different colon biopsy image datasets with various magnification factors consisting of normal and malignant labels. A brief description is provided in Table 1.

Dataset A: This dataset comprises images of H&E stained slides of colon biopsy, 5-6-μmm-thick tissue section taken at different magnifications (4X, 10X, and 40X) for normal and malignant images from the Ishita Pathology Center and Cytocare Pathology Center. Magcam CD5 with Olympus CX33 was used to capture the images. Dr. Ranjana Srivastava, the Senior Consultant at the Ishita Pathology Center analyzed the H&E slides and prepared the image labels for the dataset.

Dataset B: This dataset consists of images of H&E-stained colon biopsy samples of a 5-6μmm-thick tissue section taken at various magnifications (10X, 20X, and 40X) for normal and malignant images from Aster Medcity, Kochi, India. The NIS element viewer microscope was used to view the slides, and Nikon eclipse Ci was used to capture images. Dr. Shahin Hameed and Dr. Sarah Kuruvila, Department of Pathology, Aster Medcity, Kochi, India, analyzed H&E colon biopsy slides. They prepared the dataset providing the ground-truth labels.

ImeadiaTreat [34]: This dataset provided by Stoean et al. includes images captured from H&E-stained colon, tissue slides obtained by the Emergency County Hospital of Craiova, Romania.

GlaS dataset [30]: consists of colon images from the University Hospitals Coventry and Warwickshire, UK, collected by a team of pathologists. Zeiss MIRAX MIDI was used for collecting images of colon tissue that were stained by H&E.

The performance measures used to evaluate the proposed model is given in Table 2 with true positive (P_v), true negative (N_v), false positive (FP), and false negative (FN).

Table 1
Datasets used for evaluation

Datasets Magnification No. of Normal Images No. of Malignant Images

Dataset A 4X 100 100

10X 100 100

40X 100 100

Dataset B 10X 90 90

20X 90 90

40X 90 90

Imeadiatreat 10X 62 62

GlaS 20X 42 42

Datasets	Magnification	No. of Normal Images	No. of Malignant Images
Dataset A	4X	100	100
	10X	100	100
	40X	100	100
Dataset B	10X	90	90
	20X	90	90
	40X	90	90
Imeadiatreat	10X	62	62
GlaS	20X	42	42

Table 2

Performance evaluation measures

Parameter	Formula	Description
Accuracy	$\frac{P_{v} + N_{v}}{Pv + N_{v} + FP + FN}$	Accuracy with which a classifier can label correctly
Error rate		1-Accuracy
Sensitivity	$\frac{P_{v}}{P_{v} + FN}$	The classifier’s skill to determine the positive labels.
Specificity	$\frac{N_{v}}{N_{v} + FP}$	The classifier’s skill to to determine negative labels.
Precision	$\frac{P_{v}}{P_{v} + FP}$	Actual positive samples among correctly predicted positive samples.
False Positive rate		1- Specificity
F-Score	$2 \times \frac{Precision \times Recall}{Precision + Recall}$	It is the measurement of the accuracy of a test that as the harmonic mean of sensitivity and precision.
MCC	$\frac{P_{v} \times N_{v} - FP \times FN}{\sqrt{((P_{v} + FN) (P_{v} + FP) (N_{v} + FN) (N_{v} + FP))}}$	Its a coefficient of correlation between the binary classifications of observed and predicted.
Kappa Statistics	$\frac{observed accuracy - expected accuracy}{1 - expected accuracy}$	Its a degree of how closely the instances categorised by the machine learning classifier matched the records labeled as ground truth

4.2 Experimental results

To validate the predictive precision of the model, we used 10-fold cross-validation to formulate data for the training/testing. The proposed model is evaluated in different stages as follows:

The proposed model is evaluated with three pre-trained networks; Alexnet, VGG-16, and Inception-V3 to find the best-suited network for colon cancer detection.

We compared the BO-SVM classifier with the traditional SVM and softmax layer classifier for different pre-trained networks.

The proposed model is evaluated with various magnifications of Dataset A and B to demonstrate the robustness.

Finally, we compared the existing methods in the GlaS dataset with the proposed framework.

The proposed model is evaluated on four datasets consisting of various magnifications using three networks - Alexnet, VGG-16, and Inception-V3 - as demonstrated in Table 3. We considered 4X, 10X, and 40X magnified images from Dataset A, and Dataset B, 10X, 20X, and 40X images were also taken for training and testing with cross-validation. Thus, Datasets A and B considered all the magnified images. Moreover, Datasets A, B, and Glas have the highest accuracy of 98.33%, 99.07%, and 96.67% respectively on Inception V3 network, whereas, the Dataset ImediaTreat, gives the highest accuracy of 100% on Alexnet network. However, considering other parameters such as F-Score, Kappa Statistics, and MCC, we concluded that the Inception-V3 network is best suited for colon cancer detection with the proposed framework. F-Score of 0.9836, 0.9907, 0.9804, and 0.9655 was observed for Datasets A, B, ImediaTreat, and GlaS respectively with the Inception-V3 network, which is high when compared to other networks.

Table 3
Performance measures of proposed model on different datasets

Network Performace Measures Dataset A Dataset B ImediaTreat GlaS

Alexnet Accuracy 0.9333 0.9722 1 0.8889

Error Rate 0.0667 0.0278 0 0.1111

Sensitivity 0.9063 0.9636 1 0.8182

Specificity 0.9643 0.9811 1 1

Precision 0.9667 0.9815 1 1

False Positive Rate 0.0357 0.0189 0 0

F-Score 0.9355 0.9725 1 0.9000

MCC 0.8686 0.9446 1 0.7977

Kappa Statistics 0.8667 0.9444 1 0.7778

VGG-16 Accuracy 0.9083 0.9833 0.9808 0.8333

Error Rate 0.0917 0.0167 0.0192 0.1667

Sensitivity 0.9016 0.9677 1 0.8000

Specificity 0.9153 1 0.963 0.8750

Precision 0.9167 1 0.9615 0.8889

False Positive Rate 0.0847 0 0.037 0.1250

F-Score 0.9091 0.9836 0.9804 0.8421

MCC 0.8168 0.9672 0.9623 0.6708

Kappa Statistics 0.8167 0.9667 0.9615 0.6667

Inception-V3 Accuracy 0.9833 0.9907 0.9808 0.9667

Error Rate 0.0167 0.0093 0.0192 0.0333

Sensitivity 0.9677 1 1 1

Specificity 1 0.9818 0.9630 0.9375

Precision 1 0.9815 0.9615 0.9333

False Positive Rate 0 0.0182 0.0370 0.0625

F-Score 0.9836 0.9907 0.9804 0.9655

MCC 0.9672 0.9816 0.9623 0.9354

Kappa Statistics 0.9667 0.9815 0.9615 0.9333

The proposed method classified the samples with BO-SVM with the features extracted from each of the pre-trained networks. Figure 2 demonstrates the accuracy comparison between the proposed method with the SVM classifier and the softmax layer classifier for the three network architecture. With Alexnet, the proposed framework gave a higher accuracy of 0.933, 0.9722, 1, and 0.8889 when compared to the SVM classifier and the softmax classifier. However, with the VGG-16 network, the softmax layer classifier gave better accuracy of 1 and 0.9231 on Dataset B and GlaS when compared with other classifiers. Additionally, when comparing Inception V3 with SVM and softmax layer classifier, the proposed framework gives better accuracy of 0.9833, 0.9907, 0.9808, and 0.9667 for Datasets A, B, ImediaTreat, and GlaS respectively. Figure 3 shows the execution time taken by these classifiers on different datasets on the Inception-V3 network. It shows that the softmax layer classifier takes time for complete training and tuning in transfer learning on the Inception-V3 network than others. It is also noted that as the size of the dataset increases, the training time also increases. However, BO-SVM takes more time than SVM but produces better results than SVM in all of the datasets. Thus, by analyzing all the statistical error parameters, the proposed framework works better when compared with SVM and softmax layer classifier on the Inception-V3 network.

Fig. 2

Bayesian Optimised SVM (BO-SVM) comparison with SVM and Softmax classifier across different network architectures and datasets.

Fig. 3

Execution time comparison between SVM, softmax layer classifier and BO-SVM on different datasets.

To illustrate the superiority of SVM over other classifiers, features learned from different CNN models are classified with various classifiers - k-Nearest Neighbor (NN) with k=5, Naive Bayes, Adaboost with a decision tree, and SVM with Weka 3.8 - and the accuracy comparison is given in Fig. 4. k-NN performs least with all the networks. SVM performs better when compared with the other classifiers across datasets as well as CNN models with the highest accuracy of 0.9583, 0.9722, 0.9688, and 0.9444 with Datasets A (Inception-V3), B (Alexnet, VGG 16), ImediaTreat (Inception-V3), and GlaS (Inception-V3) respectively. Hence, in the proposed framework, the hyperparameters of SVM were optimized to enhance the performance.

Fig. 4

Accuracy comparison of feature learning with different classifiers.

The proposed framework has been evaluated for various magnifications, where 4X, 10X, and 40X; and 10X, 20X, and 40X images are considered from Dataset A and B respectively. This evaluation is shown in Fig. 5. With 4X of Dataset A, VGG-16 and Inception-V3 networks give an accuracy of 0.975, whereas, 10X and 20X in Inception-V3 network have the highest accuracy of 0.9833 and 0.9808 respectively when compared with the other networks. With Dataset B 10X magnification, all the networks obtained an accuracy of 0.9444, whereas, with 20X and 40X magnifications, the VGG-16 network performs better with 1 and 0.9722 respectively. The difference in the performance is due to the variations in the image acquisitions, the staining methods, and the different illumination conditions. However, considering all the differences and other statistical error parameters, the proposed model performed better than all the traditional models across the various magnifications.

Fig. 5

Proposed method evaluation across different magnifications for Datasets A and B.

The high-level features extracted from the trained Inception-V3 performs better than the general texture features. Figure 6 gives the comparison of the proposed framework with existing texture features including GLCM [13], Histogram [14], LBP [21], HOG [11], and hybrid (GLCM+ Histogram + LBP + HOG). Further, these features are extracted from the stain normalized and contrast-enhanced colon images and classified using BO-SVM. In this evaluation, the performance of these features varies in each of the datasets. The LBP and GLCM features perform better on Dataset A, and no particular feature performs in all the datasets. However, the proposed model features from the Inception-V3 network performed well in all the datasets with an accuracy of more than 96.5%. Thus, the high-level features extracted from the pre-trained networks can be generalized for colon cancer detection.

Fig. 6

Accuracy comparison of the proposed framework with handcrafted features on different datasets.

Moreover, analyzing the performance evaluation measures on different existing network architecture, found that the proposed methodology works best on the Inception-V3 network with the BO-SVM classifier. The feature map downsizing is usually carried out with max-pooling for AlexNet and VGG-16, where the maximum pooling followed by a convolution layer is too costly. The powerful reduction in grid size in launch V3 integrates two sets of feature maps into 640 feature maps and leads to the next stage of the inception module. Thus, the Inception-V3 network with BO-SVM detects cancer in colon biopsy images with better performance in all four datasets.

Table 4 illustrates the comparison of the proposed method with different existing methods for the GlaS dataset. The proposed generalized model was tested with four different datasets across various magnifications, and 96.67% accuracy was observed on the GlaS dataset. The overall results of the proposed model were compared to the existing methods as demonstrated in Table 4. However, [2, 24] exhibited better performance in accuracy, F-Score, and specificity as these techniques were tuned to work for this dataset. Image segmentation for finding the region of interest in these techniques depend on the magnification and have been proven only on the GlaS dataset. 10X images were also evaluated in [24], whereas the proposed model was evaluated across multiple magnifications (40X, 20X, 10X, and 4X). The handcrafted features are extracted from the region of interest in the existing methods, while high-level features are extracted from the Inception-V3 trained network without any segmentation in this study. However, existing techniques [17, 34] with ImediaTreat network are not included as these have contributed toward multiclass classification and grading and detection are less explored. Thus, the task of the proposed model detects cancer from the features extracted regardless of the segmentation tasks. Interestingly, the proposed method is a promising generalized colon cancer detection framework that extracts high-level features from the Inception-V3 trained network. It has an average classification accuracy of 96.5% -99% across four different datasets from different countries across various magnifications (40X, 20X, 10X, and 4X).

Table 4

Comparison of the proposed method with existing methods on GlaS Dataset

GlaS Dataset	Segmentation / Features / Classifier	Accuracy	F-Score	Sensitivity	Specificity
Sengar N., et al.[26]
(2016)	Intensity-based thresholding / Statistical, GLCM, GLRLM / SVM	83.33%	–	0.8181	0.8421
Saroja et al. [25]
(2017)	Adaptive k-means pillar / Lumen Circularity / Decision Tree	93.00%	0.9669	0.8076	0.9400
Dutta, et al. [7]
(2018)	Adaptive threshold / Geometric features / Linear-SVM	93.74%	–	–	–
Kainz et al. [15]
(2017)	CNN Segmentation / CNN Features / CNN Classification	95.00%	0.8890	–	–
Awan et al. [2]
(2017)	CNN Segmentation (Object-net) / Best alignment matrix / SVM	97.00%	0.9778	–	–
Rathore et al. [24]
(2019)	Multi step gland segmentation / Image, gland, patch based features / Metaclassifier(Linear, RBF, Sigmoid - SVM)	98.30%	–	0.9780	0.9880
Proposed	–/Deep features (Inception-V3) / Bayesian SVM	96.67%	0.9665	1	0.9375

5 Conclusion

This paper emphasized on colon cancer detection framework based on features extracted from existing deep learning networks, where transfer learning was deployed and classified using Bayesian optimized SVM classifier. The networks are built on the large ImageNets image dataset. Based on the specific task, the number of neurons inside the finally attached layer can be calculated and trained again with the fully connected layer parameters. The proposed framework was experimented with for three networks including the Alexnet, VGG-16, and Inception-V3 using four datasets; and the evaluations showed that the Inception-V3 network is the best for colon cancer detection with an accuracy of 0.9833, 0.9901, 0.9808, and 0.9667 for Datasets A, B, ImediaTreat, and GlaS respectively. Thus, the proposed framework performed better when compared to the existing classifiers and techniques on public datasets. However, the most important limitation lies in the heterogeneity of the histopathological representations of colon cancer. This creates problems in trying to classify an image as benign. Therefore, the emphasis of our future work should be on how this effect on the study of histopathological photos of colon cancer could be avoided or reduced. Further grading of the malignant images could also be explored.

Footnotes

Acknowledgment

We thank Ishita Pathology Center (Allahabad) and Aster Medcity (Kochi) for providing the datasets for smooth research. We also thank Dr. Ranjana Srivastava (Consultant Pathologist, Ishita Pathology Center) and Dr. Sarah Kuruvila (Former Senior Pathologist, Aster Medcity), Dr. Jyotima Agarwal (Cytocare Center, Allahabad) for extending their helping hand for the completion of this research. Dr. Shahin Hammed (Consultant, MVR Cancer Center, Poolacode, Kerala) was working with Aster Medcity at the time of dataset collection and is continuing his valuable support throughout the research.

References

Alinsaif

and Lang

, Histological image classification using deep features and transfer learning, In 2020 17th Conference on Computer and Robot Vision (CRV), pages 101–108, Los Alamitos, CA, USA, may 2020. IEEE Computer Society. doi: 10.1109/CRV50864.2020.00022.

Awan

, Sirinukunwattana

, Epstein

, Jefferyes

, Qidwai

, Aftab

, Mujeeb

, Snead

and Rajpoot

, Glandular morphometrics for objective grading of colorectal adenocarcinoma histology images, Scientific Reports7 (2017), 12. doi: 10.1038/s41598-017-16516-w.

Babu

, Singh

, Gupta

and Hameed

, Colon cancer detection in biopsy images for indian population at different magnification factors using texture features, In 2017 Ninth International Conference on Advanced Computing (ICoAC), pages 192–197, 2017.

Babu

, Gupta

, Singh

and Hameed

, Colon cancer prediction on different magnified colon biopsy images, In 2018 Tenth International Conference on Advanced Computing (ICoAC), pages 277–280, 2018.

Babu

, Gupta

, Singh

, Hameed

, Nayar

and Veena

, Cancer screening on indian colon biopsy images using texture and morphological features, In 2018 International Conference on Communication and Signal Processing (ICCSP), pages 0175–0181, 2018.

Babu

, Gupta

, Tripty

and Hameed

, Prediction of normal & grades of cancer on colon biopsy images at different magnifications using minimal robust texture & morphological features, Indian Journal of Public Health Research & Development11(01) (2020), 695. doi: 10.37506/v11/i1/2020/ijphrd/193905.

Banwari

, Sengar

and Dutta

, Image processing based colorectal cancer detection in histopathological images, International Journal of E-Health and Medical Communications9(04) (2018), 1–18. doi: 10.4018/IJEHMC.2018040101.

Bayramoglu

, Kannala

and Heikkilä

, Deep learning for magnification independent breast cancer histopathology image classification, In 2016 23rd International Conference on Pattern Recognition (ICPR), pages 2440–2445, 2016.

Bray

, Ferlay

, Soerjomataram

, Siegel

R.L.

, Torre

L.A.

and Jemal

, Global cancer statistics: Globocan estimates of incidence and mortality worldwide for 36 cancers in 185 countries, CA: A Cancer Journal for Clinicians68(6) (2018), 394–424. doi: 10.3322/caac.21492.

10.

Czarnecki

, Podlewska

and Bojarski

, Robust optimization of svm hyperparameters in the classification of bioactive compounds, Journal of Cheminformatics7, 2015.

11.

Dalal

and Triggs

, Histograms of oriented gradients for human detection, In 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), volume 1, pages 886–893, 2005.

12.

Deniz

, Sengur

, Kadiroğlu

, Guo

, Bajaj

and Budak

, Transfer learning based histopathologic image classification for breast cancer detection, Health Information Science and Systems6 (2018), 12. doi: 10.1007/s13755-018-0057-x.

13.

Haralick

R.M.

, Statistical and structural approaches to texture, Proceedings of the IEEE67(5) (1979), 786–804.

14.

Iivarinen.

, Surface defect detection with histogram-based texture features, In D.P. Casasent, editor, Intelligent Robots and Computer Vision XIX: Algorithms, Techniques, and Active Vision, volume 4197, pages 140–145. International Society for Optics and Photonics, SPIE, 2000. doi: 10.1117/12.403757.

15.

Kainz

, Pfeiffer

and Urschler

, Segmentation and classification of colon glands with deep convolutional neural networks and total variation regularization, PeerJ5, 2017.

16.

Krizhevsky

, Sutskever

and Hinton

G.E.

, Imagenet classification with deep convolutional neural networks, Commun ACM60(6) (2017), 84–90. ISSN 0001-0782. doi: 10.1145/3065386.

17.

Lichtblau

and Stoean

, Cancer diagnosis through a tandem of classifiers for digitized histopathological slides, PLOS ONE14(1) (2019), 1–20. doi: 10.1371/journal.pone.0209274.

18.

Liu

J.-E.

, An

F.-P.

and Mateos

, Image classification algorithm based on deep learning-kernel function, Sci Program 2020, Jan. 2020. ISSN 1058-9244. doi: 10.1155/2020/7607612.

19.

McCann

M.T.

, Ozolek

J.A.

, Castro

C.A.

, Parvin

and Kovacevic

, Automated histology analysis: Opportunities for signal processing, IEEE Signal Processing Magazine32(1) (2015), 78–87.

20.

Mittal

, Srivastava

and Chawla

, Review of different techniques for object detection using deep learning, In Proceedings of the Third International Conference on Advanced Informatics for Computing Research, ICAICR ’19, New York, NY, USA, 2019. Association for Computing Machinery. ISBN 9781450366526. doi: 10.1145/3339311.3339357.

21.

Ojala

, Pietikinen

and Harwood

, A comparative study of texture measures with classification based on featured distributions, Pattern Recognition29(1) (1996), 51–59. ISSN 0031-3203. doi: https://doi.org/10.1016/0031-3203(95)00067-4.

22.

Perez

and Wang

, The effectiveness of data augmentation in image classification using deep learning, CoRR, abs/1712.04621, 2017. URL http://arxiv.org/abs/1712.04621.

23.

Rathore

, Hussain

and Khan

, Automated colon cancer detection using hybrid of novel geometric features and some traditional features, Computers in Biology and Medicine65 (2015), 279–296. ISSN 0010-4825. doi: https://doi.org/10.1016/j.compbiomed.2015.03.004.

24.

Rathore

, Iftikhar

, Chaddad

, Niazi

, Karasic

and Bilello.

, Segmentation and grade prediction of colon cancer digital pathology images across multiple institutions, Cancers 11 (2019), 1700. doi: 10.3390/cancers11111700.

25.

Saroja

and Priyadharson

A.S.

, Adaptive pillar k-means clustering-based colon cancer detection from biopsy samples with outliers, Computer Methods in Biomechanics and Biomedical Engineering: Imaging & Visualization7(1) (2019), 1–11. doi: 10.1080/21681163.2017.1350603.

26.

Sengar

, Mishra

, Dutta

M.K.

, Prinosil

and Burget

, Grading of colorectal cancer using histology images, In 2016 39th International Conference on Telecommunications and Signal Processing (TSP), pages 529–532, 2016.

27.

Shallu and Mehra

, Breast cancer histology images classification: Training from scratch or transfer learning?ICT Express4(4) (2018), 247–254. ISSN 2405-9595. doi: https://doi.org/10.1016/j.icte.2018.10.007.

28.

Shin

, Roth

H.R.

, Gao

, Lu

, Xu

, Nogues

, Yao

, Mollura

and Summers

R.M.

, Deep convolutional neural networks for computer-aided detection: Cnn architectures, dataset characteristics and transfer learning, IEEE Transactions on Medical Imaging35(5) (2016), 1285–1298.

29.

Simonyan

and Zisserman

, Very deep convolutional networks for large-scale image recognition, arXiv 1409.1556, 09 2014.

30.

Sirinukunwattana

, Snead

D.R.J.

and Rajpoot

N.M.

, A stochastic polygons model for glandular structures in colon histology images, IEEE Transactions on Medical Imaging34(11) (2015), 2366–2378.

31.

Sirinukunwattana

, Raza

S.E.A.

, Tsang

, Snead

D.R.J.

, Cree

I.A.

and Rajpoot

N.M.

, Locality sensitive deep learning for detection and classification of nuclei in routine colon cancer histology images, IEEE Transactions on Medical Imaging35(5) (2016), 1196–1206.

32.

Snoek

, Larochelle

and Adams

R.P.

, Practical bayesian optimization of machine learning algorithms, In Proceedings of the 25th International Conference on Neural Information Processing Systems - Volume 2, NIPS’12, page 2951–2959, Red Hook, NY, USA, 2012. Curran Associates Inc.

33.

Spanhol

F.A.

, Oliveira

L.S.

, Petitjean

and Heutte

, A dataset for breast cancer histopathological image classification, IEEE Transactions on Biomedical Engineering63(7) (2016), 1455–1462.

34.

Stoean

, Stoean

, Sandita

, Ciobanu

, Mesina

and Gruia

C.L.

, Svm-based cancer grading from histopathological images using morphological and topological features of glands and nuclei, In G.D. Pietro, L. Gallo, R.J. Howlett and L.C. Jain, editors, Intelligent Interactive Multimedia Systems and Services 2016, pages 145–155, Cham, 2016. Springer International Publishing. ISBN 978-3-319-39345-2.

35.

Szegedy

, Vanhoucke

, Ioffe

, Shlens

and Wojna

, Rethinking the inception architecture for computer vision, In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 2818–2826, 2016.

36.

Talo

, Automated classification of histopathology images using transfer learning,43. November. ISSN, Artificial Intelligence in Medicine101 (2019), 101743. November. ISSN 0933-3657. doi: 10.1016/j.artmed.2019.101743.

37.

van Putten

P.G.

, Hol

, van Dekken

, van Krieken

J.H.

, van Ballegooijen

, Kuipers

E.J.

and van Leerdam

M.E.

, Inter-observer variation in the histological diagnosis of polyps in colorectal cancer screening, Histopathology58(6) (2011), 974–981. doi: 10.1111/j.1365-2559.2011.03822.x.

38.

, Jia

, Wang

, Ai

, Zhang

, Lai

and Chang

, Large scale tissue histopathology image classification, segmentation, and visualization via deep convolutional activation features, BMC Bioinformatics, 18, 2017.

Network	Performace Measures	Dataset A	Dataset B	ImediaTreat	GlaS
Alexnet	Accuracy	0.9333	0.9722	1	0.8889
	Error Rate	0.0667	0.0278	0	0.1111
	Sensitivity	0.9063	0.9636	1	0.8182
	Specificity	0.9643	0.9811	1	1
	Precision	0.9667	0.9815	1	1
	False Positive Rate	0.0357	0.0189	0	0
	F-Score	0.9355	0.9725	1	0.9000
	MCC	0.8686	0.9446	1	0.7977
	Kappa Statistics	0.8667	0.9444	1	0.7778
VGG-16	Accuracy	0.9083	0.9833	0.9808	0.8333
	Error Rate	0.0917	0.0167	0.0192	0.1667
	Sensitivity	0.9016	0.9677	1	0.8000
	Specificity	0.9153	1	0.963	0.8750
	Precision	0.9167	1	0.9615	0.8889
	False Positive Rate	0.0847	0	0.037	0.1250
	F-Score	0.9091	0.9836	0.9804	0.8421
	MCC	0.8168	0.9672	0.9623	0.6708
	Kappa Statistics	0.8167	0.9667	0.9615	0.6667
Inception-V3	Accuracy	0.9833	0.9907	0.9808	0.9667
	Error Rate	0.0167	0.0093	0.0192	0.0333
	Sensitivity	0.9677	1	1	1
	Specificity	1	0.9818	0.9630	0.9375
	Precision	1	0.9815	0.9615	0.9333
	False Positive Rate	0	0.0182	0.0370	0.0625
	F-Score	0.9836	0.9907	0.9804	0.9655
	MCC	0.9672	0.9816	0.9623	0.9354
	Kappa Statistics	0.9667	0.9815	0.9615	0.9333

Colon cancer prediction on histological images using deep learning features and Bayesian optimized SVM

Abstract

Keywords

1 Introduction

2 Related works

3 Proposed methodology

3.2 Feature extraction with transfer learning

3.3 Classification

4.1 Data sources and performance evaluation measures

Table 1 Datasets used for evaluation Datasets Magnification No. of Normal Images No. of Malignant Images Dataset A 4X 100 100 10X 100 100 40X 100 100 Dataset B 10X 90 90 20X 90 90 40X 90 90 Imeadiatreat 10X 62 62 GlaS 20X 42 42

Footnotes

Acknowledgment

References

Table 1
Datasets used for evaluation

Datasets Magnification No. of Normal Images No. of Malignant Images

Dataset A 4X 100 100

10X 100 100

40X 100 100

Dataset B 10X 90 90

20X 90 90

40X 90 90

Imeadiatreat 10X 62 62

GlaS 20X 42 42