Deep learning classifiers for computer-aided diagnosis of multiple lungs disease

Abstract

Background:

Computer aided diagnosis has gained momentum in the recent past. The advances in deep learning and availability of huge volumes of data along with increased computational capabilities has reshaped the diagnosis and prognosis procedures.

Objective:

These methods are proven to be relatively less expensive and safer alternatives of the otherwise traditional approaches. This study is focused on efficient diagnosis of three very common diseases: lung cancer, pneumonia and Covid-19 using X-ray images.

Methods:

Three different deep learning models are designed and developed to perform 4-way classification. Inception V3, Convolutional Neural Networks (CNN) and Long Short Term Memory models (LSTM) are used as building blocks. The performance of these models is evaluated using three publicly available datasets, the first dataset contains images for Lung cancer, second contains images for Covid-19 and third dataset contains images for Pneumonia and normal subjects. Combining three datasets creates a class imbalance problem which is resolved using pre-processing and data augmentation techniques. After data augmentation 1386 subjects are randomly chosen for each class.

Results:

It is observed that CNN when combined with LSTM (CNN-LSTM) produces significantly improved results (accuracy of 94.5 %) which is better than CNN and InceptionV3-LSTM. 3,5, and 10 fold cross validation is performed to verify all results calculated using three different classifiers

Conclusions:

This research concludes that a single computer-aided diagnosis system can be developed for diagnosing multiple diseases.

Keywords

Convolutional neural network long short-term memory lung cancer Xray

1 Introduction

The traditional methods of diagnosis of a disease are generally expensive, time taking and demand special care and attention especially in case of contagious diseases. The availability of test kits and field specialists is often limited in hospitals. Due to the lengthy acquisition time of some of the modalities and difficulties of sanitizing the equipment before making it accessible to other patients, the danger of infecting health personnel rises.

Recent pandemic of Covid-19 has revolutionize the whole world. More and more computer aided systems are now being used to perform various tasks and healthcare is no different. With the help of deep learning complex tasks that otherwise requires experts can be performed easily in lesser amount of time. This study is an attempt to explore the computer aided diagnosis of three lung diseases that includes lung cancer, pneumonia and Covid-19 using deep learning. A lot of population is affected with diseases. It is estimated in 2018 that around 9.6 million deaths are caused by lung cancer. Every year, over 800,000 children under the age of five die from pneumonia, with over 2200 fatalities per day. For every 100,000 kids, there are more than 1400 who have pneumonia. According to the Global Burden of Disease Study, pneumonia and other lower respiratory tract infections were the second leading cause of mortality in 2013. In hospitals across Europe, over 35% of patients had pneumococcal illness, and globally, 27.3% is the quotient [1]. Millions of people suffered from Covid-19 in the past three years. A system that can quickly diagnose these diseases is really the need of the hour.

All of the above mentioned diseases damage the lungs and respiratory system. In lung cancer a tumor is formed due to the abnormal growth of in the lungs. These cells spread really fast because of the bloodstream. It is seen that due to lymph flow, these cells frequently travel to the middle of the chest and metastasis occurs. It is very tough to identify lung cancer during the early stages as there are no clear symptoms and it becomes very tough to save the life of the patient. Timely diagnosis can only be done by medical imaging. CT scans and X-Rays both can be used for this purpose. Despite the fact that computed CT scans are more powerful, there is a problem with their high cost and inaccessibility in remote regions. Chest X-ray seems to be a more viable alternative for COVID-19 identification [2, 3, 19]. It has been observed that chest X-ray demonstrate signs and symptoms of COVID-19 inside four days of contamination. Consequently, the use of chest X-rays has the capability to make a distinction for doctors managing the COVID-19 disaster in underdeveloped countries [4]. Chest X-rays of pneumonia and lung cancer patients can also be helpful in diagnosing the disease as gray and white spots are observed on chest X-rays of these patients. Because X-ray scans provide comparable geographical information for other illnesses, such as lung cancer, diagnosing pneumonia by them is still a challenging endeavour. As a result, it takes a lot of time and energy to diagnose pneumonia using conventional techniques, and it is hard to use a consistent procedure to determine whether a patient has pneumonia or not.

In the past decade, deep learning techniques are getting much more effective in different fields of work, especially in the medical field. There are numerous efforts on the segmentation and classification of diseases using artificial intelligence, such as the diagnosis of lung cancer, pneumonia, and brain tumours. These methods were almost implemented in a wearable monitoring system for disease diagnosis, monitoring, and transfer to specific doctors. It is time to use artificial intelligence techniques in medicine to aid in faster and more accurate diagnosis by doctors.

To identify COVID-19, pneumonia, and cancer patients, a deep learning models, CNN (Convolutional Neural Network), and LSTM (Long Short-Term Memory) is proposed. The model is trained on COVID-19, pneumonia, cancer, and normal chest X-rays. This research aims to develop a system for the diagnosis of cancer, Covid-19, and pneumonia using a single deep learning model. Different model architectures are explored for the task that includes: CNN, LSTM and Inception. Three different datasetsets containing Pneumonia [5], COVID-19 [6] and Cancer [7] X-Rays are used for training and testing the models. Various pre-processing and dataset augmentation techniques are also applied to improve the model performance. The experimental findings demonstrates that the suggested strategy outperforms related, current methods. More diseases are classified as compared to the other models, and better accuracy is achieved.

Major Research Contributions:

The contribution of the research article are as follows:

Computer aided diagnosis of three very common lung diseases: Lung cancer, pneumonia and Covid-19 using X-Rays.

Use of data augmentation techniques to achieve better model accuracy.

Design of three Deep neural network architectures specifically for lung diseases.

Detailed analysis of the deep learning lungs disease detection models.

The rest of the paper is organized as follows: section 2 and 3 provide the background and related work, section 3 discusses methodology while section 4 and 5 elaborate results and discussion respectively and finally conclusion section concludes the findings of the current research work.

2 Background

2.1 CNN

There are systems which can detect lungs cancer and there are also systems which can detect COVID and pneumonia but there is no single system which can detect all three diseases. In this research we will implement a system which will detect all three diseases with accuracy. The proposed system will take chest X-Rays as input and by using deep learning technique CNN and LSTM will classify images as COVID, pneumonia, lung cancer or normal with accuracy [8].

2.2 How CNN works

The primary goals of employing “CNN are to perform image recognition and categorization. An image is used as the input for image categorization, in our case chest X-Rays, process it and classify it under certain categories (e.g. COVID-19, Pneumonia, Cancer and Normal). The amount of pixels in an input picture is determined by the image resolution; we will use augmentation to normalize the image scales. Every picture is processed via a series of convolution layers with filters, Pooling, fully connected layers (FC), and Softmax before being used to conduct image classification using the CNN model [9]. It classifies an item using probabilistic values ranging from 0 to 1 after applying these functions.

2.3 Backpropagation

To grasp the principle of backpropagation, you must understand the principles of forward and backpropagation first. Backpropagation is a huge and complex concept, but I will try to elaborate it as easier and possible. You effectively do forward-propagation in neural networks to get the model performance and verify whether this output is accurate or wrong to get the mistake. Backpropagation is nothing about running back to a neural network through hidden layers to input layer to find the partial weight derivatives of the defect. These partial derivatives then help us to go closer to the actual answer by updating the previous weights [10]. These weights and derivatives are then used by gradient descent algorithm to iteratively minimize the error function by adding or subtracting the weights depending on the current scenario. That’s how a neural network learns through backpropagation. So, with backpropagation you attempt to tweak the model’s weights when practicing [11].

2.4 LSTM

We can describe LSTMs as an extension of basic CNN. LSTM stands for Long Short-Term Memory. As the name suggests, the network have two types of internal memories unlike basic CNN. Instead to just one internal memory, LSTMs has two different memories. Through these memories, LSTMs can remember large dependencies of data. Therefore, learning about essential interactions that have lengthy lags in between is well-suited. LSTM modules are used as construction units for CNN layers, also called LSTM networks. LSTMs allow CNNs to recognize long-term inputs. That’s because LSTMs hold memory knowledge, just like machine memory [12].

3 Literature review

Automated detection of different diseases has gained immense popularity in recent years [13, 14]. After Covid-19, the detection of Covid-19 and other lung diseases such as pneumonia and cancer is a difficult task. This section summarizes the previous work related to the detection of lung diseases. The research [13, 15, 16], developed the ACoS system, a customized COVID screening method that uses graded classification standard machine-learning algorithms to distinguish between COVID-19, pneumonia, and normal individuals. The major benefit of the proposed system is that it can be simply displayed with a small number of images and that it can be utilized even with low resources. There are number of techniques which are used to preprocess the dataset”. The preprocessed photos are distributed into two sets: training set and testing set, which accounts for 80 percent of the total, and validation set, which accounts for 20 percent of the total. The image augmentation approach is used on pictures from the training set to create a global model that reduces noise caused by a variety of imaging circumstances.

Another research [17] uses preprocessed images from the put-in network to try to forecast what would happen next. Cohen provided the dataset for this study, which included 225 COVID-19 patients’ chest X-ray scans ranging in age from 14.9 to 58.8. A total of 131 male and 64 female patients are included in the dataset, which covers both male and female X-rays. There are 1583 normal X-rays and 4292 pneumonia patients out of a total of 5875 X-rays.

They combine four alternative network topologies with varying numbers of flexible and fully integrated layers, as well as basic image processing techniques, to assess outcomes using a variety of precision structures and approaches. COVID-19 identification in chest X-ray images is crucial; it helps physicians and patients save time and money by reducing diagnostic time and costs. Images of educational activities can be seen using artificial intelligence and in-depth reading. ConvNets were used in several trials to achieve high COVID-19 accuracy on chest X-ray images, according to the paper. Images and statistical data were utilized to test various image sizes, network architectures, machine learning models, and high-quality pre-trained networks [2, 18, 19].

One of the most deadly illnesses is lung cancer of the lungs which has taken millions of lives. It is estimated in 2018 that around 9.6 million deaths are caused by lung cancer. Cancer is also known as tumor which is made by growth of abnormal cells in lungs. These cells spread really fast because of blood stream. It is seen that due to lymph flow these cells frequently travel to middle of chest. Due to these cells, metastasis occurs. In this paper author explore different techniques and methodologies like CNN, ANN and RNN used in image classification. They draw a comparison according to their accuracy and sensitivity scores [20].

Using a deep learning approach, the authors proposed a cancer detection system from chest X-ray images in this paper. The dataset is used for this proposed system is taken from (Japanese Society of Radiological Technology) as known as JSRT which include 247 chest X-rays images which divided into two categories of cancer one is malignant 100 images and second is benign 54 images and 93 images of normal x-rays [21]. Four steps of data preprocessing applied on this dataset. The first one is histogram equalization for data normalization. In second step noise is removed by standardizing images. In third step image is resized to 224 X 224. In last step color of images is standardized [22].

CNN model is used to train data. The accuracy is achieved with this system us 84.02 percent. In this proposed method authors uses dataset of CT scans. They proposed three steps for filtering dataset. First one is unsharp masking filter. In this step images are enhanced the appearance of small scale images. The second step is OTSU thresholding in this step possible threshold values are decided, for example pixels and background of images. Third step is Adaptive Canny. In this step canny algorithm is used to calculate threshold [23].

The authors of this research propose using a deep learning technique to identify pneumonia on chest x-rays. For this research they used dataset of chest x-rays which contain more than 100,000 chest x-rays with 14 different diseases. They proposed a model ChexNet which contain 121 layer of convolutional neural network [24].

In this paper, a detection model is proposed using CNN. This system uses a Densely Connected Convolutional Neural Network (DenseNet-169). There are three different stages in the proposed architecture. The first one is preprocessing, the second stage is feature extraction and the third stage is classification. In preprocessing stage the noise is removed and images are resized into 224 X 224 pixels to reduce computation timing. In the feature extraction, the DenseNet-169 model is used. Deep Convolutional Network is one of the most useful techniques available for image recognition. There are some different types of convolutional and pooling layers which makes it more reliable. There is an issue with this model which is gradient vanished while passing through different layers [25]. DenseNets fixes the problem of gradient vanishing by balancing load and distributing networks in equal feature sizes. In the classification stage, different classifiers were used to differentiate images. The main classifiers were Support Vector Machine and Random Forest. SVM provides more productive results than Random Forest [26].

In this paper authors proposed a methodology which uses multiple pre-trained models instead of using one. The first two steps are preprocessing and data augmentation after that different models are applied on dataset. The models they uses includes DenseNet121, resNet18, AlexNet, InceptionV3 and GoogLeNet. These models can hold large dataset and the dataset they used is not that large. They added noise to the data so that it can fit in proposed model [27].

Pneumonia is a prominent cause of death worldwide, and due to a shortage of trained radiologists in hospital settings, diagnosing numerous lung illnesses such as atelectasis, cardiomegaly, lung cancer, and others can be challenging [28]. The authors of this study propose using a deep learning-based technique to diagnose pneumonia from chest X-ray images using transfer learning. In this framework, they used a transfer learning approach with pre-trained architectures. The authors of this research developed a CNN-based approach for detecting pneumonia. They used a dataset from Kaggle that had 5786 x-rays that were sorted into three categories: train, test, and Val. Using the recommended technique, they were able to attain an accuracy of 96.39 percent [29].

To diagnose pneumonia from X-ray pictures, researchers used pre-trained deep neural networks as feature extractors in combination with standard classification algorithms. They retrained the pre-trained networks using chest X-ray images and chose the two networks with the greatest accuracy and sensitivity. They showed that utilizing the concatenated properties of these networks as inputs may improve traditional classification algorithms. “Kermany et al. originally provided the dataset that they used to train and evaluate the proposed method. The images in the collection are 2D grayscale chest X-rays with an image size of 1000 X 3500 pixels on average. Each image is separated into two groups: normal and pneumonia. The database is separated into training and testing sets, with 1349 normal and 3883 pneumonia samples in the training set and 234 normal and 390 pneumonia samples in the testing set”. They used the features produced from the chosen deep neural networks as inputs to evaluate the performance of a few conventional classification techniques in order to find a suitable classifier. The accuracy is 97.44 percent, while the sensitivity is 94.02 percent [5].

A deep-CNN-based transfer learning approach for the automated diagnosis of pneumonia and associated categories is presented in this study. There are four different CNN-based deep learning systems for identifying normal, and pneumonia patients were constructed and assessed using chest x-ray. AlexNet and SqueezeNet’s pictures were decreased to 227 X 227 pixels, while ResNet18 and DenseNet201’s images were downsized to 224 X 224 pixels [30]. The pre-trained model criteria were used to normalize all of the photos. As previously stated, CNNs perform better while working with a huge dataset. The functioning database, on the other hand, is not that vast. When training deep learning algorithms, it’s typical to use data augmentation techniques to turn a small dataset into a huge one. Data augmentation has been shown to increase the classification accuracy of deep learning systems. Rather than gathering fresh data, enhancing old data might increase the performance of deep learning models. DenseNet201 beats the other three deep CNN networks, with a 98 percent accuracy for bacterial detection and a 97 percent accuracy for pneumonia detection [31]. A detailed comparison is shown in the Table 1.

Table 1
Comparison Table

Name Disease Methodology Accuracy

“Diagnosis and detection of infected tissue of COVID-19 patients based on lung x-ray image using convolutional neural network approaches [2]” COVID-19 (CNN) 93.2%

“Coronavirus disease (COVID-19) detection in Chest X-Ray images using majority voting based classifier ensemble” [15] COVID-19 Automatic COVID screening (ACoS) 91.3%

“Detection of COVID-19 from Chest X-Ray Images Using Convolutional Neural Networks” [17] COVID-19 (CNN) 94.1%

“Application of deep learning for fast detection of COVID-19 in X-Rays using nCOVnet” [32] COVID-19 nCOVnet 96.1%

“ReCoNet: Multi-level Preprocessing of Chest X-rays for COVID-19 Detection Using Convolutional Neural Networks” [20] COVID-19 ReCoNet 97.4%

“Prediction and Classification of Lung Cancer Using Machine Learning Techniques” [22] Lung Cancer CNN ANN RNN 91.6%

“EARLY LUNG CANCER DETECTION USING MACHINE LEARNING AND IMAGE” PROCESSING [23] Lung Cancer ANN 90.1%

“Lung Cancer Detection System Using Image Processing and Machine Learning Techniques” [33] Lung Cancer ANN 95.2% “Lung Cancer Detection Using Artificial Neural Network” [34] Lung Cancer KNN 96.1%

“Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learning” [26] Pneumonia Detection CNN 92.4%

“Pneumonia Detection Using CNN based Feature Extraction” [27] Pneumonia Detection CNN 80.0%

“Hybrid deep learning for detecting lung diseases from X-ray images” [35] Pneumonia, Atelectasis, Effusion, Infiltration VDSNet 73.0%

“A Novel Transfer Learning Based Approach for Pneumonia Detection in Chest X-ray Images” [29] Pneumonia Detection CNN 96.3%

“Pneumonia Detection from Chest X-ray Images Based on Convolutional Neural Network” [30] Pneumonia Detection CNN 96.2%

“A Deep Transfer Learning Framework for Pneumonia Detection from Chest X-ray Images” [31] Pneumonia Detection CNN 94.0%

“Transfer Learning with Deep Convolutional Neural Network (CNN) for Pneumonia Detection Using Chest X-ray” [36] Pneumonia Detection CNN 97.0%

Name	Disease	Methodology	Accuracy
“Diagnosis and detection of infected tissue of COVID-19 patients based on lung x-ray image using convolutional neural network approaches [2]”	COVID-19	(CNN)	93.2%
“Coronavirus disease (COVID-19) detection in Chest X-Ray images using majority voting based classifier ensemble” [15]	COVID-19	Automatic COVID screening (ACoS)	91.3%
“Detection of COVID-19 from Chest X-Ray Images Using Convolutional Neural Networks” [17]	COVID-19	(CNN)	94.1%
“Application of deep learning for fast detection of COVID-19 in X-Rays using nCOVnet” [32]	COVID-19	nCOVnet	96.1%
“ReCoNet: Multi-level Preprocessing of Chest X-rays for COVID-19 Detection Using Convolutional Neural Networks” [20]	COVID-19	ReCoNet	97.4%
“Prediction and Classification of Lung Cancer Using Machine Learning Techniques” [22]	Lung Cancer	CNN ANN RNN	91.6%
“EARLY LUNG CANCER DETECTION USING MACHINE LEARNING AND IMAGE” PROCESSING [23]	Lung Cancer	ANN	90.1%
“Lung Cancer Detection System Using Image Processing and Machine Learning Techniques” [33] Lung Cancer ANN 95.2% “Lung Cancer Detection Using Artificial Neural Network” [34]	Lung Cancer	KNN	96.1%
“Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learning” [26]	Pneumonia Detection	CNN	92.4%
“Pneumonia Detection Using CNN based Feature Extraction” [27]	Pneumonia Detection	CNN	80.0%
“Hybrid deep learning for detecting lung diseases from X-ray images” [35]	Pneumonia, Atelectasis, Effusion, Infiltration	VDSNet	73.0%
“A Novel Transfer Learning Based Approach for Pneumonia Detection in Chest X-ray Images” [29]	Pneumonia Detection	CNN	96.3%
“Pneumonia Detection from Chest X-ray Images Based on Convolutional Neural Network” [30]	Pneumonia Detection	CNN	96.2%
“A Deep Transfer Learning Framework for Pneumonia Detection from Chest X-ray Images” [31]	Pneumonia Detection	CNN	94.0%
“Transfer Learning with Deep Convolutional Neural Network (CNN) for Pneumonia Detection Using Chest X-ray” [36]	Pneumonia Detection	CNN	97.0%

4 Methodology

For computer-aided diagnosis of multiple lungs diseases, two deep learners Convolutional Neural Network (CNN) and Long Short-Term Memory (LSTM) are trained on lungs X-Rays. The diseases include COVID-19, pneumonia and lung cancer. Three different lungs X-Ray datasets, one for each disease, are used for training and evaluation of the deep classifiers. The main steps of the proposed methodology are illustrated in Fig. 1.

Fig. 1

Major steps of methodology.

4.1 Dataset

Three different benchmark datasets containing lung diseases [7] COVID-19 [6], pneumonia [5], and normal lungs X-rays are merged for training the models. The Nodule Chest X-ray dataset, which originated from Japan around the year 2000, contains cancerous images. The dataset consists of high-quality digital camera scans of Chest X-ray films. Each case in the dataset contains only one nodule, and the nodules have been rated by 20 different radiologists, with Area under the ROC Curve (AUC) scores ranging from 0.72 to 0.89. This dataset is highly suitable for evaluating the performance of nodule detection algorithms across various levels of nodule subtlety. The second dataset is a kind of database which consists of chest X-ray or CT images encompassing COVID-19 cases along with instances of Middle East Respiratory Syndrome (MERS), Severe Acute Respiratory Syndrome (SARS), and Acute Respiratory Distress Syndrome (ARDS), providing a comprehensive collection of respiratory conditions. From this dataset only COVID-19 images are used for research. The third dataset contains bacterial and viral pneumonia images alongwith noraml images. There are 4,337 X-rays of pneumonia, 308 for COVID-19, 154 for cancer and 1676 for normal lungs. Table 2 provides the detail of these datasets. All the X-rays in these four datasets are pre-processed before feeding to the deep learners.

Table 2
Dataset for COVID-19, Cancer and Pneumonia Disease

Type of X-Ray Image Number of X-Ray Images

Cancer [7] 154

COVID-19 [6] 308

Normal [5] 1676

Pneumonia [5] 4337

Type of X-Ray Image	Number of X-Ray Images
Cancer [7]	154
COVID-19 [6]	308
Normal [5]	1676
Pneumonia [5]	4337

Sample images from all the three datasets are illustrated in Figs. 2–4.

Fig. 2

Sample image from dataset [7] containing cancer.

Fig. 3

Sample image from dataset [6] with COVID-19 disease.

Fig. 4

(a) Normal image (b) image with Bacterial Pneumonia (c) Image with Viral Pneumonia from dataset [5].

4.1.1 Training/testing datasets

All lungs X-rays are sectioned into training, testing and validation sets with a ratio of 70% training, 10% validation and remaining 20% testing set. Different pre-processing steps are applied on all the three subsets of lungs X-rays. Data augmentation is applied only on training and validation sets.

4.1.2 Pre-Processing

To eliminate costly calculation by speeding up computing process and getting a good diagnosis accuracy, all the X-ray images are pre-processed and then fed to deep learners. The basic preprocessing steps applied on X-rays are grey scale conversion, rescaling and normalization.

At first, all the X-rays are converted into grey scale. Then, for reducing computational complexity, X-ray images are re-scaled to a size of 150 × 150. After that they are normalized to make them comparable by employing equations (1), (2) and (3). The normalization process maps the intensity values to a range of [0-1]. For standardization standard deviation σ is set as 1 with 0 mean μ. With these values of σ and μ X-rays are mapped to standard Gaussian/Normal distribution.

$μ = \frac{\sum_{i = 1}^{M} \sum_{j = 1}^{N} X_{i, j}}{M \times N}$ (1)

$σ = \sqrt{\frac{1}{M \times N} \sum_{i = 1}^{M} \sum_{j = 1}^{N} {(X_{i, j} - μ)}^{2}}$ (2)

$X_{s} = X - \frac{μ}{σ},$ (3)

where X is image with size M × N. X_s is image after mapping to standard Gaussian, μ is mean while σ is standard deviation.

A sample original image containing cancer, it’s grey-scale image and it’s normalised image are given in Fig. 5.

Fig. 5

(a) Original Image with cancer (b) Grey-scale Image(c) Normalised Image.

4.1.3 Data augmentation

As the three datasets used in this research are from each disease and have a different number of instances hence it raises class imbalance problem. To resolve this issue and to avoid over fitting offline data augmentation is applied which includes translation, rotation, flipping, noise addition, transformations and image enhancement etc. These augmentation techniques can be grouped as geometric transformations,

Geometric Transformation. Such augmentation resolves biases hence four geometrical transformations are applied which are two rotations along with horizontal and vertical flips. For flipping equations (4) and (5) are used.

$X_{h} (i, j) = X (- i, j),$ (4)

$X_{v} (i, j) = X (i, - j),$ (5)

where X is the original x-ray while X_h and X_v are horizontal and vertical reflections.

Once flipped images are attained, two rotations are also applied to generate more images. The x-rays are rotated by angles 90 and 270 for which equations (6) and (7) are used. $I_{θ} = [\begin{matrix} cos θ & - sin θ \\ sin θ & \cos θ \end{matrix}]$ (6) $X_{r} = X I_{θ},$ (7) where X is the original x-ray image, X_r is the rotated one, and XI_θ means rotation matrix

Colour Transformation. To generate more images two colour transformations are applied. One is for converting x-rays into a single channel known as gray-scale image. The other is heat map to capture the magnitude of colour phenomena. The heat map image is again converted into a gray-scale image. The effect of these colour transformations can be seen in Figs. 5 and 6.

Fig. 6

(a) Original Image with cancer (b) Heat map(c) Grey-scale of heatmap.

Image Enhancement. More x-ray images are generated by enhancing them in quality and adjusting hue and gamma values. Gamma (power-law) is applied to x-rays for luminance by employing (8).

$X_{e} = C X^{γ}$ (8)

Where X is the original x-ray image, X_e is the resultant image and C is a constant value which is set as 1 while γ is set as 1.3.

Hue Adjustment: Like luminant, brightness and contrast are also adjusted. the value of hue is displaced by 180 as given in equation (9) which produces more variants of images.

$Hue = 180 \times {\begin{matrix} \frac{G - B}{\max - \min}, & if R = \max \\ \frac{B - R}{\max - \min}, & if G = \max \\ \frac{R - G}{\max - \min}, & if B = \max \end{matrix}$ (9)

Where R, G and B are red, green and blue while max is the maximum value from red, green and blue.

Noise Inclusion. To avoid over-fitting and not letting the deep classifiers memorize training samples, noise is injected and more images are generated. This makes the training process robust for fault tolerance. The added noise is randomly generated form standard Gaussian distribution, given in equation (10).

$X_{noise} = \frac{X}{σ \sqrt{2 π}} e^{- \frac{{(x - μ)}^{2}}{2 σ^{2}}}$ (10)

Where X_noise and X are x-ray images before and after injecting noise, σ and μ are standard deviation and mean.

4.2 Architecture of deep classifiers

Three deep learning-based models, CNN, Inception-V3 and LSTM are trained on multiple lung disease datasets. The models are constructed layer-by-layer and in different combinations. At first, CNN architecture is defined and the model is used to extract features from x-rays and then classify them. After that another models is constructed in which Inception-V3 is used for feature extraction and LSTM is sued for classification. Finally, a third models is constructed on the basis of CNN and LSTM architecture. In this third model, CNN is used for feature extraction and LSTM classifies the lung x-rays.

4.3 CNN

The proposed CNN architecture contains 22 layers in total out of which fiver layers are convolutional ones with number of filters, defined for each layer, are 32, 64, 64, 128, and 256. The size of filters for each layer is described in 8 and complete architecture is shown in Fig. 7. Other than convolutional layers, dbatch normalization, maxPool2D, dropout, flatten and dense layers are also added in the proposed architecture. For training the models uses Adam as an optimizer and Categorical Cross-Entropy is used to calculated the loss of training. For each convolutional layer Rectified Linear Unit (ReLu), given in equation 11, is used as an aviation function while for the last dense layers Sigmoid, given in 12, is used. Total parameters of the CNN are 1,247,654 out of which 1,246,564 parameters are trainable and 1,088 parameters are non-trainable. This model is inspired from VGG-16 that doubles the number of filters in each convolution layer and reduces the length and width by half in each pooling layer. The proposed model is designed to better extract the features from the input image using optimal number of parameters. The Detail of each layer of CNN and their parameters is given in Fig. 8.

Fig. 7

CNN architecture.

Fig. 8

Summary of Convolutional Neural Network (CNN) Architecture.

$ReLu = f (x) = \max (0, x)$ (11) Where $f (x) = {\begin{matrix} 0 & | x < 0 \\ x & | otherwise \end{matrix}$

$M_{t} = β M_{t} - 1 + (1 - β) \nabla_{w} S (W, X, y)$ (12)

4.4 InceptionV3-LSTM

The second deep learner model is designed by combining the CNN flavour InceptionV3 and LSTM. For the computer-aided diagnosis of lung diseases IncpetionV3 extracts features from lungs X-rays which are provided to LSTM for classification. After Inception layer, the proposed architecture contains LSTM layer with 256 filters. LSTM layer is followed by four dense layers and three dropout layers in an alternative vogue. The dropout ratio of the three dropout layers is set as 0.5, 0.2 and 0.3 respectively. InceptionVs-LSTM based model uses Categorical Cross Entropy as a loss functions while Sigmoid is used as an optimizer with learning_rate set as 1e - 4 and momentum value as 0.9. The detail of each layer, their parameters and output shape are given in Fig. 9.

Fig. 9

Summary of InceptionV3-LSTM Architecture.

4.5 CNN-LSTM

The third model is based on CNN and LSTM. For training this deep classifier, CNN is used for feature extraction and LSTM is employed to classify the lung x-rays in COVID-19, pneumonia, cancer and normal classes. The proposed architecture of CNNLSTM is designed with 18 layers in total consisting of 4 convolutional, 3 batch normalization, 3 max pooling, 1 timedistributed, 1 LSTM, 3 dropout and 2 dense layers. The model contains 2,937,796 parameters out of which 2,937,412 are trainable and 384 are non-trainable. Each convolutional layer uses ReLu as an activation function while ReLU and Sigmoid activation functions are uses for the two dense layers respectively. The proposed model uses Adam, equation 13 as an optimizer and Categorical Cross Entropy as a loss function. Figure 10 provides an in depth look into each layer of CNNLSTM deep learners along with filters and output size of each layer.

Fig. 10

Summary of CNN-LSTM Architecture.

$x_{t} = β_{1} \times x_{t} + (1 - β_{1}) \times l_{t}, u_{t} = β_{2} \times u_{t} - 1 + (1 - β_{2}) \times l_{t}^{2} = \max (u_{t} - 1, u_{t})$ (13)

5 Experimental setup and results

Once the experiments are complete, the results of all the models for diagnosing COVID-19, pneumonia and lung cancer using chest X-ray images, are analysed. The training and testing dataset consists of a combination of three benchmark datasets having X-Rays of COVID-19, pneumonia, lung cancer and normal patients. While combining X-Ray images form different sources, data augmentation is applied in order to resolve class imbalance problem. So the results are for three models CNN, Inception V3-LSTM and CNN-LSTM which are trained on augmented X-ray images.

The first experiment is the detection of the lung diseases using CNN with 22 layers. Number of parameters for CNN are 1,247,654, out of these number of trainable parameters is 1,246,564 and number of non-trainable parameters is 1,088. The training, validation and testing accuracy achieved by CNN models is given in Table 5. Out of the three deep learners CNN-based learner got trained with the second best training accuracy i.e., 96.71%. The average test accuracy achieved by this model for all the three diseases is 91.7%.

The second experiment is the detection of the lung disease using a deep learning model that is a combination of InceptionV3 and LSTM. The first layer of this model is InceptionV3 for extracting features from X-Rays. These features are then provided to LSTM with nine layers including LSTM, dense and dropout layers. The dropout parameters are set as 0.5,0.2 and 0.3 for the three layers of dropout wile Sigmoid is used as an activation function. The details of training, validation and test accuracy is shown in Table 5. This model although got well trained by achieving a training accuracy uptill 96.8% however, it’s average test accuracy for all the three diseases is 77.4%. The model performs well during training for validation set but could not classify the unseen data properly during testing.

The third experiment is the detection of the lung diseases using a deep learning model that is a combination of CNN and LSTM with 18 layers. These layers include a combination of convolution, batch normalization, max pooling, time distributed, LSTM, dropout and dense layers. Rectified Linear Unit (ReLu) is used as an activation function in all the convolution layers. For two dense layers ReLu and Sigmoid function is used respectively. Number of parameters for CNN+LSTM model are 2,937,412. The training, validation and testing accuracy achieved by this model is given in Table 5. CNN-LSTM model is the one with the highest training accuracy and the same hold true for testing as well. The model achieves 94.44% average test accuracy for all the three diseases.

After all the experimentation, 91.7% accuracy is achieved when classification model is created using only CNN. The classification model with the combination of InceptionV3 and LSTM provides 77.4% accuracy. The best results are achieved by applying CNN with LSTM for the detection of one of the lung disease (COVID-19, pneumonia and cancer) giving 94.44% of accuracy as shown in Table tab:crossvalidation. The graphical representation of the training, validation and test accuracies for all the three models is shown in Fig. 11. It clearly shows that all the three accuracies are better when CNN-LSTM model is applied for the classification.

Fig. 11

Comparison of three proposed models.

Table 3

Lungs Disease classification using CNN-LSTM

Disease	Accuracy
Pneumonia	97.38%
COVID-19	94.2%
Cancer	96.32%
Normal	89.82%
Avg	94.44%

To reduce the risk of bias in the results, we performed 3-fold, 5 fold and 10 fold cross validation. Results obtained using all three CNN models using k fold validation are shown in the Table 5.

Table 4

Comparison of accuracy of different classifiers using cross validation

Accuracy using CNN
	3-fold	5-fold	10-fold	Average
Training	96.5±0.1	96.4±0.02	97.1±0.03	96.7
Validation	95.5±0.02	95.4±0.01	94.3±0.04	95.1
Testing	91.3±0.05	91.5±0.03	92.3±0.09	91.7
Accuracy using InceptionV3-LSTM
	3-fold	5-fold	10-fold	Average
Training	96.7±0.08	96.8±0.07	97.0±0.08	96.8
Validation	97.5±0.03	97.8±0.07	97.9±0.03	97.7
Testing	77.8±0.02	77.0±0.05	77.5±0.09	77.4
	Accuracy using CNN-LSTM
	3-fold	5-fold	10-fold	Average
Training	98.5±0.3	98.7±0.4	98.8±0.05	98.7
Validation	95.2±0.03	95.7± 0.01	95.8±0.04	95.6
Testing	94.2±0.05	94.5±0.03	95.5±0.06	94.5

Table 5

Comparison with the other State-of-the-Art Models

Disease	Proposed CNN-LSTM	State-of-the-Art
		Models	Accuracy
Pneumonia	97.38%	CNN [17]	97.47%
COVID-19	94.20%	DenseNet [37]	90.5%
Cancer	96.32%	CNN/RNN [22]	91.6%

To evaluate different classes of lungs diseases, confusion matrix was calculated which is shown in Fig. 12, these results are calculated using 5 fold validation.

Fig. 12

Confusion matrix of all classes using CNN-LSTM.

The results of the model with the best diagnosis of lungs diseases are further evaluated. Table 5 provides an insight into disease-wise diagnosis accuracy of CNN-LSTM. The best disease that is diagnosed by this model is Pneumonia which got 97.38% accuracy. Cancer also got diagnosed in a better way by achieving 96.32% accuracy. COVID-19 got diagnosed with an accuracy of 94.2%. Although the accuracy of COVID-19 diagnosis in the lowest among all the three disease but still very near to Cancer disease accuracy and above 90%. The deep learner performed poorly for Normal cases as for they are wrongly diagnosed as having some diseases. Two sample t-test is performed to check the significance of the results obtained by all three models. Using a confidence interval of 95%, it is observed that all results shown in Table 5 are significant and CNN-LSTM performs better than CNN and Inception V3-LSTM (p≤ 0.05)

6 Discussion and conclusion

As per results, the best model is proven to be CNN combined with LSTM in which feature extraction is performed by CNN and LSTM performs classification of COVID-19, pneumonia and cancer. This model is compared with the state-of-the-art techniques that includes covolutional neural network(CNN), DenseNet and CNN/RNN as given in Table 6. The comparison clearly shows that the proposed CNN-LSTM model outperforms the other techniques. The accuracy of lungs multiple disease diagnosis by CNN-LSTM is better than state-of-the art techniques. The state-of-the-art techniques used only single disease hence the dataset used in this research is different from these techniques. The current research utilizes a combination of three datasets hence its not the same. However, COVID-19 chest X-ray dataset is same in this combination of data as well as in the state of the art.

This research explores the deep learning algorithms for the diagnosis of commonly occurring lung diseases that are COVID-19, pneumonia and cancer using X-Ray images. The proposed models are created by merging CNN and LSTM neural networks to obtain much higher performance [38]. The experimentation is made by conducting three experiments including CNN model separately, then combining a flavour of CNN called InceptionV3 with LSTM and thirdly combining CNN with LSTM. The best performance was given by CNN-LSTM model. It has also outperformed other state of the art techniques by providing 94.44% accuracy. The dataset is combined by merging three benchmark datasets. The pre-processing includes data augmentation techniques that has helped in improving the performance of these models. It has been anticipated that this study will contribute to the development of more accurate Covid-19, pneumonia and cancer detection in the future. This approach has the ability to improve visual contrast without generating difficulties like checkerboard effects or washing out the look. Hence the research concludes that a single computer-aided diagnosis system can be developed for diagnosing multiple diseases.

The current research study provides a computer-aided diagnosis system for three lung diseases i.e., cancer, pneumonia and Covid-19.

With the help of data augmentation techniques the accuracy of multiple disease diagnosis via a single system is enhanced.

Three Deep neural network architectures are proposed for lung diseases diagnosis that performs better in terms of accuracy than other state of the art models.

The study also provides an analysis of the deep learning-based lungs disease detection models.

In future, this research can be applied on more data from real world. The accuracy can be further improved by exploring pre-processing techniques for feature selection and by using ensemble based models.

References

Islam

, Islam

and Asraf

A combined deep cnn-lstm network for the detection of novel coronavirus (covid-19) using x-ray images, Informatics in Medicine Unlocked 20 (2020), 100412.

Hassantabar

, Ahmadi

and Sharifi

Diagnosis and detection of infected tissue of covid-19 patients based on lung x-ray image using convolutional neural network approaches, Chaos Solitons Fractals (2020), 110170.

Kumar

and Mahapatra

R.P.

Detection and diagnosis of covid-19 infection in lungs images using deep learning techniques, International Journal of Imaging Systems and Technology 32(2) (2022), 462–475.

Reddy

C. Harinath

, Kumar

B. Koushik

, Varma

N. Sai Teja

, Vidya

, Nagaraj

and Sudar

K. Muthamil

, Risk prediction of lung disease using deep learning approach, in Second International Conference on Image Processing and Capsule Networks: ICIPCN 2021 2. Springer, 2022, pp. 462–471.

“Pneumonia Dataset,” https://www.kaggle.com/datasets/paultimothymooney/chest-xraypneumonia/.

“Covid Dataset,” https://www.kaggle.com/datasets/bachrr/covid-chest-xray/.

“Cancer Dataset,” https://www.kaggle.com/datasets/raddar/nodules-in-chest-xrays-jsrt/.

Fakhfakh

, Bouaziz

, Gargouri

and Chaari

Prognet: Covid-19 prognosis using recurrent and convolutional neural networks, medRxiv, 2020.

Pustokhin

D.A.

, Pustokhina

I.V.

, Dinh

P.N.

, Phan

S.V.

, Nguyenand

G.N.

and Joshi

G.P.

An effective deep residual network based class attention layer with bidirectional lstm for diagnosis and classification of covid-19, Journal of Applied Statistics (2020), 1–18.

10.

Alakus

T.B.

and Turkoglu

Comon of deep learning approaches to predict covid-19 infection, Chaos, Solitons & Fractals 140 (2020), 110120–paris.

11.

Dastider

A.G.

, Sadik

and Fattah

S.A.

An integrated auto encoder-based hybrid cnnlstm model for covid-19 severity prediction from lung ultrasound, Computers in Biology and Medicine 132 (2021), 104296.

12.

Zain

Z.M.

and Alturki

N.M.

Covid-19 pandemic forecasting usingcnn-lstm: a hybrid approach, Journal of Control Science and Engineering 2021 (2021).

13.

Naseer

and Zafar

Comparative analysis of raw images and meta feature based urdu ocr using cnn and lstm, International Journal of Advanced Computer Science and Applications 9(1) (2018), 419–424.

14.

Malik

Y.S.

, Tamoor

, Naseer

Wali

and Khan

, Applying an adaptive otsu-based initialization algorithm to optimize active contour models for skin lesion segmentation, Journal of X-Ray Science and Technology, 30(6) (2022), 1169–1187.

15.

Chandra

T.B.

, Verma

, Singh

, Jain

and Netam

Coronav virus disease (covid- 19) detection in chest x-ray images using majority voting based classifier ensemble, Expert Systems with Applications 165 (2020), 113909.

16.

Naseer

and Zafar

Meta features-based scale invariant ocr decision making using lst mrnn, Computational and Mathematical Organization Theory 25(2) (2019), 165–183.

17.

Islam

K.T.

, Wijewickrema

, Collins

and O’Leary

A deep transfer learning framework for pneumonia detection from chest x-ray images, pp. 286–293, 01 2020.

18.

Naseer

, Tamoor

and Azhar

Computer-aided covid-19 diagnosis and a comparison of deep learners using augmented cxrs, Journal of X-ray Science and Technology, 30(1) (2022), 89–109.

19.

Naseer

, Yasir

, Azhar

, Shakeel

and Zafar

Computer-aided brain tumor diagnosis: performance evaluation of deep learner cnn using augmented brain mri, International Journal of Biomedical Imaging 2021 (2021).

20.

Radhika

, Nair

R.A.

and Veena

A comparative study of lung cancer detection using machine learning algorithms, pp. 1–4, 2019.

21.

Naeem

and Bin-Salem

A.A.

A cnn-lstm network with multi-level feature extraction based approach for automated detection of coronavirus from ct scan and x-ray images, Applied Soft Computing 113 (2021), 107918.

22.

Pawar

Lung cancer detection system using image processing and machine learning techniques, International Journal of Advanced Trends in Computer Science and Engineering 9 (2020), 5956–5963.

23.

Ausawalaithong

, Thirach

, Marukatat

and Wilaiprasitporn

Automatic lung cancer prediction from chest x-ray images using the deep learning approach, pp. 1–5, 11 2018.

24.

Varshni

, Thakral

, Agarwal

, Nijhawan

and Mittal

Pneumonia detection using cnn based feature extraction, in 2019 IEEE international conference on electrical, computer and communication technologies (ICECCT), IEEE, (2019), pp. 1–7.

25.

Barros

, Lacerda

, Albuquerque

and Conci

Pulmonary covid-19: Learning spatio temporal features combining cnn and lstm networks for lung ultrasound video classification, Sensors 21(16) (2021), 5486.

26.

Raoof

, Jabbar

and Fathima

Lung cancer prediction using machine learning: A comprehensive approach, pp. 108–115, 03 2020.

27.

Varshni

, Thakral

, Agarwal

, Nijhawan

and Mittal

Pneumonia detection using cnn based feature extraction, pp. 1–7, 02 2019.

28.

Cohen

J.P.

, Morrison

, Dao

, Roth

, Duong

T.Q.

and Ghassemi

, Covid-19 image data collection: Prospective predictions are the future, arXiv preprint arXiv:2006.11988, 2020.

29.

Chouhan

, Singh

, Khamparia

, Gupta

, Tiwari

, Moreira

Damasevicius

, and Albuquerque

, A novel transfer learning based approach for pneumonia detection in chest x-ray images, Applied Sciences 10 (2020), 559.

30.

Jain

, Nagrath

, Kataria

, Kaushik

, J.D. Pneumonia detection in chest x-ray images using convolutional neural networks and transfer learning, Measurement 165 (2020), 108046.

31.

Rahman

, Chowdhury

, Khandakar

, Islam

Mahbub

, Kadir

, and Kashem

, Transfer learning with deep convolutional neural network (cnn) for pneumonia detection using chest x-ray, Applied Sciences 10 (2020), 3233.

32.

Panwar

, Gupta

, Siddiqui

M.K.

, Morales-Menendez

and Singh

Application of deep learning for fast detection of covid-19 in x-rays using ncovnet, Chaos, Solitons Fractals 138 (2020), 109944. x-ray images,” Informatics in Medicine Unlocked, vol. 20, p. 91.

33.

Al-Rakhami

M.S.

, Islam

M.M.

, Islam

M.Z.

, Asraf

, Sodhro

A.H.

and Ding

Diagnosis of covid-19 from x-rays using combined cnn-rnn architecture with transfer learning, Med Rxiv, pp. 2020–08, 2021.

34.

Nasser

I.M.

and Abu-Naser

S.S.

Lung cancer detection using artificial neural network, International Journal of Engineering and Information Systems (IJEAIS) 3(3) (2019), 17–23.

35.

Bharati

, Podder

and Mondal

M.R.H.

Hybrid deep learning for detecting lung diseases from x-ray images, Informatics in Medicine Unlocked 20 (2020), 100391.

36.

Rahman

, Chowdhury

M.E.

, Khandakar

, Islam

K.R.

, Islam

K.F.

Mahbub

Z.B.

, Kadir

M.A.

and Kashem

, Transfer learning with deep convolutional neural network (cnn) for pneumonia detection using chest x-ray, Applied Sciences 10(9) (2020), 3233.

37.

Mangal

, Kalia

, Rajgopal

, Rangarajan

, Namboodiri

, Banerjee

and Arora

, Covid aid: Covid-19 detection using chest x-ray, arXiv preprint arXiv:2004.09803, 2020.

38.

Demir

Deep coronet: A deep lstm approach for automated detection of covid-19 cases from chest x-ray images, Applied Soft Computing 103 (2021), 107160.

Deep learning classifiers for computer-aided diagnosis of multiple lungs disease

Abstract

Background:

Objective:

Methods:

Results:

Conclusions:

Keywords

1 Introduction

2 Background

2.1 CNN

2.2 How CNN works

2.3 Backpropagation

2.4 LSTM

3 Literature review

Table 2 Dataset for COVID-19, Cancer and Pneumonia Disease Type of X-Ray Image Number of X-Ray Images Cancer [7] 154 COVID-19 [6] 308 Normal [5] 1676 Pneumonia [5] 4337

4.1.2 Pre-Processing

4.3 CNN

References

Table 2
Dataset for COVID-19, Cancer and Pneumonia Disease

Type of X-Ray Image Number of X-Ray Images

Cancer [7] 154

COVID-19 [6] 308

Normal [5] 1676

Pneumonia [5] 4337