DDA-SSNets: Dual decoder attention-based semantic segmentation networks for COVID-19 infection segmentation and classification using chest X-Ray images

Abstract

BACKGROUND:

COVID-19 needs to be diagnosed and staged to be treated accurately. However, prior studies’ diagnostic and staging abilities for COVID-19 infection needed to be improved. Therefore, new deep learning-based approaches are required to aid radiologists in detecting and quantifying COVID-19-related lung infections.

OBJECTIVE:

To develop deep learning-based models to classify and quantify COVID-19-related lung infections.

METHODS:

Initially, Dual Decoder Attention-based Semantic Segmentation Networks (DDA-SSNets) such as Dual Decoder Attention-UNet (DDA-UNet) and Dual Decoder Attention-SegNet (DDA-SegNet) are proposed to facilitate the dual segmentation tasks such as lung lobes and infection segmentation in chest X-ray (CXR) images. The lung lobe and infection segmentations are mapped to grade the severity of COVID-19 infection in both the lungs of CXRs. Later, a Genetic algorithm-based Deep Convolutional Neural Network classifier with the optimum number of layers, namely GADCNet, is proposed to classify the extracted regions of interest (ROI) from the CXR lung lobes into COVID-19 and non-COVID-19.

RESULTS:

The DDA-SegNet shows better segmentation with an average BCSSDC of 99.53% and 99.97% for lung lobes and infection segmentations, respectively, compared with DDA-UNet with an average BCSSDC of 99.14% and 99.92%. The proposed DDA-SegNet with GADCNet classifier offered excellent classification results with an average BCCAC of 99.98%, followed by the GADCNet with DDA-UNet with an average BCCAC of 99.92% after extensive testing and analysis.

CONCLUSIONS:

The results show that the proposed DDA-SegNet has superior performance in the segmentation of lung lobes and COVID-19-infected regions in CXRs, along with improved severity grading compared to the DDA-UNet and improved accuracy of the GADCNet classifier in classifying the CXRs into COVID-19, and non-COVID-19.

Keywords

X-ray images semantic segmentation lung lobe segmentation infection segmentation genetic algorithm

1 Introduction

According to the World Health Organization (WHO), chest diseases have high death rates and cause more than 3 million deaths of individuals only due to chronic obstructive pulmonary disease (COPD) yearly. Nearly 1.5 million deaths are reported yearly due to tuberculosis (TB). Each year, around 1.5 million individuals are predicted to pass away from pneumonia and 1.5 million from lung cancer. The heart and lungs are the organs affected by potentially fatal chest disorders, which are among the most common health problems worldwide [1].

With millions of people impacted worldwide since December 2019, the SARS-CoV-2 coronavirus (COVID-19) has quickly grown into a global epidemic. Because of the COVID-19 outbreak, there is currently a global pandemic. The source of the outbreak, the coronavirus-2 (CoV-2) produces a severe acute respiratory infection that affects the human respiratory system. As of February 04, 2024, the WHO has recorded 774.59 million reported cases worldwide with 7.02 million individual deaths. COVID-19 coexists with other chest illnesses, making the diagnosis more difficult [2, 3]. Figure 1 displays a map of the COVID-19 death rate reported globally.

Fig. 1

COVID-19 death rate Worldwide as on October 21, 2022 (Source: WHO COVID-19 Dashboard).

The gold standard molecular test for detecting COVID-19 is a reverse transcription-polymerase chain reaction (RT-PCR) and is the first kind of important test [4]. Unfortunately, it takes more time and labour [5]. Computed tomography (CT) and chest radiography (CXR) are two examples of radiographic imaging techniques that have become a “successful addition” to RTPCR [6]. CXR screening is one of the most studied radiological imaging modalities due to its minimal radiation dose and ease of use. The majority of CXR images are captured with the patient in front of an X-ray machine, which uses a projection surface for the X-rays generation. Radiologists often use visual assessment, a difficult and error-prone process, to detect diseases seen on chest X-rays (CXRs), including COVID-19. Differentiating between diseases of overlaying organs can be challenging when chest disorders present as opacities around the diseased organ. CXR is helpful for diagnosing a wide variety of heart and lung conditions, including COVID-19. These anomalies could be harmful if identified late in development. The sample CXR images are displayed in Fig. 2 in grayscale.

Fig. 2

Sample CXR images: (a) Non-COVID-19 (b) COVID-19 (c) COVID-19 infected regions.

Currently, experts assess most CXRs visually, requiring a high level of knowledge and a priority on avoiding misdiagnosis of outcomes. The fact is that the CXR symptoms of different diseases might be very similar and could lead to mistakes due to exhaustion and a lack of experience. It is still a difficult and time-consuming process. Diseases like COVID-19 can be detected, and a decision-making tool is provided with the help of computer vision and computer-aided diagnostics systems (CADS).

The issue of misdiagnosis with an elevated error rate, which is a significant threat to radiologists’ employment, has been the subject of numerous researches. Since deep learning (DL) models have successful performance and may be used to create helpful decision-support tools for physicians, they have been widely used in research to reduce the rate of incorrect diagnosis. Two forms of pneumonia, lung opacities, COPD, and various cancers of the lungs, are some of the most common chest diseases [7]. But recently, COVID-19 became a dangerous chest disease. These diseases, which can manifest in various ways throughout the thorax, may be detected by DL models using CXR. This has led to increased research on chest infections, including COVID-19.

The majority of research only focuses on segmenting or classifying CXR images. Therefore there might need to be more information available at this moment to assist physicians. Before diagnosing, physicians first examine CXR images to find any infections and their locations. This shows that segmentation and classification in clinical research have a strong connection [8, 9]. We suggest segmentation and classification-based COVID-19 diagnosis methods to grade the infection for disease management. Due to the few differences across infection classes, the slight textural variations, and the random distribution of infection locations, segmenting COVID-19 infection in CXR images is challenging [10]. Artificial intelligence (AI) is effective at automatically classifying images, as demonstrated by DL methods for disease diagnosis.

Initially, recently developed DL networks for COVID-19 are reviewed in this manuscript, and propose Dual Decoder Attention-based Semantic Segmentation Networks (DDA-SSNets) such as Dual Decoder Attention-UNet (DDA-UNet) and Dual Decoder Attention-SegNet (DDA-SegNet) for COVID-19 infection segmentation and classification using CXRs. The skip connections have AGs to emphasize the features passed between the encoder and decoders. The CXRs are semantically segmented for their lung lobes and infection regions. The dual decoder structure used in the DDA-UNet and DDA-SegNet facilitates the dual segmentations such as lung lobes and infection segmentations in CXRs to grade the severity of infection present due to COVID-19 in both the lungs of CXRs. Further, a Genetic algorithm-based Deep Convolutional Neural Network (Deep CNN) classifier with the optimum number of layers, namely GADCNet, is proposed to classify the extracted regions of interest (ROI) from the CXR lung lobes into COVID-19 and non-COVID-19.

The following is a summary of this work’s main contributions:

A novel dual decoder attention-based semantic segmentation networks (DDA-SSNets) such as dual decoder attention-UNet (DDA-UNet) and dual decoder attention-SegNet (DDA-SegNet) for COVID-19 infection and lung lobes segmentation in COVID-19 and non-COVID-19 CXRs are developed.

The attention mechanism is introduced in the skip connections of DDA-SSNets to emphasize the features passed between the encoder and decoders.

The dual decoder structure used in the DDA-UNet and DDA-SegNet facilitates the dual segmentation tasks such as lung lobes and infection segmentations in CXRs (COVID-19, non-COVID-19 class). Further, the lung lobe and infection segmentations are mapped to grade the severity of COVID-19 in both the lungs of CXRs.

Furthermore, ROI is generated from the lung lobe segmentation outputs of each of the DDA-SSNets

A Genetic algorithm-based Deep CNN classifier with the optimum number of layers, namely GADCNet, is developed to classify the extracted regions of interest (ROI) from the CXR lung lobes into COVID-19 and non-COVID-19.

The rest of the article is organized as follows: Section 2 presents the introduction of relevant studies, Section 3 provides a discussion of materials and methods, Section 4 gives a description of the experiments, Section 5 is the presentation of the results and discussion, and a conclusion is provided in Section 6.

2 Related works

Deep learning (DL) models can learn from and make decisions based on a large amount of data. AI uses sophisticated algorithms to evaluate incoming data to gain information, do computations, and make predictions. We automate the feature extraction and classification procedure in DL, which has proven effective in healthcare [11]. The extensive literature on the numerous approaches used to segment and classify CXR images for COVID-19 is provided in this section.

Daniel Arias-Garzón et al. [12] used VGG19 and UNet models to detect and classify COVID-19 with the best accuracy value of 97% on CXRs. Munawar et al. [13] developed a GAN network with four different discriminators to perform lung segmentation with the best of its dice score of 97.40%. Tulin Ozturk et al. [14] presented a DarkCovidNet with 98.08% accuracy to detect COVID-19. The Deep CNN and the CNN models, which are applied to the fractal features of an image to classify with the highest accuracy of 93.2% (CNN model), are proposed by Shayan Hassantabar et al. [15]. Apostolopoulos et al. [16] developed a MobileNet-v2 for binary classification to detect COVID-19 with 96.78% accuracy. Wang. L et al. [17] implemented a DL network, namely COVID-Net, which uses CXRs to detect COVID-19 with 91.0% accuracy. Štifanić, D. et al. [18] developed lung lobes segmentation networks semantically with a dice value of 92.5%. A. Waheed et al. [19] presented a CovidGAN for an improved COVID-19 classification with 95%. M.E.H.Chowdhury et al. [20] presented transfer learning with CheXNet without data augmentation with an accuracy of 99.41%. P.K.Sethy et al. [21] provided a DL network (VGG16) with an SVM to classify COVID-19 with 97.33% accuracy. EED.Hemdan et al. [22] proposed a COVIDX-Net that uses VGG19 as a backend network to identify COVID-19 with an accuracy of 90%. Narin, A. et al. [23] used a model, namely ResNet101 with 96.1% accuracy for diagnosing COVID-19 in CXRs. Gopatoti, A. et al. [24] developed optimized DL networks to segment CXRs for lung lobes by GWO-SegNet with 98.08% accuracy. Saha.P. et al. [25] implemented an EMCNet for CXR images to identify COVID-19. This network can identify COVID-19 with 98.91% accuracy. M.J.Hasan et al. [26] developed an ensemble model with DenseNet and UNet to perform CXRs segmentation with 99.2% accuracy in 3-class classification. A.Saood et al. [27] presented semantic segmentation by SegNet and UNet on CT scan images and achieved mean accuracy value of 95% in segmenting the COVID-19.

The concept of the case study of various DL networks on COVID-19 CXR and CT has been reported by Muhammad et al. [28]. S. Mishra et al. [29] presented a modified UNet on CT images to detect COVID-19 by segmentation with a dice value of 97.68%. A. Zhang [30] performed lung segmentation with a dice value of 92.73% using CXRs. L.V. De Moura et al. [31] developed an XGBoost classifier with 82% accuracy in classifying the CXRs. JC. Souza et al. [32] proposed automatic lung lobes segmentation in CXRs with a dice value of 93.56%. DL network workflow with the UNet model is presented to perform lung and infection segmentation in COVID-19 CXR images with dice values of 96.2% and 89.4%, respectively by VV.Danilov et al. [33]. S Ahmed et al. [34] proposed an HRNet that uses segmented outputs of the UNet to classify the COVID-19 CXRs with 99.26%. COVID-19 is classified with four class classifiers, with 96.13% proposed by E Khan et al. [35]. R Rajeswari et al. [36] developed clustering to perform lung lobe segmentation with 92.41% accuracy. AK Singh et al. [37] presented an optimization algorithm to classify COVID-19. A.Gopatoti et al. [38] developed a CXGNet to diagnose COVID-19 with 100% accuracy; however this network is not consistent network against all datasets. The severity grading along with COVID-19 diagnosis is presented with an infection detection accuracy of 99.85% and a dice value of 83.20 by A Degerli et al. [39]. The localization of the COVID-19 infection by segmentation and lung lobe segmentations are performed by ED-CNN with dice values of 88.1% and 97.9%, respectively by AM Tahir et al. [40].

The related works mentioned above used CXRs for segmentation and COVID-19 classifications. Many researchers have reported binary-class segmentation and classification of CXR images. Some research regard segmentation as the first step in the classification of COVID-19. Few studies focus on lung lobe segmentation and infection as the first steps in determining COVID-19 severity classification and diagnosis. However, reported works lack infection and lung lobe segmentation in COVID-19 cases to offer severity grading for disease management.

3 Materials and methods

The proposed dual decoder attention-based semantic segmentation networks (DDA-SSNets), such as DDA-UNet, and DDA-SegNet, for COVID-19 infection segmentation and classification using CXRs are discussed in detail in this section, along with the dataset, methodology, and architectural implementation. Figure 3 provides a view of the proposed methodology of this work.

Fig. 3

A view of the proposed methodology of this work.

The methodology of implementing the novel COVID-19 infection segmentation and classification involves four tasks: lung lobe segmentation, infection segmentation, grading, and classification. The novel dual decoder architectures such as DDA-UNet and DDA-SegNet are used to perform lung and infection segmentations. These segmentation outputs are mapped to calculate the percentage of infection severity in the lungs. Further, a GADCNet is used to classify the CXRs into COVID-19 and non-COVID-19.

3.1 Dataset

The COVID-QU-Ex Dataset, which is freely accessible at the Kaggle repository, served as the source of the dataset for this study. [41, 42]. The dataset used in this work contains 5800 CXR images prepared in two classes containing 2900 COVID-19 and 2900 non-COVID-19. The non-COVID-19 CXRs class contains normal, viral, and bacterial pneumonia samples. To undertake segmentation, the CXRs in the dataset contain appropriate ground-truth lung and infection masks. Figure 4 displays some sample CXR images from the dataset utilized in this study.

All the CXRs in the dataset used in this work are resampled to 256×256 pixels. The COVID-19 CXR, along with the associated lung lobes and infection ground-truth images, are shown in the top row of Fig. 4. A non-COVID-19 CXR image, together with the corresponding lung lobes and infection ground-truth images are shown in the bottom row. The dataset utilized in this work is balanced with an equal number of COVID-19 positive and negative samples.

Fig. 4

Sample CXR images of the dataset used in this work.

3.2 Proposed dual decoder attention-based semantic segmentation networks

Radiologists often use manual segmentation assessment to diagnose diseases on chest X-rays (CXRs), including COVID-19. When chest diseases appear as opacities around the diseased organ, underlying organ diseases can be challenging to distinguish. The segmentation of COVID-19 infection in CXRs is a challenging task due to low-class differences, tiny textural variances, and random infection locations. There is heterogeneity in COVID-19 manual and automatic lung lobe segmentations in CXRs. To assess the severity of COVID-19 for disease management purposes, an automated method with high precision for segmenting lung fields and infection regions is required. Therefore we propose dual decoder attention-based semantic segmentation networks (DDA-SSNets) such as dual decoder attention-UNet (DDA-UNet) and dual decoder attention-SegNet (DDA-SegNet) for segmentation and classification of COVID-19 infection using CXRs. The features transferred between the encoder and decoders are emphasized by the AGs on the skip connections. The lung lobes and infection areas of the CXRs are semantically segmented. In order to evaluate COVID-19 infection present in both of the CXRs’ lungs, the dual decoder structure employed in the DDA-UNet and DDA-SegNet makes it easier to perform dual segmentation tasks such as lung lobes and infection segmentations. Additionally, the classification of the extracted regions of interest (ROI) from the CXR lung lobes into COVID-19 and non-COVID-19 is proposed using the genetic algorithm-based Deep CNN classifier called GADCNet, which has the optimal number of layers.

3.2.1 Dual decoder attention-UNet architecture

Building a framework for semantic segmentation requires combining features from subsequent convolutional computations to produce feature maps. The U-Net [43] architecture stands out since it was originally designed as a DL network for the analysis of medical image and segmentation. The encoder’s full feature map is transferred into the decoder’s corresponding layer in UNet. The upsampled feature map and the transferred feature map are combined to produce the final feature map in UNet. Whereas the features transferred between the encoder and decoders of dual decoder attention-UNet (DDA-UNet) are emphasized by the AGs on the skip connections. The AG in DDA-UNet differs from the normal UNet model. Also, the DDA-UNet model has a single encoder and dual decoders. The AGs are present on the skip connections between the single encoder and dual decoders of DDA-UNet. This dual decoder architecture used in DDA-UNet facilitates the dual segmentation tasks such as lung and infection segmentation. The architecture of DDA-UNet is shown in Fig. 5. A single encoder shared by the dual decoders builds the proposed DDA-UNet, a fully connected convolutional neural network. Each encoder and decoder in the DDA-UNet has 4-encoding blocks and 4-decoding blocks, respectively. There is one bottleneck between the encoder and the decoder. The bottleneck layer acts as a feature extractor. It learns to represent the input data in a more abstract and compact form, capturing essential patterns and characteristics necessary for the segmentation task. The encoder network receives the CXR image and encodes it while gradually downsampling it into an abstract feature representation. Both decoders receive the encoder network’s output, followed by a 3×3 upsampling2D convolution layer that doubles the spatial dimensions. Following that, an appropriate feature map from the encoder network is concatenated with the output of the previous decoder blocks by passing through AGs through a skip connection. To increase the strength of their feature representation in decoder blocks, these skip connections retrieve the features from earlier layers through AGs at their original resolution. An attention map of lung lobe segmentation output is produced by the final block of the first decoder network after a 1×1 convolution2D layer and sigmoid activation function. Similarly, an attention map of infection segmentation output is produced by the final block of the second decoder network after a 1×1 convolution2D layer and sigmoid activation function.

Fig. 5

Architecture of DDA-UNet.

Each layer detail used in DDA-UNet is given with a dimension of the layer, size of the filters used, the number of the filters used in each layer, input size, output size, and the parameters in Table 1.

Table 1

Details of each layer in DDA-UNet

Block	Layers	Layer	Filter	No.of	Input	Output	Parameters
		Dimension	Size	Filters	Size	Shape
Encoder Block-1	[Conv.2D+Batch.N+ReLu]+[Conv.2D+Batch.N+ReLu]+M.Pool	256×256	3×3	8	256×256×3	128×128×8	872
Encoder Block-2	[Conv.2D+Batch.N+ReLu]+[Conv.2D+Batch.N+ReLu]+M.Pool	128×128	3×3	16	128×128×8	64×64×16	3616
Encoder Block-3	[Conv.2D+Batch.N+ReLu]+[Conv.2D+Batch.N+ReLu]+M.Pool	64×64	3×3	32	64×64×16	32×32×32	14144
Encoder Block-4	[Conv.2D+Batch.N+ReLu]+[Conv.2D+Batch.N+ReLu]+M.Pool	32×32	3×3	64	32×32×32	16×16×64	55936
Bottleneck-1	[Conv.2D+Batch.N+ReLu]+	16×16	3×3	128	16×16×64	16×16×128	155994
Decoder-1 Decoder-2 Block-5	AG[Up2D+Conv2D+Batch.N+ReLu]+[Conv2D+Batch.N+ReLu]+[Conv2D+Batch.N+ReLu]	32×32	3×3	64	16×16×128	32×32×64	308674
Decoder-1 Decoder-2 Block-6	AG[Up.2D+Conv.2D+Batch.N+ReLu]+[Conv.2D+Batch.N+ReLu]+[Conv.2D+Batch.N+ReLu]	64×64	3×3	32	32×32×64	64×64×32	77538
Decoder-1 Decoder-2 Block-7	AG[Up.2D+Conv.2D+Batch.N+ReLu]+[Conv.2D+Batch.N+ReLu]+[Conv.2D+Batch.N+ReLu]	128×128	3×3	16	646432	128×128×16	19570
Decoder-1 Decoder-2 Block-8	AG[Up.2D+Conv.2D+Batch.N+ReLu]+[Conv.2D+Batch.N+ReLu]+[Conv.2D+Batch.N+ReLu]	256×256	3×3	8	128×128×16	256×256×8	4986
Decoder-1 Decoder-2 Output layer	[Conv.2D+Sigmoid]	256×256	1×1	1	256×256×8	256×256×1	18

3.2.2 Dual decoder attention-SegNet architecture

SegNet is used for pixel-by-pixel semantic segmentation, and it was primarily inspired by scene-understanding applications [44]. SegNet’s decoder network upsamples feature maps with lower resolution as a unique feature using pooling indices from the encoder’s max-pooling. There is no longer a requirement to learn how to use the decoder’s upsampling feature. When training at multiple resolutions, the architecture of SegNet takes advantage of feature maps generated by encoder-decoder pairs. The final output is put through a max pooling operation following each iteration of convolution2D, batch normalization, and ReLu activation in an encoder layer sequence. The next encoder and decoder use the output of the maximum pooling as their input. Decoders and encoders are similar in terms of functionality, but the main difference is that decoders do not add a non-linear effect.

The encoder’s full feature map is transferred into the decoder’s corresponding layer in SegNet. SegNet’s final feature map is built by combining the transmitted and the upsampled feature maps. Whereas the features transferred between the encoder and decoders of dual decoder attention-SegNet (DDA-SegNet) are emphasized by the AGs on the skip connections. The AG in DDA-SegNet differs from the typical SegNet model. Also, the DDA-SegNet model has a single encoder and dual decoders. The AGs are present on the skip connections between the single encoder and dual decoders of DDA-SegNet. This dual decoder architecture used in DDA-SegNet facilitates the dual segmentation tasks such as lung and infection segmentation. The architecture of DDA-SegNet is shown in Fig. 6.

Fig. 6

Architecture of DDA-SegNet.

Using the encoding step indices, the decoders sample the input they receive upstream. The softmax layer receives the output from the final decoder, which creates the desired results. A single encoder shared by the dual decoders allows the proposed DDA-SegNet, a fully connected convolutional neural network. Each decoder in the DDA-SegNet has 6-decoding blocks, while the encoder has 6-encoding blocks. One bottleneck exists between the encoder and the decoding blocks. The encoder network receives the CXR image and encodes it into an abstract feature representation while gradually downsampling it. Both decoders receive the output of the encoder network, which is then followed by a 3×3 max unpooling2D and convolutional layers that double its spatial dimensions. Two signals could be used as input to AGs. The feature map sent over the skip connection is used in the first input. The coarse feature map for the other input is taken from the previous decoder layer’s output. The noisy and irrelevant feature responses extracted from the coarse scale are made sense of via skip connections. The feature maps from the encoder network can be concatenated into the output by connecting the AG’s output to the next decoder via the skip connection. This process is applied to every skip connection. By enhancing feature resolution in this manner, attention weights improve segmentation performance. An attention map of lung lobe segmentation output is produced by the final block of the first decoder network after a 1×1 convolution2D layer and sigmoid activation function. Similarly, an attention map of infection segmentation output is produced by the final block of the second decoder network after a 1×1 convolution2D layer and sigmoid activation function. Each layer detail used in DDA-SegNet is given with a dimension of the layer, size of the filters used, the number of the filters used in each layer, input size, output size, and the parameters in Table 2.

Table 2

Details of each layer in DDA-SegNet

Block	Layers	Layer Dimension	Filter Size	No.of Filters	Input Size	Output Shape	Parameters
Encoder Block-1	[Conv.2D+Batch.N+ReLu]++M.Pool+Argmax2D	256*256	3×3	32	2562563	12812832	10272
Encoder Block-2	[Conv.2D+Batch.N+ReLu]+[Conv.2D+Batch.N+ReLu]+M.Pool+Argmax2D	128*128	3×3	64	12812832	646464	55936
Encoder Block-3	[Conv.2D+Batch.N+ReLu]+[Conv.2D+Batch.N+ReLu]+M.Pool+Argmax2D	64*64	3×3	128	646464	3232128	222464
Encoder Block-4	[Conv.2D+Batch.N+ReLu]+[Conv.2D+Batch.N+ReLu]+M.Pool+Argmax2D	32*32	3×3	256	3232128	1616256	887296
Encoder Block-5	[Conv.2D+Batch.N+ReLu]+[Conv.2D+Batch.N+ReLu]+M.Pool+Argmax2D	16*16	3×3	256	1616256	88256	1182208
Encoder Block-6	[Conv.2D+Batch.N+ReLu]+[Conv.2D+Batch.N+ReLu]+M.Pool+Argmax2D	8*8	3×3	512	88256	44512	3544064
Bottleneck-1	[Conv.2D+Batch.N+ReLu]	4*4	3×3	512	44512	44512	2361856
Decoder-1 Decoder-2 Block-7	AG[Maxunp.2D+Conv.2D+Batch.N+ReLu]+[Conv.2D+Batch.N+ReLu]+[Conv.2D+Batch.N+ReLu]	8*8	3×3	512	44512	88256	12067074
Decoder-1 Decoder-2 Block-6	AG[Maxunp.2D+Conv.2D+Batch.N+ReLu]+[Conv.2D+Batch.N+ReLu]+[Conv.2D+Batch.N+ReLu]	16*16	3×3	256	88256	1616256	3609986
Decoder-1 Decoder-2 Block-9	AG[Maxunp.2D+Conv.2D+Batch.N+ReLu]+[Conv.2D+Batch.N+ReLu]+[Conv.2D+Batch.N+ReLu]	32*32	3×3	128	1616256	3232128	3018882
Decoder-1 Decoder-2 Block-10	AG[Maxunp.2D+Conv.2D+Batch.N+ReLu]+[Conv.2D+Batch.N+ReLu]+[Conv.2D+Batch.N+ReLu]	64*64	3×3	64	3232128	646464	755778
Decoder-1 Decoder-2 Block-11	AG[Maxunp.2D+Conv.2D+Batch.N+ReLu]+[Conv.2D+Batch.N+ReLu]+[Conv.2D+Batch.N+ReLu]	128*128	3×3	32	646464	12812832	189474
Decoder-1 Decoder-2 Block-12	AG[Maxunp.2D+Conv.2D+Batch.N+ReLu]+[Conv.2D+Batch.N+ReLu]+[Conv.2D+Batch.N+ReLu]	256*256	3×3	32	12812832	25625632	56992
Decoder-1 Decoder-2 Output layer	[Conv.2D+Sigmoid]	256*256	1×1	1	25625632	2562561	66

The AG on the skip connections of the proposed DDA-UNet and DDA-SegNet provides a mechanism to emphasize the features passed between the encoder and decoders. By limiting feature activation in unrelated regions, the attention gate directs the model’s focus onto important regions. Due to its lightweight construction, it significantly improves the model’s ability to represent data while requiring little to no additional computation time or model parameters. The typical structure of the attention gate (AG) used in this DDA-UNet and DDA-SegNet is shown in Fig. 7. The spatial regions are selected by AG with the evaluation of the context information and also selects the gating signal (G_S) activation collected from coarser scales. Each pixel’s focus region is determined by the G_S. The attention coefficient C_a of Resampler, which is scaled based on the input feature maps (F_mj) and is the features of the j^th layer. By reducing the noise in irrelevant property responses, the resampler creates discrete activations relevant to the target class.

The AG output is obtained by multiplying the input features F_mj and the attention coefficient C_a element-wise as given in Equation (1). $Attention {Gate}_{output} = F_{mj} \times C_{a}$ (1)

Fig. 7

Typical structure of AG.

The most preferred method for achieving better accuracy is the additive attention mechanism, which yields the attention coefficient C_a and is given in Equation (2). $C_{a} = {AF}_{Sigmoid} (Ø^{T} ({AF}_{ReLu} (W_{F} F_{mj} + W_{G} G_{S} + b_{jG})) + b_{j Ø^{T}})$ (2)

Where, AF_Sigmoid, AF_ReLu are sigmoid, and ReLu activation functions, b_jG, b_jØ^T are values of bias, W_F, W_G are weighted values, and Ø^T is a non-linear function.

3.3 GADCNet classifier

The proposed dual decoder attention-based semantic segmentation networks (DDA-SSNets), such as dual decoder attention-UNet (DDA-UNet) and dual decoder attention-SegNet (DDA-SegNet), can effectively perform lung lobes segmentation and infection segmentation in COVID-19 and non-COVID-19 CXR images. The mapped output is obtained by mapping the segmented lung lobes with the corresponding input CXRs. The multi-texture features (MTFs) are obtained from the mapped output, and the fusion of these features will create ROI [45]. The generation of the ROI is shown in Fig. 8.

We propose a genetic algorithm (GA)-based Deep CNN classifier with the optimum number of layers, namely GADCNet, to classify the CXRs into COVID-19 and non-COVID-19 utilizing generated ROIs. The GADCNet classifier is built using the VGG16. The GA is used here to optimize the number of layers and the kernel used in each layer. The GA is applied to optimize the classifier model architecture [46-47]. This algorithm optimizes the number of layers and kernels used in each classifier layer. The flow chart of the genetic algorithm is shown in Fig. 9. Finding the optimal solution by the GA involves: population initialization, evaluating fitness, selection, crossover and mutation, offspring into the population, Convergence to know the criteria fulfillment, and optimal solution finding.

Fig. 8

Generation of ROI.

Fig. 9

Flow chart of the genetic algorithm.

To classify the ROI generated by the segmentation networks, the genetic algorithm (GA) begins to work and, through a series of evolutionary processes, develops the best architecture of the GADCNet classifier. During evolution, a population is initialized randomly with a predetermined population size, and the counter is initialized to zero for the current generation. The dataset evaluates each individual’s fitness and encodes a particular deep CNN classifier architecture. A new offspring is then produced using genetic operators, such as the crossover and mutation operators, once the parent individuals have been chosen based on fitness. Then, from the current population, the environmental selection selects a population of individuals who will survive into the next generation. The parent population and the offspring population produced by that population make the current population, specifically. The evolution then proceeds until the counter reaches the maximum generation previously determined, at which point the counter is incremented by one. The Genetic algorithm to develop the GADCNet classifier is given in Table 3. The architecture of the GADCNet classifier used in this work is shown in the Fig. 10. The classifier contains a total of 8 blocks with a softmax layer. It contains 11 convolutional layers with ReLu activation, one flattened layer, and two dense layers with softmax. Each layer is selected with an optimal number of filters and neurons by the genetic algorithm. Each layer detail used in GADCNet classifier selected by GA is given with a dimension of the layer, size of the filters used, the number of the filters used in each layer, input size, output size, and the parameters in Table 4.

Table 3

Genetic algorithm to develop GADCNet

GADCNet Classifier
Input: Defined Blocks, size of the population, maximum generations (m_g), maximum filters, number of parents and children, and the dataset
Output: Optimal architecture of classifier
1	P₀← initialized with the size of the population
2	Counter c← initialize to zero for the current generation
3	If c < m_g (maximum generations): evaluate the fitness of each individual in the population at ‘c’ (P_c)
4	Choose parent individuals based on fitness and produce offspring Q_c
5	Next generation P_c+1 ← P_c ∪ Q_c from the population of individuals who will survive into the next generation [environmental selection]
6	c ← c + 1
7	Evolution then proceeds until the counter reaches the maximum generation
8	Return best fitness of individuals in P_c

Fig. 10

Architecture of GADCNet classifier.

Table 4

Details of each layer in the GADCNet classifier

Layers	Layer Dimension	Filter Size	No.of Filters	Input Size	Output Shape	Parameters
Conv.2D+ReLu	256*256	3×3	64	2562563	25625664	1792
Conv.2D+ReLu	256*256	3×3	67	25625664	25625667	38659
M.Pool(2×2 Pool size)	128*128			25625667	12812867	0
Conv.2D+ReLu	128*128	3×3	98	12812867	12812898	59192
Conv.2D+ReLu	128*128	3×3	107	12812898	128128107	94481
M.Pool(2×2 Pool size)	64*64			128128107	6464107	0
Conv.2D+ReLu	64*64	3×3	243	6464107	6464243	234252
Conv.2D+ReLu	64*64	3×3	316	6464243	6464316	691408
M.Pool(2×2 Pool size)	32*32			6464316	3232316	0
Conv.2D+ReLu	32*32	3×3	389	3232316	3232389	1106705
Conv.2D+ReLu	32*32	3×3	419	3232389	3232419	1467338
Conv.2D+ReLu	32*32	3×3	452	3232419	3232452	1704944
M.Pool(2×2 Pool size)	16*16			3232452	1616452	0
Conv.2D+ReLu	16*16	3×3	486	1616452	1616486	1977534
Conv.2D+ReLu	16*16	3×3	507	1616486	1616507	2218125
M.Pool(2×2 Pool size)	8*8			1616507	88507	0
Flatten Layer				88507	32448	0
Dense+ReLu				32448 Neurons	1241 Neurons	40269209
Dense+SoftMax				1241 Neurons	2 Neurons	2484

4 Experiments

The experiments were conducted on the dataset prepared with CXRs of two classes (COVID-19 and non-COVID-19) for the dual segmentation tasks such as lung lobes and infection segmentation with proposed dual decoder attention-based semantic segmentation networks such as DDA-UNet and DDA-SegNet. The mapping of the lung lobes and infected regions involves statistical data for severity grading. By segmenting lung lobes and infection areas separately, disease analysis becomes more localized and particular. The GADCNet classifier performance is evaluated on the CXR images to identify COVID-19 and non-COVID-19. Investigations on the proposed networks involve a deep learning framework implemented with Python-3.7 using TensorFlow-2.0 on a graphics processing unit (GPU) machine with 64 GB Nvidia RAM and windows-10Pro of version 20H2.

4.1 Training and testing

The dataset used in this work contains 5800 CXR images prepared in two classes containing 2900 COVID-19 and 2900 non-COVID-19. The non-COVID-19 CXRs class contains normal, viral, and bacterial pneumonia samples. During training, 80% of the dataset had 4640 CXR samples utilized. The remaining dataset is a test dataset (20%) and is used in testing the performance of the proposed DDA-SSNets. All the CXR samples are resampled to 256×256 pixels. The CXRs in the dataset is accompanied by a ground-truth lung and infection mask to facilitate segmentation. The ADAM optimizer trains all the hyperparameters to the proposed DDA-SSNets. The learning rate of 0.001 with a batch size of 8 and 22 epochs is used in both DDA-UNet and DDA-SegNet. The GADCNet classifier utilizes a 0.0001 learning rate, batch size of 32, and epochs of 30 during the training. The populations and generations are set at 10 in the genetic algorithm with the value of 5 as parent and child. Block-1 in the GADCNet classifier (Conv.2D+Conv.2D+M.Pool) is set with the maximum number of filters at 65. The block-2 in GADCNet classifier (Conv.2D+Conv.2D+M.Pool) is set with the maximum number of filters at 129. Block-3 in the GADCNet classifier (Conv.2D+Conv.2D+M.Pool) is set with the maximum number of filters at 257. Block-4 in the GADCNet classifier (Conv.2D+Conv.2D+Conv.2D+M.Pool) is set with the maximum number of filters at 513. The block-5 in GADCNet classifier (Conv.2D+Conv.2D+M.Pool) is set with the maximum number of filters at 513. Block-6 in the GADCNet classifier (Dense layer) is set with the maximum number of neurons at 4097. Overfitting is a critical consideration in any data-driven study, and to mitigate its potential effects on the proposed model’s performance, we carefully monitored the complexity of our model architecture to strike a balance between capturing relevant patterns in the data and avoiding excessive complexity that could lead to overfitting. A penalty term was added to the loss function based on the L2 norm of the weights.

4.2 Metrics for performance evaluation

The statistical performance measures are evaluated for the proposed semantic segmentation networks such as DDA-UNet, DDA-SegNet, and the GADCNet classifier to know their dual segmentation, grading, and classification capability on the test dataset. The statistical measures evaluated in this work were binary class semantic segmentation dice coefficient (BCSSDC), binary class semantic segmentation accuracy (BCSSAC), binary class classification accuracy (BCCAC), binary class classification sensitivity (BCCSEN), binary class classification specificity (BCCSPE), binary class classification precision (BCCPRE), and binary class classification F₁(BCCF₁) score. The formulas for evaluating these measures are in Equations (3)–(9). $BCSSDC = \frac{2 TPV}{2 TPV + FPV + FNV}$ (3) $BCSSAC = \frac{TPV + TNV}{TPV + FPV + TNV + FNV}$ (4) $BCCAC = \frac{TPV + TNV}{TPV + FPV + TNV + FNV}$ (5) $BCCSEN = \frac{TPV}{TPV + FNV}$ (6) $BCCSPE = \frac{TNV}{TNV + FPV}$ (7) $BCCPRE = \frac{TPV}{TPV + FPV}$ (8) ${BCCF}_{1} = 2 * (\frac{BCCSEN * BCCPRE}{BCCSEN + BCCPRE})$ (9)

The number of correct predictions of the actual class label of COVID-19 and non-COVID-19 indicates the “true positive value (TPV)” and true negative value (TNV) respectively. The number of wrong predictions of the class label of the non-COVID-19 as COVID-19 indicates the “false positive value (FPV)”. The number of incorrect predictions of the class label of COVID-19 as non-COVID-19 indicates the “false negative value (FNV)”.

5 Results and discussion

The experimental outcomes from the proposed dual decoder attention-based semantic segmentation networks (DDA-SSNets), such as dual decoder attention-UNet (DDA-UNet) and dual decoder attention-SegNet (DDA-SegNet), for the segmentation and classification of COVID-19 infection using CXRs are presented in this section. For each dual decoder segmentation network, the qualitative visual finding and quantitative parameters were presented, along with a comprehensive list of comparative evaluations between COVID-19 and non-COVID-19 CXRs, ground-truth lung masks, infection masks, lung lobes segmentation, and infection segmentation. A qualitative and quantitative analysis of the GADCNet classifier’s performance, categorizing the CXR images into COVID-19 and non-COVID-19, was presented. The higher results produced by the networks of this work is further demonstrated by comparing the proposed GADCNet classifier performance with that of other cutting-edge networks.

5.1 Dual segmentation results

The proposed dual decoder attention-based semantic segmentation networks (DDA-SSNets), such as DDA-UNet and DDA-SegNet, are primarily designed to perform dual segmentation tasks by which the CXRs are classified into COVID-19 and non-COVID-19 along with infection severity grading for disease management. The dual segmentation is accomplished here with lung lobes and infection segmentation at the pixel level. The segmented lung lobes and the infection are binary with white and black regions. The white region represents segmented lung lobes and the infected regions, whereas the black indicates the rest of the regions in the X-ray. The severity grading of the COVID-19 infection is determined by superimposing the proposed networks’ dual segmentation output. The severity grading values of the COVID-19 infection present in the COVID-19 and non-COVID-19 CXRs are given with three sub-categories: lungs, left lung, and right lung infection. Three severity grading is determined with the counting of the pixels. The COVID-19 infection in both lungs is graded by the number of infected pixels divided by the lung lobe pixels. The COVID-19 infection severity grading in the left lung is the number of left-infected pixels divided by the left lung lobe’s pixels. The severity of right lung COVID-19 infection is the number of right infected pixels divided by the right lung lobe’s pixels. The dual segmentation performance of the proposed dual decoder attention-UNet (DDA-UNet) is measured with the statistical metrics for COVID-19, and non-COVID-19 is given in Table 5.

Table 5
Dual segmentation performance of the proposed DDA-UNet

Dual segmentation network X-ray class Performance measures (%)

Lung segmentation Infection segmentation

BCSSDC BCSSAC BCSSDC BCSSAC

DDA-UNet COVID-19 99.41 99.10 99.95 98.18

Non-COVID-19 98.87 99.20 99.89 98.20

Average 99.14 99.15 99.92 98.19

Dual segmentation network	X-ray class	Performance measures (%)
DDA-UNet	COVID-19	99.41	99.10	99.95	98.18
	Non-COVID-19	98.87	99.20	99.89	98.20
Average		99.14	99.15	99.92	98.19

The dual segmentation performance of the proposed DDA-UNet is evaluated for each X-ray class with statistical measures for lung and infection segmentation. The DDA-UNet dual segmentation network shows better segmentation with the average BCSSDC of 99.14% for lung lobes and 99.15% of average BCSSAC of all the X-ray classes of the dataset. Also, the proposed DDA-UNet dual segmentation network shows better infection segmentation with the average BCSSDC of 99.92% and 98.19% of average BCSSAC of all the X-ray classes of the dataset. The qualitative visual dual segmentation performances of the proposed DDA-UNet are shown in Fig. 11.

Fig. 11

Qualitative visual dual segmentation performances of DDA-UNet.

The qualitative visual dual segmentation results of the proposed DDA-UNet show that the DDA-UNet can reliably segment the lung lobes and the COVID-19 infection present in the COVID-19 and non-COVID-19 CXRs. The lung lobe and COVID-19 infection segmentations done by the DDA-UNet are compared to the ground-truth lung lobe and COVID-19 infection segmentations. The dual segmentation results of DDA-UNet are further used to obtain the severity grading values of the COVID-19 infection. The COVID-19 infection severity grading is provided with three grades for each sample CXR image. For the first COVID-19 CXR sample (column-1) used in Fig. 11, the COVID-19 infection severity grading is found as 39.83% in the lungs, 39.78% in the left lung, and 39.88% in the right lung. For the second COVID-19 CXR sample (column-2) used in Fig. 11, the COVID-19 infection severity grading is found as 22.82% in the lungs, 20.86% in the left lung, and 24.79% in the right lung. For the third and fourth non-COVID-19 CXR samples (column-3 and column-4) used in Fig. 11, the COVID-19 infection severity grading is found as 0% in the lungs, left lung, and right lung. By maintaining edge features, the proposed DDA-UNet has high confidence in segmenting the lung lobes and COVID-19-infected regions in each CXR class. Further, we proposed DDA-SegNet to improve the dual segmentation task for improved severity grading of the COVID-19 infection in CXRs. The dual segmentation performance of the proposed dual decoder attention-SegNet (DDA-SegNet) is measured with the statistical metrics for COVID-19 and non-COVID-19 and is given in Table 6.

Table 6

Dual segmentation performance of the proposed DDA-SegNet

Dual segmentation network	X-ray class	Performance measures (%)
		Lung segmentation		Infection segmentation
		BCSSDC	BCSSAC	BCSSDC	BCSSAC
DDA-SegNet	COVID-19	99.87	99.28	99.98	98.28
	Non-COVID-19	99.19	99.60	99.96	98.24
Average		99.53	99.44	99.97	98.26

The dual segmentation performance of the proposed DDA-SegNet is evaluated for each X-ray class with statistical measures for lung and infection segmentation. The DDA-SegNet dual segmentation network shows better segmentation with the average BCSSDC of 99.53% for lung lobes and 99.44% of average BCSSAC of all the X-ray classes of the dataset. Also, the proposed DDA-SegNet dual segmentation network shows better infection segmentation with the average BCSSDC of 99.97% and 98.26% of average BCSSAC of all the X-ray classes of the dataset. The qualitative visual dual segmentation performances of the proposed DDA-SegNet are shown in Fig. 12.

Fig. 12

Qualitative visual dual segmentation performances of DDA-SegNet.

The qualitative visual dual segmentation results of the proposed DDA-SegNet shows that the DDA-SegNet can reliably segment the lung lobes and the COVID-19 infection present in the COVID-19 and non-COVID-19 CXRs compared to DDA-UNet. The lung lobe and COVID-19 infection segmentations done by the DDA-SegNet are compared to the ground-truth lung lobe and COVID-19 infection segmentations. The dual segmentation results of DDA-SegNet are further used to obtain the severity grading values of the COVID-19 infection. The COVID-19 infection severity grading is given three grades for each sample CXR image. For the first COVID-19 CXR sample (column-1) used in Fig. 12, the COVID-19 infection severity grading is found as 68.21% in the lungs, 78.99% in the left lung, and 46.01% in the right lung. For the second COVID-19 CXR sample (column-2) used in Fig. 12, the COVID-19 infection severity grading is found as 39.64% in the lungs, 45.36% in the left lung, and 32.05% in the right lung. For the third and fourth non-COVID-19 CXR samples (column-3 and column-4) used in Fig. 12, the COVID-19 infection severity grading is found as 0% in the lungs, left lung, and right lung. By maintaining edge features, the proposed DDA-SegNet has more high degree of confidence in segmenting the lung lobes and COVID-19-infected regions in each CXR class compared to the DDA-UNet. Following the completion of the dual segmentation tasks, it can be seen that the performance of the DDA-UNet and DDA-SegNet proposed in this study are comparable to that of the works described in the literature for segmenting COVID-19 infection, and other lung lobes. As the initial steps in defining COVID-19 severity grading and diagnosis, only a few studies in the literature concentrate on lung lobe segmentation and infection segmentation. However, reported studies do not segment lung lobes and COVID-19 infection for COVID-19 cases to provide disease severity grading. As a result, the proposed study’s conclusions are reliable, accurate, and comparable with the related research works. The proposed dual segmentation networks, such as DDA-UNet and DDA-SegNet, are compared with the recent state-of-art works in Table 7.

Table 7

Comparison of proposed dual segmentation networks with recent state-of-art works

Network	Performance measures (%)
	Lung Segmentation		Infection Segmentation
	BCSSDC	BCSSAC	BCSSDC	BCSSAC
DCNN [15]	81.63	82.39	79.61	80.50
SegNet [24]	96.81	93.85	95.67	91.80
U-Net [30]	92.73	95.75	92.57	89.69
MTL-CNN [33]	97.20	96.85	89.40	87.95
ResNet50 [40]	97.95	99.10	87.78	97.93
DenseNet121 [40]	97.98	99.11	86.66	97.97
Proposed dual segmentation networks:
DDA-UNet	99.14	99.15	99.92	98.19
DDA-SegNet	99.53	99.44	99.97	98.26

As given in Table 7, most recent studies have yet to report how their networks can perform COVID-19 infection segmentation. They are performing a single segmentation task, lung lobes segmentation. Few works report dual segmentation tasks such as lung lobes segmentation and COVID-19 infection segmentation. The proposed dual segmentation models, such as DDA-UNet and DDA-SegNet, show better segmentation capabilities than the recent state-of-the-art studies presented in the comparison. Due to their superior performance, the proposed dual segmentation networks, such as DDA-UNet and DDA-SegNet, are recommended for grading the severity of COVID-19 infection and classification.

5.2 Classification results

To classify the CXRs into COVID-19 and non-COVID-19 with the segmented outputs of the DDA-UNet and DDA-SegNet by the proposed GADCNet classifier begin with the generating ROI. The mapped output is obtained by mapping the segmented lung lobes with the corresponding input CXRs. The MTFs are obtained from the mapped output, and the fusion of these features will create ROI. The feature extraction is performed to obtain the MTFs using the eight significant local features concerning the texture, namely Local Optimal Oriented Pattern (LOOP), Local Binary Pattern (LBP), Local Directional Pattern (LDP), Local Ternary Pattern (LTP), Local Directional Ternary Pattern (LDTP), Harmonic Mean Local Gradient Pattern (HLGP), Local Gradient Hexa Pattern (LGHP), and Local Gradient Pattern (LGP). After that, COVID-19 classification is accomplished by utilizing the ROIs. The MTFs derived from the mapped output are visually diverse, as the inputs CXRs were also visually diverse. The MLTFs derived from the COVID-19 CXRs are utilized to classify the CXRs into COVID-19. The MTFs derived from each class are utilized to classify the corresponding class. In contrast to approaches that extract textual features independently, research showed that fusing multi-textural features improved network recognition. The effectiveness or performance of the individual feature extraction techniques and the proposed ROI by fusion of MTFs derived from the mapped output is presented in Table 8.

Table 8
Performance of individual feature extraction techniques

Metrics (%) Feature extraction methods

LOOP LBP LDP LTP LDTP HLGP LGHP LGP ROI by Fusion of MTFs

BCCAC 89.48 86.21 76.31 79.65 84.36 84.81 77.26 86.75 99.98

BCCSEN 87.16 81.85 71.24 74.55 80.29 81.47 72.22 83.83 99.96

BCCSPE 94.03 92.32 90.56 91.04 93.83 94.83 92.68 94.36 99.94

BCCPRE 91.51 89.44 80.32 83.24 87.78 87.15 81.57 89.11 99.93

BCCF₁ 89.24 85.38 74.17 78.53 83.76 84.04 76.46 86.32 99.97

Metrics (%)	Feature extraction methods
BCCAC	89.48	86.21	76.31	79.65	84.36	84.81	77.26	86.75	99.98
BCCSEN	87.16	81.85	71.24	74.55	80.29	81.47	72.22	83.83	99.96
BCCSPE	94.03	92.32	90.56	91.04	93.83	94.83	92.68	94.36	99.94
BCCPRE	91.51	89.44	80.32	83.24	87.78	87.15	81.57	89.11	99.93
BCCF₁	89.24	85.38	74.17	78.53	83.76	84.04	76.46	86.32	99.97

The ROI contains only lung lobe fields generated by the fusion of the multi-texture features, as shown in Fig. 13.

Fig. 13

ROI generation from the multi-texture features.

The generated ROIs from each dual segmentation network are trained to the proposed GADCNet classifier. The GADCNet classifier has an optimized structure with an optimal number of layers and is developed using a genetic algorithm. The performance evaluation of the GADCNet classifier in line with the proposed dual segmentation networks is evaluated to know the classifier’s ability. The GADCNet classifier performance with the DDA-UNet dual segmentation network is given in Table 9.

Table 9

Performance of GADCNet classifier with DDA-UNet

Proposed classifier with segmentation	X-ray class	Performance measures (%)
		BCCAC	BCCSEN	BCCSPE	BCCPRE	BCCF₁
DDA-UNet+GADCNet classifier	COVID-19	99.93	99.86	100	99.82	99.93
	Non-COVID-19	99.92	100	99.85	100	99.91
Average		99.92	99.93	99.92	99.91	99.96

The DDA-UNet dual segmentation network lung lobes output was utilized to evaluate the GADCNet classifier’s classification performance for each X-ray class. The proposed GADCNet classifier with DDA-UNet achieved an average BCCAC of 99.92%, average BCCSEN of 99.93%, average BCCSPE of 99.92%, average BCCPRE of 99.91%, and average BCCF₁ of 99.96% for both the CXR classes (COVID-19, non-COVID-19). More particularly, the proposed GADCNet classifier with DDA-UNet dual segmentation network shows 99.93% BCCAC in detecting COVID-19 in the CXRs. The GADCNet classifier performance with the DDA-SegNet dual segmentation network is given in Table 10.

Table 10

Performance of GADCNet classifier with DDA-SegNet

Proposed classifier with segmentation	X-ray class	Performance measures (%)
		BCCAC	BCCSEN	BCCSPE	BCCPRE	BCCF₁
DDA-SegNet+GADCNet classifier	COVID-19	99.99	99.92	100	99.87	99.96
	Non-COVID-19	99.97	100	99.89	100	99.98
Average		99.98	99.96	99.94	99.93	99.97

Further, the DDA-SegNet dual segmentation network lung lobes output was also utilized to evaluate the GADCNet classifier’s classification performance for each X-ray class. The proposed GADCNet classifier with DDA-SegNet achieved an average BCCAC of 99.98%, average BCCSEN of 99.96%, average BCCSPE of 99.94%, average BCCPRE of 99.93%, and average BCCF₁ of 99.97% for both the CXR classes (COVID-19, non-COVID-19). More particularly, the proposed GADCNet classifier with DDA-SegNet dual segmentation network shows 99.99% BCCAC in detecting COVID-19 in the CXRs. The quantitative analysis of the proposed classifier shows that the dual segmentation networks improve the performance in classifying the CXRs into COVID-19 and non-COVID-19. Also, it was observed that both the dual segmentation networks improved GADCNet classifier performance to above 99% accuracy. Further, it was observed that there is a slight classification improvement in the GADCNet classifier with DDA-SegNet compared to the DDA-UNet.

In this work, two dual segmentation networks, DDA-UNet and DDA-SegNet, are presented with the GADCNet classifier to perform tasks such as lung lobes segmentation, infection segmentations, severity grading of the COVID-19 disease, and CXRs classification into COVID-19 and non-COVID-19. The CXRs are classified into two classes by the proposed classification network with the segmentation output of proposed dual segmentation networks. The classification performance of the proposed GADCNet classifier with DDA-UNet and DDA-SegNet is compared with the current research works in Table 11.

Table 11

Comparison of current research works with the proposed classification models

Reference		Evaluation metrics (%)
		BCCAC	BCCSEN	BCCSPE	BCCPRE	BCCF₁
DarkCovidNet [14]		98.08	95.13	95.30	98.03	96.51
CovidGAN [19]		95.00	90.00	97.00	95.00	95.00
DenseNet201 [20]		99.70	99.70	99.55	99.70	99.70
COVIDX-Net [22]		90.00	90.00	89.50	91.50	90.00
ResNet101 [23]		96.10	78.30	98.20	84.20	81.20
EMCNet [25]		98.91	97.82	96.89	100	98.89
IWS-based DeepNet [45]		91.87	93.34	88.79	89.53	91.39
This work	DDA-UNet+GADCNet Classifier	99.92	99.93	99.92	99.91	99.96
	DDA-SegNet+GADCNet Classifier	99.98	99.96	99.94	99.93	99.97

As given in Table 11, the classification results obtained by the GADCNet classifier with DDA-SegNet showed better classification results with an average BCCAC of 99.98%, followed by the GADCNet classifier with DDA-UNet with an average BCCAC of 99.92% compared with the recently reported state of art networks such as DarkCovidNet, CovidGAN, DenseNet201, COVIDX-Net, ResNet101, EMCNet, and IWS-based DeepNet. Based on the comparative analysis, we refer to the use of dual decoder attention-based semantic segmentation networks (DDA-SSNets) such as DDA-UNet and DDA-SegNet along with GADCNet classifier to classify the CXRs into COVID-19 and non-COVID-19. Also, we recommend using the proposed DDA-SSNets to know the severity grading of COVID-19 infection using CXRs. The results of the dual decoder attention-based semantic segmentation networks (DDA-SSNets) proposed for COVID-19 infection segmentation and classification using CXRs are promising and encouraging, indicating that the battle against the COVID-19 pandemic will probably involve deep learning more significantly. The deep models proposed in this work and their results will serve as a foundation for developing a method that uses the dual segmentation concept to grade the severity of COVID-19 disease. The COVID-19 characteristics may change over time. The models that are proposed may be trained on data from a certain time period, and their performance could be affected by viral replication and changes.

6 Conclusion

Extreme suffering is being caused all across the world at the moment now due to the COVID-19 pandemic. Inadequate treatment, a lack of resources, and a late diagnosis have already lost many people’s lives. The proposed DDA-SSNets can help infected patients by automatically detecting COVID-19 from CXRs and grading the severity of the infection. The proposed DDA-SSNets, such as DDA-UNet and DDA-SegNet, are primarily designed to perform dual segmentation tasks (lung lobes, infection) by which the CXRs are classified into COVID-19 and non-COVID-19 along with infection severity grading for disease management. The dual segmentation is accomplished here with lung lobes and infection segmentation at the pixel level. Using a dataset of CXRs, the proposed DDA-SegNet with GADCNet classifier offered the excellent classification results with an average BCCAC of 99.98%, followed by the GADCNet classifier with DDA-UNet, which achieved an average BCCAC of 99.92% after undergoing extensive testing and analysis. The proposed DDA-SSNets play a vital role in the severity grading of the COVID-19 infection to help doctors in early COVID-19 detection and treatment management. Therefore, we recommend dual decoder attention-based semantic segmentation networks for COVID-19 infection segmentation and classification using chest X-ray images.

Funding

The authors report no funding.

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this manuscript.

Compliance with ethical standards

No studies involving humans or animals have been reported by any of the authors in this manuscript.

References

Adnane Ait Nasser , Moulay Akhloufi

, Chest diseases classification using CXR and deep ensemble learning, In International Conference on Content-Based Multimedia Indexing, pp. 116–120. 2022.https://doi.org/10.1145/3549555.3549581.

Xiaoqin Zhang , Runhua Jiang , Pengcheng Huang , Tao Wang , Mingjun Hu Andrew Scarsbrook

, Alejandro Frangi

, Dynamic feature learning for COVID-19 segmentation and classification, Computers in Biology and Medicine (2022), 106136. https://doi.org/10.1016/j.compbiomed.2022.106136.

Wang

, Xu

, Gao

, et al. Detection of SARS-CoV-2 in different types of clinical specimens, JAMA (2020)323(18)1843–1844. https://doi.org/10.1001/JAMA2020.3786.

Tianbo Wu , Chen Tang , Min Xu , Nian Hong , Zhenkun Lei , ULNet for the detection of coronavirus (COVID-19) from chest X-ray images, Computers in Biology and Medicine 137 (2021), 104834. https://doi.org/10.1016/j.compbiomed.2021.104834.

Yafang Li , Lin Yao , Jiawei Li , Lei Chen , Yiyan Song, , Zhifang Cai , Chunhua Yang , Stability issues of RT-PCR testing of SARS-CoV-2 for hospitalized patients clinically diagnosed with COVID-19, Journal of Medical Virology 92(7) (2020), 903–908. https://doi.org/10.1002/jmv.25786.

Feng Shi , Jun Wang , Jun Shi , Ziyan Wu , Qian Wang, , Zhenyu Tang, , Kelei He, , Yinghuan Shi , Dinggang Shen , Review of artificial intelligence techniques in imaging data acquisition, segmentation, and diagnosis for COVID-19, IEEE Reviews in Biomedical Engineering 14 (2020), 4–15. https://doi.org/10.1109/RBME.2020.2987975.

Min Jae Cha , Myung Jin Chung Jeong Hyun Lee , Kyung Soo Lee , Performance of deep learning model in detecting operable lung cancer with chest radiographs, Journal of Thoracic Imaging 34(2) (2019), 86–91. https://doi.org/10.1097/RTI.0000000000000388.

, Tavares

R.S.J.

, Natal Jorge

A review on the current segmentation algorithms for medical images. In Proceedings of the first international conference on computer imaging theory and applications-IMAGAPP, (VISIGRAPP 2009), ISBN:978-989-8111-68-5, 135–140. https://doi.org/10.5220/0001793501350140.

, Tavares

J.M.R.

Effective features to classify skin lesions in dermoscopic images, Expert Systems with Applications 84 (2017), 92–101. https://doi.org/10.1016/j.eswa.2017.05.003.

10.

Deng-Ping Fan , Tao Zhou , Ge-Peng Ji , Yi Zhou , Geng Chen, , Huazhu Fu, , Jianbing Shen, , Ling Shao , Inf-net: Automatic covid-19 lung infection segmentation from ct images, IEEE Transactions on Medical Imaging 39(8) (2020), 2626–2637. https://doi.org/10.1109/TMI.2020.2996645.

11.

Hania Farag

, Lamiaa Said

A.A.

Mohamed Rizk

R.M.

, Magdy Abd ElAzim Ahmed , Hyperparameters optimization for ResNet and Xception in the purpose of diagnosing COVID-19, Journal of Intelligent & Fuzzy Systems 41(2) (2021)3555–3571. https://www.doi.org/10.3233/JIFS-210925.

12.

Daniel Arias-Garzón , Jesús Alejandro Alzate-Grisales , Simon Orozco-Arias , Harold Brayan Arteaga-Arteaga , Mario Alejandro Bravo-Ortiz , Alejandro Mora-Rubio , Jose Manuel Saborit-Torres , et al. COVID-19 detection in X-ray images using convolutional neural networks, Machine Learning with Applications 6 (2021), 100138. https://doi.org/10.1016/j.mlwa.2021.100138.

13.

Faizan Munawar , Shoaib Azmat , Talha Iqbal Christer Grönlund , Hazrat Ali , Segmentation of lungs in chest X-ray image using generative adversarial networks, IEEE Access 8 (2020), 153535–153545. https://doi.org/10.1109/ACCESS.2020.3017915.

14.

Tulin Ozturk , Muhammed Talo , Eylul Azra Yildirim , Ulas Baran Baloglu Ozal Yildirim , Rajendra Acharya

, Automated detection of COVID-19 cases using deep neural networks with X-ray images, Computers in Biology and Medicine 121 (2020), 103792. https://doi.org/10.1016/j.compbiomed.2020.103792.

15.

Shayan Hassantabar , Mohsen Ahmadi , Abbas Sharifi , Diagnosis and detection of infected tissue of COVID-19 patients based on lung X-ray image using convolutional neural network approaches,&, Fractals 140 (2020), 110170. https://doi.org/10.1016/j.chaos.2020.110170.

16.

Ioannis Apostolopoulos

, Tzani Mpesiana

, Covid-19: Automatic detection from x-ray images utilizing transfer learning with convolutional neural networks, Physical and Engineering Sciences in Medicine 43(2) (2020), 635–640. https://doi.org/10.1007/s13246-020-00865-4.

17.

Linda Wang

Zhong Qiu Lin , Alexander Wong , Covid-net: A tailored deep convolutional neural network design for detection of covid-19 cases from chest x-ray images, Scientific Reports 10(1) (2020), 1–12. https://doi.org/10.1038/s41598-020-76550-z.

18.

Štifanić

, Musulin

, Jurilj

, Šegota

, Lorencin

, Anđelić,

, Vlahinić,

, et al. Semantic segmentation of chest X-ray images based on the severity of COVID-19 infected patients, EAI Endorsed Transactions on Bioengineering and Bioinformatics 1(3) (2021). https://dx-doi-org.web.bisu.edu.cn/10.4108/eai.7-7-2021.170287.

19.

Abdul Waheed , Muskan Goyal , Deepak Gupta , Ashish Khanna Fadi Al-Turjman , Plácido Rogerio Pinheiro , Covidgan: data augmentation using auxiliary classifier gan for improved covid-19 detection, IEEE Access 8 (2020), 91916–91923. https://doi.org/10.1109/ACCESS.2020.2994762.

20.

Muhammad Chowdhury

E.H.

, Tawsifur Rahman , Amith Khandakar , Rashid Mazhar , Muhammad Abdul Kadir , Zaid Bin Mahbub , Khandakar Reajul Islam , et al. Can AI help in screening viral and COVID-19 pneumonia? IEEE Access 8 (2020), 132665–132676. https://doi.org/10.1109/ACCESS.2020.3010287.

21.

Sethy

P.K.

, Behera

S.K.

Pradyumna Ratha

, Biswas

, Detection of coronavirus disease (COVID-19) based on deep features and SVM, Int J Math Eng Manag Sci 5(4) (2020), 643–651. https://doi.org/10.33889/IJMEMS.2020.5.4.052.

22.

Ezz El-Din Hemdan

Marwa Shouman

, Mohamed Esmail Karar , Covidx-net: A framework of deep learning classifiers to diagnose covid-19 in x-ray images, arXiv preprint arXiv:2003.11055 (2020).https://doi.org/10.48550/arXiv.2003.11055.

23.

Narin

, Kaya

, Pamuk

Automatic detection of coronavirus disease (COVID-19) using X-ray images and deep convolutional neural networks, Pattern Anal Applic 24 (2021), 1207–1220. https://doi.org/10.1007/s10044-021-00984-y.

24.

Anandbabu Gopatoti , Vijayalakshmi

, Optimized chest X-ray image semantic segmentation networks for COVID-19 early detection, Journal of X-Ray Science and Technology 30(3) (2022), 491–512. https://doi.org/10.3233/XST-211113.

25.

Prottoy Saha

Muhammad Sheikh Sadi , Md Milon Islam , EMCNet: Automated COVID-19 diagnosis from X-ray images using convolutional neural network and ensemble of machine learning classifiers, Informatics in Medicine Unlocked 22 (2021), 100505. https://doi.org/10.1016/j.imu.2020.100505.

26.

Md Jahid Hasan

Md Shahin Alom , Md Shikhar Ali , Deep learning based detection and segmentation of COVID-19 & pneumonia on chest X-ray image, In 2021 International Conference on Information and Communication Technology for Sustainable Development (ICICT4SD), pp. 210–214. IEEE, 2021. https://doi.org/10.1109/ICICT4SD50815.2021.9396878.

27.

Adnan Saood , Iyad Hatem , COVID-19 lung CT image segmentation using deep learning methods: U-Net versus SegNet, BMC Medical Imaging 21(1) (2021), 1–10. https://doi.org/10.1186/s12880-020-00529-5.

28.

Muhammad Ubale Kiru , Bahari Belaton , Xinying Chew , Khaled Almotairi

, Ahmad Mohd Aziz Hussein , Maryam Aminu , Comparative analysis of some selected generative adversarial network models for image augmentation: a case study of COVID-19 x-ray and CT images, Journal of Intelligent & Fuzzy Systems 43(6) (2022), 7153–7172. https://doi.org/10.3233/JIFS-220017.

29.

Shreyas Mishra , A Novel automated method for COVID-19 infection and lung segmentation using deep neural networks, (2020).https://doi.org/10.21203/rs.3.rs-86796/v1.

30.

Alan Zhang , Covid-19 chest X-ray images: Lung segmentation and diagnosis using neural networks, Int J Comput Sci Appl IJCSA 10(5)–https://doi.org/10.5121/ijcsa.2020.10501.

31.

Luís Vinícius De Moura , Christian Mattjie , Caroline Machado Dartora , Rodrigo Barros

, Ana Maria Marques da Silva , Explainable machine learning for COVID-19 pneumonia classification with texture-based features extraction in chest radiography, Frontiers in Digital Health 3 (2021)–https://doi.org/10.3389/fdgth.2021.662343.

32.

Johnatan Carvalho Souza , João Otávio Bandeira Diniz , Jonnison Lima Ferreira , Giovanni Lucca França da Silva , Aristofanes Correa Silva, , Anselmo Cardoso de Paiva, An automatic method for lung segmentation and reconstruction in chest X-ray using deep neural networks, Computer Methods and Programs in Biomedicine 177 (2019), 285–296. https://doi.org/10.1016/j.cmpb.2019.06.005.

33.

Viacheslav Danilov

, Diana Litmanovich , Alex Proutski , Alexander Kirpich , Dato Nefaridze , Alex Karpovsky , Yuriy Gankin , Automatic scoring of COVID-19 severity in X-ray imaging based on a novel deep learning workflow, Scientific Reports 12(1) (2022), 1–22. https://doi.org/10.1038/s41598-022-15013-z.

34.

Sifat Ahmed , Tonmoy Hossain , Oishee Bintey Hoque , Sujan Sarker , Sejuti Rahman , Faisal Muhammad Shah , Automated covid-19 detection from chest x-ray images: a high-resolution network (hrnet) approach, SN Computer Science 2(4) (2021), 1–17. https://doi.org/10.1007/s42979-021-00690-w.

35.

Ejaz Khan , Muhammad Zia Ur Rehman , Fawad Ahmed , Faisal Abdulaziz Alfouzan , Nouf Alzahrani

, Jawad Ahmad , Chest X-ray classification for the detection of COVID-19 using deep learning techniques, Sensors 22(3) (2022), 1211. https://doi.org/10.3390/s22031211.

36.

Rajeswari

Veerraju Gampala

Balajee Maram , Cristin

, FWLICM-deep learning: Fuzzy weighted local information C-means clustering-based lung lobe segmentation with deep learning for COVID-19 detection, Journal of Digital Imaging (2022), 1–16. https://doi.org/10.1007/s10278-022-00667-y.

37.

Asu Kumar Singh , Anupam Kumar , Mufti Mahmud , Shamim Kaiser

, Akshat Kishore , COVID-19 infection detection from chest X-ray images using hybrid social group optimization and support vector classifier, Cognitive Computation (2021), 1–13. https://doi.org/10.1007/s12559-021-09848-3.

38.

Anandbabu Gopatoti , Vijayalakshmi

, CXGNet: A Tri-phase chest X-ray image classification for COVID-19 diagnosis using deep CNN with enhanced grey-wolf optimizer, Biomedical Signal Processing and Control (2022), 103860. https://doi.org/10.1016/j.bspc.2022.103860.

39.

Aysen Degerli , Mete Ahishali , Mehmet Yamac , Serkan Kiranyaz , Muhammad Chowdhury

EH.

, Khalid Hameed , Tahir Hamid , Rashid Mazhar , Moncef Gabbouj , COVID-19 infection map generation and detection from chest X-ray images, Health Information Science and Systems 9(1) (2021), 1–16. https://doi.org/10.1007/s13755-021-00146-8.

40.

Anas Tahir

, Muhammad Chowdhury

E.H.

, Amith Khandakar , Tawsifur Rahman , Yazan Qiblawey , Uzair Khurshid , Serkan Kiranyaz , et al. COVID-19 infection localization and severity grading from chest X-ray images, Computers in Biology and Medicine 139 (2021), 105002. https://doi.org/10.1016/j.compbiomed.2021.105002.

41.

Tawsifur Rahman , Amith Khandakar , Yazan Qiblawey , Anas Tahir , Serkan Kiranyaz , Saad Bin Abul Kashem , Mohammad Tariqul Islam , et al. Exploring the effect of image enhancement techniques on COVID-19 detection using chest X-ray images, Computers in Biology and Medicine 132 (2021), 104319. https://doi.org/10.1016/j.compbiomed.2021.104319.

42.

Anas Tahir

, Muhammad Chowdhury

E.H.

, Yazan Qiblawey , Amith Khandakar , Tawsifur Rahman , Serkan Kiranyaz , Uzair Khurshid , Nabil Ibtehaz , Sakib Mahmud , Maymouna Ezeddin , COVID-QU-Ex, Kaggle, 2021,https://doi.org/10.34740/kaggle/dsv/3122958.

43.

Olaf Ronneberger

Philipp , Thomas Brox , U-net: Convolutional networks for biomedical image segmentation, In International Conference on Medical image Computing and Computer-Assisted Intervention, pp. 234–241. Springer, Cham, 2015. https://doi.org/10.1007/978-3-319-24574-4_28.

44.

Badrinarayanan

, Kendall

, Cipolla

SegNet: A deep convolutional encoder-decoder architecture for image segmentation, IEEE Transactions on Pattern Analysis and Machine Intelligence 39(12) (2017), 2481–2495. https://doi.org/10.1109/TPAMI.2016.2644615.

45.

Anandbabu Gopatoti , Multi-texture features and optimized DeepNet for COVID-19 detection using chest x-ray images, Concurrency and Computation: Practice and Experience 34(22) (2022), e7157. https://doi.org/10.1002/cpe.7157.

46.

John Holland

, Adaptation in natural and artificial systems: an introductory analysis with applications to biology, control, and artificial intelligence. MIT press, 1992.

47.

Yanan Sun , Bing Xue , Mengjie Zhang Gary Yen

, Jiancheng Lv , Automatically designing CNN architectures using the genetic algorithm for image classification, IEEE Transactions on Cybernetics 50(9) (2020), 3840–3854. https://doi.org/10.1109/TCYB.2020.2983860.

Metrics (%)	Feature extraction methods
	LOOP	LBP	LDP	LTP	LDTP	HLGP	LGHP	LGP	ROI by Fusion of MTFs
BCCAC	89.48	86.21	76.31	79.65	84.36	84.81	77.26	86.75	99.98
BCCSEN	87.16	81.85	71.24	74.55	80.29	81.47	72.22	83.83	99.96
BCCSPE	94.03	92.32	90.56	91.04	93.83	94.83	92.68	94.36	99.94
BCCPRE	91.51	89.44	80.32	83.24	87.78	87.15	81.57	89.11	99.93
BCCF₁	89.24	85.38	74.17	78.53	83.76	84.04	76.46	86.32	99.97