Enhancing alzheimer’s diagnosis through optimized brain lesion classification in MRI with attention-driven grid feature fusion

Abstract

This paper explores cognitive interface technology, aiming to tackle current challenges and shed light on the prospects of brain-computer interfaces (BCIs). It provides a comprehensive examination of their transformative impact on medical technology and patient well-being. Specifically, this study contributes to addressing challenges in classifying brain lesion images arising from the complex nature of lesions and limitations of traditional deep learning approaches. It introduces advanced feature fusion models that leverage deep learning algorithms, including the African vulture optimization (AVO) algorithm. These models integrate informative features from multiple pre-trained networks and employ innovative fusion techniques, including the attention-driven grid feature fusion (ADGFF) model. The ADGFF model incorporates an attention mechanism based on the optimized weights obtained using AVO. The objective is to improve the overall accuracy by providing fine-grained control over different regions of interest in the input image through a grid-based technique. This grid-based technique divides the image into vertical and horizontal grids, simplifying the exemplar feature generation process without compromising performance. Experimental results demonstrate that the proposed feature fusion strategies consistently outperform individual pre-trained models in terms of accuracy, sensitivity, specificity, and F1-score. The optimized feature fusion strategies, particularly the GRU-ADGFF model, further enhance classification performance, outperforming CNN and RNN classifiers. The learning progress analysis shows convergence, indicating the effectiveness of the feature fusion strategies in capturing lesion patterns. AUC-ROC curves highlight the superior discriminatory capabilities of the ADGFF-AVO strategy. Five-fold cross-validation is employed to assess the performance of the proposed models, demonstrating their accuracy, and few other accuracy-based measures. The GRU-ADGFF model optimized with AVO consistently achieves high accuracy, sensitivity, and AUC values, demonstrating its effectiveness and generalization capability. The GRU-ADGFF model also outperforms the majority voting ensemble technique in terms of accuracy and discriminative ability. Additionally, execution time analysis reveals good scalability and resource utilization of the proposed models. The Friedman rank test confirms significant differences in classifier performance, with the GRU-ADGFF model emerging as the top-performing method across different feature fusion strategies and optimization algorithms.

Keywords

Cognitive-interface brain-computer interfaces (BCIs)brain lesion classification CNN’s pre-trained networks attention-driven feature fusion attention-driven grid-based feature fusion African vulture optimization (AVO)recurrent neural network (RNN)gated recurrent unit (GRU)

1. Background and significance

Cognitive interface technology, notably embodied in brain-computer interfaces (BCIs), stands as a trailblazer within the realm of cutting-edge innovations, forging a seamless connection between human cognition and technology. This rapidly evolving field symbolizes a transformative alliance, fostering synergy between the complexities of the human brain and computational systems. BCIs hold the potential to usher in a revolution, transcending various facets of our daily existence, ranging from healthcare and assistive technology to communication and entertainment. This research sets out on a quest to unravel the expansive possibilities within the design of computer-aided systems, with a particular focus on brain lesions. These lesions, indicative of abnormal changes in the structure or function of brain tissue, are recognized as pivotal opportunities that could shape the trajectory of future human-computer interaction. In exploring this intersection, the study aims to contribute to the ongoing discourse surrounding the potential of cognitive interface technology and its intricate relationship with neurological health.

The abnormalities in the brain can arise from various causes, including traumatic injuries, infections, tumors, vascular issues, or degenerative diseases. The prevalence of brain lesions is significant, as they affect a substantial number of individuals worldwide. The impact of brain lesions on patient health can be profound and multifaceted. Depending on the location and size of the lesion, patients may experience a wide range of symptoms, such as cognitive impairment, motor dysfunction, sensory deficits, emotional disturbances, or seizures [1, 2]. These symptoms can significantly impair daily functioning, quality of life, and overall well-being. Moreover, brain lesions can have long-term consequences, leading to permanent disabilities or even life-threatening conditions. Effective brain lesion management requires a collaborative, multidisciplinary approach involving neurologists, neurosurgeons, rehabilitation specialists, and healthcare professionals. This recognizes the complexity of brain disorders, leveraging diverse medical perspectives. Neurologists provide diagnostic and medical expertise, neurosurgeons handle surgical interventions, and rehabilitation specialists aid recovery by addressing functional impairments. Involvement of other healthcare professionals ensures a holistic approach, aiming to alleviate immediate impact and optimize long-term recovery outcomes through combined specialized knowledge and skills [3]. Accurate and automated classification of brain lesions in magnetic resonance imaging (MRI) and computed tomography (CT) scan images is pivotal for timely diagnosis and treatment, offering several compelling reasons. Automation reduces manual effort, enabling early intervention and improving patient outcomes. Standardizing the diagnostic process enhances consistency and reliability in identifying various lesions. Additionally, automated methods, particularly utilizing machine learning, deep learning, and pre-trained convolutional neural networks (CNNs), significantly boost accuracy by efficiently analyzing medical images and extracting pertinent features. The prowess of CNNs in image analysis, coupled with the efficiency gained from pre-trained networks, has markedly advanced the field, enabling more precise and swift identification of subtle features critical for effective diagnosis and treatment planning [4, 5].

The primary challenges encountered when employing conventional deep learning methods for classifying brain lesion images stem from the inherent complexity and heterogeneity of lesions, as well as the limitations of traditional approaches. Variability in lesion characteristics, inter-observer variability, and the intricate patterns present in brain lesions hinder accurate classification. Traditional methods struggle to capture the diverse and complex features of brain lesions, leading to suboptimal performance. Advanced feature fusion models address these challenges by integrating information from multiple networks and combining features from different regions of interest. This integration significantly enhances the accuracy, robustness, and generalization ability of brain lesion image classification. The main objective of this research is to develop a feature fusion model that accurately classifies brain lesion MRI images by leveraging advanced techniques, including machine learning algorithms and deep learning networks, to integrate informative features from multiple pre-trained networks, such as CNNs [6, 7]. By amalgamating features extracted from diverse regions of interest, the model seeks to provide a comprehensive representation of brain lesions and their distinctive attributes, aiming to achieve a heightened level of accuracy in classification [8, 9]. This precision is crucial for influencing the diagnostic process and facilitating informed decisions in treatment planning.

The research makes a significant contribution by developing an attention-driven grid feature fusion model specifically designed for brain lesion classification in MRI. The model incorporates an attention mechanism to identify and emphasize informative features within brain lesion images, assigning pre-assigned weights obtained from pre-trained networks [6, 7, 10]. By combining this attention-driven approach with a grid-based feature fusion technique, the model effectively integrates features with optimized weights from the pre-trained networks, greatly enhancing overall classification performance. The research explores diverse feature extraction methods to capture meaningful features and investigates the effectiveness of combining features from various pre-trained networks [11, 12, 13]. It evaluates the performance of the proposed feature fusion models in comparison to traditional approaches, aiming to significantly contribute to the field of brain lesion classification in MRI.

The attention mechanism plays a vital role by enabling the identification of essential features within the images, enhancing interpretability by indicating the significance of features during the fusion process. Additionally, the grid-based technique employed reduces the complexity of exemplar feature generation in deep learning models by dividing MRI images into horizontal and vertical grids, significantly reducing the number of exemplars. This technique allows for fine-grained control, enabling a detailed and precise analysis of the input images. Ultimately, this research has the potential to assist healthcare professionals in making more precise diagnoses, planning suitable treatment strategies, and improving patient outcomes.

The primary objective is to greatly enhance the overall classification performance of brain lesions by fusing relevant features, thereby providing finer control over the importance of different regions in the input image.To achieve these objectives, the research explores diverse feature extraction methods aimed at capturing meaningful features. It also investigates the effectiveness of combining features from various pre-trained networks and evaluates the performance of the proposed feature fusion models in comparison to traditional approaches. By introducing an attention-driven grid feature fusion model, this manuscript aims to significantly contribute to the field of brain lesion classification in MRI, improving the accuracy and diagnostic capabilities of healthcare professionals.The attention mechanism plays a vital role in this research, as it enables the identification of essential features within the images. By leveraging optimized weights obtained through the African vulture optimization (AVO) [14, 15] meta-heuristic optimizer from pre-trained networks, the attention mechanism enhances the interpretability of features by indicating their significance during the feature fusion process. Additionally, the grid-based technique employed in this framework reduces the complexity of exemplar feature generation in deep learning models. By dividing the MRI images into horizontal and vertical grids, the technique significantly reduces the number of exemplars. The incorporation of the grid-based feature generator into the framework enables the achievement of high accuracy for the given problem. Furthermore, this grid-based technique allows for fine-grained control, enabling a more detailed and precise analysis of the input images.Ultimately, this research has the potential to assist healthcare professionals in making more precise diagnoses, planning suitable treatment strategies, and ultimately improving patient outcomes.

The manuscript is organized as follows: Section 2 presents the literature studies, while Section 3 discusses the datasets, parameters, methodologies, and proposed feature fusion strategies. Section 4 covers the experimentation and result analysis, and Section 5 outlines the expected contributions and impact of the study. Finally, Section 6 concludes the study, highlighting future scopes.

2. Literature reviews

The literature review in this manuscript aims to provide a comprehensive overview of advanced medical image classification techniques using pre-trained networks and feature fusion strategies. It covers significant research findings, approaches, and advancements in automated classification and enhanced diagnostic precision. By synthesizing existing literature, this review identifies key trends, challenges, and opportunities for further investigation in this field. It highlights the effectiveness of pre-trained networks in improving the classification accuracy of medical image datasets classification. The subsequent sections of this paper build upon the literature review, setting the groundwork for additional research, discussion of novel approaches, and potential advancements in classification.

In [16], the early detection of cognitive impairment and Alzheimer’s disease (AD) using neuroimages and transfer learning is investigated. GoogLeNet, AlexNet, and ResNet-18 are employed for classification, achieving high accuracies in detecting AD. However, as network complexity increases, classification efficiency diminishes. The study focuses on class-wise performance analysis, and AlexNet performs well with transfer learning. The limitations of deep learning algorithms in identifying changes in functional connectivity within the functional brain network of patients with mild cognitive impairment are addressed in [17]. The proposed model utilizes randomized concatenated deep features from pre-trained models and achieves impressive accuracy in multiclass classification of AD. The utilization of pre-trained models and machine learning classifiers for COVID-19 diagnosis using X-ray chest radiographs is explored in [18]. Various pre-trained models, such as Inception-V3, demonstrate satisfactory performance when combined with augmentation techniques and edge-based feature extraction. A transfer learning-based approach using AlexNet for AD MRI image classification is proposed in [19]. The model achieves promising accuracy in multiclass classification of un-segmented images by fine-tuning the pre-trained architecture. In [20], a customized CNN and transfer learning are employed for the segmentation and classification of AD MRI scans. The study demonstrates the efficacy of transfer learning and gray matter segmentation in Alzheimer’s detection using MRI scans. A fusion-based framework for classifying brain MRI to diagnose AD is introduced in [21]. The proposed fusion schemes, utilizing CNNs and transforms, show improved accuracy compared to decision-level fusion. Feature-level fusion with EfficientNet-B7 achieves the highest accuracy. An efficient deep learning method, the deep feature fusion classification network (DFFCNet), is proposed in [22]. The DFFCNet, utilizing feature fusion and ensemble learning, demonstrates superior performance in disease classification tasks for COVID-19. A novel approach using deep CNNs with feature fusion and ensemble learning strategies is proposed in [23] for mammographic scan abnormalities classification. The model achieves high sensitivity, specificity, and accuracy on publicly available datasets. The scarcity of large-scale datasets in developing deep learning models for multi-modality medical images is addressed in [24]. The proposed fusion method based on evidential deep learning effectively combines information from small-scale and incomplete datasets. An efficient hybrid architecture combining CNNs and transformers is proposed in [25] for stage I multimodality esophageal cancer image classification. The RFIA-Net achieves superior results compared to other networks on the esophageal cancer dataset. A fully feature fusion based neural network (F3-Net) is introduced in [26] for COVID-19 lesion segmentation in CT images. The F3-Net architecture, along with an improved loss function, demonstrates effectiveness in accurately segmenting COVID-19 lesions.

The studies [16, 17, 18, 19, 20] aim to investigate the early detection of cognitive impairment and AD using neuroimages and transfer learning techniques. Various pre-trained networks such as GoogLeNet, AlexNet, ResNet-18, DenseNet201, SqueezeNet, Inception-V3, ResNet-50, VGG-16, and Densenet-169 are utilized for classification purposes. These studies also explore the use of hybrid pre-trained CNN models, deep feature concatenation methods, weight randomization for aligning feature maps, and gradient-weighted class activation maps to improve the generalization of medical images. On the other hand, the studies [21, 22, 23, 24, 25] primarily focus on fusion-based frameworks that leverage pre-trained networks to achieve better classification performance for medical images. The authors propose the use of transforms and CNNs for deep feature extraction to identify discriminative features. Inspired by the capabilities of these pre-trained networks, this work applies fusion at the feature level, where the fused features are used as input to classifiers for the classification of brain lesion images. The fusion approach primarily focuses on the adaptation of attention mechanisms applied to pre-trained networks to obtain useful features and a grid-based approach for dividing input images. These techniques enable localized feature extraction, improve robustness to image variations, preserve spatial context, and reduce computational complexity in the classification process.

3. Methodolog

Our proposed approach to enhance brain lesion classification in Alzheimer’s disease MRI scans is presented. The attention-driven grid feature fusion (ADGFF) technique is introduced, which utilizes an optimized mode to improve classification accuracy and reliability. The methodology includes key steps such as data acquisition, parameter selection, feature extraction using pre-trained CNN networks, attention-based and grid feature fusion, and model optimization using the AVO algorithm.

3.1 Dataset acquisitio

Table 1
Description of datasets

Datasets	Total data size	No. of training samples	No. of testing samples
ADNI	360 cognitively normal $+$ 290 alzheimer’s disease	455	195
LONI BMR	150 normal $+$ 175 disease	228	97

Two primary datasets were used in this experiment to investigate AD and brain structure-function relationships. The first dataset is the AD neuroimaging initiative (ADNI) dataset, which includes MRI scans from individuals with AD, mild cognitive impairment, and healthy controls [27]. The experiment also incorporated the LONI brain MRI image dataset (LONI BMR), consisting of high-quality brain MRI scans maintained by the laboratory of neuroimaging (LONI) [28]. The use of these datasets forms a robust foundation for the experiment, facilitating a comprehensive investigation of AD and its neuroimaging characteristics. Table 1 provides details on the datasets used, including the number of samples for training and testing.

3.2 Parameters selection

Table 2
Parameters and their selected values used for experimentation

Classification and optimization techniques	Parameters and associated values
Recurrent neural network (RNN) [3, 29]	Optimizer-Adam; Activation Function-RLU; LearningRate-0.001; Momentum-0.9; Dropout-0.1; Cell Size-50; Epochs $=$ 30.
CNN [6, 7, 8, 9]	Optimizer-Adam; Activation function-RLU; WeightRegularization-L1 $=$ 0; L2 $=$ 0.01; Dropout-0.1; Batch Size-16; Epochs $=$ 30.
Gated recurrent unit (GRU) [30, 31]	Optimizer-Adam; Activation Function-RLU; WeightRegularization-L1 $=$ 0, L2 $=$ 0.01; Dropout-0.1; Batch Size-16; Epochs $=$ 30.
Genetic algorithm (GA) [32]	Maximum Number of Iterations $=$ 50; Population Size $=$ 10; SelectionMethod-Roulette Wheel;
Differential evolution (DE) [34]	Maximum Number of Iterations $=$ 50; Crossover Rate $=$ 0.7; MutationFactor $=$ 0.5.
AVO [14, 15]	Maximum Number of Iterations $=$ 50; Number of Vultures $=$ 10.

The chosen parameters for the classifiers and optimization strategies during the experimentation process are summarized in Table 2. The table provides an overview of the specific parameters selected for each component, facilitating an understanding of the experimental setup, and allowing for reproducibility and comparison of results

3.3 Utilizing pre-trained networks for feature extraction

Pre-trained CNN networks excel in feature extraction for brain lesion MRI data classification. They learn complex patterns and meaningful features from large-scale image datasets. After training on vast image databases, these networks serve as fixed feature extractors, extracting high-level features from brain lesion MRI data [7, 16, 17, 18, 19, 20, 21, 22, 24, 25]. These features capture crucial information about lesion presence, location, and characteristics. Utilizing pre-trained networks overcomes data limitations and time-consuming feature engineering. This approach accelerates classification, enhances accuracy, and improves the reliability of brain lesion diagnosis. In this study, we employed widely used pre-trained networks: Inception-ResNet-V2 [34, 35], visual geometry group(VGG-16) [6, 35], ResNeXt-10 [37, 38], and progressive neural architecture search(PNASNet-5) [39]. Leveraging their extraction capabilities, we aimed to enhance the understanding and analysis of brain lesion data.

Inception-ResNet-V2 is an advanced neural network that combines the concepts of the inception network and ResNet. It addresses the problem of vanishing gradients by incorporating residual connections, enabling the training of deeper networks. Its architecture consists of stacked inception modules with parallel convolutional branches of different filter sizes, capturing multi-scale features. VGG-16 is a deep network with 16 layers that uses stacked 3x3 convolutional filters and minimizes the use of pooling layers. It excels at capturing significant features in brain lesion MRI images. ResNeXt-10 is a variant of ResNeXt with 10 layers that increases the network’s capacity to capture diverse features by utilizing parallel pathways. It effectively captures complex features in brain lesion data. PNASNet-5 is discovered through neural architecture search, automatically finding an optimal architecture. It captures meaningful features in brain lesion MRI images by extracting hierarchical representations. These pre-trained networks extract features from brain lesion MRI data, capturing complex patterns and structures. It has been observed that, by combining the parallel branches of inception modules, residual connections, and factorized convolutions, Inception-ResNet-V2 achieved an architecture that can effectively capture multi-scale features, propagate gradients more efficiently, reduced computational complexity, and enabled the training of deeper networks. Similarly, leveraging the pre-trained VGG-16, ResNeXt-10,andPNASNet-5networks as feature extractors empowered us to harness its remarkable capabilities, enhancing comprehension and analysis of brain lesion MRI data [16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39].

3.4 Unleashing the power of the proposed feature fusion strategies in image classification

Figure 1.

The layout of the proposed feature fusion strategiescombining features from multiple pre-trained networks, adjusting weights, optimizing fusion strategies, and prioritizing relevant features using attention weights and utilizing a grid-based structure.

The planned feature fusion methodologies involved combining features extracted from different pre-trained networks, namely Inception-ResNet-V2, VGG-16, ResNeXt-10, and PNASNet-5. This fusion aimed to take the benefit from the unique strengths and representations learned by each network to enhance the classification of brain lesion MRI data. By combining these diverse sets of features, we can capture a more comprehensive and discriminative representation of the data, potentially improving the accuracy and robustness of the classification model [8, 9, 23, 24, 25, 26, 40]. The fusion process combines the extracted features from multiple networks by employing appropriate techniques as discussed below and is depicted in Fig. 1.

(a)

One such technique is the concatenation of outputs from four pre-trained models, resulting in a batch of feature sets known as unified feature fusion (UFF), representing the most straightforward form of feature fusion which combines the 1024 number of features each from Inception-ResNet-V2, ResNeXt-10, and PNASNet-5and 512 number of features from VGG-16 which is represented in Eq. ((a)).

$\displaystyle\left[\textit{Feature∼{}set}_{\text{Inception}-\text{ResNet}-% \text{V}2},\textit{Feature∼{}set}_{\text{VGG}-16},\right.$ $\displaystyle\left.\textit{Feature∼{}set}_{\text{ResNeXt}-10},\textit{Feature∼% {}set}_{\text{PNASNet}-5}\right]$ (1)

(b)

In adaptive weight-driven feature fusion (AWDFF), the weights of the pre-trained networks are initialized to [0,1]. The combined feature set is formed by adaptively selecting weights and concatenating them with the extracted features from each respective pre-trained model.This approach allows for dynamic adjustment of the weights to create the combined feature set asrepresented in Eq. ((b)). This dynamic approach enables the weights $w_{1}$ , $w_{2}$ , $w_{3}$ and $w_{4}$ to be updated using gradient descent approach.

$\displaystyle\left[w_{1}\times\textit{Feature∼{}set}_{\text{Inception}-\text{% ResNet}-\text{V}2},w_{2}\times\textit{Feature∼{}set}_{\text{VGG}-16},\right.$ $\displaystyle\left.w_{3}\times\textit{Feature∼{}set}_{\text{ResNeXt}-10},w_{4}% \times\textit{Feature∼{}set}_{\text{PNASNet}-5}\right]$ (2)

(c)

In feature-type driven feature fusion (FTDFF), four different weights are initially assigned to the fourtypes of feature sets extracted from the pre-trained networks. The classifier is trained using these assigned weights. As part of the optimization process in this experimentation, the weights of the feature fusion strategy are updated through the utilization of specific optimization algorithms. This allows for an optimized version of the feature fusion strategy to be achieved by iteratively adjusting the weightsare done through the same Eq. ((b)) where, theweights $w_{1}$ , $w_{2}$ , $w_{3}$ and $w_{4}$ are updated based on the updating policy of the chosen optimization algorithms. In this approach, only the features from pre-trained networks with weights greater than 0.5 are considered to form the feature set. As a result, important features from the pre-trained networks whose weights fall below 0.5 are ignored, ensuring a more selective and optimized feature fusion strategy ignoring the other pre-trained network(s) which may lead to non-selection of important features from the pre-networks.

(d)

The motivation behind the attention-driven feature fusion (ADFF) strategy lies in its objective to enhance feature representation through the application of optimization algorithms to derive attention weights. By employing these attention weights, the strategy aims to selectively emphasize significant features while concurrently suppressing irrelevant or noisy elements within a given dataset. This process is motivated by the recognition that not all features contribute equally to the effectiveness of a model, and attention-driven mechanisms offer a means to prioritize and focus on the most pertinent information. Ultimately, ADFF strives to optimize feature sets, promoting improved model performance and efficiency by allowing the algorithm to dynamically allocate attention to the most relevant aspects of the data. In this strategy, the classifier is initially trained using the extracted feature sets obtained from the pre-trained networks. Subsequently, the weights of this trained classifier undergo a weight optimization process to extract the attention weights. This dynamic process allows the classifier to prioritize important features based on their relevance, as illustrated in Eq. ((d)). ADFF overcomes the limitation observed in FTDFF, where some pre-trained networks’ features were ignored, by utilizing an attention mechanism that selects the features of importance from each pre-trained network.

$\displaystyle\left[w_{1\left[{1\times 1024}\right]}\times\textit{Featureset}_{% \textit{Inception}-\textit{ResNet}-V2},w_{2\left[{1\times 512}\right]}\times% \textit{Featureset}_{\textit{VGG}-16},\right.$ $\displaystyle\left.w_{3\left[{1\times 1024}\right]}\times\textit{Featureset}_{% \textit{ResNeXt}-10},w_{4\left[{1\times 1024}\right]}\times\textit{Featureset}% _{\textit{H1:PNASNet}-5}\right]$ (3)

(e)

The attention-driven grid-based feature fusion (ADGFF) strategy expands on the ADFF approach by introducing a grid partitioning technique for each input image. Initially, the datasets are split into separate training and testing sections. Next, all the images are divided into horizontal and vertical grids. This grid configuration enables the extraction of feature sets from the pre-trained networks, considering both the horizontal and vertical grids. To train the classifier, the extracted feature sets from all eight grids are utilized, comprising four horizontal and four vertical grids. Subsequently, a weight optimization process is applied to the trained classifier, determining the attention weights as represented in Eq. ((e)). This process empowers the classifier to dynamically prioritize crucial features based on their relevance through attention mechanism and leveraging the grid-based structure of the input image as a guiding framework. As we know, brain lesions can vary in size and shape, and they may manifest at different scales within the brain. By employing this grid-based representation, the classifier can analyze brain regions at multiple scales, ensuring that lesions of various sizes are effectively captured and classified. Additionally, this grid-based approach enables the fusion of features from different imaging modalities within each grid, providing a comprehensive representation of brain regions and improving classification accuracy.

$\displaystyle\left[w_{1\left[{1\times 1024}\right]}\times\textit{Feature∼{}set% }_{\textit{H1:Inception -- ResNet -- V2}},w_{1\left[{1\times 1024}\right]}% \times\textit{Feature∼{}set}_{\textit{V1:Inception -- ResNet -- V2}},\right.$ $\displaystyle\qquad\left.w_{2\left[{1\times 512}\right]}\times\textit{Feature∼% {}set}_{\textit{H1:VGG}-16},w_{2\left[{1\times 512}\right]}\times\textit{% Feature∼{}set}_{\textit{V1:VGG}-16},\right.$ $\displaystyle\qquad\left.w_{3\left[{1\times 1024}\right]}\times\textit{Feature% ∼{}set}_{\textit{H1:ResNeXt}-10},w_{3\left[{1\times 1024}\right]}\times\textit% {Feature∼{}set}_{\textit{V1:ResNeXt}-10},\right.$ $\displaystyle\qquad\left.w_{4\left[{1\times 1024}\right]}\times\textit{Feature% ∼{}set}_{\textit{H1:PNASNet}-5},w_{4\left[{1\times 1024}\right]}\times\textit{% Feature∼{}set}_{\textit{V1:PNASNet}-5}\right]$ (4)

The attention mechanisms employed in both the ADFF and ADGFF strategies play pivotal roles in dynamically prioritizing features and enhancing the discriminatory power of the classification model. ADFF optimizes feature sets by selectively emphasizing significant features while suppressing irrelevant or noisy elements, thus improving model performance and efficiency. By training the classifier using extracted feature sets and subsequently optimizing attention weights, ADFF ensures important features are prioritized based on relevance. ADGFF builds upon this approach by introducing a grid partitioning technique, enabling the model to analyze brain regions at multiple scales and effectively capture lesions of various sizes and shapes. The attention mechanism in ADGFF dynamically prioritizes crucial features based on their relevance, leveraging the grid-based structure of input images. These strategies collectively enhance the model’s ability to handle the complexity and heterogeneity of brain lesions, resulting in improved accuracy, robustness, and generalization in classification tasks.

The proposed techniques enable the combination of features from multiple pre-trained networks, adjusting weights, optimizing fusion strategies, prioritizing relevant features using attention weights, and utilizing a grid-based structure for enhanced feature fusion and classification. The ADGFF model offers several enhancements over conventional deep learning methods specifically tailored to address the challenges encountered in brain lesion classification. By incorporating the grid-based partitioning of images, the model ensures that brain lesions of various sizes and shapes are effectively captured and analyzed at multiple scales. The attention mechanism further refines this process by emphasizing the most relevant features, thereby enhancing the model’s ability to handle the inherent complexity and heterogeneity of brain lesions. Additionally, the integration of multiple pre-trained networks allows for a richer and more comprehensive feature set, leading to improved accuracy, robustness, and generalization in classification. Subsequently, in this study, the classification and model validation processes have been undergone. We evaluate the performance of these strategies by comparing accuracy measures, conducting statistical validation, and analyzing the execution time required for the classification of ADNI and LONI BMR datasets.

4. African vultures optimization algorithm (AVO

The AVO is a metaheuristic algorithm inspired by the behavior of vultures in their search for food. This algorithm follows a series of steps to explore the search space, balance exploration and exploitation, and converge towards an optimal solution for an optimization problem [14, 15]. The algorithm can be divided into four main steps based on its operating principles. The first step involves identifying the best vultures within two groups. Initially, a population of solutions is created, and their fitness values are evaluated. The best solution in each group is determined, and the remaining solutions move towards these top vultures based on their probabilities calculated using Eq. (5). The selection probabilities are influenced by parameters $L_{1}$ and $L_{2}$ , which are assigned values between 0 and 1. These parameters control the balance between exploration and exploitation in the algorithm.

$\displaystyle\textit{Best}_{\textit{Vulture}_{i}}=\left\{{{\begin{array}[]{l}{% \textit{Best}_{\textit{Vulture}_{1}}\textit{if∼{}Probability}_{\textit{Vulture% }_{i}}=L_{1}}\\ {\textit{Best}_{\textit{Vulture}_{2}}\textit{if∼{}Probability}_{\textit{% Vulture}_{i}}=L_{2}}\\ \end{array}}}\right.$ (5)

$\displaystyle\textit{Probability}_{\textit{Vulture}_{i}}=\frac{\textit{Fitness% }_{\textit{Vulture}_{i}}}{\sum_{{i}=1}^{\textit{Pop}_{\textit{Vultures}}}% \textit{Fitness}_{\textit{Vulture}_{i}}}$ (6)

The second step focuses on the rate of starvation of vultures. The behavior of vultures varies depending on their satiation levels. Hungry vultures have lower energy levels and exhibit different behaviors compared to satiated vultures. Equation (8) models this behavior by capturing the declining satisfaction rate of vultures during the optimization process. The satisfaction rate affects the movement of vultures and determines whether they explore diverse areas or focus on existing solutions.

$\displaystyle\textit{Phase}_{\textit{Transiton}}=x\times(\sin^{z}\left({\frac{% \pi}{2}\times\frac{\textit{Current}_{\textit{Iteration}_{i}}}{\textit{Maximum}% _{\textit{Iterations}}}}\right)+\cos\left({\frac{\pi}{2}\times\frac{\textit{% Current}_{\textit{Iteration}_{i}}}{\textit{Maximum}_{\textit{Iterations}}}}% \right)-1$ (7)

$\displaystyle\textit{Vulture}_{\textit{SRate}}=\left({2\times\textit{Rand}_{1}% +1}\right)\times y\times\left({1-\frac{\textit{Current}_{\textit{Iteration}_{i% }}}{\textit{Maximum}_{\textit{Iterations}}}}\right)+\textit{Phase}_{\textit{% Transiton}}$ (8)

The third step is the exploration phase, where vultures simulate their natural behavior in search of food. Vultures carefully survey their environment and travel long distances to find food. In the AVO, vultures explore random areas based on two different strategies. The selection of a strategy is determined by the parameter $P_{1}$ , and a random number ( $\textit{Rand}_{P_{1}})$ is generated to decide which strategy to employ. Equation (9) represents this selection process, where the vultures’ positions are updated based on the chosen strategy.

$\displaystyle\textit{Position}_{\textit{Vulture}\left({i+1}\right)}=\left\{{{% \begin{array}[]{l}{\textit{Equation}\left(6\right)\textit{if}∼{}P_{1}\geqslant% \textit{Rand}_{P_{1}}}\\ {\textit{Equation}\left(8\right)\textit{if}∼{}P_{1}<\textit{Rand}_{P_{1}}}\\ \end{array}}}\right.$ (9)

$\displaystyle\textit{Position}_{\textit{Vulture}\left({i+1}\right)}=\textit{% Best}_{\textit{Vulture}_{i}}-D\left(i\right)\times\textit{Vulture}_{\textit{% SRate}}$ (10)

$\displaystyle D\left(i\right)=\left|{\textit{CF}\times\textit{Best}_{\textit{% Vulture}_{i}}-\textit{Position}_{\textit{Vulture}\left(i\right)}}\right|$ (11)

The fourth step is the exploitation phase, which consists of two internal phases. In the first phase, vultures employ rotating flight and siege-fight strategies. The selection of a strategy is influenced by the parameter $P_{2}$ , and a random number ( $\textit{Rand}_{P_{2}})$ is generated to determine the chosen strategy. Equation (13) describes this selection process. In the second phase, vultures engage in aggressive competition for food. This behavior is modeled using Eqs (14) and (15), where vultures exhibit aggressive movements towards the leading vulture.

$\displaystyle\textit{Position}_{\textit{Vulture}\left({i+1}\right)}=\textit{% Best}_{\textit{Vulture}_{i}}-\textit{Vulture}_{\textit{SRate}}+\textit{Rand}_{% 2}\times$ $\displaystyle(\left({\textit{Upper}_{\textit{bound}}-\textit{Lower}_{\textit{% bound}}}\right)\times\textit{Rand}_{3}+\textit{Lower}_{\textit{bound}})$ (12)

$\displaystyle\textit{Position}_{\textit{Vulture}\left({i+1}\right)}=\left\{{{% \begin{array}[]{l}{\textit{Equation}\left({10}\right)\textit{if}∼{}P_{2}% \geqslant\textit{Rand}_{p_{2}}}\\ {\textit{Equation}\left({13}\right)\textit{if}∼{}P_{2}<\textit{Rand}_{p_{2}}}% \end{array}}}\right.$ (13)

$\displaystyle\textit{Position}_{\textit{Vulture}\left({i+1}\right)}=D\left(i% \right)\times\left({\textit{Vulture}_{\textit{SRate}}+\textit{Rand}_{4}}\right% )-\textit{Vulture}_{\textit{Distance}}$ (14)

$\displaystyle\textit{Vulture}_{\textit{Distance}}=\textit{Best}_{\textit{% Vulture}_{i}}-\textit{Position}_{\textit{Vulture}\left(i\right)}$ (15)

Table 3

Mathematical notations, parameters, random numbers and their range of valoes used in AVO optimization equations

Mathematical representations	Descriptions	Mathematical representations	Descriptions
$\textit{Best}_{\textit{Vulture}_{1}}$ and $\textit{Best}_{\textit{Vulture}_{2}}$	First best and second-best solutions/vultures	$\textit{Probability}_{\textit{Vulture}_{i}}$	Probability of selecting the top vultures
$L_{1}$ and $L_{2}$ ; $P_{1},P_{2}$ and $P_{3}\textit{Parameter}_{u}$ and $\textit{Parameter}_{v}$ ranging between [0 $\sim$ 1]	Parameters	$\frac{\textit{Fitness}_{\textit{Vulture}_{i}}}{\sum_{\text{i}=1}^{\textit{Pop}% _{\textit{Vultures}}}\textit{Fitness}_{\textit{Vulture}_{i}}}$	Fitness value of vultures
$\textit{Phase}_{\textit{Transiton}}$	Transition from the exploration phase to the exploitation phase	$\textit{Vulture}_{\textit{SRate}}$	Satisfaction rate
$\textit{Rand}_{1}\left[{0\sim 1}\right],x\left[{-2\sim 2}\right],y\left[{-1% \sim 1}\right]$ $\textit{Rand}_{P_{1}}\left[{0\sim 1}\right],\textit{Rand}_{P_{2}}\left[{0\sim 1% }\right],$ $\textit{Rand}_{P_{2}}\left[{0\sim 1}\right],\textit{Rand}_{2}\left[{0\sim 1}% \right],$ $\textit{Rand}_{3}\left[{0\sim 1}\right],\textit{Rand}_{4}$ [0 $\sim$ 1] $\textit{Rand}_{5}$ [0 $\sim$ 1] and $\textit{Rand}_{6}$ [0 $\sim$ 1]	Random varibles and their ranges	$\textit{Pop}_{\textit{Vultures}}$	Vulture population
$\textit{Current}_{\textit{Iteration}_{i}}$ and $\textit{Maximum}_{\textit{Iterations}}$	Current itertaion and maximum iterations respectively	$D\left(i\right)$	One of the best vultures chosen through the utilization of Eq. ((a)) in the current iteration
CF	A coefficient vector that augments the random motion, varying in each iteration and computed using the formula $\textit{CF}=2\times$ rand	$\textit{Position}_{\textit{Vulture}\left(i\right)}$	Current vector position of a vulture
$\textit{Upper}_{\textit{bound}},\textit{Lower}_{\textit{bound}}$	Upper and lower ranges of variables	$\textit{Vulture}_{\textit{Distance}}$
$\textit{Spiral}_{1}$ and $\textit{Spiral}_{2}$	Spiral equation connecting all vultures with either the best vulture ( $\textit{Best}_{\textit{Vulture}_{1}})$ or the second-best vulture ( $\textit{Best}_{\textit{Vulture}_{2}})$	$\textit{Accumulation}_{1}$ and $\textit{Accumulation}_{2}$	Vector of the vulture’s position in the subsequent iteration
$d$	Problem dimensions	$\psi$	Fixed and default value of 1.5

Additionally, the AVO incorporates the concept of vulture diversity, where multiple vulture species can accumulate around a food source. Equations (4) and (20) model the movement of all vultures towards the food source, considering potential competition and starvation. The aggregation of vultures is calculated based on the best vultures in each group. Lastly, Eq. (21) captures the fierce competition between vultures for scarce food resources when the leading vultures become weak and starved. Vultures move towards the leading vulture in diverse directions, and the Levy flight mechanism is incorporated to enhance the effectiveness of the AVO. By iteratively following the equations, the AVO explores the search space, balances exploration and exploitation, and converges towards an optimal solution for the given optimization problem. The parameter values and equations used in each step influence the behavior and performance of the algorithm (depicted in Table 3).

$\displaystyle\textit{Spiral}_{1}=\textit{Best}_{\textit{Vulture}_{i}}\times% \left({\frac{\textit{Rand}_{5}\times\textit{Position}_{\textit{Vulture}\left(i% \right)}}{2\pi}}\right)\times{\cos}\left({\textit{Position}_{\textit{Vulture}% \left(i\right)}}\right)∼{}\text{and}∼{}$ $\displaystyle\textit{Spiral}_{2}=\textit{Best}_{\textit{Vulture}_{i}}\times% \left({\frac{\textit{Rand}_{6}\times\textit{Position}_{\textit{Vulture}\left(i% \right)}}{2\pi}}\right)\times{\sin}\left({\textit{Position}_{\textit{Vulture}% \left(i\right)}}\right)$ (16)

$\displaystyle\textit{Position}_{\textit{Vulture}\left({i+1}\right)}=\textit{% Best}_{\textit{Vulture}_{i}}-\left({\textit{Spiral}_{1}+\textit{Spiral}_{2}}\right)$ (17)

$\displaystyle\textit{Position}_{\textit{Vulture}\left({i+1}\right)}=\left\{{{% \begin{array}[]{l}{\textit{Equation}\left({16}\right)∼{}\textit{if}∼{}P_{3}% \geqslant\textit{Rand}_{p_{3}}}\hfill\\ {\textit{Equation}\left({17}\right)∼{}\textit{if}∼{}P_{3}<\textit{Rand}_{p_{3}% }}\hfill\\ \end{array}}}\right.$ (18)

$\displaystyle\textit{Accumulation}_{1}=\textit{Best}_{\textit{Vulture}_{1}}% \left(i\right)-\frac{\textit{Best}_{\textit{Vulture}_{1}}\left(i\right)\times% \textit{Position}_{\textit{Vulture}\left(i\right)}}{\textit{Best}_{\textit{% Vulture}_{1}}\left(i\right)\times\textit{Position}_{\textit{Vulture}\left(i% \right)}^{2}}\times\textit{Vulture}_{\textit{SRate}}∼{}\text{and}∼{}$ $\displaystyle\textit{Accumulation}_{2}=\textit{Best}_{\textit{Vulture}_{2}}% \left(i\right)-\frac{\textit{Best}_{\textit{Vulture}_{2}}\left(i\right)\times% \textit{Position}_{\textit{Vulture}\left(i\right)}}{\textit{Best}_{\textit{% Vulture}_{2}}\left(i\right)\times\textit{Position}_{\textit{Vulture}\left(i% \right)}^{2}}\times\textit{Vulture}_{\textit{SRate}}$ (19)

$\displaystyle\textit{Position}_{\textit{Vulture}\left({i+1}\right)}=\frac{% \textit{Accumulation}_{1}+\textit{Accumulation}_{2}}{2}$ (20)

$\displaystyle\textit{Position}_{\textit{Vulture}\left({i+1}\right)}=\textit{% Best}_{\textit{Vulture}_{i}}-\left|{\textit{Vulture}_{\textit{Distance}}}% \right|\times\textit{Vulture}_{\textit{SRate}}\times\textit{Levy}\left(d\right)$ (21)

$\displaystyle\textit{LF}\left(x\right)=0.01\times\frac{\textit{Parameter}_{u}% \times\sigma}{\left|{\textit{Parameter}_{v}}\right|^{\frac{1}{p}}},\sigma=% \left(\frac{\Gamma(1+\psi)\times\sin\left(\frac{\pi\psi}{2}\right)}{\Gamma(1+% \psi 2)\times\beta\times 2\left(\frac{\psi-1}{2}\right)}\right)^{\frac{1}{p}}$ (22)

The pseudocde of the propsoed feature fusion stregtegy incorporating the AVO algoirithm for wieght optimization to obtaing the attention weights is desribe in Algorithm 1.

Algorithm 1: Proposed ADGFF feature fusion based classificatio
Step 1. Split the dataset into training and testing sets
Step 2. Form the grids (4 horizontal and 4 vertical)
Step 3. Extract the features from each grid using pre-trained networks
Step 4. Train the model using extracted features using attention network
Step 5. Extract attention-based features
Step 6. Initiate optimization process
Initialize vulture positions (attention weights for attention features)
For $i=1∼{}\textit{to∼{}Maximum}_{\textit{Iterations}}$
Calculate fitness (MSE) of each vulture
Find Pbest1 for $\textit{Group}_{1}$
Find Pbest2 for $\textit{Group}_{2}$
For each individual
Calculate $\textit{Probability}_{\textit{Vulture}_{i}}$ (probability of choosing best solution) using Eq. (6)
Calculate $\textit{Best}_{\textit{Vulture}_{i}}$ using $\textit{Probability}_{\textit{Vulture}_{i}}$ using Eq. (5)
Calculate rate of $\textit{Vulture}_{\textit{SRate}}$ of vulture using Eq. (8)
If $\left\|{\textit{Vulture}_{\textit{SRate}}}\right\|\geqslant 1$
If $\textit{Probability}_{\textit{Vulture}_{i}}>\textit{Rand}_{P_{1}}$
Update location using Eq. (10)
Else
Update location using Eq. (4)
If end
If end
If $\left\|{\textit{Vulture}_{\textit{SRate}}}\right\|<1$
If ( $\left\|{\textit{Vulture}_{\textit{SRate}}}\right\|\geqslant 0.5)$
If $\textit{Probability}_{\textit{Vulture}_{i}}>\textit{Rand}_{2}$
Update location using Eq. (14)
Else
Update location using Eq. (17)
Else
If $\textit{Probability}_{\textit{Vulture}_{i}}>\textit{Rand}_{3}$
Update location using Eq. (20)
Else
Update location using Eq. (21)
If end
If end
If end
If end
End for
End for
Extract best individual $(\textit{weights}>0.5)$
Step 7. Perform classification and calculate performance

The AVO algorithm, inspired by vulture behavior, plays a crucial role in the proposed feature fusion models by optimizing solutions for complex optimization problems. Divided into four main steps, AVO navigates the search space efficiently, balancing exploration and exploitation to converge towards optimal solutions. Firstly, it identifies the best solutions within two groups based on fitness evaluations and adjusts vulture positions accordingly. Secondly, it models vulture behavior, accounting for declining satisfaction rates during optimization, which influences their movement patterns. Thirdly, it guides vultures through exploration phases, mimicking their search for food by exploring random areas using different strategies. Lastly, it facilitates exploitation phases, where vultures employ rotating flight and siege-fight strategies to converge towards optimal solutions. Through these steps, AVO dynamically adjusts vulture positions based on exploration and exploitation strategies, optimizing feature fusion models to enhance classification accuracy without explicit human intervention.

5. Experimentation and model analysis

In this study, we conducted a series of experiments to evaluate the performance of a computer system equipped with a 9th Gen Intel^® Cor $\texttrademark$ i3-9100 processor, featuring 4 cores, 6 MB cache, and up to 4.2 GHz with Inte^® Turbo Boost Technology. The system was configured with 8 GB DDR4 RAM and a 3.5" 1 TB 7200RPM SATA Hard Drive, running on the Windows 10 Pro 64-bit operating system.This section presents a comprehensive overview of the experiments conducted, and the subsequent analysis of the results validates the effectiveness of the proposed strategies.

5.1 Result analysis and validation

This section begins by conducting a direct comparison of classification accuracy between four pre-trained models and the GRU based on the mean squared error (MSE) evaluation metric. MSE was likely chosen as the evaluation metric for comparing classification accuracy between pre-trained models and the GRU classifier due to its simplicity, interpretability, and robustness. It offers a straightforward measure of prediction error, adaptable to classification tasks by quantifying the discrepancy between predicted probabilities and true class labels. This simplicity enables easy understanding and comparison of model performance, even among stakeholders with limited technical expertise. Moreover, MSE provides a balanced assessment of model performance across all classes, making it robust to class imbalance. Overall, its suitability lies in its ability to offer a clear, interpretable, and balanced evaluation of classification accuracy across different models. The comparison observed using GRU focuses on evaluating the proposed feature sets generated using UFF, AWDFF, FTDFF, and ADGFF. These results provide detailed information on various models, including their accuracies, sensitivities, specificities, and F1 scores. The goal is to gain deeper insights into the feature fusion strategies employed for achieving precise classification of the ADNI and LONI BMR datasets

Table 4
Performance comparison of pre-trained models with proposed feature fusion strategies for ADNI dataset

Pre-trained networks	ADNI dataset				LONI BMR dataset
and fusion strategies	Accuracy	Sensitivity	Specificity	F1 score	Accuracy	Sensitivity	Specificity	F1 score
Inception-ResNet-V2	0.9381	0.9387	0.9411	0.9462	0.9401	0.9321	0.9344	0.9334
VGG-16	0.9401	0.9311	0.9348	0.9457	0.9412	0.9354	0.9325	0.9348
ResNeXt-10	0.9421	0.9411	0.9349	0.9498	0.9421	0.9359	0.9344	0.9349
PNASNet-5	0.9358	0.9374	0.9399	0.9406	0.9458	0.9362	0.9361	0.9360
GRU-UFF	0.9431	0.9441	0.9498	0.9502	0.9488	0.9388	0.9391	0.9389
GRU-AWDFF	0.9487	0.9488	0.9489	0.9547	0.9491	0.9388	0.9398	0.9389
GRU-FTDFF	0.9488	0.9459	0.9491	0.9544	0.9491	0.9402	0.9388	0.9399
GRU-ADFF	0.9495	0.9489	0.9496	0.9552	0.9522	0.9424	0.9458	0.9436

The provided table (Table 4) presents the performance comparison of pre-trained models with the proposed feature fusion strategies for the ADNI and LONI BMR datasets, respectively, For the ADNI dataset, ResNeXt-10 achieved the highest accuracy of 0.9421, closely followed by VGG-16 with an accuracy of 0.9401. The GRU-ADFF demonstrated the highest sensitivity of 0.9489, while VGG-16 had the lowest sensitivity of 0.9311. In terms of specificity, GRU-UFF attained the highest value of 0.9498, while VGG-16 had the lowest specificity of 0.9348. GRU-ADFF also obtained the highest F1 score of 0.9552, indicating its overall strong performance. Particularly, GRU-ADFF achieved the highest accuracy of 0.9495 and showcased competitive performance across all evaluation metrics. Similarly, for the LONI BMR dataset, GRU-ADFF achieved the highest accuracy of 0.9522, closely followed by GRU-FTDFF and GRU-AWDFF, all-surpassing an accuracy of 0.949. GRU-ADFF also obtained the highest sensitivity of 0.9424, indicating its ability to correctly identify positive cases. GRU-ADFF and GRU-UFF exhibited the highest specificities of 0.9458 and 0.9391, respectively.Furthermore, GRU-ADFF displayed the highest F1 score of 0.9436, suggesting its overall strong performance across all evaluation metrics. Like the results from the ADNI dataset, the proposed feature fusion strategies consistently outperformed the individual pre-trained models in terms of the evaluated performance measures. It is observed that the proposed feature fusion strategies, including UFF, AWDFF, FTDFF, and ADFF, exhibited superior performance in both the ADNI and LONI BMR datasets based on GRU. The GRU-ADFF consistently demonstrated the highest accuracy, sensitivity, and F1 score, indicating its effectiveness in enhancing the classification performance for both datasets.Notably, across both datasets, the proposed feature fusion strategies consistently surpassed the performance of individual pre-trained models, underscoring their effectiveness in enhancing classification accuracy and demonstrating their potential for improving diagnostic outcomes in brain lesion classification tasks.

Table 5

Performance comparison of proposed optimized feature fusion strategies based on deep learning-based classifiers for ADNI and LONI BMR dataset

Proposed optimized feature	ADNI dataset			LONI BMR dataset
fusion strategies	CNN	RNN	GRU	CNN	RNN	GRU
FTDFF-GA	0.9591	0.9583	0.9602	0.9633	0.9633	0.9643
FTDFF-DE	0.9597	0.9594	0.9594	0.9656	0.9643	0.9659
FTDFF-AVO	0.9607	0.9598	0.9643	0.9696	0.9617	0.9701
ADFF-GA	0.9587	0.9564	0.9598	0.9663	0.9603	0.9654
ADFF -DE	0.9591	0.9544	0.9591	0.9627	0.9574	0.9610
ADFF -AVO	0.9599	0.9592	0.9599	0.9654	0.9652	0.9631
ADGFF-GA	0.9604	0.9588	0.9604	0.9658	0.9587	0.9602
ADGFF-PSO	0.9612	0.9601	0.9604	0.9699	0.9689	0.9682
ADGFF-AVO	0.9612	0.9621	0.9711	0.9702	0.9693	0.9788

After conducting comprehensive comparisons on both datasets, the proposed feature fusion strategies, namely AWDFF, FTDFF, and ADFF, have exhibited notably higher classification accuracy when integrated with the GRU classifier. To further enhance their effectiveness, an optimization effort is undertaken using the AVO algorithm, which leverages its metaheuristic capabilities. This optimization process involves fine-tuning the parameters (weights $w_{1},w_{2},w_{3}\text{and}w_{4}$ ) associated with AWDFF, FTDFF, and ADFF based on the AVO algorithm. Following this optimization, additional comparisons are conducted to assess the performance of the optimized versions of these feature fusion strategies, utilizing GA and DE optimization algorithms in conjunction with three deep learning-based classifiers, specifically CNN, RNN, and GRU. Detailed findings of these comparisons are presented in Table 5 for the ADNI and LONI BMR datasets.

The performance comparison of the proposed optimized feature fusion strategies for the ADNI and LONI BMR datasets respectively, as shown in Table 5, reveals that the GRU classifier consistently achieves the highest accuracies across various scenarios. In the ADFF-GA scenario, GRU achieves an accuracy of 0.9598, closely followed by CNN with 0.9587 and RNN with 0.9564. Similar trends are observed in other optimized strategies, such as ADFF-DE and ADFF-AVO. The FTDFF-GA strategy highlights GRU’s superior performance, achieving an accuracy of 0.9602, surpassing CNN and RNN. Furthermore, in FTDFF-AVO, GRU outperforms both CNN and RNN with the highest accuracy of 0.9643. Notably, in the ADFF-AVO case, GRU demonstrates exceptional performance with an accuracy of 0.9711, surpassing CNN and RNN. Turning to the performance comparison for the LONI BMR dataset, CNN consistently exhibits strong performance, attaining the highest accuracies in multiple scenarios. For instance, in the ADFF-GA scenario, CNN achieves an accuracy of 0.9663, while GRU closely follows with 0.9654. In the FTDFF-AVO scenario, GRU surpasses other classifiers with an accuracy of 0.9701, but CNN remains competitive at 0.9696. Additionally, CNN achieves the highest accuracy of 0.9699 in the ADGFF-PSO scenario. Noteworthy is the outstanding performance of GRU in the ADGFF-AVO scenario, where it outperforms CNN and RNN with an accuracy of 0.9788. These results underline the promising potential of the optimized feature fusion strategies to enhance classification performance, with GRU consistently demonstrating its effectiveness as a classifier in leveraging these strategies for both the ADNI and LONI BMR datasets.

Figure 2.

Learning progress of the proposed optimized feature fusion strategies obtained using GRU classifier for (a) ADNI dataset and (b) LONI BMR dataset.

Furthermore, the learning progress of the proposed optimized feature fusion strategies, namely AWDFF, FTDFF, and ADFF, obtained using optimized versions of the GRU classifier with GA, DE, and AVO, based on MSE are monitored and depicted in Fig. 2(a) and Fig. 2(b) for the ADNI and LONI BMR datasets, respectively. The primary objective of this analysis is to evaluate the performance of the feature fusion strategy ADFF in conjunction with the AVO method, utilizing the GRU classifier. Upon observing the figures, it is evident that the ADGFF-AVO strategy demonstrates convergence at approximately the 20^th iteration, while the remaining compared strategies begin converging around the 37^th iteration for both datasets. Subsequent sections will present the acquired results and provide a comprehensive evaluation of the effectiveness of this proposed strategy.

Figure 3.

Learning progress of the proposed AVO-based optimized feature fusion strategies obtained using GRU classifier for (a) ADNI and (b) LONI BMR datasets.

Furthermore, for a detailed examination of error decay measured in MSE specifically for the ADFF-AVO, FTDFF-AVO, and ADGFF-AVO feature fusion strategies using the GRU classifier, the learning progress is illustrated in Fig. 3(a) and Fig. 3(b) for the ADNI and LONI BMR datasets, respectively. Upon analysis, it is evident that the GRU model employing the ADGFF-AVO strategy converges around the 20^th iteration for both datasets. Conversely, the GRU classifier utilizing the FTDFF-AVO strategy converges around the 27^th iteration for the ADNI dataset and the 20^th iteration for the LONI BMR dataset. Similarly, the GRU classifier based on the ADFF-AVO strategy converges around the 41^st iteration for both datasets.

The AUC-ROC curves in Fig. 4(a) and Fig. 4(b) reveal insights into the discriminatory capabilities of the ADGFF-AVO strategy compared to other fusion methods for both the ADNI and LONI BMR datasets. A higher AUC-ROC value indicates better discrimination power, with 1 representing a perfect classifier. The ADGFF-AVO strategy consistently approaches an AUC-ROC value closer to 1, indicating stronger discriminatory capabilities compared to other strategies. This implies that ADGFF-AVO effectively distinguishes between brain lesion classifications, reducing false positives and negatives. The strategy’s robust feature fusion and optimization techniques contribute to its superior performance, enhancing diagnostic accuracy and patient outcomes.

Table 6

Five-fold test results of GRU-UFF for ADNI and LONI BMR dataset

ADNI dataset
#Round	TP	TN	FP	FN	ACC	SEN	SPC	AUC	PPV	NPV
1	98	82	13	2	0.9230	0.98	0.8631	0.9188	0.8828	0.9761
2	98	89	6	2	0.9333	0.98	0.9368	0.9720	0.9423	0.9780
3	98	82	13	2	0.9230	0.98	0.8631	0.9188	0.8828	0.9761
4	98	82	13	2	0.9230	0.98	0.8631	0.9188	0.8828	0.9761
5	98	83	12	2	0.9285	0.98	0.8736	0.9268	0.8909	0.9764
AVG	98	83.6	11	2	0.9261	0.98	0.8799	0.9310	0.8963	0.9765
SD	0.001	2.72	2.727	0.001	0.004	0.0001	0.0028	0.002	0.0023	0.0007
LONI BMR dataset
#Round	TP	TN	FP	FN	ACC	SEN	SPC	AUC	PPV	NPV
1	45	46	1	5	0.9381	0.9	0.9787	0.9105	0.9782	0.9019
2	45	46	1	5	0.9381	0.9	0.9787	0.9105	0.9782	0.9019
3	45	46	1	5	0.9381	0.9	0.9787	0.9105	0.9782	0.9019
4	48	46	1	2	0.9690	0.96	0.9787	0.9731	0.9795	0.9583
5	45	46	1	5	0.9381	0.9	0.9787	0.9105	0.9782	0.9019
AVG	45.6	46	1	4.4	0.9442	0.912	0.9787	0.9230	0.9784	0.9131
SD	1.20	0	0	1.20	0.0123	0.0240	0.00001	0.0250	0.0005	0.0225

Figure 4.

ROC of proposed optimized feature fusion strategies for (a) ADNI and (b) LONI BMR datasets.

The study employed five-fold cross-validation to assess the performance of the GRU model using five distinct feature fusion strategies, as indicated in Table 6 to Table 10 for GRU-UFF, GRU-AWDFF, GRU-FTDFF (optimized with AVO), GRU-ADFF (optimized with AVO), and GRU-ADGFF (optimized with AVO) concerning the ADNI and LONI BMR datasets respectively. By adopting this cross-validation approach, the research ensured a more robust estimation of the model’s performance in comparison to relying on a single train-test split. The model was assessed on different subsets of the data, which enabled the effective utilization of available data for both training and evaluation. Additionally, this multi-test set evaluation method mitigated the risk of overfitting to a specific train-test split.

The computation of various accuracy measures plays a crucial role in image classification. These measures, including true positive (TP), true negative (TN), false positive (FP), false negative (FN), accuracy (ACC), sensitivity (SEN), specificity (SPC), AUC, positive predicted value (PPV), and negative predicted value (NPV), provide valuable insights into the performance of the classification model. By analyzing these metrics, we can assess the model’s ability to correctly identify positive and negative instances, its overall accuracy, and its capacity to avoid false classifications. Additionally, computing the average (AVG) and standard deviation (SD) of these accuracy measures allows us to understand the consistency and reliability of the model’s predictions across different datasets or scenarios [41, 42]. These accuracy measuresaid in evaluating and fine-tuning image classification models, ensuring they deliver reliable and precise results.Furthermore, to establish the effectiveness of the proposed feature fusion-based classification models, the study utilized the well-known ensemble approach of majority voting. A straightforward comparison was performed between the proposed fusion approaches and the majority voting ensemble technique, and the results are presented in Table 11 for the ADNI and LONI BMR datasets, respectively.

Table 6 displays the five-fold test results of the GRU-UFF model for both the ADNI and LONI BMR datasets. In the case of the ADNI dataset, the model consistently achieved high accuracy, ranging from 92.30% to 93.33%, and maintained elevated sensitivity (86.31% to 93.68%) and specificity (91.88% to 97.20%). The AUC values consistently exceeded 0.91, indicating the model’s ability to discriminate effectively between positive and negative instances. Furthermore, the model demonstrated reliable PPV ranging from 88.28% to 94.23% and NPV ranging from 97.61% to 97.80%, underscoring its accuracy in providing precise predictions for both classes. Similarly, for the LONI BMR dataset, the GRU-UFF model consistently achieved high accuracy, ranging from 93.81% to 96.90%. Sensitivity ranged from 90% to 96%, while specificity remained stable at 97.87%, showcasing the model’s proficiency in classifying positive and negative instances. The average AUC of 92.30% reflects good discrimination ability. Moreover, the model demonstrated reliable PPV (90.19% to 97.95%) and NPV (90.19% to 95.83%), further affirming its accuracy in making predictions. The narrow range of small SD, between 0.001 to 0.004 and 0.0123 to 0.0240 for ADNI and LONI BMR datasets across different folds corroborates the model’s consistent performance and highlights its reliability and effectiveness for image classification.

Table 7

Five-fold test results of GRUAWDFF for ADNI and LONI BMR datasets

ADNI dataset
#Round	TP	TN	FP	FN	ACC	SEN	SPC	AUC	PPV	NPV
1	98	82	13	2	0.9230	0.98	0.8631	0.9188	0.8828	0.9761
2	98	83	12	2	0.9285	0.98	0.8736	0.9268	0.8909	0.9764
3	98	82	13	2	0.9230	0.98	0.8631	0.9188	0.8828	0.9761
4	98	83	12	2	0.9285	0.98	0.8736	0.9268	0.8909	0.9764
5	98	82	13	2	0.9230	0.98	0.8631	0.9188	0.8828	0.9761
AVG	98	82.4	12.6	2	0.9252	0.98	0.8673	0.922	0.8860	0.9762
SD	0.001	0.4898	0.4898	0.001	0.0002	0.0001	0.0051	0.0039	0.0039	0.0001
LONI BMR dataset
#Round	TP	TN	FP	FN	ACC	SEN	SPC	AUC	PPV	NPV
1	48	44	3	2	0.9484	0.96	0.9361	0.9541	0.9411	0.9565
2	48	44	3	2	0.9484	0.96	0.9361	0.9541	0.9411	0.9565
3	48	46	1	2	0.9690	0.96	0.9787	0.9703	0.9795	0.9583
4	48	44	3	2	0.9484	0.96	0.9361	0.9541	0.9411	0.9565
5	45	46	1	5	0.9381	0.9	0.9787	0.9105	0.9782	0.9019
AVG	47.4	44.8	2.2	2.6	0.9504	0.948	0.9531	0.9486	0.9562	0.9459
SD	1.2	0.9797	0.9797	1.2	0.0100	0.024	0.0208	0.0200	0.0184	0.0220

Table 7 presents the five-fold test results of the GRU-AWDFF model for both the ADNI and LONI BMR datasets. For ADNI, the model achieved consistently high accuracy (92.30% to 92.85%), sensitivity (86.31% to 87.36%), and specificity (91.88% to 92.68%). The AUC consistently exceeded 0.92, indicating its proficiency in distinguishing between positive and negative instances. The PPV (88.09% to 89.09%) and NPV (97.61% to 97.64%) were consistently high, showcasing the model’s reliability in generating accurate predictions for both classes. Similarly, for LONI BMR, the model exhibited an average accuracy of 95.04%, sensitivity (94.8%), and specificity (95.31%), with an average AUC of 0.9486, indicating good discrimination ability. The PPV (94.62%) and NPV (95.62%) further affirmed its reliability. The small SD for each metric, ranging from 0.0001 to 0.0051 and 0.0100 to 0.024, demonstrated the model’s consistent performance across different folds, reinforcing its reliability and effectiveness for neuroimaging data analysis on both datasets. The GRU-AWDFF model showed robustness and generalization capability, making it a promising approach for image classification tasks.

Table 8

Five-fold test results of GRUFTDFF (optimized with AVO) for ADNI and LONI BMR datasets

ADNI dataset
#Round	TP	TN	FP	FN	ACC	SEN	SPC	AUC	PPV	NPV
1	99	91	4	1	0.9487	0.99	0.9578	0.9724	0.9611	0.9891
2	99	91	4	1	0.9487	0.99	0.9578	0.9724	0.9611	0.9891
3	97	91	4	3	0.9384	0.97	0.9578	0.9711	0.9603	0.9680
4	97	91	4	3	0.9384	0.97	0.9578	0.9711	0.9603	0.9680
5	98	89	6	2	0.9333	0.98	0.9368	0.9720	0.9423	0.9780
AVG	98	90.6	4.4	2	0.9415	0.98	0.9536	0.9718	0.9570	0.9784
SD	0.8944	0.8000	0.8000	0.8944	0.0061	0.0089	0.0084	0.0058	0.0073	0.0094
LONI BMR dataset
#Round	TP	TN	FP	FN	ACC	SEN	SPC	AUC	PPV	NPV
1	48	46	1	2	0.9690	0.96	0.9787	0.9731	0.9795	0.9583
2	48	44	3	2	0.9484	0.96	0.9361	0.9541	0.9411	0.9565
3	48	46	1	2	0.9690	0.96	0.9787	0.9703	0.9795	0.9583
4	48	46	1	2	0.9690	0.96	0.9787	0.9731	0.9795	0.9583
5	48	46	1	2	0.9690	0.96	0.9787	0.9731	0.9795	0.9583
AVG	48	45.6	1.4	2	0.9648	0.96	0.9701	0.9687	0.9718	0.9579
SD	0	0.8000	0.8000	0	0.0082	0	0.0170	0.0073	0.0153	0.0007

Table 8 presents the five-fold test results of the GRU-FTDFF model (optimized with AVO) for both the ADNI and LONI BMR datasets. The model exhibited consistently high performance for the ADNI dataset, achieving an average accuracy of 94.15%, with sensitivity ranging from 97% to 99% and specificity consistently at 95.36%. The AUC consistently remained high at 0.9718, indicating its proficiency in distinguishing between positive and negative instances. Moreover, the PPV ranged from 95.70% to 96.11%, and the NPV consistently reached 97.84%, highlighting its reliability in generating accurate predictions for both classes. Similarly, for the LONI BMR dataset, the GRU-FTDFF model achieved an average accuracy of 96.48%, with sensitivity and specificity averaging at 96% and 97.02%, respectively. The average AUC of 0.9687 demonstrated good discrimination ability. Additionally, the PPV and NPV both averaged at 97.18%, affirming the model’s reliability in providing accurate predictions. The small SD for each metric, ranging from 0.0061 to 0.0094 and 0.0082 to 0.0170, further supported its consistent performance across different rounds. These results highlight the effectiveness and generalization capability of the AVO-optimizedGRU-FTDFF modelfor both ADNI and LONI BMR datasets.

Table 9

Five-fold test results of GRUADFF (optimized with AVO) for ADNI and LONI BMR datasets

ADNI dataset
#Round	TP	TN	FP	FN	ACC	SEN	SPC	AUC	PPV	NPV
1	98	89	6	2	0.9333	0.98	0.9368	0.9720	0.9423	0.9780
2	98	82	13	2	0.9230	0.98	0.8631	0.9188	0.8828	0.9761
3	98	89	6	2	0.9333	0.98	0.9368	0.9720	0.9423	0.9780
4	98	89	6	2	0.9333	0.98	0.9368	0.9720	0.9423	0.9780
5	98	89	6	2	0.9333	0.98	0.9368	0.9720	0.9423	0.9780
AVG	98	87.6	7.4	2	0.9312	0.98	0.9220	0.9613	0.9304	0.9776
SD	0.001	2.80	2.80	0.0001	0.0004	0.0001	0.0294	0.0212	0.0238	0.0007
LONI BMR dataset
#Round	TP	TN	FP	FN	ACC	SEN	SPC	AUC	PPV	NPV
1	45	46	1	5	0.9381	0.9	0.9787	0.9105	0.9782	0.9019
2	48	46	1	2	0.9690	0.96	0.9787	0.9703	0.9795	0.9583
3	48	46	1	2	0.9690	0.96	0.9787	0.9703	0.9795	0.9583
4	45	46	1	5	0.9381	0.9	0.9787	0.9105	0.9782	0.9019
5	48	46	1	2	0.9690	0.96	0.9787	0.9703	0.9795	0.9583
AVG	46.8	46	1	3.2	0.9566	0.936	0.9787	0.9463	0.9789	0.9357
SD	1.469	0.00	0.00	1.469	0.0151	0.0293	0.000001	0.0292	0.0006	0.0276

Table 9 presents the five-fold test results of the GRU-ADFF model (optimized with AVO) for both the ADNI and LONI BMR datasets. For the ADNI dataset, the model demonstrated variable performance across rounds, with an average accuracy of 93.12%. Sensitivity remained consistently high at 98%, while specificity showed some variability, averaging at 92.20%. The AUC consistently reached 0.9613, indicating the model’s proficiency in distinguishing between positive and negative instances. The PPV ranged from 93.04% to 94.23%, and the NPV consistently reached 97.76%, demonstrating the model’s reliability in generating accurate predictions for both classes. Similarly, for the LONI BMR dataset, the GRU-ADFF model achieved an average accuracy of 95.66%, with sensitivity averaging at 93.60% and specificity consistently high at 97.87%. The average AUC was 0.9463, indicating good discrimination ability. Additionally, the PPV and NPV both averaged at 97.89% and 93.57%, respectively, affirming the model’s reliability in providing accurate predictions. The small SD for most metrics, ranging from 0.0001 to 0.0294, supported the model’s consistent performance across different rounds. These results highlight the model’s effectiveness and potential for neuroimaging data analysis on both ADNI and LONI BMR datasets. However, the ADNI dataset showed slightly higher variability in specificity, potentially indicating the need for further improvement.

Table 10

Five-fold test results of GRU-ADGFF (optimized with AVO) for ADNI and LONI BMR datasets

ADNI dataset
#Round	TP	TN	FP	FN	ACC	SEN	SPC	AUC	PPV	NPV
1	99	91	4	1	0.9487	0.99	0.9578	0.9724	0.9611	0.9891
2	99	91	4	1	0.9487	0.99	0.9578	0.9724	0.9611	0.9891
3	99	91	4	1	0.9487	0.99	0.9578	0.9724	0.9611	0.9891
4	98	89	6	2	0.9333	0.98	0.9368	0.9720	0.9423	0.9780
5	99	91	4	1	0.9487	0.99	0.9578	0.9724	0.9611	0.9891
AVG	98.8	90.6	4.4	1.2	0.9456	0.98	0.9536	0.9723	0.9573	0.9868
SD	0.40	0.80	0.80	0.40	0.006	0.004	0.008	0.0001	0.0075	0.0044
LONI BMR dataset
#Round	TP	TN	FP	FN	ACC	SEN	SPC	AUC	PPV	NPV
1	49	46	1	1	0.9793	0.98	0.9787	0.9756	0.9800	0.9787
2	49	46	1	1	0.9793	0.98	0.9787	0.9756	0.9800	0.9787
3	48	46	1	2	0.9690	0.96	0.9787	0.9703	0.9795	0.9583
4	48	46	1	2	0.9690	0.96	0.9787	0.9731	0.9795	0.9583
5	49	46	1	1	0.9793	0.98	0.9787	0.9756	0.9800	0.9787
AVG	48.6	46	1	1.4	0.9751	0.972	0.9787	0.9740	0.9798	0.9705
SD	0.4898	0	0	0.489	0.0050	0.0097	0.0001	0.0021	0.0002	0.0099

Table 10 presents the five-fold test results of the GRU-ADGFF model (optimized with AVO) for both the ADNI and LONI BMR datasets. For the ADNI dataset, the model demonstrated consistently high performance, achieving an average accuracy of 94.56%. Sensitivity remained consistently high at 98%, while specificity showed some variability, averaging at 95.36%. The AUC consistently reached 0.9723, indicating the model’s proficiency in distinguishing between positive and negative instances. The PPV ranged from 95.73% to 98.68%, and the NPV consistently reached 98.68%, highlighting the model’s reliability in generating accurate predictions for both classes. Similarly, for the LONI BMR dataset, the GRU-ADGFF model achieved an average accuracy of 97.51%, with sensitivity averaging at 97.20% and specificity consistently high at 97.87%. The average AUC was 0.9740, indicating good discrimination ability. Additionally, the PPV and NPV both averaged at 97.98% and 97.05%, respectively, affirming the model’s reliability in providing accurate predictions. The small SD for most metrics, ranging from 0.0001 to 0.0099, supported the model’s consistent performance across different rounds. These results demonstrate the effectiveness and generalization capability of the GRU-ADGFF model (optimized with AVO) for both ADNI and LONI BMR datasets.

In general, theresults presented in Table 6 trough Table 10 highlight the effectiveness and potential of the GRU-UFF, GRU-AWDFF, GRU-FTDFF, GRU-ADFF and GRU-ADGFF models for neuroimaging data analysis, with some models showing better generalization capability across different datasets. However, the GRU-ADGFF is found to be more promising in comparison to the rest of the compared fusion approaches for both the datasets.

Table 11

Five-fold test results of majority voting for ADNI and LONI BMR datasets

ADNI dataset
#Round	TP	TN	FP	FN	ACC	SEN	SPC	AUC	PPV	NPV
1	95	80	15	5	0.8974	0.95	0.8421	0.8543	0.8636	0.9411
2	97	89	6	3	0.9538	0.97	0.9368	0.8911	0.9417	0.9673
3	98	81	14	2	0.9179	0.98	0.8526	0.8904	0.8750	0.9759
4	98	80	15	2	0.9128	0.98	0.8421	0.8901	0.8672	0.9756
5	95	80	15	5	0.8974	0.95	0.8421	0.8543	0.8636	0.9411
AVG	96.6	82	13	3.4	0.9158	0.966	0.8631	0.8760	0.8822	0.9602
SD	1.3564	3.5213	3.5213	1.3564	0.0206	0.0135	0.0370	0.0177	0.0300	0.0158
LONI BMR dataset
#Round	TP	TN	FP	FN	ACC	SEN	SPC	AUC	PPV	NPV
1	46	44	3	4	0.9278	0.92	0.9361	0.9154	0.9387	0.9166
2	49	45	2	1	0.9690	0.98	0.9574	0.9311	0.9607	0.9782
3	45	46	1	5	0.9381	0.90	0.9787	0.9014	0.9782	0.9019
4	44	45	2	6	0.9175	0.88	0.9574	0.8991	0.9565	0.8823
5	49	46	1	1	0.9793	0.98	0.9787	0.9412	0.9800	0.9787
AVG	46.6	45.2	1.8	3.4	0.9463	0.932	0.9616	0.9174	0.9628	0.9315
SD	2.0591	0.7483	0.7483	2.0591	0.0238	0.0411	0.0159	0.0164	0.0152	0.0398

Furthermore, another comparison has been made between theGRU-ADGFF and majority voting principle of feature fusion as presented in Table 11 for the five-fold test applied to ADNI and LONI BMR datasets. The GRU-ADGFF method demonstrates higher average accuracy (98.8% for ADNI and 48.6% for LONI BMR) compared to the majority voting method shown in Table 11 (96.6% for ADNI and 46.6% for LONI BMR). Similarly, the GRU-ADGFF method exhibits better sensitivity, specificity, and AUC values in both datasets compared to the majority voting method. However, the standard deviation values in Table 10 for GRU-ADGFF are generally higher than those in Table 11, indicating more variability in the performance of the GRU-ADGFF method. On the other hand, the majority voting method shows more consistent results with lower standard deviation values and the GRU-ADGFF method outperforms the majority voting method in terms of accuracy and discriminative ability.

After conducting a series of experiments and comparisons with pre-trained networks and proposed feature fusion strategies, it has been observed that the ADGFF feature fusion strategy, optimized with AVO, shows promising results when combined with the GRU classifier for both datasets in this study. To validate its effectiveness, a two-stage evaluation is conducted, involving the recording of execution time and a non-parametric statistical test using Friedman’s rank test. Evaluating the execution time performance of the proposed classification models is essential as it provides valuable insights into their efficiency and practical applicability. Faster execution times indicate better scalability and resource utilization, allowing the model to handle large datasets and real-time scenarios more effectively. In Fig. 5(a) and (b), we present the average training times recorded in minutes for the optimized versions of FTDFF, ADFF, and ADGFF based on GA, DE, and AVO for both datasets. Notably, FTDFF-AVO and ADGFF-AVO demonstrate remarkably similar execution times of (110.61/111.91) and (94.91/94.24) minutes, respectively, for the ADNI and LONI BMR datasets. Although the proposed grid-based approach takes slightly more time to execute compared to FTDFF-AVO, the accuracy achieved is significantly higher than that of the other models. This trade-off in execution time is justifiable due to the substantial improvement in classification accuracy.

Table 12

Results of the Friedman statistical test

Methods	ADFF- GA	ADFF- DE	ADFF- AVO	FTDFF- GA	FTDFF- DE	FTDFF- AVO	ADGFF- GA	ADGFF- DE	ADGFF- AVO
Average rank	7.13	5.47	5.46	4.23	4.91	4.88	3.19	3.22	2.58

Figure 5.

The average training time (in minutes) of proposed optimized feature fusion strategies observed for (a) ADNI and (b) LONI BMR datasets.

Here, the Friedman test [43, 44] is used to compare multiple classifiers based on the multiple feature fusion strategies utilizing GRU across both the datasets, to assess whether there are significant differences in the performance of different classifiers on those datasets. The test is performed by setting up the null hypothesis (H0), which states that there is no significant difference in the classifiers’ performance across the datasets. The alternative hypothesis (Ha) asserts that there are significant differences among classifiers. Then, for each classifier, the datase’s performances are ranked from best to worst (or vice versa) based on accuracy, and the average ranks are calculated for each classifier, considering its performance across those datasets. This test computes a test statistic that considers the ranks and the number of datasets and classifiers, following the chi-square distribution. The $p$ -value associated with the test statistic is obtained from the chi-square distribution table or through simulation, representing the probability of observing the obtained results (or more extreme) under the assumption that the null hypothesis is true. If the $p$ -value is less than a pre-defined significance level (commonly set at 0.05), the null hypothesis is rejected, indicating that there are significant differences in classifier performance across the datasets. Looking at Table 12, we can observe the average ranks of different methods on various feature fusion strategies (ADFF, FTDFF, ADGFF) and optimization algorithms (GA, DE, AVO). Among the feature fusion strategies, ADGFF with AVO (average rank: 2.58) stands out as the top-performing method, with the lowest average rank among all combinations. It is closely followed by FTDFF with AVO (average rank: 3.19) and FTDFF with DE (average rank: 3.22), both of which also show competitive performance. In comparison, ADFF feature fusion strategy without any optimization methods (ADFF-GA, ADFF-DE, ADFF-AVO) appears to have higher average ranks (ranging from 4.23 to 7.13), indicating relatively poorer performance when compared to the other fusion strategies. When considering the optimization algorithms, AVO exhibits consistent success across all feature fusion strategies, helping to achieve top-performing models. On the other hand, GA and DE seem to have a more mixed impact, with GA performing better than DE in most cases. Based on these average ranks, it is evident that the GRU with ADGFF-AVO feature fusion strategy is the most suitable feature fusion approach for the given classification task. It consistently outperforms other combinations in terms of classification performance across the datasets.

6. Research timelin

Over 10 months, from October 2022 to July 2023, this research focused on brain lesion classification using MRI data. The study began with a literature review in October 2022, followed by formulating research questions and designing five feature fusion strategies, which were optimized with the AVO algorithm. Data collection started in January 2023, and meticulous curation and preprocessing of MRI data occurred until March 2023. Extensive experiments and comparisons between proposed strategies, including data analysis, took place in April and May 2023. The manuscript drafting started in May and continued until June 2023, with subsequent months dedicated to revisions based on valuable feedback from co-authors and peer reviewers. This research timeline illustrates the systematic approach undertaken to develop and optimize the feature fusion strategies for brain lesion classification using MRI data. The study involved various stages, from the initial literature review to the final manuscript preparation and represents a significant contribution to the field of medical image analysis

7. Expected contributions, impact and limitations observe

The research aims to illuminate the transformative prospects and potential challenges inherent in the integration of cognitive interface technology with the study of brain lesions, enriching the understanding of their collective impact on the evolving landscape of BCI, including.

(a)
Novel feature fusion strategy: The manuscript introduces the ADGFF strategy, tailored for brain lesion classification using MRI data. By efficiently combining multiple pre-trained networks, leveraging attention mechanisms, and grid-based representation of data, the ADGFF enhances classification performance, promising greater accuracy and robustness.
(b)
Optimization with AVO algorithm: The ADGFF strategy is optimized using the AVO algorithm, fine-tuning the fusion process for superior performance. This commitment to state-of-the-art techniques extends the potential applicability to other medical image analysis tasks.
(c)
Enhanced brain lesion classification: Applying the ADGFF strategy optimized with AVO to MRI data is expected to improve accuracy and robustness compared to other methods. This advancement benefits medical professionals and enhances automated brain lesion detection and classification, leading to improved patient care.
(d)
Efficiency: ADGFF not only enhances classification performance but also exhibits improved efficiency in execution time and statistical significance.
(e)
Generalizability and adaptability: The ADGFF strategy’s generalizability and adaptability extends its practical utility to other medical imaging tasks and diverse classification problems beyond brain lesion analysis.
(f)
Advancement in medical image analysis: This manuscript’s contributions have the potential to advance medical image analysis and computer-aided diagnosis, paving the way for further research and innovations, and benefiting both researchers and medical practitioners.

The GRU-ADGFF model, optimized with the AVO algorithm, consistently outperforms traditional CNN and RNN classifiers due to several factors. First, the GRU architecture handles sequential data and captures temporal dependencies, crucial for accurately classifying complex brain lesion images. Unlike CNNs, which excel at spatial feature extraction, GRUs retain long-sequence information, making them better for contextual analysis. Second, the ADGFF strategy enhances feature representation by dividing images into horizontal and vertical grids, enabling multi-scale analysis and effective lesion detection. The attention mechanisms selectively emphasize informative features, improving feature quality. Third, integrating multiple pre-trained networks (Inception-ResNet-V2, VGG-16, ResNeXt-10, and PNASNet-5) provides a richer and more diverse feature set, leading to more accurate and robust classification. Lastly, the AVO algorithm optimizes model parameters, ensuring better convergence and avoiding local minima, thus improving accuracy and generalization. These factors collectively contribute to the GRU-ADGFF model’s superior performance in brain lesion classification tasks.
8. Conclusion and future scop

Brain lesions present a formidable global health challenge, significantly affecting individuals and their overall well-being. The accurate and timely classification of these lesions plays a crucial role in effective diagnosis and treatment planning. While machine learning and deep learning, particularly CNNs, have demonstrated promise in classifying brain lesions using MRI images, persistent challenges include lesion heterogeneity, limited training data, and inter-observer variability. To overcome these hurdles, advanced feature fusion models become essential. This study’s primary objective was to develop a feature fusion model for precise brain lesion classification in MRI images, leveraging attention-driven grid feature fusion to capture comprehensive lesion representations. The optimized model introduced in this research enhances overall classification performance and provides fine-grained control over the importance of different image regions, significantly improving diagnostic capabilities. The grid-based partitioning technique in the ADGFF model significantly enhances the analysis of brain lesions at multiple scales and improves classification accuracy through a detailed and localized approach. By dividing the input brain MRI images into horizontal and vertical grids, the model can focus on smaller, localized regions, which allows for the detection and analysis of lesions of varying sizes and shapes. This multi-scale analysis ensures that even small or region-specific lesions are effectively captured. The technique facilitates detailed feature extraction within each grid, enabling the model to highlight subtle and fine details crucial for accurate lesion classification. Integrating the attention mechanism with the grid-based structure allows the model to dynamically emphasize the most relevant and informative regions, enhancing overall robustness and precision. Additionally, the grid-based approach maintains the spatial context of features, preserving anatomical relationships essential for distinguishing different types of brain lesions. This spatial awareness helps in capturing structural nuances, making the model more resilient to variability in lesion presentation across different patients. After extracting features from each grid, the ADGFF model fuses these features, leveraging the strengths of multiple pre-trained networks to create a comprehensive and highly discriminative feature set. This fusion process, combined with the detailed and context-aware analysis, results in a robust and accurate classification of brain lesions, demonstrating the significant advantages of the grid-based partitioning technique. During the optimization of the GRU-ADGFF model with the AVO algorithm, several challenges were encountered. One major issue is overfitting, where the model becomes too tailored to the training data, reducing its generalizability. This can be mitigated with regularization techniques like dropout, L2 regularization, and data augmentation. Another challenge is the computational complexity due to integrating multiple pre-trained networks and grid-based partitioning, leading to longer training times and higher resource requirements. Strategies such as parallel processing, distributed computing, and efficient weight update algorithms can help, along with optimizing AVO hyperparameters. Additionally, dynamic adjustment of attention weights can cause instability, which can be addressed by tuning the attention mechanism’s parameters, using learning rate scheduling, gradient clipping, and early stopping. Variability and quality of input MRI images also pose challenges, which can be mitigated through preprocessing steps like normalization, denoising, and alignment. Implementing these strategies collectively enhances the model’s performance, stability, and generalizability.

References

Cao

Zhang

Wang

Zhang

Han

Huang

Kang

. End-to-end automatic pathology localization for alzheimer’s disease diagnosis using structural mri. Computers in Biology and Medicine. 2023 Sep 1; 163: 107110.

Yao

Wang

Yan

Wang

Zhang

Wang

Zhang

. Artificial intelligence-based diagnosis of Alzheimer’s disease with brain MRI images. European Journal of Radiology. 2023 Jun; 20: 110934.

Khojaste-Sarakhsi

Haghighi

Ghomi

Marchiori

. Deep learning for Alzheimer’s disease diagnosis: A survey. Artificial Intelligence in Medicine. 2022 Aug 1; 130: 102332.

Erdogmus

Kabakus

. The promise of convolutional neural networks for the early diagnosis of the Alzheimer’s disease. Engineering Applications of Artificial Intelligence. 2023 Aug 1; 123: 106254.

Xin

Wang

Guo

Liu

Tang

. CNN and swin-transformer based efficient model for Alzheimer’s disease diagnosis with sMRI. Biomedical Signal Processing and Control. 2023 Sep 1; 86: 105189.

Sharma

Guleria

Tiwari

Kumar

. A deep learning based convolutional neural network model with VGG16 feature extractor for the detection of Alzheimer Disease using MRI scans. Measurement: Sensors. 2022 Dec 1; 24: 100506.

Tan

Miao

Liang

Gong

Jiao

Zhang

Xing

. Attention-based and micro designed EfficientNetB2 for diagnosis of Alzheimer’s disease. Biomedical Signal Processing and Control. 2023 Apr 1; 82: 104571.

Lin

Qiao

Zhuang

Zhang

. Alzheimer’s disease diagnosis via multimodal feature fusion. Computers in Biology and Medicine. 2022 Sep 1; 148: 105901.

Zhang

Shi

. Multi-modal neuroimaging feature fusion for diagnosis of Alzheimer’s disease. Journal of Neuroscience Methods. 2020 Jul 15; 341: 108795.

10.

Rana

Bhushan

. Classifying breast cancer using transfer learning models based on histopathological images. Neural Computing and Applications. 2023 Jul; 35(18): 14243-57.

11.

Tan

Miao

Liang

Gong

Jiao

Zhang

Xing

. Attention-based and micro designed EfficientNetB2 for diagnosis of Alzheimer’s disease. Biomedical Signal Processing and Control. 2023 Apr 1; 82: 104571.

12.

Zhao

Huang

Chen

Yuan

Zhong

Pun

Huang

. IDA-Net: inheritable deformable attention network of structural MRI for Alzheimer’s disease diagnosis. Biomedical Signal Processing and Control. 2023 Jul 1; 84: 104787.

13.

Liu

Cui

Luo

Guo

Zhao

Wang

Subasi

Dogan

Tuncer

. Artificial intelligence-based breast cancer diagnosis using ultrasound images and grid-based deep feature generator. International Journal of General Medicine. 2022 Mar; 12271-82.

14.

Alanazi

Fathy

Yousri

Rezk

. Optimal reconfiguration of shaded PV based system using African vultures optimization approach. Alexandria Engineering Journal. 2022 Dec 1; 61(12): 12159-85.

15.

Ait-Saadi

Meraihi

Soukane

Yahia

Ramdane-Cherif

Gabis

. An enhanced African vulture optimization algorithm for solving the unmanned aerial vehicles path planning problem. Computers and Electrical Engineering. 2023 Sep 1; 110: 108802.

16.

Shanmugam

Duraisamy

Simon

Bhaskaran

. Alzheimer’s disease classification using pre-trained deep networks. Biomedical Signal Processing and Control. 2022 Jan 1; 71: 103217.

17.

Odusami

Maskeliūnas

Damaševičius

. An intelligent system for early recognition of Alzheimer’s disease using neuroimaging. Sensors. 2022 Jan 19; 22(3): 740.

18.

Poola

. COVID-19 diagnosis: A comprehensive review of pre-trained deep learning models based on feature extraction algorithm. Results in Engineering. 2023 Mar 16; 101020.

19.

Alsharabi

Shahwar

Rehman

Alharbi

. Implementing magnetic resonance imaging brain disorder classification via AlexNet–quantum learning. Mathematics. 2023 Jan 10; 11(2): 376.

20.

Raza

Naseer

Tamoor

Zafar

. Alzheimer disease classification through transfer learning approach. Diagnostics. 2023 Feb 20; 13(4): 801.

21.

Asgharzadeh-Bonab

Kalbkhani

Azarfardian

. An alzheimer’s disease classification method using fusion of features from brain magnetic resonance image transforms and deep convolutional networks. Healthcare Analytics. 2023 Dec 1; 4: 100223.

22.

Liu

Sun

Zhao

Jiang

. Deep feature fusion classification network (DFFCNet): Towards accurate diagnosis of COVID-19 using chest X-rays images. Biomedical Signal Processing and Control. 2022 Jul 1; 76: 103677.

23.

Haq

Ali

Wang

Lei

Ali

. Feature fusion and Ensemble learning-based CNN model for mammographic image classification. Journal of King Saud University-Computer and Information Sciences. 2022 Jun 1; 34(6): 3310-8.

24.

Chen

Yue

. Deep evidential fusion network for medical image classification. International Journal of Approximate Reasoning. 2022 Nov 1; 150: 188-98.

25.

Zhou

Sun

Tian

Xiao

Wang

Zhou

. RFIA-Net: Rich CNN-transformer network based on asymmetric fusion feature aggregation to classify stage I multimodality oesophageal cancer images. Engineering Applications of Artificial Intelligence. 2023 Feb 1; 118: 105703.

26.

Cao

Wang

Wan

. Fully feature fusion based neural network for COVID-19 lesion segmentation in CT images. Biomedical Signal Processing and Control. 2023 Sep 1; 86: 104939.

27.

www.loni.ucla.edu/.

28.

https://resource.loni.usc.edu/.

29.

Cui

Liu

, Alzheimer’s Disease Neuroimaging Initiative. RNN-based longitudinal analysis for diagnosis of Alzheimer’s disease. Computerized Medical Imaging and Graphics. 2019 Apr 1; 73: 1-0.

30.

Abuhmed

El-Sappagh

Alonso

. Robust hybrid deep learning models for Alzheimer’s progression detection. Knowledge-Based Systems. 2021 Feb 15; 213: 106688.

31.

Pan

Dai

Fan

Huang

Mei

. GRU with spatial prior for hyperspectral image classification. InIGARSS 2019–2019; IEEE International Geoscience and Remote Sensing Symposium. 2019 Jul 28; 967-970. IEEE.

32.

Wen

Gao

. A new genetic algorithm based evolutionary neural architecture search for image classification. Swarm and Evolutionary Computation. 2022 Dec 1; 75: 101191.

33.

Yang

Wang

Zhao

Heidari

Chen

Algarni

Elmannai

. Multi-level threshold segmentation framework for breast cancer images using enhanced differential evolution. Biomedical Signal Processing and Control. 2023 Feb 1; 80: 104373.

34.

McNeely-White

Beveridge

Draper

. Inception and ResNet features are (almost) equivalent. Cognitive Systems Research. 2020 Jan 1; 59: 312-8.

35.

Chen

Lin

Ding

Zeng

Liu

Xie

Huang

. Classification of lungs infected COVID-19 images based on inception-ResNet. Computer methods and programs in biomedicine. 2022 Oct 1; 225: 107053.

36.

Goyal

Dhull

Singh

Kukreja

Singh

. VGG-COVIDNet: A Novel model for COVID detection from X-Ray and CT Scan images. Procedia Computer Science. 2023 Jan 1; 218: 1926-35.

37.

Kumari

Rao

Reddy

. Design of hybrid dental caries segmentation and caries detection with meta-heuristic-based ResneXt-RNN. Biomedical Signal Processing and Control. 2022 Sep 1; 78: 103961.

38.

Yadav

Jalal

Garlapati

Hossain

Goyal

Pant

. Deep learning-based ResNeXt model in phycological studies for future. Algal research. 2020 Sep 1; 50: 102018.

39.

Yang

Guo

Lyu

. Detection and classification of damaged wheat kernels based on progressive neural architecture search. Biosystems Engineering. 2021 Aug 1; 208: 176-85.

40.

Guérin

Thiery

Nyiri

Gibaru

Boots

. Combining pretrained CNN feature extractors to enhance clustering of complex natural images. Neurocomputing. 2021 Jan 29; 423: 551-71.

41.

Tsokani

Veroniki

Pandis

Mavridis

. Assessing the performance of diagnostic test accuracy measures. American Journal of Orthodontics and Dentofacial Orthopedics. 2022 May; 161(5): 748-51.

42.

Carbonero-Ruz

Martínez-Estudillo

Fernández-Navarro

Becerra-Alonso

Martínez-Estudillo

. A two dimensional accuracy-based measure for classification performance. Information Sciences. 2017 Mar 1; 382: 60-80.

43.

https://www.statisticshowto.com/friedmans-test/.

44.

https://www.york.ac.uk/depts/maths/tables/friedman.pdf.

Enhancing alzheimer’s diagnosis through optimized brain lesion classification in MRI with attention-driven grid feature fusion

Abstract

Keywords

1. Background and significance

2. Literature reviews

3. Methodolog

3.1 Dataset acquisitio

Table 1 Description of datasets

Table 2 Parameters and their selected values used for experimentation

3.4 Unleashing the power of the proposed feature fusion strategies in image classification

5.1 Result analysis and validation

Table 4 Performance comparison of pre-trained models with proposed feature fusion strategies for ADNI dataset

7. Expected contributions, impact and limitations observe

References

Table 1
Description of datasets

Table 2
Parameters and their selected values used for experimentation

Table 4
Performance comparison of pre-trained models with proposed feature fusion strategies for ADNI dataset