Anomaly detection in electrocardiogram signals using metaheuristic optimized time-series classification with attention incorporated models

Abstract

Efforts in cardiovascular disorder detection demand immediate attention as they hold the potential to revolutionize patient outcomes through early detection systems. The exploration of diseases and treatments, coupled with the potential of artifical intelligence to reshape healthcare, highlights a promising avenue for innovation. AI-driven early detection systems offer substantial benefits by improving quality of life and extending longevity through timely interventions for chronic diseases. The evolving landscape of healthcare algorithms presents vast possibilities, particularly in the application of metaheuristics to address complex challenges. An exemplary approach involves employing metaheuristic solutions such as PSO, FA, GA, WOA, and SCA to optimize an RNN for anomaly detection using ECG systems. Despite commendable outcomes in the best and median case scenarios, the study acknowledges limitations, focusing on a narrow comparison of optimization algorithms and exploring RNN capabilities for a specific problem. Computational constraints led to the use of smaller populations and limited rounds, emphasizing the need for future research to transcend these boundaries. Significantly, the introduction of attention layers emerges as a transformative element, enhancing neural network performance. The introduced optimizer proves robust across test scenarios, effectively navigating local minimum traps. Attention layers contribute to a substantial performance boost, reducing the error rate from 0.006837 to an impressive 0.002486, underscoring their role in focusing on pertinent information. This abstract advocates for further research to expand beyond these limitations, exploring novel algorithms and addressing broader medical challenges in the pursuit of refined and advanced solutions.

Keywords

Time-series classification attention mechanism particle swarm optimization ECG RNN

1. Introduction

The leading cause of death globally is cardiovascular conditions. Lack of physical activity, diet irregularities, stress, age, as well as mostly alcohol and tobacco are the factors that affect cardiovascular health the most [1]. Health conditions related to such diseases usually develop over longer periods of time and are easy to miss, which results in severe conditions. While conditions take a while to begin deteriorating patient’s standard of living, many clinical indicators can be used to detect developing heart conditions. Increased heart pressure, irregular heart rates, loss of breath, and chronic fatigue are just some of the indicators. However, these conditions are not always caused by heart issues and can point to other conditions as well. Therefore a direct way of monitoring heart electrical activity is often used for diagnosis.

With the advancement of artificial intelligence (AI) other fields received many improvements through the use of such techniques. The smartification of devices has led to a boom in data which was one of the key drawbacks for stable predictors, as data was scarce before this. Not only did the use of computing units and memory in more devices benefit the AI field but the connection of these devices as well. This behavior results in higher quality of data as it allows for a combination of the data from these devices making for more accurate predictions. The described phenomena has been related to the Fourth Industrial Revolution (Industry 4.0), which includes many different fields among which the healthcare field is recognized as Healthcare 4.0. The equipment used by healthcare specialists collects large amounts of data on its own and with Industry 4.0 affecting all fields the case is not different with healthcare. Smart hospitals are being built with smart infrastructure, which improves the daily tasks of healthcare professionals. Furthermore, such infrastructure collects a lot of data which needs to be utilized in the right way. The possibilities are endless as scenarios like smart rooms are being evaluated by researchers [2], as well as more public datasets of specific reading like the gait measurements in Parkinson’s disease prediction [3]. AI demonstrates great potential for the improvement of patients’ treatments. With powerful insights gained through such methods, healthcare staff can adapt and change the treatment which would previously be impossible because a human is not capable of recognizing patterns as AI techniques. Additionally, pattern recognition by AI is at its best when coupled with constant monitoring of patients inside a smart hospital room. With the utilities offered in such healthcare systems, chances and the time for recovery should be improved. The advantages of constant monitoring were previously inaccessible by all individuals due to the financial limitations, but even with high financial power, no patient was able to take advantage of their data before the introduction of AI into healthcare.

AI can not only improve the chances of recovery but also the quality of life for certain chronic disease patients. By applying pattern recognition patients with movement, sight, hearing, and other impairments can greatly benefit. The variety of systems that take advantage of AI is vast and with the advancements in the technology of augmented reality (AR), the possibilities are further increased. The current direction of AI advancements puts an accent on human usability and the evolution of computers. AR kits are becoming more powerful and affordable which will soon result in even more powerful devices and in even smaller form factors. Humans have begun a new step in their evolution by slowly merging with the technologies. Human augmentation began even before the first subject received a Neuralink [4] cranial implant. Less sophisticated techniques in comparison to Neuralink include the implementation of RFID chips in human’s hand for the ease of use of their smart homes. While the risks are still to be observed over long-term use, humans seem eager for such changes in their bodies. Neuralink is an interface created with the purpose of restoring sensory and motor functions to patients but as of recently commercial use seems promising. While a large percentage of the population is concerned with putting trust in their own thoughts to a single company the benefits of such a solution are undeniable. This could be the next big step for mankind as it would allow humans to dive into cyberspace and live with their data.

While it is unknown what the future will hold it is certain that the sensors in medical field play a key role in their treatment. Personal diagnostics systems do not seem like a distant future, but what is left unanswered is the role of doctors. Will the doctors be completely replaced or will there still be a need for a human to supervise these systems? Would a patient trust a robot doctor or nurse who is to provide them with physical treatment for wounds? These are the key questions that will affect the future of the human population and it is paramount that such advancements are thoroughly planned and heavily controlled before released to the general public. For this reason, this research aims to expand the electrocardiograph research with an exhaustive search for the best technique for anomaly prediction in the ECGs. The focus is on early detection as that is one of the key factors in heart condition treatment.

The diseases of the heart are often regarded as silent killers as in most cases the symptoms are scarce. This leads to individuals being careless with what they put in their systems and how they take care of their bodies. Additionally, many refuse treatment because they do not take cardiovascular health seriously. With such individuals more precise detection mechanism and more specific in terms of the root of the problem could bring benefits and possibly better treatment if at all. Furthermore, the sensor systems previously described could help in maintaining good cardiovascular health. The patient’s activity can be tracked considering their physical activity and movement which largely affects the health of the cardiovascular system. Moreover, greater insight into how they prepare food from smart devices in their houses and possible precise nutritional value scanning of foods in the future could bring even more benefits.

The monitoring of the heart’s electrical signals is achieved through the utilization of an electrocardiogram (ECG), a widely recognized technique in the field of cardiology [5]. This system is comprised of an array of 16 strategically placed sensors distributed across the body, with the majority of sensors strategically positioned on the chest region. This comprehensive arrangement enables the accurate and efficient monitoring of not only the heart’s rhythm but also the identification of various cardiac conditions. To decipher and interpret the data obtained from these sensors, specialized expertise in the field of cardiology is indispensable, as the most valuable insights are often gleaned from the visual representations of the ECG readings.

In recent times, modern AI techniques have evolved to provide a novel approach to analyzing ECG time-series data [6]. Inspired by the intricacies of the human brain, a subset of AI known as neural networks has garnered significant attention [7, 8]. The distinctive upper hand of employing neural networks relies on their capacity to extrapolate novel insights and information from the existing ECG data. This innovation represents a promising avenue for enhancing our understanding of cardiac health and facilitating early detection of abnormalities. The advances were noted in other fields of medicine as well [9, 10, 11].

Despite excellent capabilities, AI-based solutions have a negative characteristic that requires the use of external optimization algorithms [12, 13]. These algorithms are employed out of a need to provide the best subset of parameter settings for the required solution [14, 15]. Mostly, such frameworks tend to optimize for general performance [16]. Every problem requires a different subset hence the need for an external algorithm that can do this more precisely and efficiently [17]. In this work, a recurrent neural network (RNN) is applied for time series predictions for detecting anomalies from ECG reading [18] which is then optimized by the particle swarm optimization (PSO) swarm metaheuristic. The research’s focal point is set to construct simpler models containing less layers and units in every layer.

Swarm intelligence represents a subfield of machine learning focused on nature-inspired population-based stochastic algorithms. Such solutions take inspiration from phenomena in nature that can be translated into a population that represents a swarm. The name comes from the first algorithms in this group which include insects and animals as inspiration. The key traits that defined which animals were suitable for these algorithms included way of behaving where the specimens work together in direction of a common objective unreachable on their own. These include search for food, path optimization, search for shelter, hunting, movement, and many other principles. This subfield of ML has provided solutions of high performance that have proven excellent as optimizers. Such optimizers can be applied to other ML techniques for hyperparameter optimization which can yield to models of very high accuracy. Various combinations exist but the SI algorithms shine truly when coupled with an algorithm that improves their deficiency. To elaborate, SI algorithms usually favor one of their two phases which are exploration and exploitation. During the first stage, the algorithm searches the realm of solutions globally with the goal of narrowing down the search to a smaller area. When it is successful in this task, the phase of exploitation begins in which the algorithm focuses on the area solely. The issue with SI solutions is that most of them are significantly better in one of the two, and hence comes the solutions of hybridization. Many forms of modifying algorithms through hybridization exist and the extent of this research cannot cover all of them. The most common form of this process would be an incorporation of a search strategy from a different algorithm and some sort of parameter that can control the alterations between these two search phases. With this principle, the highest performance from SI solutions can be extracted, as well as from other ML techniques that are optimized through such solutions.

This work represents an extension of the previously described research [19]. The results of the original research indicated the high potential of the proposed hybrid swarm metaheuristic of PSO and firefly algorithm (FA) for RNN optimization for time series prediction of cardiovascular diseases. For this reason, further ways of improving the proposed solution are explored. Attention-based RNNs have proven to provide performance increases to the time series type of problem as such mechanism allows for focusing on more important features which improves accuracy as a result. The experiments are repeated for the metaheuristics coupled with attention-based RNNs utilized for comparing the performance of different models. This approach has been taken due the reason that the potential of attention-calculated weights has not been explored for the problem at hand.

The following encapsulates the most significant contributions of the research:

•
The RNN potential is explored for the case of ECG anomaly detection.
•
The use of attention-based RNNs is explored for further possible improvements of the model.
•
An introduction of an altered variant of the popular PSO algorithm for hyperparameter optimization.
•
Comparison with other metaheuristic-based strong optimizers.

The structure of the research is outlined ahead: Section 2 provides fundamentals of the proposed research, the following Section 3 describes the methodology, while Section 4 provides the characteristics of the experiments performed, and the results of those experiments are given in Section 5. Proposals for future work and a summary of the achieved results are given in the final Section 6.
2. Related works

The AI implementation within the realm of medicine has garnered considerable recognition among scientists for a variety of enthralling reasons. One primary driving force is the incessantly growing healthcare demands and the escalating need for precision in medical diagnoses, which serves as a powerful incentive for numerous scientists to investigate avenues for integrating automation within the medical domain [20, 21, 22]. Additionally, the progression of networking technologies and the advent of the IoT have ushered in a heightened emphasis on security measures. In this context, AI algorithms have once again proven their efficacy, particularly in the realm of intrusion detection and malware identification [23]. Fraud detection is a large problem as more data is available and more devices are connected to the Internet. In the work of Petrovic et al. [24] XGBoost algorithm is optimized with a teaching-learning-based-optimization (TLB) algorithm to detect credit card frauds. Authors of [25] provide yet another XGBoost approach this time tuned with an improved version of the firefly swarm metaheuristic for intrusion detection in IoT systems of Healthcare 4.0. Jovanovic et al. [26] provide yet another XGBoost approach optimized by FA indicating high usability of the XGBoost method for healthcare-related prediction problems.

A fascinating area of application lies inside the reach of time series analysis. Algorithms have proven to be invaluable for observing and forecasting trends found inside continuous data, expediting the identification of patterns, directions, and correlations within datasets. Algorithms equipped with the ability to consider the temporal component in data have shown considerable promise when employed in tackling complex real-world challenges [27]. To further enhance their effectiveness, numerous cutting-edge data decomposition techniques were integrated with time-series data, leading to improved performance achieved by the dissection of signals into a sequence of component signals [28]. This approach frequently brings superior results since forecasting a complex signal can be a formidable task, whereas dealing with a series of simpler signals tends to be more manageable. Jovanovic et al. [29] explored the forecast of crude oil prices in terms of supply and demand. The authors applied a long short-term memory (LSTM) network which is a high-performing RNN optimized by the salp swarm metaheuristic. Furthermore, an even more advanced solution includes a double LSTM network system which is known as a bidirectional LSTM (Bi-LSTM) [30].

A significant challenge entailed in the effective utilization of AI algorithms revolves around the necessity for fine-tuning to achieve enhanced performance [31]. Contemporary AI algorithms are endowed with a multitude of control parameters that dictate their response to the input data at hand. Given that many algorithms are typically designed with the aim of achieving solid overall performance, adjustments become indispensable to optimize their capabilities for specific problem-solving scenarios. In this regard, metaheuristic algorithms have emerged as a valuable asset, consistently delivering exceptional results in the context of selecting hyperparameter values and elevating the performance of foundational algorithms for applications in extensive spectrum of real-world problems [32, 33, 34]. Cryptocurrency [35, 36], gold [37], and other stock related predictions [38] have gained popularity. The performance improvements of metaheuristics have been documented during the most current pandemic of COVID-19 with the uses of such predictions including case detection [39], and fake news detection [40]. The influence of swarm intelligence solutions is recorded in the field of cyber security with accents on IoT [41, 42, 43], fraud detection [44, 45, 46], and intrusion detection [47, 48, 49, 50].

Nonetheless, it’s imperative to acknowledge that not all metaheuristic algorithms are universally well-suited to address every conceivable challenge [51, 52], as aptly illustrated by the “No free lunch” theorem [53]. This theorem essentially underscores the notion that there is no one-size-fits-all solution in the world of optimization algorithms. Consequently, it becomes essential to embark on a journey of experimentation and exploration to identify the most promising approaches for each specific implementation or problem domain.

In that sense, this study delved into the intriguing realm of applying optimization metaheuristic techniques to the classification of time-series data derived from EGC. By doing so, it sought to unravel the optimal strategies for effectively discerning patterns and extracting valuable insights from this intricate and dynamic form of data. This research represents a vital step toward enhancing our understanding of how metaheuristic algorithms can be harnessed to address the unique challenges posed by ECG time-series classification [54].

2.1 Recurrent neural network

Modifications towards operations with sequential data lead to an RNN, differing from basic neural networks by incorporating recurrent connections besides regular ones among cells. This feature allows for memory storage of future inputs. The structure is similar to the original solution, consisting of sequential layers with neurons connecting to each other, along with the weights and biases that represent the connections for evaluating input, making decisions, and generating output.

With respect to the task being addressed, neural networks can and have to be customized so that they would provide optimal results [55]. For simpler structures, training and interpretation of data benefit, but nonlinear complex relationships are a struggle. For larger networks the situation is opposite.

Previous hidden state $h_{t-1}$ is incorporated to the ongoing time step $t$ and processed by the formula shown in Eq. (1).

$\displaystyle{a}_{t}={b}+{W}{h}_{t-1}+{U}{x}_{t}$ (1)

for which the input activation is $a_{t}$ , the bias term is $b$ , recurrent connections weight matrix is $W$ , while the input connections are shown as weight matrix $U$ .

The hidden state is changed after each input processing by the formula in Eq. (2). The new hidden state is determined with application of the activation function $\phi$ to the input activation $a_{t}$ shown in Eq. (2).

$\displaystyle{h}_{t}=\phi({a}_{t})$ (2)

$\phi$ function can be different based on the needs for prediction. The network output is calculated from the hidden state Eq. (3).

$\displaystyle{y}_{t}={W}{h}_{t}$ (3)

Fundamentals of RNN are described with the previous equations. In the same manner, other AI-based solutions yield tuning [56], RNNs require customized structures for optimal performance.

2.2 Luong attention-based model

The phenomenon of attention does not have a precise mathematical definition and its implementation should be regarded as a mechanism. Different expressions in mathematics that are used to describe attention include saliency detection, sliding window methods, local image features, etc. The attention mechanism is precisely defined when used with RNNs.

The networks that adhere to RNN standards and employ any attention mechanism are considered attention-based. The mechanism is used to allow the input sequence to have different weights. This allows for data capture and usability of the input-output relations. A basic way to implement this is with the use of a second RNN.

For this work, the Luong attention-based model is applied. For every timestep $t$ the source’s weight is $w_{t}$ is calculated with the goal of decoding the attention-based encoder-decoder as $\Sigma_{s}w_{t}(s)=1$ and $\forall s\;w_{t}(s)\geqslant 0$ . The function of the hidden state $h_{t}$ is the predicted timestep’s token, while the $\Sigma_{s}w_{t}(s)*\hat{h}_{s}$ .

For other mathematical implementations of attention, the weights are computed differently. The softmax function on scaled scores for each token is used with the Luong model. The score is calculated with linear transformation of the matrix $W_{a}$ of the decoder’s $h_{t}$ dot product and the encoder $\hat{h}_{s}$ to calculate the score.

2.3 Hyperparameters of Luong-attention based RNN

The addition of the attention mechanism separates the Luong RNN from the original. This mechanism allows the neural network to have a selective focus on the input sequences while creating output. The text ahead explains the hyperparameters required to be tuned in such networks.

1.
Hidden layers number ( $n_{\textit{hid}}$ ): The depth of the model corresponds to the count of hidden layers. More complex layers are capturable through the application of greater depth but the risk of overfitting increases as well.
2.
Hidden units number per layer ( $n_{\textit{unit}}$ ): Each hidden layer’s number of neurons. The same principle for increasing this parameter applies as to the hidden layers number.
3.
RNN cell type: Different RNN cells can be used alike in gated recurrent units (GRU) and LSTM RNNs [57]. The purpose of such cells is the mitigation of the vanishing gradient as well as to capture data relations of greater temporal distances.
4.
Mechanism of attention: As different mechanisms can be applied this parameter in the case of this research has the value of the Luong attention model. There are two types of attention mechanisms. The first is the global attention used to take care of all positions of the source, and the local attention focuses on the area around the current target.
5.
Scoring function for attention: The alignment scores are computed by this function between the source and target sequences. Three different scoring functions are defined by Luong attention: dot product, general (multiplicative), and concatenation (additive). The model’s performance directly depends on the selection of this function as well as its interpretability.
6.
Learning rate ( $\alpha$ ): During the training process the weights of the model are refreshed with respect to the size defined by the learning rate, making this hyperparameter paramount. More iterations are required with smaller learning rates while providing higher precision convergence. On the other hand, a large learning rated reduces the training times but the overfitting risk is higher.
7.
Dropout rate ( $p_{\textit{drop}}$ ): Overfitting prevention is performed through the dropout rate parameter. Random neurons are deactivated during the training process with the percentage of them to be deactivated in each iteration specified by this parameter.
8.
Batch size: During a single weights update the number of training samples is defined by the batch size. Models with higher sizes can be more accurate in gradient estimation and increase the speed of training but with higher computational resource requirements.
9.
Sequence length: The lengths of model’s input and output sequences. Computational complexity and risk of overfitting rise with the increase of this parameter with the goal of capturing distant data relations.

Models performance for cardiogram anomaly detection relies on proper hyperparameter selection for the Luong attention-based RNN model. This is performed through extensive experimentation, as well as optimization through other metaheuristics.
3. Methods

3.1 The original particle swarm optimization method

Kennedy and Eberhart [58] suggested the original PSO in 1995 with the main inspiration being the fish and bird flocking. Particles define the search agents which are individuals of the population. The algorithm has established satisfactory performance for discrete and continuous optimization challenges [59].

The initial step is to apply random velocities to every particle of the population. Particles move with the velocity described by three weights searching for the optimal position. The mentioned three weights are the previous velocity, the best obtained so far, and the neighbor’s best-obtained direction so far.

$\displaystyle\begin{cases}\overrightarrow{v_{i}}\leftarrow\overrightarrow{v_{i% }}+\overrightarrow{U}(0,\phi_{1})\bigotimes(\overrightarrow{p_{i}}-% \overrightarrow{x_{i}})+\overrightarrow{U}(0,\phi_{2})\bigotimes(% \overrightarrow{p_{g}}-\overrightarrow{x_{i}})\\ \overrightarrow{x_{i}}\leftarrow\overrightarrow{x_{i}}+\overrightarrow{v_{i}},% \\ \end{cases}$ (4)

for which $\bigotimes$ gives the component-wise multiplication, $v_{i}$ shows all the components defined by the range $[-V_{max},+V_{max}]$ , while the $\overrightarrow{U}(0,\phi_{1})$ describes the vector representing each particle for every generation that is uniformly distributed in the range of $0$ to $\phi_{i}$ . In $D$ -dimensional space, each particle can be a solution, and the position it is defined as

$\displaystyle X_{i}=(x_{i1},\;x_{i2},\;\ldots,\;x_{iD})$ (5)

The position that is the best so far is updated as

$\displaystyle P_{i}=(p_{i1},\;p_{i2},\;\ldots,\;p_{iD})$ (6)

The velocities are described as

$\displaystyle V_{i}=(v_{i1},\;v_{i2},\;\ldots,\;v_{iD})$ (7)

the best individual so far is given by pbest, while the gbest represents the group’s best solution. Based on both values the next position of the particle is calculated. By applying the inertia weight method, it may be modeled as:

$\displaystyle v_{id}=W*v_{id}+c_{1}*r_{1}*(P_{id}-X_{id})+c_{2}*r_{2}*(P_{gd}-% X_{id})$ (8)

where the particle velocity is shown as $v_{id}$ , the current position of the particles as $x_{id}$ , the factor of inertia as $w$ , $c_{1}$ shows the relative influence on the cognitive component while $c_{2}$ does so for the social component, random numbers are shown as $r_{1}$ and $r_{2}$ . Note that the pbest and gbest are respectively given as $p_{id}$ and $p_{gd}$ .

The application of the inertia factor follows:

$\displaystyle w=w_{max}-\frac{w_{max}-w_{min}}{T}\cdot t$ (9)

for which $w_{\max}$ shows the initial weight, $w_{\min}$ the final one, $T$ the number of maximum rounds, and $t$ the ongoing round.

3.2 Proposed PSO firefly search algorithm

While the original PSO is very versatile and shows excellent performance when applied to many challenging tasks extensive evaluations using standard CEC [60] functions certain drawbacks can be observed. A popular approach for overcoming similar issues is through hybridization with other algorithms [61]. As such an algorithm that outperforms base algorithms can be formulated.

For this work, due to excellent exploitation potential [62], the FA [63] is chosen for hybridization. This algorithm mimics the mating behaviors of fireflies observed in nature. As a population-based algorithm, initially, a population of random agents is created. Agents are evaluated with respect to the fitness function that is problem-specific. The outcomes of this function are emulated as firefly brightness by the FA. Brightness determines how attractive an agent is to others in the population.

$\displaystyle\beta_{0}=Obj(x)$ (10)

However, the brightness perceived by other agents varies depending on several factors and can be determined for each agent in the population according to the following:

$\displaystyle\beta_{i}=\beta_{0}\cdot\exp(-\gamma*d_{i}^{2})$ (11)

where $\beta$ represents the maximum brightness of an agent (objective function score), $d_{i}$ , represents the distance between the base and observed agent, and $\gamma$ is a characteristic of the media through which the simulated light travels. Once brightness’s are determined agents are relocated towards more attractive solutions. This is done in accordance with the following equation:

$\displaystyle\Delta X_{i}=\alpha\cdot\beta_{i}\cdot(x_{j}-x_{i})+x_{r}$ (12)

where the positional change for agent $i$ is denoted as $\Delta X_{i}$ , the value of $\beta$ represents the determined attractiveness, $\alpha$ represents a control parameter that determines the effect of the change and $x_{i}$ and $x_{j}$ represent the positions of the two agents being evaluated for relocation. Finally, $x_{r}$ represents a factor that introduces randomness into the equation. This equation can easily be expanded to suit problems with multidimensional search spaces.

To ensure contribution of both metaheuristics to the optimization process, an additional control $\psi$ parameter has been proposed. The initial value of $\psi$ has been empirically determined as giving the best outcomes set at $0.5$ . Following each iteration a random value $r$ is generated in rage $[0,1]$ . Should the value of $r$ exceed $\psi$ the FA search mechanism is applied in the current iteration. Otherwise, should the value of $r$ be below $\psi$ the standard PSO search is utilized.

As the original PSO is enriched with the FA’s search procedure in the proposed solution it is named the PSO firefly search algorithm (PSO-FS). The pseudocode of the introduced optimization approach is provided below:

Pseudocode of the introduced PSO-FS algorithmDefine control parameters $\gamma$ , $\alpha$ , $\psi$ Initialized a population $P$ of potential solutions Evaluate solutions using an objective function Store best solution $t<T$ Select an arbitrary number for $r$ in range $[0,1]$

$r>\psi$ $i$ in $P$ $j$ in $P$ $\beta_{j}>\beta_{i}$ Relocate agent $i$ towards $j$

Determine particle velocities based on the best agent Update agent positions based on determined velocities

Evaluate solutions based on the objective function Store best solution for future updates top-performing individual as the solution

4. Experimental setup

For the simulations that were conducted, a real-world ECG dataset was applied [64], available at https://www.physionet.org/content/ptbdb/1.0.0/ (accessed September 15th). For the dataset 16 input electrodes are used for data collection. The dataset consists of 549 records of which 268 are some forms of cardiovascular disorders. The initial 70% of data was used for the training subset, 10% for the validation subset, and finally 20% of data in the testing set. For the sequences of time series, 10 consecutive samples were applied.

In the first set of simulations metaheuristic methods were assigned to optimize the RNN for this issue. The parameters that were the subject of optimization are model’s layers number $[1,2]$ , count of cells per layer $[3,10]$ , dropout rate $[0.05,0.2]$ , learning rate $[0.0001,0.05]$ , and training epochs $[30,60]$ .

In the second set of simulations metaheuristic methods were allocated to optimize the RNN networks with attention layers under identical conditions. The count of cells in the attention layer present in the network is optimized inside boundaries of $[3,10]$ .

In both simulations the focus is to find light weight architectures and optimize their efficiently for the given task. To this end several state-of-the-art algorithms have been put to comparison alongside the proposed algorithm. These include the original PSO [58], and FA [63] algorithms as well as the well established GA [65], WOA [66] and SCA [67]. Every algorithm used in experiments was applied under the same circumstances having the population size of 6 solutions for 8 rounds. Total lags are applied $\frac{1}{3}$ as a stopping criterion to prevent overfitting. Lastly, 30 independent runs were applied. The metrics used to compare the models include standard ones which are the precision, recall, f1-score, and accuracy.

In the preceding equations TP and TN denote true positives and negatives, while FP and FN denote false positive and negative instances. The optimization of the objective function is done for the error metric.

$\displaystyle\textit{Error}=1-\textit{Accuracy}$ (13)

An additional metric included in this analysis is Cohen’s Kappa shown in Eq. 14 due to this metric’s ability to deal with imbalanced data.

$\displaystyle K=\frac{P_{\textit{obs}}-P_{\textit{exp}}}{1-P_{\textit{exp}}}$ (14)

where the observed agreements $(\textit{TP}+\textit{TN})$ are represented bz $P_{\textit{obs}}$ , and the expected agreements $[{(\textit{TP}+\textit{FN})\times(\textit{TP}+\textit{FP})}+{(\textit{FP}+% \textit{TN})\times(\textit{FN}+\textit{TN})}]$ are given as $P_{\textit{exp}}$ . The value of $K$ is in the range $[0,1]$ , while lower scores suggest less agreement.

5. Outcomes

The outcomes of the two conducted experiments are presented in the following order. Initially outcomes with simple RNN are explored. Following this outcomes using attention augmented LSTM networks are presented.

5.1 Experiments with simple RNN

Each algorithm is evaluated and compared for the fitness function results over 30 runs in Table 1. Best, worst, and mean outcomes are provided. The Cohen’s kappa comparison results are given in Table 2.

Table 1
Fitness function overall scores

Approach	Best	Worst	Mean	Median	Std	Var
RNN-PSO-FS	0.006837	0.165942	0.047131	0.009013	0.060166	0.003620
RNN-PSO	0.011809	0.152268	0.047649	0.020510	0.050720	0.002572
RNN-FA	0.007458	0.048477	0.015641	0.009323	0.014731	0.000217
RNN-GA	0.009944	0.142324	0.076963	0.089807	0.050539	0.002554
RNN-WOA	0.006837	0.186451	0.043712	0.013362	0.064568	0.004169
RNN-SCA	0.007458	0.085768	0.033147	0.017713	0.029475	0.000869

Table 2

Indicator function overall scores

Approach	Best	Worst	Mean	Median	Std	Var
RNN-PSO-FS	0.981837	0.560570	0.876110	0.976189	0.158779	0.025211
RNN-PSO	0.968882	0.578161	0.871107	0.946288	0.140824	0.019831
RNN-FA	0.980169	0.864892	0.957502	0.975413	0.041529	0.001725
RNN-GA	0.973774	0.680990	0.798729	0.763433	0.130910	0.017138
RNN-WOA	0.981926	0.347997	0.858031	0.964800	0.229515	0.052677
RNN-SCA	0.980202	0.795146	0.917558	0.952601	0.069766	0.004867

Figure 1.

Distributions of the fitness and indicator functions.

Figure 2.

Objective and indicator convergence diagrams.

Table 3

Detailed metrics comparisons among top-performance models

Approach	Metric	Normal	Abnormal	Accuracy	Macro av.	Weight av.
RNN-PSO-FS	Precision	0.995021	0.987624	0.993163	0.991322	0.993159
	Sensitivity	0.995847	0.985185	0.993163	0.990516	0.993163
	f1-score	0.995434	0.986403	0.993163	0.990918	0.993161
RNN-PSO	Precision	0.995816	0.966184	0.988191	0.981000	0.988357
	Sensitivity	0.988372	0.987654	0.988191	0.988013	0.988191
	f1-score	0.992080	0.976801	0.988191	0.984441	0.988234
RNN-FA	Precision	0.994196	0.987593	0.992542	0.990894	0.992534
	Sensitivity	0.995847	0.982716	0.992542	0.989282	0.992542
	f1-score	0.995021	0.985149	0.992542	0.990085	0.992536
RNN-GA	Precision	0.996656	0.970944	0.990056	0.983800	0.990184
	Sensitivity	0.990033	0.990123	0.990056	0.990078	0.990056
	f1-score	0.993333	0.980440	0.990056	0.986887	0.990088
RNN-WOA	Precision	0.997498	0.980488	0.993152	0.988993	0.993216
	sensitivity	0.993355	0.992593	0.993152	0.992974	0.993163
	f1-score	0.995422	0.986503	0.993152	0.990963	0.993177
RNN-SCA	Precision	0.747658	0.125000	0.744562	0.436329	0.590929
	Sensitivity	0.994186	0.002479	0.744562	0.498328	0.744562
	f1-score	0.853476	0.004843	0.744562	0.429159	0.639867
	Entries	1204	405

Figure 3.

Swarm diversity diagrams for the objective and indicator functions.

As the outcomes shown in Table 1 indicate, the introduced algorithm achieved the best outcomes in the best-case scenario and in the median case of execution. However, the original FA algorithm held the best outcomes in the worst-case scenario. This in turn allowed the FA algorithm to reach the best outcomes in the mean metric. Furthermore, the original FA algorithm attained the highest rate of stability.

Figure 4.

Best performing model PR, ROC curves and confusion metric.

Outcomes are somewhat mirrored for the indicator function, where the FA algorithm demonstrated high stability due to good performance in the worst-case scenario, resulting in a good outcome in the mean outcomes. Nevertheless, the best median outcomes are demonstrated by the introduced metaheuristic. Interestingly, the WOA demonstrated the best outcomes evaluated on the indicator function. However, it also achieved the worst results in the worst-case execution, limiting the algorithm’s reliability and demonstrating the lowest stability compared to other algorithms.

Stability comparisons regarding the fitness and indicator functions of each of the evaluated algorithms can be seen in Fig. 1.

Table 4

Selected parameters for top-performance models

Model	Learning rate	Dropout	Epochs	Layers	Units L1	Units L2
RNN-PSO-FS	0.006151	0.015985	46	2	9	8
RNN-PSO	0.009302	0.014908	60	1	10	NA
RNN-FA	0.010000	0.200000	59	2	3	NA
RNN-GA	0.007093	0.178377	60	1	9	8
RNN-WOA	0.010000	0.200000	55	2	10	10
RNN-SCA	0.004896	0.095600	56	1	7	NA

Table 5

Fitness function overall scores for RNN-ATT experiments

Method	Best	Worst	Mean	Median	Std	Var
RNN-ATT-PSO-FS	0.002486	0.006837	0.005594	0.006215	0.001522	2.32E-06
RNN-ATT-PSO	0.005594	0.006837	0.006422	0.006526	0.000463	2.15E-07
RNN-ATT-FA	0.006215	0.014916	0.009633	0.009323	0.003061	9.37E-06
RNN-ATT-GA	0.003729	0.006837	0.006008	0.006215	0.001056	1.12E-06
RNN-ATT-WOA	0.004351	0.007458	0.005076	0.004661	0.001101	1.21E-06
RNN-ATT-SCA	0.004972	0.012430	0.007665	0.007458	0.002398	5.75E-06

One important indicator of algorithm performance is the convergence rates through iterations. By observing convergence rates, details about the algorithm’s ability to avoid local minima traps as well as the algorithm’s ratio of diversification to intensification can be assessed. Comparison between convergence rates for the top-performance models generated by every optimizer is presented in Fig. 2 for both the indicator and objective functions.

Table 6

Indicator function overall scores for RNN-ATT experiments

Method	Best	Worst	Mean	Median	Std	Var
RNN-ATT-PSO-FS	0.993390	0.981837	0.985190	0.983582	0.004020	1.62E-05
RNN-ATT-PSO	0.985212	0.981926	0.983006	0.982769	0.001246	1.55E-06
RNN-ATT-FA	0.983582	0.960597	0.974380	0.975048	0.008121	6.60E-05
RNN-ATT-GA	0.990117	0.981926	0.984115	0.983555	0.002781	7.73E-06
RNN-ATT-WOA	0.988461	0.980363	0.986562	0.987633	0.002867	8.22E-06
RNN-ATT-SCA	0.986844	0.966840	0.979687	0.980299	0.006442	4.15E-05

Table 7

Detailed metrics comparisons among top-performance models for RNN-ATT experiments

Approach	Metric	Normal	Abnormal	Accuracy	Macro av.	Weight av.
RNN-ATT-PSO-FS	Precision	0.997512	0.997519	0.997514	0.997516	0.997514
	Sensitivity	0.999169	0.992593	0.997514	0.995881	0.997514
	f1-score	0.998340	0.995050	0.997514	0.996695	0.997512
RNN-ATT-PSO	Precision	0.747664	0.000000	0.745805	0.373832	0.559470
	Sensitivity	0.996678	0.000000	0.745805	0.498339	0.745805
	f1-score	0.854397	0.000000	0.745805	0.427198	0.639337
RNN-ATT-FA	Precision	0.998331	0.980535	0.993785	0.989433	0.993851
	Sensitivity	0.993355	0.995062	0.993785	0.994209	0.993785
	f1-score	0.995837	0.987745	0.993785	0.991791	0.993800
RNN-ATT-GA	Precision	0.998336	0.990172	0.996271	0.994254	0.996281
	Sensitivity	0.996678	0.995062	0.996271	0.995870	0.996271
	f1-score	0.997506	0.992611	0.996271	0.995059	0.996274
RNN-ATT-WOA	Precision	0.997506	0.990148	0.995649	0.993827	0.995654
	Sensitivity	0.996678	0.992593	0.995649	0.994635	0.995649
	f1-score	0.997092	0.991369	0.995649	0.994230	0.995651
RNN-ATT-SCA	Precision	0.998333	0.985330	0.995028	0.991832	0.995060
	Sensitivity	0.995017	0.995062	0.995028	0.995039	0.995028
	f1-score	0.996672	0.990172	0.995028	0.993422	0.995036
	Entries	1204	405

Figure 5.

Best performing model confusion metric.

Figure 6.

Objective and indicator function distribution plots for RNN-ATT simulations.

Figure 7.

Swarm diversity diagrams for the fitness and indicator functions for RNN-ATT simulations.

Figure 8.

Convergence graphs of fitness and indicator functions for RNN-ATT experiments.

Convergence rates shown in Fig. 2 indicate that in the best possible runs the original FA and SCA algorithm had a near-optimal start showing very little convergence. The convergence of the introduced algorithm demonstrated a steady convergence towards an optimal starting with a sub-optimal position. Furthermore, numerical results indicated that the algorithm also demonstrated an ability to avoid local minimum traps attaining better solutions than algorithms starting with a significant advantage that shows premature convergence. These outcomes are further reinforced in sward diversity plots shown in Fig. 3.

Additional comparisons between the top models constructed by every metaheuristics algorithm are demonstrated in detail in Table 3.

Table 8

Objective function overall outcomes for RNN-ATT simulations

Model	Learning rate	Dropout	Epochs	Layers	Units L1	Units L2	Attention neurons
RNN-ATT-PSO-FS	0.006959	0.079863	60	1	10	10	5
RNN-ATT-PSO	0.007908	0.161136	57	1	8	5	4
RNN-ATT-FA	0.010000	0.020000	60	2	10	10	6
RNN-ATT-GA	0.004460	0.188410	60	1	10	3	4
RNN-ATT-WOA	0.010000	0.020000	60	2	4	10	8
RNN-ATT-SCA	0.009184	0.144949	57	1	10	3	8

Figure 9.

Best performing model PR, ROC curves and confusion metric for RNN-ATT simulations.

As shown within Table 3 the best promising model that has been produced by the proposed metaheuristics shows the highest accuracy value alongside the highest macro average values. Furthermore, the algorithms also demonstrated the highest precision for anomaly detection compared to other optimized models. Nevertheless, the WOA demonstrated the highest precision for normal heart activity classification. However, it is important to mention that the WOA exhibited low stability stability across other metrics.

The classification PR and ROC curves for the best-performing model are demonstrated in Fig. 4 while the confusion metrics are in Fig. 5.

Hyperparameters that were selected by the algorithms are provided to incite repeatability and further research on this topic in Table 4.

5.2 Experiments with attention based RNN

Simulations outcomes that incorporate attention layers are provided in Table 8 in terms of objective function and in Table 6 in terms of indicator function. A strong stability is demonstrated by the PSO. However, the PSO did not showcase the best outcomes. Similarly the WOA demonstrated the superior scores for the mean and median values but did not manage to find an overall optimal solution.

Figure 10.

Best performing model confusion metric for RNN-ATT simulations.

Stability comparisons are also provided in Fig. 6 as distribution plots and in Fig. 7 as swarm plots. While the FA showcases high stability, it also fails to attain the best outcomes, suggesting that the lack of exploration exists with this algorithm converging prematurely towards a local minimum. Further details are presented in convergence graphs provided in Fig. 8. Convergence rates suggest that several optimizes converge prematurely towards an local optima failing to locate a best solution. The introduced optimizer showcases a slower convergence rate, but this boost in exploration helps the algorithm locate a more promising outcome.

Detailed comparisons between the most superior models produced by every optimizer are provided in Table 7.

Further details such as the PR and ROC curves for the best model are provide in Fig. 9 followed by the best models confusion matrix in Fig. 10. Finally, parameters for the best constructed models are displayed in Table 8.

The introduction of attention layers improve the performance of neural networks for the given challenge. Additionally, the introduced optimizer performed equally well under both test scenarios suggesting that the introduced modifications help overcome local minimum traps and focus on finding better potential solutions. Additionally, attention layers help the model focus on more relevant information provided to the network allowing for better outcomes boosting performance from the previous best of 0.006837 to a new best of 0.002486 error rate.

6. Conclusion

The significance of cardiovascular health in future research cannot be overstated, considering it remains the leading global cause of death. The unexplored facets of diseases and their treatments present a promising frontier for AI, holding the potential to revolutionize healthcare. AI-driven early detection systems can markedly enhance quality of life and extend longevity by enabling doctors to intervene promptly, a critical factor in managing chronic diseases. The healthcare landscape is rife with possibilities as an array of advanced AI algorithms continues to evolve, offering solutions for increasingly complex challenges. The application of metaheuristics to healthcare issues presents limitless combinations, with ongoing research aimed at unraveling the unknowns. Health data, often in time series format, serves as the tuning ground for proposed solutions. With minor adjustments and parameter optimization, the untapped potential of these solutions awaits exploration. An innovative approach involves anomaly detection using ECG systems, employing an RNN optimized by metaheuristic solutions such as PSO, FA, GA, WOA, and SCA.

This study, while showcasing commendable outcomes in the best and median case scenarios, has certain limitations. It concentrates on a specific comparison of optimization algorithms and explores the capabilities of RNNs exclusively for this problem. Computational constraints led to the use of smaller population sizes and a limited number of rounds. Future endeavors should extend beyond these limitations, delving into similar medical problems and exploring novel algorithms to enhance the proposed solution.

Noteworthy enhancements come in the form of attention layers, significantly improving neural network performance for the targeted challenge. The introduced optimizer exhibits consistent performance in various test scenarios, suggesting its effectiveness in overcoming local minimum traps and focusing on superior potential solutions. Furthermore, attention layers enable the model to concentrate on pertinent information, resulting in a substantial performance boost – reducing the error rate from the previous best of 0.006837 to an impressive new low of 0.002486. Future research should build on these findings, exploring similar medical challenges and novel algorithms to further refine and advance solutions in this domain.

References

Ezzati

Obermeyer

Tzoulaki

Mayosi

B.M.

Elliott

and Leon

D.A.

, Contributions of risk factors and medical care to cardiovascular mortality trends, Nature Reviews Cardiology 12(9) (2015), 508–530.

Plageras

A.P.

and Psannis

K.E.

, Iot-based health and emotion care system, ICT Express 9(1) (2023), 112–115.

Cuk

Bezdan

Jovanovic

Antonijevic

Stankovic

Simic

Zivkovic

and Bacanin

, Tuning attention based long-short term memory neural networks for parkinson’s disease detection using modified metaheuristics, Scientific Reports 14(1) (2024), 4309.

Musk

et al., An integrated brain-machine interface platform with thousands of channels, Journal of Medical Internet Research 21(10) (2019), e16194.

Berkaya

S.K.

Uysal

A.K.

Gunal

E.S.

Ergin

Gunal

and Gulmezoglu

M.B.

, A survey on ecg analysis, Biomedical Signal Processing and Control 43 (2018), 216–235.

Minic

Jovanovic

Bacanin

Stoean

Zivkovic

Spalevic

Petrovic

Dobrojevic

and Stoean

, Applying recurrent neural networks for anomaly detection in electrocardiogram sensor data, Sensors 23(24) (2023), 9878.

Bacanin

Zivkovic

Al-Turjman

Venkatachalam

Trojovskỳ

Strumberger

and Bezdan

, Hybridized sine cosine algorithm with convolutional neural networks dropout regularization application, Scientific Reports 12(1) (2022), 6302.

Zivkovic

Bacanin

Antonijevic

Nikolic

Kvascev

Marjanovic

and Savanovic

, Hybrid cnn and xgboost model tuned by modified arithmetic optimization algorithm for covid-19 early diagnostics from x-ray images, Electronics 11(22) (2022), 3798.

Pratama

M.R.

Suryana

A.L.

Alfiansyah

Olivia

Nurmawati

and Destarianto

, Diagnosis of stroke and diabetes mellitus with classification techniques using decision tree method, International Journal of Health and Information System 2(1) (2024), 1–8.

10.

Utomo

D.T.

Utomo

A.H.

Olivia

Maria

and Rosidania

N.P.

, Detecting dehydration based on urine color using fuzzy logic image processing and regulating water intake with an automatic water pump according to dehydration level using an iot-based, International Journal of Health and Information System 1(3) (2024), 152–164.

11.

Nugraha

G.S.

Juliansyah

and Tajuddin

, Glaucoma detection based on texture feature of neuro retinal rim area in retinal fundus image, International Journal of Health and Information System 1(3) (2024), 117–127.

12.

Bacanin

Stoean

Zivkovic

Petrovic

Rashid

T.A.

and Bezdan

, Performance of a novel chaotic firefly algorithm with enhanced exploration for tackling global optimization problems: Application for dropout regularization, Mathematics 9(21) (2021), 2705.

13.

Zivkovic

Stoean

Chhabra

Budimirovic

Petrovic

and Bacanin

, Novel improved salp swarm algorithm: An application for feature selection, Sensors 22(5) (2022), 1711.

14.

Zivkovic

Jovanovic

Ivanovic

Bacanin

Strumberger

and Joseph

P.M.

, Xgboost hyperparameters tuning by fitness-dependent optimizer for network intrusion detection, in Communication and Intelligent Systems: Proceedings of ICCIS 2021, Springer, 2022, pp. 947–962.

15.

Bacanin

Petrovic

Zivkovic

Bezdan

and Antonijevic

, Feature selection in machine learning by hybrid sine cosine metaheuristics, in International Conference on Advances in Computing and Data Sciences, Springer International Publishing Cham, 2021, pp. 604–616.

16.

Bezdan

Petrovic

Zivkovic

Strumberger

Devi

V.K.

and Bacanin

, Current best opposition-based learning salp swarm algorithm for global numerical optimization, in 2021 zooming innovation in consumer technologies conference (ZINC), IEEE, 2021, pp. 5–10.

17.

Bacanin

Petrovic

Zivkovic

Bezdan

and Chhabra

, Enhanced salp swarm algorithm for feature selection, in International conference on intelligent and fuzzy systems, Springer International Publishing Cham, 2021, pp. 483–491.

18.

Medsker

L.R.

and Jain

, Recurrent neural networks, Design and Applications 5(64-67) (2001), 2.

19.

Bansal

J.C.

and Deep

, Algorithms for intelligent systems, 2020.

20.

Jovanovic

Zivkovic

Antonijevic

Jovanovic

Ivanovic

and Jassim

H.S.

, An emperor penguin optimizer application for medical diagnostics, in 2022 IEEE Zooming Innovation in Consumer Technologies Conference (ZINC), IEEE, 2022, pp. 191–196.

21.

Dinesh

Bacanin

Askar

and Abouhawwash

, Diagnostic ability of deep learning in detection of pancreatic tumour, Scientific Reports 13(1) (2023), 9725.

22.

Jovanovic

Hajdarevic

Jovanovic

Jassim

H.S.

Strumberger

Bacanin

Zivkovic

and Antonijevic

, Tuning extreme learning machine by hybrid planet optimization algorithm for diabetes classification, in Congress on Intelligent Systems, Springer, 2022, pp. 23–36.

23.

AlHosni

Jovanovic

Antonijevic

Bukumira

Zivkovic

Strumberger

Mani

J.P.

and Bacanin

, The xgboost model for network intrusion detection boosted by enhanced sine cosine algorithm, in Third International Conference on Image Processing and Capsule Networks: ICIPCN 2022, Springer, 2022, pp. 213–228.

24.

Petrovic

Antonijevic

Strumberger

Jovanovic

Savanovic

and Janicijevic

, The xgboost approach tuned by tlb metaheuristics for fraud detection, in Proceedings of the 1st International Conference on Innovation in Information Technology and Business (ICIITB 2022) 104, Springer Nature, 2023, p. 219.

25.

Savanović

Toskovic

Petrovic

Zivkovic

Damaševičius

Jovanovic

Bacanin

and Nikolic

, Intrusion detection in healthcare 4.0 internet of things systems via metaheuristics optimized machine learning, Sustainability 15(16) (2023), 12563.

26.

Jovanovic

Antonijevic

Zivkovic

Budimirovic

Strumberger

and Bacanin

, The xgboost tuning by improved firefly algorithm for network intrusion detection, in 2022 24th International Symposium on Symbolic and Numeric Algorithms for Scientific Computing (SYNASC), IEEE, 2022, pp. 268–275.

27.

Petrovic

Damaševičius

Jovanovic

Toskovic

Simic

Bacanin

Zivkovic

and Spalević

, Marine vessel classification and multivariate trajectories forecasting using metaheuristics-optimized extreme gradient boosting and recurrent neural networks, Applied Sciences 13(16) (2023), 9181.

28.

Stankovic

Jovanovic

Bacanin

Zivkovic

Antonijevic

and Bisevac

, Tuned long short-term memory model for ethereum price forecasting through an arithmetic optimization algorithm, in International Conference on Innovations in Bio-Inspired Computing and Applications, Springer, 2022, pp. 327–337.

29.

Jovanovic

Bacanin

Jovancai Stakic

Antonijevic

Magd

Thirumalaisamy

and Zivkovic

, Multi-step crude oil price prediction based on lstm approach tuned by salp swarm algorithm with disputation operator, Sustainability 14(21) (2022), 14616.

30.

Jovanovic

Dogandzic

Jovanovic

Kumpf

Zivkovic

and Bacanin

, Metaheuristic optimized bilstm univariate time series forecasting of gold prices, in International Conference on Data Science and Applications, Springer, 2023, pp. 221–235.

31.

Jovanovic

Zivkovic

Budimirovic

Petrovic

Strumberger

and Bacanin

, Metaheuristic optimized extreme gradient boosting for solar flare prediction, in 2023 16th International Conference on Advanced Technologies, Systems and Services in Telecommunications (TELSIKS), IEEE, 2023, pp. 183–186.

32.

Zivkovic

Jovanovic

Ivanovic

Krdzic

Bacanin

and Strumberger

, Feature selection using modified sine cosine algorithm with covid-19 dataset, in Evolutionary Computing and Mobile Sustainable Networks: Proceedings of ICECMSN 2021, Springer, 2022, pp. 15–31.

33.

Strumberger

Bezdan

Ivanovic

and Jovanovic

, Improving energy usage in wireless sensor networks by whale optimization algorithm, in 2021 29th Telecommunications Forum (TELFOR), IEEE, 2021, pp. 1–4.

34.

Salb

Jovanovic

Zivkovic

Tuba

Elsadai

and Bacanin

, Training logistic regression model by enhanced moth flame optimizer for spam email classification, in Computer Networks and Inventive Communication Technologies: Proceedings of Fifth ICCNCT 2022, Springer, 2022, pp. 753–768.

35.

Petrovic

Strumberger

Bezdan

Jassim

H.S.

and Nassor

S.S.

, Cryptocurrency price prediction by using hybrid machine learning and beetle antennae search approach, in 2021 29th Telecommunications Forum (TELFOR), IEEE, 2021, pp. 1–4.

36.

Petrovic

Jovanovic

Zivkovic

Bacanin

Budimirovic

and Marjanovic

, Forecasting bitcoin price by tuned long short term memory model, in 1st International Conference on Innovation in Information Technology and Business (ICIITB 2022), Atlantis Press, 2023, pp. 187–202.

37.

Golubovic

Petrovic

Bozovic

Antonijevic

Zivkovic

and Bacanin

, Gold price forecast using variational mode decomposition-aided long short-term model tuned by modified whale optimization algorithm, in International Conference on Data Intelligence and Cognitive Informatics, Springer, 2023, pp. 69–83.

38.

Jovanovic

Milutinovic

Gajevic

Krstovic

Rashid

T.A.

and Petrovic

, Sine cosine algorithm for simple recurrent neural network tuning for stock market prediction, in 2022 30th Telecommunications Forum (TELFOR), IEEE, 2022, pp. 1–4.

39.

Zivkovic

Petrovic

Bacanin

Milosevic

Veljic

and Vesic

, The covid-19 images classification by mobilenetv3 and enhanced sine cosine metaheuristics, in Mobile Computing and Sustainable Informatics: Proceedings of ICMCSI 2022, Springer Nature Singapore Singapore, 2022, pp. 937–950.

40.

Zivkovic

Stoean

Petrovic

Bacanin

Strumberger

and Zivkovic

, A novel method for covid-19 pandemic information fake news detection based on the arithmetic optimization algorithm, in 2021 23rd International Symposium on Symbolic and Numeric Algorithms for Scientific Computing (SYNASC), IEEE, 2021, pp. 259–266.

41.

Milutinovic

Gajevic

Krstovic

Petrovic

Bacanin

and Antonijevic

, Performance of arithmetic optimization algorithm for elm tuning applied to iot security, in 2022 30th Telecommunications Forum (TELFOR), IEEE, 2022, pp. 1–4.

42.

Bacanin

Zivkovic

Hajdarevic

Janicijevic

Dasho

Marjanovic

and Jovanovic

, Performance of sine cosine algorithm for ann tuning and training for iot security, in International Conference on Hybrid Intelligent Systems, Springer, 2022, pp. 302–312.

43.

Jovanovic

Gajevic

Dobrojevic

Budimirovic

Bacanin

and Zivkovic

, Tackling iot security challenge by metaheuristics tuned extreme learning machine, in International Conference on Intelligent Sustainable Systems, Springer, 2023, pp. 507–522.

44.

Petrovic

Bacanin

Zivkovic

Marjanovic

Antonijevic

and Strumberger

, The adaboost approach tuned by firefly metaheuristics for fraud detection, in 2022 IEEE World Conference on Applied Intelligence and Computing (AIC), IEEE, 2022, pp. 834–839.

45.

Djuric

Jovanovic

Zivkovic

Bacanin

Antonijevic

and Sarac

, The adaboost approach tuned by sns metaheuristics for fraud detection, in Proceedings of the International Conference on Paradigms of Computing, Communication and Data Sciences: PCCDS 2022, Springer, 2023, pp. 115–128.

46.

Jovanovic

Antonijevic

Stankovic

Zivkovic

Tanaskovic

and Bacanin

, Tuning machine learning models using a group search firefly algorithm for credit card fraud detection, Mathematics 10(13) (2022), 2272.

47.

Petrovic

Antonijevic

Strumberger

Budimirovic

Savanovic

and Janicijevic

, Intrusion detection by xgboost model tuned by improved multi-verse optimizer, in Proceedings of the 1st International Conference on Innovation in Information Technology and Business (ICIITB 2022) 104, Springer Nature, 2023, p. 203.

48.

Jovanovic

Marjanovic

Antonijevic

Zivkovic

Budimirovic

and Bacanin

, Feature selection by improved sand cat swarm optimizer for intrusion detection, in 2022 International Conference on Artificial Intelligence in Everything (AIE), IEEE, 2022, pp. 685–690.

49.

Zivkovic

Bacanin

Arandjelovic

Rakic

Strumberger

Venkatachalam

and Joseph

P.M.

, Novel harris hawks optimization and deep neural network approach for intrusion detection, in Proceedings of International Joint Conference on Advances in Computational Intelligence: IJCACI 2021, Springer, 2022, pp. 239–250.

50.

Stankovic

Antonijevic

Bacanin

Zivkovic

Tanaskovic

and Jovanovic

, Feature selection by hybrid artificial bee colony algorithm for intrusion detection, in 2022 International Conference on Edge Computing and Applications (ICECAA), IEEE, 2022, pp. 500–505.

51.

Zivkovic

Petrovic

Bacanin

Djuric

Vesic

Strumberger

and Marjanovic

, Training logistic regression model by hybridized multi-verse optimizer for spam email classification, in Proceedings of International Conference on Data Science and Applications: ICDSA 2022, Volume 2, Springer, 2023, pp. 507–520.

52.

Bacanin

Petrovic

Antonijevic

Zivkovic

Sarac

Tuba

and Strumberger

, Intrusion detection by xgboost model tuned by improved social network search algorithm, in International Conference on Modelling and Development of Intelligent Systems, Springer Nature Switzerland Cham, 2022, pp. 104–121.

53.

Wolpert

D.H.

and Macready

W.G.

, No free lunch theorems for optimization, IEEE Transactions on Evolutionary Computation 1(1) (1997), 67–82.

54.

Jovanovic

Bacanin

Zivkovic

Antonijevic

Petrovic

and Zivkovic

, Anomaly detection in ecg using recurrent networks optimized by modified metaheuristic algorithm, in 2023 31st Telecommunications Forum (TELFOR), IEEE, 2023, pp. 1–4.

55.

Damaševičius

Jovanovic

Petrovic

Zivkovic

Bacanin

Jovanovic

and Antonijevic

, Decomposition aided attention-based recurrent neural networks for multistep ahead time-series forecasting of renewable power generation, PeerJ Computer Science 10 (2024).

56.

Jovanovic

Antonijevic

Nikolic

Bacanin

Zivkovic

and Strumberger

, Improving phishing website detection using a hybrid two-level framework for feature selection and xgboost tuning, Journal of Web Engineering 22(3) (2023), 543–574.

57.

Stankovic

Bacanin

Zivkovic

Jovanovic

Mani

and Antonijevic

, Forecasting ethereum price by tuned long short-term memory model, in 2022 30th Telecommunications Forum (TELFOR), IEEE, 2022, pp. 1–4.

58.

Kennedy

and Eberhart

, Particle swarm optimization, in Proceedings of ICNN’95-international conference on neural networks 4, IEEE, 1995, pp. 1942–1948.

59.

Bacanin

Simic

Zivkovic

Alrasheedi

and Petrovic

, Cloud computing load prediction by decomposition reinforced attention long short-term memory network optimized by modified particle swarm optimization algorithm, Annals of Operations Research (2023), pp. 1–34.

60.

Bäck

and Schwefel

H.-P.

, An overview of evolutionary algorithms for parameter optimization, Evolutionary Computation 1(1) (1993), 1–23.

61.

Todorovic

Petrovic

Toskovic

Zivkovic

Jovanovic

and Bacanin

, Multivariate bitcoin price prediction based on lstm tuned by hybrid reptile search algorithm, in 2023 16th International Conference on Advanced Technologies, Systems and Services in Telecommunications (TELSIKS), IEEE, 2023, pp. 195–198.

62.

Bacanin

Zivkovic

Sarac

Petrovic

Strumberger

Antonijevic

Petrovic

and Venkatachalam

, A novel multiswarm firefly algorithm: An application for plant classification, in International Conference on Intelligent and Fuzzy Systems, Springer International Publishing Cham, 2022, pp. 1007–1016.

63.

Yang

X.-S.

and He

, Firefly algorithm: recent advances and applications, International Journal of Swarm Intelligence 1(1) (2013), 36–50.

64.

Kreiseler

and Bousseliot

, Automatisierte ekg-auswertung mit hilfe der ekg-signaldatenbank cardiodat der ptb, 1995.

65.

Mirjalili

and Mirjalili

, Genetic algorithm, Evolutionary Algorithms and Neural Networks: Theory and Applications, 2019, pp. 43–55.

66.

Mirjalili

and Lewis

, The whale optimization algorithm, Advances in Engineering Software 95 (2016), 51–67.

67.

Mirjalili

, Sca: a sine cosine algorithm for solving optimization problems, Knowledge-Based Systems 96 (2016), 120–133.

Anomaly detection in electrocardiogram signals using metaheuristic optimized time-series classification with attention incorporated models

Abstract

Keywords

1. Introduction

2.1 Recurrent neural network

2.3 Hyperparameters of Luong-attention based RNN

3.1 The original particle swarm optimization method

5.1 Experiments with simple RNN

Table 1 Fitness function overall scores

References

Table 1
Fitness function overall scores