Tuned long short-term memory model for Ethereum price forecasting via an arithmetic optimization algorithm

Abstract

Machine learning as a subset of artificial intelligence presents a promising set of algorithms for tackling increasingly complex challenges. A notable ability of this subgroup of algorithms to tackle tasks without explicit programming coupled with the expanding availability of computational resources and information transparency has made it possible to utilize algorithms to forecast prices. In recent years, cryptocurrency has increased in popularity and has seen wider adoption as a payment method. Cryptocurrency trading and mining have become a potentially very lucrative venture. However, due to the instability of cryptocurrency prices, casting accurate predictions can be quite challenging. A novel way of approaching this challenge is by tackling it through time-series forecasting. A particularly promising method for tackling this type of problem is through the utilization of long-short-term memory artificial neural networks to attain accurate prediction results. However, the forecasting accuracy of machine learning models is highly dependent on adequate hyperparameter settings. Thus, this work presents an improved variation of the arithmetic optimization algorithm, tasked with selecting the best values of a long-short term neural network casting price predictions. The presented approach has been evaluated on publicly available real-world Ethereum trading price data. The attained results of a comparative analysis against several popular metaheuristics indicate that the presented method achieved excellent results, and outperformed aforementioned algorithms in one and four-step ahead predictions.

Keywords

Cryptocurrency Ethereum neural networks arithmetic optimization algorithm machine learning

1. Introduction

In the last decade, cryptocurrencies have seen astounding proliferation. This digital payment system employs advanced cryptographic methods to safeguard and manage transactions that are made in a peer-to-peer manner among individuals. The benefits of cryptocurrencies over conventional payment methods include decentralization, security, anonymity, immutability, ease of financial transfers, and decreased transaction costs [1], making it an increasingly popular payment modus operandi. The pioneer of all cryptocurrencies is Bitcoin which influenced the development of many other ecosystems. The idea was first presented in Satoshi Nakamoto’s 2008 article [2] and it was built upon it. The idea gained widespread acceptance and was used in a variety of applications. It also influenced the development of the cryptocurrency ecosystems and gave rise to many additional virtual currencies, including Ethereum, Tether, Binance, Stellar, Cardano, XRP, and Dash [3]. There are presently more than 20,000 different types of cryptocurrencies accessible for public trade, demonstrating the crucial role cryptocurrencies have as a form of electronic transaction media and a financial asset has sparked a lot of interest among investors, researchers, developers, and the general public [4, 5].

The pillar on which the cryptocurrency market operates is blockchain - an open ledger of all transactions that take place inside the system for a particular purpose and is available to all users. As a result, everyone using the system may see how each record has been balanced. Consequently, centralized authority is no longer required, and the entities immediately involved in the transaction are given control [6]. This ledger record consists of an ever-growing sequence of blocks that include the sender or receiver’s public key, timestamp, a cryptographic hash structure of the preceding block, and the quantity of transferred coins [7]. Wallet addresses are the label given to these public keys. A personal key must be approved by the sender in each transaction. After confirmation, the transaction is promptly broadcast throughout the network. Since the system is decentralized, each member has to perform an activity called mining. By solving a cryptographic puzzle, miners in a cryptocurrency network can validate transactions. A miner must verify the exchange or transaction, mark it as authentic, and then broadcast it throughout the network. Each hub then updates its databases with the completed transactions. For this calculation work, miners are granted a virtual bonus, encouraging more individuals to participate in the network. Once the exchange has been verified, the transaction procedure ends successfully. Ultimately, the invulnerability of the blockchain’s cryptocurrency to information change is what gives it its legitimacy.

One distinctive quality of most cryptocurrencies is very unpredictable price fluctuations, that result in an immediate impact on investments and are controlled by a variety of variables. To make strategic judgments and maximize investment returns, professional traders require a reliable forecasting tool [6]. The assessment of impending price changes is further made more challenging by the minimal correlations of digital currency with traditional assets. Furthermore, numerous factors may impact the price of cryptocurrencies, including changes in macroeconomic statuses, impact world events and even misleading news, governmental regulations, and volatile social website content [8].

Determining and effectively interpreting complex relations requires models with sufficient complexity. Additionally, a model needs to be capable of tackling data formulated as a time series. While recurrent neural networks show great promise for tackling time series, problems intrinsic to this approach such as vanishing gradients have led researchers to develop improved methods. One notable approach is the long-short-term memory(LSTM) [9] neural network. By using memory cells and a sophisticated set of control gates the LSTM network can select and retain data important for future forecasts and recall it when needed. These unique abilities make LSTM neural networks uniquely well-suited for price forecasting.

Like many artificial intelligence(AI) algorithms, the LSTM networks present a set of control parameters that govern specific behaviors. A trait shared with machine learning (ML) algorithms as well. While these parameters help ensure good general performance when handling different problems, they require appropriate tuning to better suit a specific task. With more modern algorithms there has been an increase in the number of hyperparameters. The process of selecting specific values has traditionally been handled manually. However, the increasing number of possible combinations makes this process difficult. Therefore, methods of automation have been developed in the form of hyper-parameter tuning. By framing the selection process as an optimization task, we can address the process of selection. Nevertheless, determining optimal hyperparameters is often understood to be an NP-Hard problem, and algorithms capable of handling this form of optimization are needed.

When handling optimization problems, few groups of algorithms have shown the same success swarm intelligence algorithms have had in this field. By simulating behaviors often inspired by natural groups cooperating with a goal in mind, such as hunting or reproduction, swarm intelligence algorithms have demonstrated their ability to address complex optimizations. By refining a population of agents that represent potential solutions, through carefully crafted exploration and exploitation mechanisms swarm algorithms can even handle NP-hard problems. This makes swarm intelligence algorithms a promising contender when optimizing hyperparameter selection.

The principal focus of this manuscript is, however, a cryptocurrency called Ether whose blockchain is provided by the Ethereum platform, developed and presented by Vitalik Buterin in 2013 [10] which became operational in 2015. Therefore, we are proposing a method for Ethereum price univariate forecasting by employing Long Short-Term Memory Networks that are optimized using a metaheuristic algorithm called Arithmetic Optimization Algorithm (AOA). This approach outperforms other modern metaheuristics when the mean and best cases across 15 independent iterations are considered while performing insignificantly worse in other situations. Furthermore, this work aims to expand on previous research [11].

The primary scientific focus of the conducted research can be summered as the following:

•
A proposal for a novel improved AOA metaheuristic that improves the already admirable performance of the original.
•
The introduction of a novel LSTM-based approach for accurately forecasting Ethereum prices five steps ahead.
•
The application of the introduced novel metaheuristic for optimizing hyperparameter values of a LSTM network to further improve performance.

The rest of this work is formulated as per the following: Section 2 presents the most relevant published literature tackling similarly challenging issues using novel ML and AI algorithms and presents a brief background overview of LSTM neural networks as well as swarm intelligence. The subsequent Section 3 describes in detail the methods that form the backbone of this research and the novel introduced improved algorithm. In Section 4 and Section 5, the experimental setup and attained results are presented and discussed. Finally, Section 6 presents a conclusion on the conducted research and presents proposals for future research works.
2. Background and related works

With the increased availability of computational resources researchers have started exploring potential applications of computational methods to address ever more complex problems. A recently popular subject is the topic of secure and distributed transactions. By leveraging complex peer-to-peer technology with advanced crystallographic protocols research has developed fully digital currencies that offer several advantages over traditional methods. Some improvements include independence from central authorities such as governments and banks. This new paradigm shift in computing has resulted in novel approaches being developed. The initial white paper that introduces Bitcoin [2] introduced several methods of protocols for tackling secure transactions in a distributed peer-to-peer network. These novel principles inspired many researchers to develop novel variations of the original cryptocurrencies each with their own unique contributions [12]. Nevertheless, Bitcoin had a huge first-to-market advantage and quickly established market dominance that it maintains to this day.

While Bitcoin had many advantages by being the first cryptocurrency on the market, novel currencies have managed to carve out their own potentially profitable niche. Some interesting examples tether the United States Dollar to a virtual coin in hopes of stabilizing price volatility. Other currencies reward users by performing certain actions of making their own hardware available for community use. However, few have seen the rapid success of Ethereum [13]. Ether, itself built on the basis of the Ethereum blockchain was introduced in 2015, and notably quickly established market dominance second only to Bitcoin. The rapid economic growth of Ethereum has made it a potentially very lucrative venture. However, the lack of theater of virtual currencies to real-world assets makes them prone to rapid and drastic variations in price [14]. These changes make cryptocurrencies a risky venture and often deter potential investors. A need for a robust approach that is capable of accurately forecasting prices while accounting for rapid variations is apparent.

By formulating price data as a time series domain problem and applying sophisticated algorithms capable of working within this domain can be applied to cast predictions. One promising approach that has seen an increase in popularity within the last decade is the application of neural networks. Artificial neural networks mimic structures observed in biological brains. By mathematically modeling neurons and synapses that connect them. neural networks have been able to address problems in dynamic environments and adjust connections to adapt without the need for explicit programming. By increasing the number of layers in network architecture, more complex tasks may be addressed. Deep networks have in recent years shown great promise for addressing more complex tasks. However, this comes at a cost of increased computational demands. Recent global silica shortages have limited the immediate availability of computational resources. This has further emphasized the need for more optimized and lighter networks capable of making the best of the available hardware. Therefore optimization plays an increasingly crucial role in model development and testing.

Traditional networks are incapable of addressing time series and rely on single-point predictions. One approach for handling problems in the time domain is by adapting recurrent neural networks(RNN) [15] that leverage cyclical connections within the network. This allows previous inputs to affect subsequent predictions. However, RNNs have several issues intrinsic to this approach that can cause problems when put into practice. One notable shortcoming is the problem of vanishing gradients [16] that make training less effective and more difficult. Therefore, novel methods capable of overcoming these shortcomings have been developed by researchers [9].

A novel approach that shows great promise when applied to time-series forecasting is utilizing LSTM [9] networks. This novel variant of recurrent networks uses a special cell state and various gates that allow networks to retain a certain amount of information within the network and release it when deemed necessary. Researchers have applied this category of a network to various fields including oil price forecasting [17]. However, to attain good performance adequate control parameters needed to be selected and optimized. While traditionally, this process is often handled through trial and error, increasing numbers of parameters presented in novel algorithms have created a need for techniques that automate the process of selection.

Hyperparameter tuning is an increasingly popular method used by researchers to improve the performance of existing models. By formulating algorithm performance as an optimization task, and applying novel algorithms, the process of selection can be automated. This in turn further boosts the overall performance of the resulting methods. Researchers have successfully applied this method to several algorithms and across multiple fields. Some notable examples include optimizing XGboost parameters for various purposes [18, 19, 20], as well as optimizing LSTM parameters [21, 22].

2.1 LSTM

a LSTM network [23] a variation of a of RNN, that enables data to be maintained within cell states. As a result, past inputs have an impact on those in the future, making it particularly suited to time-series predictions. A conventional network’s cells are substituted with memory nodes in network layers to impart the ability to data storage. Three different types of gates including input, output, and forget. These form the building blocks of up memory nodes, which work by judicious capturing and releasing data that passes through them.

Data flowing through a forget gate $f_{t}$ forms a primary first stage of an LSTM architecture. The forget gate decides whether to delete the present information from a specific memory node. The definition of a forget gate $f_{t}$ is given in Eq. (1).

$\displaystyle f_{t}=\sigma\left(W_{f}x_{t}+U_{f}h_{t-1}+b_{f}\right)$ (1)

where $f_{t}$ may take on values in the range [0,1], $\sigma$ denotes the sigmoid function, $W_{f}$ and $U_{f}$ signify variable-weight matrices, and $b_{f}$ represents the bias vector.

Data is chosen and stored in memory cells by the ensuing stem. Additionally, the sigmoid function determines values that will be regenerated and are established per Eq. (2) for a particular input gate.

$\displaystyle i_{t}=\sigma\left(W_{i}X_{t}+U_{i}h_{t-1}+b_{i}\right)$ (2)

Here, $i_{t}$ may appear in the interval (0,1). Additionally, $W_{i}$ , $U_{i}$ , $b_{i}$ represent a set of learnable parameters that regulate the input gate.

The $t a n h$ function is used in Eq. (3) to calculate the potential update vectors $C_{t}$ .

$\displaystyle C_{t}=\tanh\left(W_{c}x_{t}+U_{c}h_{t-1}+b_{c}\right)$ (3)

in which $W_{c}$ , $U_{c}$ , $b_{c}$ denote a set of learnable parameters.

Upon making a decision on what data will be recorded, the state of a specific cell $C_{t}$ is calculated using Eq. (4).

$\displaystyle C_{t}=f_{t}\odot C_{t-1}+i_{t}\odot C_{t}$ (4)

where $\odot$ signifies the Hadamard product. Data to be erased is represented by $C_{t-1}$ . Similarly, data to be saved is signified by $f_{t}\odot C_{t-1}$ . New information to be recorded in a memory cell $C_{t}$ is denoted as $i_{t}\odot C_{t}$ .

Sigma function computations can be employed to determine the output gate $o_{t}$ value outlining the hidden state $h_{t}$ in line with Eq. (5).

$o_{t}=\sigma\left(W_{o}x_{t}+U_{o}h_{t-1}+b_{o}\right)$ (5)

where the range of $o_{t}$ is (0,1), and $W_{o}$ , $U_{o}$ , $b_{o}$ are learnable parameters for the input gate.

Last but not least, the output value $h_{t}$ in compliance with Eq.6 is represented by the product of $o_{t}$ and the $t a n h$ value of $C_{t}$ .

$\displaystyle h_{t}=o_{t}\odot\tanh\left(C_{t}\right)$ (6)

2.2 Swarm intelligence and literature review

Beni and Wang first proposed the concept of swarm intelligence in 1989 about the intelligent behavior of cellular robotic systems [24]. This area of study examines both man-made and systems present in nature in which many individual units work together to create favorable outcomes for an entire population through decentralized control and self-organization. A typical swarm intelligence system consists of several discrete “boids” [25] that interact both locally and with their surroundings. Boids are an example of emergent behavior; unknowing to these individuals, the formation of intelligent global behavior is the result of seemingly random actions taken by individual agents while adhering to a set of basic rules in the absence of a centralized framework to govern their coordination.

The core principles behind the functioning of swarm intelligence algorithms predominantly stem from nature. The most famous algorithm inspired by the latter and initially intended to simulate social behavior [26] is Particle Swarm Optimization (PSO). This class of algorithms formulates a technique made for various tasks that could be formulated as optimizations. Other notable representatives of this family are: artificial bee colony(ABC) [27], firefly algorithm(FA) [28], sine cosine algorithm (SCA) [29], bat algorithm(BA) [30], and many others [31, 32, 29]. However, various hybrid optimization methods have proven to utilize the best traits of a swarm and conventional machine learning models, and they show an increasing trend in popularity among researchers. A few noteworthy cases include the two-stage GA-PSO-ACO algorithm [33], interactive search algorithm (ISA) [34], ABC-BA [35], Swarm-TWSVM [36], and those used for forecasting, such as a hybrid consisting of support vector machines (SVM) and RNNs, called RSVR [37].

Although they have different methods for approaching problems, and according to the no free lunch (NFL) theorem, there is no single best approach for all presented problems [38]. Swarm intelligence algorithms often do optimization tasks exceptionally well. Even NP-hard problems may be addressed and solved by swarm intelligence algorithms with satisfactory outcomes consuming reasonable resources and in a fair amount of time.

Metaheuristics algorithms have also been recently used to deal with various issues in the domain of artificial intelligence, such as stock market forecasting [39] plant classification [40], intrusion detection and security [41, 42, 43], spam detection [44], and medical applications [45, 46, 47, 48] to mention the few.

3. Proposed method

3.1 Arithmetic optimization algorithm

The AOA represents a novel algorithm that draws inspiration from fundamental operators in mathematics [49]. The procedure of refining solutions in metaheuristic algorithms is subdivided into two elementary stages referred to collectively as the search phase. These stages are exploration and exploitation. During the initial stage the algorithms are tasked with scouring the unexplored areas of the problem space, the latter stage assigns a goal of focusing on the spaces that have already been investigated.

The optimization process initiates with a randomized matrix $X$ , in which $x_{ij}$ represents a lone agent. This will represent the initial solution space set for optimization. As iterations progress, each outputs a best-candidate solution which is deemed as the finest agent reached in the preceding executions. The areas where potential optimal values can perhaps still be found exist during the course of the optimization and are computed using Divide, Multiply, Subtract, and Add operations. The Math Optimizer Accelerated (MOA) function, which will be employed throughout both key stages in the optimization cycle, is at first calculated by AOA to select the search phase:

$\displaystyle\textit{MOA}(\text{C\_}\textit{Iter})=\textit{ Min }+\text{C\_}% \textit{Iter}\times\left(\frac{\textit{ Max }-\textit{ Min }}{\text{M\_}% \textit{Iter}}\right)$ (7)

in which $\textit{MOA}(\text{C\_}\textit{Iter})$ represents the attained MOA value during the $t$ th cycle, C_Iter signifies the ongoing cycle between 1 and the maximum allowed a number of optimization cycles M_Iter. Minimum and maximum values of the accelerated function are denoted by Min and Max, respectively.

In the exploration phase, Division ( $D$ ) and Multiplication ( $M$ ) operators randomly examine the search space in various areas in order to reach a more suitable solution. Equation (8) provides a model for these tactics. When a random number $r1>\textit{MOA}$ , the MOA function limits the search in this phase. The first constraint in Eq. (8) states that the other operator ( $M$ ) in this stage won’t be considered until the first operator ( $D$ ), whose current task is confined by a random integer $r2<0.5$ , has finished it. Otherwise, the $D$ will be substituted with the alternative operator ( $M$ ) to complete the current problem.

$\displaystyle x_{i,j}(\text{C\_}\textit{Iter}+1)=\begin{cases}\text{best}(x_{j% })\div(\textit{MOP}+\epsilon)\times\left((\textit{UB}_{j}-\textit{LB}_{j})% \times\mu+\textit{LB}_{j}\right),&r2<0.5\\ \text{best}(x_{j})\times\textit{MOP}\times\left((\textit{UB}_{j}-\textit{LB}_{% j})\times\mu+\textit{LB}_{j}\right),&\text{otherwise}\end{cases}$ (8)

whereby $\epsilon$ is an arbitrary small integer, $\mu$ denotes a previous defined control value, $x_{i,j}(\text{C\_}\textit{Iter}+1)$ represents $i$ th agent in the subsequent optimization cycle, $x_{i,j}(\text{C\_}\textit{Iter})$ signifies $j$ th location of the $i$ th agent in the current optimization cycle, $\text{best}(x_{j})$ stands for the $j$ th location of the currently finest performing agent. Finally, $\textit{LB}_{j}$ and $\textit{UB}_{j}$ represent the lowest and highest boundaries for the $j$ th location, respectively.

$\displaystyle\textit{MOP}(\text{C\_}\textit{Iter})=1-\frac{\text{C\_}\textit{% Iter}^{1/\alpha}}{\text{M\_}\textit{Iter}^{1/\alpha}}$ (9)

Here, Math Optimizer Probability $\textit{MOP}(\text{C\_}\textit{Iter})$ signifies the attained value in the $t$ th optimization cycle, C_Iter is the ongoing optimization cycle, M_Iter represents the number of optimization cycles, and $\alpha$ signifies a fixed value created to measure exploitation specificity throughout the iterations.

Then, using the Addition ( $A$ ) and Subtraction ( $S$ ) strategies, that are described in Eq. (10), a deep search is conducted to exploit the search space. According to the first constraint in Eq. (10), which is bound by a random integer $r3<$ 0.5, the other operator ( $A$ ) at this stage won’t be factored until the first operator ( $S$ ) has finished the task it’s currently employed on. Otherwise, the $S$ will be substituted by ( $A$ ) which will finalize the current problem, in place of $S$ .

$\displaystyle x_{i,j}(\text{C\_}\textit{Iter}+1)=\begin{cases}\text{best}(x_{j% })-\textit{MOP}\times\left((\textit{UB}_{j}-\textit{LB}_{j})\times\mu+\textit{% LB}_{j}\right),&r3<0.5\\ \text{best}(x_{j})+\textit{MOP}\times\left((\textit{UB}_{j}-\textit{LB}_{j})% \times\mu+\textit{LB}_{j}\right),&\text{otherwise}\end{cases}$ (10)

Candidate solutions gravitate towards divergence from the potentially almost optimal solution whenever $r1>\textit{MOA}$ , and they are inclined to converge on the near-optimal solution whenever $r1<\textit{MOA}$ . To stimulate exploration and exploitation, the parameter MOA is raised incrementally starting at 0.2 up until 0.9. Lastly, it is essential to note that AOA has an $O(N\times(\textit{ML}+1))$ computational complexity.

3.2 Improved AOA with firefly search (AOA-FA)

Despite the good performance of the basic AOA, certain drawbacks have been observed [50], signifying that there is room for further improvement. A notable tendency of the original algorithm to get stuck in sub-optimal regions can lead to premature convergence. in turn, resulting in reduced accuracy. This section describes a proposed strategy to mitigate this problem and improve on the excellent results presented by the original metaheuristic.

A popular and effective approach for addressing algorithm drawbacks is hybridization. By incorporating mechanisms from other well-performing metaheuristics, a novel hybrid algorithm is created, with improved overall performance.

A popular and effective approach for addressing algorithm drawbacks is hybridization. Following extensive testing, the Firefly algorithm (FA) [28] has been selected to improve the shortcomings of the AOA due to its particularly powerful exploration mechanism. Accordingly, the proposed algorithm has named the AOA with firefly search (AOA-FA).

The improved version of the algorithm introduces a search mechanism utilized by the original FA as shown in Eq. (11).

$\displaystyle x_{i}=x_{i}+\beta_{0}e_{i}^{-\gamma r^{2}_{ij}}j(x_{j}-x_{i})+% \alpha\epsilon_{t}$ (11)

where $x_{i}$ represents the position $i$ -th agent, distance to between two agents denoted as $r_{i,j}$ and $\beta_{0}$ being the attraction of fireflies at distance $r=0$ . The media absorption coefficient is $\gamma$ , and rand is a random value in range $[0,1]$ . Finally, $\alpha$ is a randomization factor that is gradually reduced through iterations. The proposed algorithm implements the FA search mechanisms during even other iterations to improve exploration power.

The proposed algorithm’s pseudocode is presented in Algorithm 3.2.

Arithmetic optimization firefly search algorithm pseudocodeDefine values for $\alpha$ and $\mu$ . Randomize solution locations $\text{C\_}\textit{Iter}<\text{M\_}\textit{Iter}$ Evaluate all solutions using a fitness function Determine the current optimal solution Update MOA in accordance with Eq. (7). Update MOP in accordance with Eq. (9). $\text{C\_}\textit{Iter}\%2==0$ Perform firefly search with respect to Eq. (11) $i=1$ to Solutions $j=1$ to SolutionsSelect random numbers for ( $r1,r2,r3$ ) parameters within between [0,1]. $r1>MOA$ Explore search space $r2>0.5$ Use the divide ( $D$ ) action Relocate $i$ th agent in accordance with the first constraint of Eq. (8). Use the multiply ( $M$ ) action Relocate $i$ th agent in accordance with the second constraint of Eq. (8). Exploit promising areas $r3>0.5$ Use the subtract ( $S$ ) action Relocate $i$ th agent in accordance with the first constraint of Eq. (10). Use the add ( $A$ ) action Relocate $i$ th agent in accordance with the second constraint of Eq. (10). increment C_IterPresent best-attained solution.

4. Experimental findings and results

In this manuscript, univariate forecasting of Ethereum was conducted using the day’s closing cryptocurrency price using daily available data that was gathered from Investing.com – a service that offers news and analysis on worldwide financial markets. During experimentation, 70% of accessible samples were employed for model training, with the latter 30% needed for evaluation. A visual representation of the dataset and the training test split is demonstrated in Fig. 1.

Figure 1.

Train/test split of closing Ethereum price used in experiments.

The evaluation of the scheme is performed using the following metrics: mean squared error (MSE), root mean squared error (RMSE), mean absolute error (MAE), and lastly mean absolute percentage error (MAPE) shown respectively in Eqs (12)–(15). In all these equations $\hat{p}_{i}$ denotes the forecast price, $p_{i}$ signifies the true price, while $N$ is the number of data samples.

Table 1

Objective function metrics over 15 runs for five-step ahead.

Method	Best	Worst	Mean	Median	Std	Var
LSTM-AOA-FS	6.701E-04	9.411E-04	7.411E-04	6.711E-04	1.047E-04	1.100E-08
LSTM-AOA	6.724E-04	9.439E-04	7.423E-04	6.724E-04	1.052E-04	1.110E-08
LSTM-FA	6.770E-04	9.220E-04	7.633E-04	7.554E-04	9.010E-05	8.120E-09
LSTM-SCA	9.003E-04	1.345E-03	1.209E-03	1.227E-03	1.628E-04	2.650E-08
LSTM-BA	6.896E-04	6.896E-04	6.896E-04	6.896E-04	0	0
LSTM-ABC	7.532E-04	8.793E-04	7.905E-04	7.833E-04	4.640E-05	2.160E-09

Figure 2.

LSTM Comparative analysis visualizations.

$\displaystyle\textit{MSE}=\frac{1}{N}\sum_{i=1}^{N}\left(\hat{p}_{i}-p_{i}% \right)^{2}$ (12) $\displaystyle\textit{RMSE}=\sqrt{\frac{1}{N}\sum_{i=1}^{N}\left(\hat{p}_{i}-p_% {i}\right)^{2}}$ (13) $\displaystyle\textit{MAE}=\frac{1}{N}\sum_{i=1}^{N}\left|\hat{p}_{i}-p_{i}\right|$ (14) $\displaystyle\textit{MAPE}=\frac{1}{N}\sum_{i=1}^{N}\left|\frac{\hat{p}_{i}-p_% {i}}{p_{i}}\right|$ (15)

Figure 3.

LSTM Comparative analysis visualizations.

Figure 4.

Five steps ahead predictions made by best-performing model optimized by each metaheuristic.

All tested models were independently implemented in Python, using TensorFlow and Keras. The prediction model used six input steps, representing six days’ worth of data, used to predict prices five steps (days) ahead. The optimization algorithms used a population of six individuals, and 8 algorithm iterations for an optimization. Furthermore, the optimization constraint for LSTM parameters where in the following ranges: number of neurons $[20,200]$ , learning rate $[0.0001,0.01]$ , number of training epochs $[20,200]$ , and dropout $[0.001,0.01]$ , while the remaining parameters remain the same as the suggested default in the TensorFlow implementation.

To demonstrate the improvements made to the introduced algorithm it has been subjected to a comparative analysis against several well-known metaheuristics including the original AOA [49], SCA [29], FA [28], BA [30], ABC [27]. Since these algorithms were tasked with optimizing LSTM network hyperparameters these are given a LSTM suffix to improve clarity. Furthermore, experimental findings followed by a comparison with other contemporary algorithms applied to LSTM optimization are demonstrated in two types of tables.

5. Discussion

Table 2
The R2, MAE, MSE, and RMSE metrics of best-generated LSTM with 5 steps ahead

	Error indicator	LSTM-AOA-FA	LSTM-AOA	LSTM-SCA	LSTM-FA	LSTM-BA	LSTM-TLB
One-step ahead	R2	0.986272	0.985089	0.984792	0.982208	0.985477	0.984165
	MAE	0.019343	0.019815	0.020117	0.022142	0.020627	0.021575
	MSE	0.000647	0.000676	0.000678	0.000835	0.000725	0.000757
	RMSE	0.025429	0.026003	0.026038	0.028890	0.026924	0.027511
Two-step ahead	R2	0.984656	0.985554	0.985812	0.980197	0.986119	0.984507
	MAE	0.020307	0.020144	0.019846	0.023145	0.020269	0.021243
	MSE	0.000687	0.000680	0.000678	0.000888	0.000683	0.000712
	RMSE	0.026210	0.026083	0.026046	0.029807	0.026129	0.026686
Three-step ahead	R2	0.986188	0.985968	0.985989	0.981245	0.985774	0.983646
	MAE	0.020226	0.019751	0.019817	0.022496	0.020379	0.021940
	MSE	0.000687	0.000653	0.000664	0.000843	0.000691	0.000747
	RMSE	0.026202	0.025550	0.025762	0.029033	0.026281	0.027332
Four-step ahead	R2	0.986659	0.984917	0.985363	0.976741	0.986009	0.983168
	MAE	0.019368	0.020823	0.020852	0.024595	0.020116	0.021908
	MSE	0.000655	0.000722	0.000704	0.001012	0.000660	0.000751
	RMSE	0.025584	0.026872	0.026538	0.031819	0.025681	0.027399
Five-step ahead	R2	0.985282	0.986101	0.986349	0.978713	0.985822	0.982977
	MAE	0.019644	0.019579	0.020040	0.023765	0.020417	0.022458
	MSE	0.000681	0.000631	0.000661	0.000923	0.000690	0.000799
	RMSE	0.026091	0.025114	0.025706	0.030383	0.026274	0.028268
Overall results	R2	0.985839	0.985524	0.985678	0.979870	0.985838	0.983697
	MAE	0.019778	0.020022	0.020134	0.023229	0.020362	0.021825
	MSE	0.000671	0.000672	0.000677	0.000900	0.000690	0.000753
	RMSE	0.025905	0.025931	0.026020	0.030005	0.026261	0.027444

Figure 5.

Five steps ahead predictions of the best performing AOA-FS-LSTM model (above) and original AOA-LSTM model (below) vs actual price values.

The following section presents and discusses the attained results by each of the metaheuristic optimized models applied to cryptocurrency forecasting. The models are evaluated in several independent runs to ensure fair grounds for an objective comparison and account for random nature of the metaheuristic algorithms. Furthermore, each metaheuristic has been evaluated under identical test conditions, given the same population size and identical number of iterations to improve population quality. Objective metrics are captured through the executions in order to wage the performance of the algorithm.

In the first Table 2 R2, MAE, MSE, and RMSE performance indicators for the objective obtained in the best run, are shown for each step separately. The proposed algorithm outperformed all contemporary metaheuristics in one and four steps ahead, while still attaining admirable results in two, three, and five-step ahead predictions. The LSTM-SCA model outperformed the proposed algorithm in two steps ahead predictions reinforcing the NFL theorem, while the based AOA achieved better results for three and five steps ahead. However, the proposed algorithm outperformed all other algorithms in overall results.

The Best, worst, mean, median, standard deviation, and variance of the objective function(MSE) averaged over 15 independent executions are demonstrated in Table 1 for five-steps ahead prediction. To highlight the best-obtained results they are marked in bold text.

The median and best performance averaged over 15 independent runs have been attained by the novel proposed algorithm, while the LSTM-BA algorithm performed better in the works and arithmetic mean further solidifying the NFL theorem.

To further emphasize the improvements made a visual comparison of the proposed LSTM-AOA-FA algorithm result distributions can be seen in Fig. 2. Additionally, the comparison of convergence speeds of all tested algorithms is demonstrated in Fig. 3.

The forecasts made by each best-performing tested model can be seen in Fig. 4.

Finally, the overall best-performing prediction model, the AOA-FS-LSTM model, alongside or original AOA-LTMS model is shown in Fig. 5.

6. Conclusion

The research presented in this paper tackles the potentially very lucrative process of forecasting cryptocurrency prices, focusing on Ether running on the Ethereum blockchain. Since price changes can be formulated as a time series an appropriate method has been selected to tackle forecasting. Due to their ability to effectively retain data from previous inputs through the use of memory cells and various gates, LSTM neural networks have been applied to price forecasting. However, as LSTM networks, much like other AI algorithms require adequate hyperparameter tuning to ensure desired functionality, this work also introduces a novel improved version of the AOA algorithm. By leveraging the powerful search mechanism of the FA the shortcomings of the original can be overcome. Due to this mechanism being introduced the novel algorithm is therefore dubbed the AOA-FS. It is applied to selecting appropriate LSTM network hyperparameter values improving performance and accuracy.

The attained results suggest that the introduced approach is favorable for predicting market prices with reasonable resources and within an acceptable time frame. Furthermore, the proposed approach attained the best results in mean and best cases in 15 independent runs and outperformed other algorithms in one and four-step ahead predictions, while performing marginally worse in other cases, suggesting that the proposed algorithm has the potential for success used for optimization and prediction.

Future research will focus on further refining the proposed approach and prediction model accuracy through additional optimization. On top of that, a goal for future research will be exploring the potential for applying the proposed novel algorithm in various fields.

References

Mukhopadhyay

Skjellum

Hambolu

Oakley

and Brooks

R.R.

, A brief survey of Cryptocurrency systems, in: 2016 14th Annual Conference on Privacy, Security and Trust (PST), 2016, pp. 745–752.

Nakamoto

, Bitcoin: A Peer-to-Peer Electronic Cash System, Cryptography Mailing list at https://metzdowd.com, 2009.

Patel

Tanwar

Gupta

and Kumar

, A Deep Learning-based Cryptocurrency Price Prediction Scheme for Financial Institutions, Journal of Information Security and Applications 55 (2020), 102583.

Balcilar

Bouri

Gupta

and Roubaud

, Can volume predict Bitcoin returns and volatility? A quantiles-based approach, Economic Modelling 64 (2017), 74–81.

Zhang

Dai

H.-N.

Zhou

Mondal

Martínez-García

and Wang

, Forecasting Cryptocurrency Price Using Convolutional Neural Networks with Weighted and Attentive Memory Channels, Expert Systems with Applications 183 (2021), 115378.

Dutta

Kumar

and Basu

, A Gated Recurrent Unit Approach to Bitcoin Price Prediction, Papers, 1912.11166, arXiv.org, 2019. https://ideas.repec.org/p/arx/papers/1912.11166.html.

Salah

and Khan

, IoT Security: Review, Blockchain Solutions, and Open Challenges, Future Generation Computer Systems 82 (2017).

Philippas

, Media Attention and Bitcoin Prices, SSRN Electronic Journal (2019).

Sundermeyer

Schlüter

and Ney

, LSTM neural networks for language modeling, in: Thirteenth annual conference of the international speech communication association, 2012.

10.

Buterin

, A Next Generation Smart Contract & Decentralized Application Platform, 2015.

11.

Stankovic

Jovanovic

Bacanin

Zivkovic

Antonijevic

and Bisevac

, Tuned Long Short-Term Memory Model for Ethereum Price Forecasting Through an Arithmetic Optimization Algorithm, in: Innovations in Bio-Inspired Computing and Applications: Proceedings of the 13th International Conference on Innovations in Bio-Inspired Computing and Applications (IBICA 2022) Held During December 15–17, 2022, Springer, 2023, pp. 327–337.

12.

Mukhopadhyay

Skjellum

Hambolu

Oakley

and Brooks

, A brief survey of cryptocurrency systems, in: 2016 14th annual conference on privacy, security and trust (PST), IEEE, 2016, pp. 745–752.

13.

Wood

et al., Ethereum: A secure decentralised generalised transaction ledger, Ethereum project yellow paper 151 (2014) (2014), 1–32.

14.

Stankovic

Bacanin

Zivkovic

Jovanovic

Mani

and Antonijevic

, Forecasting Ethereum Price by Tuned Long Short-Term Memory Model, in: 2022 30th Telecommunications Forum (TELFOR), IEEE, 2022, pp. 1–4.

15.

Medsker

L.R.

and Jain

, Recurrent neural networks, Design and Applications 5 (2001), 64–67.

16.

Hochreiter

, The vanishing gradient problem during learning recurrent neural nets and problem solutions, International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems 6 (02) (1998), 107–116.

17.

Jovanovic

Bacanin

Jovancai Stakic

Antonijevic

Magd

Thirumalaisamy

and Zivkovic

, Multi-Step Crude Oil Price Prediction Based on LSTM Approach Tuned by Salp Swarm Algorithm with Disputation Operator, Sustainability 14 (21) (2022), 14616.

18.

Zivkovic

Jovanovic

Ivanovic

Bacanin

Strumberger

and Joseph

P.M.

, Xgboost hyperparameters tuning by fitness-dependent optimizer for network intrusion detection, in: Communication and Intelligent Systems: Proceedings of ICCIS 2021, Springer, 2022, pp. 947–962.

19.

AlHosni

Jovanovic

Antonijevic

Bukumira

Zivkovic

Strumberger

Mani

J.P.

and Bacanin

, The xgboost model for network intrusion detection boosted by enhanced sine cosine algorithm, in: Third International Conference on Image Processing and Capsule Networks: ICIPCN 2022, Springer, 2022, pp. 213–228.

20.

Petrovic

Antonijevic

Strumberger

Jovanovic

Savanovic

and Janicijevic

, The XGBoost Approach Tuned by TLB Metaheuristics for Fraud Detection, in: Proceedings of the 1st International Conference on Innovation in Information Technology and Business (ICIITB 2022), Vol. 104, Springer Nature, 2023, p. 219.

21.

Venkatachalam

Trojovskỳ

Pamucar

Bacanin

and Simic

, DWFH: An improved data-driven deep weather forecasting hybrid model using Transductive Long Short Term Memory (T-LSTM), Expert Systems with Applications 213 (2023), 119270.

22.

Bacanin

Sarac

Budimirovic

Zivkovic

AlZubi

A.A.

and Bashir

A.K.

, Smart wireless health care system using graph LSTM pollution prediction and dragonfly node localization, Sustainable Computing: Informatics and Systems 35 (2022), 100711.

23.

Hochreiter

and Schmidhuber

, Long Short-term Memory, Neural computation 9 (1997), 1735–80.

24.

Beni

and Wang

, Swarm intelligence in cellular robotic systems, in: Robots and Biological Systems: Towards a New Bionics?, Springer Berlin Heidelberg, Berlin, Heidelberg, 1993, pp. 703–712.

25.

Reynolds

C.W.

, Flocks, Herds and Schools: A Distributed Behavioral Model, SIGGRAPH Comput Graph 21 (4) (1987), 25–34.

26.

Kennedy

, The particle swarm: social adaptation of knowledge, in: Proceedings of 1997 IEEE International Conference on Evolutionary Computation (ICEC ’97), 1997, pp. 303–308.

27.

Karaboga

, Artificial bee colony algorithm, Scholarpedia 5 (3) (2010), 6915.

28.

Yang

X.-S.

and Slowik

, Firefly algorithm, in: Swarm Intelligence Algorithms, CRC Press, 2020, pp. 163–174.

29.

Mirjalili

, SCA: a sine cosine algorithm for solving optimization problems, Knowledge-based systems 96 (2016), 120–133.

30.

Yang

X.-S.

and Gandomi

A.H.

, Bat algorithm: a novel approach for global engineering optimization, Engineering computations (2012).

31.

Gandomi

A.H.

Yang

X.-S.

and Alavi

A.H.

, Cuckoo search algorithm: a metaheuristic approach to solve structural optimization problems, Engineering with Computers 29 (1) (2013), 17–35.

32.

Mirjalili

S.Z.

Mirjalili

Saremi

Faris

and Aljarah

, Grasshopper optimization algorithm for multi-objective optimization problems, Applied Intelligence 48 (4) (2018), 805–820.

33.

Deng

Chen

Liu

Yin

and Guo

, A novel two-stage hybrid swarm intelligence optimization algorithm and application, Soft Computing 16 (2012).

34.

Mortazavi

ToÄŸan

and Moloodpoor

, Solution of structural and mathematical optimization problems using a new hybrid swarm intelligence optimization algorithm, Advances in Engineering Software 127 (2019), 106–123.

35.

Karthikeyan

Ramasamy

Shankar

Lakshmanaprabu

S.K.

Varadarajan

Elhoseny

and Manogaran

, Energy consumption analysis of Virtual Machine migration in cloud using hybrid swarm optimization (ABCâ€“BA), The Journal of Supercomputing 76 (2020).

36.

Houssein

Ewees

and Elsayed Abd Elaziz

, Improving Twin Support Vector Machine Based on Hybrid Swarm Optimizer for Heartbeat Classification, Pattern Recognition and Image Analysis 28 (2018), 243–253.

37.

Sumi

Zaman

and Hirose

, A rainfall forecasting method using machine learning models and its application to the Fukuoka city case, International Journal of Applied Mathematics and Computer Science 22 (2012), 841–854.

38.

Wolpert

D.H.

and Macready

W.G.

, No free lunch theorems for optimization, IEEE transactions on evolutionary computation 1 (1) (1997), 67–82.

39.

Jovanovic

Milutinovic

Gajevic

Krstovic

Rashid

T.A.

and Petrovic

, Sine Cosine Algorithm for Simple recurrent neural network Tuning for Stock Market Prediction, in: 2022 30th Telecommunications Forum (TELFOR), IEEE, 2022, pp. 1–4.

40.

Bukumira

Antonijevic

Jovanovic

Zivkovic

Mladenovic

and Kunjadic

, Carrot grading system using computer vision feature parameters and a cascaded graph convolutional neural network, Journal of Electronic Imaging 31 (6) (2022), 061815.

41.

Jovanovic

Antonijevic

Stankovic

Zivkovic

Tanaskovic

and Bacanin

, Tuning Machine Learning Models Using a Group Search Firefly Algorithm for Credit Card Fraud Detection, Mathematics 10 (13) (2022), 2272.

42.

Zivkovic

Petrovic

Venkatachalam

Strumberger

Jassim

H.S.

and Bacanin

, Novel Chaotic Best Firefly Algorithm: COVID-19 Fake News Detection Application, in: Advances in Swarm Intelligence, Springer, 2023, pp. 285–305.

43.

Zivkovic

Bacanin

Arandjelovic

Strumberger

and Venkatachalam

, Firefly Algorithm and Deep Neural Network Approach for Intrusion Detection, in: Applications of Artificial Intelligence and Machine Learning, Springer, 2022, pp. 1–12.

44.

Salb

Jovanovic

Zivkovic

Tuba

Elsadai

and Bacanin

, Training Logistic Regression Model by Enhanced Moth Flame Optimizer for Spam Email Classification, in: Computer Networks and Inventive Communication Technologies, Springer, 2023, pp. 753–768.

45.

Bezdan

Zivkovic

Bacanin

Chhabra

and Suresh

, Feature Selection by Hybrid Brain Storm Optimization Algorithm for COVID-19 Classification, Journal of Computational Biology (2022).

46.

Zivkovic

Bacanin

Venkatachalam

Nayyar

Djordjevic

Strumberger

and Al-Turjman

, COVID-19 cases prediction by using hybrid machine learning and beetle antennae search approach, Sustainable Cities and Society 66 (2021), 102669.

47.

Bezdan

Zivkovic

Tuba

Strumberger

Bacanin

and Tuba

, Glioma brain tumor grade classification from mri using convolutional neural networks designed by modified fa, in: International conference on intelligent and fuzzy systems, Springer, 2020, pp. 955–963.

48.

Gajevic

Milutinovic

Krstovic

Jovanovic

Marjanovic

and Stoean

, Artificial Neural Network Tuning by Improved Sine Cosine Algorithm for HealthCare 4.0, in: Proceedings of the 1st International Conference on Innovation in Information Technology and Business (ICIITB 2022), Vol. 104, Springer Nature, 2023, p. 289.

49.

Abualigah

Diabat

Mirjalili

Elsayed Abd Elaziz

and Gandomi

, The Arithmetic Optimization Algorithm, Computer Methods in Applied Mechanics and Engineering 376 (2021), 113609.

50.

Chen

Zhou

and Luo

, An Improved Arithmetic Optimization Algorithm for Numerical Optimization Problems, Mathematics 10 (12) (2022), 2152.

Tuned long short-term memory model for Ethereum price forecasting via an arithmetic optimization algorithm

Abstract

Keywords

1. Introduction

2.1 LSTM

3. Proposed method

3.1 Arithmetic optimization algorithm

Table 2 The R2, MAE, MSE, and RMSE metrics of best-generated LSTM with 5 steps ahead

References

Table 2
The R2, MAE, MSE, and RMSE metrics of best-generated LSTM with 5 steps ahead