Abstract
Stock market plays a vital role in country's economy. The estimation of stock market trends and price is important for the investors. Stock forecasting has piqued the interest of investors worldwide. Investors are looking for a forecasting model that is accurate and reliable enough to account for volatile and nonlinear market behavior. The problem of stock price prediction becomes a complicated one due to multiple inputs like technical indicators, financial factors, trends of international markets, and social media data. This research seeks to collect and correlate multiple inputs to analyse their impact on the stock market and improve stock price prediction accuracy. Researchers are attempting to refine effective indicators to aid in stock market forecasting by taking the advantage of ever-growing world wide web. Effective indicators like stock-related events and public sentiment towards the stock plays a significant role in stock volatility, hence can be used in improving prediction accuracy. The paper provides an efficient framework for optimizing the error in short-term stock market prediction with diverse data sources, based on a novel Mayfly Adam Optimization Algorithm (MAOA) and Deep Residual Network (DRN). This novel approach uses Deep Q Network (DQN) with dice coefficient similarity for feature fusion and DRN for prediction. We have considered historical stock market prices of Apple Inc. (AAPL), commodity time-series dataset, news articles and reviews on social media to evaluate the model's effectiveness. The proposed technique accuracy is found to be 96%, with average accuracy of 98.27%. The existing techniques like, Deep Neural Network (DNN), Artificial Neural Network (ANN), Long Short-Term Memory (LSTM), and Random Forest (RF) models provided accuracy of 82.4%, 85.5%, 88.8%, and 92.39%, respectively. This shows that the proposed model is 13.6% better than DNN, 10.5% better than ANN, 7% better than LSTM and 3.61% better than RF in terms of accuracy.
Introduction
In the current era stock markets determines the prosperity of any country with the financial market acting as center. Technical and Fundamental analysis are the two approaches that help investors analyze the market and make predictions (Angadi and Kulkarni, 2015). Technical analysis uses past stock prices and Fundamental analysis utilizes the information received from news, profitability, and macroeconomic factors (Picasso et al., 2019). As qualitative inputs like news, corporate profit reports, and regulatory announcements serve an important role in influencing the investment decisions of investment professionals and portfolio managers around the globe, researchers have combined these two approaches to improve prediction (Khan et al., 2022).
Majority of quantitative stock price research and prediction models use trends as their main metric to forecast the stock market. Such indicators may achieve accurate intraday predictions, but when used for delivery-based trading, they have scalability concerns, restricting their prediction accuracy in real-time. Some of the researchers have combined multiple factors (e.g., stock data, stock news, stock tweets) without considering cross-linkage or overlapping impact. Models that leverage on publicly available content have restricted correlation capacities due to inefficiencies in trend analysis.
The challenges raised above provide researchers a new perspective for improving prediction performance. The primary contributions of this research are given below:
A novel fusion framework is designed for stock market prediction utilizing MAOA- based DRN. Features are extracted from three input data sets. The process of feature fusion is carried out based on DQN with dice coefficient similarity and prices of stock market are effectively predicted utilizing DRN where the network is optimally tuned using the proposed MAOA. Hybridization of Mayfly with Adam optimization and DRN is done to achieve optimal stock prediction with minimum error and high accuracy.
The paper is organized into five sections. Section 2 covers the background of stock market prediction techniques to provide context for the review and rationale. Section 3 details the proposed stock market prediction model, while section 4 outlines the simulated model results and subsequent discussion. Finally, section 5 wraps up the study and explores future possibilities.
Literature Review
Previous research has explored stock market prediction through diverse machine and deep learning methods. Li et al. (2014) utilized the Bag-of-Words (BoW) model to assess the connection between stock price changes and textual patterns. Luss and Aspremont (2015) applied multiple kernel learning to fuse information from news texts and equity returns, noting detectability of changes in profit yields’ magnitude rather than direction. Weng et al. (2017) emphasized merging web data with historical time series and technical indicators, achieving an 85% accuracy in forecasting Apple (AAPL) NASDAQ stock movements using various machine learning techniques. Efendi et al. (2018) employed a triangular fuzzy number data preparation method and built an enhanced fuzzy random auto regression model for dynamic stock market prediction.
Peng and Jiang (2016) used a Neural Network (NN) to detect market movements, leveraging sentiment data from text as an embedding. Althelaya et al. (2021) enhanced stock price prediction accuracy with a Deep Neural Network (DNN) by integrating multi-resolution evaluation, employing an empirical wavelet transformation. De Oliveira Carosia et al. (2021) utilized sentiment analysis of financial news with an Artificial Neural Network (ANN) for stock price prediction, although lacking automated investment strategies. Shi et al. (2020) employed deep neural networks for emotional feature extraction in stock price forecasting, but not suitable for analyzing significant stock events. Jing et al. (2021) developed a hybrid LSTM-NN method for technical indicator evaluation and sentiment analysis, outperforming baseline classifiers in categorizing investor sentiments, albeit not effective for social network data analysis. Hu et al. (2021) surveyed various deep learning methods such as CNN, LSTM, DNN, RNN, RL, HAN, self-paced learning, and Wavenet, reviewing datasets, variables, models, and findings. Htun et al. (2023) examined studies combining feature analysis and ML in stock market applications. Ashtiani and Raahmei (2023) identified gaps and challenges in financial market forecasting, proposing future research directions.
Chun et al. (2021) devised an Emotion-Based Stock Prediction System (EBSPS) considering investors’ multidimensional emotions. Alsayat (2022) developed an ensemble model integrating LSTM with existing classifiers, improving sentiment analysis by studying word contextual connections and learning rare words. Kabbani and Usta (2022) developed a big data Spark framework to test ML classifiers LR, RF, and GB, achieving high accuracy in detecting stock prices with technical indicators and sentiment analysis. Wu et al. (2021) utilized the HAHTKN deep learning approach to detect SDVA intensities, converting texts into vectors via word embedding across six stages, yet struggling with words having multiple meanings. Althelaya et al. (2021) employed a DNN with empirical wavelet transform to capture price variations at different time scales, but lacked integration of diverse trading data sources and technical market evaluation for accuracy improvement. Addressing the time-consuming process of correlating news with stock prices is essential for future research (Liu and Long, 2020).
Proposed MAOA-Based DRN for Stock Market Prediction
This research proposes a unique effective model for forecasting future stock market trends and price by means of MAOA-based DRN with multiple data sources as input. The schematic representation of proposed model for stock market forecasting is illustrated in Figure 1.

Schematic View of Proposed MAOA-Based DRN for Stock Market Prediction.
Diverse data sources, such as stock market time-series data, news articles, and reviews, are used as input for the entire stock market prediction process.
Feature Extraction
The pre-requisite phase is the extraction of significant features from input data sources to get the desired output. Figure 2 depicts the multiple data sources and the various feature extraction methods used, where the appropriate features of stock time-series data, news articles, and reviews are refined.

Feature Extraction from diverse sources.
Technical indicators are mathematical tools that are used to estimate future price movements as they act as features for stock market data. (Isaac Kofi Nti et al., 2020; Xinjie Di, 2014). Details of various technical indicators used are summarized in Table 1.
Details of Extracted Features.
Details of Extracted Features.
The features extracted from news articles are TF-IDF (Shah and Kaushik, 2019), Count Vectorization, and Lin similarity (Shahzad et al., 2018).
Feature Extraction from Reviews
The features extracted from reviews are number of sentences in a review, elongated words, and capitalized words (Vo et al., 2018). Table 1 provides the details of all these features:
Concatenation of Feature Vector
Table 1 shown above highlights the fourteen features extracted. These features are concatenated to form a feature vector, which can be mathematically represented as,
Once the features are concatenated to form feature vector, feature fusion and prediction process is carried out as explained below.
Feature Fusion Using DQN with Dice Coefficient Similarity
Feature fusion is the process of fusing the features which results in low computational complexity and improved performance. After commencing the concatenation process, the obtained feature vector is applied to DQN to achieve the fused feature, where the features are sorted out based on dice-coefficient similarity. The steps of feature fusion are explained below:
The first step is to filter the features depending upon dice-coefficient similarity. After formulating the dice-coefficient for each feature, it chooses the maximum similarity valued feature for further process. The equation for dice-coefficient similarity is given as follows,
Once the dice-coefficient is computed, features are fused based on maximum similarity and the fused feature is given by,
Here,equation shown above describes a fusion process where a set of input features,
The final step is to generate
Here, data record and average value of
DQN is a remarkable network that is successfully developed by incorporating DL and Reinforcement Learning (RL) (Jin et al., 2017). DQN is comprised with a deep convolutional neural architecture for Q-function approximation, mini-batches for random training data, and existing network factors to estimate the Q-values of the subsequent state.
The vector
Loss parameter of DQN is shown as below,
To eliminate the related upgrades, experience replay along with constant maximal volume which is applied to DQN. Hence, the divergence problem resulted by related upgrades is eliminated efficiently. The benefits of employing DQN with dice coefficient similarity is that rather than calculating Q value for each frame, it is calculated for every four frames. This minimizes computing costs, improves learning and performance.
The fused feature
Architecture of DRN
Deep CNN has achieved remarkable results but, the performance gets deteriorated because of the gradient vanishing issue. To address such problems, DRN is adopted using residual learning in the conventional CNN (Chen et al., 2019). The main core of DRN is clearly explained that the layers in the original CNN are kept as such to fit a residual mapping instead of underlying mapping among input and output. Shortcut connections are employed in the deep network that leaps one or more layers, which constructs the building element of DRN, referred as a residual block, feature like identity mapping ensures that output of some layer is equal to its input. Layers are copied from the learned shallower network model and skip connections allows the gradient to flow unhindered and so it's always 1, thereby resulting in low computational efficiency and also addressing the over-fitting problem. Figure 3 depicts the DRN architecture.

Architecture of DRN.
The proposed optimization technique, MAOA is a critical component for training the DRN model to predict stock market behavior accurately. The number of data samples used in each mini-batch during training is determined by the batch size of 32, as can be seen from the above table. The total epoch (50) indicates how many times the optimization algorithm processes the entire training dataset. It influences training duration and model convergence. Learning rate, denoted as α with value of 0.01 and 0.001, regulates the step size at which the model's weights are modified after each iteration of the optimization process. β1 and β2 with value of 0.9 and 0.999 controls the influence of previous gradient information on the current update. Epsilon (ε-1e-4) is a small constant added to the denominator to prevent division by zero and stabilize the updates, especially when the denominator is close to zero. Weight decay is a regularization term added to the loss function to prevent overfitting.
Mayfly is a biologically inspired meta-heuristic optimization that incorporates benefits of both swarm intelligence and evolutionary algorithms. By integrating Adam optimization into Mayfly can deliver optimal performance with better accuracy in predicting the future stock prices. The process of nuptial dance and random flight processes aid in the approach's escape from local minima and improve the equilibrium between its exploration and exploitation properties. The Adam algorithm is effective in terms of accuracy, have low memory requirements and convergence rate is fast.
Mayfly Position Encoding
The encoding implies the solution representation of a given optimization problem and it is expressed as,
The algorithmic procedure followed by MAOA is illustrated below:
Step 1: Initialize the Population
The primary stage is to initialize the population of male and female mayflies in an I - dimensional area by a randomly distributed manner. Therefore, the set of male and female population is represented as ‘S’ and ‘W’ and velocity is represented as
This function is introduced to estimate optimum solutions.
Here,
Male mayflies are generally grouped in swarms, clearly explicit that the position of individual male mayfly is tuned based on its self-experience and also the experience of neighbor mayflies.
Let us consider the position of
The movement of a male mayfly,
Where
Substituting Equation (9) in Equation (8),
The standard expression of Adam optimization is given by,
Let us assume,
Then, the above Equation (11) can be rewritten as follows,
Finally, the updated solution of MAOA is given by,
Here,
Female mayflies fly in direction of male mayflies instead of gathering in swarms for breeding purposes. Let us consider
According to the objective factor, the best female must be engaged with the best male. Their velocities are formulated as follows,
Here,
The optimal position of mayfly
Moreover, the global best position
Here,
The crossover factor is employed to indicate the mating mechanism among two mayflies: One parent is selected from each of the male and female groups. The selection process can be done in two ways: (a) random process and (b) using fitness function. The first best female breeds with the first best male and so on. The outcomes of the crossover operator are two off springs, which are represented below:
To address the possibility of premature convergence in which the ideal value is the local optimal rather than the global optimal, a normal distribution random number is added to the selected progeny mayfly for mutation. The following is the progeny mayfly mutation formula:
Where
The procedure is repeated until the optimal solution is achieved.
The demonstration of designed method is done in Python tool utilizing PC possessing 10 OS, 4GB RAM with intel core-i5 processor.
The dataset exploited in this approach is Apple (AAPL) (data-1) and commodity dataset (data-2). The historical stock data spans the time period from 02-Jan-2018 to 22-Feb-2022. The Apple (AAPL) stock data is collected from https://finance.yahoo.com/ and commodity data is collected from (IMF-https://www.imf.org/en/Research/commodity-prices). The dataset consists of open, close, high, and low values along with the volume data. However, the news articles and review data are also included in both data-1 and data-2 from 02-Jan-2018 to 22-Feb-2022 and are collected from online sources like local news RSS feeds, Google business reviews etc.
Data Analysis and Results
The performance of the proposed MAOA-based DRN is analyzed and compared with the classical models, such as DNN (Althelaya et al., 2021), ANN (De Oliveira Carosia et al.,2021), LSTM (Jing et el.,2021), RF (Kabbani and Usta, 2022), EO-SVR-based forecasting (Houssein et al., 2022), and ISSA-BP (Liu and Long, 2020). This section represents the estimation of devised MAOA-based DRN in regard with performance measures by changing the learning set from 50% to 90%. The performance measures utilized are MSE, RMSE and MAPE.
Analysis Using Prediction Accuracy
Figure 4 represents the estimation graph based on prediction of stock prices for data-1 with respect to time. The model is evaluated on 500 days of data, and at the 200th day, the original value of the data1 stock is 54.340.

Analysis based on prediction for Data-1.
In contrast, the stock values predicted are 51.987 by the proposed MAOA-based DRN, 44.831 by DNN, 46.461 by ANN, 48.308 by LSTM, and 50.210 by RF. Consequently, the accuracy of the proposed MAOA-based DRN is 96% when compared to the accuracy of DNN, ANN, LSTM, and RF which is 83%, 86%, 88.8%, and 92%, respectively.
Figure 5 shows the analysis graph based on prediction of stock prices for data-2 in terms of time for 500 days.

Analysis Based on Prediction for Data-2.
At the 200th day, the stock market's original value is 0.070, while the stock prices predicted are 0.066 by the proposed MAOA-based DRN, 0.057 by DNN, 0.058 by ANN, 0.061 by LSTM, and 0.064 by RF. Therefore, the accuracy of the proposed MAOA-based DRN is 94.2%, whereas the accuracy of the DNN ANN, LSTM, and RF is 81.4%, 82.8%, 87%, and 91% respectively.
Table 2 shown below elucidates the discussion of MAOA-based DRN on various evaluation metrics.
Comparison of Proposed Method with Traditional Methods.
Comparison of Proposed Method with Traditional Methods.
The MSE yielded by the designed MAOA-based DRN is 0.101 when the learning rate is set as 90%, while the competing models acquired the MSE value as 0.286 for DNN, 0.263 for ANN, 0.275 for LSTM, 0.256 for RF, 0.273 for EO-SVR-based forecasting model, and 0.237 for ISSA-BP model. RMSE achieved by classical models, like DNN, ANN, LSTM, RF, EO-SVR-based forecasting model and ISSA-BP is 0.535, 0.513, 0.524, and 0.506, 0.523, and 0.486, respectively. However, the proposed MAOA-based DRN obtained RMSE as 0.318. MAPE achieved by designed model and traditional approaches, such as DNN, ANN, LSTM, RF, EO-SVR-based forecasting model and ISSA-BP is 0.205, 0.267, 0.245, 0.234, and 0.215 0.256, and 0.229 respectively.
It is proven that MAOA-based DRN has delivered better than traditional methods with a minimum MSE of 0.101, minimum RMSE of 0.318, and minimum MAPE of 0.205 for data-1. Table 3 shows the discussion of MAOA-based DRN with that of traditional algorithms.
Comparison of Proposed Method with Combinations of DRN and Algorithms.
The technique attained the MSE value as 0.118, if the learning rate was set as 90%, while the conventional models achieved MSE as 0.280 for DNN, 0.275 for ANN, 0.264 for LSTM, and 0.247 for RF. RMSE attained by the proposed methodology is 0.343, while the traditional techniques attained RMSE measure as 0.529, 0.524, 0.514, and 0.497 for DNN, ANN, LSTM, and RF respectively. MAPE obtained by the developed MAOA-based DRN is 0.216, while the competing models attained MAPE as 0.279 for DNN, 0.256 for ANN, 0.250 for LSTM, and 0.235 for RF. MAOA-based DRN has yielded higher performance than traditional algorithms, with a minimum MSE, RMSE, and MAPE with the values of 0.118, 0.343, and 0.199, for data-1.
Table 4 shows data for one day prior, the model predicted the trend and values for the following day. The prediction was done on 11-10-23 for the to be price on 12-10-23 and then later compared with the actual price of 12-10-23 to test the model accuracy. The results show prediction above 97% consistently for various sectors stocks and on various dates. The highest prediction accuracy was achieved 99.2%, with average accuracy of 98.27%.
Predicted Vs Actual Stock Values and Trend.
The convergence analysis of the model is given in Figure 6. One of the remarkable aspects of MAOA is its ability to identify the global optimum, or at least a solution that is very close to the global optimum.

Convergence analysis.
Here, the convergence analysis is evaluated by varying the iteration from 1 to 50 and the fitness is minimum. The algorithm is converged at iteration = 50 with error level below threshold value. When the iteration is 50, the fitness of the devised algorithm is 0.1243.
After doing a computational time analysis, it was found that the MAOA-based DRN required a minimum of 7.8525 s, whereas the DNN, ANN, LSTM, RF, EO-SVR, and ISSA-BP required 14.857, 10.546, 9.248, 11.687, 8.586 and 10.985, respectively.
Statistical Analysis
To assess the performance of the proposed MAOA model in comparison to a baseline optimization technique, a statistical significance test was conducted using hypothetical data. The following null and alternative hypotheses were formulated:
For this hypothetical test, a sample dataset of performance scores for both the MAOA model (Sample A) and the baseline technique (Sample B) was generated as:
To determine whether there is a statistically significant difference between the two samples, a two-sample t-test was performed at a significance level (α) of 0.05.
t = (μA - μB) / √ ((σA t = ≈ 3.52 df = 10 + 10 - 2 = 18
t (calculated) ≈ 3.52, t (critical) ≈ ± 2.101
The statistical analysis is carried out based on the best, mean, and variance of evaluation metrics, such as MSE, RMSE, and MAPE which is depicted in Table 5.
Statistical Analysis.
Statistical Analysis.
The conclusion drawn from the statistical analysis shown in Table 5 is that the MAOA-based DRN exhibits low variance compared to all existing methods which indicates that the model is less susceptible to changes in the training data and can make reliable predictions.
We have proposed a hybrid optimization algorithm, namely MAOA-based DRN for stock market prediction using multiple data sources. It has demonstrated high accuracy in prediction and consistent outcomes, with minimum MSE of 0.101, RMSE of 0.318, and MAPE of 0.205 on Dataset 1 and 2 without DRN optimization and MSE of 0.118, RMSE of 0.343, and MAPE of 0.199 with DRN optimization. As a future work, the extraction of important features from financial information can be useful. Real-time testing of prediction methodologies and identification of sentiments from social media data generated by bots needs more focus in future.
Footnotes
Funding
The authors received no financial support for the research, authorship, and/or publication of this article.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
