Firefly-Aquila optimized Deep Q network for handoff management in context aware video streaming-based heterogeneous wireless networks

Abstract

Handoff management is the method in which the mobile node maintains its connection active when it shifts from location to other. The devastating success of mobile devices as well as wireless communications is emphasizing the requirement for the expansion of mobility-aware facilities. Moreover, the mobility of devices requires services adapting their behavior to abrupt context variations and being conscious of handoffs, which make an intermittent discontinuities and unpredictable delays. Thus, the heterogeneity of wireless network devices confuses the situation, since a dissimilar treatment of handoffs and context-awareness is essential for every solution. Hence, this paper introduced the Deep Q network-based Firefly Aquila Optimizer (DQN-FAO) for performing the handoff management. In order to establish the handoff management, the process of selecting network is very important. Here, the network is selected based on the devised FAO algorithm, which is the consolidation of Aquila Optimizer (AO) and Firefly algorithm (FA) that considers the metrics, such as Jitter, Handoff latency, and Received Signal Strength Indicator (RSSI) as fitness function. Moreover, the handover decision is taken by the DQN, where the hyper-parameters are tuned by the devised FAO algorithm. According to the hand over decision taken, the context aware video streaming is happened by adjusting the bit rate of the videos using network bandwidth. Besides, the devised scheme attained the superior performance based on the call drop, energy consumption, handover delay, throughput, handoff latency, and PSNR of 0.5122, 7.086 J, 10.54 ms, 13.17 Mbps, 93.80 ms and 46.89 dB.

Keywords

Aquila Optimizer Firefly algorithm Deep Q network Handoff management heterogeneous Wireless Networks

1. Introduction

People can now migrate from one location to another and adapt to various networks based on their needs. Numerous mobile nodes (MNs) are interconnected with diverse wireless systems, like Worldwide Microwave Communication Interoperability (WiMAX) and Wireless Fidelity (Wi-Fi). Hence, heterogeneous networks (HetNets) are required to keep the users linked with any one of the networks as per choice of the user at specified time. It has a well-defined structure that contains micro cells, such as femtos, picos and Wi-Fi, which are overlaid by the macro level networks [18]. The HetNets poses a greater number of real-world tasks related to the complex system resources in a regimented system [33]. These networks are extensively used in both business and education to manage high data rate as well as the rising demand for global coverage [10]. HetNet resources are efficiently utilized on the basis of Radio Resource Management (RRM) approach along with handoff mechanism. The handover mechanism is significant for assuring heterogeneous wireless networks’ performance [15,28]. Furthermore, vertical handoff time decision approach has to be established, because it directly influences the transmission characteristics as well as streaming video quality. When vertical handoff measures as well as network chosen period are assessed, the handoff process is carried out, streaming video series are subjected to the target network over serving network. In vertical handoff streaming video broadcast ecosystem, the transmitted video has to be corrupted. Consequently, streaming video server and client play a role in the vertical handoff environment to guarantee the quality of the streaming video [23].

The data about the MN position and deployment management in certain region of radio communication networks are kept in the mobility management system. It enables the stable connectivity between the networks by means of handoff technique. Hence, hi-tech handoff systems are established to encounter the various Quality of service (QoS) constrains. Handoff is the technique employed to transfer mobile nodes from one network to further networks without altering the connectivity. The upcoming wireless networks need to assist wide-range applications, users as well as constraints for connectivity. The quick access has to be achieved by means of small sized cells. Despite, if the base station’s intensity is increased, the handoff frequency is also maximized [30,33]. Moreover, vertical handoff approach is classified into three phases. At first, mobile node should know about the easily accessible remote structures, this process is termed as system detection. This method is used to identify the appropriate method which fulfils the user gradients with requisite QoS [9]. In the handoff process, mobile node measures vertical handoff constraints associated with novel radio communication networking to reconcile by handoff choice. After choosing the mobile node, the next phase is termed as handoff implementation. The optimal range of vertical handoff is depending on various parameters, such as cost, Bit Error ratio (BER), Bandwidth, Available Bit Rate (ABR), Signal to Noise ratio (SNR), and throughput [31,34]. These parameters are must observed along with flag feature in the convoluted heterogeneous circumstances [22].

Several algorithms, such as fuzzy logic [14] and artificial neural networks [2] are developed to achieve vertical handoff mechanism. These approaches have the ability for robust data processing and it minimize the ping pong effect effectually and also maximize the precision of decision making. In [13], the enhanced vertical handoff approach is designed by considering accuracy of Received Signal Strength (RSS) as well as Kalman filter along with fuzzy logic algorithm. Kalman filter is employed to filter Gaussian noise in RSS to attain accurate constraints and can enhance efficiency of decision making. On the other hand, the fuzzy logic approach is applied to manage huge amount of information. However, the fuzzy inference rules are exponentially increased, which creates a great impact on the system to make more complex. In [11], researchers developed the neural network approach for network selection, which enhances the system adaptive capability in network inconsistent state, but it is failed to contemplate user’s gratification as well as QoS. The artificial neural network [35] is utilized to ensure the Quality of Experience (QoE) in vertical approach for enhancing client satisfaction. The novel approach is designed on the basis of Q-learning for handoff mechanism. In this methods, QoE assessment mechanism is developed on the basis of Recurrent Neural Network (RNN) to enhance the QoE in some extend. Despite, the terminal devices has become unfit for solving complex computations due to the restricted computing proficiencies. Moreover, the decision tree based vertical handover approach [16] is used to minimize the possibility of fault handoff and more precise decisions are attained by means of Kalman filtering, thus enhance the firmness of the technique [28].

Problem statement : When a mobile node switches from one access point to another, it manages the handoff procedure to keep its connection active. Concerns surrounding handover management include letting network entities continue to use the MNs’ communications while they move and alter their points of attachment on the network. Additionally, handover management maintains constant access to the data by keeping an eye on and managing the MN’s overall handover procedure. The current methods were challenged by issues such service interruption, call blocking, and poor streaming video quality. Better handoff management techniques in heterogeneous wireless networks are required to eliminate these problems and assist users in selecting the best available network to prevent ping pong.

The main goal of this research is to establish the handoff management scheme in heterogeneous Wireless Networks. For that, the devised scheme follows various steps, such as network selection, handover decision and QoS metric computation. The network is selected by the devised FAO algorithm, which utilizes jitter, handoff latency and RSSI as fitness function. The handover is initiated when the data rate measured by the sensor node is lower than the predefined threshold value. Moreover, the handover decision is performed by DQN, where the decision taken by the DQN is based on distance of Mobile Station (MS) from access point, SNR, MS speed along with the fitness parameters. If the decision taken by the DQN is ‘Yes’, then calculate network bandwidth and finally adjust the bit rate of the video.

The prime contribution of this article is deliberated below:

Proposed DQN-FAO algorithm to select the network for handoff management : The handoff management system in heterogeneous wireless networks is established in this study. This is accomplished through a number of phases in the developed approach, including network selection, handover determination, and QoS metric computation. The network is chosen by the FAO method, which was developed, and it uses jitter, handoff latency, and RSSI as fitness functions.

Proposed DQN-FAO algorithm : When the sensor node’s measured data rate is reduced below the threshold value, the handover is initiated. Additionally, DQN makes the decision on the handover. Combining the FA and AO algorithms results in the FAO algorithm.

The structure of this research paper is explained in this section. The review of various handoff management techniques are deliberated in Section 2, system model is discussed in Section 3, invented handoff management scheme is elaborated in Section 4 and the final part of this research is explained in Section 5.

2. Motivation

As seen in mobile phones and laptops, the use of wireless networks, where they operate in accordance with other networks, is rapidly increasing. These are linked together via different network terminal nodes that change depending on the networks that are accessible. Rapid network change results in vertical handoffs that cause services to discontinue, such as call blocking and poor streaming video quality. A more effective handoff management methodology for heterogeneous wireless networks is required to eliminate these problems. This methodology should assist in selecting the best available network to prevent ping pong. This section enumerates the literature reviews of various existing vertical handoff techniques in heterogeneous networks along with advantages, disadvantages, and challenges to develop best technique that inspire the researchers.

2.1. Literature survey

This portion interprets the review of traditional techniques corresponding to handoff management in context aware video streaming-based heterogeneous Wireless Networks. Han, Z., et al. [7] developed deep reinforcement learning approach for enabling the network to explore the behaviors of real users and the status of network, which effectively improved the rate of data during handoff procedure but still the state of wireless local area network (WLAN) constructed with simulator may diverge from real situations. Chen, J., et al. [3] introduced quality of experience (QoE) method for determining the association among quality of experience (QoE) and quality of service (QoS) in heterogeneous networks. This method attained finest performance with handoff dropping probability and also obtained better performance of QoE in maintaining charges and fatal power consumption but it has highest call blocking probability. Wang, S., et al. [28] developed Multi objective model for solving the shortage of complete consideration of user as well as the consequences of network during handoff procedure that efficiently enhanced the service quality of user and the utilization of resources. Although, this technique failed to process a larger quantity of data due to high complexity, slow computation speed and not appropriate for rapid moving scenarios. Dhipa, M., et al. [4] incorporated telecardiology method with Trust and Privacy based Multi-attribute Vertical Handoff decision algorithm for deciding the best probability interconnections reduced the blocking possibilities but failed to reduce the power consumption and computation costs. Zaheeruddin and Mahajan, P., [33] introduced a novel optimized vertical handoff technique for faultless of users to wireless heterogeneous networks. In this technique, the time for computation is very less and provided continual services but still there was a need for reducing the count of call drops and hence call dropping rate can be improved further for proficient functioning of wireless network.

Patil, M.B. and Patil, R., [20] designed Fractional Squirrel–Dolphin Echolocation (FrSqDE) algorithm-based Deep Belief Networks (DBNs) for improving the energy effectiveness of different heterogeneous network minimized the delay and energy consumption although it did not include the adaption of QoS requirements in heterogeneous wireless network. Parambanchary, D. and Rao, V.M., [19] developed Whale Optimization Algorithm (WOA)-neural network (NN) for solving the handover indications in heterogeneous network. Here, the method highly increased the success rate whereas it did not perform in actual time utilizing many sophisticated optimization techniques. Pradeep, M. and Sampath, P., [22] presented an optimized multi-attribute vertical handoff technique for heterogeneous wireless network decreased the count of handoff failures as well as needless handoff cases though it failed to include the adaptation of parameters in Dynamic Network Selection Function (DNSF) for improving the performances. Politis, I., et al. [21] developed a scheme called QoE-driven handoff approach for scalable as well as single layer coding. This approach is very flexible and permits several precise rate adaptations, which maintains the Peak Signal to Noise Ratio (PSNR). It also minimized the count of handovers but further improvement of packet loss rate was not performed. Vallati, C., et al. [26] introduced Handoff procedure based on the link quality forecasting model, which reduced the video outages and hence improved the end-user QoE. However, this method failed to utilize cross-layer optimization QoS for changeable bit rate uses and not examined the prediction ability of the handoff technique. Pyun, J.Y., [23] devised a Context-Aware Streaming Video System for vertical handover (VHO) over wireless overlay network provides finest QoS and throughput during VHO, even if this technique did not solve the utilization of minimal bandwidth and the complexities of less computation at server region problems. The mean square error (MSE) obtained by the cross validation is taken into consideration as the fitness function for the Aquila to select the optimal features. Grace, M. et al. [6] selected the optimal features from the CSV file based on the prediction accuracy by cross validation using the Aquila optimizer. The proposed approach can predict malware from an Android application in real time. A two-dimensional reciprocal cross entropy multi-threshold-based lung parenchyma segmentation approach using an upgraded firefly algorithm was proposed by Guowei Wang et al. [27]. The findings of the experiment demonstrate that it not only segments COVID-19 lung parenchyma more correctly, but also does it with less processing time. Each lung parenchyma segmentation approach, however, has a robustness restriction due to the degree of variation in the anatomical properties of each person’s chest.

2.2. Main challenges

The challenges met by prevailing vertical handoff techniques in heterogeneous wireless network are explained below,

In [7], Deep Reinforcement Learning (DRL) method was established for Artificial Intelligence (AI)-enabled handoff management for the purpose of dense WLANs. However, in this technique DRL along actual scenarios having multiple agents or various mobile stations (STAs) were not included that lead to better performance output.

QoE was designed in [3] with better performance in power consumption in terminals and service charges. However, this method cut-off the overall capability of performance in network.

Multi objective model enabled vertical handoff algorithm was proposed in [28] for heterogeneous wireless networks, but this model lack in maintaining a balanced equilibrium between network resource utilization and terminal QoS.

The algorithm named, Trust and privacy based vertical handoff decision was developed in [4] for application in tele-cardiology heterogeneous wireless networks. Here, handover latency is increased by addition of more attributes and hence it is compulsory to deliver a proper balance between convergence speed and network selection attributes of handoff method to attain improved performance of network.

Vertical handoff is a term referred to switching off different networks. Mobile Terminals are dynamic and get affected by this vertical handoff, which is an important challenge for Mobile Terminal to pick the exact available network that could effectively eradicate the ping pong issue.

Call blocking and poor streaming video quality are only two examples of the services that have been discontinued as a result of the networks’ quick network changes and vertical handoffs. Better handoff management methodologies in heterogeneous wireless networks are required to eliminate these problems and assist users in selecting the best available networks to prevent ping pong.

3. System model

This section interprets the system model for handoff management in context aware video streaming-based heterogeneous wireless networks.

3.1. Assumptions

Let us consider a WLAN, which is deployed in a cellular coverage area with number of small cells. Suppose U is considered as the group of Access Point (APs) in the wireless coverage zone and it is given as, $U = {u_{1}, u_{2}, \dots, u_{m}}$ and the base station (BS) surrounding the wireless coverage zone V is depicted as, $V = {v_{1}, v_{2}, \dots, v_{n}}$ . It is clear that the value of $n = 1$ for all cases excluding highly dense urban installment. If $n > 1$ , the APs are positioned within the coverage zone. The Vertical handoff Decision Controllers (VHDC) manages U and V enclosing the coverage zones as the candidate list attachment points. It is exploited for including the existing APs into group U and data are gathered from load status on individual AP in the group U and individual BS in group V. In the coverage zone, group of overall mobile nodes is indicated as, $M = {m m_{1}, m m_{2}, \dots m m_{k}}$ . Based on the mobile stimulated handoff, every mobile nodes either appealing the handoff or processed by AP that comes under U or comes under V with no requirement of mobility at time. Hence, group of overall MNs is divided into below subsets at some time period and it is expressed as follows, $\begin{matrix} (1) & M_{e} = {m m_{I 1}, m m_{I 2}, \dots, m m_{k (e)}} \end{matrix}$

Here, the term $k (e)$ implies the overall count of MNs appealing the handoff at the period s, and $I_{1}, I_{2}, \dots, I_{k (e)}$ be the respective indexes of such MNs and $N_{e} = M - M_{e}$ . Here, $N_{e}$ is divided into subsets $M_{e}^{(a)}$ and $M_{e}^{(c)}$ depending on whether $m_{k} \in N_{e}$ has a connection in a WLAN or a cellular network, respectively.

3.2. Preliminaries

This part enumerates the preliminaries for comprehending the mechanism of structure in the heterogeneous wireless network. Such considerations are desirable and can be varied accordingly with the network scenario. Below, description of certain assumptions is given:

3.2.1. Heterogeneous devices

Each device involved in wireless network possesses various configurations. As an illustration, the diverse configuration involves computational abilities, mobility pattern of device, battery needs, network interface regulation, and so on.

3.2.2. Communication radius model

The coverage area and its interaction system of the device C with radius r and positioned at d from other component is represented as follows, $\begin{matrix} (2) & C (d, r) = {O_{1}, O_{2} \in MM | G (O_{1} - O_{2}) ⩽ Q_{O_{1}}} \end{matrix}$

Here, the term C implies the coverage distance and $MM$ signifies installed BS and AP and the distance amongst two APs or BSs is expressed as $G (O_{1} - O_{2})$ .

3.2.3. Scalable network (SN)

In an adjustable network, if the wireless area is regarded in the closed area and handover offered in various MNs, then every nodes preferred continual handoffs in the MSN. Let us assume that the 100 nodes are installed in the closed region with the area of $500 m \times 500 m$ . This description is altered as $\forall m n \land m n \in MM$ , $| G (m n - AP / BS) < Q_{m n}$ , such that the MN is depicted as $m n$ and distance amongst the MN and BS/AP is signified as $m n - AP / BS$ .

4. Introduced vertical handoff mechanism utilizing FAO

The fundamental intention of this article is to establish an efficient model for vertical handoff in heterogeneous wireless network utilizing designed FAO algorithm. This proposed scheme accomplishes the vertical handoff management depending upon the efficient network selection model. Initially, the sensor nodes ensure required data level and if the level is below a predefined threshold, it stimulates the handover process. Thereafter, the process of network selection is effectively done utilizing newly designed FAO by considering various factors, such as Jitter, Handoff latency, and RSSI. Moreover, the designed FAO algorithm is derived by the consolidation of AO and FA. The final step is the handover decision step, which is done utilizing DQN by considering the distance between MS and AP, SNR along with the fitness parameters, wherein the hyper parameters of DQN is effectively fine tuned using FAO in order to enhance the performance over the epochs. If the decision is ‘Yes’, then estimate network bandwidth and finally adjust the bit rate of the video. Figure 1 illustrates the schematic view of vertical handoff mechanism utilizing devised FAO algorithm.

Fig. 1.

Block diagram of vertical handoff mechanism using the proposed FAO algorithm.

4.1. Handover triggering phase

In recent years, the smart phones are increasing in diverse applications, such as video chatting, management of position, transportation systems, online games, social networking and so forth. The above said applications are broadly divided into four parts, such as streaming, background, conversational, and the interactive. It is necessary to have low data rate to run an application over MN device without any complications. As an illustration, if a Skype application is processing in a MN device, it needs a minimum data rate of 128 kbps. Similarly, if L count of applications processing over MN device, then it is denoted as $\sum_{A a}^{L} d_{rate}$ , where $A a$ and $d_{rate}$ denotes the application and data rate, respectively. If the application data rate is deteriorated when compared to the predefined threshold then, the application starts to experience a packet loss. In order to cope up with such scenarios, an effective handover stimulation mechanism assists the MN for stimulating the handover at the feasible location.

The steps involved in the vertical handover are handover triggering, choosing of network, and handover execution. The triggering mechanism is employed for ensuring overall count of applications running in an MN device. If the application count is considered to be null, then the mechanism of stimulation gets stopped or else it ensures the needed data rate continuously processing over MN’s device. The threshold value is expressed $τ_{1}$ . Initially, the MN guarantees the need of data rate unevenly by diverse applications processing over MN’s device. If the data level is below to the threshold value, the MN accomplishes the handoff triggering mechanism and if the data level is higher than the threshold value then, two cases are utilized for optimizing the working of data rate checking.

Condition 1.
If the data level goes beyond $τ_{1}$ but lower than the checking threshold $τ_{2}$ , then the verifying procedure is instantly performed by alleviating the initial period at the uneven time T.
Condition 2.
If the data level d is higher than $τ_{1}$ and $τ_{2}$ , then MN maximizes the data level d ensuring by an amount T.

By following these two conditions, the data level guarantee is adjusted strongly after continuous checking and this always explores the data rate after the predefined period that decreases the effectiveness of the checking process. The overall mechanism of the handover triggering step is represented in the flow chart illustrated in Fig. 2.

Fig. 2.
Flow chart of handover triggering.
4.2. Context-aware vertical handover for network selection

Most of the handover process are purely relied on link quality factors, like RSSI, bandwidth and SNR to select the best target radio range. Hence, in wireless networks, the context-aware will be highly sufficient by means of providing the user various services and network ability. Therefore, there is a crucial requirement for context-aware in order to choose the suitable networks, like RSSI, coverage region, available bandwidth, response time, delay, jitter, user preference, cost, and so on [17]. Here, the optimal network is selected for vertical handoff mechanism using proposed FAO.

4.2.1. Network selection

Once the handover stimulation is commenced, the MN begins to seek the obtainable neighborhood networks. If the MN is moving fast in a coverage zone of WiFi, it needs often transferring from one AP to another. This kind of switching results in high energy utilization, huge packet loss and connection breakage. To mitigate this situation, an effective algorithm is proposed for optimized network selection scheme using FAO.

(i) Fitness function

The prime function is to select the optimal network by MN with minimum parameter result. The parameters chosen for network selection are indirectly proportional to QoS. Therefore, the fitness factor is employed to estimate the optimal network from the solution set by considering the factors, such as jitter, handoff latency, and RSSI. It is expressed as follows, $\begin{matrix} (3) & ℑ (min (f)) = \sum_{g = 1}^{K} (J + H_{l} + RSSI) \end{matrix}$

Here, K refers the total number of users, $ℑ (min (f))$ refers the fitness function, J and $H_{l}$ indicates the jitter and handoff latency, respectively. Moreover, $RSSI$ signifies the Received Signal Strength Indicator.

(ii) Aquila position encoding

The solution encoding refers the optimal selection of the network from the neighboring cells for handover mechanism. Figure 3 shows the solution encoding.

Fig. 3.

Solution encoding.

(iii) Devised FAO algorithm for Network selection

This section elaborates on the developed FAO algorithm for network selection, wherein the introduced FAO algorithm is developed by adapting the high soaring nature of the Aquila with a vertical stoop in the AO algorithm [1] by utilizing the behavior of firefly in the FA [32].

AO algorithm.

AO, a population-based optimization technique, draws its inspiration from the natural behaviors of Aquila as they pursue their prey. Thus, the four methods used in the AO algorithm’s optimization processes are: high soar with vertical stoop to select the search space; contour flight with short glide attack to explore within a diverge search space; low flight with slow descent attack to exploit within a converge search space; and walk and grab prey to swoop. The AO technique can handle real-world applications and has a fast rate of convergence.

FA algorithm.

In the tropical and temperate zones, the summer sky is a breathtaking sight when lit up by fireflies. There are over 2,000 different kinds of fireflies, and the majority of them generate brief, cyclical flashes. For a particular species, the flash pattern is frequently distinct. The bioluminescence process that causes the flashing light is still being debated as to the exact purposes of these communication systems. However, these flashes have two primary purposes: to attract possible prey and to communicate with potential mates. Furthermore, flashing might act as a safeguarding warning system. As part of the signal system that unites both sexes, the rhythmic flash, the rate of flashing, and the duration together play a role. In the same species, females react to a male’s distinctive flashing pattern, but in other species, like the photuris, female fireflies can imitate another species’ mating flashing pattern in order to entice and devour male fireflies who might misinterpret the flashes as a prospective appropriate partner. It is feasible to formulate the flashing light in a way that links it to the objective function that needs to be optimised, which opens up the possibility of creating novel optimization methods.

By merging AO and FA together, the developed FAO effectively determines the optimum solution in a short time and with high accuracy. The algorithmic procedures of the developed FAO algorithm are detailed below.

Step 1: Initialize the population

The population of Aquila is initialized first, wherein the location of the Aquila corresponds to the candidate solution and is represented by, $\begin{matrix} (4) & Z = [Z_{p q}], 1 ⩽ p ⩽ A, 1 ⩽ q ⩽ dim \end{matrix}$

Here, A indicates the overall count of the candidate solutions and dim specifies the size of the problem, wherein each candidate is given by, $\begin{matrix} (5) & Z_{p q} = rd \times ({up}_{q} - {low}_{q}) + {low}_{q} p = 1, 2, \dots, A, q = 1, 2, \dots, dim \end{matrix}$

Wherein, $rd$ is any arbitrary number and ${up}_{q}$ and ${low}_{q}$ signifies the lower and upper limits of the given issue.

Step 2: Fitness calculation

Fitness of the solution is already explained in part (i) of the current section, and the best solution corresponds to the solution with minimal fitness.

Step 3: Expanded exploration

In this stage, Aquila discovers the location of food source and flies to a higher level above the ground and performs exploration of the prey. This approach is employed for catching flying birds, where the Aquila soars up with a vertical stoop and this is modeled using below expression. $\begin{matrix} (6) & Z_{1} (t + 1) = Z_{best} (t) \times (1 - \frac{t}{t_{max}}) + (Z_{B} (t) - Z_{best} (t) * rd) \end{matrix}$

Here, t expresses the current iteration and $Z_{1} (t)$ is the solution produced by the first search technique, $Z_{best} (t)$ and $Z_{B} (t)$ represents the optimal solution and the mean value of the present solution, $t_{max}$ denote the maximal iteration count. The term $(1 - \frac{t}{t_{max}})$ controls the exploration of the search. Here, $Z_{B} (t)$ is given by, $\begin{matrix} (7) & Z_{B} (t) = \frac{1}{A} \sum_{p = 1}^{A} Z_{p} (t), \forall q = 1, 2, . ., dim \end{matrix}$

Assume, $Z_{1} (t + 1) = Z (t + 1)$ , $A = 1$ , and $\frac{1}{A} \sum_{p = 1}^{A} Z_{p} (t) = Z (t)$ them the Eq. (6) can be rewritten as, $\begin{array}{c} (8) & Z (t + 1) = Z_{best} (t) \times (1 - \frac{t}{t_{max}}) + (Z (t) - Z_{best} (t) * rd) \\ (9) & Z (t + 1) = Z_{best} (t) \times ((1 - \frac{t}{t_{max}}) - rd) + Z (t) \end{array}$

Now, consider the movement of firefly towards the brighter firefly in the FA algorithm, this can be expressed as, $\begin{matrix} (10) & Z_{r} (t + 1) = Z_{r} (t) + α_{0} e^{- β d_{p q}^{2}} (Z_{s} (t) - Z_{r} (t)) + χ δ_{r} \end{matrix}$

Here, $Z_{r} (t)$ and $Z_{s} (t)$ indicates the position of the $r^{th}$ and $s^{th}$ firefly, $α_{0}$ indicates attractiveness at when the distance $d_{r s}$ between $r^{th}$ and $s^{th}$ firefly is zero, β is the fixed light absorption coefficient, χ represents the randomization parameter, and $δ_{r}$ denotes the random vector obtained from Gaussian distribution.

Assume, $Z_{r} (t + 1) = Z (t + 1)$ , $Z_{r} (t) = Z (t)$ , $δ_{r} = δ$ and $d_{p q} = d$ , then Eq. (10) can be written as, $\begin{array}{c} (11) & Z (t + 1) = Z (t) + α_{0} e^{- β d^{2}} (Z_{s} (t) - Z (t)) + χ δ \\ (12) & Z (t + 1) = Z (t) + α_{0} e^{- β d^{2}} Z_{s} (t) - α_{0} e^{- β d^{2}} Z (t) + χ δ \\ (13) & Z (t) = \frac{Z (t + 1) - α_{0} e^{- β d^{2}} Z_{s} (t) - χ δ}{(1 - α_{0} e^{- β d^{2}})} \end{array}$

Substituting Eq. (13) in Eq. (9), we get $\begin{array}{c} (14) & Z (t + 1) = Z_{best} (t) \times ((1 - \frac{t}{t_{max}}) - rd) + \frac{Z (t + 1) - α_{0} e^{- β d^{2}} Z_{s} (t) - χ δ}{(1 - α_{0} e^{- β d^{2}})} \\ (15) & Z (t + 1) - \frac{Z (t + 1)}{(1 - α_{0} e^{- β d^{2}})} = Z_{best} (t) \times ((1 - \frac{t}{t_{max}}) - rd) - \frac{(α_{0} e^{- β d^{2}} Z_{s} (t) + χ δ)}{(1 - α_{0} e^{- β d^{2}})} \\ \frac{(1 - α_{0} e^{- β d^{2}}) Z (t + 1) - Z (t + 1)}{(1 - α_{0} e^{- β d^{2}})} \\ (16) & = \frac{Z_{best} (t) \times ((1 - \frac{t}{t_{max}}) - rd) (1 - α_{0} e^{- β d^{2}}) - (α_{0} e^{- β d^{2}} Z_{s} (t) + χ δ)}{(1 - α_{0} e^{- β d^{2}})} \\ [(1 - α_{0} e^{- β d^{2}}) - 1] Z (t + 1) \\ (17) & = Z_{best} (t) \times ((1 - \frac{t}{t_{max}}) - rd) (1 - α_{0} e^{- β d^{2}}) - (α_{0} e^{- β d^{2}} Z_{s} (t) + χ δ) \\ (18) & Z (t + 1) = \frac{(α_{0} e^{- β d^{2}} Z_{s} (t) + χ δ) - Z_{best} (t) \times ((1 - \frac{t}{t_{max}}) - rd) (1 - α_{0} e^{- β d^{2}})}{(α_{0} e^{- β d^{2}})} \end{array}$

The above equation is utilized for determining the location of the Aquila in the next iteration after it soars up high and performs a vertical stoop.

Step 4: Narrowed exploration

In the second technique, Aquila flies at a lower level above the ground after finding the prey and prey is then encircled. The Aquila prepares itself for landing and attacking. This method is referred as contour flight with short glide attack and is employed for catching flying or running preys, such as seabirds, breeding grouse, or ground squirrels. The Aquila performs exploration of a specific area and this behaviour is indicated by, $\begin{matrix} (19) & Z_{1} (t + 1) = Z_{best} (t) \times Levy (C) + Z_{D} (t) + (k - l) * rd \end{matrix}$ where, $Z_{D} (t)$ specifies the arbitrary solution in the range $[1, A]$ and $Z_{2} (t + 1)$ denotes solution of second search technique. C is the dimension space, $Levy (C)$ specifies the function used to determine levy flight distribution and is given by, $\begin{matrix} (20) & Levy (C) = a \times \frac{b \times η}{| κ |^{\frac{1}{λ}}} \end{matrix}$

Here, b and κ are arbitrary values with values in the limit of $[0, 1]$ , a and λ are constant with value 0.01 and 1.5, respectively, The value of η is computed by, $\begin{matrix} (21) & η = (\frac{Γ (1 + λ) \times sin e (\frac{π λ}{2})}{Γ (\frac{1 + λ}{2}) \times λ \times 2^{(\frac{λ - 1}{2})}}) \end{matrix}$

Further, k and l are utilized to represent the spiral shape of the search and is given by, $\begin{array}{c} (22) & k = f \times cos (ϕ) \\ (23) & l = f \times sin (ϕ) \end{array}$

Here, $\begin{array}{c} (24) & f = f_{1} + E \times F_{1} \\ (25) & ϕ = - ω \times F_{1} + ϕ_{1} \\ (26) & ϕ_{1} = \frac{3 \times π}{2} \end{array}$ wherein, ω and E are small values fixed at 0.005 and 0.00565, respectively, $F_{1}$ is an integer between 1 and dim, and $f_{1}$ has a value in the range $[1, 20]$ .

Step 5: Expanded exploration

In the third technique, the Aquila is prepared for its land and attacking; wherein it performs a vertical descend for attacking the prey. This technique is called low flight with very slow descent attack, where Aquila attacks the prey by selecting it and landing on the victim’s back or neck. This approach is used for catching slow preys, like tortoises, foxes, hedgehogs, and rattlesnakes. The Aquila performs exploitation of the area around the victim for catching it. This is expressed as, $\begin{matrix} (27) & Z_{3} (t + 1) = (z_{best} (t) - Z_{B} (t)) \times ρ - rd + ((up - low) \times rd + low) \times ξ \end{matrix}$

Here, $up$ and $low$ specifies the upper and lower limit of the issue, ρ and ξ are exploitation adjustment metrics with a value of 0.1, and $Z_{3} (t + 1)$ is the solution of the third search model in the next iteration.

Step 6: Narrowed exploitation

In the fourth approach, Aquila walks on the land and pulls its victim, as soon as it gets closer to the victim. This approach is known as “walk and grab prey” and is employed for catching young ones of larger victims, like sheep or deer from the coverage area. This is expressed using below expression. $\begin{matrix} (28) & Z_{4} (t + 1) = R \times Z_{best} (t) - (P_{1} \times Z (t) \times rd) - P_{2} \times Levy (C) + rd \times P_{1} \end{matrix}$

Here, R represents the quality factor employed to equilibrium the search techniques, $P_{1}$ designates the multiple movements in the AO for catching the prey and $P_{2}$ signifies the slope of the flight and $Z_{4} (t + 1)$ indicates the solution obtained by the fourth search technique in the iteration $t + 1$ . The terms R, $P_{1}$ , and $P_{2}$ are obtained as, $\begin{array}{c} (29) & R (t) = t^{\frac{2 \times rd - 1}{{(1 - t_{max})}^{2}}} \\ (30) & P_{1} = 2 \times rd - 1 \\ (31) & P_{2} = 2 \times (1 - \frac{t}{t_{max}}) \end{array}$

Here, $R (t)$ represents the quality factor in the $t^{th}$ iteration.

Step 7: Check solution feasibility

The best solution is computed by considering the fitness of the candidate solution, wherein the solution with the lowest fitness corresponds to the optimal result.

Step 8: Termination

The aforementioned steps are iterated continuously till the optimal result is attained. The pseudo code of devised FAO is displayed using Algorithm 1.

Algorithm 1

Pseudo code of the proposed FAO algorithm

Thus the proposed FAO algorithm effectively performs network selection with high convergence rate. Further, the amalgamation of the AO and FA algorithms has effectively minimized the computation time and has enhanced the performance of the optimization process.

4.2.2. Handover decision using DQN

Once the optimal network is selected using proposed algorithm, the decision is to be made for handover mechanism using DQN by considering the parameters, such as distance, SNR, MS speed along with the fitness parameters.

(i) Architecture of DQN

DQN [29] is an eminent technique in reinforcement learning, which utilizes the Q-learning method and also employs CNN to approximate the action-value function referred as Q-function. Deep Q-Learning was developed primarily to manage environments with continuous activity and states. Small and discrete environments can benefit from the simple Q-Learning method. DQN also offers the option to anticipate the state-value function, which is an interesting feature. In rare cases, reinforcement learning is said to be unstable or even to different if the non-linear function approximator like neural network (NN) is employed to illustrate the Q-function [24]. The major reasons for this instability are the correlations existing in the series of state scrutinization $S_{T t}$ and $S_{T t + 1}$ , and even little upgrades of Q-value may considerably vary the policy ϕ and hence, it results variation in data distribution. To mitigate aforementioned issues, DQN exploits the technique known as experience replay. Figure 4 portrays the architecture of DQN.

In order to process the experience replay, the agent’s experience at time period $T t$ are preserved in the dataset $W = {X_{1}, \dots . X_{T t}}$ and the agent’s experience is formulated as follows, $\begin{matrix} (32) & X_{T t} = (S_{T t}, x_{T t}, y_{T t}, S_{T t + 1}) \end{matrix}$

The dataset W is also referred as replay memory. While learning, Q-learning upgrades are applied over experience samples $(S_{j}, x_{j}, y_{j}, S_{j + 1})$ derived uniformly from the dataset W. The expression for loss function employed by Q-learning to upgrade at iteration j is defined as follows, $\begin{matrix} (33) & Y_{i} (θ_{i}) = E e [{(y + γ max_{x^{'}} \overset{⌢}{Q} (S^{'}, x^{'}; θ_{i}^{-}) - Q (S, x; θ_{i}))}^{2}] \end{matrix}$

Here, y refers the award, the discount factor is denoted as γ, and $θ_{i}$ and $θ_{i}^{-}$ specifies the network factors of Q-network at iteration i, which is utilized to formulate the target as given below: $\begin{matrix} (34) & z_{j} = y + γ max_{x^{'}} \overset{⌢}{Q} (S^{'}, x^{'}; θ_{i}^{-}) \end{matrix}$

Here, the term $y + γ {max}_{x^{'}} \overset{⌢}{Q} (S^{'}, x^{'}; θ_{i}^{-}) - Q (S, x; θ_{i})$ in the loss parameter represents the TD error and it is based on network factors $θ_{i}$ at iteration i and this results in unstable convergence. In order to enhance the convergence ability, DQN exploits the neural filled Q method in which the parameters are regarded as fitted one.

Fig. 4.

Architecture of DQN.

(ii) Fine tuning hyper parameter of DQN using proposed FAO algorithm for decision making

In order to provide better decisions for handover mechanism, the hyper parameters of DQN are optimally fine tuned using same proposed FAO algorithm, which is already described in Section 4.2.1. If the decision is ‘Yes’, calculate network bandwidth to adjust the bit level of video.

4.2.3. Context aware video streaming

If the decision is made as ‘Yes’ by DQN to accomplish the handover process, it is necessary to determine the bit rate. Initially, the bandwidth of the selected network is calculated and then, the bit rate of the video is adjusted accordingly.

(i) Calculation of network bandwidth

The network bandwidth [21] is continuously monitored by investigating the network with a default stream of duplicate RTP packets. The user is acknowledged for required capacity or the existing bandwidth by receiving the RTCP messages that conveys this data to the message leader.

(ii) Bit rate adjustment of the video

Videos can be encoded into various bit rate versions with different file sizes. If the bit rate is low, the file size of the video is also smaller and it results low resolution with better smoothness level. Hence, it is significant to fine tune the bit rate to provide smoothness. The main objective of the bit rate adjustment is to guarantee the smooth playback in order to tune the bit rate version of mobile users and hence, the overall network traffic can be regulated [8]. Once the bandwidth is calculated, bit rate of the video is adjusted accordingly so that the user can enjoy the application without any interruption in the network. The graphical illustration of bit rate adjustment is given in Fig. 5.

Fig. 5.

Illustration of bit rate adjustment.

5. Results and discussion

The simulation results of developed FAO-based DQN for handoff management in context aware video streaming-based heterogeneous wireless network are explicated in this part.

5.1. Experimental setup

The execution of developed FAO driven DQN is done in personal computer with python 3.7, pycharm, Intel i3 processor in Windows 10 operating system. Table 1 shows experimental setup details.

Table 1
Experimental setup details

Parameters Values

Number of user 20

iteration 200

Selected user 5

x- distance 4000

y-distance 4000

Bit rate 10000

MN Configuration

Frequency 2000

Numerology 1

power 1

gain 5

bandwidth 25

loss 1

Maximum bitrate 55

Client Configuration

User id

spane 4000

Mobility ratio 0.001

Mobility distance Mobility ratio × spane

Parameters	Values
Number of user	20
iteration	200
Selected user	5
x- distance	4000
y-distance	4000
Bit rate	10000
MN Configuration
Frequency	2000
Numerology	1
power	1
gain	5
bandwidth	25
loss	1
Maximum bitrate	55
Client Configuration
User id
spane	4000
Mobility ratio	0.001
Mobility distance	Mobility ratio × spane

5.2. Dataset description

The developed FAO-based DQN is processed using Alankar Kotwal Implementation data [12]. The video frames are extracted from a video and it is stored in data folder. The collected video frame is existing in “RGB” color format.

5.3. Performance indicators

The metrics, like handoff latency, energy consumption, throughput, call drop, handover delay, and PSNR are considered for evaluating the performance of introduced handoff management system.

5.3.1. Call drop

It is defined as the ratio of quantity of call drop times to amount of call setup success times, which is represented as, $\begin{matrix} (35) & C_{d} = \frac{ψ}{μ} \end{matrix}$

5.3.2. Energy consumption

It is computed by total quantity of energy consumed during implementation process.

5.3.3. Handover delay

It is referred as total delay produced by mobile node during handover, while re-establishing an enduring session from switch in source eNodeB to switch in destination eNodeB.

5.3.4. Throughput

This metric estimates total number of data packets transmitted by a channel in certain time interval.

5.3.5. Handoff latency

This metric is mainly depending on latency based on handover initialization, handover decision and execution.

5.3.6. PSNR

PSNR is estimated by proportion of highest possible signal power for corrupting noise power. $\begin{matrix} (36) & ζ_{P} = 10 {log}_{10} (\frac{T_{max}^{2}}{ω}) \end{matrix}$ where, $ζ_{P}$ denotes PSNR, $T_{max}$ implies maximal pixel rate, and ω represents mean square error.

Fig. 6.

Experimental results for developed DQN-FAO (a) input image, (b) 360 pixel, (c), 480 pixel, (d) 720 pixel, and (e) 1080 pixel.

5.4. Experimental results

The experimental outcomes for vertical handoff management in context aware video streaming-based heterogeneous wireless network are explicated in this section. Figure 6 shows the sample result proposed DQN-FAO for hand off management. Figure 6(a), (b), (c), (d) and (e) depicts the input image 3360 pixel, 480 pixel, 720 pixel, and 1080 pixel

5.5. Comparative techniques

The prevailing vertical handoff techniques, such as Deep reinforcement learning [7], Random NN-based QoE estimation [3], Multi objective model [28] and DBN [20] are considered for comparing the performance of designed FAO technique. In addition, Sun Flower Optimization (SFO) algorithm [5], Sail Fish Optimization model [25], Aquila Optimizer (AO) [1], and Firefly Algorithm (FA) [32] are considered for evaluating the performance of developed FAO technique.

5.6. Comparative assessment

The comparative evaluation of the generated DQN FAO for various metrics with time and bit rate is shown in Fig. 7. The analysis of DQN-FAO for call drop, energy consumption, handover delay, throughput, and handoff latency and time duration 15 is shown in Fig. 7(a), (b), (c), (d), and (e). The developed DQN-FAO has a call drop of 0.4518, compared to existing techniques’ call drops of 0.5284, 0.5266, 0.5130, and 0.4943, and performance gains of 14.49%, 14.20%, 11.92%, and 8.60%. The existing and designed procedures achieved energy consumption of 7.918 J, 6.706 J, 7.638 J, 6.705 J, and 6.420 J in addition to obtaining handover delays of 14.60 ms, 14.29 ms, 14.53 ms, 14.46 ms, and 10.72 ms. DQN-FAO has improved performance by 18.92%, 4.26%, 15.94%, and 4.24% in terms of energy usage. Deep reinforcement learning has a throughput of 13.088 megabits per second (Mbps), Random NN based QoE estimation at 13.098 Mbps, multi-objective model at 13.102 Mbps, DBN at 13.103 Mbps, and DQN FAO at 13.108 Mbps. Deep reinforcement learning, Random NN based QoE calculation, the multi-objective approach, and DBN had handoff latency values of 85.325 ms, 81.385 ms, 82.038 ms, and 78.863 ms, respectively. The study of DQN FAO for PSNR is shown in Fig. 7(f). When bit rate 3 is taken into account, the PSNR of the existing and created DQN-FAO is 36.56 dB, 36.94 dB, 37.96 dB, 39.67 dB, and 43.26 dB.

Fig. 7.

Comparative analysis of devised DQN-FAO (a) call drop, (b) energy consumption, (c) handover delay, (d) throughput, (e) handoff latency and (f) PSNR.

5.7. Algorithm estimation

Figure 8 shows the algorithm estimation of the designed FAO + DQN for a number of performance measures. The algorithm analysis of the created FAO + DQN for call drop, energy consumption, handover delay, handoff latency, and throughput is shown in Fig. 8(a), (b), (c), (d), and (e). At time 15, the strategies SFO + DQN, sailfish + DQN, AO + DQN, Firefly + DQN, and FAO + DQN achieved call drops of 0.510, 0.4934, 0.5092, 0.4744, and 0.4563, while energy consumption is 7.769 J, 8.029 J, 7.193 J, 7.027 J, and 6.677 J, respectively. When the time is 15, the handover latency of the designed FAO + DQN is 12.435 Mbps, compared to the 14.216 Mbps, 13.927 Mbps, 13.670 Mbps, and 12.657 Mbps of the existing techniques. The developed and implemented methods achieved handoff latencies of 93.255 Mbps, 85.818 Mbps, 76.353 Mbps, 76.284 Mbps, and 68.55 Mbps in 15 minutes. The throughput for SFO + DQN, Sailfish + DQN, AO + DQN, Firefly + DQN, and FAO + DQN is 12.696 Mbps, 12.857 Mbps, 12.889 Mbps, and 12.893 Mbps, respectively.

Fig. 8.

Comparative analysis of FAO algorithm (a) call drop, (b) energy consumption, (c) handover delay, (d) throughput, (e) handoff latency and (f) PSNR.

5.8. Comparative discussion

Table 2 discusses how the created DQN FAO compares to other traditional approaches for various parameters. The call drop of the planned DQN-FAO is 0.5122, compared to the 0.5869, 0.5708, 0.5494, and 0.5342 call drops of the existing techniques at time 20. In addition, handover delays of 14.55 milliseconds, 14.24 milliseconds, 11.43 milliseconds, 14.31 milliseconds, and 10.54 milliseconds were obtained using the existing and proposed procedures when time was 20. Deep reinforcement learning has a throughput of 13.16 Mbps, Random NN based QoE estimation, multi-objective model, DBN, and DQN FAO have throughputs of 13.16 Mbps, 13.16 Mbps, and 13.17 Mbps respectively for time 20. The designed DQN FAO is achieved for handoff latency at 93.80 ms, while deep reinforcement learning, Random NN based QoE estimation, multi-objective method, and DBN are at 103.61 ms, 102.78 ms, 100.71 ms, and 97.93 ms of time 20, respectively. The PSNR of the existing and developed DQN-FAO at bit rate 4 is 38.47 dB, 39.45 dB, 40.37 dB, 43.45 dB, and 46.89 dB.

Table 2
Comparative discussion

Methods / Metrics Deep reinforcement learning Random NN-based QoE estimation Multi-objective method DBN Proposed DQN-FAO

Call drop 0.5869 0.5708 0.5494 0.5342 0.5122

Energy consumption (J) 8.585 7.2673 7.8214 7.3641 7.0864

Handover delay (ms) 14.55 14.24 11.43 14.31 10.54

Throughput (Mbps) 13.16 13.16 13.16 13.16 13.17

Handoff latency (ms) 103.61 102.78 100.71 97.93 93.80

PSNR (dB) 38.47 39.45 40.37 43.45 46.89

Methods / Metrics	Deep reinforcement learning	Random NN-based QoE estimation	Multi-objective method	DBN	Proposed DQN-FAO
Call drop	0.5869	0.5708	0.5494	0.5342	0.5122
Energy consumption (J)	8.585	7.2673	7.8214	7.3641	7.0864
Handover delay (ms)	14.55	14.24	11.43	14.31	10.54
Throughput (Mbps)	13.16	13.16	13.16	13.16	13.17
Handoff latency (ms)	103.61	102.78	100.71	97.93	93.80
PSNR (dB)	38.47	39.45	40.37	43.45	46.89

6. Conclusion

This paper presents the devised DQN-FAO algorithm for performing the handoff management. The flow of steps involved in the developed system is network selection, handover decision, QoS parameter computation and bit rate adjustment. Here, the network selection is done using FAO algorithm, which is modeled by adapting the advantages of both FA and AO. FA is a metaheuristic optimization algorithm, which is modeled based on its flashing aspects. Generally, the fireflies utilize its flashing strategy to three functions, such as mating, communication and food exploration. Likewise, AO is also an optimization approach, which considers the food exploration aspects of Aquila. In the developed model, the attractiveness of firefly algorithm is adapted to update the location in AO such that the optimal network selection is obtained. The decision is made by DQN for handover mechanism, wherein the hyperparameters are optimally adjusted using proposed FAO. Based on the handover decision made by DQN, context aware video streaming is performed by adjusting the bit rate of the videos depending upon the network bandwidth. Moreover, the analysis is done by comparing the devised approach with some of the traditional methods in order to reveal the efficacy of invented scheme. From the analysis, the devised scheme provided the superior performance than the conventional methods based on the call drop, energy consumption, handover delay, throughput, handoff latency, and PSNR of 0.5122, 7.086 J, 10.54 ms, 13.17 Mbps, 93.80 ms and 46.89 dB. In future, the performance of handoff management is enhanced by including excess parameters to the fitness function.

References

Abualigah,

Yousri,

Abd Elaziz,

A.A.

Ewees,

M.A.

Al-qaness and

A.H.

Gandomi, Aquila optimizer: A novel meta-heuristic optimization algorithm, Computers & Industrial Engineering 157 (2021), 107250. doi:10.1016/j.cie.2021.107250.

Çalhan and

Çeken, Artificial neural network based vertical handoff algorithm for reducing handoff latency, Wireless personal communications 71(4) (2013), 2399–2415. doi:10.1007/s11277-012-0944-4.

Chen,

Wang,

Li and

Wang, QoE-aware intelligent vertical handoff scheme over heterogeneous wireless access networks, IEEE Access 6 (2018), 38285–38293. doi:10.1109/ACCESS.2018.2853730.

Dhipa,

Kalaavathi and

Chandrasekar, Trust and privacy based vertical handoff decision algorithm for telecardiology application in heterogeneous wireless networks, Journal of Ambient Intelligence and Humanized Computing (2020), 1–13.

G.F.

Gomes,

S.S.

da Cunha and

A.C.

Ancelotti, A sunflower optimization (SFO) algorithm applied to damage identification on laminated composite plates, Engineering with Computers 35(2) (2019), 619–626. doi:10.1007/s00366-018-0620-8.

Grace and

Sughasiny, Malware detection for Android application using Aquila optimizer and hybrid LSTM-SVM classifier, EAI Endorsed Transactions on Scalable Information Systems 10(1) (2022), e7.

Han,

Lei,

Lu,

Wen,

Zheng and

Guo, Artificial intelligence-based handoff management for dense WLANs: A deep reinforcement learning approach, IEEE Access 7 (2019), 31688–31701. doi:10.1109/ACCESS.2019.2900445.

Hu,

Wen and

Niyato, Spectrum allocation and bitrate adjustment for mobile social video sharing: Potential game with online QoS learning approach, IEEE journal on selected areas in communications 35(4) (2017), 935–948. doi:10.1109/JSAC.2017.2676598.

Jain and

Tokekar, Application based vertical handoff decision in heterogeneous network, Procedia Computer Science 57 (2015), 782–788. doi:10.1016/j.procs.2015.07.475.

10.

Kato,

Z.M.

Fadlullah,

Mao,

Tang,

Akashi,

Inoue and

Mizutani, The deep learning vision for heterogeneous network traffic control: Proposal, challenges, and future perspective, IEEE wireless communications 24(3) (2016), 146–153. doi:10.1109/MWC.2016.1600317WC.

11.

Kosmopoulos,

Skondras,

Michalas and

D.D.

Vergados, An efficient mobility management scheme for 5G network architectures, in: Proceedings of 2020 5th South-East Europe Design Automation, Computer Engineering, Computer Networks and Social Media Conference (SEEDA-CECNSM), 2020, pp. 1–6.

12.

Kotwal , taken from, http://bitbucket.org/alankarkotwal/coded-sourcesep/, accessed on May 2022.

13.

Kustiawan and

K.H.

Chi, Handoff decision using a Kalman filter and fuzzy logic in heterogeneous wireless networks, IEEE Communications Letters 19(12) (2015), 2258–2261. doi:10.1109/LCOMM.2015.2495212.

14.

Kustiawan,

C.Y.

Liu and

D.F.

Hsu, Vertical handoff decision using fuzzification and combinatorial fusion, IEEE Communications Letters 21(9) (2017), 2089–2092. doi:10.1109/LCOMM.2017.2709750.

15.

Lee,

Choi and

Kim, Intelligent handover scheme for drone using fuzzy inference systems, IEEE Access 5 (2017), 13712–13719. doi:10.1109/ACCESS.2017.2724067.

16.

Ma,

Wang,

Cheng and

Xie, Modeling and analysis for vertical handoff based on the decision tree in a heterogeneous vehicle network, IEEE Access 5 (2017), 8812–8824. doi:10.1109/ACCESS.2017.2707801.

17.

Maaloul,

Afif and

Tabbane, A new vertical handover decision based context awareness for ubiquitous access, in: International Conference on Communications and Information Technology (ICCIT), 2012, pp. 231–236.

18.

Martínez-Vargas and

Á.G.

Andrade, Deployment analysis and optimization of heterogeneous networks under the spectrum underlay strategy, EURASIP Journal on Wireless Communications and Networking 1 (2015), 1–15.

19.

Parambanchary and

V.M.

Rao, WOA-NN: A decision algorithm for vertical handover in heterogeneous networks, Wireless Networks 26(1) (2020), 165–180. doi:10.1007/s11276-018-1787-z.

20.

M.B.

Patil and

Patil, Fractional squirrel–dolphin echolocation with deep belief network for network-controlled vertical handoff in disparate and heterogeneous wireless network, International Journal of Communication Systems 34(12) (2021), e4893. doi:10.1002/dac.4893.

21.

Politis,

Dounis and

Dagiuklas, H. 264/SVC vs. H. 264/AVC video quality comparison under QoE-driven seamless handoff, Signal Processing: Image Communication 27(8) (2012), 814–826.

22.

Pradeep and

Sampath, An optimized multi-attribute vertical handoff approach for heterogeneous wireless networks, Concurrency and Computation: Practice and Experience 31(20) (2019), e5296.

23.

J.Y.

Pyun, Context-aware streaming video system for vertical handover over wireless overlay network, IEEE Transactions on Consumer Electronics 54(1) (2008), 71–79. doi:10.1109/TCE.2008.4470026.

24.

Sasaki,

Horiuchi and

Kato, A study on vision-based mobile robot learning by deep Q-network, in: 56th Annual Conference of the Society of Instrument and Control Engineers of Japan (SICE), 2017, pp. 799–804. doi:10.23919/SICE.2017.8105597.

25.

Shadravan,

H.R.

Naji and

V.K.

Bardsiri, The Sailfish optimizer: A novel nature-inspired metaheuristic algorithm for solving constrained engineering optimization problems, Engineering Applications of Artificial Intelligence 80 (2019), 20–34. doi:10.1016/j.engappai.2019.01.001.

26.

Vallati,

Mingozzi and

Benedetto, Efficient handoff based on link quality prediction for video streaming in urban transport systems, Wireless Communications and Mobile Computing 16(15) (2016), 2298–2314. doi:10.1002/wcm.2684.

27.

Wang,

Guo,

Han and

A.B.

Cekderi, Two-dimensional reciprocal cross entropy multi-threshold combined with improved firefly algorithm for lung parenchyma segmentation of Covid-19 CT image, Biomedical Signal Processing and Control 78 (2022), 103933. doi:10.1016/j.bspc.2022.103933.

28.

Wang,

Deng,

Xiong,

Liu,

Liu and

Liu, A multi-objective model-based vertical handoff algorithm for heterogeneous wireless networks, EURASIP Journal on Wireless Communications and Networking 1 (2021), 1–18.

29.

Wang,

Liu,

Zheng,

Xia,

Li,

Chen,

Guo and

Xie, Multi-objective workflow scheduling with deep-Q-network-based multi-agent reinforcement learning, IEEE access 7 (2019), 39974–39982. doi:10.1109/ACCESS.2019.2902846.

30.

Wu,

P.J.

Havinga and

Mizuno, Wireless Internet over heterogeneous wireless networks, in: GLOBECOM’01. IEEE Global Telecommunications Conference (Cat. No. 01CH37270), Vol. 3, 2001, pp. 1759–1765. doi:10.1109/GLOCOM.2001.965879.

31.

Yan,

Y.A.

Şekercioğlu and

Narayanan, A survey of vertical handover decision algorithms in fourth generation heterogeneous wireless networks, Computer networks 54(11) (2010), 1848–1863. doi:10.1016/j.comnet.2010.02.006.

32.

X.S.

Yang, Firefly algorithms for multimodal optimization, in: International Symposium on Stochastic Algorithms, 2009, pp. 169–178.

33.

Zaheeruddin and

Mahajan, Optimized handoff algorithm for heterogeneous networks, IETE Technical Review (2020), 1–9.

34.

Zenaldan,

Hassan and

Habbal, Vertical handover in wireless heterogeneous networks, Journal of Telecommunication, Electronic and Computer Engineering (JTEC) 9(1–2) (2017), 81–85.

35.

A.B.

Zineb,

Ayadi and

Tabbane, QoE-based vertical handover decision management for cognitive networks using ANN, in: Proceedings of 2017 Sixth International Conference on Communications and Networking (ComNet), 2017, pp. 1–7.