A hybrid differential evolution and simulated annealing algorithm for global optimization

Abstract

Differential evolution (DE) is one of the most effective ways to solve global optimization problems. However, considering the traditional DE has lower search efficiency and easily traps into local optimum, a novel DE variant named hybrid DE and simulated annealing (SA) algorithm for global optimization (HDESA) is proposed in this paper. This algorithm introduces the concept of “ranking” into the mutation operation of DE and adds the idea of SA to the selection operation. The former is to improve the exploitation ability and increase the search efficiency, and the latter is to enhance the exploration ability and prevent the algorithm from trapping into the local optimal state. Therefore, a better balance can be achieved. The experimental results and analysis have shown its better or at least equivalent performance on the exploitation and exploration capability for a set of 24 benchmark functions. It is simple but efficient.

Keywords

Differential evolution simulated annealing ranking mutation operator selection operator

1 Introduction

Global optimization problems are very common in the process of theoretical research and practical application. Most of the traditional analytical optimization methods need to calculate the derivative value of the objective function and some other auxiliary information to determine the search direction. What’s more, there are many restrictions on the convexity and linearity of the function. However, in science or engineering practice, the objective function is not always differentiable, or the problem simply cannot be expressed by a function. Therefore, many of the optimization problems cannot be addressed well by traditional methods. Since the 1980 s, intelligent optimization algorithms have developed rapidly. Especially, evolutionary algorithms (EAs) have become increasingly popular due to it is derivative-free mechanism and excellent optimization ability to avoid local optimum. Differential evolution (DE) [1, 2] is one of the most effective EAs to solve continuous global optimization problems, whether the function is convex or non-convex, which was proposed by Storn and Price in 1995. The general idea of DE is to generate fresh individuals from the randomly generated initial individuals after each mutation and crossover operation, and then compare their fitness values with the previous ones, and the better ones are selected to save as offspring. Continue to iterate according to this process, and ultimately the optimal solution is obtained. The simple coding, high optimization efficiency and excellent robustness of the DE algorithm make it one of the most frequently used algorithms for solving various test and practical problems in diverse domains, such as chemical engineering [3], image processing [4], power engineering [5, 6], industrial engineering [7], electrical engineering [8] and so on.

However, when solving some complex optimization problems with high dimensions, the traditional DE is likely to appear premature and trap into the local optima. Besides, the convergence rate of DE is not high enough, so that it will affect the versatility of DE [9, 10]. In response to the two shortcomings mentioned above, various work about DE variants have been done to improve the performance of the traditional DE, which are mainly reflected in the following aspects: the setting of control parameters (i.e., population size NP, the crossover rate C_r and the scaling factor F), the improvement of evolution strategies (i.e., mutation, crossover and selection operators) and the application of hybrid algorithms.

Plenty of work can be found in the literature on finding out the proper control parameters settings to improve the versatility of DE. In order to obtain a good solution, Storn and Price suggested that NP should be 5D - 10D, F should be 0.5 and C_r should be 0.1 or 0.9, where D represents the dimension of the problem [2]. Gamlerle et al. [11] believed that NP should be selected from 3D to 8D, F should be set to 0.6 and C_r should lie in the interval [0.3, 0.9]. Ronkkonen et al. [12] recommended that NP should be selected from 2D to 4D, F should lie in the interval [0.4, 0.95] with the initial value of 0.9, and C_r should lie between [0, 0.2] for a separable function while for multimodal or non-separable function it should be in the range of [0.9, 1]. Zielinski and Laur [13] suggested that the setting F and C_r ⩾ 0.6 are good choices to obtain good results in many cases. In summary, it is not difficult to find that researchers mentioned above have reached a consistent conclusion that F should lie between [0.4, 1], C_r should be either close to 1 or 0 [14]. As for the setting of NP, there is no consensus yet.

Many attempts have also been made on the modifications of evolution strategies to improve the convergence rate and population diversity. According to the order of DE steps, the first is mutation operation, which is also the core step of DE. Zhong et al. [15] proposed a novel DE variant with a fresh mutation operator which took the influence of reference individuals into account. Yu et al. [16] raised a novel adaptive mutation operator for DE, which was adaptive only when the individuals clustered around the local optimum. Tan et al. [17] put forward a new DE variant equipped with an adaptive mutation operator based on the fitness landscape. Mezura-Montes et al. [18] introduced a new mutation operator adopting the information of the individual who performed best in the current population to generate a new trial vector. Biswas et al. [19] introduced a fresh mutation operator called parent-centric mutation based on neighborhoods to decrease niching behavior in DE. Sun and Cai [20] used a dynamic neighborhood learning (DNL) aiming at enhancing the efficiency of mutation operator. Mohamed [21] proposed a triangular mutation operator, where three vectors were randomly selected, and the difference between best and worst vector was added to its mutation rule. Kaelo and Ali [22] presented a mutation operator that applied the attraction-repulsion idea of electromagnetism to enhance DE performance. Das et al. [23] adopted the concept of neighborhood to improve the “current-to-best/1” strategy by using “target-to-best/1” strategy. Zhang et al. [24] introduced the “current-to-pbest/1” strategy, where the top 100 p % individuals have the possibility to enter into the next generation. Lu et al. [25] proposed an improved elite archive mutation operator according to the quantitative research on chromosome variability as well as population diversity.

Crossover operation is the next step, where a new trial vector is created. Zaharie [26] was committed to the study of comparing binomial and exponential crossover operator. Zhao and Suganthan [27] proposed that the exponential crossover operator was better for optimization problems with high dimensions. Wang et al. [28] introduced a novel crossover operator named orthogonal crossover operator. Fan et al. [29] proposed an enhanced DE variant with adaptive crossover strategies. Deng et al. [30] introduced a new crossover operator with Jrand number decreasing mechanism and feedback guide technique.

The selection operation is the last step of the evolution process. Since the selection operation is a special step that makes it different from other algorithms, there are not many variants based on this step. Yi et al. [31] introduced a p-best selection operator to improve population diversity. In this operator, the p-best of the offspring with a better fitness value is suitable for exploitation. Liang et al. [32] proposed a distance-based elite selection mechanism.

In addition to the above modifications, hybridization is increasingly popular. Arab and Alfi [33] hybridized memetic algorithm (MA) with local search (LS) called MA-ALS to improve the convergence rate and speed. Mousavi and Alfi [34] proposed a hybrid algorithm based on fractional particle swarm optimization (FPSO) and memetic algorithm (MA) called FPSOMA to solve optimization problems. Moreover, many hybridizations based on DE and other promising global optimization algorithms have been proposed to improve the performance of traditional DE. Zhang et al. [35] proposed a new hybridization of DE and cuckoo search (CS) called CSDE. In CSDE, the population are divided into two groups and adopted DE and CS respectively to exchange information and make full use of the advantages of each other. Shimpi Singh Jadon et al. [36] proposed a HABCDE algorithm which is a hybridization of the artificial bee colony algorithm (ABC) and DE. The onlooker bee operator of ABC adopted the DE/best/1/bin evolution strategy of DE in HABCDE. Liu et al. [37] proposed a hybrid whale optimization algorithm (WOA) enhanced by Lévy flight and DE (WOA-LFDE), which employed the “DE/current-to-pbest/1” strategy to help WOA to escape from the local optima and improve the robustness of WOA. Babak Mohammadi et al. [38] proposed a MLP-PSODE algorithm. In MLP-PSODE, multi-layer perceptron (MLP) is hybridized with PSO and then, adding DE.

In the search process, simulated annealing (SA) accepts inferior solutions with a certain probability that decreases as the temperature reduces and eventually approaches to zero [39]. Through affecting the probability, the temperature can control the algorithm’s progress. In the beginning, the random search accounts for more, and as the temperature is reduced, the greedy search starts to prevail. This feature of SA can effectively avoid the procedure trapping into the local optima and ultimately achieve the global optima. Furthermore, the strong local exploitation capability of SA algorithm is conducive to improving the convergence rate of DE. These advantages urge many scholars to choose SA as an appropriate algorithm to hybridize with DE to enhance its performance. Hu et al. [40] proposed a SaDESA algorithm, which is based on self-adaptive DE and SA. Olenšek et al. [41] introduced a parallel algorithm based on DE and SA, which used a greedy strategy to accept individuals in local search and Metropolis criterion in SA to accept individuals in the rest. Zhao et al. [42] proposed a iSADE which adopted the idea of SA to jump out of the possible local optimum attraction. Zeng et al. [43] introduced a hybrid algorithm using the Metropolis criterion in SA to further optimize the solution after the selecting operation of DE, aiming at improving the global optimization ability. Assadi et al. [44] proposed a hybrid DE with population-based SA. He et al. [45] employed SA algorithm to the mutation operator of DE by using an adaptive selection mechanism where the vectors were chosen adaptively according to their historical performance. Mirsadeghi et al. [46] proposed a novel hybrid algorithm, where the concept of temperature in SA was utilized to strike a balance between the exploration and exploitation capability.

Although a lot of work on improving the performance of DE has been done, the defects such as complex coding, excessive control parameters, and inability to balance the exploration and exploitation capability prompted us to continue to study the improvement of DE performance. It is also observed that most of the work mentioned above focuses on the improvement of mutation operations, whereas few variants on selection operations. Moreover, as mentioned previously, hybrid DE with SA can make full use of the advantages of both algorithms to improve the global and local searching ability at the same time. Based on these considerations, this paper proposes a hybrid DE and SA algorithm for global optimization (HDESA). Concerning the previous research, the control parameter F and C_r are respectively set to 0.5 and 0.9 in this algorithm, and mutation operator and selection operator are modified simultaneously. The coding of the algorithm is very simple, and it is relatively efficient. First of all, the concept of ranking is added when the parent individuals are randomly selected in the mutation operation. The probability of being selected for each individual is related to the ranking of their fitness values at the current generation [47]. The higher the ranking of the vector, the more likely it is selected as the parent vector, which improves the exploitation ability of DE. Furthermore, to strike a balance between the exploration and exploitation capability of DE, considering the Metropolis criterion in SA can primarily prevent premature convergence, our algorithm replaces the greedy selection operator in DE with the accept with a certain probability one in SA. In other words, Metropolis criterion is used to determine whether to accept the trial vector after mutation operation. This modification aims to increase the population diversity as well as improve the exploration capability. According to the experimental results, HDESA can primarily improve the performance of DE.

The rest of paper is organized as follows. The basic procedures of DE and SA are briefly introduced in Section 2. Section 3 gives the novel DE variant HDESA. Experimental results and analysis are presented in Section 4. At last, Section 5 draws the conclusion from this paper.

2 Differential evolution (DE) and simulated annealing (SA)

In this section, the basic steps of traditional differential evolution (DE) and simulated annealing (SA) are introduced.

2.1 Differential evolution (DE)

For the optimization problem: $min f (x_{1}, x_{2}, \dots, x_{D})$ (1) $s . t x_{j}^{L} ⩽ x_{j} ⩽ x_{j}^{U}, j = 1, 2, \dots, D$ (2)

Where D denotes the dimension of the solution space, $x_{j}^{L}$ , $x_{j}^{U}$ espectively denote the lower and upper bounds of the j ^th component x_j. The basic operations of DE are listed as follows.

(1) Generate initial population

The initial population ${x_{i} (0) | x_{j, i}^{L} ⩽ x_{j, i}$ $(0) ⩽ x_{j, i}^{U}, i = 1, 2, \dots, NP; j = 1, 2, \dots, D}$ is generated from the feasible region randomly according to a uniform distribution between 0 and 1. $x_{j, i} (0) = x_{j, i}^{L} + rand (0, 1) \times (x_{j, i}^{U} - x_{j, i}^{L})$ (3)

Where x_j,i (0) represents the j ^th gene of the i ^th individual of the 0^th eneration and NP is the population size.

(2) Mutation operation

At each generation g, a mutant vector v_i (g) is created in this operation based on the parent population x_i (g) according to: $v_{i} (g) = x_{r_{1}} (g) + F \cdot (x_{r_{2}} (g) - x_{r_{3}} (g))$ (4)

The above strategy is the most classic mutation strategy named DE/rand/1 [1, 2]. Besides, the mutation vectors can also be generated in some other common ways:

DE/best/1: $v_{i} (g) = x_{best} (g) + F \cdot (x_{r_{1}} (g) - x_{r_{2}} (g))$ (5)

DE/current-to-best/1:

$\begin{matrix} v_{i} (g) = x_{i} (g) + F \cdot (x_{best} (g) - x_{i} (g)) \\ + F \cdot (x_{r_{1}} (g) - x_{r_{2}} (g)) \end{matrix}$ (6)

DE/rand/2:

$\begin{matrix} v_{i} (g) = x_{r_{1}} (g) + F \cdot (x_{r_{2}} (g) - x_{r_{3}} (g)) \\ + F \cdot (x_{r_{4}} (g) - x_{r_{5}} (g)) \end{matrix}$ (7)

DE/best/2:

$\begin{matrix} v_{i} (g) = x_{best} (g) + F \cdot (x_{r_{1}} (g) - x_{r_{2}} (g)) \\ + F \cdot (x_{r_{3}} (g) - x_{r_{4}} (g)) \end{matrix}$ (8)

Where F represents the mutation scaling factor, x_best denotes the best component in the current generation, r₁∼ r₅ ∈ { 1, 2, …, NP } ∖ { i }, and r₁ ≠ r₂ ≠ r₃ ≠ r₄ ≠ r₅. DE is good at global searching but a little weak in local searching in that r₁ ∼ r₅ are randomly selected, especially when the mutation operator does not adopt the best individual (i.e., x_best) [23].

(3) Crossover operation

At each generation g, a trial vector u_j,i (g) is created in this operation based on the parent population x_i (g) and the middle generation v_i (g): $u_{j, i} (g) = {\begin{matrix} v_{j, i} (g) if {rand}_{j} (0, 1) ⩽ C_{r} or j = j_{rand} \\ x_{j, i} (g) otherwise \end{matrix}$ (9)

Where C_r denotes the crossover rate, j_rand denotes an integer randomly selected between 1 and D.

(4) Selection operation

DE adopts a greedy algorithm to select individuals who enter the next generation: $x_{i} (g + 1) = {\begin{matrix} u_{i} (g) iff (u_{i} (g)) ⩽ f (x_{i} (g)) \\ x_{i} (g) otherwise \end{matrix}$ (10)

For each individual, the solution obtained is better or equal to the global optimum achieved by the individual through mutation, crossover, and selection operations [1, 2]. Following Fig. 1 explains the schematic diagram of the traditional DE.

Fig. 1

The schematic diagram of the traditional DE.

2.2 Simulated annealing (SA)

SA is inspired by the solid annealing process, and the cooling schedule is used to control the algorithm progress [48]. When the control parameter T slowly drops and tends to zero, the algorithm will obtain the global optimal solution. The basic steps of SA are listed as follows:

(1)
Set the cooling schedule parameters and randomly generate an initial solution x₀, where the parameters in the cooling schedule are initial temperature T₀, decay function, end temperature T_f, and Markov chain length L.

Perturb the current solution x₁ to generate a new solution x₂, the increment df = f (x₂) - f (x₁).

If df < 0, then p = 1, else $p = \exp (- \frac{df}{T})$ where p denotes the Boltzmann probability function, T denotes the current temperature.

If rand (0, 1) < p, x₂ is accepted as the new current solution x₁ = x₂, else keep the current solution x₁, the above judgment criterion is Metropolis criterion.

If the termination condition is met, the current solution x₁ is output as the optimal solution, else go to step (2) after reducing the temperature.

SA does not depend on the initial value, which means that there is no necessary connection between the solution obtained by the algorithm and the initial solution. Therefore, SA has good robustness. Furthermore, the use of Metropolis criterion improves the exploration ability of SA. To illustrate the method more clearly, a schematic diagram of SA is shown as Fig. 2.

Fig. 2
The schematic diagram of SA.
3 Hybrid DE and SA algorithm (HDESA)

The hybrid differential evolution and simulated annealing algorithm, HDESA, is proposed in this paper. The motivation of the work and the construction process of HDESA are given in this section.

3.1 Motivation of the work

Many DE variants add self-adaptive approaches to enhance the performance of DE. Although the performance has been improved to a certain extent, the defects such as complex coding and too many control parameters appear [6 , 30]. Similarly, some hybrid algorithms also ignore the simplicity of the code due to the pursuit of algorithm performance. Moreover, most hybrid algorithms have disadvantages such as too many control parameters and dependence on specific problems [45, 46]. It can also be observed that most of the DE variants focus on improving mutation operations, whereas few variants focus on the selection operation, so some of them cannot achieve a balance between exploration and exploitation capability. Based on the considerations mentioned above, a new DE variant named hybrid DE and SA algorithm (HDESA) is proposed to solve continuous global optimization problems. Unlike some other hybrid algorithms, HDESA has a relatively simple coding and has almost no additional parameters. The algorithm introduces the concept of ranking into the mutation operation of DE and adds the idea of SA to the selection operation. The former is to enhance the exploitation capability of DE and increase the convergence rate, and the latter is to enhance the exploration ability and prevent it from trapping into the local optimum, then an excellent balance can be achieved.

3.2 Framework of hybrid DE and SA algorithm (HDESA)

HDESA adopts the same initial population generation operation and crossover operation as described in (1) and (3) of the traditional DE steps in Section 2. The details of the mutation and selection operations of our algorithm are described as follows.

3.2.1 Ranking based mutation operation

In the conventional DE, three individuals are randomly selected from the current population, which results in the poor exploitation. According to the selection pressure widely used in the conventional DE, the better the individual, the higher the chance of being offspring. The current population is ranked according to their fitness values. Choosing each vector based on the ranking of their fitness values in the current population instead of selecting it randomly in this step. That is to say, the higher the ranking of the vector, the more likely it is selected as the parent vector [47]. The pseudocode of the ranking-based mutation operator is given in Table 1.

Table 1
Pseudocode of the ranking-based mutation operator

Algorithm 1 Ranking-based mutation operator

Input: The target individual index i

1: Sort the individuals according to the fitness values

calculated above

2: Calculate the selection probability according to

Equation (12).

3: for i = 1toNPd o

4:       Randomly select r₁ ∈ {1, 2, …, NP}

5:       while rndreal [0 , 1) > p_{r
₁}r₁ = = id o

6:       Randomly select r₁ ∈ {1, 2, …, NP}

7: end while

8: Randomly select r₂ ∈ {1, 2, …, NP}

9: while rndreal [0 , 1) > p_{r
₂}r₁ = = r₂o rr₁ = = id o

10:       Randomly select r₂ ∈ {1, 2, …, NP}

11: end while

12: Randomly select r₃ ∈ {1, 2, …, NP}

13: whiler₃ = = r₂orr₃ = = r₁orr₃ = = id o

14:       Randomly select r₃ ∈ {1, 2, …, NP}

15: end while

16: end for

Output: The selected individual indexes r₁, r₂ and r₃.

(1) Weight assignment

To enhance the local search capability of DE, weights are assigned for each vector according to the ranking of their fitness values. First, based on the fitness value of each vector, sort the individuals in the population in ascending order (i.e., from the best to the worst). Then, calculate the weights of the vectors as follows: $R_{i} = NP - i, i = 1, 2, \dots, NP$ (11)

Where NP denotes the population size.

(2) Selection probability

After obtaining the weight for each vector, calculate the selection probability p_i of the i ^th vector x_i according to: $p_{i} = \frac{R_{i}}{NP}, i = 1, 2, \dots, NP$ (12)

According to the above formula, the better the vectors, the higher the selection probability they will get.

(3) Vector selection

After obtaining the selection probability of each vector, another question is, will all of the vectors be selected based on the selection probability? The answer is NO. In this paper, the selection of r₁ and r₂ is linked to their rankings, whereas r₃ should be selected randomly as the traditional DE does in Section 2. The reason why the selection of r₃ is not also dependent on its ranking is that the population diversity should be ensured. Otherwise, it may cause the algorithm to premature convergence and easily trap into local optimum. The specific operations for selecting r₁ and r₂ are as follows:

Randomly select different r₁, r₂∈ { 1, 2, …, NP } ∖ { i }, and calculate their selection probability p_{r
₁}, p_{r
₂};

Compare the random number r randomly generated between 0 and 1 with the selection probability p_{r
₁}, p_{r
₂}. If p_{r
₁}, p_{r
₂} > r, select r₁ and r₂, otherwise, return to step a.

3.2.2 Selection operation based on SA

In the mutation operation, we add the concept of ranking to enhance the local search capability of the algorithm. In this operation, what we need is to avoid premature convergence. The Metropolis criterion of SA can primarily prevent the algorithm from trapping into the local optima and achieve the global optimum by designing a constantly changing probability during the search process. Besides, the competitive local search capability of SA is also helpful to increase the convergence rate of DE. Based on the above considerations, the original greedy selection operator in DE is replaced with the “accept with a certain probability” operator in SA. It is consistent with the cooling progress of SA. The improved selection operation steps are presented as follows:

At each generation g:

(1)
Calculate the fitness value f (x_i (g)) and f (u_i (g)) of the parent vector x_i (g) and the trial vector u_i (g) and the increment delta (i) = f (u_i (g)) - f (x_i (g));

Compare the value of f (u_i (g)) and f (x_i (g)), that is, if delta (i) ⩽ 0, then accept the test vector u_i (g) into the next generation, otherwise, compare the value of exp (- delta/T) with the randomly generated number r between 0 and 1. If exp (- delta/T) ⩾ rand (0, 1), also accept the test vector u_i (g) into the next generation, otherwise, keep x_i (g):

$\begin{matrix} x_{i} (g + 1) = \\ {\begin{matrix} u_{i} (g), iff (u_{i} (g)) ⩽ f (x_{i} (g)) or exp (- \frac{delta}{T}) ⩾ rand (0, 1) \\ x_{i} (g), otherwise \end{matrix} \end{matrix}$ (13)

Where T is the current temperature. Besides, it should be noted that to keep this process consistent with the cooling progress in SA, we set DE to de-temperature every 20 iterations. In this way, the population diversity of DE can be increased, and the competitive local search ability of SA can be retained as well. The pseudocode of this operator is given in Table 2.

Table 2
Pseudocode of SA-based Selection operator

Algorithm 2 SA-based Selection operator

Input: NP: the population size

1: Calculate the fitness value of the individuals

2: fori = 1 NPdo

3:      delta (i)=f(u_i(g)) –f(x_i(g))

4:      if delta (i) < orexp (- delta/T) ⩾ rndreal [0, 1) t hen

5:          Replace x_i with u_i

6:      end if

7: end for

Output: the best individual in the population

Adding the ranking-based mutation operator and the SA-based selection operator we previously proposed, a hybrid DE and SA algorithm (HDESA) is presented subsequently. The pseudocode of HDESA is given in Table 3. With these two novel operations, DE will perform better.

Table 3
Pseudocode of HDESA

Algorithm 3 HDESA

Input: NP: the population size

k: the temperature reduction parameter

G: the maximum number of iterations

T: the current temperature

m: the number of SA iterations

1: Generate an initial population sampling from the feasible

solution space randomly

2: Calculate the fitness value of the individuals

3: for g = 1toGd o

4:    forl = 1tomd o

5:        Perform mutation operation as shown in Algorithm 1

6:     end for

7:     forj = 1toDd o

8:        j_rand=rndreal(1, D)

9:        if rndreal [0 , 1) < C_ro rj = = j_randt hen

10:           u_i,j = x_r₁,j + F · (x_r₂,j - x_r₃,j)

11:        else

12:           u_i,j = x_i₁,j

13:        end if

14:     end for

15:     fori = 1 NPdo

16:        Perform selection operation as shown in Algorithm 2

17:     end for

18:        T = k × T

19:     end for

20: end for

Output: the best individual in the population

3.3 Complexity analysis

In this subsection, both time and space complexity of HDESA are presented [49, 50].

3.3.1 Time complexity

If NP is the number of the population, D is the dimension of the solution space, G is the maximum iteration number, then the time complexity of each operation of HDESA is analyzed as follows:

The HDESA initializes the population in O(1) time.

Mutation operator requires O(NP×(D + log(NP)+1)) time, where population sorting and probability calculation are added.

Crossover operator requires O(NP×D) time.

Selection operator requires O(NP×(D + 1)) time, where probability calculation is added.

In general, the value of D is larger than the value of log(NP). Thus, the total time complexity of HDESA is equal to O(NP×D×G) for maximum number of iterations G.

3.3.2 Space complexity

HDESA uses the number of population which is NP to calculate the space complexity, and the dimension of the search space is D. Therefore, the total space complexity of HDESA is O(NP×D).

4 Experimental results

Some experimental results and analysis are given in this section to study the performance of the proposed algorithm HDESA.

4.1 Benchmark functions

Twenty-four benchmark functions proposed at the CEC2005 special session were used to study the performance of HDESA. If the number of test instances is smaller, the results may not be comprehensive enough, and the algorithm may be biased towards certain types of problems. More details for each function are given in [51]. These 24 functions can be divided into four groups:

(1)
Unimodal functions (F₁ - F₅)

Basic Multimodal functions (F₆ - F₁₂)

Expanded multimodal functions (F₁₃ - F₁₄)

Hybrid composition functions (F₁₅ - F₂₄)

4.2 Parameter settings

In the experimental research of this paper, to compare the proposed HDESA with other algorithms about their performance fairly, unified parameters are set in this study: the population size NP = 50, the dimension of the functions D = 30 and 50, the maximum number of evaluations for each function FES = 500000. Furthermore, the initial temperature in SA involved in this paper T₀ = 10000; the de-temperature parameter k = 0.9; the number of iterations at each temperature m = 20 (which is similar to Markov chain length L in SA). More details on parameter settings for algorithms mentioned in this study are given in Table 4.

Table 4
Parameter settings for algorithms mentioned in this study

Algorithm Parameter settings

SA T₀ = 10000, L = 500, k = 0.9

DE NP = 50, F = 0.5, C_r = 0.9

rank-DE NP = 50, F = 0.5, C_r = 0.9

HDESA NP = 50, F = 0.5, C_r = 0.9, T₀ = 10000,

k = 0.9, m = 20

Algorithm	Parameter settings
SA	T₀ = 10000, L = 500, k = 0.9
DE	NP = 50, F = 0.5, C_r = 0.9
rank-DE	NP = 50, F = 0.5, C_r = 0.9
HDESA	NP = 50, F = 0.5, C_r = 0.9, T₀ = 10000,
	k = 0.9, m = 20

4.3 Comparison of HDESA with other algorithms

The evaluation of the performance of HDESA is obtained by comparing with other algorithms, which are SA, DE and rank-DE [47].

To avoid the large deviation caused by one run, the number of independent runs of the algorithm is set to 30. Each function performs 30 independent evolutions, and the optimal value obtained in each run is obtained and the mean of these results can be computed. The mean and standard deviation function error values of SA, DE, rank-DE and HDESA over 30 independent runs on 24 benchmark functions at D = 30 and D = 50 are presented in Table 5 and 6. To further analyze the performance of the algorithm, the Wilcoxon signed-rank test at α = 0.05 was adopted to compare the HDESA with the traditional DE, classic SA and rank-DE, respectively. The comparison results can also be obtained in Tables 5 and 6. In addition, the final rankings of the four algorithms mentioned above calculated by the Friedman test for all functions are given in Table 7.

Table 5
Experimental results of SA, DE, rank-DE and HDESA at D = 30

Function SA DE rank-DE HDESA

Mean Error±Std Dev Mean Error±Std Dev Mean Error±Std Dev Mean Error±Std Dev

Unimodal Functions F ₁ 14E+0±4.64E-1+ 3.18E-28±4.10E-28 – 8.32E-2±3.99E-1+ 1.02E-19±2.50E-20

F ₂ 72E+1±6.09E+0+ 7.46E-20±2.87E-19 – 1.53E-26±1.79E-26 – 1.67E-19±4.40E-20

F ₃ 05E+6±1.01E+6+ 1.40E+5±8.51E+4+ 5.44E+4±3.07E+4+ 4.61E+4±1.95E+4

F ₄ 21E+5±9.98E+4+ 7.09E-16±2.28E-15+ 7.38E-22±1.36E-21 – 1.98E-19±8.07E-20

F ₅ 46E+3±1.49E+3+ 2.69E+2±2.72E+2+ 5.64E+2±4.32E+2+ 5.83E+1±1.43E+2

Basic Multimodal F ₆ 11E+3±3.23E+3+ 3.52E+2±1.04E+3+ 1.94E+6±1.00E+7+ 3.65E+1±4.78E+1

Functions F ₇ 22E+0±3.63E-2+ 1.14E-2±1.02E-2+ 9.83E-2±2.70E-1+ 1.15E-3±3.02E-3

F ₈ 11E+1±4.42E-2 ≈ 2.09E+1±6.09E-2 ≈ 2.09E+1±5.84E-2 ≈ 2.09E+1±4.71E-2

F ₉ 94E+2±9.50E+1+ 1.59E+1±4.06E+0+ 2.19E+1±5.67E+0+ 1.55E+1±5.25E+0

F ₁₀ 80E+2±1.54E+2+ 7.07E+1±5.52E+1+ 4.14E+1±1.94E+1+ 2.78E+1±1.09E+1

F ₁₁ 55E+1±3.72E+0+ 1.59E+1±1.07E+1+ 1.47E+1±3.73E+0+ 1.34E+1±3.34E+0

F ₁₂ 73E+2±5.21E+3 – 3.89E+3±4.39E+3 – 8.45E+3±1.00E+4+ 6.60E+3±7.83E+3

Expanded F ₁₃ 14E+4±7.74E+3+ 2.76E+0±5.47E-1 ≈ 3.04E+0±5.93E-1+ 2.76E+0±5.53E-1

Functions F ₁₄ 40E+1±3.07E-1+ 1.30E+1±1.96E-1+ 1.45E+1±2.30E-2+ 1.24E+1±3.85E-1

Hybrid Composition F ₁₅ 63E+2±2.39E+2+ 3.50E+2±1.14E+2+ 3.63E+2±9.96E+1+ 3.10E+2±9.23E+1

Functions F ₁₆ 68E+2±2.32E+2+ 1.33E+2±1.40E+2 – 1.61E+2±1.45E+2 ≈ 1.60E+2±1.64E+2

F ₁₇ 91E+3±4.52E+2+ 2.77E+2±8.58E+1+ 2.27E+2±2.05E+2+ 1.84E+2±1.95E+2

F ₁₈ 19E+2±1.68E+2 – 9.05E+2±1.84E+0+ 9.09E+2±7.03E+0+ 9.03E+2±2.10E-1

F ₁₉ 17E+2±1.72E+2 – 9.05E+2±1.45E+0+ 9.08E+2±3.13E+0+ 9.03E+2±1.92E-1

F ₂₀ 19E+2±1.74E+2 – 9.06E+2±1.94E+0+ 9.09E+2±4.46E+0+ 9.03E+2±2.05E-1

F ₂₁ 17E+3±2.30E+2+ 5.00E+2±1.73E-13 ≈ 5.84E+2±1.67E+2+ 5.00E+2±1.73E-13

F ₂₂ 71E+3±3.15E+2+ 8.73E+2±1.41E+1+ 8.81E+2±1.84E+1+ 8.59E+2±1.37E+1

F ₂₃ 43E+3±1.17E+2+ 5.78E+2±1.30E+2+ 6.07E+2±1.54E+2+ 5.34E+2±6.82E-1

F ₂₄ 48E+3±7.50E+1+ 2.00E+2±0.00E+0 – 2.27E+2±1.50E+2 – 3.87E+2±3.45E+2

+ 19 16 19

– 4 5 3

≈ 1 3 2

Function		SA	DE	rank-DE	HDESA
Unimodal Functions	F ₁	14E+0±4.64E-1+	3.18E-28±4.10E-28 –	8.32E-2±3.99E-1+	1.02E-19±2.50E-20
	F ₂	72E+1±6.09E+0+	7.46E-20±2.87E-19 –	1.53E-26±1.79E-26 –	1.67E-19±4.40E-20
	F ₃	05E+6±1.01E+6+	1.40E+5±8.51E+4+	5.44E+4±3.07E+4+	4.61E+4±1.95E+4
	F ₄	21E+5±9.98E+4+	7.09E-16±2.28E-15+	7.38E-22±1.36E-21 –	1.98E-19±8.07E-20
	F ₅	46E+3±1.49E+3+	2.69E+2±2.72E+2+	5.64E+2±4.32E+2+	5.83E+1±1.43E+2
Basic Multimodal	F ₆	11E+3±3.23E+3+	3.52E+2±1.04E+3+	1.94E+6±1.00E+7+	3.65E+1±4.78E+1
Functions	F ₇	22E+0±3.63E-2+	1.14E-2±1.02E-2+	9.83E-2±2.70E-1+	1.15E-3±3.02E-3
	F ₈	11E+1±4.42E-2 ≈	2.09E+1±6.09E-2 ≈	2.09E+1±5.84E-2 ≈	2.09E+1±4.71E-2
	F ₉	94E+2±9.50E+1+	1.59E+1±4.06E+0+	2.19E+1±5.67E+0+	1.55E+1±5.25E+0
	F ₁₀	80E+2±1.54E+2+	7.07E+1±5.52E+1+	4.14E+1±1.94E+1+	2.78E+1±1.09E+1
	F ₁₁	55E+1±3.72E+0+	1.59E+1±1.07E+1+	1.47E+1±3.73E+0+	1.34E+1±3.34E+0
	F ₁₂	73E+2±5.21E+3 –	3.89E+3±4.39E+3 –	8.45E+3±1.00E+4+	6.60E+3±7.83E+3
Expanded	F ₁₃	14E+4±7.74E+3+	2.76E+0±5.47E-1 ≈	3.04E+0±5.93E-1+	2.76E+0±5.53E-1
Functions	F ₁₄	40E+1±3.07E-1+	1.30E+1±1.96E-1+	1.45E+1±2.30E-2+	1.24E+1±3.85E-1
Hybrid Composition	F ₁₅	63E+2±2.39E+2+	3.50E+2±1.14E+2+	3.63E+2±9.96E+1+	3.10E+2±9.23E+1
Functions	F ₁₆	68E+2±2.32E+2+	1.33E+2±1.40E+2 –	1.61E+2±1.45E+2 ≈	1.60E+2±1.64E+2
	F ₁₇	91E+3±4.52E+2+	2.77E+2±8.58E+1+	2.27E+2±2.05E+2+	1.84E+2±1.95E+2
	F ₁₈	19E+2±1.68E+2 –	9.05E+2±1.84E+0+	9.09E+2±7.03E+0+	9.03E+2±2.10E-1
	F ₁₉	17E+2±1.72E+2 –	9.05E+2±1.45E+0+	9.08E+2±3.13E+0+	9.03E+2±1.92E-1
	F ₂₀	19E+2±1.74E+2 –	9.06E+2±1.94E+0+	9.09E+2±4.46E+0+	9.03E+2±2.05E-1
	F ₂₁	17E+3±2.30E+2+	5.00E+2±1.73E-13 ≈	5.84E+2±1.67E+2+	5.00E+2±1.73E-13
	F ₂₂	71E+3±3.15E+2+	8.73E+2±1.41E+1+	8.81E+2±1.84E+1+	8.59E+2±1.37E+1
	F ₂₃	43E+3±1.17E+2+	5.78E+2±1.30E+2+	6.07E+2±1.54E+2+	5.34E+2±6.82E-1
	F ₂₄	48E+3±7.50E+1+	2.00E+2±0.00E+0 –	2.27E+2±1.50E+2 –	3.87E+2±3.45E+2
+		19	16	19
–		4	5	3
≈		1	3	2

* “Mean Error” and “Std Dev” denote the average and standard deviation of the function error values obtained 30 runs, respectively. “+”, “–”, “≈” represent that the performance of our approach is better than, worse than, and the same as that of its competitors according to the Wilcoxon signed-rank test at α= 0.05, respectively.

Table 6

Experimental results of SA, DE, rank-DE and HDESA at D = 50

Function		SA	DE	rank-DE	HDESA
		Mean Error±Std Dev	Mean Error±Std Dev	Mean Error±Std Dev	Mean Error±Std Dev
Unimodal Functions	F ₁	68E+1±1.77E+0+	3.93E-27±1.05E-26 –	1.10E+1±2.63E+1+	1.70E-19±3.10E-20
	F ₂	74E+2±3.31E+1+	7.80E-5±1.10E-4+	4.30E-11±7.24E-11+	1.94E-11±3.84E-11
	F ₃	96E+6±2.30E+6+	5.31E+5±2.07E+5+	3.74E+5±1.27E+5+	3.98E+5±1.50E+5
	F ₄	28E+5±3.66E+5+	1.06E-1±1.05E-1+	5.18E-3±9.87E-3+	2.72E-3±3.51E-3
	F ₅	33E+4±1.90E+3+	3.24E+3±7.26E+2+	3.69E+3±7.68E+2+	2.72E+3±5.94E+2
Basic Multimodal	F ₆	85E+3±4.97E+3 –	6.36E+5±2.73E+6 –	2.33E+7±7.68E+7+	1.60E+7±2.94E+7
Functions	F ₇	67E+0±6.25E-2+	9.66E-3±1.63E-2+	2.03E+0±4.60E+0+	1.23E-3±3.97E-3
	F ₈	12E+1±3.96E-2 ≈	2.11E+1±3.53E-2 ≈	2.11E+1±3.73E-2 ≈	2.11E+1±4.01E-2
	F ₉	64E+2±1.26E+2+	3.58E+1±9.70E+0+	5.61E+1±1.36E+1+	3.45E+1±7.56E+0
	F ₁₀	27E+3±1.66E+2+	2.66E+2±1.22E+2+	7.89E+1±1.92E+1+	7.44E+1±7.29E+1
	F ₁₁	33E+2±4.32E+1+	7.31E+1±1.42E+0+	4.06E+1±1.85E+1+	4.09E+1±1.66E+1
	F ₁₂	3.48E+4±1.55E+4 –	2.44E+4±1.74E+4 –	3.38E+4±2.42E+4 –	3.88E+4±3.27E+4
Expanded Functions	F ₁₃	12E+4±1.25E+4+	5.12E+0±8.85E-1+	5.41E+0±1.05E-1+	4.62E+0±5.93E-1
	F ₁₄	40E+1±2.23E-1+	2.29E+1±2.42E-1+	2.26E+1±3.18E-1+	2.27E+1±3.22E-1
Hybrid Composition	F ₁₅	77E+2±1.94E+2+	3.17E+2±9.18E+1+	3.62E+2±6.33E+1+	2.50E+2±6.42E+1
Functions	F ₁₆	6.30E+2±1.83E+2+	2..44E+2±1.04E+2+	1.66E+2±1.52E+2 ≈	1.61E+2±1.69E+2
	F ₁₇	20E+3±5.54E+2+	3.76E+2±5.94E+1+	3.47E+2±6.50E+1 –	3.52E+2±7.27E+1
	F ₁₈	30E+3±1.40E+2+	9.22E+2±9.22E+0+	9.41E+2±1.41E+1+	9.13E+2±4.95E-1
	F ₁₉	25E+3±9.83E+1+	9.24E+2±4.45E+0+	9.35E+2±5.61E+0+	9.13E+2±8.21E-1
	F ₂₀	28E+3±1.12E+2+	9.22E+2±3.98E+0+	9.40E+2±1.01E+1+	9.12E+2±4.44E-1
	F ₂₁	30E+3±1.01E+2+	6.29E+2±2.20E+2 –	6.84E+2±2.39E+2 –	1.00E+3±1.09E-0
	F₂₂x.	91E+3±2.23E+2+	9.16E+2±1.86E+1+	9.32E+2±2.19E+1+	9.01E+2±4.23E+0
	F ₂₃	51E+3±1.04E+2+	7.76E+2±2.30E+2 –	7.57E+2±2.33E+2 –	1.01E+3±1.12E+0
	F ₂₄	60E+3±5.83E+1+	2.00E+2±0.00E+0 ≈	2.00E+2±0.00E+0 ≈	2.00E+2±0.00E+0
+		21	17	18
–		2	5	4
≈		1	2	2

* “Mean Error” and “Std Dev” denote the average and standard deviation of the function error values obtained 30 runs, respectively. “+”, “–”, “≈”represent that the performance of our approach is better than, worse than, and the same as that of its competitors according to the Wilcoxon signed-rank test at α= 0.05, respectively.

Table 7

Average rankings of SA, DE, rank-DE and HDESA according to the Friedman test for all functions

D = 30		D = 50
Algorithm	Ranking	Algorithm	Ranking
SA	3.2708	SA	3.7917
DE	1.9792	DE	2.2667
rank-DE	2.9167	rank-DE	2.3333
HDESA	1.8333	HDESA	1.7083

Obviously, HDESA has better performance than the other three algorithms as a whole according to the last three rows of Tables 5 and 6 and the average rankings showed in Table 7. The detailed description is presented as follows:

Unimodal functions (F₁ - F₅). On these five unimodal functions, no matter D = 30 or 50, HDESA performs the best. For D = 30, HDESA is significantly better than DE and rank-DE on three benchmark functions, respectively. SA cannot outperform HDESA on any unimodal function. For D = 50, HDESA outperforms SA, DE and rank-DE on 5, 4 and 5 benchmark functions, which performs even better than it does at D = 30. On the one hand, the possibility of choosing better solutions among the population is increased by using the ranking-based mutation operator, which can speed up the convergence rate when solving unimodal functions. On the other hand, the SA-based selection operator increases the population diversity and avoids over-exploitation. Hence, it can help strike a balance between the exploitation ability and the exploration ability.

Basic multimodal functions (F₆ - F₁₂). Clearly, on these seven basic multimodal functions, HDESA is the best among the four algorithms at both D = 30 and 50. For D = 30, it outperforms SA, DE and rank-DE on 5, 5 and 6 benchmark functions, respectively. SA and DE outperform HDESA on one function, and rank-DE cannot perform better than HDESA on any problems. For D = 50, HDESA is significantly better than SA, DE and rank-DE on four benchmark functions, respectively. We can also observe that rank-DE outperforms DE only on one test function (1 out of 7) no matter D = 30 or 50, which confirms that the ranking-based mutation operator only may lead to over-exploitation and local optima. Our proposed HDESA adopts SA-based selection operator to balance the exploitation and exploration ability, proving very useful.

Expanded multimodal functions (F₁₃ - F₁₄). It is clear that HDESA remains the winner of the four algorithms on these two expanded multimodal functions at both D = 30 and 50. SA, DE and rank-DE cannot outperform HDESA on any test function.

Hybrid composition functions (F₁₅ - F₂₄). These functions are far more complicated than other test functions mentioned above because each of them contains ten sub-functions. So, it is challenging to solve them for almost all algorithms. On these ten test functions, HDESA outperforms SA, DE and rank-DE on 7, 8 and 8 tests at D = 30, respectively. The performance of SA, DE and rank-DE can only surpass HDESA on 0, 2 and 3 tests at D = 50. Thus, HDESA is still the winner on the ten hybrid composition functions. The superior performance of HDESA has been further demonstrated here.

With respect to the Friedman test showed in Table 7, the average rankings of SA, DE, rank-DE and HDESA for all functions are presented. From the results, it can be seen that HDESA obtains the best ranking at both D = 30 and 50, which also tells us that HDESA performs best among these four algorithms.

To further clarify the comparison of the HDESA algorithm with other algorithms, the logarithm of mean function error values of SA, DE, rank-DE and HDESA over 30 independent runs on 24 test functions at D = 30 versus the number of FES is plotted in Fig. 3. It can be seen from Fig. 3 that the overall convergence speed of HDESA is faster than the other three algorithms on most functions, and there is no local optimum.

Fig. 3

Convergence graph of SA, DE, rank-DE and HDESA on 24 test functions.

Moreover, the search history and trajectory are given in Fig. 4. The first column of Fig. 4 is the benchmark function. The second column of Fig. 4 is the search history, which tells that the solution is searched extensively at the beginning and the optimal solution is exploited at the end. Besides, the third column of Fig. 4 gives the trajectory, from which we can draw the conclusion that the search ability of HDESA is quite strong and the diversity of the population is guaranteed. Finally, the fourth and fifth columns of Fig. 4 prove the high convergence rate of the algorithm once again.

Fig. 4

Search history and trajectory of HDESA.

In summary, according to the results and analysis, it is not difficult to notice that the proposed HDESA can vastly improve the performance of DE on various problems, which are either unimodal or multimodal, separable or non-separable, clean or noisy, continuous or non-continuous. This is because HDESA uses ranking-based mutation operator, which speeds up the convergence of the algorithm and introduces the idea of SA in the selection operation. Metropolis criterion is employed to choose the individuals as offspring rather than selecting them greedily, enabling the algorithm to have a strong global search capability without losing its exploitation capability.

4.4 Discussion

Different settings of the number of iterations at each temperature m may influence the performance of HDESA. The m is an important parameter, which may influence the performance of the algorithm. It is necessary to test the influence of the parameter. To verify the performance of HDESA with different value of m, twelve typical benchmark functions are adopted to perform at m = 20 and 25, respectively. The twelve functions selected cover four types of functions, which are unimodal functions, basic multimodal functions, expanded multimodal functions and hybrid composition functions. Table 8 gives the average function error values of HDESA over 30 independent runs on twelve benchmark functions at m = 20 and 25, respectively.

Table 8
Experimental results of HDESA over 30 independent runs on 12 test functions at m = 20 and 25, respectively

Function m = 20 m = 25

Mean Error Mean Error

F ₃ 61E+04 4.04E+04

F ₄ 98E-19 6.19E-19

F ₅ 83E+01 5.65E+02

F ₆ 65E+01 2.34E+01

F ₈ 09E+01 2.09E+01

F ₁₀ 78E+01 3.08E+01

F ₁₃ 76E+00 2.21E+00

F ₁₄ 24E+01 1.28E+01

F ₁₈ 03E+02 9.03E+02

F ₂₀ 03E+02 9.03E+02

F ₂₂ 59E+02 8.63E+02

F ₂₄ 87E+02 2.00E+02

Function	m = 20	m = 25
F ₃	61E+04	4.04E+04
F ₄	98E-19	6.19E-19
F ₅	83E+01	5.65E+02
F ₆	65E+01	2.34E+01
F ₈	09E+01	2.09E+01
F ₁₀	78E+01	3.08E+01
F ₁₃	76E+00	2.21E+00
F ₁₄	24E+01	1.28E+01
F ₁₈	03E+02	9.03E+02
F ₂₀	03E+02	9.03E+02
F ₂₂	59E+02	8.63E+02
F ₂₄	87E+02	2.00E+02

There are no significant differences in the performance of the proposed algorithm HDESA when m takes 20 and 25, which shows that the setting of m has little influence on the performance of the HDESA algorithm. So, we simply set m to 20.

5 Conclusion

In the ranking-based mutation operation, the higher the ranking of the vector, the more likely it is selected as the parent vector, which can enhance the exploitation capability of DE. However, it may lead to trapping in local optimum for some cases. The Metropolis criterion in SA can effectively improve the population diversity and ultimately achieve the global optimum, replacing the original greedy selection operator in DE. Inspired by this, we propose a hybrid DE and SA algorithm to balance the exploration and exploitation ability, which is simple but efficient. The main innovation points of the algorithm are listed as follows:

The concept of ranking is added to the mutation operation to make full use of the information of the better individuals in the current population, enhancing the exploitation ability of DE greatly.

The greedy selection operator in DE is replaced with the accept with a certain probability one in SA, increasing the population diversity and balancing the exploitation and exploration capability.

Unlike some other hybrid algorithms, HDESA has a relatively simple coding and has almost no additional parameters without losing its efficiency.

The experimental research has been conducted on 24 benchmark global functions proposed at the CEC2005 special session. HDESA was compared with three other algorithms, i.e., SA, DE and rank-DE. The experimental results and analysis have shown its better or at least equivalent performance on exploitation and exploration ability. Besides, the influence of the number of iterations at each temperature m on the algorithm is experimentally studied.

In this paper, we introduce SA-based selection operator to DE algorithm. There remain other effective algorithms that we can utilize to enhance the performance of DE. Thus, we will try to hybrid some other advanced algorithms such as PSO or some ideas from other algorithms with DE to get better results in our near future work.

Footnotes

Acknowledgments

This research was funded by the China Natural Science Foundation (No.71974100), Natural Science Foundation (No. BK20191402) in Jiangsu Province, Major Project of Philosophy and Social Science Research in Colleges and Universities in Jiangsu province (2019SJZDA039), Qing Lan Project (R2019Q05), and philosophy and Social Sciences in Universities of Jiangsu (No. 2020SJA0182).

References

Storn

and Price

, Minimizing the real functions of the ICE’96 contest by differential evolution, Proc. IEEE Conf 1996, 842–844In.

Storn

and Price

, Differential evolution —A simple and efficient heuristic for global optimization over continuous spaces, J Global Optim 11(4) (1997), 341–359.

Chen

, Du

and Qian

, Solving chemical dynamic optimization problems with ranking-based differential evolution algorithms, Chinese Journal of Chemical Engineering 24(11) (2016), 1600–1608.

Manjit

and Dilbag

, Multi-modality medical image fusion technique usingmulti-objective differential evolution based deep neural networks, Journal of Ambient Intelligence and Humanized Computing (2020).

Balaji

, Balamurugan

and Lakshminarasimman

, Mathematical approach assisted differential evolution for generator maintenance scheduling, International Journal of Electrical Power & Energy Systems 82 (2016), 508–518.

Chamorro

H.R.

, et al., Synthetic inertia control based on fuzzy adaptive differential evolution, International Journal of Electrical Power & Energy Systems 105 (2019), 803–813.

Kadhar

K.M.A.

, Baskar

and Amali

S.M.J.

, Diversity Controlled Self Adaptive Differential Evolution based design of non-fragile multivariable PI controller, Engineering Applications of Artificial Intelligence 46 (2015), 209–222.

Stopar

, et al., Electric arc modeling of the EAF using differential evolution algorithm, Materials and Manufacturing Processes 32(10) (2017).

Liu

, Li

and Gong

, Rethinking the differential evolution algorithm, Service Oriented Computing and Applications, (2020) (prepublish).

10.

Bilal , et al., Differential Evolution: A review of more than two decades of research, Engineering Applications of Artificial Intelligence 90 (2020).

11.

Gamlerle

, Muller

S.D.

and Koumoutsakos

, A parameter study for differential evolution, Adv Intell Syst 10 (2002), 293–298.

12.

Ronkkonen

, Kukkonen

and Price

K.V.

, Real-Parameter Optimization with Differential Evolution, In: 2005 IEEE International Conference on Evolutionary Computation 1 (2005), 506–513.

13.

Zielinski

K.L.R.

, Constrained Single-Objective Optimization Using Differential Evolution, In: 2006 IEEE International Conference on Evolutionary Computation (2006), 223–230.

14.

Wang

, Cai

and Zhang

, Differential Evolution with Composite Trial Vector Generation Strategies and Control Parameters, Ieee Transactions on Evolutionary Computation 15(1) (2011), 55–66.

15.

Xuxu

, Peng

and Guoqiang

, An Improved Differential Evolution Algorithm Based on Dual-Strategy, Mathematical Problems in Engineering 2020 (2020).

16.

, Cai

and Cao

, A novel mutation differential evolution for global optimization, Journal of Intelligent & Fuzzy Systems 28(3) (2015), 1047–1060.

17.

Zhiping

, Kangshun

and Yi

, Differential evolution with adaptive mutation strategy based on fitness landscape analysis, Information Sciences 549 (2021).

18.

Mezura-Montes

, and

J.V.-R.

and Coello Coello

C.A.

, Modified differential evolution for constrained optimization, IEEE Press (2006), 332–339.

19.

Biswas

, Kundu

and Das

, An Improved Parent-Centric Mutation with Normalized Neighborhoods for Inducing Niching Behavior in Differential Evolution, Ieee Transactions on Cybernetics 44(10) (2014), 1726–1737.

20.

Sun

and Cai

, Differential evolution with dynamic neighbourhood learning strategy-based mutation operators, Int. J. of Computational Science and Engineering 19(1) (2019).

21.

Mohamed

A.W.

, An improved differential evolution algorithm with triangular mutation for global numerical optimization, Computers & Industrial Engineering 85 (2015), 359–375.

22.

Kaelo

and Ali

M.M.

, Differential evolution algorithms using hybrid mutation, Computational Optimization and Applications 37(2) (2007), 231–246.

23.

Das

, et al., Differential Evolution Using a Neighborhood-Based Mutation Operator, Ieee Transactions on Evolutionary Computation 13(3) (2009), 526–553.

24.

Zhang

and Sanderson

A.C.

, JADE: Adaptive Differential Evolution With Optional External Archive, Ieee Transactions on Evolutionary Computation 13(5) (2009), 945–958.

25.

, Zhang

and Wang

, Differential evolution with improved elite archive mutation and dynamic parameter adjustment, Cluster Computing: The Journal of Networks, Software Tools and Applications 22(4) (2019).

26.

Zaharie

, A comparative analysis of crossover variants in differential evolution, Comput Sci Inf Technol 2007, 171–181.

27.

Zhao

S.-Z.

and Suganthan

P.N.

, Empirical investigations into the exponential crossover of differential evolutions, Swarm and Evolutionary Computation 9 (2013), 27–36.

28.

Wang

, Cai

and Zhang

, Enhancing the search ability of differential evolution through orthogonal crossover, Information Sciences 185(1) (2012), 153–177.

29.

Fan

and Zhang

, Self-adaptive differential evolution algorithm with crossover strategies adaptation and its application in parameter estimation, Chemometrics and Intelligent Laboratory Systems 151 (2016), 164–171.

30.

LiBao

, Haili

and Chunlei

, JDF-DE: a differential evolution with Jrand number decreasing mechanism and feedback guide technique for global numerical optimization, Applied Intelligence 51(1) (2020).

31.

, et al., An improved adaptive differential evolution algorithm for continuous optimization, Expert Systems with Applications 44 (2016), 1–12.

32.

Jing

, et al., A clustering-based differential evolution algorithm for solving multimodal multi-objective optimization problems, Swarm and Evolutionary Computation 60 (2021).

33.

Arab

and Alfi

, An adaptive gradient descent-based local search in memetic algorithm applied to optimal controller design, Information Sciences 299 (2015), 117–142.

34.

Mousavi

and Alfi

, A memetic algorithm applied to trajectory control by tuning of Fractional Order Proportional-Integral-Derivative controllers, Applied Soft Computing 36 (2015), 599–617.

35.

Zhang

, Ding

and Jia

, A hybrid optimization algorithm based on cuckoo search and differential evolution for solving constrained engineering problems, Engineering Applications of Artificial Intelligence 85 (2019), 254–268.

36.

Jadon

S.S.

, et al., Hybrid Artificial Bee Colony algorithm with Differential Evolution, Applied Soft Computing 58 (2017), 11–24.

37.

Liu

, Yao

and Li

, Hybrid whale optimization algorithm enhanced with Lévy flight and differential evolution for job shop scheduling problems, Applied Soft Computing 87 (2020), 105954.

38.

Mohammadi

, et al., Implementation of hybrid particle swarm optimization-differential evolution algorithms coupled with multi-layer perceptron for suspended sediment load estimation, CATENA 198 (2021), 105024.

39.

, et al., An improved simulated annealing algorithm based on residual network for permutation flow shop scheduling, Complex & Intelligent Systems (2020), (prepublish).

40.

, et al., Self-adaptive Hybrid differential evolution with simulated annealing algorithm for numerical optimization, in 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence). (2008).

41.

Olenšek

, et al., A new asynchronous parallel global optimization method based on simulated annealing and differential evolution, Applied Soft Computing Journal 11(1) (2010).

42.

Zhao

, et al., A new hybrid differential evolution with simulated annealing and self-adaptive immune operation, Computers & Mathematics with Applications 66(10) (2013), 1948–1960.

43.

Zeng Yurong

Z.J.

, Lu

and lin

, Hybrid differential evolution supported by simulated annealing and its application in integrated joint replenishment-delivery problems, Application Research of Computers 35(04) (2018), 1037–1041.

44.

Assadi

M.T.

and Bagheri

, Differential evolution and Population-based simulated annealing for truck scheduling problem in multiple door cross-docking systems, Computers & Industrial Engineering 96 (2016), 149–161.

45.

, et al., Adaptive Differential Evolution Based on Simulated Annealing for Large-Scale Dynamic Economic Dispatch with Valve-Point Effects, Mathematical Problems in Engineering 2018 (2018).

46.

Mirsadeghi

and Khodayifar

, Hybridizing particle swarm optimization with simulated annealing and differential evolution, Cluster Computing (2020), (prepublish).

47.

Gong

and Cai

, Differential Evolution With Ranking-Based Mutation Operators, Ieee Transactions on Cybernetics 43(6) (2013), 2066–2081.

48.

Julian

and David

, A simulated annealing algorithm with a dual perturbation method for clustering, Pattern Recognition (2020), (prepublish).

49.

Anari

, Akbari Torkestani

and Rahmani

A.M.

, Automatic data clustering using continuous action-set learning automata and its application in segmentation of images, Applied Soft Computing 51 (2017), 253–265.

50.

Zhou

, Ling

and Luo

, Levy flight trajectory-based whale optimization algorithm for engineering optimization, Engineering Computations 35(7) (2018), 2406–2428.

51.

Suganthan

P.N.

, N.H., Liang

J.J.

, Deb

, Chen

Y.-P.

, Auger

and Tiwari

, Problem definitions and evolution criteria for the CEC 2005 specoal session on real-parameter optimization, (2005).

A hybrid differential evolution and simulated annealing algorithm for global optimization

Abstract

Keywords

1 Introduction

2 Differential evolution (DE) and simulated annealing (SA)

2.1 Differential evolution (DE)

3.1 Motivation of the work

3.2 Framework of hybrid DE and SA algorithm (HDESA)

3.2.1 Ranking based mutation operation

3.3.1 Time complexity

3.3.2 Space complexity

4 Experimental results

4.1 Benchmark functions

(1) Unimodal functions (F1 - F5) Basic Multimodal functions (F6 - F12) Expanded multimodal functions (F13 - F14) Hybrid composition functions (F15 - F24) 4.2 Parameter settings

Table 4 Parameter settings for algorithms mentioned in this study Algorithm Parameter settings SA T0 = 10000, L = 500, k = 0.9 DE NP = 50, F = 0.5, C r = 0.9 rank-DE NP = 50, F = 0.5, C r = 0.9 HDESA NP = 50, F = 0.5, C r = 0.9, T0 = 10000, k = 0.9, m = 20

Footnotes

Acknowledgments

References

(1)
Unimodal functions (F₁ - F₅)

Basic Multimodal functions (F₆ - F₁₂)

Expanded multimodal functions (F₁₃ - F₁₄)

Hybrid composition functions (F₁₅ - F₂₄)

4.2 Parameter settings

Table 4
Parameter settings for algorithms mentioned in this study

Algorithm Parameter settings

SA T₀ = 10000, L = 500, k = 0.9

DE NP = 50, F = 0.5, C_r = 0.9

rank-DE NP = 50, F = 0.5, C_r = 0.9

HDESA NP = 50, F = 0.5, C_r = 0.9, T₀ = 10000,

k = 0.9, m = 20