A meta-optimizer based on the upper confidence bound algorithm and orthogonal arrays: A preliminary empirical study

Abstract

It is well-known that tuning a metaheuristic is a critical task because the performance of a metaheuristic and the quality of its solutions depend on its parameter values. However, finding a good parameter setting is a time-consuming task. In this work, we apply the upper confidence bound (UCB) algorithm to automate offline tuning in a (1 + 1)-evolution strategy. Preliminary results show that our proposed approach is a less costly method.

Keywords

Upper confidence bound algorithm meta-optimizer bandit problems reinforcement learning

1 Introduction

Metaheuristics solve a wide umbrella of optimization problems, covering areas from wellness [22] and engineering [11] to health [19] and arts [3]. The exploration and exploitation phases in metaheuristics are related to their parameter values. The parameter setting task specifies a suitable value for any parameter for a metaheuristic to improve the way it can move through the search space. Commonly, there are two approaches to setting parameter values in a metaheuristic [6]: online and offline parameter tuning methods. The first method consists of establishing the parameter values before the run of the metaheuristic. Then the parameter values remain fixed during the run. On the other hand, the second method changes parameter values on the fly of the run. The above suggests that the metaheuristic changes the parameter values according to the course of the run.

The no-free lunch (NFL) theorem [24] indicates that no optimization algorithm solves all optimization problems. The above implies that researchers and end users need not only to adapt the selected metaheuristic to achieve a good solution for a specific problem but also to perform the parameter setting task every time they face new problems [6]. A study in [1] reports that developing a new metaheuristic takes up only 10% of the time required, while the remaining 90% is spent on setting parameter values. In consequence, the tuning task is a time-consuming and tedious process.

All the above motivates us to propose a meta-optimizer based on UCB1 [2] and orthogonal arrays to automate offline tuning. The term meta-optimizer (tuner) was first coined by Mercer and Sampson [12]. It refers to the fact that the offline tuning problem is also an optimization problem that searches for the optimal set of parameters for the metaheuristic. We focus on offline parameter tuning method due to like work in [6] mentions, the advantage of dealing with it is its universality. In other words, if one can develop an efficient tuning method, it can serve to tune various metaheuristics.

UCB1 is part of the branch of machine learning (ML) algorithms called reinforcement learning (RL) algorithms. The integration of ML algorithms into the offline tuning problem has been reported in studies such as [6, 9], and [21].

To verify the effectiveness of our proposed meta-optimizer, we applied it to tune a (1 + 1)-evolution strategy [23] for solving the positioning of unmanned aerial vehicles (UAVs) [16].

The remainder of the work is organized as follows. Section 2 describes the UCB1 algorithm and the (1 + 1)-evolution strategy. Consequently, Section 3 gives details about our proposed meta-optimizer. Next, Section 4 shows preliminary results. Finally, Section 5 concludes this work and gives future directions.

1.1 Related work

In recent years, there has been an increasing number of proposed automated algorithm configuration techniques to tune metaheuristics through a computer-intensive search process. They can be classified as follows:

-
Experimental design techniques or sampling methods. Researchers have adopted statistical techniques like hypothesis testing to evaluate performance significance and avoid failures in the trial-and-error process. The most commonly used sampling methods are Latin-Square and Taguchi orthogonal designs [5].
-
Configurations based on surrogate models. These approaches are referred to as Bayesian optimization. They predict configuration performance based on previous results from running similar configurations on problem instances [4, 17].
-
Selection methods. The idea of selection methods is to identify the best parameter vector from a set of vectors with a minimum number of tests.

According to [25], the most relevant parameter-tuning techniques are:

ParamILS performs an iterated local search in the parameter space [8] and treats the problem as a task that only considers the configuration of categorical variables. Therefore, the numerical parameters need to be discretized, which can be achieved by generating a series of discrete values for each parameter using prior or random knowledge. For initialization, ParamILS requires an initial configuration as input and generates a small number of random configurations.

The SMAC configurator implements a surrogate model-based search of the parameter space [7]. Unlike ParamILS, SMAC handles numerical and categorical parameters natively, meaning it does not discretize numerical parameters.

The I-Race package [20] implements configuration procedures using an iterative mechanism that involves (i) generating candidate algorithm configurations through a probabilistic mechanism, (ii) selecting the best configurations through runs, and (iii) updating the probabilistic model used to generate candidate configurations, focusing on the elite candidate configurations.

REVAC [18] is a heuristic method that can be described as an evolutionary algorithm, working with a population of m parameter vectors. This population is updated by selecting parent vectors, which are recombined and mutated to produce a second vector that is inserted into the population.

EVOCA [13] is an evolutionary tuner that works with a set of parameters to be calibrated. The population size is calculated based on the number of parameters and their domains. It includes a set of relevant values for each parameter independently in its initial population. A set of well-distributed values is selected for each tuned parameter, and those values are combined in calibrations using a Latin hypercube design.

After conducting a thorough review of the literature, it became clear that there was a lack of exploration regarding reinforcement learning procedures. In the upcoming section, we will outline our bold approach of utilizing the UCB1 algorithm to fine-tune a metaheuristic.
2 Background

2.1 Upper confidence bound 1

UCB1 is one of the RL algorithms to solve the multi-armed bandit (MAB) problem. Briefly, the MAB problem aims to find which machine in a casino will give us the maximum average reward. To do so, an agent performs an action a (pull an arm from the gambling machine) at each time step t to receive a reward. This simple idea has many practical applications as identifying a suitable advertisement banner.

The maximum average reward is defined as follows [15]:

Q (a^{*}) = arg max_{a} Q_{t} (a)

(1)

where

Q_{t} (a) = \frac{R (a)}{N (a)}

(2)

In (1),

arg max_{a}

is the action a for which the formula that follows is maximized. On the other hand, in (2), R (a) is the sum of rewards received from action a. Consequently, N (a) is the total number of times the action a was taken.

UCB1 performs considering the principle of optimism in the face of uncertainty. It selects the best action a taking into account a confidence interval. In other words, UCB1 chooses an arm whose UCB1 value is the highest to explore [14] as follows:

Arm = arg max_{a} [Q_{t} (a) + \sqrt{\frac{2 \ln (t)}{N (a)}}]

(3)

where t is the total number of rounds done so far.

Next, in Algorithm 1, we describe the steps to select the best arm according to UCB1 policy. Algorithm 1 is divided into two phases: the initialization and the selection. The first stage is performed from STEP 2 to STEP 7 which is to play at each arm once. The count vector keeps track of the number of times the i - th arm is pulled. The sum _ rewards vector keeps the cumulative rewards obtained from the i - th arm. The Q vector stores the average rewards obtained from the i - th arm.

After, Algorithm 1 performs the selection phase from STEP 9 to STEP 19. This stage identifies Q (a^*). To do so, Algorithm 1 computes the UCB1 policy for each arm at each iteration. This is shown from STEP 10 to STEP 13. In STEP 10, num _ of _ arms refers to the total number of arms to play. Then, in STEP 14, Algorithm 1 chooses the arm to pull at each iteration according to UCB1 policy. Once, Algorithm 1 identifies the arm to pull at each iteration according to (3); the count vector, the Q vector, and the sum _ rewards vector are updated for that arm. Then, when Algorithm 1 achieves the total number of iterations (num _ of _ rounds in Algorithm 1), it takes the highest Q value from Q vector as (1) indicates. The above is shown in STEP 19.

Algorithm 1 UCB1

1: Initialization phase ⊳ Play each arm once

2: for i = 1, 2, …, num _ of _ armsdo

3: Obtain reward from arm i and keep it in variable reward _ arm

4: count [i] = count [i] +1

5: sum _ rewards [i] = sum _ rewards [i] + reward _ arm

6: Q [i] = sum _ rewards [i]/count [i]

7: end For

8: Selection phase ⊳ Select the arm which has maximum Q value

9: for i = 1, 2, …, num _ of _ roundsdo

10: for arm = 1, 2, …, num _ of _ armsdo ⊳ Select the arm to explore

11: $upper_bound = \sqrt{\frac{2 * \ln * i}{count [arm]}}$

12: ucb1 [arm] = Q [arm] + upper _ bound

13: end For

14: Select the arm that has the maximum value in ucb1 vector and obtain the reward from this arm

15: Update the count vector of that arm

16: Update the sum _ rewards vector with the reward from that arm

17: Update the Q vector for that arm

18: end For

19: Select the arm which has a maximum Q value

In subsequent sections, we refer to arms by using the term bandits.

2.2 The metaheuristic to be tuned

We aim to tune offline a (1 + 1)-evolution strategy to locate in a disaster zone a set of unmanned aerial vehicles (UAVs) to provide voice services to ground users (victims and first responders) as Fig. 1 shows.

Fig. 1

Optimization problem.

On the other hand, Fig. 2 deploys the idea behind the (1 + 1)-evolution strategy. The flow chart begins by requesting the step size σ, the step size adaptation z, and the number of iterations t. Then, a parent solution X is randomly generated. This parent solution X is evaluated. The parent solution X is mutated, and a child X’ is generated. Subsequently, the feasibility of the child solution X’ is evaluated. If the child’s solution is feasible, it is evaluated in the objective function. Otherwise, it is penalized with zero. The fitness of the parent solution X and the child solution X’ is compared. If the child solution X’ is better than the parent solution X, the child solution X’ becomes the parent solution for the next generation. Then, the success counter is increased. Otherwise, the parent solution X is retained as is for the next generation, and the failure counter is increased. Next, it is checked whether it is time to update the step size σ. If it is time to update the step size σ, there are three options: increase it (σ/c), decrease it (σ · c), or keep it the same (σ). If the stopping condition (number of iterations) is met, the last parent individual is the solution to the problem. Otherwise, another iteration of the algorithm begins from the step of evaluating the parent solution X.

Fig. 2

Flow chart of (1 + 1)-evolution strategy.

(1 + 1)-evolution strategy returns as a solution the geographic coordinates to achieve the maximum probability of successfully providing voice services to ground users.

We identify by previous experiments that the parameters that most influence the exploration/exploitation trade-off in the (1 + 1)-evolution strategy are: the step size σ and the step size adaptation z. In short, σ controls mutation, and z indicates how many iterations elapse to adapt σ. Table 1 shows the parameters used for the (1 + 1)-evolution strategy.

Table 1

Parameters used for the (1 + 1)-evolution strategy

Parameter	Values
Population size	1
Number of offspring	1
Step size adaptation z	5,10,20 iterations
Step size σ	0.0001,0.001,0.004
Total number of iterations	500
Adaptation speed c	0.817

The (1 + 1)-evolution strategy applied to locate UAVs, uses the one-fifth success rule [23]. This rule suggests that the step size σ should be increased if more than 20% of the new solutions are successful, meaning that the offspring is better than the parent. On the other hand, if less than 20% of the new solutions are successful, the step size σ should be decreased.

3 Upper confidence bound 1 algorithm and orthogonal arrays for tuning a metaheuristic

The Fig. 3 depicts the methodology. It can be seen that in step 1, a set of configurations is defined for use in the calibration of any metaheuristic, using the Taguchi methodology (steps 1 to 5, in [10]).

Fig. 3

Methodology to tune a metaheuristic utilizing orthogonal design and UCB1.

Before using this methodology, it is necessary to identify the parameters of interest to calibrate in the metaheuristic. Based on this, we define the different configurations to be evaluated. The parameter combinations form orthogonal arrays proposed by Taguchi; these are designs that possess the property of orthogonality, which is also found in factorial designs. These arrays are complete factorial, fractional, or mixed designs, depending on the number of factors under investigation in a particular case.

Taguchi named these arrays as follows: L_a (b) ^C, where: a represents the number of tests or experimental conditions to be carried out. In other words, the number of rows or lines in the array. b represents the different levels at which each factor will be taken. C is the number of independent effects that can be analyzed, meaning the number of columns in the array.

Once the arrays are obtained, each bandit will be associated with a parameter configuration, as shown in step 2 of Fig. 3. In this way, when the stopping condition is met in the UCB1 algorithm, we will have the configuration (see step 3 of Fig. 3) that is suitable for the problem under study (see step 4 of Fig. 3).

3.1 Upper confidence bound 1 algorithm approach for tuning the (1 + 1)-evolution strategy

Orthogonal design is an experimental design commonly used to test the effectiveness of different configurations on metaheuristic algorithms. Each configuration is made up of a set of parameters. The idea behind orthogonal design is to create a set of combinations, where each combination is unique and ensures that the effect of one experiment does not interfere with the effect of any other experiment. This means that each experiment is independent and produces reliable results. Table 2 illustrates the application of orthogonal design by presenting nine experimental designs that are used to test the effectiveness of nine different configurations. The parameters z and σ are the ones that most impact the evolution strategy according to prior simulation results.

Table 2
Taguchi orthogonal array L₉3⁽²⁾

L₉3⁽²⁾

Parameters

Bandits Step size

adaptation z Step size σ

Bandit 1 5 0.0001 (0.012 km)

Bandit 2 5 0.001 (0.12 km)

Bandit 3 5 0.004 (0.48 km)

Bandit 4 10 0.0001 (0.012 km)

Bandit 5 10 0.001 (0.12 km)

Bandit 6 10 0.004 (0.48 km)

Bandit 7 20 0.0001 (0.012 km)

Bandit 8 20 0.001 (0.12 km)

Bandit 9 20 0.004 (0.48 km)

L₉3⁽²⁾
	Parameters
Bandits	Step size
	adaptation z	Step size σ
Bandit 1	5	0.0001 (0.012 km)
Bandit 2	5	0.001 (0.12 km)
Bandit 3	5	0.004 (0.48 km)
Bandit 4	10	0.0001 (0.012 km)
Bandit 5	10	0.001 (0.12 km)
Bandit 6	10	0.004 (0.48 km)
Bandit 7	20	0.0001 (0.012 km)
Bandit 8	20	0.001 (0.12 km)
Bandit 9	20	0.004 (0.48 km)

The steps involved in typical RL algorithm for tuning the (1 + 1)-evolution strategy [14] are:

First, the agent interacts with the environment by performing an action that comprises the execution of the (1 + 1)-evolution strategy with one set of parameters (z and σ) from Table 2 to evaluate their effect on the algorithm performance.

The agent performs an action and moves from one state to another. An agent on the UCB1 algorithm has a single state, so this step does not apply.

The agent will receive a reward based on the action performed. In this research, the reward is the probability of a successful voice service.

Based on the reward the agent will understand whether the action was good or bad. In this research, our interest is in the trending of actions to select the most relevant orthogonal design, i.e., the best configuration (bandit) to tune the (1 + 1)-evolution strategy.

If the action was good, i.e., if the agent received a positive reward, the agent will prefer to use the successful orthogonal design (exploitation). Otherwise, the agent will try another orthogonal design to gain a positive reward (exploration).

The UCB1 algorithm’s stop condition is met when a bandit reaches a minimum of 34 calls. The winning arm will then determine the configuration for the (1 + 1)-evolution strategy and its associated problem.

4 Results

The simulations were executed on a desktop computer. The specifications of the computer are as follows: AMD Ryzen 2600 processor, (2 × 8 dual channel) 16 GB of Crucial Ballistix DDR4 memory running at 3000 MHz, a 480 GB Crucial brand SSD, and an 8 GB GDDR5 AMD Radeon Rx 570 video card from Gigabyte. All components are mounted on a B450 AORUS ELITE Motherboard. The operating system is 64-bit Windows 10. The code was developed in Python language.

As shown in Fig. 4, there is a clear breakdown of the total number of times that each bandit was selected. Bandit 1 [5, 0.0001] appears to have poor exploration capability, mainly because of its small step size σ. This small step size leads to a constant fluctuation due to the one-fifth success rule in the (1 + 1)-evolution strategy, resulting in a shaky movement. On the other hand, Bandit 5 [10, 0.001] has a longer review time but has better results due to its increased exploration capabilities when its step size increases.

Fig. 4

Final values for the accumulated number of bandit selections.

The performance of each configuration is shown in Fig. 5. It is noticeable that Bandit 5 maintains high values throughout the iterations, while Bandit 1 steadily obtains low values over time. It is possible to uncover where the UCB1 algorithm explores or exploits in the set of configurations. Bandits 5 and 6 are more often called, making an exploitation process while the rest of the configurations share similar values of calls along the iterations as shown in Fig. 6, here again, Bandit 1 has the least number of calls (nine).

Fig. 5

Exploration-exploitation of UCB1 on the set of experiments.

Fig. 6

Average probability of successful voice services Q(a) by iteration on bandits.

The box plots depicted in Fig. 7 collectively reveal five distinct sets of bandits with the most closely matched 3rd quartile values, after excluding outliers from their respective ranges. For a comprehensive data overview from Fig. 7, please refer to Table 3.

Fig. 7

Average probability of successful voice services per bandit.

Table 3

Summary of Experimental Designs on the (1 + 1)-evolution strategy performance

	Bandit 1	Bandit 2	Bandit 3	Bandit 4	Bandit 5	Bandit 6	Bandit 7	Bandit 8	Bandit 9
IQR:	0.0319	0.1203	0.0375	0.0458	0.0113	0.0441	0.1024	0.0583	0.0056
Min.:	0.0000	0.0000	0.6044	0.3766	0.6828	0.0000	0.5229	0.5156	0.6277
1st Qu.:	0.1773	0.3869	0.6543	0.4580	0.7015	0.6275	0.5282	0.5356	0.6330
3rd Qu.:	0.2092	0.5072	0.6918	0.5038	0.7128	0.6716	0.6306	0.5939	0.6386
Max.:	0.3138	0.5462	0.7540	0.5598	0.7528	0.6832	0.7503	0.7207	0.6953
Median:	0.1995	0.4926	0.6867	0.4856	0.7053	0.6561	0.5638	0.5687	0.6351
Mean:	0.1942	0.4012	0.6706	0.4759	0.7063	0.6045	0.5906	0.5729	0.6379

The first set has the Bandit 1, characterized by a minimal value of zero and a mean of 0.1942. Its 3rd quartile value, Q (a) =0.2092, signifies that 75% of the total data generated by the (1 + 1)-evolution strategy using this configuration falls below this threshold. The second set comprises Bandit 2 and Bandit 4, their mean is 0.4012 and 0.4759 respectively; here the 3rd quartile is 0.5072 and 0.5038 respectively, increasing substantially their value compared to the first set. In the third set, we find Bandit 7 and Bandit 8, their mean are 0.5638 and 0.5687, values higher than the previous set; having similar behavior when we compare the 3rd quartile of this set (0.6303 and 0.5939) with the second set. The fourth set includes Bandit 3, Bandit 6, and Bandit 9; their Q(a) value mean are 0.6867, 0.6045, and 0.6379, and their values in the 3er quartile are 0.6918, 0.6716, and 0.6386 respectively. Finally, the fifth set has the best bandit configuration with the 50% of total data produced by the (1 + 1)-evolution strategy falling below 0.7128 and up to 0.7015; while their maximal value is 0.7528. This configuration gives a Q(a*) = 0.7063.

The time used by the UCB1 algorithm for the exploration-exploitation process is irrelevant compared with the time used by the (1 + 1)-evolution strategy algorithm. Figure 8 shows the time consumed by the (1 + 1)-evolution strategy on executing 500 iterations and sending a Q value related to the i - th bandit.

Fig. 8

Time on seconds to provide a solution by the (1 + 1)-evolution strategy.

The most time-consuming configuration was the Bandit 5, this Bandit was selected 34 times, finishing with a Q(a) of 0.7063 as the best configuration suggested by the UCB1 algorithm. In contrast, the most effective configuration considered time-consuming was the configuration on Bandit 9, this bandit was selected 27 times concluding with a Q(a) of 0.6351 as the fourth best. Since we aim to maximize the probability of successful voice services, we take the configuration suggested by Bandit 5.

5 Conclusion and future work

Once we finish analyzing the experimental results, we gather the following conclusions: -

Using statistical techniques to tune metaheuristics can be advantageous. However, this task is challenging if a human expert performs it. To address this challenge, the UCB1 algorithm and the DOE technique provide an automated approach to tune metaheuristics.

Utilizing the UCB1 algorithm’s exploitation-exploration process for early stopping when a trending configuration emerges, can significantly reduce the time and resources required for algorithm tuning.

We plan to find a suitable termination condition to stop UCB1 for future work. Another future direction includes applying our proposed meta-optimizer to tune toy optimization problems. Finally, we plan to apply Thompson sampling as another approach to tune a metaheuristic.

Footnotes

Acknowledgments

We thank Daniel Sánchez-Bautista for coding the UCB1 algorithm. Gabriela L. Rodríguez-Cortés gratefully acknowledges the scholarship from CONAHCyT to pursue her graduate studies. M.A. Cosío-León and Anabel Martínez-Vargas extend their thanks to the Centro Nacional de Supercómputo at IPICYT for providing essential insights into the UCB1 algorithm through the Diploma in Applied Artificial Intelligence.

References

Belarmino Adenso-Díaz Manuel Laguna , Fine-tuning ofalgorithms using fractional experimental designs and local search, Operations Research54(1) (2006), 99–114.

Peter Auer Nicolò Cesa-Bianchi Paul Fischer , Finite-timeanalysis of the multiarmed bandit problem, Machine Learning47(2) (2002), 235–256.

Danny Dijkzeul Nielis Brouwer Iris Pijning Levi Koppenhol Daan van den Berg , Painting with evolutionary algorithms. In Tiago Martins, Nereida Rodryguez-Fernandez and Sergio M. Rebelo, editors, Artificial Intelligence in Music, Sound, Art and Design, pages 52–67, Cham, 2022. Springer International Publishing.

Onas Mockus , et al., Bayesian Heuristic approach to discrete and global optimization: Algorithms, visualization, software, and applications, Springer Science & Business Media, 2013.

Yuin Wu Genichi Taguchi Subir Chowdhury Taguchi’s Quality Engineering Handbook. Wiley, 2004.

Changwu Huang Yuanxiang Li Xin Yao , A survey of automaticparameter tuning methods for metaheuristics, IEEE Transactionson Evolutionary Computation24(2) (2020), 201–216.

Frank Hutter Holger Hoos

Kevin Leyton-Brown Sequential model-based optimization for general algorithm configuration. In Carlos A. Coello Coello, editor, Learning and Intelligent Optimization, pages 507–523, Berlin, Heidelberg, 2011. Springer Berlin Heidelberg.

Frank Hutter Holger Hoos

Kevin Leyton-Brown Thomas Stützle , Paramils: An automatic algorithm configurationframework, J. Artif. Int. Res.36(1) (2009), 267–306.

Maryam Karimi-Mamaghan Mehrdad Mohammadi Patrick Meyer AmirMohammad Karimi-Mamaghan El-Ghazali Talbi , Machine learning atthe service of meta-heuristics for solving combinatorialoptimization problems: A state-of-theart, European Journal ofOperational Research296(2) (2022), 393–422.

10.

Misael Lopez-Sanchez Cosío-León

M.A.

Anabel Martínez-Vargas Comparative Analysis of Constraint Handling Techniques Based on Taguchi Design of Experiments, pages 285–315. Springer Singapore, Singapore, 2021.

11.

Anabel Martínez-Vargas Jesús Gómez-Avilés

Cosío-León

M.A.

Ángel Andrade

, Explaining the walkingthrough of a team of algorithms, Computer56(2) (2023), 67–81.

12.

Mercer

R.E.

Sampson

J.R.

, Adaptive search using a reproductivemeta-plan, Kybernetes7(3) (1978), 215–228.

13.

Elizabeth Montero María-Cristina Riff NicolásRojas-Morales , Tuners review: How crucial are set-up values to findeffective parameter values?Engineering Applications ofArtificial Intelligence76 (2018), 108–118.

14.

Ravichandiran

Hands-On Reinforcement Learning with Python: Master Reinforcement and Deep Reinforcement Learning Using OpenAI Gym and TensorFlow. Packt Publishing, 2018.

15.

Andrew Barto

Richard Sutton

Reinforcement Learning: An Introduction, Bradford Books, 2018.

16.

Gabriela Rodríguez-Cortés

Anabel Martínez-Vargas Oscar Montiel-Ross

Cosío-León

M.A.

Daniela Martínez

Deployment of unmanned aerial vehicles for maximum coverage in emergency scenarios using the (1 + 1) evolution strategy with one-fifth success rule. In 2021 Mexican International Conference on Computer Science (ENC), pages 1–7, 2021.

17.

Bobak Shahriari Kevin Swersky Ziyu Wang Ryan Adams

Nandode Freitas , Taking the human out of the loop: A review of bayesianoptimization, Proceedings of the IEEE104(1) (2016), 148–175.

18.

Selmar Kagiso Smit Parameter Tuning and Scientific Testing in Evolutionary Algorithms. PhD thesis, Vrije Universiteit Amsterdam, 2012.

19.

Mahnaz Sohrabi Mostafa Zandieh Mohammad Shokouhifar , Sustainable inventory management in blood banks considering healthequity using a combined metaheuristic-based robust fuzzy stochasticprogramming, Socio-Economic Planning Sciences86 (2023), 101462–.

20.

Stützle Manuel López-Ibáñez , Automated Design of Metaheuristic Algorithms, pages 541–579. Springer International Publishing, Cham, 2019.

21.

El-Ghazali Talbi , Machine learning into metaheuristics: A survey and taxonomy, ACM Comput. Surv.54(6) (2021).

22.

Maritzol Tenemaza Sergio Luján-Mora Angélica De Antonio Jaime Ramírez , Improving itinerary recommendations fortourists through metaheuristic algorithms: An optimization proposal, IEEE Access8 (2020), 79003–79023.

23.

Peter Krause Bäck Christophe Foussette Contemporary Evolution Strategies. Springer Berlin, Heidelberg, 2013.

24.

Wolpert

D.H.

Macready

W.G.

, No free lunch theorems foroptimization, IEEE Transactions on Evolutionary Computation1(1) (1997), 67–82.

25.

Zhu

Z.Q.

Dawei Liang Kan Liu , Online parameter estimation forpermanent magnet synchronous machines: An overview, IEEEAccess9 (2021), 59059–59084.