A convergence-diversity balanced fitness evaluation mechanism for decomposition- based many-objective optimization algorithm

Abstract

Many-objective optimization problems (MaOPs) have great challenges for traditional Pareto-based multi-objective evolutionary algorithms (MOEAs). One of the main reasons is that Pareto dominance cannot effectively rank/evaluate the solutions, resulting in almost random selection of the new population and no quality guarantee of the solutions in the new population. To overcome this shortcoming, in this paper, a new fitness evaluation mechanism is proposed for MaOPs. It uses a tailor-made integration of convergence and diversity measures, and can well balance convergence and diversity in solution evaluation. Specifically, to cooperate it with the subsequent selection mechanism, a new global convergence strength-based convergence measure is designed to avoid the interference from solution’s local information. To overcome the shortcoming that the existing angle-based diversity measure cannot distinguish the contributions to diversity of two solutions which have the same angle with reference direction, a new angle-distance based diversity measure is designed, which can accurately evaluate the contribution of each solution to diversity. Furthermore, to well balance the convergence and diversity, the proposed convergence measure and diversity measure have the same order of magnitude. As a result, the proposed convergence-diversity balanced fitness evaluation mechanism can accurately evaluate each solution’s quality and then properly increase the selection pressure. The proposed method is compared with five state-of-the-art many-objective evolutionary algorithms by experiments on 120 test instances of 24 benchmark problems with up to 20 objectives and a real world problem. The results indicate the competitiveness and effectiveness of our proposed algorithm in keeping a good balance between convergence and diversity.

Keywords

Many-objective optimization reference direction fitness evaluation mechanism convergence diversity

1. Introduction

Many real-world industrial problems [1, 3, 20, 26, 53, 61] and engineering applications [19, 45, 46, 63, 70] involve optimization problems, some of which often contain more than one conflicting objectives, such as real estate property maintenance [54], mountain railway alignment optimization [33], environmentally sustainable road network design [51, 60, 66] and shape generation of grid structures [2, 28, 29, 30, 47, 52], etc. These optimization problems are called multi-objective optimization problems (MOPs) and they can be mathematically formulated as follows:

$\displaystyle\textit{Min}F(x)=[{f_{1}}(x),{f_{2}}(x),\ldots,{f_{m}}(x)]^{T}$ (1) $\displaystyle s.t.x\in\Omega\subseteq{R^{n}}$

where $\Omega=\prod^{n}_{i=1}[a_{i},b_{i}]\subseteq R^{n}$ is the $n$ -dimensional decision space, $x={({x_{1}},{x_{2}},\ldots,{x_{n}})^{T}}\in\Omega$ represents each candidate solution in the decision space, and $F:\Omega\rightarrow R^{m}$ is the objective function vector which comprises $m$ conflicting objectives, where $f_{i}(x)$ $(i=1,2,\ldots,m)$ is the ith objective to be minimized. The purpose of MOPs is to find a set of best compromise solutions among the $m$ conflicting objectives.

In the past 20 years, much effort has been devoted to developing evolutionary multiobjective optimization (EMO) algorithms for solving the optimization problems with two and three objectives [50, 57, 58, 69]. However, many real-world applications, such as water resource management [25, 27], crash-worthiness design of vehicles [34, 40] and car cab design [12], often involve four or more conflicting objectives and also known as many-objective optimization (MaOPs). For MaOPs, with the number of objectives increasing, the proportion of Pareto non-dominated solutions among the population is growing rapidly, which inevitably leads to lots of difficulties in the evaluation of solutions. Some traditional Pareto-based multi-objective evolutionary algorithms (MOEAs) [39, 56], such as nondominated sorting genetic algorithm-II (NSGA-II) [14], strength Pareto evolutionary algorithm 2 (SPEA2) [71] and so on, which have shown strong ability for solving MOPs, become ineffective when solving MaOPs. The main reason is that with the increasing number of objectives, the selection pressure of these MOEAs based on Pareto dominance is seriously weakened, leading to the solution selection of the new population becoming almost randomly, which hinders the convergence of MOEAs to the Pareto optimal front seriously.

To improve the ability of MOEAs for solving MaOPs, various convergence enhancement strategies have been proposed for dealing with MaOPs. These strategies can be roughly grouped as three types.

The first type of strategies is by modifying the definition of the traditional Pareto-dominance to enhance the selection pressure toward the PF. $\epsilon$ -dominance [21], average and maximum ranking relations [5], $\alpha$ - dominance [23], $C E$ -dominance [48], and $C N$ - dominance [7] are typical examples. Take $\alpha$ - dominance as an example, which modifies the Pareto dominance as follows: a solution $x$ is defined to $\alpha$ -dominate another solution $y$ if $x$ is slightly inferior to $y$ in one objective while largely superior to $y$ in some other objectives. The new definition of dominance evidently expands the dominated area of each solution, thus enhances the selection pressure toward the PF. However, the main shortcoming of these modified dominance methods is that some of them only emphasize the solutions residing in the middle part of the PF, while others emphasize the solutions which only have good performance on some objectives. This may lead to a poor overall performance [17].

The second type mainly refers to the strategies that combine an additional convergence criterion with the Pareto dominance to enhance the convergence, such as the recently proposed knee point-driven EA (KnEA) [68], which uses the definition of “knee point” as a secondary selection basis to enhance the convergence pressure.

The third type adopts the performance metrics as selection criterions to improve convergence. Hypervolume [43] is a widely used metric. It is well-known that the computational complexity for computing hypervolume grows exponentially with the increasing number of objectives. To avoid the high computational complexity, a fast hypervolume-based many-objective optimization algorithm (HypE) [4] and novel hypervolume driven selection mechanism (HAGA) [44] are proposed for MaOPs. However, these methods used approximate hypervolume values, and their performance is also decreased.

Except for the strategies to enhance the convergence of solutions, the strategies to maintain the diversity of obtained solutions are also very important factors to improve the ability of MOEAs. Less diversity would cause the algorithm to easily trap into local optimal solutions or to get a worse distribution of solutions.

To improve the diversity, a variety of diversity enhancement strategies have been proposed for MaOPs. One of the most powerful strategies is the decom- position-based method, which uses a set of uniformly distributed weight vectors to decompose the whole objective space into a number of subspaces, and then the search is made on these subspaces respectively so as to make the searched area as wide as possible. MOEA based on decomposition (MOEA/D) [67], is a representative of this sort of algorithms. It decomposes a MaOP into several optimization subproblems by a set of predefined evenly distributed weight vectors, and then optimizes these subproblems simultaneously by solving single objective optimization problems. By using the idea of MOEA/D, some variants have been designed to solve MaOPs, such as MOEA/D-DE [31], which uses a differential evolution (DE) [41] operator and a polynomial mutation operator [11] in MOEA/D. dMOPSO [38], which uses the decomposition-based approach to update the position of each particle and designs a memory re-initialization process to provide diversity to the swarm, and MOEA/D-M2M [36], which decomposes an MaOP into a number of simple multi-objective subproblems and solves these subproblems in a collaborative way by reducing the computational effort at each generation. In addition to the above methods, recently, a new kind of strategies [6, 40] appears in many-objective optimization, which aims to provide a new method of evaluating solutions based on a set of well-distributed reference points. The first work of this type is the nondominated sorting genetic algorithm III (NSGA-III) [13], where a niche-preservation operation based on a set of well-distributed reference points is designed to replace the crowding distance used in NSGA-II [14]. Since NSGA-III prefers to the non-dominated solutions close to the reference vector, it emphasizes diversity more than convergence when the Pareto dominance lacks enough selection pressure. To improve the convergence of NSGA-III in MaOPs, $\theta$ -dominance-based evolutionary algorithm ( $\theta$ -DEA) [65] and RP-dominance-based NSGA-II (RPD-NSGA-II) [17] design new dominance methods based on the well-distributed reference points to balance the convergence and diversity. MOEA/DD [32], by combining the dominance and decomposition, facilitates the local density estimation and improves the selection procedure of MOEA/D. Besides, based on the framework of NSGA-III, a vector angle-based evolutionary algorithm (VaEA) [62] is proposed, where a maximum-vector-angle-first principle is used to select solutions and can keep a good balance between the convergence and diversity. The performance of VaEA is better than NSGA-III and MOEA/DD for many test problems [62].

Recently, a strength Pareto evolutionary algorithm based on reference direction (SPEA/R) [24] is proposed for multi-objective and many-objective optimization, which is a substantial extension of strength Pareto evolutionary algorithm (SPEA) [72] and an improved version of SPEA2 [71]. In SPEA, a fitness assignment mechanism is first defined by how many individuals a solution dominates and is dominated by. This fitness assignment can better measure each solution’s convergence strength. However, SPEA does not take the density estimation into account, i.e., when many solutions have the same convergence strength, none or very little information can be obtained to distinguish them. To tackle this issue, SPEA2 incorporates the nearest neighbor density estimation into the fitness assignment of SPEA to discriminate between individuals. Then the improved fitness assignment can better distinguish individuals with identical convergence strength and well evaluate each solution. It performs well for optimization problems with two or three objectives. However, with increasing of the number of objectives, the objective space becomes very large and the population members are likely to be widely separated. Thus the nearest neighbor-based density estimation method becomes inefficient in MaOPs. SPEA/R [24] inherits the advantage of fitness assignment in SPEA2, but replaces the nearest neighbor density estimation by a reference direction-based one. In SPEA/R, a new fitness assignment scheme based on the decomposed objective space is designed to directly evaluate each solution, where both Pareto dominance and diversity in each decomposed subspace are considered, and a diversity-first-and-convergence-second selection strategy is proposed based on the decomposed subspaces. SPEA/R has been demonstrated efficient on various MaOPs.

However, there are also some shortages in SPEA/R, the first is that the fitness evaluation scheme in SPEA/R cannot fit the environmental selection strategy well, which will hinder the performance of SPEA/R to solve more challenging MaOPs. The second is that the diversity measure in the fitness assignment of SPEA/R cannot accurately evaluate the contributions to the diversity of such solutions with the same angle to a reference vector. The third is that the scales of convergence measure and diversity measure in SPEA/R may have a big difference, which cannot efficiently balance the convergence and diversity. Since an effective fitness evaluation mechanism plays a key role for evolutionary algorithms, therefore, it is important to require a further study on this issue, and more factors should be taken into account to make the fitness evaluation mechanism more accurately to quantify each solution’s diversity and convergence contributions for MaOPs.

In this paper, to overcome these shortcomings of fitness assignment scheme in the recent proposed algorithm SPEA/R [24], we design a new fitness evaluation mechanism for MaOPs, which takes both convergence and diversity into account, and more importantly, we design a tailor-made method to integrate convergence measure and diversity measure subtly, so that the convergence measure can better fit the selection scheme and the diversity measure can accurately measure each solution’ contribution to the diversity. Based on these, an improved SPEA/R is proposed and named as ISPEA/R. Experiments are conducted on a widely used benchmark suite and the proposed algorithm is compared with five state-of-art algorithms. The experimental results have shown the good performance of our proposed algorithm in keeping a good balance between convergence and diversity. In addition, the efficacy of the proposed method on a realistic water resource management problem is showcased.

The remainder of this paper is organized as follows. In Section 2, the proposed algorithm ISPEA/R and the new fitness evaluation mechanism are described in detail. Experimental design and the results are presented and analyzed in Section 3. In addition, the efficacy of ISPEA/R on a realistic water management problem is demonstrated in Section 4. Finally, the conclusions are given in Section 5.

2. Proposed algorithm: ISPEA/R

2.1 General framework

[b] Basic Framework of ISPEA/R Input: the maximal number of fitness evaluations (MFEs) Output: approximated Pareto-optimal front $P_{t+1}$ [1] Generate a diverse reference direction set $W$ ; Create an initial Population $P_{0}$ ; Set $t=0$ ; stopping criterion is not met Generate the offspring set $\bar{P}_{t}$ by applying genetic operators on $P_{t}$ ; Set $Q_{t}=P_{t}\cup\bar{P}_{t}$ ; Normalize objective population $Q_{t}$ to get $\bar{Q}_{t}$ ; each reference direction $\lambda_{i}\in W$ Find subspace $E(i)$ of solutions in $\bar{Q}_{t}$ associating with $\lambda_{i}$ ; Calculate fitness value of each solution $x$ in $E(i)$ : $FV(x)=$ Fitness_Assignment ( $x$ ); Select next generation population $P_{t+1}$ by using $E(i)$ and $W$ ; $t=t+1$ ;

The proposed ISPEA/R shares a common framework that is employed by SPEA/R [24]. The pseudo code of ISPEA/R is shown in Algorithm 1. Firstly, to uniformly decompose the whole objective space into a number of non-intersection subspaces, a set of reference directions are generated by Das and Dennis’s [8] systematic approach (when $H\geqslant m$ , two-layered reference vectors with small values of $H$ as suggested in [13]). Then based on the predefined number of reference directions, an initial population is randomly produced. Steps 4–14 are iterated until the termination criterion is satisfied. In step 5, the offspring population $\bar{P}_{t}$ is produced by applying the widely used genetic operators (the simulated binary crossover (SBX) [10] and the polynomial mutation [11]) on the parent population $P_{t}$ . Then the offspring population is combined with the parent population to form a new population. In step 7, an objective normalization strategy [24] is first performed on the combined population in objective space to normalize the disparately scaled objectives. Then, the normalized population is associated with the predefined set of reference directions. In this way, the solutions in the combined population are allocated to different subspaces. In each subspace, a new proposed fitness evaluation mechanism is used to evaluate the quality of the internal solutions, which is different from the fitness assignment scheme of SPEA/R and is the key technique in ISPEA/R, and will be introduced later. Once the fitness assignment of all subspaces are finished, the environmental selection strategy [24] based on the principle of the diversity-first and convergence-second is employed to construct a new parent population for the next generation.

In the following subsection, considering that the fitness evaluation mechanism in ISPEA/R is based on the decomposed objective space and is derived by member association operation, we will give a brief introduction to the member association operation first, and then describe the implementation detail of the fitness evaluation mechanism in ISPEA/R.

2.2 Member association

To decompose the whole objective space into the predefined number of subspaces, we need to associate the normalized population in the objective space with the predefined set of reference directions. To be specific, suppose that $N$ reference directions constitute a set $W$ of reference directions. For each reference direction $\lambda_{i}\in W$ , a subspace $E(i)$ in the objective space is defined as follows:

$\displaystyle E(i)=\{\bar{F}(x)\in\bar{Q}|\langle\bar{F}(x),\lambda_{i}\rangle% \leqslant\langle\bar{F}(x),\lambda_{j}\rangle\}$ (2)

where $x\in\Omega$ , $\lambda_{j}\in W$ . $\bar{F}(x)$ is the normalized objective function vector of $x$ . $\langle\bar{F}(x),\lambda_{i}\rangle$ denotes the acute angle between vector $\bar{F}(x)$ and $\lambda_{i}$ .

Obviously, all those solutions which have the smallest angle to the reference direction $\lambda_{i}$ are associated with subspace $E(i)$ . In this way, the whole objective space will be divided into $N$ subspaces and each solution will be assigned to (associated with) a corresponding subspace.

2.3 Proposed new fitness evaluation mechanism

In this section, we will first give a brief introduction to the fitness assignment of the algorithm SPEA/R [24], which is the basis of our proposed new fitness evaluation mechanism. Then, the motivations for designing our new fitness evaluation mechanism to overcome some shortages of the fitness evaluation mechanism in SPEA/R will be described. Finally, the detail of the proposed new fitness evaluation mechanism will be given.

2.3.1 Fitness assignment in SPEA/R

When the whole objective space is divided into $N$ subspaces, SPEA/R assigns a fitness value for each solution in each subspace to evaluate its quality. The detailed idea is as follows.

For each solution in subspace $E(i)$ , SPEA/R first assigns it a “local” strength value $S(a)$ by:

$\displaystyle S(a)=C(\{b|a\prec b,\>b\in E(i)\})$ (3)

where $C(\cdot)$ denotes the cardinality of a set, i.e., $S(a)$ represents the number of solutions in $E(i)$ dominated by $a$ . Then based on the local strength value $S(a)$ , its local raw fitness $R(a)$ is calculated as follows:

$\displaystyle R(a)=\sum\limits_{c\in E(i),c\prec a}S(c)$ (4)

where the local raw fitness of solution $a$ is the sum of the local strength values of its dominators in the same subspace. The smaller the value of $R(a)$ , the better the convergence of solution $a$ in the subspace.

Considering the situation that once all solutions in a subspace are non-dominated by each other, then their local raw fitness values will be zero. In this situation, these solutions cannot be distinguished by their local raw fitness values. SPEA/R takes another information, solutions’ acute angle values, into account. Specifically, for each non-dominated solution $a\in E(i)$ , the acute angle value between $\bar{F}(a)$ and the associated reference direction $\lambda_{i}$ , denoted as $\theta_{a}$ , is used to compute the density $D(a)$ of the solution $a$ by

$\displaystyle D(a)=\frac{\theta_{a}}{\theta_{a}+\theta_{m}}$ (5)

where $\theta_{m}=\max\limits_{1\leqslant i\leqslant N}\min\limits_{j\neq i}(\lambda_% {i},\lambda_{j})$ , i.e., the largest acute angle between two neighboring reference directions is used to ensure that the density $D(a)$ is smaller than one. Thus, the local fitness value of individual $a$ , named as $FV_{l}(a)$ , is defined by:

$\displaystyle FV_{l}(a)=R(a)+D(a)$ (6)

SPEA/R realizes that just using the local fitness value to evaluate each solution may impair global convergence if all solutions in one subspace are dominated by solutions in other subspaces. To avoid this situation, SPEA/R takes each solution’s global raw fitness, denotes as $FV_{g}(a)$ , into account. The detail process of defining $FV_{g}(a)$ is as follows.

For a solution $a\in E(i)$ , its “global” strength value $G(a)$ is first calculated by

$\displaystyle G(a)=C(\{d|a\prec d\})$ (7)

where $d$ represents the solution in $\bar{Q}_{t}\subset R^{m}$ and $\bar{Q}_{t}$ is defined in Algorithm 1. So $G(a)$ is the number of solutions Pareto dominated by $a$ in the whole solution set in the objective space. The larger the value $G(a)$ , the better the convergence of solution $a$ .

Based on the global strength value $G(a)$ of each solution $a$ in $E(i)$ , its global raw fitness $FV_{g}(a)$ is defined as follows:

$\displaystyle FV_{g}(a)=\sum\limits_{e\in Q_{t},e\prec a}G(e)$ (8)

where the global raw fitness of solution $a$ is adopted from SPEA2 [71], and is the sum of the global strength values of its dominators in the whole objective space. Here the global raw fitness of each solution is to be minimized.

Then, the final fitness of solution $a$ , denoted as $FV(a)$ , is defined by

$\displaystyle FV(a)=\left\{\begin{array}[]{l}FV_{l}(a)\\ FV_{l}(a)+FV_{g}(a)\end{array}\right.=\left\{\begin{array}[]{l}R(a)+D(a),\text% {if}|E(i)|=1\\ R(a)+D(a)+FV_{g}(a),\text{otherwise}.\end{array}\right.$ (9)

where the smaller the value of $FV(a)$ , the better the quality of solution $a$ .

2.3.2 Motivations for our new evaluation mechanism

Generally, different fitness evaluation mechanisms should fit different strategies of environmental selection. However, in SPEA/R, the fitness evaluation scheme cannot fit the environmental selection strategy well, which will hinder the performance of SPEA/R for more challenging MaOPs. In fact, note that the environmental selection strategy in SPEA/R, which selects the best solution from each subspace in turn, is a very reasonable way to select solutions, especially for MaOPs, since it not only maintains the diversity of obtained new population, but also takes into account the quality of each solution. However, the fitness assessment mechanism in SPEA/R has the following problems.

The first is that the environmental selection has used the local information, but the fitness assessment redundantly used the local information, which are unnecessary. In fact, in SPEA/R [24], the environmental selection scheme selects the best solution of each subspace into the population, which has considered the local information of solutions. While the fitness assignment also uses local information of local strength in convergence measure. Also, this redundant use of local information can increase the computation amount. Furthermore, a strong “local” strength solution may interfere a good “global” strength solution and result in the good “global” strength solution being missed.

The second is that the scales of convergence measure and diversity measure in the fitness assignment of SPEA/R may be in the different order of magnitude, i.e., they may have a big difference. In fact, in Eq. (9), the value of $D(a)$ is in the interval [0, 1) while the values of $R(a)$ and $FV_{g}(a)$ are both larger than 1, or even more larger, which may lead to an imbalance between convergence measure and diversity measure and finally cause unreasonable evaluation for solutions.

The third is that although the angle-based diversity evaluation in the fitness assignment of SPEA/R or some others (such as NSGAIII [13], VaEA [62]) can accurately evaluate the diversity in most cases, it cannot well evaluate the diversity in one case. Figure 1 gives a simple example to illustrate this case. The ith subspace $E(i)$ in the objective space associated by $\lambda_{i}$ contains solutions $b, c, d$ , where $b, c, d$ have the same angle $\theta$ to the reference direction $\lambda_{i}$ . In this situation, the angle-based diversity evaluation cannot distinguish the contributions of $b, c, d$ to the diversity. They are seen as of the same contribution to the diversity. But, their contributions to the diversity are obvious different. As shown in Fig. 1, the perpendicular distance of $b$ to the reference direction $\lambda_{i}$ is smaller than those of $c$ and $d$ . The contribution of $b$ to the diversity is greater than those of $c$ and $d$ . Thus, if some solutions have the same angle to a reference direction $\lambda_{i}$ , we evaluate the contributions of them to the diversity by their perpendicular distances. The smaller the perpendicular distance, the more the contribution to the diversity.

Figure 1.

Illustration of angle-based diversity evaluation.

Based on the above observations, we propose a new fitness evaluation mechanism to more accurately quantify the quality of each solution.

2.3.3 The details of the new fitness evaluation mechanism

The proposed fitness evaluation mechanism is based on the decomposed objective sub-spaces, the details are as follows:

Case 1. When a subspace contains only one solution, no matter whether it is dominated or not by solutions in other subspaces, we will directly allow it to enter the next iteration since the survival of this solution in the subspace is very promising for the unexploited area. The fitness value of this solution is set to 0 to protect it in the environment selection. Case 2. When a subspace contains more than one solutions, we shall define the diversity measure and convergence measure respectively in the following:

Figure 2.

Illustration of the perpendicular distances $d^{i}(x)$ .

Diversity measure. For the diversity measure of each solution, as discussed in Section 2.3.2, the popular angle-based diversity evaluation cannot distinguish the diversity contributions of such solutions which are in the same subspace and have the same angle to a reference direction ${\lambda_{i}}$ . In this situation, we evaluate the diversity contributions of these solutions by their perpendicular distances. The smaller the perpendicular distance, the bigger the contribution to the diversity. To be specific, for the normalized objective function $\bar{F}(x)={({\bar{f}_{1}}(x),{\bar{f}_{2}}(x),...,{\bar{f}_{m}}(x))^{T}}$ of solution $x$ which is one of the solutions having the same value of acute angle to $\lambda_{i}$ , suppose the ideal point is taken as the origin and $\theta_{x}$ is the acute angle between vector $\bar{F}(x)$ and $\lambda_{i}$ . Then the perpendicular Euclidean distance between $\bar{F}(x)$ and ${\lambda_{i}}$ , denoted as $d^{i}(x)$ , can be calculated as follows [64] and intuitively demonstrated by Fig. 2:

$\displaystyle d^{i}(x)=\|\bar{F}(x)\|\sin\theta_{x}$ (10)

Then, an angle-distance-based diversity measure of solution $x$ in subspace $E(i)$ , denotes $Dm(x)$ , is defined by

$\displaystyle Dm(x)=\left\{\begin{array}[]{ll}D(x),&\text{if }\theta_{x}\text{% is different}\\ &\text{from }\theta_{y}\text{ of any}\\ &\text{other }y\text{ in }E(i);\\ d^{i}(x),&\text{otherwise}.\\ \end{array}\right.$ (11)

where $D(x)$ is the normalized acute angle value of $\theta_{x}$ defined by Eq. (5). Besides, since the value $d^{i}(x)$ is computed by the normalized objective function $\bar{F}(x)$ , it is still smaller than one. Thus, the ranges of the density $Dm(x)$ is in [0, 1] and the smaller value of $Dm(x)$ means more contribution of $x$ to the diversity.

II)

Convergence measure. Considering that a diversity-first-and-convergence-second selec- tion strategy [24] is used in the environmental selection, i.e., the best solution in each subspace will be selected in turn, thus, for each solution in the subspace, its local convergence ability should be avoided in the convergence measure to prevent missing the solutions with good global convergence ability. For a solution $x$ in the subspace, we only use its “global” strength value $G(x)$ to measure its convergence ability, where the “global” strength value of solution $x$ , representing the number of solutions Pareto dominated by $x$ in the whole solution set in the objective space, is calculated by Eq. (7). The larger the value of $G(x)$ , the better the convergence of solution $x$ . As a part of fitness evaluation mechanism, the convergence measure of solution $x$ is defined as

$\displaystyle{\bar{G}}(x)=\cos\left(\frac{G(x)}{Gm}\right)$ (12)

where $Gm=\max\limits_{x\in Q_{t}}\{G(x)\}$ , i.e., the maximum “global” strength value among the combined population $Q_{t}$ . It is used to normalize the “global” strength, i.e., $\frac{G(x)}{Gm}\in$ [0, 1]. The effect of cos function is to keep the scale of convergence measure consistent with that of the diversity measure. Based on the new defined convergence measure and diversity measure of each solution in subspace $E(i)$ , we propose a new fitness assignment scheme as follows:

$\displaystyle FV(x)\!=\!\left\{\begin{array}[]{l}\!0,\text{if}|E(i)|=1\\ \!\bar{G}(x)\!+\!Dm(x),\text{otherwise}.\end{array}\right.$ (13)

where ${\bar{G}}(x)$ is the convergence measure of solution $x$ . Note that the smaller the first item ${\bar{G}}(x)$ in Eq. (13) is, the stronger the convergence of the solution $x$ will be. Also, the smaller the second item $Dm(x)$ in Eq. (13) is, the bigger the contribution of solution $x$ to diversity will be. The new fitness evaluation mechanism not only takes both convergence measure and diversity measure into account, but also keeps the scales of convergence measure and diversity measure of each solution to be of the same order of magnitude. Thus it can efficiently balance the convergence and diversity. The smaller the fitness value of a solution is, the better the convergence and the contribution to diversity it will have.

2.4 Discussion

This section further illustrates similarities and differences between ISPEA/R and SPEA/R.

2.4.1 Similarity

In both algorithms, a set of reference vectors is employed to decompose the whole objective space and guide the selection procedure. Each solution in the objective space is associated with a reference vector. Besides, a diversity-first-and-convergence-second selection strategy is used in both algorithms.

2.4.2 Differences

The fitness evaluation mechanism in both ISPEA/R and SPEA/R takes the situation ( $|E(i)|=1$ ) into account, leading to the similar formulas of the fitness functions, i.e., both are defined in two cases. However, the principles and details of these two fitness assessment mechanisms are different in each case:

a) 1.
In the case $|E(i)|=1$ , i.e., there is only one solution in the subspace, fitness function in SPEA/R (i.e., Eq. (9)) still calculates this solution’s fitness value and compares with the solutions in other subspaces of $|E(i)|>1$ . It only removes the calculation of the global raw fitness $FV_{g}(x)$ , but its fitness value is still positive, which cannot guarantee this unique solution to be kept. In our new fitness evaluation mechanism (i.e., Eq. (13)), we set the fitness value of the unique solution to 0 directly, which not only saves the computations, but also ensures the unique solution to be preserved. The survival of this unique solution in the subspace is very promising for the unexploited area.
2.
In the case $|E(i)|>1$ , although these two fitness evaluation mechanisms take convergence and diversity into account, the ways of defining convergence measure and diversity measure are different, and the way of integrating convergence measure and diversity measure is also different. For convergence measure in SPEA/R, each solution’ local strength and global strength are both calculated, as we have mentioned in Section 2.3.2, using solution’ local strength not only increases the computation amount, but also may miss the really good “global” strength solution in the selection process. While in our new fitness evaluation mechanism, to void missing the good solution, we just use each solution’s global strength to measure the convergence. In SPEA/R, the popular angle-based diversity measure is used to measure the contribution of each solution to diversity, but it cannot distinguish the contributions of such solutions with same angle to the reference direction. In our new fitness evaluation mechanism, the angle-distance-based diversity measure is used, which can distinguish the contributions of solutions with same angle to the reference direction.

2.5 Computational complexity of ISPEA/R and SPEA/R

Since the main framework of ISPEA/R is the same as that of SPEA/R, and the only difference between them is the process of fitness assignment. Here we just analyze the computational complexity of the fitness assignment in these two algorithms from two aspects:

1) 1.
For the convergence measure, SPEA/R first calculates the “global” strength value and “local” strength value of all solutions in the combined population, the computation amount for computing the “global” strength value is $O(4mN^{2})$ , where $m$ is the number of objectives and $2N$ is the combined population size. For computing the “local” strength value, in the worst case, that is, all $2N$ solutions get trapped into one subspace and other subspaces are all empty, the computation amount is $O(4mN^{2})$ . Then the global raw fitness and local raw fitness of all solutions are computed based on their “global” strength value and “local” strength value, respectively. For computing the global raw fitness of all solutions, in the case that the first solution is dominated by $2N-1$ solutions, the second solution is dominated by $2N-2$ solutions, similarly, the $(2N-1)$ -th solution is dominated by one solution and the $2N$ -th solution is dominated by non solution. The computation complexity will reach $O(2(2N-1)N^{2})$ . The calculation of local raw fitness will also cost $O(2(2N-1)N^{2})$ in the worst case. Thus, the computation complexity of SPEA/R is $O(4mN^{2})+O(4mN^{2})+O(2(2N-1)N^{2})+O(2(2N-1)N^{2})+O(2N)=O(N^{3}+mN^{2})$ .

In ISPEA/R, only the “global” strength value is used to measure the convergence of solutions that are in the subspace where $|E(i)|>1$ . Here we consider the worst situation, that is, there is no subspace which contains only one solution, then all solutions’ “global” strength value $G(x)$ need to be calculated, the computational amount is $O(4mN^{2})$ . Then complexity of computing the maximum “global” strength value $G m$ , $\frac{G(x)}{Gm}$ and $\cos(\frac{G(x)}{Gm})$ is $O(2N)$ , respectively. Thus, ISPEA/R requires a total of $O(4mN^{2}+6N)=O(mN^{2})$ computations. Compared with SPEA/R, ISPEA/R requires less computations for convergence measure.

Table 1
The computations of fitness assignment in ISPEA/R and SPEA/R

ISPEA/R SPEA/R

Convergence measure $O(4mN^{2}+6N)$ $O(8N^{3}+8mN^{2}-4N^{2}+2N)$

Diversity measure $O(4N^{2}+6N)$ $O(6N)$

Fitness assignment $O(4mN^{2}+4N^{2}+12N)$ $O(8N^{3}+8mN^{2}-4N^{2}+8N)$

2.
For the diversity measure, both ISPEA/R and SPEA/R consider the solutions in the subspace where $|E(i)|>1$ . In the worst situation that there is no subspace which contains only one solution, that is, the diversity measure should be computed for all solutions. In SPEA/R, calculating the density $D(x)$ (i.e., $\frac{\theta_{x}}{\theta_{x}+\theta_{m}}$ ) of all solutions requires $O(6N)$ computations. In ISPEA/R, firstly, all solutions in each subspace need to be divided into two categories based on whether their angle values are different from the one of any others in the subspace, this operation requires $O(4N^{2})$ computations in the worst case that all the $2N$ solutions get trapped into one subspace. Then the solutions belong to different categories will be treated differently. For the solutions whose angle values are different from the one of any others in the subspace, their $D(x)$ values are calculated. For the solutions having same angle values, in order to well distinguish them, the perpendicular distances $d^{i}(x)$ of these solutions are calculated. The complexity of this operation is $O(6N)$ in the worst case in which the angle between each solution in the same subspace and the reference direction is different. Thus, in the process of diversity measure, ISPEA/R requires extra $O(4N^{2})$ computations than SPEA/R.

As shown in Table 1, the computation cost of fitness assignment in ISPEA/R is $O(4mN^{2}+4N^{2}+12N)=O(mN^{2})$ while in SPEA/R is $O(8N^{3}+8mN^{2}-4N^{2}+8N)=O(N^{3}+mN^{2})$ . The computational cost of ISPEA/R is obviously less than that of SPEA/R.

3. Experimental study

	ISPEA/R	SPEA/R
Convergence measure	$O(4mN^{2}+6N)$	$O(8N^{3}+8mN^{2}-4N^{2}+2N)$
Diversity measure	$O(4N^{2}+6N)$	$O(6N)$
Fitness assignment	$O(4mN^{2}+4N^{2}+12N)$	$O(8N^{3}+8mN^{2}-4N^{2}+8N)$

In this section, to investigate the performance of ISPEA/R, we compare it with five state-of-the-art MOEAs via experiments for many-objective optimization problems. Empirical experiments are conducted on 24 benchmark test problems taken from three widely used test suites. The source codes of these compared algorithms are obtained on a Matlab platform named PlatEMO [55] (Matlab version: R2015b 64-bit). We perform all the experiments on this platform for a fair comparison environment. In the following subsections, we will first briefly introduce the 24 benchmark test problems and the performance indicators used in our comparative studies. Then, the five state-of-the-art many-objective optimization evolutionary algorithms which are employed for comparison are briefly introduced, followed by the experimental parameter settings used in the comparisons. Finally, the empirical results and discussion are presented.

3.1 Benchmark test problems

The 24 benchmark test problems are taken from three widely used test suites, i.e., seven test problems from DTLZ1 to DTLZ7 are taken from DTLZ test suite [15], eight test problems (most of them are scaled versions of DTLZ problems) are selected from MaF test suite for CEC’2018 competition on evolutionary many-objective optimization algorithms [42] and nine test problems WFG1 to WFG9 are taken from WFG test suite [22], which are also the most used benchmarks in many-objective optimization. These test problems contain diverse properties, covering a good representation of various real-world scenarios and challenges multifarious abilities of an algorithm. In Table 2, we summarize the main features of all the adopted test problems.

Table 2
Features of the test problems

Problem	Features	Problem	Features	Problem	Features
DTLZ1	Linear, multi-modal	MaF1	Linear (modified inverted DTLZ1)	WFG1	Mixed, biased, scaled
DTLZ2	Concave	MaF2	Concave (DTLZ2BZ)	WFG2	Convex, disconnected, multi-modal, non-separable, scaled
DTLZ3	Concave, multi-modal	MaF4	Concave, multi-modal (inverted badly-scaled DTLZ3)	WFG3	Linear, degenerate, non-separable, scaled
DTLZ4	Concave, biased	MaF5	Concave, biased (concave badly-scaled DTLZ4)	WFG4	Concave, multi-modal, scaled
DTLZ5	Concave, degenerate	MaF6	Concave, degenerate (DTLZ5 (I, M))	WFG5	Concave, deceptive, scaled
DTLZ6	Concave, degenerate, biased	MaF7	Mixed, disconnected, multi-modal	WFG6	Concave, non-separable, scaled
DTLZ7	Mixed, disconnected, multi-modal	MaF9	Linear, degenerate (multi-line distance minimization problem)	WFG7	Concave, biased, scaled
		MaF10	Mixed, biased	WFG8	Concave, biased, non-separable, scaled
				WFG9	Concave, biased, multi-modal, deceptive, non-separable, scaled

All these test problems can be scaled to any number of objectives and decision variables. In our experiment, we consider the number of objectives from 5 to 20, i.e., $m\in\{5,8,12,16,20\}$ . The number of decision variables on DTLZ test suite is given by $n=m+k-1$ . To be specific, for DTLZ1, the parameter $k$ is set to 5, 10 for DTLZ2-DTLZ6 and 20 for DTLZ7 [13]. For the eight MaF test problems, the number of decision variables of each test problem is same as that setting in CEC’2018 competition [42]. For nine WFG test problems, the number of decision variables is set to 24 and the position related parameter is set to $m-1$ according to the reference [22].

3.2 Performance Indicators

In this paper, the Hypervolume (HV) indicator [72], Inverted generational distance (IGD) [73] and Spacing [49] are used as the performance measures of the compared algorithms in the experiments. The details of those three indicators are as follows:

1) 1.
Hypervolume (HV) [72]: HV can assess both the convergence and diversity of each obtained solution set. The calculation of the HV-indicator needs to set a reference point ${y^{}}=(y_{1}^{},y_{2}^{},\ldots,\linebreak y_{m}^{})$ in the objective space. Then the obtained final solution set $A$ in the objective space can be measured by the volume of the region which is dominated by $A$ , and each solution in $A$ dominates ${y^{}}$ . HV indicator can be calculated by

$\displaystyle HV(A)=L\left(\bigcup\limits_{x\in A}{[{f_{1}}(x),{y_{1}}^{}]}% \times\ldots\right.\left.\phantom{\bigcup\limits_{x\in A}}[{f_{m}}(x),{y_{m}}^% {}]\right)$ (14)

where $L(\cdot)$ is the Lebesgue measure of a set $A$ . We take the reference point $y^{}$ as 1.1 times of the upper bounds of the true PFs. For HV, a larger value will indicate a better quality of $A$ for approximating the whole true PF.
2.
Inverted generational distance (IGD) [73]: The IGD indicator can also measure both the convergence and diversity of the obtained solution set. It computes the minimum distance from each true Pareto solution to all solutions of obtained optimal set. Let $P^{}$ be a set of solutions uniformly sampled from the true Pareto front and $A$ be the set of non-dominated solutions obtained in the objective space by an algorithm. The IGD between $P^{}$ and $A$ can be mathematically defined as follows:

$\displaystyle\textit{IGD}(A,{P^{}})=\frac{1}{{|{P^{}}|}}\sum\limits_{i=1}^{|% {P^{}}|}{\mathop{\min}\limits_{f\in A}d({f_{i}}^{},f)}$ (15)

where $|P^{}|$ denotes the number of elements in $P^{}$ and ${f_{i}}^{}\in P^{}$ . $d({f_{i}}^{},f)$ denotes the Euclidean distance between the point ${f_{i}}^{}$ and $f$ . The IGD value for different sampled points can vary a bit due to the different geometries of the Pareto fronts. In this paper, the number of sampled points (i.e., the number of solutions in $P^{*}$ ) in true PF is set to a number closest to 10000 (due to the fact that it is impossible to set an exact number 10000 of sample points for each test instance). The set $A$ with a smaller IGD value indicates a better performance.
3.
Spacing [49]: The Spacing indicator ( $S$ ) is used to measure the uniformity of the distribution of the obtained solution set. For an obtained solution set $A$ , Spacing indicator can be calculated by

$\displaystyle\left\{\begin{array}[]{l}S(A)=\sqrt{\frac{1}{|A|-1}\sum\limits_{x% \in A}(\bar{d}-d(x))^{2}}\\ d(x)=\min\{\|x-y\||x\neq y,y\in A\}\\ \bar{d}=\frac{1}{|A|}\sum\limits_{x\in A}d(x)\end{array}\right.$ (16)

when computing the Spacing value of a solution set $A$ , we do not need to know the true PF. The set $A$ with a smaller Spacing value indicates a better uniformity of the distribution.

3.3 Algorithms under comparisons

To verify the proposed ISPEA/R, we compare it with five state-of-the-art MOEAs, namely SPEA/R [24], NSGA-III [13], PICEA-g [59], dMOPSO [38] and VaEA [62], representing reference vector-based, angle-based and fitness assignment-based methods with different strategies for MaOPs. Here we just present the general principle of each compared algorithm in the following.

1) 1.
SPEA/R [24]: SPEA/R inherits the advantage of fitness assignment in SPEA2 [71], and designs a new fitness assignment scheme based on the decomposed objective space to evaluate the quality of each solution, where both Pareto dominance and diversity of solutions in each decomposed subspace are considered, and then a diversity-first-and-convergence-second selection strategy is proposed based on the decomposed subspaces. SPEA/R has been demonstrated the high efficiency on various test problems.
2.
NSGA-III [13]: NSGA-III is a reference point-based evolutionary algorithm based on NSGA-II for handling MaOPs. The basic framework of NSGA-III remains similar to that of NSGA-II, while the difference is the way to select solutions from the last front $F_{l}$ , and NSGA-III replaces the crowding distance operator used in NSGA-II by a reference point-based niche-preservation operation.
3.
PICEA-g [59]: PICEA-g is a posteriori pre- ference-based algorithm. During the search, PICEA-g coevolves a family of decision maker’ preferences together with a population of candidate solutions, and then designs a fitness assignment method based on weight vectors, which takes preference information into account. PICEA-g outperforms several state-of-the-art methods in terms of convergence and diversity on WFG [22] test problems.
4.
dMOPSO [38]: dMOPSO is a decomposition-based multi-objective particle swarm algorithm, it updates the position of each particle using a set of solutions considered as the global best, where the penalty-based boundary intersection (PBI) approach [67] is used to benefits the diversity of solutions in the memory re-initialization process.
5.
VaEA [62]: VaEA is a vector angle-based evolutionary algorithm, which has a similar framework to that of NSGA-III, and both of them consider a niching technique to evaluate the solutions in the last front $F_{l}$ . The weight vectors in VaEA are adaptively adjusted according to the distribution of the current population and then the solutions are selected one by one according to the maximum-vector-angle-first principle.

3.4 Experimental settings

The experimental settings of ISPEA/R, SPEA/R [24], NSGA-III [13], PICEA-g [59], dMOPSO [38] and VaEA [62] in the experiments are as follows.

1) 1.
Population size: In order to make a fair comparison, we set the same population sizes for these six algorithms. As shown in Table 3, the population size is determined by the simplex-lattice design factor $H$ together with the number of objective $m$ [8]. For problems with $m\geqslant 8$ , a two-layered reference vector method [13] is applied to generate reference directions on not only the outer boundaries but also the inside layers of the PFs.

Table 3
Setting of the population size

No. of objectives (m) Divisions (H) Population size (N)

5 6 210

8 (4, 3) 450

12 (3, 3) 728

16 (3, 2) 952

20 (3, 1) 1560

2.
Number of runs and termination criterion: All algorithms are independently run 20 times on each test problem. The termination condition of an algorithm on each run is that the maximum function evaluations (MFEs) are reached. For these three test suites DTLZ1-7, WFG1-9 and a subset of MaF functions, the MFEs are set to 63000, 135000, 218400, 285600 and 468000 when $m$ is 5, 8, 12, 16 and 20, respectively.
3.
Significance test: To test the difference for statistical significance, the Wilcoxon rank sum test is adopted to compare the results achieved by ISPEA/R and other five compared algorithms at a significant level of 0.05. Symbol ‘ $+$ ’ indicates that the compared algorithm significantly outperforms ISPEA/R according to the Wilcoxon rank sum test, while ‘ $-$ ’ represents that ISPEA/R significantly outperforms the compared algorithm, and ‘ $\approx$ ’ means that there is no statistically significant difference between the results achieved by ISPEA/R and the compared algorithm.
4.
Settings for crossover and mutation operators: SBX [10] and polynomial mutation [11] are used in all the considered algorithms except for dMOPSO. For ISPEA/R, SPEA/R, NSGA-III, PICEA-g and VaEA, the crossover probability $p_{c}=$ 1.0 and distribution index $\eta_{c}=$ 30 are used in SBX [10]. For the polynomial mutation [11], the distribution index and the mutation probability are set to $\eta_{m}=$ 20 and $p_{m}=1/n$ , respectively.
5.
Specific parameter setting in each algorithm: For dMOPSO, the age threshold $T_{a}=$ 2. For ISPEA/R and SPEA/R, the number of parent candidates $K=$ 20.

Figure 3.
The variations of HV values achieved by the six compared algorithms on 16-objective DTLZ7.

3.5 Performance comparisons on DTLZ test suite

The statistical results of ISPEA/R and other five state-of-the-art MOEAs in terms of IGD and HV values on DTLZ test suite are presented in Tables 4 and 5, respectively. The best results are highlighted in bold. From these statistical results, it is clear that ISPEA/R is the best optimizer as it wins on most comparisons. For IGD measure, among 35 test instances, ISPEA/R got the best results on 13 instances, while SPEA/R, dMOPSO, VaEA, NSGA-III and PICEA-g only got the best results on 3, 7, 2, 8, 2 instances, respectively. Moreover, for problems with more than 10 objectives, ISPEA/R performs much better than any compared algorithm. In fact, for total 21 test instances with more than 10 objectives, ISPEA/R got the best results on 13 instances, while SPEA/R, dMOPSO, VaEA, NSGA-III and PICEA-g only got the best results on 3, 3, 0, 1, 1 instances, respectively. From the statistical results of Wilcoxon rank sum test in the last line of Table 4, we can see that, among the 35 test instances, the IGD results obtained by ISPEA/R are significantly better than those obtained by SPEA/R, dMOPSO, VaEA and PICEA-g in most test instances (in 22, 24, 20 and 23 test instances, respectively), while the IGD results of SPEA/R, dMOPSO, VaEA, and PICEA-g are significantly better than those obtained by ISPEA/R in only a few test instances (in 11, 7, 8, 6, respectively). Moreover, NSGA-III performs similar to ISPEA/R on IGD measure.

For HV measure, the comparison results are similar. In fact, ISPEA/R got the best results on 14 instances, while SPEA/R, dMOPSO, VaEA, NSGA-III and PICEA-g only got the best results on 3, 5, 1, 9, 3 instances, respectively. Moreover, for problems with more than 10 objectives, ISPEA/R performs much better than any compared algorithm. In fact, for total 21 test instances with more than 10 objectives, ISPEA/R got the best results on 12 instances, while SPEA/R, dMOPSO, VaEA, NSGA-III and PICEA-g only got the best results on 2, 3, 1, 2, 1 instances, respectively. From the statistical results of Wilcoxon rank sum test in the last line of Table 5, we can see that, among the 35 test instances, ISPEA/R performs significantly better than SPEA/R, dMOPSO, VaEA, NSGA-III and PICEA-g in 22, 21, 22, 14 and 12 test instances, respectively, while SPEA/R, dMOPSO, VaEA, NSGA-III and PICEA-g only perform significantly better than ISPEA/R in 9, 6, 10, 13 and 8 test instances, respectively.

To illustrate this further, Fig. 3 displays the HV variations achieved by ISPEA/R and other five compared algorithms on 16-objective DTLZ7 instance. It can be seen from this figure that ISPEA/R achieves the best HV values in all iterations, and NSGA-III and PICEA-g achieved similar the HV values to those obtained by ISPEA/R in the late iteration. The HV values achieved by SPEA/R, VaEA and dMOPSO are relatively poor, which is coincident with the statistical results in Tables 4 and 5.

Therefore, for DTLZ test suite, ISPEA/R performs better than 5 compared algorithms, especially on problems with more than 10 objectives. And for most test instances, ISPEA/R performs best.

3.6 Performance comparisons on MaF test suite

To further investigate the performance of ISPEA/R and other five state-of-the-art MOEAs, we test them on a subset of MaF test problems. As shown in Table 2, MaF1, MaF2, MaF4, MaF5, MaF6 and MaF7 are the variants of DTLZ test suite, MaF9 and MaF10 have the different characteristics from DTLZ test suite. They have more challenge for the algorithms than DTLZ test suite. Tables 6 and 7 present the comparison results of ISPEA/R with other five algorithms in terms of IGD and HV values on 40 test instances of the eight MaF test problems. The best results are highlighted in bold. For IGD measure, from the statistical results in Table 6, we can see that, among the 40 test instances, ISPEA/R performs much better than SPEA/R, dMOPSO, NSGA-III and PICEA-g and a little bit worse than VaEA. For problems with more than 10 objectives, ISPEA/R also performs much better than SPEA/R, dMOPSO, NSGA-III and PICEA-g, and similar to VaEA. From the results of Wilcoxon rank sum test in the last line of Table 6, we can see that ISPEA/R

Table 4
Statistical results (mean and standard deviation) of the IGD values obtained by six algorithms for DTLZ problems

Problem	$M$	SPEA/R	dMOPSO	VaEA	NSGA-III	PICEA-g	ISPEA/R
DTLZ1	5	8.9915e-2 (2.29e-2) $+$	2.8406e+0 (3.54e+0) $-$	9.9058e-2 (2.41e-2) $+$	5.3251e-2 (5.62e-4) $+$	2.6486e-1 (5.55e-2) $-$	1.8478e-1 (4.68e-2)
	8	1.2516e-1 (1.23e-2) $+$	1.0483e+0 (7.68e-1) $-$	1.6102e-1 (2.16e-2) $\approx$	8.1318e-2 (4.58e-4) $+$	2.7759e-1 (4.82e-2) $\approx$	2.6155e-1 (1.01e-1)
	12	3.1390e-1 (1.09e-1) $\approx$	6.7085e-1 (4.38e-1) $-$	2.8727e-1 (9.23e-2) $\approx$	1.0212e-1 (1.36e-2) $+$	3.0421e-1 (5.47e-2) $\approx$	2.2893e-1 (7.93e-2)
	16	7.8748e-1 (2.38e-1) $-$	3.6397e-1 (1.35e-1) $-$	3.8476e-1 (1.16e-1) $-$	2.3152e-1 (6.40e-3) $\approx$	3.6418e-1 (4.09e-2) $-$	2.2955e-1 (8.51e-2)
	20	6.6246e-1 (1.32e-1) $-$	3.0001e-1 (3.04e-2) $\approx$	7.4018e-1 (1.20e-1) $-$	2.5047e-1 (3.82e-2) $\approx$	3.8124e-1 (3.63e-2) $-$	2.4660e-1 (3.47e-2)
DTLZ2	5	1.6522e-1 (6.49e-5) $+$	2.4649e-1 (6.47e-3) $-$	1.6905e-1 (1.57e-3) $+$	1.6519e-1 (1.64e-5) $+$	2.4493e-1 (1.32e-2) $-$	1.8446e-1 (5.28e-3)
	8	2.8523e-1 (7.50e-4) $+$	4.1447e-1 (1.63e-2) $-$	3.0439e-1 (2.49e-3) $+$	2.8327e-1 (2.46e-4) $+$	3.5508e-1 (1.86e-3) $\approx$	3.5514e-1 (1.90e-2)
	12	3.9080e-1 (4.54e-4) $+$	5.4519e-1 (3.73e-3) $-$	4.6703e-1 (6.23e-3) $\approx$	4.0198e-1 (3.51e-2) $\approx$	5.0226e-1 (4.12e-3) $\approx$	4.7633e-1 (2.72e-2)
	16	4.8009e-1 (2.18e-3) $+$	7.4977e-1 (4.35e-2) $-$	5.9349e-1 (1.24e-2) $+$	5.3209e-1 (3.21e-2) $+$	5.4065e-1 (1.77e-2) $+$	6.7811e-1 (2.47e-2)
	20	4.5899e-1 (4.14e-3) $+$	9.2415e-1 (1.53e-2) $-$	7.3180e-1 (9.84e-3) $+$	5.3622e-1 (3.64e-2) $+$	6.1823e-1 (5.16e-2) $+$	8.2406e-1 (5.69e-2)
DTLZ3	5	5.6501e+0 (2.86e+0) $-$	1.8847e+2 (1.68e+1) $-$	8.4003e-1 (8.79e-1) $+$	1.4712e+0 (8.69e-1) $+$	5.5869e-1 (2.01e-2) $+$	2.9237e+0 (7.32e-1)
	8	9.3851e+0 (3.41e+0) $-$	1.7303e+2 (2.45e+1) $-$	5.8958e+0 (2.38e+0) $\approx$	6.0835e-1 (2.45e-1) $+$	7.4121e-1 (6.09e-2) $+$	2.5572e+0 (2.01e+0)
	12	5.2984e+0 (1.12e+0) $-$	1.9275e+2 (6.77e+0) $-$	4.3606e+1 (9.79e+0) $-$	1.7659e+0 (9.21e-1) $\approx$	9.1397e-1 (6.95e-2) $\approx$	1.0197e+0 (3.69e-1)
	16	9.4700e+1 (1.18e+1) $-$	1.7950e+2 (9.16e+0) $-$	4.8671e+1 (1.07e+1) $-$	1.0251e+1 (8.30e+0) $-$	9.7612e-1 (6.42e-2) $-$	7.7845e-1 (7.18e-2)
	20	6.3820e+1 (2.43e+1) $-$	1.7190e+2 (7.24e+0) $-$	4.6233e+1 (1.52e+1) $-$	1.4698e+0 (8.58e-1) $-$	1.0460e+0 (6.58e-2) $-$	6.9465e-1 (5.30e-2)
DTLZ4	5	1.6541e-1 (1.04e-4) $+$	4.5863e-1 (1.74e-2) $-$	1.7080e-1 (1.01e-3) $\approx$	1.6520e-1 (8.08e-5) $+$	2.4233e-1 (4.87e-3) $-$	1.7250e-1 (2.60e-3)
	8	2.8856e-1 (1.06e-3) $+$	5.5797e-1 (2.05e-2) $-$	3.1448e-1 (3.16e-3) $\approx$	2.8361e-1 (3.22e-4) $+$	3.7010e-1 (4.01e-2) $-$	3.1680e-1 (2.17e-3)
	12	3.9770e-1 (2.46e-3) $-$	6.8704e-1 (9.83e-3) $-$	4.8964e-1 (1.23e-2) $-$	3.8751e-1 (4.28e-2) $-$	4.9678e-1 (2.92e-3) $-$	3.7390e-1 (2.48e-3)
	16	5.0723e-1 (5.72e-3) $-$	7.0409e-1 (1.33e-2) $-$	6.3506e-1 (1.32e-2) $-$	4.9775e-1 (9.99e-5) $\approx$	5.4228e-1 (5.17e-3) $-$	4.9720e-1 (1.48e-3)
	20	4.7999e-1 (5.01e-3) $-$	7.1888e-1 (7.15e-3) $-$	7.4595e-1 (1.09e-2) $-$	4.2782e-1 (1.85e-3) $-$	5.6292e-1 (4.88e-3) $-$	4.1052e-1 (2.01e-3)
DTLZ5	5	1.8500e-1 (1.77e-2) $\approx$	1.8741e-2 (1.05e-3) $+$	1.2543e-1 (2.90e-2) $\approx$	8.3767e-2 (1.84e-2) $+$	8.6749e-2 (2.80e-2) $+$	1.3734e-1 (6.03e-2)
	8	4.2466e-1 (2.95e-2) $-$	1.0963e-1 (3.65e-3) $+$	3.0960e-1 (3.34e-2) $-$	2.8746e-1 (4.05e-2) $-$	1.6619e-1 (3.43e-2) $+$	2.3880e-1 (5.32e-2)
	12	8.2105e-1 (1.28e-1) $-$	7.8718e-2 (3.05e-3) $-$	4.9014e-1 (1.30e-1) $-$	2.6030e-1 (1.15e-1) $-$	2.7131e-1 (5.38e-2) $-$	1.2686e-2 (3.00e-2)
	16	1.0977e+0 (4.27e-1) $-$	1.5604e-2 (5.33e-3) $\approx$	5.1375e-1 (1.64e-1) $-$	3.1364e-1 (6.42e-2) $-$	3.1634e-1 (1.93e-2) $-$	1.3448e-2 (4.50e-2)
	20	1.0570e+0 (4.64e-1) $-$	3.0522e-2 (3.36e-3) $-$	3.8466e-1 (1.04e-1) $-$	2.8254e-1 (3.68e-2) $-$	2.7017e-1 (2.61e-2) $-$	1.5222e-2 (1.65e-2)
DTLZ6	5	2.9262e-1 (6.88e-2) $-$	2.2416e-2 (1.24e-5) $+$	2.3758e-1 (1.71e-2) $-$	1.4744e-1 (4.02e-2) $\approx$	1.8117e-1 (4.16e-2) $\approx$	1.5377e-1 (1.44e-2)
	8	8.6999e-1 (1.93e-1) $-$	1.0005e-2 (7.02e-6) $+$	2.7044e+0 (3.07e-1) $-$	3.6188e-1 (2.38e-1) $-$	3.0671e-1 (5.51e-2) $-$	1.9704e-1 (4.48e-2)
	12	6.9315e-1 (3.86e-1) $-$	1.0544e-2 (1.40e-6) $+$	2.6805e+0 (4.07e-1) $-$	4.3532e-1 (3.07e-1) $-$	3.6941e-1 (6.71e-2) $-$	2.5663e-1 (1.07e-1)
	16	6.5740e+0 (7.36e-1) $-$	2.4299e-2 (1.85e-6) $+$	3.2562e+0 (1.58e-1) $-$	2.5415e+0 (5.83e-1) $-$	4.1279e-1 (4.60e-2) $-$	2.1774e-1 (1.01e-1)
	20	8.7844e+0 (8.47e-1) $-$	3.9095e-2 (2.56e-6) $+$	3.4402e+0 (3.93e-1) $-$	3.8830e-1 (2.07e-1) $\approx$	5.6467e-1 (8.59e-2) $-$	3.7582e-1 (1.74e-1)
DTLZ7	5	3.5378e-1 (9.42e-3) $+$	4.5401e-1 (3.80e-2) $\approx$	2.7449e-1 (3.10e-3) $+$	2.9713e-1 (2.15e-2) $+$	6.6381e-1 (3.65e-1) $-$	4.5822e-1 (6.36e-2)
	8	8.3479e-1 (5.99e-2) $+$	1.1667e+0 (1.39e-1) $\approx$	6.3835e-1 (1.11e-2) $+$	5.8443e-1 (1.33e-2) $+$	3.2537e+0 (6.55e-1) $-$	1.4574e+0 (1.98e-1)
	12	3.2969e+0 (3.39e-1) $-$	2.8881e+0 (2.22e-1) $-$	2.8290e+0 (4.40e-2) $-$	3.5294e+0 (2.66e-1) $-$	5.8311e+0 (1.51e-1) $-$	2.1359e+0 (2.07e-1)
	16	8.4542e+0 (4.68e-2) $-$	6.6543e+0 (3.99e-1) $-$	6.5174e+0 (9.07e-1) $-$	8.3382e+0 (1.21e+0) $-$	1.0800e+1 (6.36e-2) $-$	5.7587e+0 (7.38e-1)
	20	3.1823e+1 (1.32e-1) $-$	3.3222e+1 (5.99e-1) $-$	1.9978e+1 (2.38e-1) $-$	2.8164e+1 (6.11e-1) $-$	2.3737e+1 (6.46e-2) $-$	1.4076e+1 (5.94e-1)
$+/-/\approx$		11/22/2	7/24/4	8/20/7	14/14/7	6/23/6

‘ $+$ ’, ‘ $-$ ’ and ‘ $\approx$ ’ indicate that the result is significantly better, significantly worse and statistically similar to that obtained by ISPEA/R, respectively.

Table 5

Statistical results (mean and standard deviation) of the HV values obtained by six algorithms for DTLZ problems

Problem	$M$	SPEA/R	dMOPSO	VaEA	NSGA-III	PICEA-g	ISPEA/R
DTLZ1	5	4.7356e-2 (1.47e-3) $+$	1.7777e-3 (3.98e-3) $-$	4.5231e-2 (2.42e-3) $+$	4.9259e-2 (3.08e-5) $+$	2.8634e-2 (9.62e-3) $-$	3.2397e-2 (4.13e-3)
	8	8.1416e-3 (1.81e-4) $+$	4.5886e-4 (6.31e-4) $-$	8.0306e-3 (2.81e-4) $+$	8.3633e-3 (8.36e-7) $+$	6.0105e-3 (1.18e-3) $-$	6.8812e-3 (1.82e-3)
	12	3.0303e-4 (1.56e-4) $-$	1.2427e-4 (1.32e-4) $-$	5.0624e-4 (2.19e-4) $-$	6.7604e-4 (8.03e-8) $\approx$	6.0988e-4 (1.26e-4) $-$	6.7720e-4 (1.53e-4)
	16	1.0701e-6 (2.23e-6) $-$	2.1318e-5 (1.34e-5) $-$	2.8789e-5 (1.92e-5) $-$	7.0113e-5 (4.17e-10) $+$	4.9872e-5 (9.91e-6) $-$	5.8937e-5 (1.63e-5)
	20	1.8118e-7 (3.11e-7) $-$	2.7058e-6 (7.93e-7) $-$	1.3185e-7 (1.83e-7) $-$	5.4157e-6 (2.50e-10) $\approx$	5.0115e-6 (7.35e-7) $-$	5.9896e-6 (8.10e-7)
DTLZ2	5	1.3059e+0 (5.11e-4) $+$	8.6078e-1 (2.14e-2) $-$	1.2717e+0 (4.32e-3) $+$	1.3068e+0 (5.31e-4) $+$	1.2540e+0 (1.03e-2) $+$	1.1270e+0 (1.52e-2)
	8	2.0166e+0 (2.60e-3) $\approx$	1.2778e+0 (4.07e-2) $-$	1.9661e+0 (7.06e-3) $\approx$	2.0138e+0 (1.18e-3) $\approx$	2.0765e+0 (1.60e-3) $\approx$	1.9687e+0 (8.05e-3)
	12	3.1008e+0 (6.66e-3) $+$	1.7677e+0 (3.94e-2) $-$	2.9093e+0 (2.30e-2) $+$	3.0803e+0 (2.20e-2) $+$	2.8944e+0 (3.58e-3) $\approx$	2.7746e+0 (1.42e-1)
	16	4.5659e+0 (2.46e-3) $+$	2.8966e+0 (1.52e-1) $\approx$	4.6815e+0 (5.64e-2) $+$	4.5534e+0 (9.56e-3) $+$	4.5787e+0 (5.55e-3) $+$	2.6759e+0 (1.40e-1)
	20	6.7203e+0 (1.04e-3) $+$	4.3287e+0 (1.76e-1) $+$	5.8555e+0 (9.88e-2) $+$	6.7083e+0 (1.15e-2) $+$	6.7071e+0 (1.79e-2) $+$	3.3846e+0 (6.41e-1)
DTLZ3	5	0.0000e+0 (0.00e+0) $\approx$	0.0000e+0 (0.00e+0) $\approx$	6.2524e-1 (5.77e-1) $+$	2.2437e-1 (5.02e-1) $+$	6.4217e-1 (7.22e-2) $+$	0.0000e+0 (0.00e+0)
	8	0.0000e+0 (0.00e+0) $\approx$	0.0000e+0 (0.00e+0) $\approx$	0.0000e+0 (0.00e+0) $\approx$	9.7118e-1 (7.29e-1) $+$	9.6320e-1 (1.77e-1) $+$	6.9061e-2 (6.36e-2)
	12	0.0000e+0 (0.00e+0) $-$	0.0000e+0 (0.00e+0) $-$	0.0000e+0 (0.00e+0) $-$	5.0199e-1 (1.12e+0) $-$	1.3025e+0 (4.04e-1) $\approx$	1.4269e+0 (8.88e-1)
	16	0.0000e+0 (0.00e+0) $-$	0.0000e+0 (0.00e+0) $-$	0.0000e+0 (0.00e+0) $-$	3.4789e-3 (7.78e-3) $-$	2.3894e+0 (3.64e-1) $-$	3.7129e+0 (4.97e-1)
	20	0.0000e+0 (0.00e+0) $-$	0.0000e+0 (0.00e+0) $-$	0.0000e+0 (0.00e+0) $-$	1.9219e+0 (2.07e+0) $-$	2.9041e+0 (6.33e-1) $-$	6.2492e+0 (5.57e-1)
DTLZ4	5	1.3037e+0 (9.10e-4) $+$	9.3252e-1 (2.10e-2) $-$	1.2706e+0 (3.25e-3) $+$	1.3058e+0 (9.14e-4) $+$	1.2671e+0 (5.87e-3) $+$	1.2002e+0 (8.84e-3)
	8	2.0051e+0 (2.50e-3) $+$	1.5807e+0 (3.38e-2) $-$	1.9370e+0 (1.03e-2) $+$	2.0260e+0 (9.97e-4) $+$	1.9987e+0 (2.65e-2) $+$	1.8683e+0 (3.62e-2)
	12	3.0517e+0 (7.66e-3) $\approx$	2.6600e+0 (4.89e-2) $-$	2.8863e+0 (4.86e-2) $\approx$	3.0769e+0 (1.04e-3) $\approx$	3.0900e+0 (1.16e-3) $\approx$	2.9652e+0 (9.66e-3)
	16	4.1481e+0 (6.15e-3) $-$	4.1831e+0 (6.55e-2) $-$	4.1065e+0 (7.33e-2) $-$	4.3878e+0 (3.73e-2) $-$	4.2417e+0 (1.06e-2) $-$	4.5287e+0 (8.87e-2)
	20	6.5133e+0 (2.37e-3) $-$	6.3154e+0 (2.80e-2) $-$	5.8536e+0 (9.09e-2) $-$	6.4961e+0 (1.44e-2) $-$	6.5231e+0 (5.76e-4) $\approx$	6.7540e+0 (8.01e-2)
DTLZ5	5	2.5980e-3 (1.57e-3) $-$	7.3468e-3 (4.02e-4) $\approx$	6.8667e-3 (3.40e-4) $-$	7.2126e-3 (1.75e-4) $-$	7.5226e-3 (3.95e-4) $\approx$	7.9680e-3 (2.96e-4)
	8	2.8121e-8 (3.84e-8) $-$	1.6893e-5 (1.79e-6) $\approx$	1.5541e-5 (5.94e-7) $-$	5.7840e-6 (6.51e-6) $-$	1.8085e-5 (4.23e-7) $\approx$	1.8389e-5 (1.82e-7)
	12	0.0000e+0 (0.00e+0) $-$	4.7315e-11 (1.60e-12) $\approx$	3.6689e-11 (3.47e-12) $-$	3.7552e-11 (4.05e-12) $-$	4.7411e-11 (8.52e-13) $\approx$	4.8000e-11 (5.99e-13)
	16	0.0000e+0 (0.00e+0) $-$	5.2552e-19 (4.65e-21) $+$	3.5444e-19 (9.17e-20) $-$	3.7126e-19 (1.01e-19) $-$	5.2000e-19 (2.93e-21) $\approx$	5.1684e-19 (3.82e-21)
	20	0.0000e+0 (0.00e+0) $-$	2.2194e-29 (1.59e-31) $+$	1.4439e-29 (2.26e-30) $-$	1.4024e-29 (3.42e-30) $-$	2.2056e-29 (1.56e-31) $+$	2.1677e-29 (2.81e-31)
DTLZ6	5	6.8249e-4 (1.12e-3) $-$	9.0800e-3 (1.64e-5) $+$	2.9067e-3 (3.26e-3) $-$	7.3451e-3 (5.67e-4) $\approx$	7.5600e-3 (5.86e-4) $\approx$	7.8522e-3 (2.74e-4)
	8	0.0000e+0 (0.00e+0) $-$	1.9503e-5 (5.47e-8) $+$	0.0000e+0 (0.00e+0) $-$	1.1427e-5 (7.76e-6) $-$	1.7565e-5 (5.75e-7) $\approx$	1.6882e-5 (1.32e-7)
	12	4.0209e-18 (8.99e-18) $-$	5.0437e-11 (1.06e-13) $+$	0.0000e+0 (0.00e+0) $-$	1.8813e-11 (2.58e-11) $-$	4.1692e-11 (6.69e-13) $-$	4.6941e-11 (1.46e-13)
	16	0.0000e+0 (0.00e+0) $-$	5.1495e-19 (1.75e-21) $\approx$	0.0000e+0 (0.00e+0) $-$	0.0000e+0 (0.00e+0) $-$	5.1534e-19 (3.03e-21) $\approx$	5.1990e-19 (7.56e-22)
	20	0.0000e+0 (0.00e+0) $-$	2.1402e-29 (5.66e-32) $\approx$	0.0000e+0 (0.00e+0) $-$	1.7481e-29 (9.77e-30) $-$	8.7438e-30 (1.20e-29) $-$	2.1829e-29 (3.36e-32)
DTLZ7	5	2.1712e+0 (1.59e-2) $+$	1.3498e+0 (1.39e-1) $-$	2.1206e+0 (1.90e-2) $+$	2.1586e+0 (5.24e-2) $+$	2.1086e+0 (2.49e-1) $\approx$	1.9966e+0 (6.17e-2)
	8	2.1069e+0 (1.64e-1) $-$	1.1473e+0 (9.58e-2) $-$	2.0728e+0 (1.71e-2) $-$	2.4484e+0 (4.14e-2) $\approx$	2.1817e+0 (6.29e-2) $-$	2.4327e+0 (5.30e-2)
	12	1.0804e+0 (3.55e-1) $-$	4.5053e-1 (5.37e-1) $-$	1.4435e+0 (1.23e-1) $-$	2.3749e+0 (4.04e-2) $\approx$	2.3052e+0 (3.33e-2) $-$	2.4011e+0 (4.60e-2)
	16	1.7864e+0 (5.18e-2) $-$	4.2816e-2 (9.57e-2) $-$	7.0284e-1 (2.09e-1) $-$	2.2931e+0 (9.96e-2) $\approx$	2.2247e+0 (1.65e-2) $\approx$	2.3283e+0 (1.40e-1)
	20	1.3992e+0 (8.44e-2) $-$	8.5083e-9 (1.34e-8) $-$	1.6798e-1 (1.38e-1) $-$	2.2314e+0 (9.83e-2) $+$	2.1872e+0 (1.23e-2) $\approx$	2.0528e+0 (1.22e-1)
$+/-/\approx$		9/22/4	6/21/8	10/22/3	13/14/8	8/12/15

‘ $+$ ’, ‘ $-$ ’ and ‘ $\approx$ ’ indicate that the result is significantly better, significantly worse and statistically similar to that obtained by ISPEA/R, respectively.

Table 6

Statistical results (mean and standard deviation) of the IGD values obtained by six algorithms for MaF problems

Problem	$M$	SPEA/R	dMOPSO	VaEA	NSGA-III	PICEA-g	ISPEA/R
MaF1	5	1.8835e-1 (3.41e-3) $+$	2.3656e-1 (2.82e-2) $-$	2.7790e-1 (3.31e-4) $-$	2.4678e-1 (1.19e-2) $-$	2.3058e-1 (3.05e-3) $-$	2.1156e-1 (2.42e-2)
	8	4.2623e-1 (3.26e-2) $-$	4.6775e-1 (2.00e-2) $-$	2.6611e-1 (3.42e-4) $-$	2.6361e-1 (1.47e-2) $-$	1.9524e-1 (3.15e-3) $+$	2.4402e-1 (1.71e-2)
	12	6.4419e-1 (2.83e-2) $-$	5.0571e-1 (1.75e-2) $-$	2.1015e-1 (1.24e-3) $+$	2.6369e-1 (3.15e-3) $+$	2.7363e-1 (1.34e-2) $+$	3.2709e-1 (8.37e-3)
	16	5.5241e-1 (2.46e-2) $-$	5.3098e-1 (1.15e-2) $-$	2.3403e-1 (1.06e-3) $+$	3.5253e-1 (5.88e-3) $-$	3.5034e-1 (1.29e-2) $-$	3.2712e-1 (6.11e-3)
	20	6.3758e-1 (2.08e-2) $-$	6.2565e-1 (5.26e-3) $-$	2.5595e-1 (1.90e-3) $+$	4.5109e-1 (6.18e-3) $\approx$	4.4909e-1 (6.27e-3) $\approx$	4.5447e-1 (9.52e-3)
MaF2	5	1.1898e-1 (9.68e-4) $+$	1.0705e-1 (7.56e-4) $+$	9.0353e-2 (1.98e-3) $+$	1.1469e-1 (3.79e-3) $+$	1.4617e-1 (7.55e-3) $+$	2.7237e-1 (3.78e-2)
	8	1.7318e-1 (4.23e-4) $+$	1.8228e-1 (2.12e-3) $+$	1.3848e-1 (2.53e-3) $+$	1.5677e-1 (1.06e-2) $+$	3.8388e-1 (2.54e-2) $+$	6.5090e-1 (6.46e-2)
	12	2.7575e-1 (6.60e-2) $+$	2.3793e-1 (2.32e-3) $+$	1.7165e-1 (2.34e-3) $+$	1.7043e-1 (4.66e-3) $+$	5.9120e-1 (4.25e-3) $+$	6.8676e-1 (5.08e-2)
	16	2.6126e-1 (1.23e-3) $+$	4.4218e-1 (9.93e-3) $+$	1.7290e-1 (1.70e-3) $+$	1.7048e-1 (4.56e-3) $+$	6.9632e-1 (1.10e-3) $\approx$	7.0886e-1 (4.42e-2)
	20	1.6895e-1 (1.66e-3) $+$	5.1771e-1 (1.34e-2) $+$	1.6293e-1 (8.10e-4) $+$	2.5665e-1 (1.19e-2) $+$	7.1567e-1 (2.13e-3) $\approx$	7.2226e-1 (3.83e-2)
MaF4	5	1.4226e+1 (8.86e+0) $-$	1.2532e+3 (1.22e+3) $-$	2.1460e+0 (2.03e-1) $+$	2.3409e+0 (1.85e-1) $+$	6.5046e+0 (7.77e-2) $-$	4.1101e+0 (5.53e-1)
	8	1.6964e+2 (1.99e+2) $-$	3.2369e+3 (2.03e+3) $-$	1.1579e+1 (1.30e-1) $+$	2.1086e+1 (5.60e-1) $+$	2.7770e+1 (5.12e+0) $+$	3.6966e+1 (2.68e+0)
	12	2.8760e+3 (3.94e+3) $-$	5.7646e+4 (6.66e+4) $-$	2.0453e+2 (5.58e+1) $+$	3.2466e+2 (2.80e+1) $+$	3.6383e+2 (1.19e+2) $\approx$	4.5258e+2 (4.06e+1)
	16	8.3291e+4 (6.63e+4) $-$	3.7051e+5 (2.50e+5) $-$	9.2901e+3 (2.30e+2) $-$	8.5655e+3 (3.90e+2) $-$	8.5502e+3 (1.07e+3) $-$	6.7233e+3 (1.53e+3)
	20	1.0442e+5 (3.84e+4) $-$	7.2418e+6 (7.10e+6) $-$	1.8610e+4 (1.77e+3) $-$	8.1656e+4 (8.81e+3) $-$	5.9215e+4 (9.17e+3) $-$	1.1944e+4 (1.37e+4)
MaF5	5	1.9752e+0 (6.80e-3) $-$	1.0016e+1 (2.53e-1) $-$	1.9370e+0 (3.04e-2) $-$	1.9703e+0 (3.67e-3) $-$	2.7463e+0 (5.07e-1) $-$	1.7693e+0 (4.37e-2)
	8	1.5605e+1 (3.23e-1) $-$	8.7157e+1 (1.98e-1) $-$	1.3131e+1 (2.37e-1) $-$	1.4645e+1 (2.89e-1) $-$	1.4866e+1 (6.02e-1) $-$	1.0953e+1 (2.93e-1)
	12	2.0584e+2 (1.44e+1) $-$	1.1013e+3 (2.63e-1) $-$	1.7625e+2 (3.89e+0) $\approx$	1.9943e+2 (3.92e+0) $-$	2.1735e+2 (9.62e+0) $-$	1.6254e+2 (5.93e+0)
	16	3.3160e+3 (1.97e+2) $-$	1.3980e+4 (1.72e-2) $-$	1.8340e+3 (1.17e+2) $-$	2.5068e+3 (7.28e+1) $-$	2.5909e+3 (1.23e+2) $-$	1.6094e+3 (1.25e+2)
	20	3.6288e+4 (5.23e+3) $-$	1.7095e+5 (8.13e-2) $-$	9.1346e+3 (7.48e+2) $-$	1.8408e+4 (4.44e+3) $-$	1.4541e+4 (9.64e+2) $-$	6.0570e+3 (1.82e+2)
MaF6	5	9.3975e-2 (8.33e-3) $\approx$	2.5614e-1 (4.59e-2) $-$	2.5547e-3 (9.07e-5) $+$	1.6023e-2 (5.28e-3) $+$	7.3363e-1 (1.03e-2) $-$	1.1687e-1 (1.87e-2)
	8	1.8603e-1 (7.58e-2) $\approx$	1.4799e-1 (2.41e-2) $\approx$	1.6375e-1 (3.63e-1) $\approx$	1.5269e-1 (3.29e-1) $\approx$	5.6781e-1 (2.82e-1) $-$	2.0861e-1 (5.66e-2)
	12	2.9456e-1 (2.81e-2) $\approx$	1.1284e-1 (1.84e-2) $+$	1.7344e+0 (5.00e-1) $-$	8.0868e-1 (2.49e-1) $-$	6.9484e-1 (8.06e-2) $-$	2.6655e-1 (7.39e-2)
	16	2.3457e+1 (2.26e+1) $-$	9.5539e-2 (2.31e-2) $+$	1.7573e+0 (4.76e-1) $-$	7.5701e-1 (3.00e-1) $-$	6.3670e-1 (3.16e-1) $-$	2.0554e-1 (6.56e-2)
	20	4.4536e+1 (2.55e+1) $-$	7.9623e-2 (1.91e-2) $+$	2.0637e+0 (1.12e+0) $-$	1.1594e+0 (1.27e-1) $-$	9.2848e-1 (9.09e-2) $-$	2.9699e-1 (5.57e-2)
MaF7	5	3.5222e-1 (5.38e-3) $+$	4.3370e-1 (1.21e-2) $\approx$	2.6573e-1 (6.63e-3) $+$	2.8966e-1 (9.48e-3) $+$	5.3798e-1 (1.97e-1) $\approx$	6.3735e-1 (3.37e-1)
	8	7.2448e-1 (1.75e-2) $+$	1.2308e+0 (1.67e-1) $\approx$	6.3308e-1 (1.04e-2) $+$	5.9585e-1 (1.35e-2) $+$	3.5362e+0 (4.76e-2) $-$	1.9935e+0 (8.10e-1)
	12	3.9554e+0 (1.05e+0) $\approx$	1.8588e+0 (2.08e-1) $+$	1.2687e+0 (5.37e-2) $+$	3.8074e+0 (3.45e-1) $-$	5.8912e+0 (2.31e-1) $-$	3.2427e+0 (9.03e-1)
	16	8.3627e+0 (6.18e-2) $+$	2.6790e+0 (2.91e-1) $+$	1.3856e+1 (3.19e-1) $-$	7.4554e+0 (1.17e+0) $+$	1.0800e+1 (7.22e-2) $-$	1.0267e+1 (5.60e-1)
	20	1.1849e+1 (1.29e-1) $+$	3.1829e+0 (5.07e-1) $+$	1.6046e+1 (9.83e-2) $-$	1.8129e+1 (1.22e+0) $-$	1.3669e+1 (1.19e-1) $\approx$	1.3479e+1 (3.83e-1)
MaF9	5	9.7081e-1 (3.80e-1) $\approx$	9.6565e-2 (2.46e-4) $+$	5.2051e-1 (2.33e-1) $+$	2.7251e-1 (1.13e-1) $+$	6.3159e-1 (7.68e-2) $+$	1.1252e+0 (3.43e-1)
	8	2.3139e+0 (1.20e+0) $-$	1.4131e-1 (3.69e-4) $+$	1.2455e-1 (2.18e-2) $+$	6.1374e-1 (2.71e-1) $\approx$	6.7035e-1 (1.24e-1) $\approx$	1.3230e+0 (9.68e-1)
	12	7.4446e+0 (3.16e+0) $-$	4.2712e-1 (7.34e-4) $+$	1.0393e-1 (2.34e-2) $+$	3.9309e+0 (3.23e+0) $-$	8.8055e-1 (3.89e-1) $\approx$	1.1566e+0 (2.20e-1)
	16	1.0491e+1 (4.56e+0) $-$	9.9902e-1 (7.83e-4) $\approx$	1.8474e-1 (2.11e-1) $+$	2.6647e+0 (4.43e+0) $\approx$	7.6991e-1 (1.00e-1) $+$	3.2040e+0 (4.53e+0)
	20	1.0953e+1 (7.81e+0) $\approx$	1.4303e+0 (4.42e-4) $\approx$	9.5632e-2 (5.44e-2) $+$	3.0113e-1 (4.86e-2) $+$	9.1338e-1 (1.26e-1) $\approx$	1.6184e+0 (1.13e+0)
MaF10	5	7.6146e-1 (3.75e-2) $+$	2.1141e+0 (1.08e-2) $-$	1.0473e+0 (8.77e-2) $-$	8.8445e-1 (1.13e-1) $\approx$	9.4249e-1 (7.12e-1) $-$	8.7143e-1 (5.74e-2)
	8	1.0685e+0 (4.92e-2) $-$	2.9230e+0 (3.10e-2) $-$	1.7592e+0 (7.96e-2) $-$	1.3088e+0 (8.75e-2) $-$	2.1229e+0 (9.20e-1) $-$	9.2165e-1 (7.41e-2)
	12	1.3876e+0 (2.49e-2) $-$	3.7518e+0 (6.11e-2) $-$	2.7691e+0 (1.03e-1) $-$	1.5621e+0 (8.26e-2) $-$	2.5047e+0 (1.35e-2) $-$	1.2390e+0 (6.05e-2)
	16	1.9324e+0 (6.12e-2) $\approx$	4.6031e+0 (8.61e-2) $-$	3.4141e+0 (1.67e-1) $-$	1.9998e+0 (8.43e-2) $-$	2.7371e+0 (7.26e-2) $-$	1.9301e+0 (1.50e-1)
	20	3.5627e+0 (2.72e-1) $\approx$	6.3518e+0 (2.53e-2) $-$	4.1304e+0 (2.00e-1) $-$	3.5168e+0 (1.20e-1) $\approx$	3.6072e+0 (5.61e-1) $\approx$	3.5011e+0 (1.49e-1)
$+/-/\approx$		11/21/8	14/21/5	20/18/2	15/19/6	8/22/10

‘ $+$ ’, ‘ $-$ ’ and ‘ $\approx$ ’ indicate that the result is significantly better, significantly worse and statistically similar to that obtained by ISPEA/R, respectively.

Table 7

Statistical results (mean and standard deviation) of the HV values obtained by six algorithms for MaF problems

Problem	$M$	SPEA/R	dMOPSO	VaEA	NSGA-III	PICEA-g	ISPEA/R
MaF1	5	1.0449e-2 (2.80e-4) $\approx$	7.8644e-3 (1.16e-3) $-$	1.8900e-2 (2.69e-4) $+$	1.0026e-2 (6.49e-4) $-$	1.6322e-2 (4.55e-4) $+$	1.0690e-2 (9.25e-4)
	8	6.4828e-6 (2.13e-6) $-$	4.7943e-6 (7.78e-7) $-$	7.8459e-5 (1.04e-5) $+$	4.7020e-5 (6.44e-6) $+$	8.9042e-5 (4.69e-6) $+$	3.2687e-5 (5.82e-6)
	12	5.7342e-12 (1.09e-11) $-$	3.3452e-10 (6.82e-11) $-$	0.0000e+0 (0.00e+0) $-$	3.1140e-8 (7.51e-10) $+$	1.7525e-8 (7.11e-9) $\approx$	8.7108e-9 (1.39e-9)
	16	7.2233e-14 (4.27e-14) $-$	4.0768e-14 (4.80e-15) $-$	0.0000e+0 (0.00e+0) $-$	5.8715e-12 (1.52e-12) $+$	1.8793e-12 (7.16e-13) $\approx$	1.5689e-12 (2.18e-13)
	20	1.1148e-17 (5.18e-18) $-$	4.3292e-18 (8.31e-19) $-$	0.0000e+0 (0.00e+0) $-$	4.2419e-16 (1.52e-16) $-$	2.7754e-16 (1.99e-16) $-$	6.3748e-16 (1.68e-16)
MaF2	5	4.5102e-2 (4.90e-4) $+$	4.5915e-2 (9.13e-4) $+$	5.1587e-2 (1.79e-4) $+$	4.9949e-2 (3.69e-4) $+$	5.0861e-2 (4.82e-4) $+$	3.1933e-2 (2.67e-3)
	8	2.6660e-2 (2.82e-4) $+$	2.7037e-2 (5.13e-4) $+$	3.2172e-2 (1.22e-4) $+$	3.1388e-2 (5.51e-4) $+$	3.4317e-2 (1.99e-4) $+$	1.5098e-2 (2.98e-3)
	12	9.8568e-4 (1.52e-4) $\approx$	1.1176e-3 (1.25e-5) $+$	1.5731e-3 (1.16e-5) $+$	1.5448e-3 (3.34e-5) $+$	1.7249e-3 (1.00e-5) $+$	7.1127e-4 (8.67e-5)
	16	1.9395e-5 (4.34e-7) $+$	1.6543e-5 (2.76e-7) $+$	2.2169e-5 (3.24e-7) $+$	2.4229e-5 (2.58e-7) $+$	2.3200e-5 (2.31e-7) $+$	9.7587e-6 (9.66e-7)
	20	7.9476e-8 (1.68e-9) $+$	6.9276e-8 (4.96e-10) $+$	9.1395e-8 (1.96e-9) $+$	9.3213e-8 (8.43e-10) $+$	9.3235e-8 (9.70e-10) $+$	4.0458e-8 (3.63e-9)
MaF4	5	1.8931e+2 (2.69e+2) $-$	0.0000e+0 (0.00e+0) $-$	5.0288e+3 (6.12e+2) $+$	3.4232e+3 (3.70e+2) $-$	2.0805e+2 (9.07e+1) $-$	4.2404e+3 (3.63e+2)
	8	4.6026e+7 (9.57e+7) $-$	0.0000e+0 (0.00e+0) $-$	3.9192e+8 (7.58e+7) $\approx$	3.5720e+8 (9.10e+6) $-$	6.8029e+6 (4.79e+6) $-$	4.0638e+8 (4.72e+7)
	12	9.6292e+11 (2.07e+12) $-$	0.0000e+0 (0.00e+0) $-$	5.4141e+18 (2.54e+18) $-$	1.6592e+19 (9.03e+17) $-$	0.0000e+0 (0.00e+0) $-$	1.9281e+19 (3.28e+18)
	16	7.5916e+28 (1.47e+29) $-$	0.0000e+0 (0.00e+0) $-$	0.0000e+0 (0.00e+0) $-$	4.8420e+34 (3.03e+33) $+$	0.0000e+0 (0.00e+0) $-$	3.4355e+34 (6.01e+33)
	20	7.2636e+53 (9.47e+53) $-$	0.0000e+0 (0.00e+0) $-$	0.0000e+0 (0.00e+0) $-$	3.2806e+54 (4.32e+53) $-$	0.0000e+0 (0.00e+0) $-$	4.4464e+54 (5.63e+53)
MaF5	5	4.2676e+4 (3.83e+1) $+$	1.0298e+4 (2.21e+3) $-$	4.1541e+4 (1.02e+2) $+$	4.2810e+4 (2.67e+1) $+$	3.9590e+4 (2.76e+3) $\approx$	3.9893e+4 (6.81e+2)
	8	1.3710e+11 (1.53e+8) $+$	2.6933e+10 (3.30e+9) $-$	1.3433e+11 (7.06e+8) $+$	1.3920e+11 (6.25e+7) $+$	1.3807e+11 (2.78e+8) $+$	1.2851e+11 (9.69e+8)
	12	8.9055e+23 (8.15e+21) $\approx$	1.6455e+23 (1.27e+22) $-$	8.8784e+23 (6.90e+21) $\approx$	9.3636e+23 (1.57e+20) $+$	9.3630e+23 (6.02e+20) $+$	8.7648e+23 (1.35e+22)
	16	3.5983e+41 (1.13e+40) $\approx$	7.6277e+40 (8.68e+39) $-$	3.4451e+41 (1.56e+40) $-$	3.9968e+41 (1.98e+37) $+$	3.9916e+41 (4.43e+37) $+$	3.7318e+41 (7.52e+39)
	20	9.9431e+63 (5.11e+61) $-$	2.1570e+63 (1.77e+62) $-$	9.5243e+63 (3.75e+62) $-$	1.1068e+64 (6.93e+58) $+$	1.1063e+64 (6.37e+59) $+$	1.0525e+64 (2.05e+62)
MaF6	5	8.1328e-3 (4.13e-5) $\approx$	1.1455e-3 (3.10e-4) $-$	9.2362e-3 (3.96e-5) $+$	8.8242e-3 (1.34e-4) $+$	6.4764e-3 (1.21e-5) $-$	7.9333e-3 (1.66e-4)
	8	1.6748e-5 (1.42e-7) $\approx$	8.7585e-7 (7.05e-7) $\approx$	1.5771e-5 (8.82e-6) $\approx$	1.5527e-5 (8.68e-6) $\approx$	1.7608e-5 (1.10e-6) $\approx$	3.7196e-6 (7.95e-6)
	12	3.8940e-11 (3.02e-12) $\approx$	1.5237e-12 (1.44e-12) $\approx$	0.0000e+0 (0.00e+0) $-$	0.0000e+0 (0.00e+0) $-$	4.7304e-11 (5.55e-13) $\approx$	3.6871e-11 (2.02e-11)
	16	5.2973e-20 (1.18e-19) $-$	5.6868e-21 (2.74e-21) $-$	0.0000e+0 (0.00e+0) $-$	0.0000e+0 (0.00e+0) $-$	4.0690e-19 (2.28e-19) $\approx$	5.1028e-19 (1.66e-21)
	20	0.0000e+0 (0.00e+0) $-$	2.3561e-31 (2.91e-31) $-$	0.0000e+0 (0.00e+0) $-$	0.0000e+0 (0.00e+0) $-$	0.0000e+0 (0.00e+0) $-$	1.6016e-29 (6.64e-30)
MaF7	5	2.1574e+0 (1.65e-2) $+$	1.4220e+0 (5.59e-2) $-$	2.1545e+0 (2.10e-2) $+$	2.1784e+0 (2.82e-2) $+$	2.1926e+0 (1.88e-1) $+$	1.9780e+0 (1.09e-1)
	8	2.0702e+0 (1.01e-1) $-$	1.1149e+0 (4.11e-2) $-$	2.0768e+0 (3.94e-2) $-$	2.2250e+0 (7.42e-2) $-$	2.1435e+0 (2.66e-2) $-$	2.4733e+0 (8.00e-2)
	12	4.9686e-1 (5.24e-1) $-$	6.5188e-1 (6.04e-1) $-$	1.4782e+0 (9.88e-2) $-$	2.2898e+0 (1.34e-1) $-$	2.2810e+0 (4.15e-2) $-$	2.5234e+0 (1.53e-1)
	16	1.8436e+0 (6.47e-2) $-$	6.3279e-5 (1.32e-4) $-$	6.8006e-1 (1.15e-1) $-$	2.1914e+0 (1.60e-1) $-$	2.1813e+0 (1.62e-2) $-$	2.3128e+0 (1.26e-1)
	20	1.3963e+0 (1.53e-1) $-$	2.8828e-7 (6.03e-7) $-$	5.1001e-1 (1.91e-1) $-$	1.9851e+0 (6.78e-2) $-$	2.0931e+0 (1.54e-2) $\approx$	2.2232e+0 (7.31e-2)
MaF9	5	2.0448e+0 (1.92e+0) $\approx$	9.4627e+0 (1.20e-2) $+$	5.1304e+0 (1.99e+0) $+$	7.2823e+0 (1.11e+0) $+$	3.7903e+0 (7.35e-1) $+$	1.9363e+0 (8.99e-1)
	8	1.3912e+0 (2.75e+0) $\approx$	1.4460e+1 (2.95e-2) $+$	1.4451e+1 (5.40e-1) $+$	6.1575e+0 (2.79e+0) $\approx$	3.1444e+0 (1.43e+0) $\approx$	3.8758e+0 (2.52e+0)
	12	1.2384e-1 (2.77e-1) $-$	4.4529e+1 (2.17e-1) $+$	5.4309e+1 (2.91e+0) $+$	1.0515e+1 (1.47e+1) $-$	7.0625e+0 (6.07e+0) $-$	1.1784e+1 (3.49e+0)
	16	1.8790e-1 (4.20e-1) $-$	9.4815e+1 (1.55e-1) $+$	1.5145e+2 (3.64e+1) $+$	2.2014e+1 (3.59e+1) $-$	1.6038e+1 (1.11e+1) $-$	2.8927e+1 (1.69e+1)
	20	1.0114e+0 (1.39e+0) $-$	1.7896e+2 (3.90e-1) $+$	4.3623e+2 (6.10e+1) $+$	2.6346e+2 (3.53e+1) $+$	8.7115e+0 (9.35e+0) $-$	6.9919e+1 (6.73e+1)
MaF10	5	4.8319e+3 (1.19e+2) $+$	1.6339e+3 (1.27e+1) $-$	3.6609e+3 (1.96e+2) $-$	4.2897e+3 (2.45e+2) $\approx$	6.0289e+3 (8.48e+0) $+$	4.2741e+3 (1.25e+2)
	8	1.7336e+7 (9.93e+5) $-$	4.7297e+6 (8.19e+4) $-$	9.1639e+6 (7.32e+5) $-$	1.3164e+7 (1.08e+6) $-$	1.7727e+7 (2.47e+3) $-$	1.8186e+7 (6.34e+5)
	12	4.9559e+12 (2.80e+11) $-$	1.0475e+12 (1.55e+10) $-$	1.6011e+12 (1.30e+11) $-$	3.3784e+12 (3.09e+11) $-$	5.0616e+12 (1.63e+8) $-$	5.1348e+12 (1.89e+11)
	16	4.5219e+18 (4.56e+14) $-$	8.6515e+17 (2.81e+16) $-$	1.2439e+18 (1.09e+17) $-$	3.2481e+18 (9.87e+16) $-$	4.6225e+18 (3.31e+13) $-$	4.6848e+18 (1.39e+16)
	20	1.0854e+25 (2.72e+20) $-$	1.8899e+24 (3.23e+22) $-$	2.9214e+24 (3.84e+23) $-$	8.3650e+24 (1.10e+24) $-$	1.0794e+25 (7.85e+19) $-$	1.0868e+25 (2.05e+22)
$+/-/\approx$		8/23/9	10/28/2	17/20/3	18/19/3	14/18/8

‘ $+$ ’, ‘ $-$ ’ and ‘ $\approx$ ’ indicate that the result is significantly better, significantly worse and statistically similar to that obtained by ISPEA/R, respectively.

Table 8

Statistical results (mean and standard deviation) of the IGD values obtained by six algorithms for WFG problems

Problem	$M$	SPEA/R	dMOPSO	VaEA	NSGA-III	PICEA-g	ISPEA/R
WFG1	5	1.0717e+0 (6.70e-2) $+$	2.1238e+0 (1.15e-2) $-$	1.5619e+0 (5.36e-2) $-$	1.1993e+0 (9.06e-2) $\approx$	7.3888e-1 (1.03e-1) $+$	1.1778e+0 (4.36e-2)
	8	1.3972e+0 (7.77e-2) $+$	2.9347e+0 (1.86e-2) $-$	2.2107e+0 (6.59e-2) $-$	1.7326e+0 (1.04e-1) $-$	2.2286e+0 (4.66e-1) $-$	1.5288e+0 (5.47e-2)
	12	1.6144e+0 (1.13e-1) $+$	3.7676e+0 (2.91e-2) $-$	2.8966e+0 (6.24e-2) $-$	2.0341e+0 (1.55e-1) $-$	2.5251e+0 (2.75e-2) $-$	1.7845e+0 (1.10e-1)
	16	1.8453e+0 (7.73e-2) $+$	4.6480e+0 (7.75e-2) $-$	3.0978e+0 (1.89e-1) $-$	1.7910e+0 (4.51e-2) $+$	2.7316e+0 (5.61e-2) $-$	2.4742e+0 (1.12e-1)
	20	2.8164e+0 (4.30e-1) $+$	6.2028e+0 (7.81e-2) $-$	2.8395e+0 (1.03e-1) $+$	2.3101e+0 (9.36e-2) $+$	3.5666e+0 (5.31e-1) $+$	4.1510e+0 (2.92e-1)
WFG2	5	7.5422e-1 (6.28e-2) $\approx$	4.1252e+0 (6.77e-1) $-$	7.8673e-1 (1.12e-1) $\approx$	8.8038e-1 (1.96e-1) $-$	2.7912e+0 (2.27e-1) $-$	7.8419e-1 (1.77e-1)
	8	9.9099e-1 (1.40e-1) $\approx$	7.0955e+0 (2.28e-1) $-$	1.8581e+0 (1.66e-1) $-$	1.4238e+0 (2.91e-1) $-$	5.6205e+0 (4.13e-1) $-$	1.0469e+0 (1.86e-1)
	12	3.0903e+0 (6.92e-1) $-$	1.7976e+1 (3.66e-1) $-$	3.6591e+0 (3.83e-1) $-$	5.7180e+0 (3.30e-1) $-$	1.1014e+1 (4.17e-1) $-$	2.1777e+0 (3.37e-1)
	16	1.2032e+0 (2.44e-1) $\approx$	2.6774e+1 (3.84e-1) $-$	1.5627e+0 (3.81e-1) $\approx$	1.1162e+1 (2.38e+0) $-$	1.7726e+1 (2.96e-1) $-$	1.4158e+0 (4.15e-1)
	20	9.4330e-1 (6.04e-1) $\approx$	3.5758e+1 (4.08e-1) $-$	1.0981e+0 (4.31e-1) $\approx$	7.4304e+0 (3.15e+0) $-$	2.2651e+1 (1.60e+0) $-$	2.5042e+0 (1.32e+0)
WFG3	5	4.6813e-1 (2.68e-2) $+$	6.6371e-1 (3.60e-2) $\approx$	6.1632e-1 (8.91e-2) $\approx$	5.9660e-1 (5.44e-2) $\approx$	9.0939e-1 (1.39e+0) $\approx$	6.3615e-1 (1.44e-1)
	8	1.6175e+0 (6.22e-2) $\approx$	1.5823e+0 (4.72e-2) $\approx$	1.4203e+0 (9.11e-2) $\approx$	1.4179e+0 (2.42e-1) $\approx$	1.6677e+0 (2.19e+0) $\approx$	1.5724e+0 (9.87e-2)
	12	2.3146e+0 (1.75e-1) $\approx$	5.6878e+0 (3.85e-1) $-$	2.2208e+0 (1.73e-1) $\approx$	1.1080e+0 (2.67e-1) $+$	5.3978e+0 (4.29e+0) $\approx$	2.1153e+0 (1.79e-1)
	16	3.0555e+0 (6.64e-2) $-$	9.1188e+0 (2.09e+0) $-$	3.3640e+0 (2.89e-1) $-$	1.1109e+0 (2.06e-1) $+$	1.1827e+1 (5.46e-1) $-$	2.9016e+0 (9.05e-2)
	20	3.7660e+0 (4.82e-2) $\approx$	1.9740e+1 (4.29e+0) $-$	4.0596e+0 (3.43e-1) $-$	2.0153e+0 (1.14e+0) $+$	1.1715e+1 (2.45e+0) $-$	3.6414e+0 (4.68e-1)
WFG4	5	9.7366e-1 (7.42e-4) $-$	1.4657e+0 (1.31e-1) $-$	9.5007e-1 (6.92e-3) $+$	9.6742e-1 (1.43e-3) $-$	1.3377e+0 (4.67e-2) $-$	9.5409e-1 (1.62e-3)
	8	2.5474e+0 (5.72e-3) $-$	6.4820e+0 (8.42e-2) $-$	2.3672e+0 (6.89e-3) $+$	2.5208e+0 (4.25e-3) $\approx$	2.9563e+0 (3.71e-2) $-$	2.5299e+0 (3.57e-3)
	12	4.8308e+0 (9.03e-2) $-$	1.2326e+1 (2.92e-1) $-$	4.7598e+0 (2.04e-2) $-$	4.7299e+0 (4.51e-2) $-$	6.6409e+0 (1.19e+0) $-$	4.6537e+0 (7.54e-2)
	16	5.9468e+0 (1.87e-2) $-$	1.7186e+1 (3.23e-1) $-$	7.1395e+0 (1.15e-1) $-$	5.9672e+0 (1.40e-1) $-$	1.0579e+1 (2.33e+0) $-$	5.7601e+0 (1.02e-2)
	20	5.6143e+0 (1.62e-2) $\approx$	2.1955e+1 (1.26e-1) $-$	9.6422e+0 (5.13e-2) $-$	5.9508e+0 (8.22e-1) $-$	1.2567e+1 (1.80e+0) $-$	5.6173e+0 (9.67e-3)
WFG5	5	9.5739e-1 (9.34e-4) $\approx$	1.2438e+0 (1.90e-2) $-$	9.5409e-1 (5.33e-3) $\approx$	9.5683e-1 (1.10e-3) $\approx$	1.3201e+0 (2.73e-2) $-$	9.5697e-1 (1.33e-3)
	8	2.5439e+0 (3.92e-3) $-$	4.2496e+0 (2.22e-1) $-$	2.3948e+0 (8.67e-3) $+$	2.5490e+0 (2.08e-3) $-$	3.0457e+0 (2.71e-2) $-$	2.5216e+0 (6.04e-3)
	12	5.1372e+0 (1.48e-1) $-$	7.6239e+0 (2.04e-1) $-$	4.5764e+0 (3.10e-2) $+$	4.4583e+0 (2.66e-2) $+$	6.0053e+0 (9.50e-2) $-$	5.1075e+0 (1.07e-1)
	16	5.7217e+0 (8.44e-3) $-$	1.0850e+1 (5.31e-1) $-$	6.6672e+0 (3.02e-2) $-$	5.6176e+0 (1.09e-2) $-$	8.2801e+0 (2.00e-1) $-$	5.5984e+0 (1.03e-2)
	20	5.6283e+0 (4.70e-3) $\approx$	1.5795e+1 (1.69e-1) $-$	9.0391e+0 (7.26e-2) $-$	5.5975e+0 (3.99e-2) $\approx$	1.0854e+1 (8.58e-1) $-$	5.6269e+0 (1.01e-2)
WFG6	5	9.6312e-1 (6.22e-4) $\approx$	2.1974e+0 (5.93e-2) $-$	9.7550e-1 (4.98e-3) $-$	9.6515e-1 (6.68e-4) $\approx$	1.3679e+0 (3.65e-2) $-$	9.6256e-1 (2.96e-3)
	8	2.5498e+0 (3.91e-3) $\approx$	6.9881e+0 (2.67e-1) $-$	2.4152e+0 (1.44e-2) $+$	2.5399e+0 (9.41e-3) $\approx$	3.1813e+0 (3.20e-2) $-$	2.5473e+0 (6.62e-3)
	12	5.1974e+0 (8.69e-2) $\approx$	1.2280e+1 (2.06e-1) $-$	4.6112e+0 (3.48e-2) $+$	5.1300e+0 (8.11e-1) $\approx$	6.1945e+0 (4.10e-2) $-$	5.2304e+0 (7.17e-2)
	16	5.7780e+0 (1.35e-2) $-$	1.6291e+1 (1.33e-1) $-$	6.6277e+0 (7.84e-2) $-$	5.7871e+0 (1.42e-2) $-$	8.7975e+0 (8.95e-1) $-$	5.7510e+0 (8.23e-3)
	20	5.6205e+0 (1.85e-2) $\approx$	2.0850e+1 (1.13e-1) $-$	8.6447e+0 (4.63e-2) $-$	7.1192e+0 (2.20e+0) $\approx$	9.8262e+0 (5.23e-1) $-$	5.6193e+0 (7.28e-3)
WFG7	5	9.6782e-1 (3.77e-4) $\approx$	1.8074e+0 (7.07e-2) $-$	9.6400e-1 (9.43e-3) $\approx$	9.6383e-1 (1.64e-3) $\approx$	1.4874e+0 (2.96e-2) $-$	9.6784e-1 (3.06e-3)
	8	2.5552e+0 (5.13e-3) $\approx$	6.3808e+0 (4.65e-1) $-$	2.3776e+0 (1.63e-2) $+$	2.5479e+0 (9.82e-3) $\approx$	3.1938e+0 (4.12e-2) $-$	2.5649e+0 (5.56e-3)
	12	5.4757e+0 (3.41e-2) $-$	1.2638e+1 (1.32e-1) $-$	4.6251e+0 (2.30e-2) $+$	4.5728e+0 (1.97e-1) $+$	6.1251e+0 (7.95e-2) $-$	5.2766e+0 (6.12e-2)
	16	5.9313e+0 (2.45e-2) $\approx$	1.6613e+1 (3.36e-1) $-$	6.6837e+0 (3.59e-2) $-$	6.9347e+0 (4.98e-1) $-$	7.9538e+0 (8.02e-2) $-$	5.9571e+0 (5.37e-2)
	20	6.6171e+0 (9.46e-2) $\approx$	2.1768e+1 (2.91e-1) $-$	9.0185e+0 (8.96e-2) $-$	7.7869e+0 (4.09e-1) $-$	1.0241e+1 (3.36e-1) $-$	6.6162e+0 (8.52e-2)
WFG8	5	9.6329e-1 (1.30e-3) $\approx$	1.4843e+0 (3.63e-2) $-$	1.0030e+0 (6.98e-3) $-$	9.5975e-1 (1.40e-3) $\approx$	1.4096e+0 (6.30e-2) $-$	9.5839e-1 (1.77e-3)
	8	2.6186e+0 (7.89e-3) $-$	6.2221e+0 (3.84e-1) $-$	2.5869e+0 (3.60e-2) $-$	2.5848e+0 (1.21e-2) $-$	3.1380e+0 (2.02e-1) $-$	2.5257e+0 (1.15e-2)
	12	5.3682e+0 (2.64e-2) $\approx$	1.1603e+1 (4.74e-1) $-$	4.8596e+0 (4.71e-2) $+$	4.7868e+0 (4.75e-1) $+$	7.4697e+0 (6.29e-2) $-$	5.3571e+0 (1.52e-2)
	16	5.8312e+0 (1.85e-2) $-$	1.6592e+1 (1.62e-1) $-$	6.6324e+0 (6.43e-2) $-$	7.6650e+0 (2.30e-1) $-$	1.0367e+1 (1.39e-1) $-$	5.7675e+0 (1.60e-2)
	20	5.7252e+0 (1.27e-2) $\approx$	2.1274e+1 (2.94e-1) $-$	8.8103e+0 (8.15e-2) $-$	7.1667e+0 (4.06e-1) $-$	1.3210e+1 (4.32e-1) $-$	5.6741e+0 (5.57e-1)
WFG9	5	9.7568e-1 (1.34e-2) $-$	1.7631e+0 (4.54e-2) $-$	9.7028e-1 (8.50e-3) $\approx$	9.4560e-1 (6.39e-3) $+$	1.2860e+0 (3.37e-2) $-$	9.6890e-1 (1.06e-2)
	8	2.6085e+0 (9.24e-3) $-$	5.9522e+0 (1.55e-1) $-$	2.4731e+0 (1.93e-2) $+$	2.5247e+0 (6.48e-3) $+$	2.8733e+0 (8.85e-2) $-$	2.5645e+0 (1.57e-2)
	12	4.9476e+0 (1.08e-1) $-$	1.0996e+1 (4.74e-1) $-$	4.4043e+0 (3.69e-2) $+$	4.4971e+0 (6.24e-2) $+$	5.7835e+0 (9.32e-2) $-$	4.9107e+0 (1.12e-1)
	16	5.6964e+0 (2.79e-2) $\approx$	1.4885e+1 (1.08e+0) $-$	6.3854e+0 (7.47e-2) $-$	5.8136e+0 (3.36e-1) $-$	8.8880e+0 (1.97e-1) $-$	5.6959e+0 (2.84e-2)
	20	6.5218e+0 (4.75e-2) $\approx$	1.6667e+1 (6.18e-1) $-$	9.3664e+0 (6.75e-2) $-$	8.2160e+0 (6.86e-1) $-$	1.1494e+1 (5.05e-1) $-$	6.5142e+0 (6.16e-2)
$+/-/\approx$		6/16/23	0/43/2	12/24/9	11/21/13	2/40/3

‘ $+$ ’, ‘ $-$ ’ and ‘ $\approx$ ’ indicate that the result is significantly better, significantly worse and statistically similar to that obtained by ISPEA/R, respectively.

Table 9

Statistical results (mean and standard deviation) of the HV values obtained by six algorithms for WFG problems

Problem	$M$	SPEA/R	dMOPSO	VaEA	NSGA-III	PICEA-g	ISPEA/R
WFG1	5	3.7825e+3 (7.67e+1) $+$	1.6423e+3 (4.31e+0) $-$	2.4814e+3 (1.08e+2) $-$	3.2537e+3 (1.94e+2) $-$	5.0125e+3 (4.47e+1) $+$	3.6489e+3 (7.36e+1)
	8	1.3072e+7 (7.95e+5) $\approx$	4.6790e+6 (2.13e+5) $-$	6.8733e+6 (3.56e+5) $-$	9.4077e+6 (7.13e+5) $-$	2.0017e+7 (9.50e+4) $+$	1.3333e+7 (9.96e+5)
	12	4.2223e+12 (2.56e+11) $\approx$	1.0359e+12 (1.27e+10) $-$	1.4317e+12 (9.43e+10) $-$	2.6069e+12 (2.18e+11) $-$	5.2613e+12 (2.08e+8) $+$	4.0538e+12 (3.17e+11)
	16	4.7193e+18 (6.43e+14) $\approx$	8.6100e+17 (2.42e+16) $-$	1.4812e+18 (1.19e+17) $-$	3.7950e+18 (2.22e+17) $-$	4.7185e+18 (3.65e+13) $\approx$	4.7204e+18 (1.93e+15)
	20	1.0852e+25 (1.94e+19) $\approx$	2.1232e+24 (1.58e+23) $-$	1.0854e+25 (4.85e+19) $\approx$	1.0853e+25 (7.72e+20) $\approx$	1.0854e+25 (1.59e+17) $\approx$	1.0866e+25 (1.69e+20)
WFG2	5	6.0576e+3 (1.89e+1) $+$	5.2255e+3 (5.97e+1) $-$	6.0067e+3 (1.87e+1) $\approx$	6.0181e+3 (1.92e+1) $\approx$	6.0939e+3 (1.02e+1) $+$	6.0208e+3 (2.57e+1)
	8	2.1595e+7 (5.17e+4) $\approx$	1.8225e+7 (7.11e+5) $-$	2.1537e+7 (3.28e+4) $\approx$	2.1626e+7 (7.53e+4) $\approx$	2.1784e+7 (1.76e+4) $\approx$	2.1801e+7 (1.02e+5)
	12	6.0207e+12 (8.54e+9) $\approx$	4.9783e+12 (1.33e+11) $-$	6.0468e+12 (2.52e+10) $\approx$	6.1174e+12 (2.45e+10) $+$	6.1539e+12 (9.81e+8) $+$	6.0195e+12 (1.59e+10)
	16	6.2483e+18 (9.79e+15) $\approx$	5.0096e+18 (6.50e+16) $-$	6.2567e+18 (2.23e+15) $+$	6.2728e+18 (1.68e+16) $+$	6.2990e+18 (1.46e+15) $+$	6.2367e+18 (2.23e+16)
	20	1.7055e+25 (2.62e+22) $\approx$	1.4891e+25 (3.40e+23) $-$	1.7064e+25 (6.47e+21) $\approx$	1.7076e+25 (1.12e+22) $\approx$	1.7061e+25 (1.88e+21) $\approx$	1.7095e+25 (2.10e+22)
WFG3	5	1.8262e+0 (3.11e-1) $+$	6.1716e-1 (1.63e-1) $\approx$	7.8377e-1 (4.15e-1) $\approx$	1.1253e+0 (2.57e-1) $\approx$	3.1201e+0 (2.86e-1) $+$	8.8856e-1 (2.20e-1)
	8	0.0000e+0 (0.00e+0) $\approx$	0.0000e+0 (0.00e+0) $\approx$	0.0000e+0 (0.00e+0) $\approx$	0.0000e+0 (0.00e+0) $\approx$	3.1565e-2 (6.89e-4) $+$	0.0000e+0 (0.00e+0)
	12	0.0000e+0 (0.00e+0) $\approx$	0.0000e+0 (0.00e+0) $\approx$	0.0000e+0 (0.00e+0) $\approx$	0.0000e+0 (0.00e+0) $\approx$	2.0192e-8 (4.18e-9) $+$	0.0000e+0 (0.00e+0)
	16	0.0000e+0 (0.00e+0) $\approx$	0.0000e+0 (0.00e+0) $\approx$	0.0000e+0 (0.00e+0) $\approx$	0.0000e+0 (0.00e+0) $\approx$	4.1057e-19 (3.94e-19) $\approx$	0.0000e+0 (0.00e+0)
	20	0.0000e+0 (0.00e+0) $\approx$	0.0000e+0 (0.00e+0) $\approx$	0.0000e+0 (0.00e+0) $\approx$	0.0000e+0 (0.00e+0) $\approx$	7.1216e-34 (1.06e-33) $\approx$	0.0000e+0 (0.00e+0)
WFG4	5	4.8421e+3 (1.03e+1) $\approx$	3.4509e+3 (1.45e+2) $-$	4.5751e+3 (2.94e+1) $-$	4.7755e+3 (2.61e+1) $-$	4.8776e+3 (1.64e+1) $+$	4.8482e+3 (7.03e+0)
	8	1.9852e+7 (1.25e+5) $\approx$	8.7436e+6 (2.91e+5) $-$	1.8889e+7 (1.25e+5) $-$	1.9515e+7 (1.02e+5) $-$	1.9994e+7 (7.96e+3) $\approx$	1.9831e+7 (6.60e+4)
	12	5.5171e+12 (1.11e+11) $\approx$	1.6767e+12 (2.34e+11) $-$	5.5142e+12 (3.24e+10) $\approx$	5.5457e+12 (6.05e+10) $\approx$	5.8415e+12 (2.05e+11) $+$	5.4981e+12 (9.71e+10)
	16	6.0563e+18 (2.74e+16) $-$	1.8787e+18 (1.30e+17) $-$	5.7755e+18 (4.87e+16) $-$	6.0255e+18 (7.54e+16) $-$	6.0605e+18 (1.51e+17) $-$	6.2121e+18 (3.07e+16)
	20	1.6934e+25 (5.27e+22) $\approx$	4.8731e+24 (4.73e+23) $-$	1.6396e+25 (5.21e+22) $-$	1.6929e+25 (8.93e+22) $\approx$	1.6937e+25 (1.15e+23) $\approx$	1.6946e+25 (4.86e+22)
WFG5	5	4.6717e+3 (5.17e+0) $\approx$	3.1384e+3 (6.81e+1) $-$	4.4790e+3 (3.94e+1) $-$	4.6498e+3 (4.96e+0) $-$	4.5795e+3 (2.14e+1) $-$	4.6683e+3 (1.05e+1)
	8	1.9191e+7 (3.66e+4) $-$	4.4588e+6 (1.83e+6) $-$	1.8303e+7 (8.13e+4) $-$	1.9116e+7 (3.62e+4) $-$	1.9387e+7 (2.77e+4) $\approx$	1.9467e+7 (2.07e+4)
	12	4.5821e+12 (3.17e+11) $-$	4.8258e+11 (8.32e+10) $-$	5.2924e+12 (3.31e+10) $+$	5.3994e+12 (1.88e+10) $+$	5.6610e+12 (3.38e+9) $+$	4.6682e+12 (2.84e+11)
	16	5.7498e+18 (2.24e+15) $-$	3.3656e+17 (5.30e+16) $-$	5.4703e+18 (2.67e+16) $-$	5.7419e+18 (3.54e+15) $-$	5.7482e+18 (1.01e+16) $-$	5.7825e+18 (2.89e+15)
	20	1.5761e+25 (5.79e+21) $\approx$	8.4976e+23 (4.04e+22) $-$	1.5192e+25 (3.67e+22) $-$	1.5707e+25 (1.03e+22) $-$	1.5692e+25 (4.16e+22) $-$	1.5757e+25 (5.40e+21)
WFG6	5	4.5758e+3 (1.55e+1) $-$	2.7157e+3 (1.07e+2) $-$	4.3681e+3 (6.86e+1) $-$	4.5731e+3 (4.64e+1) $-$	4.5867e+3 (4.34e+1) $-$	4.6279e+3 (1.58e+1)
	8	1.8922e+7 (1.10e+5) $\approx$	6.7057e+6 (4.64e+5) $-$	1.7967e+7 (1.28e+5) $-$	1.8917e+7 (1.63e+5) $\approx$	1.9184e+7 (1.14e+5) $\approx$	1.9204e+7 (9.82e+4)
	12	4.2388e+12 (2.47e+11) $-$	1.3399e+12 (1.72e+11) $-$	5.1203e+12 (3.37e+10) $+$	5.2562e+12 (1.03e+11) $+$	5.6204e+12 (5.07e+10) $+$	4.5260e+12 (1.77e+11)
	16	5.4900e+18 (9.53e+16) $-$	1.0803e+18 (2.60e+17) $-$	5.2643e+18 (4.34e+16) $-$	5.6404e+18 (1.17e+17) $-$	5.6402e+18 (3.89e+16) $-$	5.7126e+18 (7.93e+16)
	20	1.5599e+25 (4.43e+23) $\approx$	2.0107e+24 (2.60e+23) $-$	1.5613e+25 (2.89e+23) $\approx$	1.5410e+25 (5.27e+23) $\approx$	1.6670e+25 (2.92e+23) $+$	1.5564e+25 (2.69e+23)
WFG7	5	4.8693e+3 (1.37e+1) $\approx$	2.6180e+3 (5.54e+1) $-$	4.6857e+3 (2.32e+1) $-$	4.8183e+3 (2.21e+1) $-$	4.7880e+3 (1.38e+1) $-$	4.8554e+3 (1.32e+1)
	8	2.0060e+7 (6.53e+4) $-$	7.2028e+6 (7.03e+5) $-$	1.9583e+7 (1.45e+5) $-$	1.9975e+7 (5.26e+4) $-$	2.0004e+7 (3.72e+4) $-$	2.1976e+7 (1.16e+5)
	12	4.3939e+12 (1.71e+11) $-$	1.0028e+12 (1.21e+11) $-$	5.7109e+12 (1.65e+10) $+$	5.6918e+12 (1.58e+11) $+$	6.0779e+12 (2.43e+9) $+$	4.5306e+12 (2.29e+11)
	16	6.1131e+18 (1.76e+16) $-$	7.4693e+17 (1.59e+17) $-$	5.9912e+18 (1.41e+16) $-$	6.2363e+18 (8.62e+15) $\approx$	6.1610e+18 (1.53e+16) $-$	6.2388e+18 (7.52e+15)
	20	1.6839e+25 (5.96e+22) $-$	1.2827e+24 (1.36e+23) $-$	1.6723e+25 (3.54e+22) $-$	1.7102e+25 (9.35e+21) $+$	1.7079e+25 (1.64e+22) $\approx$	1.7019e+25 (4.87e+22)
WFG8	5	4.3904e+3 (1.09e+1) $-$	2.3997e+3 (5.49e+1) $-$	4.1755e+3 (3.37e+1) $-$	4.4060e+3 (2.91e+1) $-$	4.3174e+3 (3.91e+1) $-$	4.5254e+3 (8.25e+0)
	8	1.8344e+7 (8.27e+4) $-$	2.1152e+6 (3.73e+5) $-$	1.6245e+7 (9.08e+4) $-$	1.8012e+7 (9.01e+4) $-$	1.8609e+7 (1.40e+5) $\approx$	1.8685e+7 (1.57e+5)
	12	4.2195e+12 (1.58e+11) $-$	1.0230e+12 (1.43e+11) $-$	4.6808e+12 (9.59e+10) $+$	5.1794e+12 (1.48e+11) $+$	5.6559e+12 (4.44e+9) $+$	4.3719e+12 (8.73e+10)
	16	6.0436e+18 (2.21e+16) $+$	8.8288e+17 (1.53e+17) $-$	5.3833e+18 (1.22e+17) $-$	5.8735e+18 (4.28e+16) $\approx$	6.0695e+18 (2.04e+16) $+$	5.9300e+18 (5.45e+16)
	20	1.6764e+25 (7.50e+22) $-$	1.9037e+24 (2.56e+23) $-$	1.5957e+25 (8.02e+22) $-$	1.6719e+25 (9.62e+22) $-$	1.6869e+25 (2.01e+22) $\approx$	1.6874e+25 (2.06e+22)
WFG9	5	4.2610e+3 (1.96e+2) $-$	3.5046e+3 (1.34e+2) $-$	4.1220e+3 (1.41e+2) $-$	4.3595e+3 (1.22e+2) $\approx$	4.3678e+3 (2.54e+1) $\approx$	4.3799e+3 (1.47e+2)
	8	1.7729e+7 (5.31e+5) $\approx$	9.2372e+6 (1.14e+6) $-$	1.7604e+7 (3.09e+5) $\approx$	1.8118e+7 (5.00e+5) $\approx$	2.0128e+7 (4.54e+4) $+$	1.7593e+7 (4.07e+5)
	12	4.8911e+12 (2.26e+11) $\approx$	1.8565e+12 (2.15e+11) $-$	5.2406e+12 (1.89e+10) $+$	5.2062e+12 (5.55e+10) $+$	5.8623e+12 (4.03e+10) $+$	4.8963e+12 (1.36e+11)
	16	5.6670e+18 (4.82e+16) $-$	1.4989e+18 (5.65e+17) $-$	5.5401e+18 (2.02e+16) $-$	5.8353e+18 (2.78e+16) $\approx$	5.7934e+18 (7.61e+15) $\approx$	5.8930e+18 (2.41e+16)
	20	1.6034e+25 (4.89e+22) $-$	6.5483e+24 (2.04e+24) $-$	1.5618e+25 (4.75e+22) $-$	1.6268e+25 (7.23e+22) $-$	1.6911e+25 (1.63e+22) $\approx$	1.7213e+25 (7.32e+22)
$+/-/\approx$		4/18/23	0/40/5	6/26/13	8/19/18	19/10/16

‘ $+$ ’, ‘ $-$ ’ and ‘ $\approx$ ’ indicate that the result is significantly better, significantly worse and statistically similar to that obtained by ISPEA/R, respectively.

performs significantly better than SPEA/R, dMOPSO, VaEA, NSGA-III and PICEA-g in 21, 21, 18, 19 and 22 test instances, while SPEA/R, dMOPSO, VaEA, NSGA-III and PICEA-g perform significantly better than ISPEA/R on only 11, 14, 20, 15 and 8 test instances, respectively. For HV measure, ISPEA/R performs much better than 5 compared algorithms, especially for problems with more than 10 objectives. From the results of Wilcoxon rank sum test in the last line of Table 7, we can see that ISPEA/R performs significantly better than SPEA/R, dMOPSO, VaEA, NSGA-III and PICEA-g in 23, 28, 20, 19 and 18 test instances, respectively, while SPEA/R, dMOPSO, VaEA, NSGA-III and PICEA-g perform significantly better than ISPEA/R only in 8, 10, 17, 18 and 14 test instances, respectively.

From the above statistical results of six algorithms on MaF test suite, we can find that the overall performance of ISPEA/R is the best among the six compared algorithms, and the performance of ISPEA/R is much better than that of 5 compared algorithms on problems with more than 10 objectives. Also, ISPEA/R is more effective to deal with hard MaOPs which are of concave PFs combining with multi-modal, degenerate and biased PFs, respectively, and of mixed (concave and convex) PFs combining with disconnected, multi-modal and biased PFs, respectively, especially for problems with a large number of objectives.

3.7 Performance comparisons on WFG test suite

By introducing a series of composable complexities (such as non-separability, multi-modality, biased parameters, and mixed PF geometries), the WFG test suite poses a significant challenge for algorithms to obtain a well-converged and well-distributed solution set. Tables 8 and 9 present the comparison results of ISPEA/R with other five algorithms in terms of IGD and HV values on the 45 test instances of the nine WFG test problems. The best results are highlighted in bold. For IGD measure, from the statistical results in Table 8, we can see that, among the 45 test instances, ISPEA/R got 14 best results, while SPEA/R, dMOPSO, VaEA, NSGA-III and PICEA-g got 9, 0, 9, 12 and 1 best results, respectively. For 27 instances with more than 10 objectives, ISPEA/R got 11 best results, while SPEA/R, dMOPSO, VaEA, NSGA-III and PICEA-g got 5, 0, 2, 9 and 0 best results, respectively. Thus, the overall performance of ISPEA/R is the best among 6 compared algorithms, especially for problems with more than 10 objectives. From the statistical results of Wilcoxon rank sum test in the last line of Table 8, we can see that ISPEA/R is significantly better than SPEA/R, dMOPSO, VaEA, NSGA-III and PICEA-g in 16, 43, 24, 21 and 40 test instances, respectively, while SPEA/R, dMOPSO, VaEA, NSGA-III and PICEA-g are significantly better than ISPEA/R in 6, 0, 12, 11 and 2 test instances, respectively. Thus, for IGD measure, ISPEA/R performs better than other 5 compared algorithms, and for problems with more than 10 objectives, the performance of ISPEA/R is much better.

For HV measure, we can see from Table 9 that among the 45 test instances, ISPEA/R got 19 best results, while SPEA/R, dMOPSO, VaEA, NSGA-III and PICEA-g got 3, 0, 0, 1 and 22 best results, respectively. For 27 instances with more than 10 objectives, ISPEA/R got 11 best results, while SPEA/R, dMOPSO, VaEA, NSGA-III and PICEA-g got 1, 0, 0, 1 and 14 best results, respectively. Thus, ISPEA/R performs a little bit worse than PICEA-g and much better than SPEA/R, dMOPSO, VaEA and NSGA-III for HV measure. Thus, the overall performance of ISPEA/R is obviously better than SPEA/R, dMOPSO, VaEA and NSGA-III for HV measure, especially on problems with more than 10 objectives. We can see from the statistical results of Wilcoxon rank sum test in the last line of Table 9 that ISPEA/R performs much better than 5 compared algorithms in 18, 40, 26, 19 and 10 test instances, respectively, while SPEA/R, dMOPSO, VaEA, NSGA-III and PICEA-g are significantly better than ISPEA/R in 4, 0, 6, 8 and 19 test instances, respectively. We can get the similar conclusion that ISPEA/R performs obviously better than SPEA/R, dMOPSO, VaEA and NSGA-III and a little bit worse than PICEA-g for HV measure.

To further compare ISPEA/R with PICEA-g, we take a look at Table 8 for IGD measure. PICEA-g only got 1 best results while ISPEA/R got 14 best results for IGD measure, and for instances with more than 10 objectives, PICEA-g got non best results while ISPEA/R got 11 best results for IGD measure. PICEA-g got the worse IGD results and the better HV results indicates that PICEA-g can only obtain a few high quality solutions which can result in good HV results, but it cannot get a lot of high quality and well distributed solutions which will result in good IGD results. Therefore, the overall performance of ISPEA/R is better than that of PICEA-g.

Figure 4.

Parallel coordinates of final solutions obtained by six algorithms on the 16-objective WFG5 instance.

Figure 5.

Parallel coordinates of final solutions obtained by ISPEA/R on WFG4 with (a) 30, (b) 40 and (c) 50 objectives.

To illustrate this further, Fig. 4 displays the parallel coordinates of final solutions obtained by ISPEA/R and other five compared algorithms on 16-objective WFG5 instance. From these six plots, we can see that the approximate PFs obtained by ISPEA/R, SPEA/R and NSGA-III show better uniformity and promising convergence on this test instance, followed by PICEA-g and dMOPSO, which is coincident with the statistical results in Tables 8 and 9.

The above experimental studies show that ISPEA/R has great potential for WFG test suite, followed by SPEA/R. The experiments also show that PICEA-g can get a few high quality solutions which can result in a good HV measure, but cannot get a lot of high quality and well distributed solutions because its performance for IGD measure is very poor. Furthermore, ISPEA/R can not only improve the performance of SPEA/R, but also balance the diversity and convergence better, especially for problems with a larger number of objectives.

3.8 Performance of ISPEA/R on problems with more objectives

To demonstrate the efficiency and effectiveness of ISPEA/R and the proposed convergence-diversity balanced fitness assignment scheme for MaOPs, we test ISPEA/R on MaOPs with more objectives. With the number of objectives becoming larger and larger, nearly all solutions in the population will become the nondominated solutions, and general fitness assignment scheme will face the difficulty to distinguish the quality of solutions and thus become ineffective. To test the effectiveness of the proposed fitness assignment scheme and the efficiency of ISPEA/R, we test ISPEA/R on WFG4 with 30, 40 and 50 objectives, respectively. For these three test instances, the population size is set to 495, 860 and 1325, respectively, and the maximum function evaluations (MFEs) are set to 247500, 430000 and 662500 for 30, 40 and 50 objectives, respectively.

Figure 5 shows the parallel coordinates of final solutions obtained by ISPEA/R for the three instances. It can be seen from Fig. 5 that for problems with 30, 40 and 50 objectives, ISPEA/R can still obtain a set of convergence-diversity balanced solutions. This demonstrate the effectiveness of the proposed fitness assignment scheme and the efficiency of ISPEA/R.

4. Application to a water resource management problem

One important engineering problem, namely water resource management problem for urban drainage planning and design [17, 25, 18], is used to test the performance of the proposed ISPEA/R on solving real-world engineering problems.

4.1 Problem description

The water resource management problem is an optimal planning problem for storm-drainage systems in urban areas. For a particular sub-basin within a watershed, suppose that it is hydrologically independent of other sub-basins and has its own treatment plant, on-site detention storage facility, drainage network and tributary to a receiving water body. The main idea of this problem is to examine the sub-basin in terms of its storm drainage needs, where three decision variables are considered in the sub-basin’s storm drainage system. Figure 6 presents the design variables (denoted by $x_{1},x_{2},x_{3}$ ) of the water resource management problem as well as how they conceptually interlink, where $x_{1}$ represents local detention storage capacity (basin $\times$ inches), $x_{2}$ represents maximum treatment rate (basin $\times$ inches/hour), and $x_{3}$ represents maximum allowable overflow rate (basin $\times$ inches/hour).

The optimization problem consists of finding the values of these three decision variables such that the five objectives in the following are minimized simultaneously:

1.
Drainage network cost
2.
Storage facility cost
3.
Treatment facility cost
4.
Expected flood damage cost
5.
Expected economic loss due to flooding

The mathematical formulation of the five-objective water resource management problem is given as follows [17]:

$\displaystyle\min\left\{\begin{array}[]{ll}f_{1}(x)=&106780.37\times(x_{2}+x_{% 3})\\ &{}+61704.67\\ f_{2}(x)=&3000\times x_{1}\\ f_{3}(x)=&(305700\times 2289x_{2})/\\ &(0.06\times 2289)^{0.65}\\ f_{4}(x)=&250\times 2289\times\exp(-39.75x_{2}\\ &{}+9.9x_{3}+2.74)\\ f_{5}(x)=&25\times((1.39/x_{1}x_{2})+4940x_{3}\\ &{}-80)\\ \end{array}\right.$

subject to

$\displaystyle\begin{array}[]{ll}g_{1}(x)=&(0.00139/(x_{1}x_{2})+4.94x_{3}-0.08% \leqslant 1\\ g_{2}(x)=&(0.000306/(x_{1}x_{2})+1.082x_{3}-0.0986\\ &{}\leqslant 1\\ g_{3}(x)=&(12.307/(x_{1}x_{2})+49408.24x_{3}\\ &{}+4051.02\leqslant 50000\\ g_{4}(x)=&(2.098/(x_{1}x_{2})+8046.33x_{3}-696.71\\ &{}\leqslant 16000\\ g_{5}(x)=&(2.138/(x_{1}x_{2})+7883.39x_{3}-705.04\\ &{}\leqslant 10000\\ g_{6}(x)=&(0.417/(x_{1}x_{2})+1721.26x_{3}-136.54\\ &{}\leqslant 2000\\ g_{7}(x)=&(0.164/(x_{1}x_{2})+631.13x_{3}-54.48\\ &{}\leqslant 550\\ \end{array}$

where $0.01\leqslant x_{1}\leqslant 0.45,0.01\leqslant x_{2}\leqslant 0.1,0.01% \leqslant x_{3}\leqslant 0.1$ .

Figure 6.
Conceptualization of a subbasin’s storm drainage system.

Table 10
Statistical results (mean and standard deviation) of the IGD and Spacing values obtained by six algorithms for water resource management problem

SPEA/R dMOPSO VaEA NSGA-III PICEA-g ISPEA/R

IGD Best 1.9759e+6 ( $-$ ) 1.9181e+6 ( $\approx$ ) 1.9169e+6 ( $\approx$ ) 1.9186e+6 ( $\approx$ ) 1.9271e+6 ( $\approx$ ) 1.9167e+6

Mean 1.9870e+6 ( $-$ ) 1.9187e+6 ( $+$ ) 1.9370e+6 ( $-$ ) 1.9272e+6 ( $\approx$ ) 1.9497e+6 ( $-$ ) 1.9244e+6

Std 3.5012e+4 ( $-$ ) 3.6481e-1 ( $+$ ) 5.8049e+2 ( $+$ ) 1.1021e+3 ( $+$ ) 6.0335e+3 ( $-$ ) 3.1012e+3

Spacing Best 9.2079e+4 ( $-$ ) 1.1733e+5 ( $-$ ) 6.8354e+4 ( $-$ ) 4.1660e+4 ( $\approx$ ) 4.7305e+4 ( $-$ ) 3.9400e+4

Mean 1.2445e+5 ( $-$ ) 1.1864e+5 ( $-$ ) 1.2609e+5 ( $-$ ) 6.5807e+4 ( $-$ ) 6.4846e+4 ( $-$ ) 6.2246e+4

Std 8.9523e+4 ( $-$ ) 1.1454e+3 ( $+$ ) 8.4421e+4 ( $-$ ) 2.0809e+4 ( $+$ ) 3.9346e+4 ( $+$ ) 5.1637e+4

‘ $+$ ’, ‘ $-$ ’ and ‘ $\approx$ ’ indicate that the result is significantly better, significantly worse and statistically similar to that obtained by ISPEA/R, respectively.

4.2 Experimental results

		SPEA/R	dMOPSO	VaEA	NSGA-III	PICEA-g	ISPEA/R
IGD	Best	1.9759e+6 ( $-$ )	1.9181e+6 ( $\approx$ )	1.9169e+6 ( $\approx$ )	1.9186e+6 ( $\approx$ )	1.9271e+6 ( $\approx$ )	1.9167e+6
	Mean	1.9870e+6 ( $-$ )	1.9187e+6 ( $+$ )	1.9370e+6 ( $-$ )	1.9272e+6 ( $\approx$ )	1.9497e+6 ( $-$ )	1.9244e+6
	Std	3.5012e+4 ( $-$ )	3.6481e-1 ( $+$ )	5.8049e+2 ( $+$ )	1.1021e+3 ( $+$ )	6.0335e+3 ( $-$ )	3.1012e+3
Spacing	Best	9.2079e+4 ( $-$ )	1.1733e+5 ( $-$ )	6.8354e+4 ( $-$ )	4.1660e+4 ( $\approx$ )	4.7305e+4 ( $-$ )	3.9400e+4
	Mean	1.2445e+5 ( $-$ )	1.1864e+5 ( $-$ )	1.2609e+5 ( $-$ )	6.5807e+4 ( $-$ )	6.4846e+4 ( $-$ )	6.2246e+4
	Std	8.9523e+4 ( $-$ )	1.1454e+3 ( $+$ )	8.4421e+4 ( $-$ )	2.0809e+4 ( $+$ )	3.9346e+4 ( $+$ )	5.1637e+4

The proposed ISPEA/R has been tested to solve the water resource management problem and its performance has been compared with those of SPEA/R [24], dMOPSO [38], VaEA [62], NSGA-III [13] and PICEA-g [59]. In the experiment, both the population size $N$ and the number of reference directions are set to 210, the MFEs is set to 63000, and all the compared algorithms are independently run 20 times. We use two indicators IGD [73] and Spacing [49] to assess the performance of the compared algorithms on the water resource management problem.

For the constraints of this problem, we adopt Deb’s constraint handling strategy [9]: for two solutions $p$ and $q$ , $p$ is said to constraint-dominate $q$ if one of the following conditions holds:

•
$p$ is feasible and $q$ is infeasible.
•
both $p$ and $q$ are feasible and $p$ dominates $q$ .
•
both $p$ and $q$ are infeasible and $p$ has a smaller overall constraint violation value.

The statistical results (best, mean and standard deviation) of the IGD and Spacing values obtained by the six compared algorithms are given in Table 10. For IGD measure, it can be clearly seen that ISPEA/R obtains the best IGD value among the six compared algorithms, where the result obtained by ISPEA/R is better than those obtained by dMOPSO, NSGA-III, VaEA and PICEA-g at least $e+1$ , and much better than that obtained by SPEA/R. For the mean IGD value, ISPEA/R performs much better than SPEA/R, VaEA and PICEA-g, little bit better than NSGA-III, but worse than dMOPSO. The statistical results of mean values of Wilcoxon rank sum test also indicate ISPEA/R performs significantly better than SPEA/R, VaEA and PICEA-g, similarly to NSGA-III, but significantly worse than dMOPSO, however, the statistical results of best values of Wilcoxon rank sum test indicate ISPEA/R performs similarly to dMOPSO. Thus, the overall performance of ISPEA/R for IGD measure is best among all 6 compared algorithms. This indicates that ISPEA/R can better balance the convergence and diversity than other five compared algorithms.

For Spacing measure, we can see from Table 10 that ISPEA/R performs best with respect to both the best values and the best mean values. For best values, the result obtained by ISPEA/R is better than those obtained by the compared algorithms at least 2260, and for the best mean, the result obtained by ISPEA/R is better than those obtained by the compared algorithms at least 2600. The statistical results of best values of Wilcoxon rank sum test indicate ISPEA/R is significantly better than SPEA/R, dMOPSO, VaEA and PICEA-g and similar to NSGA-III. The statistical results of best mean values of Wilcoxon rank sum test indicate ISPEA/R is significantly better than all other 5 compared algorithms.

It can be seen from the analysis to the experimental results on this real world problem that ISPEA/R performs better than 5 compared algorithms, can well balance both convergence and diversity of the solution set, and can obtain a set of uniformly distributed solutions.
5. Conclusion

To overcome the shortcomings of fitness assignment scheme in the recent proposed algorithm SPEA/R [24] and propose an efficient algorithm for MaOPs, in this paper, a new fitness evaluation mechanism-based evolutionary algorithm, called ISPEA/R, has been proposed. The new fitness evaluation mechanism in ISPEA/R properly integrates the convergence and diversity of solutions so that it can better balance the convergence and diversity in evaluating the quality of solutions. Furthermore, the proposed convergence measure can better cooperate the selection scheme, and the diversity measure can more accurately measure each solution’ contribution to the diversity.

To demonstrate the strong competitiveness of ISPEA/R, we compare ISPEA/R with other five state-of-the-art many-objective evolutionary algorithms by experiments on 120 test instances of 24 test problems from three test suites DTLZ, WFG and MaF with up to 20 objectives and on a real world problem with 5 objectives. Experimental results have shown the potential of ISPEA/R to get a set of well distributed solutions and the competitiveness and effectiveness of ISPEA/R in keeping a good balance between convergence and diversity of solutions.

However, from these experimental results, we can also find some deficiency of ISPEA/R, it seems that it is ineffective to the pure concave problem. This is an important issue to be further studied in the future. In addition, as discussed in [16], the performance of an evolutionary many-objective optimization algorithm might degenerate with the increase of number of decision variables. Therefore, it is also worthwhile to investigate the scalability of ISPEA/R for large-scale problems [35, 37].

Footnotes

Acknowledgments

This work was supported by National Natural Science Foundation of China (No. 61872281 and 614722 97).

References

Adeli

, and Kumar

Distributed Computer-Aided Engineering: For Analysis, Design, and Visualization. CRC Press, Boca Raton, Florida, 1998.

Adeli

, and Sarma

K.C.

Cost Optimization of Structures: Fuzzy Logic, Genetic Algorithms, and Parallel Computing. John Wiley and Sons, West Sussex, United Kingdom, 2006.

Almeida

M.A.

, and Pedrino

E.C.

Hybrid evolvable hardware for automatic generation of image filters. Integrated Computer-Aided Engineering, 25(3): 289–303, 2018.

Bader

, and Zitzler

Hype: an algorithm for fast hypervolume-based many-objective optimization. Evolutionary Computation, 19(1): 45–76, 2014.

Bentley

P.J.

, and Wakefield

J.P.

Finding acceptable solutions in the pareto-optimal range using multiobjective genetic algorithms. In Soft Computing in Engineering Design and Manufacturing, pages 231–240. Springer, London, 1998.

Cheng

Jin

Olhofer

, et al. A reference vector guided evolutionary algorithm for many-objective optimization. IEEE Transactions on Evolutionary Computation, 20(5): 773–791, 2016.

Dai

Wang

, and Ye

A new evolutionary algorithm based on contraction method for many-objective optimization problems. Applied Mathematics and Computation, 245: 191–205, 2014.

Das

, and Dennis

J.E.

Normal-boundary intersection: A new method for generating the pareto surface in nonlinear multicriteria optimization problems. Siam Journal on Optimization, 8(3): 631–657, 2006.

Deb

An efficient constraint handling method for genetic algorithms. Computer Methods in Applied Mechanics and Engineering, 186(2): 311–338, 2000.

10.

Deb

, and Agrawal

R.B.

Simulated binary crossover for continuous search space. Complex Systems, 9(3): 115–148, 1994.

11.

Deb

, and Goyal

A combined genetic adaptive search (geneas) for engineering design. Computer Science and Informatics, 26(4): 30–45, 1996.

12.

Deb

Gupta

Daum

, et al. Reliability-based optimization using evolutionary algorithms. IEEE Transactions on Evolutionary Computation, 13(5): 1054–1074, 2009.

13.

Deb

, and Jain

An evolutionary many-objective optimization algorithm using reference-point-based nondominated sorting approach, part i: Solving problems with box constraints. IEEE Transactions on Evolutionary Computation, 18(4): 577–601, 2014.

14.

Deb

Pratap

Agarwal

, et al. A fast and elitist multiobjective genetic algorithm: Nsga-ii. IEEE Transactions on Evolutionary Computation, 6(2): 182–197, 2002.

15.

Deb

Thiele

Laumanns

, et al. Scalable multi-objective optimization test problems. In Proceedings of the 2002 Congress on Evolutionary Computation. CEC’02, Honolulu, HI, USA, USA, 12–17 May 2002, volume 1, pages 825–830. IEEE, 2002.

16.

Durillo

J.J.

Nebro

A.J.

Coello

C.A.C.

, et al. A study of multiobjective metaheuristics when solving parameter scalable problems. IEEE Transactions on Evolutionary Computation, 14(4): 618–635, 2010.

17.

Elarbi

Bechikh

Gupta

, et al. A new decomposition-based nsga-ii for many-objective optimization. IEEE Transactions on Systems Man and Cybernetics Systems, 48(7): 1191–1210, 2018.

18.

Elarbi

Bechikh

, and Said

L.B.

On the importance of isolated infeasible solutions in the many-objective constrained nsga-iii. Knowledge-Based Systems, 2018, doi: 10.1016/j.knosys.2018.05.015.

19.

Gutierrez Soto

, and Adeli

Many-objective control optimization of high-rise building structures using replicator dynamics and neural dynamics model. Structural and Multidisciplinary Optimization, 56(6): 1521–1537, 2017.

20.

Gutierrez Soto

, and Adeli

Vibration control of smart base-isolated irregular buildings using neural dynamic optimization model and replicator dynamics. Engineering Structures, 156: 322–336, 2018.

21.

Hadka

, and Reed

Borg: an auto-adaptive many-objective evolutionary computing framework. Evolutionary Computation, 21(2): 231–259, 2013.

22.

Huband

Hingston

Barone

, et al. A review of multiobjective test problems and a scalable test problem toolkit. IEEE Transactions on Evolutionary Computation, 10(5): 477–506, 2006.

23.

Ikeda

Kita

, and Kobayashi

Failure of pareto-based moeas: Does non-dominated really mean near to optimal? In Evolutionary Computation, 2001. Proceedings of the 2001 Congress on, volume 2, pages 957–962. IEEE, Seoul, South Korea, 2001.

24.

Jiang

, and Yang

A strength pareto evolutionary algorithm based on reference direction for multiobjective and many-objective optimization. IEEE Transactions on Evolutionary Computation, 21(3): 329–346, 2017.

25.

Musselman

, and Talavage

A tradeoff cut approach to multiple objective optimization. Operations Research, 28(6): 1424–1435, 1980.

26.

Kabir

A.M.

Langsfeld

J.D.

Kaipa

K.N.

, et al. Identifying optimal trajectory parameters in robotic finishing operations using minimum number of physical experiments. Integrated Computer-Aided Engineering, 25(2): 111–135, 2018.

27.

Kasprzyk

J.R.

Reed

P.M.

Kirsch

B.R.

, et al. Managing population and drought risks using many-objective water portfolio planning under uncertainty. Water Resources Research, 45(12): 170–170, 2009.

28.

Kociecki

, and Adeli

Two-phase genetic algorithm for size optimization of free-form steel space-frame roof structures. Journal of Constructional Steel Research, 90: 283–296, 2013.

29.

Kociecki

, and Adeli

Two-phase genetic algorithm for topology optimization of free-form steel space-frame roof structures with complex curvatures. Engineering Applications of Artificial Intelligence, 32: 218–227, 2014.

30.

Kociecki

, and Adeli

Shape optimization of free-form steel space-frame roof structures with complex geometries using evolutionary computing. Engineering Applications of Artificial Intelligence, 38: 168–182, 2015.

31.

, and Zhang

Multiobjective optimization problems with complicated pareto sets, moea/d and nsga-ii. IEEE Transactions on Evolutionary Computation, 13(2): 284–302, 2009.

32.

Deb

Zhang

, et al. An evolutionary many-objective optimization algorithm based on dominance and decomposition. IEEE Transactions on Evolutionary Computation, 19(5): 694–716, 2015.

33.

Schonfeld

, et al. Mountain railway alignment optimization with bidirectional distance transform and genetic algorithm. Computer-Aided Civil and Infrastructure Engineering, 32(8): 691–709, 2017.

34.

Liao

Yang

, et al. Multi-objective optimization for crash safety design of vehicles using stepwise regression model. Chinese Journal of Mechanical Engineering, 35(6): 561–569, 2008.

35.

Liu

Wang

Liu

, et al. A two phase hybrid algorithm with a new decomposition method for large scale optimization. Integrated Computer-Aided Engineering, 25(4): 349–367, 2018.

36.

Liu

H.L.

, and Zhang

Decomposition of a multiobjective optimization problem into a number of simple multiobjective subproblems. IEEE Transactions on Evolutionary Computation, 18(3): 450–455, 2014.

37.

Liu

, et al. A multiobjective evolutionary algorithm based on decision variable analyses for multiobjective optimization problems with large-scale variables. IEEE Transactions on Evolutionary Computation, 20(2): 275–298, 2016.

38.

Martinez

S.Z.

, and Coello

C.A.C.

A multi-objective particle swarm optimizer based on decomposition. In Proceedings of the 13th Annual Conference on Genetic and Evolutionary Computation, GECCO ’11, Dublin, Ireland, July 12–16, 2011, pages 69–76. ACM New York, NY, USA, 2011.

39.

Padillo

Luna

Herrera

, et al. Mining association rules on big data through mapreduce genetic programming. Integrated Computer-Aided Engineering, 25(1): 31–48, 2018.

40.

Pan

Tian

, et al. A region division based diversity maintaining approach for many-objective optimization. Integrated Computer-Aided Engineering, 24(3): 279–296, 2017.

41.

Price

Storn

R.M.

, and Lampinen

J.A.

Differential Evolution: A Practical Approach to Global Optimization (Natural Computing Series). Springer-Verlag New York, Inc., 2005.

42.

Ran Cheng

M.L.

, and Tian

Benchmark functions for the cec’2018 competition on evolutionary many-objective optimization. Technical Report No. CSR-17-01, School of Computer Science, University of Birmingham, U.K., January 2017.

43.

Rostami

, and Neri

Covariance matrix adaptation pareto archived evolution strategy with hypervolume-sorted adaptive grid algorithm. Integrated Computer-Aided Engineering, 23(4): 313–329, 2016.

44.

Rostami

, and Neri

A fast hypervolume driven selection mechanism for many-objective optimisation problems. Swarm and Evolutionary Computation, 34: 50–67, 2017.

45.

Rostami

Neri

, and Epitropakis

M.G.

Progressive preference articulation for decision making in multi-objective optimisation problems. Integrated Computer-Aided Engineering, 24(4): 315–335, 2017.

46.

Sarma

K.C.

, and Adeli

Bilevel parallel genetic algorithms for optimization of large steel structures. Computer-aided Civil and Infrastructure Engineering, 16(5): 295–304, 2001.

47.

Sarma

K.C.

, and Adeli

Data parallel fuzzy genetic algorithm for cost optimization of large space steel structures. International Journal of Space Structures, 18(3): 195–205, 2003.

48.

Sato

Aguirre

H.E.

, and Tanaka

Controlling dominance area of solutions and its impact on the performance of moeas. In International Conference on Evolutionary Multi-criterion Optimization, pages 5–20. Springer, Berlin, Heidelberg, 2007.

49.

Schott

J.R.

Fault tolerant design using single and multicriteria genetic algorithm optimization. Cellular Immunology, 37(1): 1–13, 1995.

50.

Siddique

, and Adeli

Computational Intelligence: Synergies of Fuzzy Logic, Neural Networks and Evolutionary Computing. John Wiley and Sons Inc, West Sussex, United Kingdom, 2013.

51.

Sousa

Al çada Almeida

, and Coutinho-Rodrigues

J.a.

Bi-objective modeling approach for repairing multiple feature infrastructure systems. Computer-Aided Civil and Infrastructure Engineering, 32(3): 213–226, 2017.

52.

, et al. Shape generation of grid structures by inverse hanging method coupled with multiobjective optimization. Computer-Aided Civil and Infrastructure Engineering, 33(6): 498–509, 2018.

53.

Sülflow

Drechsler

, and Drechsler

Robust multi-objective optimization in high dimensional spaces. In International Conference on Evolutionary Multi-criterion Optimization, pages 715–726. Springer-Verlag, Berlin, Heidelberg, 2007.

54.

Taillandier

Fernandez

, and Ndiaye

Real estate property maintenance optimization based on multi-objective multidimensional knapsack problem. Computer-Aided Civil and Infrastructure Engineering, 32(3): 227–251, 2017.

55.

Tian

Cheng

Zhang

, et al. Platemo: A matlab platform for evolutionary multi-objective optimization [educational forum]. IEEE Computational Intelligence Magazine, 12(4): 73–87, 2017.

56.

Valenzuela

Jiang

Carrillo

, et al. Multi-objective genetic algorithms to find most relevant volumes of the brain related to alzheimer’s disease and mild cognitive impairment. International Journal of Neural Systems, 28(9): 1850007(1–23), 2018.

57.

Wang

Zhang

, and Zhang

Cooperative differential evolution with multiple populations for multiobjective optimization. IEEE Transactions on Cybernetics, 46(12): 2848–2861, 2016.

58.

Wang

Liu

H.-L.

Yuan

, et al. Optimizing the energy-spectrum efficiency of cellular systems by evolutionary multi-objective algorithm. Integrated Computer-Aided Engineering, 25(4): 1–14, 2018.

59.

Wang

Purshouse

R.C.

, and Fleming

P.J.

Preference-inspired coevolutionary algorithms for many-objective optimization. IEEE Transactions on Evolutionary Computation, 17(4): 474–494, 2013.

60.

Wang

, and Szeto

W.Y.

Multiobjective environmentally sustainable road network design using pareto optimization. Computer-Aided Civil and Infrastructure Engineering, 32(11): 964–987, 2017.

61.

Wright

, and Jordanov

Quantum inspired evolutionary algorithms with improved rotation gates for real-coded synthetic and real world optimization problems. Integrated Computer-Aided Engineering, 24(3): 203–223, 2017.

62.

Xiang

Zhou

, et al. A vector angle-based evolutionary algorithm for unconstrained many-objective optimization. IEEE Transactions on Evolutionary Computation, 21(1): 131–152, 2017.

63.

Yang

Emmerich

Back

, et al. Multi-objective inventory routing with uncertain demand using population-based metaheuristics. Integrated Computer-Aided Engineering, 23(3): 205–220, 2016.

64.

Yuan

, and Wang

An improved nsga-iii procedure for evolutionary many-objective optimization. In 16th Genetic and Evolutionary Computation Conference (GECCO), Vancouver, CANADA, JUL 12–16, 2014, pages 661–668, 2014.

65.

Yuan

Wang

, et al. A new dominance relation-based evolutionary algorithm for many-objective optimization. IEEE Transactions on Evolutionary Computation, 20(1): 16–37, 2016.

66.

Wang

Moran

, et al. Multi-objective path optimization for critical infrastructure links with consideration to seismic resilience. Computer-Aided Civil and Infrastructure Engineering, 32(10): 836–855, 2017.

67.

Zhang

, and Li

Moea/d: A multiobjective evolutionary algorithm based on decomposition. IEEE Transactions on Evolutionary Computation, 11(6): 712–731, 2007.

68.

Zhang

Tian

, and Jin

A knee point-driven evolutionary algorithm for many-objective optimization. IEEE Transactions on Evolutionary Computation, 19(6): 761–776, 2015.

69.

Zhao

Guo

Zhou

, et al. A quantum-inspired genetic algorithm-based optimization method for mobile impact test data integration. Computer-Aided Civil and Infrastructure Engineering, 33(5): 411–422, 2018.

70.

Zhu

Xiao

, et al. Global path planning of wheeled robots using a multi-objective memetic algorithm. Integrated Computer-Aided Engineering, 22(4): 387–404, 2015.

71.

Zitzler

Laumanns

, and Thiele

Spea2: Improving the strength pareto evolutionary algorithm for multiobjective optimization. In Evolutionary Methods for Design, Optimization and Control with Applications To Industrial Problems. Athens. Greece, September 19–21, pages 95–100, 2001.

72.

Zitzler

, and Thiele

Multiobjective evolutionary algorithms: a comparative case study and the strength pareto approach. IEEE Transactions on Evolutionary Computation, 3(4): 257–271, 1999.

73.

Zitzler

Thiele

Laumanns

, et al. Performance assessment of multiobjective optimizers: An analysis and review. IEEE Transactions on Evolutionary Computation, 7(2): 117–132, 2003.

No. of objectives (m)	Divisions (H)	Population size (N)
5	6	210
8	(4, 3)	450
12	(3, 3)	728
16	(3, 2)	952
20	(3, 1)	1560

A convergence-diversity balanced fitness evaluation mechanism for decomposition- based many-objective optimization algorithm

Abstract

Keywords

1. Introduction

2.1 General framework

2.2 Member association

2.3.1 Fitness assignment in SPEA/R

2.4.1 Similarity

2.4.2 Differences

3.1 Benchmark test problems

Table 2 Features of the test problems

3.6 Performance comparisons on MaF test suite

Table 4 Statistical results (mean and standard deviation) of the IGD values obtained by six algorithms for DTLZ problems

4. Application to a water resource management problem

4.1 Problem description

Footnotes

Acknowledgments

References

Table 2
Features of the test problems

Table 4
Statistical results (mean and standard deviation) of the IGD values obtained by six algorithms for DTLZ problems