Research of Strategies of Maintaining Population Diversity for MOEA/D Algorithm

Abstract

In recent years, MOEA/D algorithm has been recognized by the industry for its inherent advantages in dealing with super multi objective optimization problems, and its application is also very extensive. However, MOEA/D algorithm also has the problem of lack of population diversity during the later stage of evolution, resulting in slow convergence speed. In this paper, it makes a research on the strategy of maintaining population diversity based on MOEA/D algorithm and proposes three population diversity maintenance strategies, namely SBX-DE operator competition, mutation probability adaptive modulation, and double-faced mirrors theory boundary processing. The experiments’ result shows that all of these three strategies can effectively improve the diversity of the MOEA/D algorithm in the late evolutionary population, and contribute to the convergence speed of the MOEA/D algorithm.

1 Introduction

Multi Objective Optimization Problems (MOPs) play an important role in scientific research, production practice, and engineering applications. The research on MOP issues has important theoretical and academic value. Compared with single-objective optimization, the goal of MOP is no longer a single optimal solution, but a set of mutually constrained solution sets, that is, the improvement of one optimization target may be accompanied by the deterioration of other optimization goals. To this end, the academic community generally uses the Pareto solution set to represent a set of compromise solution sets of various objectives in multi-objective optimization, and the projection of Pareto optimal solution sets in the target domain is called Pareto Front (PF) [1].

The early multi objective evolutionary algorithms are evolutionary algorithms based on the Pareto dominance relationship, such as SPEA [2], SPEA-II [3], NSGA-II [4], PESA [5], PESA-II [6] and so on. Although these evolutionary algorithms have achieved remarkable results in the field of multi-objective optimization, they face challenges when facing the problem of ultra-multi-objective optimization (the number of objectives is greater than or equal to 3). In 2007, Zhang [7] and others proposed a decomposition-based multi-objective evolutionary algorithm (MOEA/D), which decomposes multi-objective optimization problems into a set of single-objective optimization problems, and then uses evolutionary algorithm to solve these decomposition problems simultaneously. At the same time, MOEA/D algorithm also introduces neighbor relation to share evolution information. In addition, the MOEA/D algorithm is directly applicable to the super multi-objective optimization problem, and overcomes the limitations of the evolutionary algorithm based on the Pareto dominance relation. Therefore, the MOEA/D algorithm framework has become a research hotspot in recent years.

Although MOEA/D algorithm has made great progress in the field of multi-objective optimization, there are also some problems such as the lack of diversity and slow convergence rate in the late evolution. For this reason, scholars have proposed various improvement schemes of MOEA/D algorithm, mainly in the following aspects: improvement of weight vector, improvement of decomposition method, improvement of evolutionary operator, improvement of matching selection, improvement of replacement selection. However, these studies have not done much to solve the problem of population diversity loss in MOEA/D algorithm, and also lack systematic research programs for diversity loss.

In order to solve the problem of missing population diversity of MOEA/D algorithm, this paper systematically studied the population diversity maintenance strategy of MOEA/D algorithm, and carried out research on the improvement strategies of three aspects: one is to adopt a competitive employment evolution strategy combined with simulated binary crossover (SBX) [8] and Differential Evolution (DE) [9]. The second is to adopt the mutation probability adaptive adjustment strategy. The mutation probability will be adjusted for adaptive growth according to the diversity change of the population in the evolution process. At the same time, if the evolution falls into the stagnation state, the algorithm will temporarily increase the mutation probability for breaking through the stagnation state. Third, the boundary processing problem of double specular reflection principle and the problem of overcoming the boundary aggregation in the process of population evolution. Simulation experiments show that the above three population diversity maintenance strategies can enhance the diversity of MOEA/D algorithm populations in the late evolution stage and improve the evolution rate of MOEA/D algorithm.

2 Related Work

2.1 Multi Objective Optimization Problems and the Related Concepts

Taking the minimization of multi objective problems as an example, the MOP problem can be described as:

The decision variable $X = (x_{1} ,x_{2} , \ldots ,x_{n} )$ should satisfy the constraint:

$$ \left\{ {\begin{array}{*{20}c} {g_{i} (X) \ge 0} & {i = 1,2, \ldots ,k} \\ {h_{i} (X) = 0} & {i = 1,2, \ldots ,k} \\ \end{array} } \right. $$

(1)

With $m$ objective functions, the optimization objective is expressed as

$$ F(X) = (f_{1} (X),f_{2} (X), \ldots ,f_{n} (X))^{T} $$

(2)

Then the MOP problem is to find a set of solution sets $X^{*} = (x_{1}^{*} ,x_{2}^{*} , \ldots ,x_{n}^{*} )$ in the decision space to make $F(X^{*} )$ minimum under the premise of satisfying the constraints. On this basis, the following definitions are given:

• (1) Feasible solution set $X_{f}$: a set of decision variables $x$ satisfying constraint condition (2). That is $X_{f} = \left\{ {x \in X|g(x) \ge 0,h(x) = 0} \right\}$

• (2) Pareto dominance:

  Suppose $p$ and $q$ are any two different individuals in the evolutionary group, if the following relationship is satisfied: ① All sub-targets of $p$ are no worse than $q$, namely $f_{k} (p) \le f_{k} (q)(k = 1,2, \ldots ,n)$; (2) $p$ at least has one sub-goal that is better than $q$, namely $\exists l \in \{ 1,2, \ldots ,n\} ,f_{k} (p) < f_{k} (q)$, then $p$ dominates $q$, $p$ dominates and $q$ is dominated.

• (3) Pareto’s optimal solutions:

  Suppose $x^{*} \in X_{f}$, if there is no other solution $x'^{*} \in X_{f}$ to make $f_{i} (x'^{*} ) \le f_{i} (x^{*} )(i = 1,2, \ldots ,m)$, and at least one is a strict inequality, then $x^{*}$ is said to be the Pareto’s optimal solution of this MOP problem.

• (4) Pareto’s optimal solution set:

  The set of all Pareto optimal solutions of MOP problem is Pareto Set (PS).

• (5) Pareto Front:

  The projection of the Pareto optimal solution set on the target space is the Pareto Front (PF), namely $PF = \{ F(x)|x \in PS\}$.

2.2 Introduction to MOEA/D Algorithm

MOEA/D algorithm uses aggregation function to decompose MOPs into N subproblems, and optimizes these subproblems at the same time. The algorithm divides neighborhood for each subproblem by calculating Euclidean distance between weight vectors. Each subproblem realizes coevolution through neighborhood. MOEA/D algorithm flow is as follows:

• Step 1: Initialization

  • a) Initialize the weight vector $\lambda = \{ \lambda_{1} ,\lambda_{2} , \ldots ,\lambda_{N} \}$ and divide it into neighborhoods. Assuming the individual $x_{i} \in X$, then $B(i) = \{ i_{1} ,i_{2} , \ldots ,i_{T} \}$ is the neighborhood of the No. $i$ weight vector.

  • b) Initialize the population $pop = \{ x_{1} ,x_{2} , \ldots ,x_{N} \}$ and calculate its target value $FV_{i} = F(x_{i} )$.

  • c) Calculate the ideal point $z = \{ z_{1} ,z_{2} , \ldots ,z_{m} \}^{T}$, where $m$ is the target number and $z_{i}$ is the optimal value of $f_{i}$ in the current generation.

• Step 2: Update

  • a) Evolution: Firstly, two individuals $x_{j}$ and $x_{k}$ are randomly selected from the neighborhood $B(i)$, and a new individual $x_{d}$ is generated by using a simulated binary crossover (SBX) algorithm. Then, $x_{d}$ is subjected to polynomial mutation to obtain a mutant individual $y$, and finally, the mutation individual $y$ is improved to $y'$ according to the constraint of the problem.

  • b) Update the ideal point: for $j = 1,2, \ldots ,m$, if $f_{j} (y^{{\prime }} ) < z_{j}$, then $z_{j} = f_{j} (y^{{\prime }} )$.

  • c) Update the neighborhood: For $j \in B(i)$, if the aggregate function value of $y^{{\prime }}$ is not greater than $x_{{^{j} }}$, then $x_{{^{j} }} = y^{{\prime }}$, and the corresponding target value $FV_{{^{j} }} = F(y^{{\prime }} )$ is updated at the same time.

• Step 3: Stop iteration

  • If the stop condition is satisfied, stop iteration and output the best solution set; Otherwise, continue to Step 2.

3 Study on Population Diversity Maintenance Strategy of MOEA/D Algorithm

According to the above MOEA/D algorithm flow, the simulation experiment of the test function was carried out on the MOEA/D algorithm. The experiments show that it’s easy to have the problems of premature convergence and lack of diversity.

Taking the test function DTLZ3 as an example, MOEA/D algorithm is used to solve the evolution of the test function DTLZ3. The test function DTLZ3 is propagated for 200 generations. The average of the Pareto optimal solution sets of the three optimization targets of each generation is calculated to reflect the evolution degree of the MOEA/D algorithm. The variation curve of the average value of Pareto optimal solution sets is shown in Fig. 1. It can be seen from Fig. 1 that after the MOEA/D population has multiplied to 40 generations, the average reduction trend of Pareto optimal solution sets of the three optimization objectives of the algorithm is significantly slower, indicating that the evolution speed of the MOEA/D algorithm has slowed down.

Fig. 1.

Pareto optimal solution set mean curve of MOEA/D algorithm

The reason why MOEA/D algorithm’s evolution speed slows down is the lack of diversity after the population has multiplied to a certain extent, which leads to the algorithm falling into a local optimal solution. Variance can well reflect the diversity of the population. The larger the variance is, the better the diversity of the population, and vice versa. For this reason, the mean value of the variance of each dimension in each generation is further studied and counted, as shown in Fig. 2. It can be seen that the significant decrease of variance mean in all dimensions of the population is basically synchronous with the slow evolution of the algorithm, which also proves that the slow evolution of MOEA/D algorithm is indeed related to the lack of population diversity.

Fig. 2.

Population variance mean curve of MOEA/D algorithm

Fig. 3.

Pareto optimal solution set distribution of MOEA/D algorithm

Fig. 4.

Population variance mean curve of improved MOEA/D algorithm (SBX-DE competition induction strategy)

Fig. 5.

Pareto optimal solution set mean curve of improved MOEA/D algorithm (SBX-DE competition induction strategy)

Fig. 6.

Variation probability adaptive adjustment curve

The study also found that the Pareto optimal solution set was unevenly distributed in the simulation process of MOEA/D algorithm, which showed that the solution set was aggregated near the boundary while the other regions were sparsely distributed, as shown in Fig. 7. This will also affect the population diversity of MOEA/D algorithm.

Fig. 7.

Population variance mean curve of improved MOEA/D algorithm (adaptive adjustment strategy of mutation probability)

The reason of uneven distribution of Pareto optimal solution set of MOEA/D algorithm is that the algorithm adopts the following methods when dealing with boundary problems:

$$ y = \left\{ {\begin{array}{*{20}l} {X_{\hbox{min} } ,} \hfill & {y < X_{\hbox{min} } } \hfill \\ {X_{\hbox{max} } ,} \hfill & {y > X_{\hbox{max} } } \hfill \\ {y,} \hfill & {others} \hfill \\ \end{array} } \right. $$

(3)

Where $y$ is an new individual obtained by evolution, $X_{\hbox{min} }$ is its lower boundary limit, and $X_{\hbox{max} }$ is its upper boundary limit. This method will make all the trans boundary individuals to gather and project onto the boundary.

In order to solve the problem of lack of population diversity in MOEA/D algorithm, this paper tries three strategies to maintain population diversity.

3.1 The SBX-DE Operator Competition Strategy

SBX-DE operator competition strategy is that the algorithm starts to use DE algorithm to perform crossover operation, and counts the number of neighbor updates in each round of evolution in real time. When the number of neighbor updates is significantly reduced in five consecutive rounds of evolution, it indicates that the evolution speed is slowing down. SBX algorithm is used to replace DE algorithm to perform crossover operation at this time, and then this competition polling mechanism is used to alternately use the two evolutionary algorithms.

The following simulation and verification experiments are carried out. The DTLZ3 test function is taken as the test object, the number of decision variables is set to 12, the dimension of the problem is set to 3, and the population reproduction algebra is set to 200 generations. The variance mean curve of each dimension of MOEA/D algorithm population was obtained after adopting the SBX-DE operator competition strategy. As shown in Fig. 4, it can be seen that the population variance decrease smoothly, while it’s has a sudden decrease in Fig. 2, indicating that the SBX-DE operator competition strategy can enrich the diversity of the MOEA/D algorithm population.

After enriching the diversity of the population, in order to further study the influence of the strategy on the convergence speed of MOEA/D algorithm, the experiment continued to count the Pareto optimal solution set mean of the improved algorithm, and obtained the Pareto optimal solution set mean curve of the improved MOEA/D algorithm as shown in Fig. 5. Compared with Fig. 1, it can be seen that the Pareto optimal solution set mean continues to evolve to a better value after 40 generations of reproduction. It shows that SBX-DE operator competition strategy can improve the convergence speed of MOEA/D algorithm.

3.2 Adaptive Adjustment Strategy of Mutation Probability

Another solution to maintain population diversity is to increase the probability of mutation. The adaptive adjustment strategy of mutation probability is to make the individual mutation probability adapt to the evolution process, and the adjustment rule is to make the mutation probability change in a negative exponential growth law with the evolution process. The variation curve of mutation probability is shown in Fig. 6. The population diversity performs very well at the beginning of evolution process, and the mutation probability is low. But with the evolution process the population diversity decrease, which need to increase the mutation probability to improve it. However, the mutation probability should tend to be stable after reach a certain value to avoid the problem of reducing convergence speed caused by the high mutation probability. The mutation probability computation formula is as follows:

$$ Muta\_rate = 0.2*( - e^{{ - \frac{gen}{\chi }}} + 1) $$

(4)

Where $gen$ is the population evolution algebra and $\chi$ is the scaling factor, here set to 400.

In addition, by recording the number of neighbor updates, when it is detected that the algorithm is stuck in a stagnant state (the number of neighbor updates counted for 5 generations of continuous evolution is less than a set threshold), the mutation probability will be temporarily increased to seek a breakthrough in the stagnant state. Figure 7 is the mean variance curve of every population dimension of improved MOEA/D algorithm by adopting adaptive adjustment strategy of mutation probability. It can be seen that the strategy can also mitigate the attenuation trend of the MOEA/D algorithm population diversity.

In terms of convergence, Fig. 8 shows the mean curve of Pareto optimal solution set of improved MOEA/D algorithm by adopting adaptive adjustment strategy of mutation probability. Compared with Fig. 1, it can be seen that the evolution rate is faster and does not fall into the evolutionary stagnation state.

Fig. 8.

Pareto optimal solution set mean value curve of improved MOEA/D algorithm (adaptive mutation probability)

3.3 Double-Faced Mirrors Theory Boundary Processing Strategy

The principle of double-faced mirrors theory: as shown in Fig. 9, the boundary is $[X_{\hbox{min} } ,X_{\hbox{max} } ]$, $y$ is set as the crossing point, where the upper and lower boundaries are two mirrors. $y$ is the propagating beam and the size of $y$ indicates the light intensity. The intermediate medium has optical loss. Then, after a large amount of double-faced mirrors reflection, the light will eventually be exhausted at a certain point $y'$ in the boundary due to the intermediate medium loss. If there are several beams of continuous intensity values, the final dissipation point of these beams after double-faced mirrors reflection will be evenly distributed in the boundary region, as shown in Fig. 10.

Fig. 9.

The principle of double specular reflection

Fig. 10.

Processing effect of the double-faced mirrors reflection principle

If the cross-boundary value is regarded as a beam of light, the cross-boundary problem can be dealt with by the principle of double-faced mirrors reflection. The boundary’s value will eventually fall to a certain point in the boundary after double-mirrors reflection. Continuous and uniform coverage in the boundary area can be realized for multiple random cross-boundary values. The formula for dealing with boundary problems by the principle of double-faced mirrors reflection is as follows:

$$ y^{{\prime }} = \left\{ {\begin{array}{*{20}c} {X_{\hbox{max} } - \bmod (y - X_{\hbox{max} } ,X_{\hbox{max} } - X_{\hbox{min} } ),} & {y > X_{\hbox{max} } } \\ {X_{\hbox{min} } + \bmod (X_{\hbox{min} } - y,X_{\hbox{max} } - X_{\hbox{min} } ),} & {y < X_{\hbox{min} } } \\ \end{array} } \right. $$

(5)

Apply the principle of the double-faced mirrors reflection to the boundary processing of MOEA/D algorithm and observe the distribution of Pareto optimal solution set in the evolution process, as shown in Fig. 11. Compared with Fig. 3, there is no Pareto optimal solution sets gathering at the boundary. Meanwhile, the mean variance of each dimension of the improved MOEA/D algorithm using the double-faced mirrors reflection boundary processing test was calculated, as shown in Fig. 12. It can be seen that the decrease trend of variance was slower than that in Fig. 2, indicating that the double-faced mirrors reflection boundary processing strategy can also effectively improve the diversity of the algorithm population.

Fig. 11.

Pareto optimal solution set distribution diagram of improved MOEA/D algorithm (double-faced mirrors reflection boundary processing strategy)

Fig. 12.

Population variance mean change curve of improved MOEA/D algorithm (double-faced mirrors reflection boundary processing strategy)

In terms of convergence, Pareto optimal solution set mean curve of the improved MOEA/D algorithm using the double-faced mirrors theory boundary processing strategy is shown in Fig. 13, which is better than the overall evolution speed of Fig. 1, but there are also some problems of slow convergence of some optimized goals.

Fig. 13.

Pareto optimal solution set mean curve of improved MOEA/D algorithm (double-faced mirrors theory boundary processing strategy)

4 Conclusion

In view of the problem that the loss of population diversity of MOEA/D algorithm in the late evolution process which leads to the slowdown or even stagnation of evolution, this paper studies three population diversity maintenance strategies of MOEA/D algorithms. Among them, the SBX-DE operator competitive strategy and the mutation probability adaptive adjustment strategy can improve the population diversity of MOEA/D algorithm and meanwhile improving its convergence speed. Although the double-faced mirrors theory boundary processing strategy can also improve population diversity, there is also the problem of slow convergence of some evolutionary goals. Due to the different positions of the three population diversity maintenance strategies in the MOEA/D algorithm, the three strategies can be used together in the practical application process, which will achieve better improvement results.