MAP-Elites for Constrained Optimization

Abstract

Constrained optimization problems are often characterized by multiple constraints that, in the practice, must be satisfied with different tolerance levels. While some constraints are hard and as such must be satisfied with zero-tolerance, others may be soft, such that non-zero violations are acceptable. Here, we evaluate the applicability of MAP-Elites to “illuminate” constrained search spaces by mapping them into feature spaces where each feature corresponds to a different constraint. On the one hand, MAP-Elites implicitly preserves diversity, thus allowing a good exploration of the search space. On the other hand, it provides an effective visualization that facilitates a better understanding of how constraint violations correlate with the objective function. We demonstrate the feasibility of this approach on a large set of benchmark problems, in various dimensionalities, and with different algorithmic configurations. As expected, numerical results show that a basic version of MAP-Elites cannot compete on all problems (especially those with equality constraints) with state-of-the-art algorithms that use gradient information or advanced constraint handling techniques. Nevertheless, it has a higher potential at finding constraint violations versus objectives trade-offs and providing new problem information. As such, it could be used in the future as an effective building-block for designing new constrained optimization algorithms.

1 Introduction

Several real-world applications, for instance, in engineering design, control systems and health care, can be described in the form of constrained continuous optimization problems, i.e., problems where a certain objective/cost function must be optimized within a certain search space, subject to some problem-dependent constraints. Without loss of generality, these problems can be formulated as

$$\begin{aligned} \underset{\mathbf{x \in \mathbf{D} }}{{\text {minimize}}}&f(\mathbf{x} ) \\ {\text {subject\;to:}}&g_i(\mathbf{x} ) \le 0, \quad i = 1, 2, \dots , m\\&h_j(\mathbf{x} ) = 0, \quad j = 1, 2, \dots , p \end{aligned}$$

where (1) $\mathbf {x} \in \mathbf {D} \subseteq \mathbb {R}^n$ is a candidate solution to the problem, being n the problem dimensionality, and $\mathbf {D}$ the search space, typically defined in terms of bounding box constraints $lb_k \le x_k \le ub_k ~ \forall k \in \{1, 2, \dots , n\}$, where $lb_k$ and $ub_k$ are the lower and upper bound, respectively, for each kth variable; (2) $f(\mathbf{x} ): \mathbb {R}^n \rightarrow \mathbb {R}$ is the objective function; (3) $g_i(\mathbf{x} )$ and $h_j(\mathbf{x} )$ (both defined as: $\mathbb {R}^n \rightarrow \mathbb {R}$) are, respectively, inequality and equality constraints. A summary of the aforementioned symbols, as well as the symbols used in the rest of the paper, is reported in Table 1.

Table 1 List of symbols used in the paper (symbols in boldface indicate vectors)

In the past three decades, a large number of computational techniques have been proposed to solve efficiently this class of problems, among which Evolutionary Algorithms (EAs) [17] have shown great potential due to their general applicability and effectiveness. So far, most of the research in the field has focused on how to improve the feasible results obtained by EAs, for instance, developing ad hoc evolutionary operators, specific constraint repair mechanisms, or constraint handling techniques (CHTs). However, in various real-world applications it could be desirable, or at least acceptable, to consider also infeasible solutions. This could be obtained for instance by defining different tolerance levels for each constraint, so to reason on the effect of relaxing a certain constraint (and, if so, how much to do that) in order to obtain an improvement on the objective function, and therefore find different trade-offs in terms of constraint violations versus objective [25]. Despite these application needs, to date little research effort has been put on how to allow EAs to identify, rather than a single optimal solution, a diverse set of solutions characterized by different trade-offs of this kind. In this sense, the most notable exceptions that explicitly addressed this problem—although with contrasting results—have focused on multiobjective approaches, where the constraint violations were considered as additional objectives to be minimized [5, 10, 26, 33, 35], or surrogate methods [2].

In this paper, our goal is to evaluate the applicability of the Multi-dimensional Archive of Phenotypic Elites (MAP-Elites) [3, 19], an EA recently introduced in the literature in the context of robotic tasks, for tackling these problems, specifically to provide trade-off solutions in constrained optimization. Differently from conventional EAs, MAP-Elites conducts the search by mapping the highest-performing solutions found during the search (elites) into another multi-dimensional discretized space, defined by problem-specific features (the latter space is separate from the original search space, and typically of a lower dimensionality). These features are uncorrelated to the actual objective function, and describe some domain-specific properties of the candidate solutions. By means of this mapping, the algorithm “illuminates” the search space by showing the potential value of each area of the feature space, and the corresponding trade-off between the objective and the features of interest.

The functioning of MAP-Elites is simple and intuitive. First, the multi-dimensional feature space is discretized into a multi-dimensional grid, where each bin (i.e., a cell in the grid, which is, in general, a hyper-rectangle) represents a different “niche”. Then, an EA-like search is performed by means of selection and variation (mutation and crossover), but instead of keeping a population of solutions that may or may not be diverse, MAP-Elites explicitly maintains diversity by keeping in each niche one elite, which identifies the best solution characterized by the corresponding feature values. At the end of the optimization procedure, a full map of possible solutions is provided (rather than a single optimal solution, as in conventional single-objective EAs), each characterized by different features. This map is shown in the form of a multidimensional heatmap, which allows for an easy visual inspection of how the objective function changes across the feature space.

In order to apply MAP-Elites to constrained optimization, the main idea we propose here is to define the feature space based on a discretization of the constraint violations. It is worth noting that in practical applications the discretization has a concrete, domain-dependent meaning: it can be seen as a set of tolerance levels, which as we mentioned can be different for each constraint. With this approach, we are able to produce a visual representation of the objective values in the feature space (in this case, space of constraint violations), thus uncovering possible correlations between the constraints and the objective. Thanks to this visualization, it is indeed easy to understand “where”, with respect to the boundaries of the constraints, the best solutions lie. It also is easy to inspect the best overall solution, and check if the algorithm was able to produce particularly interesting solutions violating some of the constraints. As said, this insight can be helpful in cases where the violation of some constraints (within a certain tolerance level) can be an acceptable trade-off for a better overall performance.

As we will see in detail in the paper, despite its simplicity the proposed approach has various advantages: (1) it can be easily adapted/extended to include custom evolutionary operators; (2) it does not necessarily need explicit CHTs, but it can also include them; (3) it implicitly preserves diversity; (4) it allows the user to easily define custom tolerance levels, different for each constraint; (5) it “illuminates” the search space as it provides additional information on the correlation between constraints and objective, which might be of interest in practical applications; (6) it facilitates the interpretation of results through an intuitive visualization.

The rest of the paper is structured as follows. In Sect. 2, we will briefly summarize the most recent works on MAP-Elites and constrained optimization. Then, Sect. 3 describes the basic MAP-Elites algorithm and how it can be applied to constrained optimization. In Sect. 4, we describe the experimental setup (benchmark and algorithmic settings), followed by the analysis of the numerical results, reported in Sect. 5. Finally, Sect. 6 concludes this work and suggests possible future developments.

2 Related Work

The study of MAP-Elites and, more in general, EAs explicitly driven by novelty [12, 18, 24] or diversity [4, 20], rather than the objective alone, is a relatively new area of research in the Evolutionary Computation community. Among these algorithms, MAP-Elites [3, 19] has attracted quite some attention in the field, due to its simplicity and general applicability. Since its introduction in 2015, MAP-Elites has been mostly used as a means to identify repertoires of different agent behaviors, e.g., in evolutionary robotics setups. Various examples of applications to maze navigation, legged robot gait optimization, and anthropomorphic robot trajectory optimization can be found in [1, 3, 4, 19, 23, 30,31,32]. More recently, MAP-Elites has been applied also to Workforce Scheduling and Routing Problem (WSRP) [29] and Genetic Programming [6]. To the best of our knowledge, no prior work exists on the explicit use of MAP-Elites for solving constrained optimization problems.

Evolutionary constrained optimization is, on the other hand, a much more mature area of research: hundreds of papers have shown in the past three decades various algorithmic solutions and real-world problems where EAs were successfully applied to constrained optimization. Summarizing all the recent advances in this area would be impossible, and is obviously outside the scope of this paper. A thorough survey of the literature is performed, for instance, in [13], to which we refer the interested reader for a comprehensive analysis of the state of the art updated to 2016. Another interesting study, published at the end of 2018 by Hellwig and Beyer [9], covers all the aspects related to benchmarking EAs for constrained optimization, including a thorough analysis of the most important benchmark suites available in the literature. Among these, the CEC 2010 benchmark [15] has attracted in the past few years a large body of works that showed how to solve its functions efficiently, and is often used nowadays for benchmarking new algorithms. Currently, the state-of-the-art results on this benchmark have been obtained by $\varepsilon$DEag, an $\varepsilon$ constrained Differential Evolution algorithm with an archive and gradient-based mutation proposed by Takahama and Sakai [28], followed by ECHT-DE, another variant of Differential Evolution that includes an ensemble of four constraint handling techniques, proposed by Mallipeddi and Suganthan [14]. These two works are also good examples of two of the most successful recent trends in the field, which use of gradient-based information (if available, or at least approximable), and the combination of multiple CHTs into a single evolutionary algorithm.

3 Methodology

The basic version of MAP-Elites, as introduced in [3, 19], is shown in Algorithm 1. In the pseudocode, $\mathbf{x}$ and $\mathbf{x} '$ are candidate solutions (i.e., n-dimensional vectors defined in the search space $\mathbf {D}$); $\mathbf{b} '$ is a feature descriptor, which is a location in a user-defined discretized feature space, corresponding to the candidate solution $\mathbf{x} '$, (i.e., an N-dimensional vector of user-defined features that characterize $\mathbf{x} '$, typically with $N<n$); $p'$ is the performance of the candidate solution $\mathbf{x} '$ (i.e., the scalar value returned by the objective function $f(\mathbf{x} ')$; the function itself is assumed to be a black-box, that is its mathematical formulation, if any, is unknown to the algorithm); $\mathcal {P}$ is a <feature descriptor, performance> map (i.e., an associative table that stores the best performance associated to each feature descriptor encountered by the algorithm); $\mathcal {X}$ is a <feature descriptor, solution> map (i.e., an associative table that stores the best solution associated to each feature descriptor encountered by the algorithm); $\mathcal {P}(\mathbf{b} ')$ is the best performance associated to the feature descriptor $\mathbf{b} '$ (it can be empty); $\mathcal {X}(\mathbf{b} ')$ is the best solution associated to the feature descriptor $\mathbf{b} '$ (it can be empty).

Following the pseudocode, the algorithm first creates the two maps $\mathcal {P}$ and $\mathcal {X}$, which are initially empty. Then, a loop of I iterations (i.e., function evaluations) is executed. For each of the first G iterations, G solutions are randomly sampled in the search space $\mathbf {D}$, which are used for initializing the two maps $\mathcal {P}$ and $\mathcal {X}$. Then, starting from the iteration $G+1$, a solution $\mathbf{x}$ is randomly selected from the current map $\mathcal {X}$, and a randomly modified copy of it, $\mathbf{x} '$, is generated. The feature descriptor $\mathbf{b} '$ and performance $p'$ associated to this new, perturbed solution are then evaluated. At this point, the two maps $\mathcal {P}$ and $\mathcal {X}$ are updated: if the performance associated to $\mathbf{b} '$, $\mathcal {P}(\mathbf{b} ')$, is empty (which can happen if this is the first time that the algorithm generates a solution with that feature descriptor), or if it contains a value that is worse than the performance $p'$ of the newly generated solution (in Algorithm 1, we assume a minimization problem, therefore we check the condition $\mathcal {P}(\mathbf{b} ') > p'$), the new solution $\mathbf{x} '$ and its performance $p'$ are assigned to the elements of the maps corresponding to its feature descriptor $\mathbf{b} '$, namely $\mathcal {P}(\mathbf{b} ')$ and $\mathcal {X}(\mathbf{b} ')$. Once the loop terminates, the algorithm returns the two maps $\mathcal {P}$ and $\mathcal {X}$, which can be later analyzed for further inspection and post-processing.

It can be immediately noted how simple the algorithm is. With reference to the pseudocode, in order to apply MAP-Elites to a specific problem the following methods must be defined:

• random_solution(): returns a randomly generated solution;

• $\text {random}\_\text {selection}{(\mathcal {X})}$: randomly selects a solution from $\mathcal {X}$;

• $\text {random}\_\text {variation}{(\mathbf{x} )}$: returns a modified copy of $\mathbf{x}$;

• $\text {feature}\_\text {descriptor}(\mathbf{x} )$: maps a candidate solution $\mathbf{x}$ to its representation in the feature space, $\mathbf{b}$;

• $\text {performance}(\mathbf{x} )$: evaluates the objective function corresponding to the candidate solution $\mathbf{x}$.

The first three methods are rather standard, i.e., they can be based on general-purpose operators typically used in EAs. However, it is possible to customize them according to the specific need. For instance, the basic version MAP-Elites randomly selects at each iteration one solution, and applies only Gaussian mutation operator; on the other hand, the algorithm can be easily configured to use a different selection mechanism (e.g., an informed operator that introduces some selection pressure/bias) or select multiple solutions at each iteration so to apply a recombination operator (crossover) or some other search mechanism such as a local search. We will see in Sect. 4 the details of three different algorithm configurations that we have used in our experimentation.

As for what concerns $\text {feature}\_\text {descriptor}(\mathbf{x} )$ and $\text {performance}(\mathbf{x} )$, these are obviously problem-dependent: the first one, being dependent on how the user defines the features of interest and the corresponding feature space; the latter, being dependent on the specific objective function at hand.

The application of MAP-Elites to constrained optimization is then quite straightforward: here, we map each constraint of a constrained optimization problem to a different feature in the feature space explored by MAP-Elites, such that each candidate solution is associated to a feature descriptor that is basically a vector of constraint violations. In this specific case then, the user does not necessarily have to define any additional feature, but the features themselves are already part of the problem definition. Leaving aside the algorithmic details (selection and variation) and parameters (the only two parameters of the algorithm are the total and initial number of iterations, respectively, I and G, which can be easily set by the user based on computing resources and/or time constraints), the only input required from the user is the discretization of the features (constraint violations) space.

An intuitive way of discretizing this space is to define, for each constraint, a certain number of tolerance levels, i.e., amounts of constraint violation used as discretization steps. These can be easily expressed in absolute terms (based on the values of $g_i(\mathbf{x} )$ and $h_j(\mathbf{x} )$ in case of violations), or normalized w.r.t. known minimum and maximum violations. A simple example of discretization steps is $\{0, \varepsilon , 2\varepsilon , \dots \}$, where $\varepsilon$ is a user-defined parameter. However, as we will show in Sect. 4, also non-linear discretization is possible. In general, the discretization strategy should be based on domain knowledge and defined in such a way that solutions whose violations are equivalent, from a practical point of view, are grouped in the same bin. This would allow to “illuminate” the relation between objective function and constraint violations in a significant, meaningful way. Finally, we must note that while in general a different set of tolerance levels can be defined for each constraint (especially if these are expressed in absolute terms), if all constraints have the same codomain (or, if they are normalized), the same tolerance levels can be used for all of them.

4 Experimental Setup

We evaluated the performance of the proposed approach on the benchmark functions defined for the CEC 2010 Competition on Constrained Real-Parameter Optimization [15]. This benchmark presents 18 problems with different landscape characteristics, subject to a varying number (up to four) of equality and/or inequality constraints. To assess the scalability of MAP-Elites, we tested these problems in 10 and 30 dimensions. All the details of the experimental setup were set according to the CEC indications [15], with the main parameters listed in Table 2.

Table 2 Parameters used in the experimental setup

As for the discretization steps, the selected values correspond to having each feature discretized into 5 bins, namely: $\{0\}$, (0, 0.0001], (0.0001, 0.01], (0.01, 1.0] and $(1.0,\inf )$. This last aspect deserves attention, since as we have seen in Sect. 3 this is what allows the application of MAP-Elites to constrained optimization. Here, we have defined as discretization steps the three tolerance levels defined in [15], i.e., 0.0001, 0.01, and 1.0 (see Sect. 5), in addition to an explicit step corresponding to zero-tolerance (corresponding to solutions with $g_i(\mathbf{x} ) \le 0$ and $h_j(\mathbf{x} ) = 0$, respectively for inequality and equality constraints).

The features used in MAP-Elites follow the same order as they appear in the corresponding problem definition, with inequality constraints considered before equality constraints. Since the problems contained in the CEC 2010 benchmark have a variable number of equality/inequality constraints, we define a variable-sized feature space, where for each problem there are as many MAP-Elites features as constraints. For visualization purposes, we represent the map $\mathcal {P}(\mathbf{b} ')$ obtained in each run of MAP-Elites in the form of a multi-dimensional heatmap, as explained in [3, 19]. The color represents the objective value corresponding to the solution contained in each bin. The first axis (abscissa) corresponds to the first constraint violation, the second axis (ordinate) corresponds to the second constraint violation, and so on for the third and fourth constraints. Feature dimensions are “nested” such that the feature space is first discretized along the 1st and 2nd axes (so to obtain a 2D grid of bins), while the following features (if any) are represented by an “inner” (1D or 2D) discretization inside each bin in the “outer” grid. Obviously, this visualization procedure can be easily extended to handle more than four constraints, although the visual interpretability of the results tends to decrease with the number of features shown in the heatmap.

It should be noted that according to the CEC 2010 benchmark definition [15], a solution $\mathbf{x}$ is considered feasible iff $g_i(\mathbf{x} ) \le 0 ~\forall i \in \{1, 2, \dots , m\}$ and $|h_j(\mathbf{x} )| - \epsilon \le 0 ~\forall j \in \{1, 2, \dots , p\}$, where $\epsilon$ is the equality constraint tolerance, set to 0.0001. Otherwise, the solution is considered infeasible. From what we have just discussed, it follows then that feasible solutions in the sense of the CEC 2010 definition can be found: (1) for what concerns inequality constraints, in the first bin ($\{0\}$) along each feature dimension; (2) for what concerns equality constraints, in the first two bins ($\{0\}$, (0, 0.0001]). In plain terms, this means that we can easily identify feasible solutions found by MAP-Elites by simply looking at the lower-left corner of the heatmap, while solutions with increasing constraint violations are found scanning the heatmap (and each inner bin in case of more than two constraints) towards the upper right side. Some examples of heatmaps obtained by MAP-Elites are shown in Fig. 1.

Fig. 1

Final heatmaps found in a single run of MAP-Elites on C1, Configuration 1 (left), C2, Configuration 1 (center), C16, Configuration 3 (right) in 10D. The three benchmark functions are characterized, respectively, by 2I, 2I-1E, and 2I-2E, where ‘I’ and ‘E’ stand for inequality and equality constraints, respectively. In each heatmap, the color of each bin is proportionate to the objective value corresponding to the solution present in it (assuming minimization, the lower the better), while the red bin indicates the solution with the best objective value (regardless its feasibility). It can be observed that the maps allow to “illuminate” the search space of each problem, identifying various trade-off solutions in terms of objective value versus constraint violations, such as solutions with a high performance but with some violated constraints. Note that in case of 3 or 4 constraints, the discretization along the first two (outer) dimensions of the heatmap is indicated by a thicker black line, while the discretization along the “nested” (inner) dimensions are indicated by a thinner black line

As for the evolutionary operators (selection and variation, as shown in Algorithm 1), we defined three different algorithmic settings. In all cases, selection is performed according to a uniform distribution over the current map. Variation is instead applied according to the following configurations:

• Configuration 1: mutation ($\sigma =0.1$), without crossover

• Configuration 2: mutation ($\sigma =0.5$), without crossover

• Configuration 3: mutation ($\sigma =0.1$), with crossover.

In all three cases, mutation is implemented by applying to the selected solution (with probability 0.5 for each variable) a Gaussian mutation with $\mu =0$ and the given value of $\sigma$. Boundary constraints are handled according to a toroidal mechanism: given a decision variable x constrained to the interval [a, b], if the corresponding mutated variable $x'$ exceeds the upper bound b (i.e., $x'=b+\zeta$), $x'$ is transformed into $x'=a+\zeta$, $\zeta >0$. Similarly, if $x'=a-\zeta$, $x'$ is transformed into $x'=b-\zeta$, $\zeta >0$.

In Configuration 3, at each iteration, two solutions are randomly selected from the current map, after which uniform crossover (with probability 0.5 for each variable) is applied by swapping the corresponding variables from the two parents. Then, the first of the two offspring generated by crossover undergoes Gaussian mutation, as in Configurations 1 and 2, and is evaluated in terms of feature descriptor and performance, as shown in Algorithm 1.

The entire experimental setup was implemented in Python 3,¹ and the experimentation was performed on a Ubuntu 18.10 workstation, with a CPU Intel Core i9-7940X @3.10 GHz and 64 GB DDR4.

5 Numerical Results

We present here the results obtained on the experimental setup described in Sect. 4. In Tables 3, 4, 5, 6, 7 and 8, we report the results for all the CEC 2010 functions in 10 and 30 dimensions, for the three algorithm settings described above.² In the tables, we report the results as suggested in [15], where for each function we show:

• 1. The objective value corresponding to the best, worst and median solution³ (over 25 runs) obtained at the end of the computational budget; next to each objective value, we show in parenthesis the no. of violated constraints corresponding to each of these three solutions.

• 2. The number of violated constraints at the median solution, $c=(c_1, c_2, c_3)$ (where each element $c_i, i=1,2,3$ represents the number of violations higher than three tolerance levels set to 1, 0.01, and 0.0001, respectively), and the corresponding mean violation $\bar{v}$, calculated as

  $$\begin{aligned} \bar{v}=\frac{\sum _{i=1}^{m}{G_i(x)}+\sum _{j=1}^{p}{H_j(x)}}{m+p} \end{aligned}$$

  (1)

  where $G_i(x)=g_i(x)$ if $g_i(x)>0$ (otherwise zero), and $H_j(x)=|h_j(x)|$ if $|h_j(x)|-\epsilon >0$ (otherwise zero), being $\epsilon$ is the equality constraint tolerance (as seen earlier, 0.0001).

• 3. The average objective value (over 25 runs) of the final solutions obtained at the end of the budget, and its std. dev.

• 4. The Feasibility Rate (FR), that is, for each function, the ratio between the number of runs during which at least one feasible solution was found within the budget, and the total number of runs (in our case, 25).

Table 3 Numerical results (rank based on CEC rules) obtained from 25 independent of MAP-Elites (Configuration 1) on 10D problems

Table 4 Numerical results (rank based on CEC rules) obtained from 25 independent of MAP-Elites (Configuration 2) on 10D problems

Table 5 Numerical results (rank based on CEC rules) obtained from 25 independent of MAP-Elites (Configuration 3) on 10D problems

Table 6 Numerical results (rank based on CEC rules) obtained from 25 independent of MAP-Elites (Configuration 1) on 30D problems

Table 7 Numerical results (rank based on CEC rules) obtained from 25 independent of MAP-Elites (Configuration 2) on 30D problems

Table 8 Numerical results (rank based on CEC rules) obtained from 25 independent of MAP-Elites (Configuration 3) on 30D problems

Table 9 Numerical results (rank based on CEC rules) obtained from 25 independent of $\varepsilon$DEag on 10D problems (taken from [28])

Table 10 Numerical results (rank based on CEC rules) obtained from 25 independent of $\varepsilon$DEag on 30D problems (taken from [28])

For reference, we report in Tables 9 and 10 the results for all the CEC 2010 functions after $I=2.0\mathrm {e}5$ and $I=6.0\mathrm {e}5$ NFEs, respectively for 10D and 30D, obtained by $\varepsilon$DEag [28], the best algorithm on the CEC 2010 benchmark.

From the tables, we can observe that in all three configurations, MAP-Elites solves with 100% FR C1, C7, C8, C14, C15 in 10D, i.e., all the functions with inequality constraints only (except C13, that is, however, the only function with inequality constraints only whose volume of the feasible region is approximately zero); in 30D, it also finds feasible solutions on C18 in 100% of the runs in all configurations (92% in 10D for Configuration 3). The peculiarity of this function is that despite it has one equality constraint, the volume of its feasible region is non-zero. The only other functions on which a non-zero FR is obtained, although not in all configurations and dimensionalities, are C2, C9, C10, C16, C17. Except C10 that has 1 “rotated” equality constraint, all other functions have only separable constraints, which could explain why in some cases even by Gaussian mutation only (which acts independently on each variable) it is possible to reach the feasible region. Overall, among the 3 configurations Configuration 3 has the highest FR across all the tests, while it results that an excessively high value of $\sigma$ in Gaussian mutations (as in Configuration 2) is detrimental.

From these observations, we can conclude that the basic MAP-Elites algorithm we have used in our experimentation is not able to solve efficiently either problems with non-separable equality constraints, or with an approximately zero-volume feasible region. This is not surprising though, as the algorithm is only based on simple genetic operators (Gaussian mutations and uniform crossover in our case) that do not use any information about the constraints. In contrast, $\varepsilon$DEag [28] encapsulates highly efficient CHTs and uses gradient information about constraints that allows the algorithm to reach a 100% FR on all functions in 10D and 30D (except, respectively, for the two algorithms, C12 in 30D, and C11-C12 in both 10D and 30D), as reported in the original papers.

This comparison encourages though the idea to explore in the future the possibility to include into the MAP-Elites scheme at least one dedicated technique for better handling equality constraints, such as the $\varepsilon$ constrained method, initially introduced in [27] and since then used in most of the state-of-the-art algorithms for constrained optimization. Notably, the strength of this method is that it guides the search by allowing $\varepsilon$ level comparisons with a progressively shrinking relaxation (defined by the $\varepsilon$ parameter) of the constraint boundaries.

Considering the objective values, similar considerations can be drawn: limiting the analysis on the functions with 100% FR, it results that MAP-Elites is less efficient at finding optimal values than $\varepsilon$DEag [28]. In all cases MAP-Elites is several orders of magnitude worse than $\varepsilon$DEag, except C1 in 10D where instead the configuration with crossover finds a better optimal value.⁴ Once again, this conclusion is not surprising and is also in line with what was observed by Runarsson and Yao [22], who identified the reason for the sometimes poor results obtained by multi-objective approaches (such as [10, 26, 33, 35]): in fact, when applied to constrained optimization, the Pareto ranking leads to a “bias-free” search that is not able to properly guide the search towards (and within) the feasible region. In other words, allowing the search to spend too many evaluations in the infeasible region makes it harder to find feasible solutions, but also to find feasible solutions with optimal values of the objective function. This might be the case also of MAP-Elites, where some form of bias (such as the $\varepsilon$ constrained method) might be needed.

6 Conclusions

In this paper, we have explored the use of MAP-Elites for solving constrained continuous optimization problems. In the proposed approach, each feature in the feature space explored by MAP-Elites corresponds, quite straightforwardly, to the violation of each constraint, discretized according to user-defined steps (tolerance levels). In this way, the algorithm allows to “illuminate” the search space and thus uncover possible correlations between the constraints and the objective. The visualization of MAP-Elites also gives users the possibility to focus on different solutions characterized by different values of constraint violations. We have tested this approach on a large number of benchmark problems in 10 and 30 dimensions, characterized by up to four equality/inequality constraints. Our numerical results showed that while MAP-Elites obtains results that are not particularly competitive with the state-of-the-art on all problems (especially those with equality constraints), it is still able to provide new valuable, easy-to-understand information that can be of great interest for practitioners. Additionally, the algorithm can be easily implemented and applied without any specific tuning to various real-world problems, for instance, in engineering design, where different tolerance levels can be defined depending on the specific constraints.

Since our goal was to evaluate the performance of the basic MAP-Elites on constrained optimization, the proposed approach is purposely quite simplistic, but clearly it can be extended in various ways. First of all, the basic MAP-Elites algorithm we used in this work (as shown in Algorithm 1 can be replaced with some more advanced variants recently proposed in the literature. In particular, the version of MAP-Elites based on centroidal Voronoi tessellation (CVT-MAP-Elites) [31] can be used instead of the basic one in order to scale the algorithm to a larger number of constraints. In order to better handle the unbounded feature spaces (thus avoiding the need for an explicit “upper” bin, $(1.0,\inf )$ in our case), the “expansive” MAP-Elites variants introduced in [30] can be employed instead, which are able to expand their bounds (in the feature space) based on the discovered solutions. Other possibilities will be to use the “directional variation” operator introduced in [32], that exploits inter-species (or inter-elites) correlations to accelerate the search, add, most of all, specific constraint handling techniques (especially for handling equality constraints, which as we have seen is the main weakness of this approach) [7, 14]. It is also worth considering the use of surrogate models, such as in [34], in order to further speed up the search and guide it towards the feasible region, still allowing the algorithm to keep infeasible solutions as part of the map. Another improvement can be obtained by using an explicit Quality Diversity measure [1, 4, 20, 21], so to enforce at the same time a better coverage of the feature space and a further improvement in terms of optimization results. It is also possible to hybridize the basic MAP-Elites algorithm with local search techniques (such that MAP-Elites explores the feature space and local search is applied within one bin to further refine the search), or to devise memetic computing approaches based on a combination of MAP-Elites and other metaheuristics, such as CMA-ES, that has been recently applied successfully also to constrained optimization [8, 11, 16].

Finally, on the application side, it will be interesting to evaluate the applicability of this approach on combinatorial constrained optimization problems, which can be obtained by simply modifying the variation operators, or multi-objective constrained optimization, which can be obtained by adding a Pareto-dominance check, as recently shown in the context of robotic experiments [23].