Efficient implicit constraint handling approaches for constrained optimization problems

Abstract

Many real-world optimization problems, particularly engineering ones, involve constraints that make finding a feasible solution challenging. Numerous researchers have investigated this challenge for constrained single- and multi-objective optimization problems. In particular, this work extends the boundary update (BU) method proposed by Gandomi and Deb (Comput. Methods Appl. Mech. Eng. 363:112917, 2020) for the constrained optimization problem. BU is an implicit constraint handling technique that aims to cut the infeasible search space over iterations to find the feasible region faster. In doing so, the search space is twisted, which can make the optimization problem more challenging. In response, two switching mechanisms are implemented that transform the landscape along with the variables to the original problem when the feasible region is found. To achieve this objective, two thresholds, representing distinct switching methods, are taken into account. In the first approach, the optimization process transitions to a state without utilizing the BU approach when constraint violations reach zero. In the second method, the optimization process shifts to a BU method-free optimization phase when there is no further change observed in the objective space. To validate, benchmarks and engineering problems are considered to be solved with well-known evolutionary single- and multi-objective optimization algorithms. Herein, the proposed method is benchmarked using with and without BU approaches over the whole search process. The results show that the proposed method can significantly boost the solutions in both convergence speed and finding better solutions for constrained optimization problems.

Introduction

Constrained optimization problems arise naturally in most disciplines where finding optimized feasible solutions, especially with multiple objectives, is challenging. Despite the considerable variety of techniques developed in optimization fields and other disciplines to tackle these problems, the complexity of their solutions calls for alternative solution methods. Moreover, the computational cost has become another major concern due to the complexity of modern real-world problems. Also, the demand for optimal design and its applications in engineering and industry have become even more significant to satisfy the need for more strategic designs in modern engineering practices¹. Engineering optimization problems usually consist of constraints that may be physical, geometrical, or operational; handling such constraints to find a single feasible solution is a challenging task^2,3. For multi-objective problems, constraint handling is critical since a set of feasible solutions, i.e., Pareto front set, is sought. To tackle real-world constrained optimization problems, the constraint handling technique (CHT) has been combined with an evolutionary algorithm (EA) to achieve constrained evolutionary algorithm optimization (CEAO)². Most CHTs proposed in the literature are explicit methods, e.g., CHTs by penalty or other fix-ups².

The CHTs come in two main forms: explicit and implicit. Explicit techniques involve the explicit definition of constraints as part of the problem formulation. They are designed to respect and enforce constraints explicitly, guiding the search towards feasible regions of the solution space and addressing violations directly.

For example, several existing CHTs in the literature are based on feasibility and infeasibility regions^4,5,6, priority assignment⁷, and tournament selection and a selection operator^7,8. Some studies also provide surveys of CHTs^8,9,10,11. In contrast, implicit techniques do not necessitate the explicit form of constraints. They handle constraints inherently during the search for the optimal solution, avoiding the need for additional objective function evaluations. Implicit CHT simplifies the problem formulation, but it may face challenges in handling complex or nonlinear constraints effectively.

For instance, Diwekar and Rubin¹² proposed an implicit constraint handling technique for the ASPEN chemical process, which is based on a mixed integer mathematical programming approach consisting of master and subproblems. Since most of the relations in the above-mentioned problems are implicit¹², proposed a method for partitioning the variables to significantly decrease computational time.

Raghavan et al.¹³ developed an implicit constraint handling technique for optimization based on the proper orthogonal decomposition (POD) of shapes, further producing a bi-level reparameterization approach for structural geometries. Uemura et al.¹⁴ proposed a real-coded genetic algorithm for implicitly constrained black-box optimization, in which the method employs the weighted mean of the best individuals in a population to find the optimal solution.

Mirabel and Lamiraux¹⁵ proposed a method to handle all constraints explicitly and implicitly that deals with manipulation planning, where the constraints are solved explicitly as much as possible. Implicit handling is alternatively employed with few variables in case all constraints cannot be handled explicitly. Nomura et al.¹⁶ introduced a natural evolution strategy called DX-NES-IC for implicit handling of constrained black-box optimization, which demonstrated better performance than DX-NES, xNES, and CMA-ES.

The current study extends the work of Gandomi and Deb⁶ by extending and applying boundary update (BU) to single-objective and multi-objective optimization problems (MOOPs). As novel strategies, two switching mechanisms are suggested that transform the landscape and variables to the original problem when the feasible region is found. Then, the optimization process is continued without the BU method.

In the work by Gandomi and Deb⁶, the authors proposed a novel approach, boundary update (BU), as an implicit CHT that updates variable bounds by directly using the constraints and then applying them to several single-objective optimization problems. Since, the BU method is an implicit CHT, it should be coupled with an explicit CHT, that is feasibility rules in our study¹⁷. As for the strategy without BU, only an explicit CHT (feasibility rules) is applied to solve the problem. The BU method is an implicit constraint handling approach that aims to cut the infeasible search space over iterations to find the feasible region faster. Although the BU method directs the search operators to the feasible space and reaches the first feasible solution reasonably fast, it twists the search space, making the optimization problem more challenging. The current study tries to tackle the above-mentioned problem by proposing two switching approaches.

The current study augments existing research by introducing two novel switching mechanisms. In the initial method, called Hybrid-cvtol, the BU approach is employed until constraint violations reach zero, ensuring zero violations across the entire population. Subsequently, the algorithm transitions to the optimization process without utilizing the BU approach. The second switching mechanism, called Hybrid-ftol, in this study involves employing the BU method during the initial phase until the objective space remains unchanged for a specified number of generations. At that point, the optimization problem shifts to a state without utilizing the BU approach.

The remainder of the paper is organized as follows. Section “Proposed approach” presents the proposed approach. Section “Research methodology” discusses the research methodology. Section “Numerical examples” provides numerical examples. Finally, Sects. “Discussion”, “Limitation”, and “Conclusion” give the discussion, limitation, and conclusion.

Proposed approach

A constrained optimization problem can be formulated in Eqs. (1–3) as follows:

$$\mathrm{Maximize }({\text{Minimize}})\mathrm{ F}({\text{x}})=({{\varvec{f}}}_{1}\left({\varvec{x}}\right),\dots ,{{\varvec{f}}}_{{\varvec{t}}}\left({\varvec{x}}\right))$$

(1)

$${\text{s}}.{\text{t}}. {{\varvec{h}}}_{{\varvec{i}}}\left({\varvec{x}}\right)\le 0{\varvec{f}}{\varvec{o}}{\varvec{r}}{\varvec{i}}\in \{1,\dots ,{\varvec{n}}\}$$

(2)

$${{\varvec{g}}}_{{\varvec{j}}}\left({\varvec{x}}\right)=0\mathrm{for j}\in \{1,\dots ,{\varvec{m}}\}$$

(3)

$${\text{LB}}\le {\varvec{x}}\le {\varvec{U}}{\varvec{B}}$$

(4)

where $F(x)$ is the objective vector that consists of several objectives (t is the number of objective functions); n and m are the number of inequality and equality constraints, respectively; and x is the decision variable bounded by the lower bound (LB) and upper bound (UB) vectors. These equations yield several Pareto optimal solutions rather than producing a single solution¹⁸. In the case of single-objective optimization, a single solution can be found as a solution to $F\left(x\right)={f}_{1}\left(x\right)$. In the optimization process, the boundaries of decision variables, represented by lower bounds (LB) and upper bounds (UB) vectors, undergo dynamic changes as the algorithm progresses through iterations. This dynamic nature enables the optimization process to adapt and explore a continually evolving solution space, illustrating the inherent variability and adaptability of decision variable boundaries throughout the iterative stages. Hence, the iterative nature of optimization involves the dynamic evolution of the i-th decision variable boundaries throughout each iteration.

The proposed method uses the constraints to narrow down variable space and then forces the algorithm to focus its search in the feasible region by limiting the viable search space for the variable(s). In the BU method, the boundaries change iteratively and are updated during the optimization procedure. Mathematically, it could be written as follows⁶:

$$\exists i\in \left\{1,\dots ,m\right\}:[\forall j\in \left\{1,\dots ,n\right\}:{x}_{i}\ge {l}_{i,j}\left({x}_{\ne i}\right)\cup {x}_{i}\le {u}_{i,j}\left({x}_{\ne i}\right)]$$

(5)

where ${l}_{i,j}$ and ${u}_{i,j}$ are dynamic lower bound and upper bound for the ith decision variable, respectively. Updating the bounds involves the following scenarios:

$$\mathrm{If }{lb}_{i}=-\mathrm{\infty and }{ub}_{i}=+\infty :$$

(6)

$${lb}_{i}^{u}={l}_{i,j}({x}_{\ne i})$$

(7)

$${ub}_{i}^{u}={u}_{i,j}({x}_{\ne i})$$

(8)

Else:

$${lb}_{i}^{u}={\text{min}}({\text{max}}\left({l}_{i,j}\left({{\text{x}}}_{\ne {\text{i}}}\right),{lb}_{i}\right),{ub}_{i})$$

(9)

$${ub}_{i}^{u}={\text{max}}({\text{min}}\left({u}_{{\text{i}},{\text{j}}}\left({{\text{x}}}_{\ne {\text{i}}}\right),{ub}_{i}\right),{lb}_{i})$$

(10)

where (${lb}_{i}^{u},{ub}_{i}^{u})$ are updated boundaries. BU begins by selecting a repairing variable that can handle the most significant number of constraints without overlapping other repairing variables. As such, if there is more than one candidate, the one that handles the greatest number of constraints is selected. Also, if there is still another candidate for variable selection, one variable is chosen randomly⁶.

If a repairing variable handles the first ${k}_{i}$ constraints, then:

$${lb}_{i}^{u}={\text{min}}({\text{max}}[{l}_{i,1}\left({x}_{\ne i}\right),\dots ,{l}_{i,{k}_{i}}\left({x}_{\ne i}\right),{lb}_{i}],{ub}_{i})$$

(11)

$${ub}_{i}^{u}={\text{max}}({\text{min}}[{u}_{i,1}\left({x}_{\ne i}\right),\dots ,{u}_{i,{k}_{i}}\left({x}_{\ne i}\right),{ub}_{i}],{lb}_{i}) (\mathrm{where }{k}_{i}\le m)$$

(12)

If another repairing variable $({x}_{r})$ is defined:

$${lb}_{i}^{u}={\text{min}}({\text{max}}\left[{l}_{i,1}\left({x}_{\ne i,r}\right),\dots ,{l}_{i,{k}_{i}}\left({x}_{\ne i,r}\right),{lb}_{i}\right],{ub}_{i})$$

(13)

$${ub}_{i}^{u}={\text{max}}({\text{min}}\left[{u}_{i,1}\left({x}_{\ne i,r}\right),\dots ,{u}_{i,{k}_{i}}\left({x}_{\ne i,r}\right),{ub}_{i}\right],{lb}_{i})$$

(14)

$${lb}_{r}^{u}={\text{min}}\left({\text{max}}\left[{l}_{r,{k}_{i}+1}\left({x}_{\ne r}\right),\dots ,{l}_{r,{k}_{i}+{k}_{r}}\left({x}_{\ne r}\right),{lb}_{r}\right],{ub}_{r}\right)$$

(15)

$${ub}_{r}^{u}={\text{max}}\left({\text{min}}\left[{u}_{r,{k}_{i}+1}\left({x}_{\ne r}\right),\dots ,{u}_{r,{k}_{i}+{k}_{r}}\left({x}_{\ne r}\right),{ub}_{r}\right],{lb}_{r}\right)\mathrm{ where }r\in \left\{1,\dots ,n\right\} and r\ne i$$

(16)

In the BU method, a repairing variable could be substituted with a generalized semi-independent variable⁶ and then rewritten with lower and upper bounds as:

$$x\in \left\{{x}_{1},\dots ,{x}_{h},{x}_{h+1},\dots ,{x}_{n}\right\}\to mx\in \left\{{p}_{1},\dots ,{p}_{h},{x}_{h+1},\dots ,{x}_{n}\right\}$$

(17)

where ${p}_{1},\dots ,{p}_{h}$ are semi-independent variables; and mx indicates the set of selected repairing variables.

$$x\in \left\{{x}_{1},\dots ,{x}_{h},{x}_{h+1},\dots ,{x}_{n}\right\}\to mx\in \left\{{p}_{1},\dots ,{p}_{h},{x}_{h+1},\dots ,{x}_{n}\right\}$$

(18)

$${x}_{i}={lb}_{i}^{u}+{p}_{i}({ub}_{i}^{u}-{lb}_{i}^{u})\mathrm{ for i}=1, \dots ,\mathrm{ h where }0\le {p}_{i}\le 1$$

(19)

After selecting repairing variables, the search operator will be applied to the problem, the boundaries of non-repairing variables will be checked, and the $mx$-vector will be updated. During the solution procedure, the boundaries of repairing variables will be updated, and then the semi-independent variables (${p}_{i},i=1,\dots ,h)$ are remapped to the actual variables using updated boundaries. In the end, those constraints that were not involved in the repairing variable boundary, along with fitness values, will be evaluated using actual variables. Algorithm 1 illustrates the BU method in a constrained optimization problem.

Algorithm 1

Implementation of the BU method in a constrained optimization problem.

As an illustrative example, the following multi-objective optimization problem (MOOP) is considered to explain BU:

$$\mathrm{min }{f}_{1}\left(x\right)=({x}_{1}^{2}+{x}_{2}^{2})$$

(20)

$$\mathrm{min }{f}_{2}\left(x\right)=-{\left({x}_{1}-1\right)}^{2}-{x}_{2}^{2}$$

(21)

$${g}_{1}\left({x}_{1}+{x}_{3}\right)={x}_{1}+{x}_{3}\le 1$$

(22)

$$-2\le {x}_{1}\le 2$$

(23)

$$-2\le {x}_{2}\le 2$$

(24)

$$-2\le {x}_{3}\le 2$$

(25)

Either ${x}_{1}\mathrm{ or }{x}_{3}$ can be selected as a repairing variable of the single constraint. All the variables are in the range of − 2 to 2. From the variable selection strategy⁶, as explained earlier, ${x}_{3}$ is selected as a repairing variable. The inequality constraint is solved with respect to ${x}_{3}$ as the lower and upper bound functions. The repairing variable (${x}_{3}$) is substituted with the mapped variable, $p$, in the range of 0–1. Therefore, the lower and upper bounds of the variables are {− 2, − 2, 0} and {2, 2, 1}, respectively. When the search algorithm is applied to the problem, the repairing variable is substituted with the mapped variable, and the boundary of the repairing variable is mapped for the rest of the search cycle⁶. The original variable can be calculated by the following formula to compute the objectives and constraints:

$${x}_{3}={lb}_{3}^{u}+{p}_{i}\times ({ub}_{3}^{u}-{lb}_{3}^{u})$$

(26)

where ${ub}_{3}^{u}\mathrm{ and }{lb}_{3}^{u}$ are the lower and upper bounds for the repairing variable ${x}_{3}$, respectively. The updated bounds can be written as follows:

$${lb}_{3}^{u}={\text{min}}\left(\mathit{max}\left({lb}_{3},1-{x}_{1}\right),{ub}_{3}\right)$$

(27)

$${ub}_{3}^{u}={\text{max}}\left(\mathit{min}\left({ub}_{3},1-{x}_{1}\right),{lb}_{3}\right)$$

(28)

The repairing variable is remapped to the actual boundary when the fitness values are evaluated. Figure 1a–d compare Pareto-optimal fronts found by the BU method coupled with a CHT (here, feasibility rules) and by the method without the BU method (only the feasibility rules approach is considered) for a population size of 100 for different generations. Figure 1 shows that the BU method produces more non-dominated solutions than the approach without BU. However, the number of non-dominated solutions is not always a good indicator for comparison. In the following sections, other indicators are explained in detail to compare the algorithms.

Figure 1

Pareto-optimal fronts obtained with and without the BU method.

Although the BU method directs the search operator to the feasible space and reaches the feasible region in a reasonable time, it twists the search space, which can make the optimization problem more challenging and, hence, does not always bear good results and could potentially result in premature convergence. Consider the following example:

$$Max f\left(x\right)={x}_{1}^{2}+{x}_{2}^{2}-2\times {x}_{1}-2\times {x}_{2}+2$$

(29)

$$s.t. {g}_{1}\left(x\right):-\left(3\times {x}_{1}+{x}_{2}^{2}-5.5\right)\le 0$$

(30)

$${g}_{2}\left(x\right):-({x}_{1}^{2}+2\times {x}_{2}-4.5)\le 0$$

$${g}_{3}\left(x\right):0.8+{x}_{1}^{3}-{x}_{2}\le 0$$

$$0\le {x}_{1}\le 4$$

(31)

$$0\le {x}_{2}\le 4$$

(32)

In case of selecting variable ${x}_{2}$ as a repairing variable, the design space for the original variables ${x}_{1}$ and ${x}_{2}$ (Fig. 2a) is converted to the new design space (Fig. 2b). Twisting landscape may result in challenges for optimization problems, such as premature convergence. Some of these issues are presented in the following examples.

Figure 2

Contour plot of the search space.

Therefore, two distinct hybrid methods are introduced, each incorporating its own unique switching mechanism. In the initial approach, the BU method is employed until all constraint violations are reduced to zero, ensuring the absence of violations across the entire population. Subsequently, the algorithm transitions to the optimization process, excluding the use of the BU approach. In the second switching mechanism, the BU method is utilized in the initial phase until the objective space remains unchanged for a specified number of generations. Following this, the optimization problem transitions to a state where the BU approach is no longer employed. In both approaches, the optimization process is halted once the criteria and threshold tolerance are met. In the subsequent phase, the optimization problem is solved without the BU method, using the final population from the previous phase as the initial population for the new optimization problem. Figure 3 illustrates that the handling of boundaries and constraints occurs after the application of search operators.

Figure 3

Optimization process of the proposed method.

During the first phase, the proposed methods simultaneously check constraints and examine variable boundaries, making the optimization process an iterative search procedure for handling constraints and boundaries concurrently. If the criteria are met, the second phase of the optimization process is conducted without utilizing the BU approach, constituting a sequential iterative search procedure where constraint handling occurs after boundary checking. The implementation of the proposed hybrid method is detailed in Algorithm 2.

Algorithm 2

Implementation of the proposed method (hybrid methods).

Research methodology

To evaluate the proposed method, Python and Pymoo libraries¹⁹ were used, which include well-stabilized algorithms. The specific algorithm parameters are presented in Table 1. Furthermore, because the implemented algorithms act stochastically, each of these algorithms were ran 31 times with random initial point(s).

Table 1 Specific parameter settings for the algorithms used in this study.

Several examples, including benchmarks and real-world problems, were solved using three methods (without the BU method, with the BU method, and hybrid method). The presented examples are of different types: constrained by one, two, and three objective functions. For the problem with one objective function, the proposed method was implemented using evolutionary strategy (ES), differential evolution (DE), stochastic ranking evolutionary strategy (SRES)²⁰, and biased random key genetic algorithm (BRKGA)²¹. For two objective functions, the BU method was applied using NSGA-II^17,22. For three and five objective functions, some other EAs were used. As such, NSGA-III²³, U-NSGA-III²⁴, AGE-MOEA²⁵, NSDE²⁶, and NSDER²⁷ were applied to optimization problems. The following metrics were evaluated to assess the accuracy of the obtained results.

• Number of non-dominated solutions: The BU method was coupled with an explicit constraint handling technique and compared to the approach without BU (only feasibility rules were applied to the optimization problem). Since the BU method aims to reduce the search space, the number of non-dominated solutions was determined for comparison.

• Constraint violation (CV): Compared to unconstrained optimization problems, constrained optimization problems are more challenging since a large proportion of infeasible regions appears in the search space (meaning the hit ratio is low), especially for highly constrained problems where these regions lead to some difficulties, such as feasibility-hardness and convergence-hardness. Therefore, even finding one feasible solution is a significant achievement. To test the effectiveness of the proposed method, the first feasible solutions found by the methods were compared and the whole population was tracked against generations.

• Performance indicators: Generational Distance (GD)²⁸, Generational Distance Plus (GD+)²⁹, Inverted Generational Distance (IGD)³⁰, Inverted Generational Distance Plus (IGD+)²⁹, and Hypervolume (HV) are the indicators that were analyzed.

• Running metric³¹: It is possible to trace the difference in the objective space followed by generations.

Moreover, the population was fixed at 100, and the maximum number of experiments was set to 11.

Numerical examples

The proposed approach was applied to several single-objective and multi-objective optimization examples, including benchmarks and real-world problems. The presented examples include problems constrained by one, two, and three objective functions. The performance of the three strategies was evaluated using an example that has linear constraints. Following this numerical example, the proposed approach was evaluated to solve a surrogate model for a car-side impact design problem. This design problem consists of two versions of optimization, i.e., single- and multi-objective optimizations, and is an example of a black-box optimization problem. The speed reducer problem, a single-objective constrained engineering optimization problem, was also considered. For the multi-objective optimization problem, two benchmarks, namely OSY and BNH, and three real-world constrained optimization problems, including welded beam design, Cantilevered Beam design, and multi-objective car-side design problems were solved via the three strategies.

Several metaheuristic methods have been developed and implemented to reduce the total runtime of the optimization problems^2,32, but still produce results that are almost as accurate and precise as conventional solving methods. Also, real-world engineering problems involve some type of optimization that is often constrained, most of which are considered MOOPs. No single solution exists for a MOOP; instead, different solutions generate trade-offs for various objectives. Furthermore, MOOPs arise naturally in most fields, and solving them has been a challenging problem for researchers^32,33. EA methods have been identified as more effective in tackling the challenges that arise from MOOPs, for which the form of the Pareto-optimal front (discontinuity, nonconvexity, etc.) is not important^34,35. Moreover, most multiobjective evolutionary algorithms (MOEAs) use the dominance concept^36,37. In this work, for the single-objective optimization problems, Genetic Algorithm (GA), Differential Evolution (DE)³⁸, Evolutionary Strategy (ES)³⁹, Stochastic Ranking Evolutionary Strategy (SRES)²⁰, and Biased Ranking Key Genetic Algorithm (BRKGA)⁴⁰ were considered. NSGA-II⁴¹ is applied to bi-objective and multi-objective optimization problems with three objective functions and to multi-objective optimization problems with more than three objective functions, which are called many-objective optimization problems. NSGA-III²³, U-NSGA-III²⁴, and Adaptive Geometry Estimation-based Multi-objective Evolutionary Algorithm (AGE-MOEA)²⁵ were further implemented for optimizing the problems. For all problems, the BU method was coupled with feasibility rules proposed by Deb⁴¹. Table 2 presents the studied examples in this work to assess the effectiveness of the proposed method in comparison with the other methods.

Table 2 Test problems considered in this study.

Single optimization problems

In this section, three single-optimization problems, namely G1, car-side design problem, and speed reducer, are considered to be solved using the different strategies discussed above.

G1 problem

The G1 problem is a famous mathematical constrained optimization problem that includes 13 decision variables, a quadratic objective function, and nine linear constraints. The problem is an excellent example of a highly linearly constrained problem, with a feasibility ratio of 0.111%⁶ (see the Appendix). In other words, it is a highly challenging to find a feasible solution for this constrained problem. The three different strategies were applied to this problem. According to the variable selection strategy, ${x}_{10}, {x}_{11}, and$ ${x}_{12}$ were selected as repairing variables for handling all nine constraints. The global optimum for this problem is -15, and the optimization results by using GA, DE, ES, SRES, and BRKGA with the BU method, without the BU method, and hybrid methods are presented in Tables 3, 4, 5 and 6 and Fig. 4. The Friedman rank test at a significant level of 5% was used to detect differences in treatments across test attempts (Table 4). Also, the Post-hoc Nemenyi Friedman test was performed to identify exactly which groups have different means (Tables 5, 6).

Table 3 Statistical results of different methods on G1 problem (The best performing method (objective function values) is marked in bold).

Table 4 Summary of the p-value of the Friedman rank test over all runs.

Table 5 Summary of the p-value of the posthoc_nemenyi_friedman over all runs.

Table 6 Summary of the p-value of the posthoc_nemenyi_friedman over all runs (continue).

Figure 4

Violin plot of implementation of using different algorithms constraint handling methods (G01 problem).

The comprehensive analysis of algorithm performance on the G01 optimization problem, characterized by 13 continuous decision variables, nine linear constraints, and a quadratic objective function, unveils intricate relationships between problem features and algorithmic behavior. Each algorithm's response to the problem's unique attributes provides valuable insights into their adaptability and efficacy.

The hybrid methods, consistently showcasing superior performance across diverse algorithms, reflect an inherent versatility that aligns with the multi-faceted nature of the G01 problem. The distinct advantage of the hybridization approach becomes particularly evident in the handling of constraints. The BU method, serving as an implicit constraint-handling technique, demonstrates a keen ability to navigate the infeasible search space effectively. This characteristic proves invaluable in problems like G01, where constraints play a crucial role in shaping the feasible region. The hybrid methods, by strategically integrating the BU method, adeptly leverage this constraint-handling mechanism, enabling faster convergence to the feasible area.

Considering the explicit constraints and the quadratic objective function in G01, it becomes apparent that the algorithmic adaptation to these features significantly impacts performance. Genetic algorithms (GA), when combined with the BU method (as hybrid methods), exhibit a noteworthy convergence pattern (as illustrated in Fig. 5). The initial phase, focused on constraints handling, ensures the algorithm converges to the feasible area efficiently. Subsequently, the transition to the optimization process without BU capitalizes on the algorithm's newfound knowledge of the feasible region, resulting in a streamlined path to the final solution. This adaptive approach to the problem's structure showcases how algorithmic behavior is intricately linked to specific features, allowing for a dynamic and context-aware optimization process.

Figure 5

Constraint violation implemented by GA on the G01 test problem.

The inadequacy of the SRES algorithm in this context sheds light on the sensitivity of algorithmic performance to the problem's characteristics. The SRES algorithm, designed for stochastic optimization, might face challenges when confronted with the structured nature of the G01 problem. The lack of significant improvement suggests that algorithmic suitability is not universal but rather contingent on the problem's intricacies. This observation underscores the importance of aligning algorithmic choices with the specific features and constraints of the optimization problem at hand.

Furthermore, the reduction in standard deviation observed in the hybrid methods implies a more stable and reliable convergence behavior. This heightened stability is particularly valuable in the presence of complex problem features, such as the quadratic objective function, where rapid fluctuations in convergence patterns might hinder progress. The hybrid methods' ability to maintain a consistent trajectory towards the optimal solution underscores their resilience in handling the intricacies introduced by specific problem characteristics.

In summary, the algorithmic performance on the G01 problem intricately relates to its unique features. The hybrid methods showcase adaptability and versatility in addressing constraints, demonstrating a keen understanding of the problem structure. The nuanced convergence patterns, strategic switching mechanisms, and sensitivity to problem-specific attributes provide valuable insights into the dynamic interplay between algorithmic behavior and the intricacies of the optimization problem.

Car-side problem

The car-side impact design problem, a notorious challenge in engineering, stands out as a complex and time-consuming optimization task, as depicted in Fig. 6. This problem, adaptable to both single-objective and multi-objective formulations, adds layers of intricacy due to its quadratic objective function and explicit models. Specifically, when addressing the single-objective version with the goal of minimizing car weight, the constraints and objective function take on quadratic forms, amplifying the intricacies involved. The thicknesses of the inner floor side (${X}_{3}$) are strategically selected as decision variables to solve the explicit models, and their bounds are dynamically adjusted to accommodate the interconnected variables, adding an additional layer of complexity.

Figure 6

Schematic of car side impact design problem⁴².

The results of this formidable optimization challenge are presented in Tables 7, 8, 9 and 10 and Figure 7, offering a quantitative assessment of algorithmic performance. Statistical tests (Tables 8, 9, 10) and visualizations in Figure 7 highlight the superiority of the hybrid methods, particularly when combined with the genetic algorithm (GA). This dominance is established through the comparative analysis, showcasing that the hybrid approach consistently outperforms both the individual methods and, notably, requires fewer function evaluations (FE) to attain the final solution.

Table 7 Statistical results of different methods on car side design impact problem.

Table 8 Summary of p value of the Friedman rank test over all runs.

Table 9 Summary of the p-value of the posthoc_nemenyi_friedman over all runs.

Table 10 Summary of the p-value of the posthoc_nemenyi_friedman over all runs (continue).

Figure 7

Violin plot of implementation of using different algorithms constraint handling methods (Carside problem).

Significantly, the examination of the results reveals nuanced differences between the differential evolution (DE) and evolutionary strategy (ES) methods, with the hybrid method showcasing slightly superior outcomes. The incorporation of the BU method, along with the hybrid methods, demonstrates a tangible advantage, resulting in enhanced solution quality. The visual representation of the switching point for GA in Figure 8 provides further clarity on the optimization process. The algorithm strategically transitions from the BU-assisted phase to the standard optimization phase, showcasing an adaptive mechanism designed to optimize the convergence path. However, the study identifies limitations within the context of the car-side impact design problem. While GA, BU, and hybrid methods exhibit notable improvements, the performance of biased random key genetic algorithm (BRKGA) and stochastic ranking evolution strategy (SRES) fails to show enhancement when incorporated into the hybrid framework. This suggests that BRKGA and SRES may not be suitable for addressing the intricacies of this particular problem, underscoring the need for algorithmic adaptability in the face of diverse engineering challenges.

Figure 8

Switch point for GA to normal optimization without the BU method.

In conclusion, the car-side impact design problem proves to be a formidable optimization challenge, and the hybrid methods, particularly in conjunction with GA, emerge as the most effective approach. The adaptability of the hybrid methods, strategic switching mechanisms, and the incorporation of the BU method collectively contribute to enhanced solution quality and computational efficiency. Nonetheless, the study sheds light on the context-specific nature of algorithmic suitability, emphasizing the importance of tailoring optimization strategies to the unique characteristics of complex engineering problems.

Speed reducer

The examination of the speed reducer design problem, featuring one objective function and seven continuous decision variables, has yielded multifaceted insights into the interplay between algorithmic approaches and problem-specific characteristics. With the overarching goal of minimizing the total weight of the speed reducer, the complexity of this task is compounded by a combination of linear and nonlinear constraints. Strategically leveraging three repairing variables—specifically, the face width and shaft diameters (${x}_{1}$, ${x}_{6}$, and ${x}_{7}$)—proves instrumental in managing these constraints effectively.

The results, meticulously presented in Tables 11, 12, 13 and 14 and Fig. 9, reveal a consistent trend: the hybrid approaches consistently outperform standalone methods, yielding the best objective function values across all algorithms considered. This dominance suggests that the hybridization process contributes to algorithmic robustness, offering a versatile strategy to navigate the intricate optimization landscape presented by the speed reducer design problem.

Table 11 Statistical results of different methods on speed reducer problem.

Table 12 Summary of the p-value of the Friedman rank test over all runs.

Table 13 Summary of the p-value of the posthoc_nemenyi_friedman over all runs.

Table 14 Summary of the p-value of the posthoc_nemenyi_friedman over all runs (continue).

Figure 9

Violin plot of implementation of using different algorithms constraint handling methods (speed reducer problem).

Notably, the algorithms (GA, DE, and BRKGA) exhibit improved performance when integrated into hybrid frameworks, signifying a synergistic effect where the strengths of individual algorithms complement one another. The statistical tests conducted (Tables 12, 13, 14) affirm the significance of these improvements, providing a robust validation of the observed enhancements and instilling confidence in the findings.

A closer examination of GA within the hybrid methods reveals its particular suitability for this problem, consistently outperforming standalone and BU-assisted methods. This positions GA as a robust choice for addressing the complexities inherent in the optimization landscape of the speed reducer design problem.

The incorporation of the BU method consistently leads to a substantial reduction in the number of function evaluations required to reach the final solution. This reduction in computational cost is a valuable contribution, emphasizing the efficiency gains afforded by the BU method, a crucial consideration in engineering optimization scenarios.

Despite the positive outcomes, the study highlights that algorithms like BRKGA and SRES do not exhibit improved performance within the hybrid framework, suggesting their unsuitability for this particular problem. This observation underscores the importance of algorithmic adaptability and tailoring strategies to specific problem characteristics.

In conclusion, the insights gained from the speed reducer design problem analysis underscore the effectiveness of hybrid approaches in enhancing algorithmic performance. The adaptability of GA, efficiency gains with the BU method, and the nuanced trade-offs between algorithms offer valuable guidance for tackling real-world engineering optimization challenges. These findings contribute not only to the understanding of algorithmic behavior in this specific context but also pave the way for broader applications in engineering design optimization.

Multi-objective optimization problems

The following subsections provide two benchmarks and two real-world optimization problems to assess the effectiveness of the proposed hybrid method for multi-objective optimization problems.

Osyczka and Kundu (OSY)

Osyczka and Kundu⁴⁴ introduced the OSY test problem, a widely recognized benchmark featuring two nonlinear objective functions and six linear and nonlinear constraints. As highlighted earlier, the BU method is a central focus of this work, particularly in its ability to directly handle constraints and its subsequent integration with other explicit constraint-handling techniques. The BU approach strategically reduces the feasible search space by altering one or more variables, compelling the algorithm to concentrate on exploring regions that adhere to the defined constraints. For the OSY problem, a specific repairing variable selection strategy is employed, targeting variables ${x}_{1}$, ${x}_{4}$, and ${x}_{6}$ identified as effective for handling all constraints inherent in the problem.

Following the application of the BU approach, NSGA-II is employed to navigate the complex landscape of the OSY problem. NSGA-II, known for identifying viable Pareto fronts, initially achieves an entirely feasible population after 1500 evaluations. In contrast, the coupling of the BU method with the algorithm accelerates this process, achieving a fully feasible population after only 300 evaluations. Subsequently, the optimization process seamlessly transitions to routine optimization without the continued use of the BU method.

The evaluation of performance indicators, presented in Tables 15, 16, 17 and 18 and Fig. 10, offers a comprehensive understanding of the algorithmic performance. It becomes evident that the hybrid methods, particularly when incorporating the BU approach, consistently outperform other methods. Statistical analyses, including the Friedman rank and posthoc tests, validate this observation by demonstrating a significant improvement in all performance indicators (excluding IGD+) when the hybrid approach is implemented.

Table 15 Statistical results of different methods on the OSY problem.

Table 16 Summary of the p-value of the Friedman rank test over all runs.

Table 17 Summary of the p-value of the posthoc_nemenyi_friedman over all runs.

Table 18 Summary of the p-value of the posthoc_nemenyi_friedman over all runs (continue).

Figure 10

Violin plot of implementation of using different indicators (OSY problem).

Examining Fig. 10 through the lens of a violin plot further illustrates the superiority of the hybrid methods. The best values for all performance indicators, with the exception of IGD, show marked improvement when the BU and hybrid methods are employed. This enhancement in performance underscores the effectiveness of the BU method in combination with the hybrid approach, providing valuable insights into the algorithmic strategies that lead to improved convergence and solution quality for the OSY problem.

In summary, the OSY problem, coupled with the BU method and hybrid approach, serves as a compelling case study. The specific variable selection strategy, the seamless transition between BU-assisted and routine optimization, and the consistent improvement in performance indicators highlight the adaptability and effectiveness of the proposed approach. These findings contribute not only to the optimization of the OSY problem but also offer broader insights into the potential of the BU method and hybridization in addressing complex, constrained optimization scenarios.

Bin and Korn Test Problem (BNH)

BNH is a test problem proposed by Binh and Korn⁵, characterized by two conflicting objectives and two constraints, introduces a challenging landscape for optimization. The nonlinear nature of the objectives and constraints adds an additional layer of complexity, requiring sophisticated algorithmic strategies to navigate the intricate solution space effectively. The objectives and constraints may exhibit intricate relationships, posing a considerable challenge to traditional optimization approaches.

In addressing this nonlinear bi-objective optimization problem, the algorithmic focus will be on developing robust strategies that balance the optimization of both objectives while adhering to the imposed constraints. The inherent trade-offs between conflicting objectives necessitate a careful exploration of the Pareto front to identify solutions that represent optimal compromises. This problem involves two continuous variables, ${x}_{1}$ and ${x}_{2}$, which can be strategically chosen as repairing variables. The findings presented in Tables 19, 20, 21, 22, 23, 24 and 25 and Fig. 11 illuminate a substantial disparity between the hybrid method and alternative approaches. Particularly noteworthy is the superior performance of the hybrid method, especially when objective function tolerance is considered, showcasing noteworthy enhancements over other methods across various cases. The results unequivocally demonstrate the hybrid approaches' consistent outperformance in tackling this bi-objective optimization problem characterized by nonlinear constraints and objective functions. Furthermore, the performance indicator values associated with the hybrid approaches are significantly smaller compared to their counterparts, underscoring the effectiveness of the proposed methodology in achieving superior convergence and solution quality. These insights not only contribute to the optimization of the BNH problem but also offer valuable considerations for addressing the nuances of nonlinear bi-objective optimization challenges across diverse applications.

Table 19 Statistical results of different methods on the BNH problem.

Table 20 Summary of the p-value of the Friedman rank test over all runs.

Table 21 Summary of the p-value of the posthoc_nemenyi_friedman over all runs.

Table 22 Summary of the p-value of the posthoc_nemenyi_friedman over all runs- IGD value comparison.

Table 23 Summary of the p-value of the posthoc_nemenyi_friedman over all runs-IGD+value comparison.

Table 24 Summary of the p-value of the posthoc_nemenyi_friedman over all runs- GD value comparison.

Table 25 Summary of the p-value of the posthoc_nemenyi_friedman over all runs- GD+value comparison.

Figure 11

Violin plot of implementation of using different indicators (BNH problem).

Welded beam design problem

The welded beam fabrication is a well-known engineering optimization benchmark problem⁴⁸ that examines the trade-off between the strength and cost of a beam. The problem includes four constraints (Eqs. 32–35) and four design variables, which are:

• ${x}_{1}$ = h, the thickness of the welds

• ${x}_{2}$ = l, the length of the welds

• ${x}_{3}$ = t, the height of the beam

• ${x}_{4}$ = b, the width of the beam

The constraints are:

$${g}_{1}=\left(\frac{1}{{t}_{max}}\right)\times \left(t-{t}_{max}\right)\le 0$$

(33)

$${g}_{2}=\left(\frac{1}{{s}_{max}}\right)\times \left(s-{s}_{max}\right)\le 0$$

(34)

$${g}_{3}=\left(\frac{1}{5-0.125}\right)\times \left({x}_{1}-{x}_{4}\right)\le 0$$

(35)

$${g}_{4}=\left(\frac{1}{P}\right)\times (P-{P}_{c})\le 0$$

(36)

where ${t}_{max}=13600, {s}_{max}=30000, P=6000, {P}_{c}=64746.022 \times \left(1 - 0.0282346 \times {x}_{3}\right)\times {x}_{3}\times {x}_{4}^{3}, s=6\times P\times 14/({x}_{4}\times {x}_{3}^{2})$. The two objective functions are to minimize the fabrication cost of the beam and minimize the deflection of the end of the beam:

$${f}_{1}=1.10471 \times {x}_{1}^{2}\times {x}_{2}+0.04811\times {x}_{3}\times {x}_{4}\times (14.0+{x}_{2}))$$

(37)

$${f}_{2}=2.1952/({x}_{4}\times {x}_{3}^{3})$$

(38)

The four variables of this problem include: (1) thickness of the weld, (2) welded joint's length, (3) beam's width, and (4) beam's thickness. Using the three strategies, NSGA-II was applied to this problem, and fitness values were evaluated.

The exploration of sampling and crossover operators in EAs stands as a critical endeavor, influencing the convergence speed and efficacy of these algorithms, particularly in the context of noisy optimization. In the context of the Welded Beam Design problem, where the impact of noise is significant, the experimentation involved diverse operators, including real random sampling, Latin hypercube sampling, and two distinct crossovers: real-SBX crossover and uniform crossover. Four performance indicators, namely GD, GD+ , IGD, and IGD+, were meticulously evaluated against these operators to glean insights into their comparative effectiveness. The compelling findings, detailed in Tables 26, 27, 28, 29, 30, 31 and 32 and illustrated in Fig. 12, underscore the superiority of the hybrid approach over the two alternative methods.

Table 26 Statistical results of different methods on the welded beam design problem (sampling).

Table 27 Statistical results of different methods on the welded beam design problem (crossover).

Table 28 Summary of the p-value of the Friedman rank test over all runs.

Table 29 Summary of the p-value of the posthoc_nemenyi_friedman test evaluated over all runs (sampling).

Table 30 Summary of the p-value of the posthoc_nemenyi_friedman test evaluated over all runs (sampling-Continue).

Table 31 Summary of p value of the posthoc_nemenyi_friedmantest evaluated over all runs (crossover).

Table 32 Summary of p value of the posthoc_nemenyi_friedmantest evaluated over all runs (crossover-Continue).

Figure 12

Pareto-optimal fronts obtained for the welded beam design problem for both BU and without BU compared with the true Pareto-optimal front (Generation 20).

Examining the Pareto optimal solutions generated by the hybrid methods across different generations reveals a consistent outperformance, particularly noteworthy when considering constraint violations as tolerance with real random sampling. This suggests that the hybrid approach, when coupled with appropriate operators, significantly improves the optimization process for the Welded Beam Design problem. Notably, the hybrid method's effectiveness is evident with Latin hypercube sampling, showcasing its compatibility and superior performance within this optimization context. Furthermore, the hybrid method incorporating objective function tolerance using uniform crossover emerges as a robust option, outperforming optimization without the BU approach.

In contrast to initial expectations regarding the BU approach as a constraint handling technique, the results indicate that it may not be the optimal choice for this specific nonlinear bi-objective model. However, the hybrid approaches, through strategic integration of various operators, demonstrate a substantial enhancement in performance. This nuanced observation highlights the importance of selecting appropriate operators within hybrid approaches, suggesting that the effectiveness of the BU method can be context-dependent.

The statistical analyses, including the Friedman rank test and the Posthoc_Nemenyi_Friedman, contribute to the robustness of the findings. These tests not only affirm the significant differences between the hybrid approach and the two other methods but also reinforce the superiority of the proposed hybrid method for solving the Welded Beam Design problem. The identification of the switching point for the second phase of the optimization process at evaluation 300 further exemplifies the efficiency of the hybrid approach compared to the without BU approach, where the feasible region is reached at evaluation 1600 (Fig. 13).

Figure 13

Constraint violation evaluated by NSGA-II (convergence of the whole population to feasible solutions).

In conclusion, the investigation into sampling and crossover operators, coupled with the nuanced insights from performance indicators and statistical analyses, positions the hybrid approach as the preferred strategy for solving the Welded Beam Design problem. The adaptability of the hybrid method, evident in its performance across various operators, reinforces its efficacy in addressing the challenges posed by the specific characteristics of this nonlinear bi-objective optimization scenario.

Cantilevered stepped beam design problem

The cantilevered stepped design problem is an example of large-scale size problem. In the cantilevered beam design problem, the stepped cantilever beam must be able to carry a prescribed end load⁴⁶. This problem is originally a single-objective optimization problem considering minimizing the beam volume; however, this study has extended the original problem and added one more objective function, which minimize end deflection subject to various engineering design variables (see Appendix). The beam supports the given load, P, at a fixed distance L from the support. Besides, designers of the beam can vary the width (${b}_{i}$) and height (${h}_{i}$) of each section (Fig. 14). The problem is highly constraint and large-dimensional; however, for this example five segments (N = 5) including 10 dimensions (10 constraints) are considered. In the pursuit of solving both the single- and multi-objective versions of this intricate problem, repairing variables ${x}_{2}$, ${x}_{4}, {x}_{6,}$ ${x}_{8},$ and ${x}_{10}$ have been identified. The application of the BU method to this constrained nonlinear optimization problem is showcased, providing a valuable demonstration of the BU method's utility. Figure 15a–d visually presents the optimal solutions discovered by NSGA-II employing different methods across various generations.

Figure 14

Schematic of the stepped beam design problem.

Figure 15

Pareto solutions found by NSGA-II by different approaches and generations.

To assess the performance of the proposed approaches, the hypervolume (HV) indicator is utilized, offering a comprehensive measure of the quality of solutions (Tables 33, 34). The results, detailed in Tables 35, 36, 37, 38 and 39 and visualized in Fig. 16, unequivocally demonstrate the efficacy of the hybrid approaches in enhancing solution quality. This improvement is particularly noteworthy in the case of Latin hypercube sampling and simulated binary crossover (SBX), where the hybrid approaches exhibit a significant enhancement in solution quality.

Table 33 Results of different methods on the cantilevered stepped design problem (HV values-sampling).

Table 34 Results of different methods on the the cantilevered stepped design problem (HV values-crossover).

Table 35 Summary of the p-value of the Friedman rank test over all runs.

Table 36 Summary of the p-value of the posthoc_nemenyi_friedman test evaluated over all runs (sampling).

Table 37 Summary of the p-value of the posthoc_nemenyi_friedman test evaluated over all runs (sampling-Continue).

Table 38 Summary of p value of the posthoc_nemenyi_friedmantest evaluated over all runs (crossover).

Table 39 Summary of p value of the posthoc_nemenyi_friedmantest evaluated over all runs (crossover-Continue).

Figure 16

Violin plot of implementation of using different methods (Stepped beam design problem problem).

This example provides valuable insights into the application of the BU method in addressing complex, large-scale, constrained optimization problems. The extension of the original single-objective problem to a multi-objective setting showcases the flexibility of the proposed methodology. The utilization of hypervolume as a performance indicator reaffirms the substantial improvements achieved by the hybrid approaches. Overall, this study contributes not only to the optimization of the Cantilevered Stepped Design problem but also sheds light on the broader potential of the BU method in tackling diverse engineering optimization challenges.

Multi-objective car-side impact problem formulation

In scenarios where the black-box constraints of an optimization problem exhibit highly complex behavior, the direct evaluation of these constraints can be prohibitively time-consuming. In such instances, the use of surrogates becomes a valuable strategy to mitigate the complexity and expedite the optimization process. Surrogates, effectively approximating the actual constraints, enable the optimization algorithm to learn from previous evaluations and make informed decisions. The surrogate model, if reducible to a single variable, provides an opportunity to apply the BU approach to solve the constrained surrogate model efficiently, as detailed in the Appendix.

The car side impact problem stands out as a prime example of a complex and time-consuming optimization challenge, focusing on the multi-objective optimization of a vehicle's side impact crashworthiness 50. Repairing variables ${X}_{2} and{ X}_{3}$ are identified for this problem. Various evolutionary algorithms, including NSGA-II, NSGA-III, UNSGA-III, and AGEMOEA, are implemented to explore the solution space. Tables 40, 41, 42, 43, 44, 45 and 46 present the results of the non-dominated solutions obtained by these methods.

Table 40 Results of different methods on the multi-objective car-side impact problem (HV values-sampling).

Table 41 Results of different methods on the multi-objective car side design problem (HV values-crossover).

Table 42 Summary of the p-value of the Friedman rank test over all runs.

Table 43 Summary of the p-value of the posthoc_nemenyi_friedman test evaluated over all runs (sampling).

Table 44 Summary of the p-value of the posthoc_nemenyi_friedman test evaluated over all runs (sampling).

Table 45 Summary of the p-value of the posthoc_nemenyi_friedman test evaluated over all runs (Crossover).

Table 46 Summary of the p-value of the posthoc_nemenyi_friedman test evaluated over all runs (Crossover).

The results underscore the superiority of the hybrid methods, implemented with the proposed evolutionary algorithms, in terms of diversity. The hybrid approaches exhibit a well-distributed population across the entire search space, leading to enhanced performance. While the application of the BU approach alone may face efficiency challenges, particularly in the context of a three-objective optimization problem, the hybrid approach proves adept at overcoming these difficulties. Statistical analyses of the results reveal that, in some cases and with different algorithms, the BU method may not be as efficient, yet the hybrid approach consistently outperforms it, signifying the added benefits of the hybridization strategy.

Comparative statistical tests emphasize the significant advantage of the hybrid approaches over the BU method. Notably, although some statistical tests indicate no significant difference between the hybrid and BU methods, the hybrid method consistently achieves higher hypervolume (HV) values. This suggests that, even when statistical significance is not apparent, the hybrid approach provides a qualitative improvement in solution quality.

In conclusion, the exploration of the car side impact problem demonstrates the efficacy of hybrid approaches in addressing complex, multi-objective optimization challenges with intricate black-box constraints. The incorporation of surrogates and the application of the BU method within a hybrid framework contribute to improved diversity and solution quality. These findings not only advance the optimization of the car side impact problem but also offer valuable insights into the broader applicability of hybrid evolutionary algorithms in addressing real-world, complex optimization scenarios.

Discussion

The BU method is an implicit constraint-handling technique coupled with an explicit constraint-handling technique that aims to cut the infeasible search space over iterations to find the feasible region faster. Since optimization is an iterative process whose boundaries can be changed during the process, the i-th decision variable boundaries in each iteration exhibit a dynamic nature. By applying this philosophy to boost the optimization process in the BU method, the bounds of selected variable (s) are repaired (repairing variables) to satisfy the constraints. As a result, new solutions are created within the updated boundary of the latest search space. The above-mentioned process helps the algorithm search for a solution within the feasible region instead of the whole area. Although the BU method directs the search operator to the feasible space and reaches the feasible region in a reasonable time, it twists the search space, making the optimization problem more challenging. As such, the results are not always accurate, and the algorithm may result in premature convergence. Therefore, in this paper, two “hybrid” switching mechanisms are proposed. In the proposed approach, the BU method is applied to the problem after initializing the algorithm, and in this phase, a threshold tolerance is set. The BU method optimizes the problem based on the threshold tolerances so that the population has zero constraint violations or there is no more change in the objective space. Then, the optimization problem is continued with the BU method.

The results suggest that:

• Advantageously, the hybrid method can be used with and without the BU method. In the first phase of the hybrid method, the BU method helps the optimization algorithm to avoid searching inside the infeasible region and boosts it to explore the feasible area more efficiently. This way, the optimization algorithm finds the feasible area with fewer FEs. However, because the BU brings about changes to the landscape of the search space, it may result in premature convergence. Hence, in the second phase of the hybrid method, the optimization algorithm searches the feasible area without using the BU approach.

• GA has the least standard deviation among the other algorithms for the single-objective function optimization problem.

• The statistical tests for single-objective optimization problems show there is a significant difference between hybrid methods and two other methods (with BU and without BU) using GA. Moreover, GA works well for this method and boosts optimization problems. Due to the fact that GA can benefit from different operator settings, such as crossover and sampling, better results by choosing the appropriate operator setting may be achieved.

• Regarding the multi-objective optimization problems, the hybrid method could find more non-dominated solutions with better distribution compared to with and without the BU methods. The BNH test problems indicated that the hybrid method produced promising results for IGD and HV values compared to with BU and without BU approaches.

• Although the BU and hybrid methods outperformed the other algorithms in single- and multi-objective optimization, they are still case-dependent, and there are EAs that are not suitable in some cases.

• By boosting the optimization process, the hybrid method finds the first and whole populations much faster than the classic BU and without BU methods for most problems (i.e., the single- or multi-objective optimization problems of this paper). Moreover, regarding the car side impact design problem, the hybrid method obtained good diversity, whereas without the BU method searched for solutions in a specific feasible space.

• For some case studies, although the hybrid method resulted in the same (or close-to-final) solution as the two other methods, it requires fewer FEs to converge to a solution.

• Compared to unconstrained optimization problems, constrained optimization problems are more challenging since many infeasible regions appear in the search space (meaning the hit ratio is low). This makes solving a constrained problem more challenging, especially highly constrained problems that can lead to difficulties such as convergence-, diversity-, and feasibility-hardness. Therefore, finding just one feasible solution is a significant achievement. The primary goal of the BU method is to minimize the search space by narrowing down the variable range and eliminating infeasible options. To compare the effectiveness of the proposed approach, the first feasible solution found by different methods is compared to one another, and the whole population is tracked against generations. The hybrid method uses this philosophy to switch the optimization process and avoid reaching premature convergence. On the other hand, once the whole feasible area is found by the BU method, the final solution (global) could be achieved with fewer FEs in a reasonable time.

• Regarding the multi-objective car side impact design problem, the hybrid approach works significantly better than without the BU method when incorporated in the employed multi-objective algorithms. It is worth mentioning that although there is no significant difference in the final HV values between the hybrid and BU methods, as mentioned earlier, the hybrid method could achieve the final solution in fewer FEs.

• In scenarios where explicit formulation of constraints is challenging, the use of surrogate models, also known as metamodels, proves to be a valuable solution. Surrogate models have demonstrated effectiveness in situations where deriving explicit formulations for constraints is impractical. These models serve as approximations of intricate simulations, providing an efficient means to evaluate constraints during optimization without the computational overhead of running resource-intensive simulations. For instance, consider optimization problems associated with finite element models, such as those involving nodal displacements and bar stresses in truss optimization. In these cases, surrogate models can be strategically trained to emulate the intricate behavior observed in simulation results. This approach becomes particularly advantageous when the direct formulation of these constraints proves to be impractical due to their complexity or reliance on computationally expensive procedures.

In EAs, sampling and crossover are critical factors that can strongly affect convergence speed and are often used in noisy optimization problems where reducing the negative impact of noise is crucial. Moreover, sampling is a popular strategy for dealing with noise. Therefore, experiments with different sampling operators, including real random sampling, Latin hypercube sampling, and two different real-SBX and uniform crossovers, were conducted to reduce the noise impact.

Limitations

Although this study provides valuable insights into optimization problems, there are certain limitations to this work. One significant limitation of this work lies in the relatively small size of variables in the problems. It should be noted that this study aims to contribute by addressing a specific aspect of the optimization problem. In this study, the benchmark libraries have not been used and some specific problems with solvable constraints with respect to at least one variable have been used. While the limited number of variables in our optimization problems may restrict the immediate applicability of our findings to large-scale size problems, it is clear the insights gained from this research serve as a valuable foundation for further research in the field.

Another limitation of this study is related to the experimental section, where the algorithms compared with the proposed algorithm are not the most recent. It is important to note that the aim of this work is not focused on proposing and developing state-of-the-art algorithms; rather, the advancement of constraint handling techniques and the development of the BU approach are being concentrated on. Consequently, well-known algorithms such as NSGA-II in the analysis have been included. However, as a future study, it would be valuable to consider incorporating state-of-the-art algorithms to provide a more comprehensive evaluation of our proposed approach in comparison to the latest advancements in the field.

Moreover, while the current selection of repairing variables is designed to be applicable even in scenarios with multiple disjoint feasible regions, the evaluation of constraints can pose challenges in situations where the behavior is non-trivial. In such cases, alternative approaches to repairing variable selection, involving book-keeping or linking of these disjoint feasible ranges, may be necessary. This intriguing aspect has been highlighted and it is proposed as a subject for future investigation, intending to delve deeper into the nuanced evaluation of constraints in conditions assessment.

Conclusion

This study presents an extension of the work proposed by⁶ for solving constrained optimization problems efficiently. The proposed methods use an implicit constraint handling approach called boundary updating (BU), which focuses on directly handling constraints and is coupled with other explicit constraint-handling techniques (here, feasibility rules).

In this study, two augmented versions of the BU method were applied to single-objective, multi-objective, and many-objective constrained optimization problems, including benchmarks and real-world problems. To accomplish this goal, two thresholds are considered, each representing different switching method. In the initial method, the optimization process shifts to a state that does not involve the BU approach once constraint violations reach zero. In the second approach, the optimization process enters a phase without BU methods when there is no longer any observed change in the objective space. For single-objective constrained optimization problems, popular algorithms like GA, DE, ES, SRES, and BRKGA were employed, and NSGA-II, a renowned algorithm, was used for bi-objective problems. For the multi-objective constrained optimization problems, NSGA-III, UNSGA-III, and AGEMOEA were applied. The results show that the hybrid approach performs significantly better than with- and without the BU methods, and can find better solutions with fewer iterations.

As a future study, it is suggested that the proposed BU method be coupled with other constraint-handling techniques, for example different types of penalty methods with different objective functions. Also, additional test problems in the literature, such as many-objective and dynamic functions, could be considered for further validating the proposed hybrid approach. After BU generalization, it can also be applied to benchmark libraries such as CEC, NEVERGRAD, BBOB, GNBG, and DIRECTGOLib. Although BU is not defined for any specific optimization algorithms, implementing BU to a wider range of optimization algorithms can be investigated in another future study. Moreover, as a prospective avenue for future research, exploring the applicability of the proposed approach to constrained problems featuring multiple disconnected regions is a promising direction. Many constrained optimization scenarios exhibit this characteristic, with only certain regions potentially close to the true optimum. Investigating the effectiveness and adaptability of our approach in addressing such complex problem structures holds significant potential for enhancing the robustness and versatility of the proposed methodology. This important consideration will be a key focus in our forthcoming studies.

Appendix

G01 formulation:

$${f}_{1}\left(x\right)=5\times \left({x}_{1}+{x}_{2}+{x}_{3}+{x}_{4}\right)-5\times \left({x}_{1}^{2}+{x}_{2}^{2}+{x}_{3}^{2}+{x}_{4}^{2}\right)$$

$$-({x}_{5}+{x}_{6}+{x}_{7}+{x}_{8}+{x}_{9}+{x}_{10}+{x}_{11}+{x}_{12}+{x}_{13})$$

(39)

S.t.

$${g}_{1}\left(x\right)=2\times {x}_{1}+2\times {x}_{2}+{x}_{10}+{x}_{11}\le 10$$

(40)

$${g}_{2}\left(x\right)=2\times {x}_{1}+2\times {x}_{3}+{x}_{10}+{x}_{12}\le 10$$

(41)

$${g}_{3}\left(x\right)=2\times {x}_{2}+2\times {x}_{3}+{x}_{11}+{x}_{12}\le 10$$

(42)

$${g}_{4}\left(x\right)=-8\times {x}_{1}+{x}_{10}\le 0$$

(43)

$${g}_{5}\left(x\right)=-8\times {x}_{2}+{x}_{11}\le 0$$

(44)

$${g}_{6}\left(x\right)=-8\times {x}_{3}+{x}_{12}\le 0$$

(45)

$${g}_{7}\left(x\right)=-2\times {x}_{4}-{x}_{5}+{x}_{10}\le 0$$

(46)

$${g}_{8}\left(x\right)=-2\times {x}_{6}-{x}_{7}+{x}_{11}\le 0$$

(47)

$${g}_{9}\left(x\right)=-2\times {x}_{8}-{x}_{9}+{x}_{12}\le 0$$

(48)

$$0\le {x}_{1}\cdots {x}_{9}\le 1, 0\le {x}_{10},{x}_{11},{x}_{12}\le 100, 0\le {x}_{13}\le 1$$

(49)

Car-side problem formulation:

$${f}_{1}\left(x\right)=1.98+4.90\times {x}_{1}+6.67\times {x}_{2}+6.98\times {x}_{3}+4.01\times {x}_{4}+1.78\times {x}_{5}+2.73\times {x}_{7}$$

(50)

$${g}_{1}\left(x\right)=1.16-0.3717\times {x}_{2}\times {x}_{4}-0.00931{\times x}_{2}\times {x}_{10}-0.484\times {x}_{3}\times {x}_{9}+0.01343\times {x}_{6}\times {x}_{10}$$

(51)

$${g}_{2}\left(x\right)=46.36-9.9\times {x}_{2}-12.9\times {x}_{1}\times {x}_{8}+0.1107\times {x}_{3}\times {x}_{10}$$

(52)

$${g}_{3}\left(x\right)=33.86+2.95\times {x}_{3}+0.1792\times {x}_{10}-5.057\times {x}_{1}\times {x}_{2}-11.0\times {x}_{2}\times {x}_{8}-0.021\times {x}_{5}\times {x}_{10}-9.98\times {x}_{7}\times {x}_{8}+22.0\times {x}_{8}\times {x}_{9}$$

(53)

$${g}_{4}\left(x\right)=28.98+3.818\times {x}_{3}-4.2\times {x}_{1}\times {x}_{2}-5.0207\times {x}_{5}\times {x}_{10}+6.63\times {x}_{6}\times {x}_{9}-7.7\times {x}_{7}\times {x}_{8}+0.32\times {x}_{9}\times {x}_{10}$$

(54)

$${g}_{5}\left(x\right)=0.261-0.0159\times {x}_{1}\times {x}_{2}-0.188\times {x}_{1}\times {x}_{8}-0.019\times {x}_{2}\times {x}_{7}+0.0144\times {x}_{3}\times {x}_{5}+0.0008757\times {x}_{5}\times {x}_{10}+0.08045\times {x}_{6}\times {x}_{9}+0.00139\times {x}_{8}\times {x}_{11}+0.00001575\times {x}_{10}\times {x}_{11}$$

(55)

$${g}_{6}\left(x\right)=0.214+0.00817\times {x}_{5}-0.0131\times {x}_{1}\times {x}_{8}-0.0704\times {x}_{1}\times {x}_{9}+0.03099\times {x}_{2}\times {x}_{6}-0.018\times {x}_{2}\times {x}_{7}+0.0208\times {x}_{3}\times {x}_{8}+0.121\times {x}_{3}\times {x}_{9}-0.00364\times {x}_{5}\times {x}_{6}+0.0007715\times {x}_{5}\times {x}_{10}-0.0005354\times {x}_{6}\times {x}_{10}+0.00121\times {x}_{8}\times {x}_{11}+0.00184\times {x}_{9}\times {x}_{10}-0.02\times {x}_{2}^{2}$$

(56)

$${g}_{7}\left(x\right)=0.74-0.61\times {x}_{2}-0.163\times {x}_{3}\times {x}_{8}+0.001232\times {x}_{3}\times {x}_{10}-0.166\times {x}_{7}\times {x}_{9}+0.227\times {x}_{2}^{2}$$

(57)

$${g}_{8}\left(x\right)=4.72-0.5\times {x}_{4}-0.19\times {x}_{2}\times {x}_{3}-0.0122\times {x}_{4}\times {x}_{10}+0.009325\times {x}_{6}\times {x}_{10}+0.000191\times {x}_{11}^{2}$$

(58)

$${g}_{9}\left(x\right)=10.58-0.674\times {x}_{1}\times {x}_{2}-1.95\times {x}_{2}\times {x}_{8}+0.02054\times {x}_{3}\times {x}_{10}-0.0198\times {x}_{4}\times {x}_{10}+0.028\times {x}_{6}\times {x}_{10}$$

(59)

$${g}_{10}=16.45-0.489\times {x}_{3}\times {x}_{7}-0.843\times {x}_{5}\times {x}_{6}+0.0432\times {x}_{9}\times {x}_{10}-0.0556\times {x}_{9}\times {x}_{11}-0.000786\times {x}_{11}^{2}$$

(60)

$$0.5\le {x}_{1},{x}_{3},{x}_{4}\le 1.5, 0.45\le {x}_{2}\le 1.35, 0.875\le {x}_{5}\le 2.625, 0.4\le {x}_{6},{x}_{7}\le 1.2, 0.19\le {x}_{8},{x}_{9}\le 0.35, -30\le {x}_{10},{x}_{11}\le 30$$

(61)

Speed reducer formulation:

$$\mathrm{min }z=0.785\times {x}_{1}\times {x}_{2}^{2}\times (3.33\times {x}_{3}^{2}+14.9334\times {x}_{3}-43.0934)-1.508\times {x}_{1}\times ({x}_{6}^{2}+{x}_{7}^{2})+7.4777\times ({x}_{6}^{3}+{x}_{7}^{3})+ 0.7854\times ({x}_{4}\times {x}_{6}^{2}+{x}_{5}\times {x}_{7}^{2})$$

(62)

$${\text{S}}.{\text{t}}.$$

(63)

$$27/({x}_{1}\times {x}_{2}^{2}\times {x}_{3})\le 1$$

(64)

$$397.5/{x}_{1}\times {x}_{2}^{2}\times {x}_{3}^{2}\le 1$$

(65)

$$1.93\times {x}_{4}^{3}/{x}_{2}\times {x}_{3}{\times x}_{6}^{4}\le 1$$

(66)

$$1.93\times {x}_{5}^{3}/{(x}_{2}\times {x}_{3}\times {x}_{7}^{4})\le 1$$

(67)

$$\sqrt{{\left(745\times \frac{{x}_{4}}{{x}_{2}\times {x}_{3}}\right)}^{2}+1.69\times {10}^{7}}/(110\times {x}_{6}^{3})\le 1$$

(68)

$$\sqrt{{\left(745\times \frac{{x}_{5}}{{x}_{2}\times {x}_{3}}\right)}^{2}+1.575\times {10}^{8}}/(85\times {x}_{7}^{3})\le 1$$

(69)

$$({x}_{2}\times {x}_{3})/40\le 1$$

(70)

$$(5\times {x}_{2})/{x}_{1}\le 1$$

(71)

$${x}_{1}/(12\times {x}_{2})\le 1$$

(72)

$$(1.5\times {x}_{6}+1.9)/{x}_{4}\le 1$$

(73)

$$(1.1\times {x}_{7}+1.9)/{x}_{5}\le 1$$

(74)

$$2.6\le {x}_{1}\le \mathrm{3.6,0.7}\le {x}_{2}\le 0.8, 17\le {x}_{3}\le 28,$$

$$7.3\le {x}_{4}\le 8.3, 7.8\le {x}_{5}\le 8.3, 2.9\le {x}_{6}\le 3.9,$$

$$5.0\le {x}_{7}\le 5.5$$

(75)

OSY formulation:

$$\mathrm{min }{f}_{1}\left(x\right)=-[25\times {\left({x}_{1}-2\right)}^{2}+{\left({x}_{2}-2\right)}^{2}+{\left({x}_{3}-1\right)}^{2}+{\left({x}_{4}-4\right)}^{2}+{\left({x}_{5}-1\right)}^{2}]$$

(76)

$$\mathrm{min }{f}_{2}\left(x\right)={x}_{1}^{2}+{x}_{2}^{2}+{x}_{3}^{2}+{x}_{4}^{2}+{x}_{5}^{2}+{x}_{6}^{2}$$

$${\text{S}}.{\text{t}}.$$

(77)

$${C}_{1}\left(x\right)={x}_{1}+{x}_{2}-2\ge 0$$

(78)

$${C}_{2}\left(x\right)={6-x}_{1}-{x}_{2}\ge 0$$

(79)

$${C}_{3}\left(x\right)={2-{x}_{2}+x}_{1}\ge 0$$

(80)

$${C}_{4}\left(x\right)={2-{x}_{1}+3\times x}_{2}\ge 0$$

(81)

$${C}_{5}\left(x\right)={4-{{(x}_{3}-3)}^{2}-x}_{4}\ge 0$$

(82)

$${C}_{6}\left(x\right)={{{(x}_{5}-3)}^{2}+x}_{6}-4\ge 0$$

(83)

$$0\le {x}_{1},{x}_{2},{x}_{6}\le 10\cdots 1\le {x}_{3},{x}_{5}\le 5\cdots 1\le {x}_{4}\le 6$$

(84)

BNH formulation:

$$\mathrm{min }{f}_{1}\left(x\right)=4\times {x}_{1}^{2}+4\times {x}_{2}^{2}$$

(85)

$$\mathrm{min }{f}_{2}\left(x\right)={({x}_{1}-5)}^{2}+{({x}_{2}-5)}^{2}$$

(86)

$${\text{S}}.{\text{t}}.$$

(87)

$${C}_{1}\left(x\right)={({x}_{1}-5)}^{2}+{x}_{2}^{2}\le 25$$

(88)

$${C}_{2}\left(x\right)={({x}_{1}-8)}^{2}+{({x}_{2}+3)}^{2}\ge 7.7$$

(89)

$$0\le {x}_{1}\le 5\cdots 0\le {x}_{2}\le 3$$

(90)

Cantilevered beam design problem:

$$\mathrm{Min }{{\text{f}}}_{1}\left({\text{x}}\right)={100\times ({\text{x}}}_{5}\times {{\text{x}}}_{6}+{{\text{x}}}_{7}\times {{\text{x}}}_{8}+{{\text{x}}}_{9}\times {{\text{x}}}_{10}+{{\text{x}}}_{1}\times {{\text{x}}}_{2}+{{\text{x}}}_{3}\times {{\text{x}}}_{4})$$

(91)

$$\mathrm{Min }{{\text{f}}}_{2}\left({\text{x}}\right)=10000\times (\frac{244}{{{\text{x}}}_{5}\times {{\text{x}}}_{6}^{3}}+\frac{148}{{{\text{x}}}_{7}\times {{\text{x}}}_{8}^{3}}+\frac{76}{{{\text{x}}}_{9}\times {{\text{x}}}_{10}^{3}}+\frac{28}{{{\text{x}}}_{1}\times {{\text{x}}}_{2}^{3}}+\frac{4}{{{\text{x}}}_{3}\times {{\text{x}}}_{4}^{3}})$$

(92)

Subject to

$${{\text{g}}}_{1}\left({\text{x}}\right)=10.7143-\frac{{{\text{x}}}_{5}\times {{\text{x}}}_{6}^{2}}{1000}\le 0$$

(93)

$${{\text{g}}}_{2}\left({\text{x}}\right)=8.5714-\frac{{{\text{x}}}_{7}\times {{\text{x}}}_{8}^{2}}{1000}\le 0$$

(94)

$${{\text{g}}}_{3}\left({\text{x}}\right)=6.4286-\frac{{{\text{x}}}_{9}\times {{\text{x}}}_{10}^{2}}{1000}\le 0$$

(95)

$${{\text{g}}}_{4}\left({\text{x}}\right)=4.2857-\frac{{{\text{x}}}_{1}\times {{\text{x}}}_{2}^{2}}{1000}\le 0$$

(96)

$${{\text{g}}}_{5}\left({\text{x}}\right)=2.1429-\frac{{{\text{x}}}_{3}\times {{\text{x}}}_{4}^{2}}{1000}\le 0$$

(97)

$${{\text{g}}}_{6}\left({\text{x}}\right)={{\text{x}}}_{6}-20\times {{\text{x}}}_{5}\le 0$$

(98)

$${{\text{g}}}_{7}\left({\text{x}}\right)={{\text{x}}}_{8}-20\times {{\text{x}}}_{7}\le 0$$

(99)

$${{\text{g}}}_{8}\left({\text{x}}\right)={{\text{x}}}_{10}-20\times {{\text{x}}}_{9}\le 0$$

(100)

$${{\text{g}}}_{9}\left({\text{x}}\right)={{\text{x}}}_{2}-20\times {{\text{x}}}_{1}\le 0$$

(101)

$${{\text{g}}}_{10}\left({\text{x}}\right)={{\text{x}}}_{4}-20\times {{\text{x}}}_{3}\le 0$$

(102)

$${{\text{x}}}_{{\text{l}}}= [1, 30, 2.4,\mathrm{ 45,2.4,45,1},\mathrm{30,1},30]$$

(103)

$${{\text{x}}}_{{\text{u}}}=[5, 65, 3.1, 60, 3.1, 60, 5, 65, 5, 65]$$

(104)

Multi-objective car side formulation:

$${f}_{1}\left(x\right)=1.98+4.9\times {x}_{1}+6.67\times {x}_{2}+6.98\times {x}_{3}+4.01\times {x}_{4}+1.78\times {x}_{5}+0.00001\times {x}_{6}+2.73{\times x}_{7}$$

(105)

$${f}_{2}\left(x\right)=4.72-0.5\times {x}_{4}-0.19\times {x}_{2}\times {x}_{3}$$

(106)

$${f}_{3}\left(x\right)=0.5\times (10.58-0.674\times {x}_{1}\times {x}_{2}-0.67275\times {x}_{2}+16.45-0.489\times {x}_{3}\times {x}_{7}-0.843\times {x}_{5}\times {x}_{6}$$

(107)

$${g}_{1}\left(x\right)=1.16-0.3717\times {x}_{2}\times {x}_{4}-0.0092928\times {x}_{3}\le 1$$

(108)

$${g}_{2}\left(x\right)=0.261-0.0159\times {x}_{1}\times {x}_{2}-0.06486\times {x}_{1}-0.019\times {x}_{2}\times {x}_{7}+0.0144\times {x}_{3}\times {x}_{5}+0.0154464\times {x}_{6}\le 0.32$$

(109)

$${g}_{3}\left(x\right)=0.214+0.00817\times {x}_{5}-0.045195\times {x}_{1}-0.0135168\times {x}_{1}+0.03099\times {x}_{2}\times {x}_{6}-0.018\times {x}_{2}\times {x}_{7}+0.007176\times {x}_{3}+0.023232\times {x}_{3}-0.00364\times {x}_{5}\times {x}_{6}-0.018\times {x}_{2}^{2}\le 0.32$$

(110)

$${g}_{4}\left(x\right)=0.74-0.61\times {x}_{2}-0.31296\times {x}_{3}-0.031872\times {x}_{7}+0.227\times {x}_{2}^{2}\le 0.32$$

(111)

$${g}_{5}\left(x\right)=28.98+3.818\times {x}_{3}-4.2\times {x}_{1}\times {x}_{2}+1.2729\times {x}_{6}-2.68065\times {x}_{7}\le 32$$

(112)

$${g}_{6}\left(x\right)=33.86+2.95\times {x}_{3}-5.057\times {x}_{1}\times {x}_{2}-3.795\times {x}_{2}-3.4431\times {x}_{7}+1.45728\le 32$$

(113)

$${g}_{7}\left(x\right)=46.36-9.9\times {x}_{2}-4.4505\times {x}_{1}\le 32$$

(114)

$${g}_{8}\left(x\right)=4.72-0.5\times {x}_{4}-0.19\times {x}_{2}\times {x}_{3}\le 4$$

(115)

$${g}_{9}\left(x\right)=10.58-0.674\times {x}_{1}\times {x}_{2}-0.672\times {x}_{2}\le 9.9$$

(116)

$${g}_{10}\left(x\right)=16.45-0.489\times {x}_{3}\times {x}_{7}-0.843\times {x}_{5}\times {x}_{6}\le 15.7$$

(117)

$$0.5\le {x}_{1},{x}_{3},{x}_{4}\le 1.5, 0.45\le {x}_{2}\le 1.35, 0.875\le {x}_{5}\le 2.625, 0.4\le {x}_{6}, {x}_{7}\le 1.2,$$

(118)