Convergence rates of the efficient global optimization algorithm for improving the design of analog circuits

Abstract

Optimal sizing of analog circuits is a hard and time-consuming challenge. Nowadays, analog designers are more than ever interested in developing solutions for automating such a task. In order to overcome well-known drawbacks of the conventional equation-based and simulation-based sizing techniques, analog designers are being attracted by the so-called metamodeling techniques and recently have used them for establishing accurate models of circuits’ performances. Metamodels have been associated to optimization routines to maximize circuits’ performances. In this work we deal with the newly proposed efficient global optimization (EGO) algorithm that intrinsically offers both the metamodel generation and the optimization routine. Furthermore, it performs the requested task by using a relatively very small number of performance evaluations. Firstly, we focus on the convergence rates of the EGO technique via twenty benchmark test problems. Then, we use EGO for the optimal design of a couple of analog CMOS circuits. Comparison between EGO performances and those obtained using two surrogate-assisted metaheuristics is provided to show potentialities of the proposed approach. Finally, The case of muti-objective problems is also considered. The multi-objective efficient global optimization algorithm is used for generating Pareto fronts of conflicting perormances of two analog circuits. Obtained results are compared to those of the conventional in-loop optimization technique.

1 Introduction

Analog circuit sizing and optimization problems are tedious and time consuming. Conventional approaches use the equation-based and/or the simulation-based techniques [1,2,3,4,5,6]. Despite the fact that the last overcomes limitations of the former (i.e. the use of approximated and error-prone models) due to the use of SPICE-like numerical simulators for the evaluation of performances and intrinsic constraints, the evaluation time rapidly becomes prohibitif for (not only) complex circuits. It is worth mentioning that the use of ‘rapid’ metaheuristics has considerably contributed to reducing computing time of the optimization routines and have allowed solving complex and hard problems. The available literature offers a plethora of works dealing with this problematic, see for instance [7,8,9]. However, evaluations remain expensive and very much time consuming the optimization routines, as stressed above.

Few years ago metamodeling techniques, also known as surrogate modeling, have been proposed in the specialized literature. Among these techniques we can mention the Kriging model [10, 11], the Radial Basis Functions [12, 13] and the Polynomial Regression model [14]. Such surrogate models allow establishing precise models of the handled performances, which evaluation time is very short. These techniques have successfully been used in many engineering domains [15,16,17,18,19,20,21,22], in particular in analog circuit design, see for instance [23,24,25,26,27]. The approach, called surrogate based optimization, consists of constructing a model of the considered performance by fitting a function through an initial design sampling and then using that model within the optimization routine. This model is thus used to predict values of future search samples. Subsequently, high performance regions of the design space can be identified more rapidly [25,26,27,28,29].

The efficient global optimization technique (EGO) was proposed in the late 90s [30]. EGO technique fits a surrogate model to an initial design of samples by evaluating the considered performance at few additional design points (generally, the Kriging modeling technique is used), thus decreasing the predicted error. The differential evolution algorithm is used to maximize the (expected) improvement function, which is of the form: E[I(x)] = E[max(f_min − Y, 0)]. Where f_min is the best current function value evaluated at some point x, and Y is a uniformly distributed random variable. The expected improvement (E[I(x)]) criterion aims to evaluate and update the so far constructed Kriging model [30,31,32,33,34,35,36].

The mono-objective EGO algorithm has shown its effectiveness in various domains of engineering, such as automotive problem [31] and electronics for the optimal design of analog circuits [34,35,36].

Currently, many studies are being proposed [37,38,39,40,41] to modify the mono-objective EGO to deal with multi-objective problems.

This multi-objective optimization approach consists of optimizing EIs of all objectives, and consider the obtained Pareto solutions as promising candidates. In fact, the multi-objective EI criteria have the same routine as the mono-objective EGO algorithm. Nonetheless, when the number of objectives is higher than two, the formulas of these multi-objective EI criteria are hard to provide, which makes the computations of these multi-objective EI criteria typically time-consuming. In order to solve this problem, a new approach is proposed that is based on the concept of the expected improvement matrix (EIM) [41], which is simplier to evaluate regardless of the number of objectives and the one of non-dominated points.

In this paper, firstly, we are interested in evaluating potentialities of the EGO technique in the aim to apply it to optimally design analog circuits. As it will be shown below, this will allow taking benefits from advantages of both conventional sizing/optimization techniques: precision of the model and rapidity of the evaluation.

For the sake of comparison, two Kriging-based metaheuristic sizing approaches are considered: the first uses an evolutionary technique, namely the genetic algorithm, and it will be denoted ‘GA-Kriging’, whereas the second uses a swarm intelligence approach, namely, the particle swarm optimization technique, and it is denoted as ‘PSO-Kriging’.

Secondly, we deal with adapting the conventional EGO technique to handle multi-objective problems using the EIM approach, and we show via comparisons with results obtained using an PSO-based inloop technique, showcased via two analog circuits, that the multi-objective EGO (MOEGO) offers a considerable reduction in computing time.

The rest of the paper is structured as follows. In Sect. 2, we present an overview of the EGO technique and give details about the Kriging modeling approach, the expected improvement criterion, and the differential evolution routine. Then, in Sect. 3 we deal with the surrogate-assisted metaheuristic approaches developed for comparison purposes, as explained above. In Sect. 4, convergence rates and comparisons between EGO performances and those provided by the Kriging-assisted metaheuristic approaches are provided. Twenty bench mark test problems are considered. The Wilcoxon signed-rank test is used as a metric. In Sect. 5, we focus on the application of the EGO technique for the optimal sizing of two CMOS analog circuits, namely a current conveyor and a voltage follower. Ditto, comparisons with the Kriging-based metaheuristic optimization approaches, regarding convergence rates and robustness, are provided. Section 6 presents the multi-objective EGO algorithm, offers details regarding the EIM concept, and gives a comparison with results obtained using PSO for generating Pareto front for the circuits handled in Sect. 5. Finally, in Sect. 7 we conclude the work.

2 The efficient global optimization algorithm: an overview

The efficient global optimization algorithm is an optimization process assisted by a surrogate model (a Kriging model). This algorithm is based on the use of the error estimation provided by the Kriging model to sequentially enrich the design of experiments with new points. In order to improve the quality of the optimum, the process maximizes the so-called expected improvement criterion (see Sect. 2.2).

EGO algorithm encompasses two-steps [32]. In the first step, an initial Kriging model is built using an initial design sampling. In the second step, new samples are added to the initial data which serve for constructing/updating the Kriging model while optimizing the EI criterion using the Differential Evolution (DE) algorithm [42, 43]. The corresponding pseudo code can be summarized as follows:

where X is the set of samples and Y is the simulation result set of X.

In the following subsections, we present details of main subroutines of the EGO algorithm, i.e. the Kriging modeling, and the Expected improvement criterion.

2.1 The Kriging model

Surrogate modeling is gaining interest in various domains of engineering, such as aeronautics [44] and electronics [26, 27]. It is able to approximate very complex non-linear functions by a simple and an accurate surrogate model.

Kriging modeling technique was initially developed to solve problems in geostatistics [45]. Rapidly, it became very famous in analog circuit design and optimization. The Kriging model treats the target function as a realization of a Gaussian process. A Kriging model can be expressed as:

$$y(x) = \mu + \varepsilon (x)$$

(1)

where x is the sample point, µ is the mean of the Gaussian process, ε(x) is the error term which is normally distributed with mean zero and variance σ².

The correlation between two points x(i) and x(j) is defined by:

$$corr\left[ {\varepsilon \left( {x\left( i \right)} \right) \cdot \varepsilon \left( {x\left( j \right)} \right)} \right] = exp\left( { - \sum\limits_{k = 1}^{d} {\mathop \theta \nolimits_{k} } \mathop {\left| {x_{k} \left( i \right) - x_{k} \left( j \right)} \right|}\nolimits^{{{\rm P}_{k} }} } \right)$$

(2)

where d is the dimension of the design space.

Equation (2) indicates that a small distance between two points is synonym of a large correlation, and vice versa. The distance is measured by parameters P_k (which is related to the smoothness of the function in coordinate direction k) and θ_k can be interpreted as a measure of the activity of variable x_k.

The best linear unbiased predictor of y(x) [30] is:

$$\hat{y}(x) = \hat{\mu } + r^{T} R^{ - 1} (y - \hat{\mu })$$

(3)

The mean squared error of the predictor $s^{2}$ is defined by:

$$s^{2} (x) = \hat{\sigma }^{2} \cdot \left[ {1 - r^{T} R^{ - 1} r + \frac{{(1 - 1^{T} R^{ - 1} r)^{2} }}{{1^{T} R^{ - 1} 1}}} \right]$$

(4)

where $\hat{\mu }$ and $\hat{\sigma }^{2}$ are estimations of $\mathop \mu \limits^{{}}$ and σ² which are derived by maximizing the likelihood of the observed samples.

In this equation, R is a matrix with entry $R_{ij} = corr\left[ {\varepsilon (x^{(i)} ),\varepsilon (x^{(j)} )} \right]$. r is an n dimensional vector with entry $r_{i} = corr\left[ {\varepsilon (x),\varepsilon (x^{(i)} )} \right]$ and $y = \left( {y^{(1)} ,y^{(2)} , \ldots ,y^{(n)} } \right)$ is the vector of the n observed function values.

2.2 The expected improvement criterion

The EGO algorithm enriches the design of samples at each iteration with a new sample point. This sample is chosen according to the maximum expected improvement (EI) criterion proposed by Jones [30].

Since the value of an un-sampled point Y(x) can be processed as a Gaussian process with a mean value $\hat{y}$(x), and a variance s²(x), then, the improvement of this point beyond the current best-observed value y_min is a random value that can be expressed as follows:

$$I(x) = \hbox{max} \left( {y_{\hbox{min} } - Y(x),0} \right)$$

(5)

Therefore, the expected improvement is defined by:

$$E\left[ {I\left( x \right)} \right] = \left( {y_{\hbox{min} } - \hat{y}\left( x \right)} \right) \cdot\Phi \left( {\frac{{y_{\hbox{min} } - \hat{y}}}{s\left( x \right)}} \right) + s\left( x \right) \cdot \phi \left( {\frac{{y_{\hbox{min} } - \hat{y}(x)}}{s(x)}} \right)$$

(6)

where Φ is the standard normal density, φ is the distribution function and $s\left( x \right)$ is the square root of the Kriging prediction variance.

The advantage of the expected improvement function is to provide a suitable trade-off between local search and global search. In Eq. (6), the first term of the expected improvement function increases when the prediction $\hat{y}$(x) decreases which causes the local search exploitation (it will be close to the best observed point). The second term of the expected improvement function is enhanced when the variance s(x) is increasing, which leads to the global exploration.

3 The Kriging-assisted metaheuristics

The conventional in-loop optimization kernels consist of the use of expensive simulations (in terms of computing time, generally). To fix this problem, surrogate models can be used. Firstly, a design of experiment (DoE) is generated using the Latin hypercube technique (LHC) [46]. Then, the design sampling is evaluated. We use Hspice simulator for this purpose. The established database will serve as an input for the modeling routine.

The constructed model will be used inside an optimization kernel [47, 48].

For the sake of further comparison results, we consider two well-known robust metaheuristics (GA and PSO).

The Kriging-assisted metaheuristic algorithm can be summarized as depicted by the flowchart given in Fig. 1.

Fig. 1

The Kriging-assisted metaheuristic

4 EGO performances: convergence rates and comparison results

A benchmark of twenty test problems has been considered [49, 50], see Table 1. Functions’ expressions are given in Appendix. EGO, GA-Kriging and PSO-Kriging were considered for comparison. Accuracy of the obtained results was evaluated using Eq. (7). The statistical evaluation was performed using the Wilcoxon rank test metric [51] for 100 runs of the three algorithms for each test problem, see corresponding results in Table 2.

$$relative\;Error = \left| {\frac{Theoretical\;result - Simulation\;result}{Theoretical\;result}} \right| \times 100$$

(7)

Table 1 Benchmark test problems

Table 2 Results of pairwise comparisons for affording the best solution for each benchmark problem by utilizing Wilcoxon rank test (α = 0.05)

The size of the database set is equal to 10.d in the case of the EGO algorithm, where d is the dimension of the test problem. 100 initial samples were used for the two other algorithms. The Kriging models were built using the DACE toolbox [52]. The following settings were considered:

• For the DE algorithm:

  • Maximum number of iterations: 100.

  • DE-step size: 0.8.

  • Crossover probability: 0.8.

  • Strategy: DE/rand/1/bin.

• For the PSO algorithm:

  • Social parameter: 2.

  • Cognitive parameter = 2.

  • Inertia weight: (number of generations-number of iterations)/(number of generations).

• For the GA algorithm:

  • Mutation rate = 0.1

  • Crossover rate = 0.5

We examined the relative success for 100 runs of the EGO algorithm in solving the different benchmark problems. The Wilcoxon Signed-Rank Test was used for pairwise comparisons, with the statistical significance value α = 0.05. The null hypothesis H0 is: ‘The difference between the relative error obtained by algorithm A and the relative error achieved by algorithm B is identical to the same benchmark problem’. To determine if algorithm A is statistically more successful than algorithm B, or if not, the alternative hypothesis ‘is validated’ (i.e. T + and T − as described in [53]) is checked.

Table 2 presents the pairwise comparisons of the three algorithms (PSO-Kriging vs. EGO and GA-Kriging vs. EGO) using the Wilcoxon Signed-Rank Test metric. The + and − indicate cases in which the null hypothesis was rejected. The ‘=’ indicates the cases when the two algorithms are identical and successful in solving the problems. The null hypothesis H0 was valid. In fact, the sign ‘ +’ indicates the best performance, and the ‘−’ indicates the worst one, when solving the statistical comparison problems. The three statistical significance cases (marked with ‘+’, ‘=’ or ‘−’) present in the last row of Table 2 the total counts in the (+/−/=) format.

Table 2 clearly shows that the EGO algorithm is statistically more successful and outperforms both Kriging-assisted metaheuristics.

5 Application to the optimal design of analog circuits

In this section, we are interested in the design of two analog CMOS circuits, namely, a second-generation current conveyor (CCII) [54] and a voltage follower (VF) [55]. EGO technique is used for this purpose, and a comparison with both Kriging-assisted metaheuristics is performed. Wilcoxon Signed-Rank Test metric and robustness check are provided to further highlight performances of the proposed technique.

As introduced above, an LHC database was generated for each circuit. Hspice simulator was used for evaluating the considered performances. It is to be noted that two experiments were considered when dealing with EGO: the first is performed via 5 initial samples and the second is done via 35 initial samples (The objective is to highlight efficiency of EGO when dealing with a small initial database).

5.1 Application #1: a CMOS second-generation current conveyor

In the first application, we deal with maximizing the current transfer cut-off frequency (Fci) and minimizing the parasitic X-port resistance (R_X) of a second-generation CMOS current conveyor. Figure 2 shows the considered CCII. The circuit’s variables are the channel widths WN, WP of the NMOS and the PMOS transistors, respectively. All transistors are constrained to operate in the saturation mode. The AMS 0.35 µm technology is used.

Fig. 2

A CMOS second-generation current conveyor

Tables 3 and 4 present optimal values of the CCII variables obtained using the three sizing techniques.

Table 3 Rx results

Table 4 Fci-results

Figures 3 and 4 show reached performances, respectively for Rx and Fci using optimal parameters’ values obtained using the proposed technique.

Fig. 3

CCII X-port parasitic resistance

Fig. 4

CCII high current cutoff frequency

5.2 Application #2: a CMOS voltage follower

Figure 5 depicts the considered CMOS voltage follower (VF) [55]. Two performances were considered, namely maximization of the voltage transfer high cutoff frequency (Fcv) and minimization of the voltage offset (V_offset).

Fig. 5

A CMOS voltage follower

Tables 5 and 6 show obtained results.

Table 5 V_offset results

Table 6 Fcv results

Figures 6 and 7 show Hspice simulation results using optimal parameters’ values obtained thanks to the proposed approach.

Fig. 6

VF voltage offset

Fig. 7

VF high voltage cutoff frequency

Below, we present a pairwise statistical comparison results for 100 runs of the three sizing techniques. Wilcoxon Signed-Rank Test metric is used for this purpose, as introduced above. Tables 7 and 8 present results for the CCII and the VF, respectively.

Table 7 Results of pairwise comparisons for the three algorithms of the CCII circuit (α = 0.05)

Table 8 Results of pairwise comparisons for the three algorithms of the VF circuit (α = 0.05)

The ‘+’ indicates cases where the EGO algorithm performance is statistically better, and the null hypothesis was rejected.

From Tables 7 and 8, it is clear that the EGO algorithm is statistically more successful than the other algorithms. The relative error of the EGO algorithm is smaller than the one of both the GA and the PSO based algorithms.

For further highlighting efficiency of EGO, we performed a 100-run robustness test and proceeded to the same comparison while considering a 5-point database for EGO and a 35-sample point for both GA-Kriging and PSO-Kriging algorithms. Tables 9 and 10 present the obtained results. Figures 8 and 9 present the boxplots corresponding to the robustness tests of the relative errors for each algorithm.

Table 9 Results of pairwise comparisons for different databases of the CCII circuit using the Wilcoxon Signed-Rank Test (α = 0.05)

Table 10 Results of pairwise comparisons for different databases of the VF circuit using the Wilcoxon Signed-Rank Test (α = 0.05)

Fig. 8

CCII: Boxplots of the 100-runs for the three algorithms

Fig. 9

VF: Boxplots of the 100-execution results for the three algorithms

6 The MOEGO algorithm

6.1 Multi-objective optimization problem: an overview

A multi-objective problem comprises several (non-commensurable/conflicting) objective functions to be optimized concurrently [9]. It can be expressed as follows.

$$\begin{aligned} Minimize\;\vec{f}(\vec{x}) & = \left[ {f_{1} (\vec{x}),f_{2} (\vec{x}), \ldots ,f_{m} (\vec{x})} \right] \\ with\;\vec{x} & = \left[ {x_{1} ,x_{2} , \ldots ,x_{\text{n}} } \right] \in {\mathbb{R}}^{\text{n}} \\ \end{aligned}$$

(8)

where $\vec{x}$ is the decision vector and f_i is the ith objective function. In addition, the general problem can be associated to a number of inequality and equality constraints.

A dominance routine is applied for solving such a multi-objective problem [1, 5]. Solutions that are non-dominated within the entire search space are named Pareto optimal ones. They constitute the so-called Pareto front. This process will be integrated within the conventional EGO technique to propose a new multi-objective efficient global optimization approach (MOEGO).

6.2 The proposed MOEGO

As a first attempt to deal with multi-objective problems, EI criteria was used [37,38,39,40]. However, it has been shown that the multi-objective EI criteria are very expensive to compute when the number of objectives is higher than two, which is not practical to use in complex problems [41]. As a mean of fact, the use of the expected improvement matrix (EIM) within MOEGO has been proposed to improve its performances [41].

The proposed EIM criteria are remarkably fast to compute since their computation scales linearly with the number of objectives and have better theoretical properties when compared to the previously proposed linear time EI criteria. This criterion has been incorporated within the EGO algorithm. Algorithm 2 gives the corresponding pseudo code. Details regarding EIM are given in the following subsection.

6.3 The concept of EIM

For multi-objective optimization, the current best solution is, in fact, a two-dimensional matrix [41]:

$$\left[ {\begin{array}{*{20}c} {f_{1}^{1} } & \cdots & {f_{m}^{1} } \\ \vdots & \ddots & \vdots \\ {f_{1}^{k} } & \cdots & {f_{m}^{k} } \\ \end{array} } \right]$$

(9)

f_min is the current best solution in mono-objective optimization. The number of points in the current best solution increases from one to k; the dimension of each point changes from one to m.

Inspired by this, the function EI(x) in mono-objective optimization can also be expanded into a matrix for multi-objective optimization, specifically, the expected improvement matrix (EIM) [41] which can be defined as follows.

$$\left[ {\begin{array}{*{20}c} {EI_{1}^{1} \left( x \right)} & \cdots & {EI_{m}^{1} \left( x \right)} \\ \vdots & \ddots & \vdots \\ {EI_{1}^{k} \left( x \right)} & \cdots & {EI_{m}^{k} \left( x \right)} \\ \end{array} } \right]$$

(10)

and

$$EI_{i}^{j} \left( x \right) = \left( {f_{i}^{j} - \hat{y}_{i} \left( x \right)} \right) \cdot\Phi \left( {\frac{{f_{i}^{j} - \hat{y}_{i} (x)}}{{s_{i} \left( x \right)}}} \right) + s_{i} \left( x \right) \cdot \phi \left( {\frac{{f_{i}^{j} - \hat{y}_{i} \left( x \right)}}{{s_{i} (x)}}} \right)$$

(11)

where i = 1;2; …; m and j = 1;2; …;k. The element $EI_{i}^{j} \left( x \right)$ in EIM represents the expected improvement of the considered point x beyond the jth non-dominated front point in ith objective.

Actually, replacing the term $f_{i}^{j} - y_{i} \left( x \right)$ in the multi-objective improvement functions by the term $EI_{i}^{j} \left( x \right)$ derives the formulas of the EIM criteria.

The Euclidean distance improvement was defined by Keane [56] as the Euclidean distance between the objective vector of x to its nearest non-dominated front point:

$$I_{e} (x) = \mathop {\hbox{min} }\limits_{j = 1}^{k} \sqrt {\sum\limits_{i = 1}^{m} {(f_{i}^{j} - y_{i} (x))^{2} } }$$

(12)

The Euclidean distance-based EIM criterion can be given as:

$$EIM_{e} (x) = \mathop {\hbox{min} }\limits_{j = 1}^{k} \sqrt {\sum\limits_{i = 1}^{m} {(EI_{i}^{j} (x))^{2} } }$$

(13)

6.4 Application of MOEGO to the optimal design of analog circuits

In this section, we applied the proposed EIM-based MOEGO approach to the optimal sizing of a two conflicting performances of two analog circuits, namely, a second-generation current conveyor (see Fig. 2) and a voltage follower (see Fig. 5). We presented a comparative study between the proposed MOEGO and an in-loop based multi-objective particle swarm optimization algorithm using the crowding distance (MOPSO-CD) [57, 58].

Regarding CCII, the goal is to optimize two objectives of each circuit; the current transfer cutoff frequency (fci) is maximized and the X-port parasitic resistance (Rx) is minimized for the CCII. For the VF, the voltage transfer high cutoff frequency (Fcv) is maximized and the voltage offset (V_offset) is minimized.

For MOPSO-CD, a population of 50 individuals, and 100 iterations have been considered.

Figures 10 and 11 show the Pareto fronts obtained by both approaches. Table 11 presents the execution time for both algorithms, i.e. MOEGO algorithm and MOPSO algorithm.

Fig. 10

Pareto fronts of CCII performances

Fig. 11

Pareto fronts of the VF performances

Table 11 Execution time for both algorithms

From this table, we can see that execution time of the MOPSO-CD based inloop approach is considerably higher then EIM-MOEGO one. The proposed algorithm reduces the execution time from about 1 h to only few seconds for generating the Pareto front, which confirms the efficiency of the proposed EGO algorithm.

7 Conclusion

In this paper we proposed the use of the EGO technique for the optimal design of analog circuits. The proposed approach allows combining benefits of both conventional sizing/optimizing techniques, namely, precision of the models and rapidity of their evaluation. Two CMOS circuits were considered, namely a second generation current conveyor and a voltage follower. Two performances for each circuit were handled: the parasitic X-port input resistance and the high current high cut-off frequency for the current conveyor, and the voltage offset and the high voltage cut-off frequency for the voltage follower.

Firstly, EGO performances were checked via its application to a benchmark of 20 test problems. A comparison with two conventional techniques that are based on the use of a Kriging model of the performance inside a metaheuristic based optimization routine. Due to its stochastic aspect, statistical tests were performed to check the good convergence of the proposed algorithm. Then, the later was applied for maximizing performances of the analog circuits. Ditto, the Wilcoxon metric was used for evaluating robustness of the considered algorithms. Further, since EGO offers the important advantage to be able to construct accurate models using a relatively small initial database, different tests were performed, and we showed that the proposed approach is able to correctly converge to the global optimum using a reduced number of initial starting points. Robustness tests and statistical metric results show that EGO clearly outperforms the conventional used techniques.

The case of multi-objective problems has also been considered. We proposed the use of the so-called expected improvement matrix within a muti-objective EGO routine to generate Pareto fronts linking non-commensurable and conflicting performances of the considered analog circuits. A comparision with results obtained MOPSO was provided.

For both cases, i.e. mono- and multi-objective EGO, we showed that the proposed approach is very suitable to be integrated within a CAD tool. Indeed, the statistical test metric has highlighted that EGO outperforms the classical metamodel-assisted metaheuristics. Further, a very considerable reduction of computing time is ensured (fom 1 h to a couple of tens of seconds for generating a Pareto front).

Appendix: The test functions benchmark

Problem		Equation	Optimum value
F1	Six-hump camel-back	$f\left( x \right) = \left( {4 - 2.1x_{1}^{2} + \frac{{x_{1}^{4} }}{3}} \right)x_{1}^{2} + x_{1} x_{2} + \left( { - 4 + 4x_{2}^{2} } \right)x_{2}^{2}$ $- 3 \le x_{1} \le 3$ $- 2 \le x_{2} \le 2$	− 1.0316
F2	Goldstein-Price	$f\left( x \right) = \left[ {1 + \left( {x_{1} + x_{2} + 1} \right)^{2} \left( {19 - 14x_{1} + 3x_{1}^{2} - 14x_{2} + 6x_{1} x_{2} + 3x_{2}^{2} } \right)} \right]*\left[ {30 + \left( {2x_{1} - 3x_{2} } \right)^{2} \left( {18 - 32x_{1} + 12x_{1}^{2} + 48x_{2} - 36x_{1} x_{2} + 27x_{2}^{2} } \right)} \right]$ $- 2 \le x_{1} , x_{2} \le 2$	3
F3	Branin	$f\left( x \right) = a\left( {x_{2} - bx_{1}^{2} + cx_{1} - r} \right)^{2} + s\left( {1 - t} \right)\cos \left( {x_{1} } \right) + s$ ${\text{a}} = 1,\;{\text{b}} = 5.1/\left( {4\uppi2} \right),\;{\text{c}} = 5/\uppi,\;{\text{r}} = 6,\;{\text{s}} = 10\;{\text{and}}\;{\text{t}} = 1/\left( {8\uppi} \right)$ $- 5 \le {\text{x}}_{1} \le 10$ $0 \le x_{2} \le 15$	− 0.3979
F4	Harman3	$f\left( x \right) = - \mathop \sum \limits_{i = 1}^{4} \alpha \exp \left( { - \mathop \sum \limits_{j = 1}^{3} A_{ij} (x_{j} - P_{ij} )^{2} } \right)$ $\upalpha\left( {1.0,1.2,3.0,3.2} \right)^{\text{T}}$ ${\text{A}} = \left( {\begin{array}{*{20}c} {\begin{array}{*{20}c} {3.0} & {10} & {30} \\ {0.1} & {10} & {30} \\ {3.0} & {10} & {30} \\ {0.1} & {10} & {35} \\ \end{array} } \\ \end{array} } \right)$ ${\text{P}} = 10^{ - 4} \left( {\begin{array}{*{20}c} {\begin{array}{*{20}c} {3689} & {1170} & {2673} \\ {4699} & {4387} & {7470} \\ {1091} & {8732} & {5547} \\ {381} & { 5743} & { 8828} \\ \end{array} } \\ \end{array} } \right)$ $0 \le {\text{x}}_{1} ,\;{\text{x}}_{2} ,\;{\text{x}}_{3} \le 1$	− 3.86278
F5	Harman6	$f\left( x \right) = - \mathop \sum \limits_{i = 1}^{4} \alpha \;exp\left( { - \mathop \sum \limits_{j = 1}^{6} A_{ij} (x_{j} - P_{ij} )^{2} } \right)$ $\alpha \left( {1.0,1.2,3.0,3.2} \right)^{T}$ ${\text{A}} = \left( {\begin{array}{*{20}c} {\begin{array}{*{20}c} {10} & 3 & {17 } & {3.5} & {1.7} & 8 \\ {0.05} & {10} & {17} & {0.1} & 8 & {14} \\ 3 & {3.5} & {17} & {10} & {17} & 8 \\ {17} & 8 & {0.05} & {10} & {0.1} & {14} \\ \end{array} } \\ \end{array} } \right)$ ${\text{P}} = 10^{ - 4} \left( {\begin{array}{*{20}c} {\begin{array}{*{20}c} {1312} & {1696} & {5569} & {124} & {8283} & {5886} \\ {2329} & {4135} & {8307} & {3736} & {1004} & {9991} \\ {2348} & {1451} & {3522} & {2883} & {3047} & {6650} \\ {4047} & { 8828} & { 8732} & {5743} & {1091} & { 381} \\ \end{array} } \\ \end{array} } \right)$	− 3.32237
F6	Michalewicz	$f\left( x \right) = - \mathop \sum \limits_{i = 1}^{d} sin\left( {x_{i} } \right)sin^{2m} \left( {ix_{i}^{2} /\pi } \right)$ ${\text{d}} = 2;\;{\text{m}} = 100 \le {\text{x}}_{1} ,\;{\text{x}}_{2} \le\uppi$.	− 1.8013
F7	Beale	$f\left( x \right) = \left( {1.5 - x_{1} + x_{1} x_{2} } \right)^{2} + \left( {2.25 - x_{1} + x_{1} x_{2}^{2} } \right)^{2} + \left( {2.625 - x_{1} + x_{1} x_{2}^{3} } \right)^{2}$ $- 4.5 \le x_{1} ,\;x_{2} \le 4.5$	0
F8	Rosenbrock	$f\left( x \right) = - 100\left( {x_{2} - x_{1}^{2} } \right)^{2} + \left( {x_{1} - 1} \right)^{2}$ $- 5 \le x_{1} ,\;x_{2} \le 10$	0
F9	Mccormick	$f\left( x \right) = sin\left( {x_{1} + x_{2} } \right) + \left( {x_{1} - x_{2} } \right)^{2} - 1.5x_{1} + 2.5x_{2} + 1$ $- 1.5 \le x_{1} \le 4$ $- 3 \le x_{2} \le 4$	− 1.9133
F10	Eggholder	$f\left( x \right) = - \left( {x_{2} + 47} \right)sin\left( {\sqrt {\left| {x_{2} + \frac{{x_{1} }}{2} + 47} \right|} } \right) - x_{1} sin\left( {\sqrt {\left| {x_{1} - \left( {x_{2} + 47} \right)} \right|} } \right)$ $- 512 \le {\text{x}}_{1} ,\;{\text{x}}_{2} \le 512$	− 959.6407
F11	Holder table	$f\left( x \right) = - \left| {\sin \left( {x_{1} } \right)\cos \left( {x_{2} } \right)\exp \left( {\left| {1 - \frac{{\sqrt {x_{1}^{2} + x_{2}^{2} } }}{\pi }} \right|} \right)} \right|$ $- 10 \le x_{1} ,\;x_{2} \le 10$	− 19.2085
F12	Bohachevsky1	$f\left( x \right) = x_{1}^{2} + 2x_{2}^{2} - 0.3\,cos\left( {3 \pi x_{1} } \right) - 0.4\,cos\left( {4 \pi x_{2} } \right) + 0.7$ $- 100 \le {\text{x}}_{1} ,\;{\text{x}}_{2} \le 100$	0
F13	Forrester	$f\left( x \right) = \left( {6x - 2} \right)^{2} sin\left( {12x - 4} \right)$ $0 \le x \le 1$	− 6.0207
F14	Three-hump camel	$f\left( x \right) = 2x_{1}^{2} - 1.05x_{1}^{4} + \frac{{x_{1}^{6} }}{6} + x_{1} x_{2} + x_{2}^{2}$ $- 5 \le {\text{x}}_{1} ,\;{\text{x}}_{2} \le 5$	0
F15	Cross-in-Tray	$f\left( x \right) = - 0.0001\left( {\left| {s\left( {x_{1} } \right)sin\left( {x_{2} } \right)exp\left( {\left| {100 - \frac{{\sqrt {x_{1}^{2} + x_{2}^{2} } }}{ \pi }} \right|} \right)} \right| + 1} \right)^{0.1}$ $- 10 \le x_{1} ,\;x_{2} \le 10$	− 2.0626
F16	Drop-wave	$f\left( x \right) = - \frac{{1 + cos\left( {12\sqrt {x_{1}^{2} + x_{2}^{2} } } \right)}}{{0.5\left( {x_{1}^{2} + x_{2}^{2} } \right) + 2}}$ $- 5.12 \le {\text{x}}_{1} ,\;{\text{x}}_{2} \le 5.12$	− 1
F17	Levy N.13	$f\left( x \right) = sin^{2} \left( {3 \pi x_{1} } \right) + \left( {x_{1} - 1} \right)^{2} \left[ {1 + sin^{2} \left( {3\pi x_{2} } \right)} \right] + \left( {x_{2} - 1} \right)^{2} \left[ {1 + sin^{2} \left( {2\pi x_{2} } \right)} \right]$ $- 10 \le x_{1} ,\;x_{2} \le 10$	0
F18	Booth	$f\left( x \right) = \left( {x_{1} + 2x_{2} - 7} \right)^{2} + \left( {2x_{1} + x_{2} - 5} \right)^{2}$ $- 10 \le x_{1} ,\;x_{2} \le 10$	0
F19	Matyas	$f\left( x \right) = 0.26\left( {x_{1}^{2} + x_{2}^{2} } \right) - 0.48x_{1} x_{2}$ $- 10 \le x_{1} ,\;x_{2} \le 10$	0
F20	Schaffer N.2	$f\left( x \right) = 0.5 + \frac{{sin^{2} \left( {x_{1}^{2} - x_{2}^{2} } \right) - 0.5}}{{\left[ {1 + 0.001\left( {x_{1}^{2} + x_{2}^{2} } \right)} \right]^{2} }}$ $- 100 \le x_{1} ,\;x_{2} \le 100$	0