Empirical analysis of five nature-inspired algorithms on real parameter optimization problems

Abstract

In the past few years nature-inspired algorithms are seen as potential tools to solve computationally hard problems. Tremendous success of these algorithms in providing near optimal solutions has inspired the researchers to develop new algorithms. However, very limited efforts have been made to identify the best algorithms for diverse classes of problems. This work attempts to assess the efficacy of five contemporary nature-inspired algorithms i.e. bat algorithm (BA), artificial bee colony algorithm (ABC), cuckoo search algorithm (CS), firefly algorithm (FA) and flower pollination algorithm (FPA). The work evaluates the performance of these algorithms on CEC2014 30 benchmark functions which include unimodal, multimodal, hybrid and composite problems over 10, 30, 50 and 100 dimensions. Control parameters of all algorithms are self-adapted so as to obtain best results over benchmark functions. The algorithms have been evaluated along three perspectives (a) statistical significance using Wilcoxon rank sum test (b) computational time complexity (c) convergence rate of algorithms. Experimental results and analysis revealed that ABC algorithm perform best for majority of the problems on high dimension, while on small dimension, CS is the best choice. FPA attain the next best position follow by BA and FA for all kinds of functions. Self adaptation of above algorithms also revealed the best values of input parameters for various algorithms. This study may aid experts and scientists of computational intelligence to solve intricate optimization problems.

1 Introduction

Nature-inspired algorithm is an emerging field of research since last two decades. These algorithms simulate the collective behavior of natural swarms such as echolocation behavior of bats, flashing behavior of fireflies, foraging behavior of honey bees, etc to solve intricate problems of various domains. Swarm intelligence based algorithms, a subset of nature-inspired algorithms, works on multiple agents (swarms) that are self controlled and work in unison to achieve the desired outcome. These algorithms are classified based on their natural source of inspirations such as physical based, chemistry based, bio-inspired etc. (Fister et al. 2013a). Most of the real world optimization problems, applied in engineering disciplines, are NP-hard, that is, the solution for these problems does not exists in polynomial time. Additionally, these optimization problems are too hard to model mathematically. Nature-inspired algorithms provide near optimal solutions to such problems using metaheuristic techniques for optimizing complex functions.

Due to the enhanced capabilities of nature-inspired algorithms in providing potential solution to hard problems, researchers have developed many new algorithms in this field such as bat algorithm (BA), cuckoo search algorithm (CS), firefly algorithm (FA), flower pollination algorithm (FPA) etc. Various studies are performed to evaluate the performance of these algorithms.

Table 1 depicts the concise literature study of five nature-inspired algorithms. Pham and Castellani (2014) examined four nature-inspired algorithms i.e. the bees algorithm (BEA), evolutionary algorithms (EA), particle swarm optimization (PSO) and the artificial bee colony algorithm (ABC) on the basis of their evolutionary operators and learning parameters. These algorithms were tested on 25 benchmark continuous optimization problems based on their accuracy and speed for small dimensions only. Khaze et al. (2013) discussed two metaheuristic algorithms namely: ABC and FA for optimizing continuous problems. The work however compared the efficiency of just two algorithms on two and three dimension benchmark problems only. This comparison is made on the basis of their accuracy and reliability of convergence.

Table 1 Brief literature study of contemporary algorithm

Hashmi et al. (2013) presented the comparative study of three swarm intelligence based algorithms i.e. FA, CS and BA on five benchmark functions for population size 10, 20 and 40. Dimension size was kept constant i.e.10 throughout the experiment for evaluating their accuracy and convergence speed. Civicioglu and Besdok (2011) statistically analyzed the performance of four nature-inspired algorithms i.e. CS, PSO, Differential Evolution (DE) and ABC on 50 different benchmark problems over various dimensions. The efficiency of algorithms was studied on the basis of success rate and time complexity. However, the set of benchmark functions did not include various standard and hybrid (Suganthan et al. 2005) functions. Yang (2010a) assessed the performance of BA with respect to Genetic algorithm (GA) and PSO on only 10 benchmark functions for 16, 128 and 256 dimensions. Yang (2009) introduced another new algorithm, firefly algorithm (FA), based on flashing behavior of fireflies aiming to optimize multimodal functions. The performance of FA was evaluated only on 10 benchmark functions on the basis of success rate and efficiency at 16, 128 and 256 dimensions. Moreover, author evaluated their algorithm with respect to GA and PSO but not with their own previous algorithm i.e. BA. Yang and Deb (2009) projected a new metaheuristic CS, motivated by brood parasitic behavior of cuckoo birds. The efficiency of CS was compared with PSO and GA on 10 benchmark functions only. Yang proposed three nature-inspired algorithms but did not evaluate their performances with each other. Moreover, above mentioned algorithms were not evaluated on hybrid functions and over large dimensions. Singh and Singh (2014) optimized and compared three swarm intelligence based algorithm on 15 mathematical test functions on the basis of processing time and optimized fitness functions.

Additionally, few other studies were also introduced. Yang (2014) analyzed the efficiency of two algorithms i.e. FA and CS in terms of global optimization. The tuning and control parameters were studied in the work. Banati and Mehta (2012) presented a automated tool (SEVO) for evolutionary algorithms i.e. GA, MA, PSO and SFLA and tested these algorithms over sixteen benchmark functions on SEVO. Agarwal and Mehta (2014) highlighted the specific features such as input parameters and evolutionary operators of 12 nature-inspired algorithms. Author provided the tabular analysis of minimum and maximum dimensions evaluated as well as automated tools available for these algorithms. Banati and Mehta (2013) presented improved SFLA and tested it against GA, MA, PSO and SFLA for higher dimensions. Agarwal and Mehta (2015) had analyzed various nature-inspired algorithms on data clustering. An improved version of FPA algorithm on data clustering was proposed in Agarwal and Mehta (2016).

The aforementioned literature study indicates that analyzing the performance of contemporary nature-inspired algorithms such as BA, ABC, CS, FA and FPA is still a research problem. Moreover, in above studies algorithms were evaluated only on few classical benchmark functions over small dimensions only. Recent nature-inspired algorithms were not evaluated on standard set of benchmark functions in any of the studies. Hence comparing the basic algorithm on standard benchmark problems and finding the suitable algorithm along with control parameters values on specific class of benchmark functions is a research problem.

In present work, experimental analysis of five contemporary nature-inspired algorithms i.e. BA, ABC, CS, FA and FPA is done on real parameter single objective optimization functions of CEC2014 test suite (Liang et al. 2014). This analysis includes performance evaluation, statistical test, computational complexity and run length distribution of these algorithms. Studies reveal that it is impossible to construct general purpose algorithm that gives optimal solution for all types of optimization problems. Hence, it becomes necessary to know which algorithm performs best on which kind of objective function. This paper aids researchers to understand and identify the configurations of the best algorithm on specific class of fitness function, analyze computational time complexity and determine the most appropriate algorithm on small as well as high dimensions.

The rest of the paper is organized as follows: Sect. 2 reviews the five contemporary nature-inspired algorithms. Benchmark functions and parameter setting of each algorithm is discussed in Sect. 3. Result analysis of five nature-inspired algorithms is highlighted in Sect. 4. Section 5 finally concludes the paper.

2 Review of five contemporary nature-inspired algorithms

The algorithms studied in this work are population based belonging to the class of meta-heuristic techniques. The basic processes involved in finding near optimal solution by all nature-inspired algorithms are intensification and diversification of search space. Diversification explores the new search space to find global solution and intensification exploits the explored search space to obtain optimal solution. The goal of nature-inspired algorithms is to minimize n-dimensional objective function where n is the number of variables to be optimized. The work employs standard algorithms and is at high level of abstraction. Detailed explanation of algorithms is given in their respective reference. Brief review of five nature-inspired algorithms is as follows:

2.1 Artificial bee colony (ABC) algorithm

Artificial bee colony algorithm is inspired by hunting behavior of honey bees with motive of locating best food sources (Karaboga and Basturk 2008). The paper consider basic version only while modified algorithm is defined in Akay and Karaboga (2012).The honey bees belonging to different groups communicate with each other to share the information of food sources. In order to get maximum nectar amount, they mutually decide which food source is better to exploit. They follow principles of division of labor and self-organization. There are different groups of bees: first group consists of employed bees, which sucks nectar from flowers and brings it to hive. Half of the colony size contains employed bees while the other half is occupied by second group of bees: onlooker bees. Onlooker bees wait in bee hive area to communicate with other bees and determine which food source has highest amount of nectar. Third category of bees is scout bees, which goes out to explore food sources randomly when no better food source is discovered by employed bees and onlooker bees.

This algorithm has been applied on various unimodal and multi-modal optimization functions (Karaboga and Basturk 2007). In ABC algorithm, primarily, initial random solution is determined using Eq. (1).

$$\begin{aligned} X_{ij} =lb_j +rand\left( {0,1} \right) \left( {ub_j -lb_j } \right) \end{aligned}$$

(1)

where $i=1{\ldots } {FS}$ and $j=1{\ldots } D$, where FS is the number of food sources and D is dimension parameter of optimization problem. ub and lb are upper and lower bound parameters defining search space of a problem. Initial solution obtained in Eq. (1) is provided to employed bees and onlooker bees. Employed bees randomly choose food source position using Eq. (2) and then these bees determine their nectar amount using Eq. (3).

$$\begin{aligned} {\textit{POS}}_{ij} =X_{ij} +\varPhi _{ij} \left( {X_{ij} -X_{kj} } \right) \end{aligned}$$

(2)

where j is a random integer in the range [1, D] and $\hbox {k} \in (1, 2, \ldots { FS})$ is a randomly chosen index that has to be different from i. FS is the population of food source. $\varPhi _{ij}$ is a uniformly distributed real random number in the range [−1, 1]. f denotes objective function value in Eq. (3).

$$\begin{aligned} {\textit{Fitness}}=\left\{ {{\begin{array}{ll} \frac{1}{1+f_i },&{}\quad if\, f_i >0 \\ 1+abs\left( {f_i } \right) ,&{}\quad if\, f<0 \\ \end{array} }} \right. \end{aligned}$$

(3)

Employed bees communicate with onlooker bees to inform about food source. They find the new food position in neighborhood of previous position using Eq. (2) and again the fitness of their new position is determined by using Eq. (3). Thereafter, greedy selection technique is applied between new and previous nectar amounts (fitness value) and the one with highest amount is chosen. If new calculated amount is less than previous one then position couldn’t be improved. Onlooker bees find the new food source using Eq. (2) depending upon the probability obtained from Eq. (4).

$$\begin{aligned} Pr_i =\frac{{\textit{fitness}}_i }{\mathop \sum \nolimits _{i=1}^{fs}{\textit{fitness}}_i } \end{aligned}$$

(4)

Onlooker bees send employee bees to the selected food source. Bees memorize the best food source position, consisting of highest nectar amount. If throughout the cycle nectar amount doesn’t changes then that food source is rejected and corresponding employed bee becomes scout. The algorithm enters into scout bee phase if food source is rejected by certain threshold value. In ABC algorithm, Limit is a parameter defined for entering into scout bee phase. If the solution in employed and onlooker bee phase does not improve by a threshold value (defined by Limit parameter) then algorithm enters into scout bee phase where solution is determined randomly. Random solutions are found by scout using Eq. (1) and if its value is equal to colony size then corresponding food source is rejected. The exploitation process of food source (extracting nectar amount) is carried out by employed and onlooker bees while exploration process (searching food source) is carried out by scouts (Karaboga and Basturk 2008).

Parameters of algorithm are FoodNumber viz. half of population (colony) size, Limit, a trial counter variable initialized to 0.

2.2 Bat algorithm

Bat algorithm was introduced by Yang (2010a, b), Yang and He (2013). Bats have a very strong sense of hearing. They produce loud sound to detect its prey. This sound bounces back with some frequency and process is called echolocation. Echolocation detects object by reflected sound. It is used to know how far the prey is from background object. Bats can distinguish prey and obstacle through this frequency. They fly randomly with some velocity, frequency and sound (loudness) to search for food. By observing the bounced frequency of sound, bats are able to sense the distance of an obstacle or prey in nearby surroundings. Bat algorithm simulates the echolocation behavior of microbats as they are capable of generating high echolocation. Solution of minimization problem is expressed in terms of distance to its prey. The frequency and zooming parameters maintain the balance between exploration and exploitation processes. The parameters of bat algorithm are: position of bat, velocity, frequency, loudness and emitted pulse rate. Each bat i has a certain position vector $X_i^t$, velocity $V_i^t$ at $t{\mathrm{th}}$ iteration, frequency $F_{i}$ which varies between defined values. When bat fly randomly, it produces sound with loudness $A_{0}$ and pulse emission rate $r_{i}$. Frequency is tuned to explore potential solution using Eq. (5). Velocity of bats is updated using Eq. (6). Position vector represents potential solution to problem and is updated using Eq. (7). Fitness of solution is computed by automatic zooming. This factor helps in exploiting explored positions. If a new solution (new position of bat) is better than the local best solution computed previously then solution gets updated. Accordingly loudness is reduces by Eq. (8) and pulse rate is increased by Eq. (9). These factors are adjusted automatically depending upon the proximity of prey.

$$\begin{aligned} F_{i}= & {} F_{min} + (F_{max }- F_{min}{} { )\beta } \end{aligned}$$

(5)

$$\begin{aligned} V_i^t= & {} V_i^{t-1}+(X_i^t - X^{*})F_{i} i \end{aligned}$$

(6)

$$\begin{aligned} X_i^t= & {} X_i^{t-1} +V_i^t \end{aligned}$$

(7)

$$\begin{aligned} A^{t+1}= & {} \alpha A^{\mathrm{t}} \end{aligned}$$

(8)

$$\begin{aligned} r_i^{t+1}= & {} r_i^0 \left[ {1-e^{-\gamma t}} \right] \end{aligned}$$

(9)

where $\alpha$ and $\gamma$ are random values which lie in the range [0, 1]. Pseudocode of bat algorithm is as follows.

2.3 Cuckoo search (CS)

Cuckoo search algorithm Yang and Deb (2010) is based on reproduction strategy of cuckoo bird. They lay eggs in host bird’s nest and remove host bird egg to increase the hatching probability of its own egg. Cuckoo search algorithm assumes following three principles for its implementation:

• 1. At a particular generation, each cuckoo lays only one egg and places it in randomly selected nest.

• 2. The current best solution indicates high quality of eggs and is passed to next generation.

• 3. The probability of alien eggs or destroying the host nest, which are fix in number, is $p_{a} \in [0, 1]$.

For generating new solution at $t\mathrm{th}$ iteration following equation is deployed

$$\begin{aligned} X_i^{t+1} =X_i^t +\alpha \oplus L\hat{e}vy\left( \lambda \right) \end{aligned}$$

(10)

where $X_i^{t+1}$ is new solution (nest) of $i\mathrm{th}$ cuckoo, $\oplus$ entry wise multiplication of $\alpha$ step size and $L\bar{e}vy(\lambda )$ is levy flight with $\lambda$ as levy exponent taken as 1.5 (in our simulation). Probability that n nest is replaced by new nest is $p_{a}$.

Pseudo code of CS is as follows:

2.4 Firefly algorithm (FA)

Firefly algorithm was developed by Yang (2010b), Fister et al. (2013b). It is inspired by flashing behavior of fireflies and uses following three rules:

• 1. All fireflies are attracted towards each other as they are unisex creatures.

• 2. The attractiveness of firefly is directly proportional to the light intensity or brightness of another firefly in neighborhood. Attractiveness ‘$\beta$’ and distance ‘r’ are inversely proportional and is given by

  $$\begin{aligned} \beta =\beta _0 exp\left[ {-\gamma r^{2}} \right] \end{aligned}$$

  (11)

  A less bright firefly i moves towards brighter firefly j at $t{\mathrm{th}}$ generation. Hence new location (solution) of firefly is obtained by following equation

  $$\begin{aligned} x_i^{t+1} =x_i^t +\beta e^{-\gamma r_{ij}^2 }\left( {x_j^t -x_i^t } \right) +\alpha _t \in \end{aligned}$$

  (12)

  where second term is representing attractiveness in which $\beta$ is variation of attractiveness, r is distance between two files $x_{i}$ and $x_{j}$ and $\gamma$ is the light absorption coefficient. The third term represents random variables with $\alpha _{t}$ as randomization parameter determining step size and $\in$ is Gaussian or uniform distribution.

• 3. Objective function determines the brightness of firefly.

Pseudo code of firefly algorithm is shown below.

2.5 Flower pollination algorithm (FPA)

Flowers reproduce through pollination. Flower pollination takes place by transmission of pollen grains between different flowers of same species or other species. The algorithm Yang (2013) assumes that each flower contains only one pollen grain which is associated with the solution of problem. There are two types of pollination namely, biotic and abiotic. 90% of pollination is biotic form, where pollen grains are carried by pollinators such as insects, birds etc, while 10% of pollination is abotic which uses wind and water diffusion for pollination. Pollination can be self pollination, where flowers are fertilized with pollens of same plant species, or cross pollination, where flowers are fertilized with pollens of different flowers species. It is assumed that birds and bees follow Levy’s distribution to take distant step for flying or jumping. Hence flower pollination algorithm follows following rules:

• 1. Flower constancy is the probability of reproduction which is proportional to the flowers similarity.

• 2. Global pollination is carried by biotic and cross pollination procedures. Rule 1 and 2 can be combined to give following equation

  $$\begin{aligned} x_i^{t+1} = x_i^t +L (x_i^t -g_*) \end{aligned}$$

  (13)

  where $x_i^t$ is $i{\mathrm{th}}$ solution of $t{\mathrm{th}}$ generation, $g_*$ is the current best solution at present generation and L represents levy flight distribution (Yang 2013; Pavlyukevich 2007) depicting potency of pollination.

• 3. Local pollination is carried out by abiotic and self pollination. Rule 1 and 3 are combined to give following equation

  $$\begin{aligned} x_i^{t+1}= x_i^t + \in (x_j^t -x_k^t ) \end{aligned}$$

  (14)

  where $x_j^t$ and $x_k^t$ are flowers of same species and $\in$ is the uniform distribution in range [0, 1].

• 4. A switch probability p is used to control the local and global pollination and it ranges from [0, 1].

Pseudo code of FPA is given below.

3 Experimental setup

3.1 Evaluation criteria

The five nature-inspired algorithms are assessed on 30 benchmark single objective minimization functions of CEC 2014 (Liang et al. 2014) with real parameter optimization. These benchmark problems form test functions that are composed of several new features of basic function for the purpose of optimization. Summary of CEC2014 benchmark functions are given in Table 2.

Table 2 Summary of CEC2014 benchmark functions

These benchmark functions are evaluated over 10, 30, 50 and 100 dimension problems. Thirty functions are subdivided into four subclasses named as: unimodal functions, simple multimodal functions, hybrid functions and composition functions. Search range is defined from $-100$ to $+100$ for all test functions. Bias $(\hbox {F}_{\mathrm{i}}^{*})$ for all test functions ranges from 100 for function F1 with increment of 100 for every next consecutive function upto 30,000 for function F30. Maximum function evaluations (MaxFes) are treated as terminating criteria for all algorithms evaluating CEC2014 benchmark functions. As per Liang et al. (2014) its value is assume to be 10,000*D, where D is the dimension of the problem. Algorithms are executed 51 times independently that means there are in total 51 runs and in each run algorithm is executed by 10,000*D maximum function evaluations. Results are recorded in the form of minimum value and finally error is computed as below:

$$\begin{aligned} Error=F_{min} -F_i^*\end{aligned}$$

(15)

Error value less than $10^{-8}$ is taken 0. Performance is assessed on basis of best, worst, mean, median and SD of 51 error values (runs) on each function. In this work all algorithms are executed on Matlab R2013a with hardware configuration as: 64 bit Windows7 operating system, Intel Core i5 processor with 16.0 GB RAM.

3.2 Parameter regulations

In order to attain good results on simple and complex functions, it was found that small population size is more appropriate (Akay and Karaboga 2009). Hence, in this work standard population size viz. 20 is used in all algorithms of different problem sizes (Yang and Deb 2010; Alsariera et al. 2014).

Parameter values of algorithms are self-adapted so as to achieve the best results on CEC 2014 benchmark functions. The parameters are initialized with certain values at the beginning of an algorithm (shown in Table 3) and then self-adapted during the execution of algorithm. This self-adaptation or self-tuning of parameters is made either by choosing random number between certain range (for CS and FPA is shown in Table 3) or by some equations (Eqs. 8, 9 for BA and Eq. (16) for FA). Hence, each parameter tunes itself with different value on each individual function. Thus, self-adaptive parameters help an algorithm to obtain its optimal value with best parameter settings. Mean and SD of parameter values on 30 functions are shown in “Appendix” and initial values of parameters are shown in Table 3.

Table 3 Parameters of algorithms

Initial parameter values of firefly algorithm are obtained from Yang (2010b), while the value of randomized parameter $\upalpha$ is modified on basis of Eq. (16) defined in Fister et al. (2013c).

$$\begin{aligned} \alpha ^{t+1}=\left( {1-\varDelta } \right) *\alpha ^{t},\quad \hbox {where } \varDelta =1-\left( {\frac{10^{-4}}{0.9}} \right) ^{1/Max\_Gen} \end{aligned}$$

(16)

For cuckoo search algorithm, parameters are tuned (Yang and Deb 2010) efficiently. It has been observed from studies Mlakar et al. (2015) that population size and ‘$\hbox {p}_{\mathrm{a}}$’ varies from 15 to 25 and 0.15 to 0.30 respectively. Initial parameter values of bat algorithm are taken from Fister et al. (2013d), Alsariera et al. (2014) for acquiring nearest optimal solution. As number of iterations increases, values of parameters are adapted based on Eqs. 8 and 9. In flower pollination algorithm, optimal range of switch probability between local and global pollination is defined in Draa (2015). In ABC algorithm, Limit value (Karaboga and Akay 2009) is defined by product of FoodNumber and dimension D.

4 Experimental results and study

The error values of five contemporary nature-inspired algorithms i.e. ABC, FA, CS, BA and FPA on CEC2014 30 benchmark (Liang et al. 2014) functions are presented in Tables 4, 5 6 and 7. Each and every algorithm starts with same initial population. All functions use real parameters and returns single objective value. Tables 4, 5, 6 and 7 depicts the error values for unimodal, simple multimodal, hybrid and composition functions respectively. Bold values in tables represent minimum mean error value among five algorithms. Error value is computed from recorded function values using Eq. (15). The results obtained over benchmark test problems are analyzed on the basis of

• i. Wilcoxon rank sum test—performed by taking mean of error values over 51 runs.

• ii. Computational time complexity—performed by computing mean time taken by each algorithm to run F18 function (one of CEC 2014 benchmark function) over 5 independent runs.

• iii. Convergence speed via run length distribution—performed by recording the error values at 14 iterations defined by (0.01, 0.02, 0.03, 0.05, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0)*MaxFES (maximum fitness function evaluations) for each benchmark function. At each iteration point, mean of 51 run is calculated (Liang et al. 2014). Line graph is plotted between log of error values and 14 iteration points.

Subsections are organized as follows: Sect. 4.1 gives Wilcoxon rank sum test analysis on 10, 30, 50 and 100 dimensions. Section 4.2 illustrates the computational time complexity of all five algorithms on F18 function. Section 4.3 describes the run length distribution of all five algorithms on 100 dimensions for each class of fitness function in its sub section.

Table 4 Best, worst, median, mean and SD of fitness values of unimodal function for 10D, 30D, 50D and 100D

Table 5 Best, worst, median, mean and SD of fitness values of simple multimodal functions for 10D, 30D, 50D and 100D

Table 6 Best, worst, median, mean and SD of fitness values of hybrid functions for 10D, 30D, 50D and 100D

Table 7 Best, worst, median, mean and SD of fitness values of composition functions for 10D, 30D, 50D and 100D

4.1 Wilcoxon rank sum test analysis

Wilcoxon rank sum was performed to validate the statistical significance of results over the mean error values of each algorithm. For each function, rank sum ‘p’ is computed for two compared algorithms over 51 runs. Accordingly, ‘p’ value defines the probability of difference between mean values of algorithms that occur purely by chance. In this work, the difference between the errors is considered to be significant at 5% level of confidence. Mean of 30 error values is calculated for compared algorithm (mean_error_value1) versus ABC/CS (mean_error_value2). Pseudocode of computing statistical difference is given below:

Table 8 Wilcoxon rank sum test analysis: (a) ABC, (b) CS

Wilcoxon rank sum test shown in Table 8a, b summarize the experimental results of every algorithm over 10, 30, 50 and 100 dimensions. It was observed from Tables 4, 5, 6 and 7 that ABC and CS gives best mean error values for majority of the functions. Hence, Wilcoxon rank sum test is first performed against ABC (Table 8a) and then against CS algorithm (Table 8b). When the statistical evaluation is made versus ABC algorithm, it can be noticed that ABC perform better than FA for 28, 30, 29 and 29 functions on 10, 30, 50 and 100 dimensions respectively out of total 30 CEC2014 benchmark functions. Similarly, ABC also gives superior performance against BA and FPA algorithms. However, when comparison is made against CS, ABC gives better performance on 50 and 100 dimensions but worse at 10 and 30 dimensions. This fact is observed from Table 8a It shows CS is better on total 14 and 16 functions at 10 and 30 dimensions respectively. However it is worse than ABC algorithm on total 19 and 21 functions at 50 and 100 dimensions respectively. Hence this test was performed with respect to CS (as shown in Table 8b). It can be observed that for 10 and 30 dimensions CS is better than FA (28 and 29 functions), ABC (14 and 16 functions), BA (28 and 28 functions) and FPA (25 and 23 functions). However, when CS was compared with ABC on 50 and 100 (high) dimensions, ABC was found better on 19 and 21 functions respectively.

Hence, from Wilcoxon rank sum test analysis, it can be observed that both ABC and CS depict relatively competitive performance against BA, FPA and FA. However for small dimensions, CS seems to be the better choice and for large dimensions ABC may suit better. In order to further substantiate these results, time complexity of these algorithms is analyzed in the next section.

4.2 Time complexity analysis

Time complexity is the computational time taken by each algorithm to execute a particular benchmark function. As per CEC2014, computational time complexity of F18 benchmark function is computed using Eq. (17)

$$\begin{aligned} {\textit{Time}}\,{\textit{complexity}}=\widehat{T2}-T1/T0 \end{aligned}$$

(17)

where T0 is the time required for executing the test program (given below) defined in CEC2014 (Liang et al. 2014), T1 is the time required for executing F18 function, is the mean time of 5 independent runs taken by each algorithm to execute F18 function. The pseudo code of test program is shown below:

For both T1 and T2, number of iterations at each dimension is fixed to 200,000 (Liang et al. 2014). Table 9 shows computational complexity of all five algorithms viz. measured in terms of T0, T1, and at 10, 30, 50 and 100 dimensions.

Table 9 Comparison of 5 nature-inspired algorithms on the basis of run time complexity

Run time given in Table 9 is measured in seconds. It has been observed that time complexity of BA is minimum among all five algorithms. ABC takes second position, FPA third, CS fourth and FA fifth. Though BA completes its iterations in least time but it is unable to attain optimal value. ABC takes second position in terms of time complexity and attains best optimal solution as compared to other algorithms (as seen in previous section). Since computational time of CS is quite high (approximately 70% more than ABC) and also escalates with the increase in number of dimensions; it can be inferred that ABC is the most appropriate algorithm for complex optimization problems. FPA is next to ABC. These results are further established using run length distribution in the next section.

4.3 Run length distribution analysis

Performance of ABC, FA, CS, BA and FPA were compared in terms of their convergence speed. For visualizing the convergence rate of algorithms, convergence graphs are plotted in the form of line graph at 100 dimensions representing mean error value of each algorithm over 51 runs on fixed number of iterations. The x axis represents iteration number which varies from 0 to 1,000,000. The y axis represent log of mean error value. Since CEC2014 benchmark functions are divided into four sub groups, the analysis in sub sections are made on basis of these subgroups.

4.3.1 Unimodal functions

Unimodal functions are the functions with one global optimum. Results of algorithms including ABC, FA, CS, BA and FPA are compared on the basis of their best, worst, mean, median and standard deviation error values on unimodal functions as shown in Table 4. Table 10 presents the algorithms achieving minimum mean error values at lower and higher dimensions. F1, F2 and F3 are unimodal non-separable functions where F1 has very high number of quadratic conditions, F2 has polished but slender ridge and F3 is narrow sensitivity at one direction. Results depict that CS gives better outcome among all five algorithms on small dimensions for majority of the functions. Additionally, CS has high efficacy on the function which is sensitive at one direction only i.e. F3. However on higher dimension ABC gives best values for majority of functions.

Table 10 Result analysis of minimum mean error value on unimodal function

At higher dimension, further analysis of all five algorithms is made on basis of convergence graph. For an algorithm to be best, it should have overall better convergence speed. This parameter is analyzed on basis of run length distribution (RLD). RLD of unimodal functions F1, F2 and F3 are represented in Figs 1, 2 and 3 respectively.

Fig. 1

RLD of algorithms for F1

Fig. 2

RLD of algorithms for F2

Fig. 3

RLD of algorithms for F3

On F1 and F2, ABC algorithm depicts fast convergence speed as compared to other algorithms as it achieves its minima in 1.00E$+$06 and 2.00E$+$05 iterations respectively. Although on F2 function, algorithm attempts to diversify its search space but reach at same optima over the time. On F3 function, CS shows a linear decrement in error value i.e. good convergence speed among all five algorithms and reaches minimum error value in 1.00E$+$06 iterations. Though difference in minimum error value of ABC and CS is quite high but ABC reaches its global solution at 3.00E$+$05 iterations. This might be due to function’s property that is skewed in one direction. Also on F1 and F2 function CS converges at a very early stage but its minima is far from global minimum value.

FPA is the second best algorithm after ABC. At F1 function, FPA convergence speed is approximately same as that of ABC while on F2 and F3 function, FPA is the second best in terms of convergence speed and minimum error value.

It has been observed from above analysis that ABC algorithm is the best algorithm in terms of mean error value at high dimension. Next to ABC, FPA shows uniform behavior throughout all unimodal function. While at lower dimension, CS is the best algorithm. Thus, depending on requirement of application appropriate algorithm can be chosen for achieving desired result.

4.3.2 Simple multimodal functions

Multimodal functions are the functions having one or more than one global optima. Table 5 represents error values of five nature-inspired algorithms on simple multimodal functions of CEC2014 benchmark suite. Best, mean, median, worst and SD values for 51 runs are depicted. Analysis of mean values on simple multimodal functions at lower and higher dimension is mentioned in Table 11. ABC algorithm proved to be the best algorithm on majority of simple multimodal functions.

Table 11 Result analysis of minimum mean error value on simple multimodal function

It can be precisely observed that ABC algorithm is the best performer on small dimensions. While on higher dimension further analysis of algorithms is made on the basis of RLD. RLD is represented by line graphs shown in Figs. 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15 and 16. It can be observed that in most of the cases convergence speed of ABC algorithm in achieving minimum error value is faster than any other algorithm. It attains global optima at 1.00E$+$06 and 7.00E$+$05 iterations on F4 and F6 functions respectively. ABC algorithm tries to explore more search space after achieving best value at 5.00E$+$05 and 7.00E$+$05 iterations on F7 and F8 functions respectively. ABC algorithm has ability to explore more search space for obtaining global optima. On F10 function, performance of ABC algorithm improves in steps over the iterations. It attains its minima at 9.00E$+$05 iterations and settles down after that. This might be due to some significant characteristics of functions such as F8 and F10 which are shifted and separable functions with large number of local optima. The behavior of ABC algorithm may be due to the use of real random integer variable in exploiting neighbor food sources or type of objective function used. On F11, F13, F14, F15 and F16 functions, i.e. multimodal, shifted, rotated and non separable, ABC algorithm achieves minimum mean error value in early stages i.e. at 6.00E$+$05, 2.00E$+$05, 3.00E$+$04, 2.00E$+$05 and 7.00E$+$05 iterations respectively.

On F5 function, ABC performs next to BA though difference in mean error value is very small i.e. ABC lags by 1.5% only. ABC achieves its minima at 8.00E$+$05 iterations. On F9 function viz. non separable with large number local optima, though FPA achieves best mean error value but ABC converges earlier than FPA. ABC attains its minima at 6.00E$+$05 iterations while FPA at 1.00E$+$06 iterations. ABC lags FPA by approximately 15% in terms of minimum mean error value. After ABC, it is CS which achieves minimum mean error value on majority of multimodal functions.

Thus, it can be observed from above analysis that for simple multimodal functions, ABC algorithm is the best algorithm for attaining minimum error value. CS takes second position while FPA is on third position.

Fig. 4

RLD of algorithms for F4

Fig. 5

RLD of algorithms for F5

Fig. 6

RLD of algorithms for F6

Fig. 7

RLD of algorithms for F7

Fig. 8

RLD of algorithms for F8

Fig. 9

RLD of algorithms for F9

Fig. 10

RLD of algorithms for F10

Fig. 11

RLD of algorithms for F11

Fig. 12

RLD of algorithms for F12

Fig. 13

RLD of algorithms for F13

Fig. 14

RLD of algorithms for F14

Fig. 15

RLD of algorithms for F15

Fig. 16

RLD of algorithms for F16

4.3.3 Hybrid functions

In hybrid functions, variables are decomposed into several subcomponents. Each subcomponent is associated with different basic function. Generally, hybrid functions are unimodal or multimodal constituting non separable functions with specific traits. Table 6 represents best, mean, median, worst and standard deviation values of hybrid function on 51 runs. Table 12 scrutinize Table 6 and represents algorithms having minimum mean error values on lower and higher dimension on each function.

Table 12 Result analysis of minimum mean error value on hybrid function

It has been observed from Table 12 that CS is the better performer as compared to other algorithms on small dimensions while on high dimensions results need more analysis. To illustrate algorithms’ reliability on high dimension, further analysis is made through convergence graphs.

Run length distribution of all five algorithms on hybrid functions at high dimensions is defined by line graphs as shown in Figs. 17, 18, 19, 20, 21 and 22. It has been analyzed that ABC is the best performer on F18, F19 and F22 functions. ABC portrays fast convergence speed and attains minima at 9.00E$+$05 iterations on F18 and F19 functions. It shows early convergence at 7.00E$+$05 iterations on F22 function. On F17 and F21 functions viz. separable, non separable, ill conditioned and huge number of local optima, though FPA depicts best performance, CS converges very close to FPA. ABC converges at around 7.00E$+$05 iterations while FPA and CS both converge at 1.00E$+$06 iterations. On F20 function, CS algorithm performance continuously improves throughout the iterations and reaches its global minima at 1.00E$+$06 iterations while ABC attains minima at 4.00E$+$05 iterations. CS stagnates to local optima on majority of hybrid functions. Thus, it can be observed that due to different phases of bees in optimizing objective function, ABC algorithm obtain fast convergence rate on most of the functions.

Hence from above investigation and analysis it can be established that ABC is the best algorithm at higher dimensions on hybrid functions in terms convergence speed and minimum mean error value. FPA is the second best while CS attains next position after FPA.

Fig. 17

RLD of algorithms for F17

Fig. 18

RLD of algorithms for F18

Fig. 19

RLD of algorithms for F19

Fig. 20

RLD of algorithms for F20

Fig. 21

RLD of algorithms for F21

Fig. 22

RLD of algorithms for F22

4.3.4 Composition function

Composition functions (F23–F30) are composed of various basic functions. These functions consist of various components of variables possessing different basic functions. Table 13 investigates Table 7 for acquiring best algorithm at each function on the basis of mean error values obtained over 51 runs. It has been observed that for small dimensions CS gives the most promising result on majority of composition functions. While on high dimension, ABC and CS both depict good performances. Hence there is tradeoff between ABC and CS on high dimension. In order to establish the results at high dimension and obtain the most suitable algorithm for composition functions, further analysis is made on run length distribution (RLD).

Table 13 Result analysis of minimum mean error value on composition function

RLD of algorithms on composition functions are analyzed for determining the most fitted algorithm among the existing algorithms in terms of convergence speed. RLD of composition functions are represented by line graphs shown in Figs. 23, 24, 25, 26, 27, 28, 29 and 30.

On functions F24, F27 and F29, having different properties at different subcomponents, ABC depicts minimum mean error value as compared to other algorithms. However, on F27 and F29, convergence of ABC algorithm is at 7.00E$+$05 and 6.00E$+$05 iterations respectively. ABC is second best with small difference on F23, F25, F26, F28 and F30 functions. On F23 and F28 functions viz. is composed of different functions having asymmetry, multimodal and non separable properties at various local optima, CS attains best mean error value at 1.00E$+$06 iterations while ABC achieves its global optima at 9.00E$+$05 and 8.00E$+$05 iterations respectively. Moreover, in terms of mean error values ABC lags by 0.3 and 14% only on F23 and F28 functions respectively. On F25 and F26 functions, FPA shows minimum mean error value and succeeds ABC by 4.4 and 1.5% only. The performance of ABC, CS and FPA on these functions is almost same. Hence, it can be inferred that ABC portrays better performance for majority of composition functions and CS is the second best algorithm. This is due to the fact that CS uses some probability to abandon host nest and its update equations gives excellent exploration and exploitation capabilities to avoid local optima. ABC uses 3 stages i.e. employed, onlooker and scout bee phase to explore quality solutions.

Therefore, it can be substantiated from above study that ABC algorithm is the most appropriate algorithm for composition functions on high dimensions.

Fig. 23

RLD of algorithms for F23

Fig. 24

RLD of algorithms for F24

Fig. 25

RLD of algorithms for F25

Fig. 26

RLD of algorithms for F26

Fig. 27

RLD of algorithms for F27

Fig. 28

RLD of algorithms for F28

Fig. 29

RLD of algorithms for F29

Fig. 30

RLD of algorithms for F30

On the whole, analysis of contemporary nature-inspired algorithms on CEC2014 benchmark functions establishes that ABC algorithm performs best for more or less all kinds of functions. Both CS and FPA attain the second position as FPA performed better than CS on unimodal and hybrid functions while CS performed better for multimodal and composite functions at high dimension. Hence, ABC, FPA and CS all perform better than BA and FA for all kinds of functions. However, when analysis is made dimension wise, then CS is the best choice for small dimension while ABC should be chosen for high dimension problems. From time complexity analysis, it has been observed that ABC takes least time as compared to CS and FPA. FPA is second to ABC and CS takes third position. The study also presented the best value of input parameters required for various algorithms.

5 Conclusion

The work presents the comprehensive analysis of five contemporary self-adaptive nature-inspired algorithms i.e. ABC, FA, FPA, CS and BA on standard CEC2014 benchmark functions. The algorithms are compared on the basis of error values computed from fitness function values on unimodal, simple multimodal, hybrid and composition functions at 10, 30, 50 and 100 dimensions. Analysis over minimum error value attained, computational time complexity and convergence speed depicts that ABC is the most suitable algorithm for majority of the functions. ABC is followed by CS and FPA in terms of minimum error value attained. FA and BA attain second last and last position in terms of efficiency. This study also provides the best values of control parameters on each function for various algorithms.

Future work relies on numerous routes for fulfilling objective of developing best nature-inspired algorithm. The paper is helpful to design hybrid nature-inspired algorithm by combining strengths of various nature-inspired algorithms i.e. artificial bee colony algorithm, cuckoo search algorithm and flower pollination algorithm. Algorithms can also be tested on higher dimensions (200, 500–1000) to overcome the problem of curse of dimensionality. Future work could be evaluating the efficacy these algorithms on real world problems.

Appendix

See Tables 14 and 15.

Table 14 Parameter values of FA, CS, BA and FPA on 10, 30, 50 and 100 dimensions

Table 15 List of nomenclature