Intelligent Firefly Algorithm for Global Optimization

Abstract

Intelligent firefly algorithm (IFA) is a novel global optimization algorithm that aims to improve the performance of the firefly algorithm (FA), which was inspired by the flashing communication signals among firefly swarms. This chapter introduces the IFA modification and evaluates its performance in comparison with the original algorithm in twenty multi-dimensional benchmark problems. The results of those numerical experiments show that IFA outperformed FA in terms of reliability and effectiveness in all tested benchmark problems. In some cases, the global minimum could not have been successfully identified via the firefly algorithm, except with the proposed modification for FA.

1 Introduction

To date, a significant work has performed on global optimization and there have been significant algorithmic and computational developments including their applications to a wide variety of real-life and engineering problems. For illustration, recent advances in global optimization have been discussed by Floudas and Gounaris [1]. In particular, global optimization has and continues to play a major role in a number of engineering and science applications. As expected, finding the global optimum is more challenging than finding a local optimum and, in some real-life applications, the location of this global optimum is crucial because it corresponds to the correct and desirable solution. Basically, global optimization methods can be classified into two broad categories: deterministic and stochastic methods. The former methods can provide a guaranteed global optimum but they require certain properties of objective function and constraints such as continuity and convexity. On the other hand, stochastic global optimization methods aim at finding the global minimum of a given function reliably and effectively. These methods are attractive for solving challenging global optimization problems arising from real-life applications because they are applicable to any problem without the assumptions of continuity and differentiability of the objective function. Until now, several stochastic methods have been proposed and tested in challenging optimization problems using continuous variables and they include simulated annealing, genetic algorithms, differential evolution, particle swarm optimization, harmony search, and ant colony optimization. Recent nature-inspired optimization algorithms include cuckoo optimization [2], artificial bee colony optimization [3–5], honey bee mating optimization [6], and multi-colony bacteria foraging optimization [7]. In general, these methods may show different numerical performances and, consequently, the search for more effective and reliable stochastic global optimization methods has been an active area of research.

In particular, the Firefly algorithm (FA) [8] is a novel nature-inspired stochastic optimization method. It was inspired by the flashing behavior of fireflies. A few variants of this algorithm were recently developed, e.g.: discrete FA [9], multiobjective FA [10], Lagranian FA [11], Chaotic FA [12] and a hybrid between FA and ant colony optimization [13]. This relatively new method has gained popularity in finding the global minimum of diverse science and engineering application problems. For example, it was rigorously evaluated by Gandomi et al. [14] for solving global optimization problems related to structural optimization problems, and has been recently used to solve the flow shop scheduling problem [9], financial portfolio optimization [13], economic dispatch problems with valve loading effects [15] and phase and chemical equilibrium problems. Even FA is usually robust for continuous global optimization; the performance of available FA algorithms may fail to solve challenging application problems. Therefore, it is convenient to study new algorithm modifications and improvements to enhance its reliability and efficiency especially for large-scale problems. The aim of this chapter is to present a modification to the existing FA algorithm and to evaluate its performance in comparison with the original algorithm. The remainder of this chapter is divided as follows: Sect. 2 introduces the Firefly algorithm. Section 3 introduces the proposed modification and our new FA algorithm, namely the Intelligent Firefly Algorithm. The numerical experiments performed to evaluate the modification are presented in Sect. 4. The results of the numerical experiments are presented and discussed in Sect. 5. Section 6 summarizes the conclusions of this chapter.

2 Firefly Algorithm (FA)

As stated in this book, FA is a nature-inspired stochastic global optimization method that was developed by Yang [8]. The FA algorithm imitates the mechanism of firefly communications via luminescent flashes. In the FA algorithm, the two important issues are the variation of light intensity and the formulation of attractiveness. The brightness of a firefly is determined by the landscape of the objective function. Attractiveness is proportional to brightness and, thus, for any two flashing fireflies, the less bright one moves towards the brighter one.

In FA, the attractiveness of a firefly is determined by its brightness, which is equal to the objective function. The brightness of a firefly at a particular location x was chosen as I( x ) $=$ f( x ). The attractiveness is judged by the other fireflies. Thus, it was made to vary with the distance between firefly i and firefly j. The attractiveness was made to vary with the degree of absorption of light in the medium between the two fireflies. Thus, the attractiveness is given by

$$\begin{aligned} \beta =\beta _{min} +\left( {\beta _o -\beta _{min} } \right) e^{-\gamma r^{2}} \end{aligned}$$

(1)

The distance between any two fireflies i and j at ${x}_{i}$ and ${x}_{j}$ is the Cartesian distance:

$$\begin{aligned} r_{{ij}}= {{\varvec{x}}}_{i}-{{\varvec{x}}}_{j} = \sqrt{\sum \limits _{{k = 1}}^{d}{\left( {x_{{i,k}}-x_{{j,k}}}\right) ^{2}}} \end{aligned}$$

(2)

The movement of a firefly attracted to another more attractive (brighter) firefly j is determined by

$$\begin{aligned} {{\varvec{x}}}_i ={{\varvec{x}}}_i +\beta \left( {{{\varvec{x}}}_j -{{\varvec{x}}}_i } \right) +\alpha \epsilon _i \end{aligned}$$

(3)

The second term is due to the attraction, while $\epsilon _i$ in the third term is a vector of random numbers usually drawn from a uniform distribution in the range [$-$0.5, 0.5]. $\alpha$ is a parameter that controls the step. It can also vary with time, gradually reducing to zero, as used in many studies [8, 14, 16] which can help to improve the convergence rate of the optimization method.

3 Intelligent Firefly Algorithm (IFA)

In the original FA, the move, i.e., Eq. (3), is determined mainly by the attractiveness of the other fireflies; the attractiveness is a strong function of the inter distance between the fireflies. Thus, a firefly can be attracted to another firefly merely because it is close, which may take it away from the global minimum. The fireflies are ranked according to their brightness, i.e. according to the values of the objective function at their respective locations. However, this ranking, which is a valuable piece of information per se, is not utilized in the move equation. A firefly is pulled towards each other firefly as each of them contributes to the move by its attractiveness. This behavior may lead to a delay in the collective move towards the global minimum. The idea behind of our Intelligent Firefly Algorithm (IFA) is to make use of the ranking information such that every firefly is moved by the attractiveness of a fraction of fireflies only and not by all of them. This fraction represents a top portion of the fireflies based on their rank. Thus, a firefly is acting intelligently by basing its move on the top ranking fireflies only and not merely on attractiveness.

A simplified algorithm for the IFA technique is presented in Fig. 1. The new parameter $\phi$ is the fraction of the fireflies utilized in the determination of the move. The original firefly algorithm is retained by setting $\phi$ to 1. This parameter is used as the upper limit for the index j in the inner loop. Thus, each firefly is moved by the top $\phi$ fraction of the fireflies only.

Fig. 1

Simplified algorithm of intelligent firefly algorithm (IFA)

The strength of FA is that the location of the best firefly does not influence the direction of the search. Thus, the fireflies are not trapped in a local minimum. However, the search for the global minimum requires additional computational effort as many fireflies wander around uninteresting areas. With the intelligent firefly modifications, the right value of the parameter $\phi$ can maintain the advantage of not being trapped in a local minimum while speeding up the search for the global minimum. The right value of $\phi$ gives a balance between the ability of the algorithm not to be trapped in a local minimum and its ability to exploit the best solutions found. An iterative procedure can be used to reach a good value of $\phi$ that is suitable for the problem to be solved. This iterative procedure is demonstrated in the numerical experiments below. We will show that this straightforward modification can improve significantly the performance of FA for continuous global optimization.

4 Numerical Experiments

Twenty classical benchmark functions were used to evaluate the performance of IFA as compared to the original FA. Tables 1 and  2 show the benchmark functions used along with their names, number of variables, variable limits and the value of the global minimum. The value of the new FA parameter $\phi$ was made to vary at four different values: 0.05, 0.1, 0.25 and 0.5. The effect of this parameter was studied for the twenty benchmark problems. The parameters for the original FA were kept constant in all experiments, i.e.: we used the value of 1 for $\beta _{o}$, 0.2 for $\beta _{min}$, 1 for $\gamma$, and $\alpha$ was made to decrease with the increase in the iteration number, k, in order to reduce the randomness according to the following formula:

$$\begin{aligned} \alpha _k =\left( {1.11\times 10^{-4}} \right) ^{b/{ {iter}}_{max}}\alpha _{k-1} \end{aligned}$$

(4)

Thus, the randomness is decreased gradually as the optima are approached. This formula was adapted from Yang [16]. The value of the parameter b was taken equal to 5 except for the Himmelblau, Powell, Wood, Rastrigin, Rosenbrock, Sine envelope sine wave, and Zacharov functions. This change in value was necessary to obtain solutions closer to the global optimum for those functions.

Table 1 Benchmark functions used for testing the performance of FA and IFA

The 20 problems constitute a comprehensive testing for the reliability and effectiveness of the suggestion modification to the original FA. Eight functions have two variables only, yet some of them are very difficult to optimize. Surface plots of these functions are shown in Fig. 2.

Table 2 Decision variables and global optimum of benchmark functions used for testing the performance of FA and IFA

To complete the evaluation of the IFE in comparison with the original FA algorithm, we have employed performance profile (PP) reported by Dolan and Moré [17], who introduced PP as a tool for evaluating and comparing the performance of optimization software. In particular, PP has been proposed to represent compactly and comprehensively the data collected from a set of solvers for a specified performance metric. For instance, number of function evaluations or computing time can be considered performance metrics for solver comparison. The PP plot allows visualization of the expected performance differences among several solvers and to compare the quality of their solutions by eliminating the bias of failures obtained in a small number of problems.

To introduce PP, consider ${n}_{s}$ solvers (i.e. optimization methods) to be tested over a set of ${n}_{p}$ problems. For each problem p and solver s, the performance metric ${t}_{ps}$ must be defined. In our study, reliability of the stochastic method in accurately finding the global minimum of the objective function is considered as the principal goal, and hence the performance metric is defined as

$$\begin{aligned} t_{ps} =f_{calc} -f^{*} \end{aligned}$$

(5)

where ${f}^{*}$ is the known global optimum of the objective function and ${f}_{calc}$ is the mean value of that objective function calculated by the stochastic method over several runs. In our study, ${f}_{calc}$ is calculated from 30 runs to solve each test problem by each solver; note that each run is different because of random number seed used and the stochastic nature of the method. So, the focus is on the average performance of stochastic methods, which is desirable [18].

Fig. 2

Surface plots of the two-variable benchmark functions used in this study: a Ackley, b Beale, c Booth, d Carrom table, e Cross-leg table, f Himmelblau, g Levy 13, and h Schaffer

For the performance metric of interest, the performance ratio $r_{ps}$ is used to compare the performance on problem p by solver s with the best performance by any solver on this problem. This performance ratio is given by

$$\begin{aligned} r_{ps} =\frac{t_{ps}}{\text {min}\{t_{ps} :1\le s \le n_{s}\}}. \end{aligned}$$

(6)

The value of ${r}_{ps}$ is 1 for the solver that performs the best on a specific problem p. To obtain an overall assessment of the performance of solvers on ${n}_{p}$ problems, the following cumulative function for ${r}_{ps}$ is used:

$$\begin{aligned} \rho _{s}(\varsigma )=\frac{1}{n_{p}}size\{p:r_{ps}\le \zeta \} \end{aligned}$$

(7)

where $\rho$($\varsigma$) is the fraction of the total number of problems, for which solver s has a performance ratio ${r}_{ps}$ within a factor of $\varsigma$ of the best possible ratio. The PP of a solver is a plot of $\rho$ $_{s}$($\varsigma$) versus $\varsigma$; it is a non-decreasing, piece-wise constant function, continuous from the right at each of the breakpoints [17]. To identify the best solver, it is only necessary to compare the values of $\rho$ $_{s}$($\varsigma$) for all solvers and to select the highest one, which is the probability that a specific solver will “win” over the rest of solvers used. In our case, the PP plot compares how accurately the stochastic methods can find the global optimum value relative to one another, and so the term “win” refers to the stochastic method that provides the most accurate value of the global minimum in the benchmark problems used.

Fig. 3

Evolution of mean best values for IFA (with $\phi$ $=$ 0.05, 0.1, 0.25, 0.5) and the original FA algorithm ($\phi$ $=$ 1) for: a Ackley, b Beale, c Booth, d Carrom table, e Cross-leg table, f Himmelblau, g Levy 13 and h Schaffer functions

Fig. 4

Evolution of mean best values for IFA (with $\phi \,=\,$ 0.05, 0.1, 0.25, 0.5) and the original FA algorithm ($\phi \,=\,$ 1) for: a Helical valley, b Powell, c Wood, d Cube, e Sphere, f Egg holder, g Griewank and h Rastrigin functions

5 Results and Discussion

As stated, each of the numerical experiments was repeated 30 times with different random seeds for IFA with four different-parameter values ($\phi$ $=$ 0.05, 0.1, 0.25, 0.5) and for the original FA algorithm ($\phi$ $=$ 1). The objective function value at each iteration for each trial was recorded. The mean and the standard deviation of the function values were calculated at each iteration. The global optimum was considered to be obtained by the method if it finds a solution within a tolerance value of 10$^{-10}$. The progress of the mean values is presented in Figs. 3, 4 and  5 for each benchmark function and a brief discussion of those results follows.

The Ackley function has one minimum only. The global optimum was obtained using all methods, as shown in Fig. 3a. However, the effectiveness of IFA is better than FA for all parameter values. No difference in the impact of the parameter value was observed. The improvement in performance was also clear with the Beale function Fig. 3b for $\phi$ values of 0.05 and 0.1. Beale has one minimum only, which was obtained satisfactory by all methods. Further iterations will improve their accuracies. The most effective method was the IFA with $\phi$ $=$ 0.05. The pattern of behavior for the five methods was also observed for the Booth function; IFA was significantly more effective than the FA as shown in Fig. 3c. The best performance was with the parameter value of 0.05, 0.1 and 0.25. The first three functions are relatively easy to optimize; the global optima were easily obtained.

For the Carrom table function, FA was unable to obtain the global minimum within the used tolerance, as shown in Fig. 3d. The global minimum was obtained by IFA with parameter value of 0.05, 0.1 and 0.25. On the other hand, the cross-leg table function is a difficult one to minimize. Its value at the global minimum is $-$1. IFA performed significantly better than FA with no apparent differences in performance between the four values of $\phi$, as shown in Fig. 3e.

FA and IFA were both able to identify the global minimum of the Himmelblau function. Figure 3f shows the evolution of the mean best values. IFA performed more effectively than FA. The effect of the value of $\phi$ was minor. Both algorithms were also able to identify the minimum of the Levy 13 function Fig. 3g. However, IFA was significantly more effective when the $\phi$ parameter values of 0.1 and 0.05 were used.

Fig. 5

Evolution of mean best values for IFA (with $\phi \,=\,$ 0.05, 0.1, 0.25, 0.5) and the original FA algorithm ( $\phi \,=\,$ 1) for: a Rosenbrock, b Sine envelope sine wave, c Trigonometric and d Zacharov functions

The Schaffer function is multimodal. Both FA and IFA failed to converge to the global minimum within the used tolerance. However, IFA was able to arrive at a better solution when the $\phi$ parameter values of 0.1 and 0.05 were used, as shown in Fig. 3h. IFA with $\phi \,=\,$ 2.5 and 0.5 failed to improve the solution in comparison with that obtained by FA. Schaffer function concludes the 2-variable functions. IFA, with $\phi \,=\,$ 0.05 and 0.1, performed better than FA in all of them.

The helical valley function has three variables. The evolution of the mean best values of FA, shown in Fig. 4a, showed that performance of IFA with $\phi \,=\,$ 0.05 was considerably better than the performance of FA. No improvement were observed, when $\phi \,=\,$ 0.5, 0.25 or 0.1. In addition, all methods failed to obtain the global minimum for the used tolerance. For the Powell function, which has four variables, IFA with $\phi \,=\,$ 0.05 obtained ten orders of magnitude improvement in the solution, as shown in Fig. 4b. Although the solution was not obtained within the acceptable tolerance but it was very close. IFA with high values of the $\phi$ parameter were also unable to identify the global minimum and converged to a higher solution. Those three algorithms seem to have been trapped in a local minimum.

Table 3 Values of the mean minima $({f}_{calc})$ and standard deviations ($\sigma$) obtained by the FA and IFA algorithms with different values of the $\phi$—parameter for the benchmark problems used in this study

A similar pattern was observed with the Wood function, which has four variables as well. The global minimum was not obtained within the acceptable tolerance used in this study. However, IFA with $\phi \,=\,$ 0.05 obtained a six orders of magnitude improvement in the solution, as shown in Fig. 4c. Higher values of the $\phi$-parameter failed to improve the performance of the FA algorithm. In fact, IFA with $\phi \,=\,$ 0.5 and 0.25 obtained worse results. For the cube function, which has five variables, Fig. 4d shows that IFA with $\phi \,=\,$ 0.05 improved the solution slightly. However, none of the methods was able to identify successfully the global minimum for this function. The sphere function also has five variables but it is easier to solve for its global optimum than the cube function. All methods successfully identified the global optimum with slight improvement in performance when IFA was used as shown in Fig. 4e.

The egg holder function was used with 50 variables and the stochastic methods were run up to 5,000 iterations. IFA obtained solutions lower than that of FA, regardless of the value of the parameter $\phi$, as shown in Fig. 4f. IFA with $\phi \,=\,$ 0.1 and 0.05 obtained the best result. Griewank function has 50 variables also. IFA with $\phi \,=\,$ 0.1 and 0.05 successfully identified the global minimum, whereas the other methods failed, as shown in Fig. 4g. Improvement of performance with the IFA algorithm is shown clearly with this benchmark problem.

The results of the optimization of Rastrigin function were peculiar. The global optimum was not obtained by FA or IFA and the performance was improved with the use of high values of $\phi$ (0.5 and 0.25) as shown in Fig. 4h. This is the only function for which the use of IFA with $\phi \,=\,$ 0.05 did not improve the result.

Fig. 6

Performance profiles of the FA and IFA methods for the global optimization of the benchmark problems used in this study

The global minimum of the Rosenbrock function was also not successfully identified by all methods, as shown in Fig. 5a. IFA, in general, gave better results than FA. The global minimum of the sine-envelope-sine function was also not successfully identified by all methods. This is a multimodal function and it is easy for any optimization method to be trapped in one of the local minima. IFA with $\phi \,=\,$ 0.1 gave the best solution followed by IFA with $\phi \,=\,$ 0.05. The results of IFA with higher values of $\phi$ were worse than the FA result, as shown in Fig. 5b.

The familiar improvement pattern with the use of IFA was obtained with the trigonometric function, as shown in Fig. 5c. The lowest value of $\phi$ gave the best solution but failed to find the global minimum within the acceptable tolerance used in this study. This was not the case with the Zacharov function (Fig. 5d) since IFA with $\phi \,=\,$ 0.05 successfully identified the global minimum within the required tolerance with twelve orders of magnitude improvement in the solution. All other methods were trapped in some local minima and no improvement in performance was obtained with IFA with other values of the parameter $\phi$.

Table 3 shows a summary of the evaluation results for the twenty benchmark problems. IFA was able to provide better solutions to all challenging problems. For example, IFA with $\phi \,=\,$ 0.05, found the global optimum of the helical valley function within 10$^{-4}$ tolerance, Powell and Trigonometric functions within 10$^{-5}$, and Zacharov function, while the FA obtained a solution several orders of magnitude higher as shown in Table 3. The performance profiles, shown in Fig. 6, summarize the results of the IFA evaluation with the four different values of the $\phi$-parameter in comparison with FA. Figure 6 clearly shows IFA with $\phi$ = 0.05 was the best performing algorithm in 18 out of the 20 cases considered. FA has not outperformed IFA in any of the benchmark problems tested. Further, PP indicates that $\phi$ is an important parameter to improve the performance of IFA. The value of $\phi$ = 0.05 appears to be the best value among those tested for reliability as indicated by the cumulative highest performance ratio in Fig. 6.

6 Conclusions

In this chapter, we propose a modification to FA through limiting the number of fireflies affecting the moves to a fraction of top performing fireflies. This modification was evaluated by attempting to find the global optimum of twenty benchmark functions. The newly developed IFA led to improved reliability and effectiveness of the algorithm in all tested benchmark problems. In some cases, the global minimum could not have been obtained via the firefly algorithm, except with this modification. IFA was found to perform best, when the new parameter $\phi$ was set to 0.05, which was the lowest value evaluated.