1 Introduction

Searching for the optimal state is one of the most fundamental principles in our world, and many real-life problems in science and engineering fields can be reduced to optimization problems. In most cases, due to the complex properties of those optimization problems, such as multimodality, high nonlinearity, non-differentiability and non-convexity, they cannot be solved by traditional methods. Therefore, intelligent optimization method has attracted a great interest of researchers and many feasible and effective algorithms have been presented, such as differential evolution [1], particle swarm optimization [2], joint operations algorithm [3], biogeography-based optimization [4], whale optimization algorithm [5], monkey algorithm [6], group teaching optimization algorithm [7], brain storm optimization [8] and more.

Differential evolution (DE), first proposed by Storn and Price [9], is one of the most prominent intelligent optimization algorithms. Owing to its simple structure, convenient operation, fast convergence, strong robustness and so on, it has been widely used in power systems [10], engineering design [11], electromagnetic emission [12], flow shop scheduling [13, 14] and many other real-life optimization problems [15,16,17].

The performance of DE mainly relies on two factors: the choice of the mutation strategy and the setting of three pivotal control parameters (scale factor F, crossover rate CR and population size NP). Over the past two decades, many DE variants based on these two factors are proposed with better performance in term of final accuracy, convergence rate and robustness [18]. For example, the variants of JADE [19], CoDE [20] and GPDE [21] introduced some new mutation strategies; jDE [22], SHADE [23] and SinDE [24] adopted some new parameter configuration mechanisms. In addition, some improvement frameworks are proposed to recover the vitality when the DE algorithms are failed to search for better solutions, such as adopting an event-triggered impulsive control scheme [25], using a successful-parent-selecting framework [26] and utilizing an adaptive dimension level adjustment framework [27].

No matter which method mentioned above is used, balancing the local exploitation ability and global exploration ability is one key to enhance the algorithm’s performance, and it’s also one important guideline for strategy selection and parameter setting. However, balancing the local exploitation ability and global exploration ability is a very tricky problem. To balance these two abilities well and renovate stagnated individuals during the evolutionary process, we propose a regeneration framework relied on two strategies with different functional emphasis to randomly produce a substitutive individual from an adjusted search space. The framework is adaptively triggered and based on the individual’s status in the current population. The dynamic space adjustment mechanism depends on a decreasing macroparameter and an individual state-based microparameter, and its design philosophy is also based on the principle of balancing local exploitation ability and global exploration ability. To verify the effectiveness of the proposed ARSA framework, two basic DE algorithms and six excellent DE variants with or without that framework are tested on 30 benchmark functions with 30, 50 and 100 dimensions from the IEEE CEC 2017 test platform and three real-world optimization problems. Simulation results demonstrate that the proposed framework can visibly improve the performance of each DE algorithm used for comparison.

The remainder of this paper is organized as follows. Section 2 introduces the basic DE algorithm. Section 3 states some work related to our framework. The proposed ARSA framework is described explicitly in Sect. 4. The simulation results are presented and analyzed in Sect. 5, and the conclusion is drawn in Sect. 6.

2 Basic DE algorithm

In this section, the basic DE algorithm is introduced, which contains four operations, namely initialization, mutation, crossover and selection.

2.1 Initialization

DE algorithm usually begins the searching process by seeding an initial population consists of NP individuals which can be expressed as D-dimensional vectors:

$$\begin{aligned} \overrightarrow{X}_{i}^0=\left\{ x_{1,i}^0, x_{2,i}^0,\ldots , x_{D,i}^0\right\} ,\quad i=1,2,\ldots, {\text{NP}}. \end{aligned}$$
(1)

The initial population can be produced randomly by a uniform distribution within the search space via the formula as follows:

$$\begin{aligned} x_{j,i}^{0}=x_{j,{\min}}+{\text{rand}}\,[0,1]\cdot (x_{j,{\max}}-x_{j,{\min}}), \end{aligned}$$
(2)

where vector \(\overrightarrow{X}_\mathrm{max}=\left\{ x_{1,{\max}}, x_{2,{\max}},\ldots , x_{D,{\max}}\right\} \) denotes the upper bound of search space and vector \(\overrightarrow{X}_\mathrm{min}=\left\{ x_{1,{\min}}, x_{2,{\min}}, \ldots , x_{D,{\min}}\right\} \) represents the lower bound of search space. The variable rand [0, 1] is a random number generated from range [0,1] and obeys the uniform distribution.

2.2 Mutation

Mutation is a perturbation operation with a random change on the individuals to locate better positions. Some kinds of mutation strategies are applied on the target vector \(X_{i}^g\) to generate a corresponding mutant vector \(V_{i}^g\). Five most popular strategies are presented as follows:

  1. (1)

    DE/Rand/1:

    $$\begin{aligned} \overrightarrow{V}_{i}^g=\overrightarrow{X}_{r_{1}}^g+F\cdot (\overrightarrow{X}_{r_{2}}^g-\overrightarrow{X}_{r_{3}}^g) \end{aligned}$$
    (3)
  2. (2)

    DE/Best/1:

    $$\begin{aligned} \overrightarrow{V}_{i}^g=\overrightarrow{X}_{\mathrm{best}}^g+F\cdot (\overrightarrow{X}_{r_{1}}^g-\overrightarrow{X}_{r_{2}}^g) \end{aligned}$$
    (4)
  3. (3)

    DE/Current-to-best/1:

    $$\begin{aligned} \overrightarrow{V}_{i}^g=\overrightarrow{X}_{i}^g+F\cdot (\overrightarrow{X}_{\mathrm{best}}^g-\overrightarrow{X}_{i}^g)+F\cdot (\overrightarrow{X}_{r_{1}}^g-\overrightarrow{X}_{r_{2}}^g) \end{aligned}$$
    (5)
  4. (4)

    DE/Rand/2:

    $$\begin{aligned} \overrightarrow{V}_{i}^g=\overrightarrow{X}_{r_{1}}^g+F\cdot (\overrightarrow{X}_{r_{2}}^g-\overrightarrow{X}_{r_{3}}^g)+F\cdot (\overrightarrow{X}_{r_{4}}^g-\overrightarrow{X}_{r_{5}}^g) \end{aligned}$$
    (6)
  5. (5)

    DE/Best/2:

    $$\begin{aligned} \overrightarrow{V}_{i}^g=\overrightarrow{X}_{\mathrm{best}}^g+F\cdot (\overrightarrow{X}_{r_{1}}^g-\overrightarrow{X}_{r_{2}}^g)+F\cdot (\overrightarrow{X}_{r_{3}}^g-\overrightarrow{X}_{r_{4}}^g) \end{aligned}$$
    (7)

    The parameter F is a positive real number for scaling the difference vectors, and \(r_{1}\), \(r_{2}\), \(r_{3}\), \(r_{4}\) and \(r_{5}\) are unequal integers randomly generated from the set {1, 2, ..., NP}, which are all different from the index i. \(\overrightarrow{X}_{\mathrm{best}}^g\) denotes the individual with best fitness in the current population.

2.3 Crossover

A trial vector \(\overrightarrow{U}_{i}^g=\left\{ u_{1,i}^g, u_{2,i}^g,\ldots , u_{D,i}^g\right\} \) will be generated by executing the crossover operation after mutation, which can be described as follow:

$$\begin{aligned} u_{j,i}^g= {\left\{ \begin{array}{ll} {v}_{j,i}^g &\quad \text{if}\,\mathrm{rand} \,[0,1] \leqslant \text{CR\,\,or }\,j={j}_\mathrm{rand}\\ {x}_{j,i}^g &\quad \text{otherwise}, \end{array}\right. } \end{aligned}$$
(8)

where crossover rate CR is a parameter evaluated within the interval [0,1], which can control the fraction of components copied from the mutant vector. The variable \(j_{\rm rand}\in \{1,2, \ldots , D\}\) is a random integer which ensures that at least one dimension of trial vector is inherited from the mutant vector.

2.4 Selection

The selection operation is applied to determine either target vector \(\overrightarrow{X}_{i}^g\) or trial vector \(\overrightarrow{U}_{i}^g\) can survive to the next generation based on their fitness values. For the minimization problem, the selection operation can be described as follow:

$$\begin{aligned} \overrightarrow{X}_{i}^{g+1}= {\left\{ \begin{array}{ll} \overrightarrow{U}_{i}^g &\quad \text {if }\,f(\overrightarrow{U}_{i}^g) \leqslant f(\overrightarrow{X}_{i}^g)\\ \overrightarrow{X}_{i}^g &\quad \text {otherwise}. \end{array}\right. } \end{aligned}$$
(9)

The implementation process of basic DE algorithm can be summarized as: when the initial population is produced, it continuously repeats the operations of mutation, crossover and selection in turns until it finds the optimal solution or reaches the preset termination condition.

3 Related work

In this section, we only briefly review several DE variants which have excellent performance and are related to our research. For a comprehensive survey, please refer to [18] and [28].

3.1 Adapting control parameters

Selecting suitable values of control parameters is an indispensable and problem-dependent task which usually needs a time-consuming process of trial and error [29]. To overcome this problem, many researchers have proposed various parameter adjustment schemes, which can be categorized into deterministic, adaptive and self-adaptive [30].

  1. (1)

    Deterministic control schemes modify parameters according to some predefined rules without using any feedback information from the evolutionary process. Sun et al. [21] employed a Gaussian function and one periodic function to dynamically regulate the values of F and CR for every individual in the current population. Amer et al. [24] introduced a novel DE algorithm (named SinDE) which adopted two kinds of intrinsically related sinusoidal functions to calculate the required values of F and CR during the evolutionary process. Based on the work of SinDE, Draa et al. [31] proposed a new variant of SinDE, namely OCSinDE with a compound sinusoidal formula to adjust the parameters F and CR.

  2. (2)

    Adaptive control schemes gather some feedback information from the evolutionary process and apply them to determine the values of parameters. Yu et al. [32] proposed a two-level (population-level and individual-level) adaptive parameter control scheme to adjust the values of scale factor and crossover rate. Mohamed et al. [33] introduced an enhanced fitness-adaptive DE variant based on the distribution conditions information of each individual. Tatsis et al. [34] established a grid-based parameter adaptation strategy to dynamically select the most suitable crossover operation and determine the most appropriate related parameters.

  3. (3)

    Self-adaptive schemes mean that the control parameters are integrated into the individual and optimized with the iterative process. Qin et al. [35] proposed a self-adaptive DE (SADE) algorithm which adopts a learning strategy to adaptively select the potential mutation strategy and the relevant parameter configuration. Liu et al. [36] introduced a novel DE algorithm which simultaneously used the successful experience learned from the whole population and the information of individual state to dynamically adjust the mutation strategy of each individual and its associated parameters. Sun et al. [37] adopted three self-adaptive functions with time-varying characteristics that can automatically regulate the values of F and CR during the search process to design a novel DE algorithm.

3.2 Improving mutation strategy

Mutation is one core operation of DE algorithm. During the past twenty years, numerous researchers devoted to improving mutation strategies to enhance DE’s performance and have accumulated some significant experience that can be used to design more effective mutation strategies. Zhang and Sanderson [19] proposed a significant DE variant (JADE) which introduced a novel mutation operation called DE/Current-to-pbest with optional external archive. Gao et al. [38] presented a novel JADE variant by incorporating a chaotic local search tactics into JADE to alleviate its premature convergence problem. Kumar et al. [39] proposed a scheme named modified random localization which could strategically select the individuals that were used to generate mutant vectors. Gong and Cai [40] provided the ranking-based mutation operators which applied the ranking information of every individual in the current population to determine its chance to be used in the mutation operators. Wu et al. [41] proposed a DE algorithm which applied a mode based on multi-population to achieve the synergy of multiple mutation strategies. Wang et al. [42] introduced a composite DE algorithm which combined three different strategies with distinct advantages to generate the new trial vectors. Raunak et al. [43] introduced a uniform distribution-based mutation strategy to generate new candidate solutions. Sun et al. [44] constructed two new mutation strategies by using the elite group of dynamic adjustment. Tian and Gao [45] introduced a stochastic mixed mutation technique with using two mutation strategies controlled by a cosine perturbed probability parameter. Zhan et al. [46] introduced an adaptive distributed DE algorithm based on a master–slave multi-population distributed framework.

3.3 Hybridizing with other algorithms

Making use of the respective advantages of different algorithms to form hybrid algorithms has become a hot spot in recent years. Hendtlass [47] firstly integrated DE and particle swarm optimization (PSO) to update the particles’ position, in which DE actually influenced the particle swarm only at a specified interval. Tang et al. [48] presented a new self-adaptive PSO by integrating DE and PSO aiming at achieving a tradeoff between the global and local search capabilities. Nenavath and Jatoth [49] recently mixed sine–cosine algorithm with DE for solving global optimization and object tracking problems. Myszkowski et al. [50] combined DE and greedy algorithm to handle project scheduling problem. Luo et al. [51] combined whale optimization algorithm with differential evolution. Aguitoni et al. [52] incorporated simulated annealing into DE to deal with synthesize heat exchanger problems. Huang et al. [53] introduced Lagrange interpolation argument algorithm into DE to speed up the convergence rate of DE algorithm.

3.4 Adding an improvement framework

Bringing a potent framework into DE algorithm is a promising method to enhance its performance, then more and more researchers are devoted to exploring improvement framework. Guo et al. [26] proposed an improvement framework based on a successful-parent-selecting strategy which can strategically select the parents from the archive or the current population to enhance DE algorithms. Based on the dynamic adjustment of search space, Deng et al. [27] introduced a mechanism similar to our ARSA framework proposed in this paper, but their mechanism is implemented at the dimension level, and the trigger condition depends on one parameter given in advance. Guo and Yang [54] introduced a crossover operator which can utilize the covariance matrix’s eigenvectors of individuals to make the crossover rotationally invariant. Deng et al. [55] proposed a multi-angle searching framework used in crossover operator to improve the performance of DE algorithms. Deng et al. [56] introduced a regeneration framework to effectively use the information provided by the neighborhood area of the elite individuals. Wu et al. [57] proposed a multi-population-based framework to mix several DE variants with different merits that individuals can cooperate with each other to solve optimization problems. Leon et al. [58] proposed a new memetic framework to enhance DE algorithms via combining alopex local search. Civicioglu et al. [59] introduced a weighted differential evolution by utilizing a random crossover-based swarming strategy to generate interaction pattern.

4 ARSA framework

Based on the above-mentioned literature analysis, we know that introducing an appropriate improvement framework into DE is an effective way to recover the activity of stagnant population or stagnant individuals, and then improve the DE’s performance. There are two important problems that must be solved when using improvement framework, one is evaluating the current status of population or individuals, and the other is designing an efficient recovery strategy to change the current terrible state. In our new proposed ARSA framework, we track the updating situation of each individual separately to judge their current state, and apply a dynamic space adjustment mechanism-based adaptive triggered regeneration strategy, which is controlled by two new position generating methods and some adaptive parameters, to generate a new substitute individual when the current individual is caught in stagnant state. The adopted new strategies and related control parameters will be described in detail in the following subsections.

4.1 Adaptive control parameters

Firstly, a global parameter GP is used to control the subsequent parameters and regulate the local exploitation ability and global exploration ability from the macrolevel. Its operational formula can be expressed as follow.

$$\begin{aligned} {\text {GP}}=\left( \frac{G_{\max}-g+1}{G_{\max}}\right)^2. \end{aligned}$$
(10)

The parameter g is the index of the current generation, and \(G_\mathrm{max}\) represents the maximum number of iterations allowed. Equation (10) implies that the value of parameter GP is monotonically decreased with the increase of iterations.

In addition to the macroparameter GP, a parameter IS is used to evaluate the current status of every individual and is also applied to guide the execution of exploitation work and exploration work from the microlevel. The status of each individual relies on its fitness value, the worst and best fitness values of the current population and the value of IS can be defined by the following equation,

$$\begin{aligned} {\text {IS}}=\frac{f(\overrightarrow{X}_{i}^g)-f_{\mathrm{best}}^{g}}{f_{\mathrm{worst}}^{g}-f_{\mathrm{best}}^{g}+\epsilon }, \end{aligned}$$
(11)

where \(f(\cdot )\) expresses the objective function, \(f_{\mathrm{best}}^{g}\) and \(f_{\mathrm{worst}}^{g}\) respectively represent the obtained optimal and worst fitness value in the gth generation, and \(\epsilon \) is a very small constant which is applied to avoid the situation of \(f_{\mathrm{best}}^{g}=f_{\mathrm{worst}}^{g}\) and set to be \(10^{-99}\) in our paper. Equation (11) shows that the smaller the value of parameter IS, the better the corresponding individual fitness value.

If the current individual can not get improved for several iterations, i.e. meeting the trigger condition (TC) which will be described latter, it is necessary to determine whether the current individual is updated or not and the percentage of updates in the dimensions. We take parameters RP and CR to represent the current renewal probabilities of the stagnant individual and its dimensions. The corresponding computational formulas can be defined as follows:

$$\begin{aligned}&{\text {RP}}=(1-{\text {GP}})\cdot {\text {IP}}. \end{aligned}$$
(12)
$$\begin{aligned}&{\text {CR}}=\sqrt{0.5\cdot {\text {RP}}}. \end{aligned}$$
(13)

The intermediate parameter \({\text {IP}}=\sqrt{0.5\cdot {\text {IS}}}\) is related to the individual status IS and is used to mitigate the effects of individual information, especially in case of extreme values. Obviously, the renewal probability RP of stagnant individual is determined by both the macroparameter (GP) and microparameter (IS). The basic idea is that the probability of stagnant individuals being updated in the early stage of the optimization process is relatively small, i.e. the impact of the ARSA framework is relatively small, which helps to give full play to the role of the original algorithm and makes the ARSA framework more robust. In the later stage of the optimization process, the function of ARSA framework should be given full play to assist the original algorithm to get rid of the dilemma and thus improve its performance. In addition, the better the individual is, the less likely it is to be updated, the goal is to retain as much good information as possible when changing, which is consistent with the basic principles of the optimization algorithm.

In summary, for each individual in the current population, if the trigger condition is met, we assume the current individual is in trouble. Furthermore, if the discriminant condition \((\text{rand}[0,1]<{\text {RP}})\) is satisfied at the same time, the stagnant individual is updated by a regeneration operation and its percentage of updates in the dimensions relies on the parameter CR, otherwise it will remain unchanged in the current generation. In order to prevent the two parameters from undue influence on the framework performance, and to ensure that individual changes remain in a relatively stable range, we adopt Eq. (13) to regulate parameter CR (It is also affected by parameters GP and IS, but it’s less affected than that of RP).

Before performing the regeneration operation, it needs to determine the scope of current search space. Except for the center of search space, the search radius is another important factor. We adopt a scale parameter (SP) to limit the maximum search radius allowed in the current iteration. For one given dimension of the current individual that needs to update, the jth dimension’s value of parameter R can be calculated via the following formula,

$$\begin{aligned} R_{j}={\text {SP}}_{i}\cdot (x_{j,{\max}}-x_{j,{\min}}) \end{aligned}$$
(14)

where \({\text {SP}}_{i}={\text {GP}}\cdot {\text {IP}}\) and \(x_{j,{\max}}-x_{j,{\min}}\) represents the initial search interval span of the ith dimension. From Eq. (14), it is not hard to realize that the setting of parameter SP is based on the balance between local exploitation and global exploration. To be more specific, the macroparameter GP gradually reduces the search radius which means to shift the focus of the search from global exploration to local exploitation during the evolutionary process. Moreover, the microparameter IS assigns more exploitation work around the individuals with good fitness values, while allows individuals with poor fitness values to focus on the exploration work.

4.2 Space adjustment strategies and coordination mechanism

In order to effectively balance the work of exploitation and exploration, we propose two types of strategies with different functional emphasis to determine the current searching space used for individual reinitialization. The relevant lower and upper bounds adjusted by the two strategies can be calculated by the following formulas,

$$\begin{aligned}{\text {IL}}_{j,i}^{g}= {\left\{ \begin{array}{ll} x_{j,i}^{g}-r\cdot R_{j}^{g} &\quad \text {if}~ x_{j,i}^{g}-r\cdot R_{j}^{g}\ge x_{j,{\min}} \\ x_{j,{\min}} &\quad \text {otherwise} \end{array}\right. } \end{aligned}$$
(15)
$$\begin{aligned}{\text {IU}}_{j,i}^{g}= {\left\{ \begin{array}{ll} x_{j,i}^{g}+r\cdot R_{j}^{g} &\quad \text {if}\;x_{j,i}^{g}+r\cdot R_{j}^{g}\le x_{j,{\max}} \\ x_{j,{\max}} &\quad \text {otherwise} \end{array}\right. } \end{aligned}$$
(16)
$$\begin{aligned}{\text {OL}}_{j,i}^{g}= {\left\{ \begin{array}{ll} x_{j, {\rm best}}^{g}-r\cdot R_{j}^{g} &\quad \text {if}\; x_{j,{\rm best}}^{g}-r\cdot R_{j}^{g}\ge x_{j,{\min}} \\ x_{j,{\min}} &\quad \text {otherwise} \end{array}\right. } \end{aligned}$$
(17)
$$\begin{aligned} {\text {OU}}_{j,i}^{g}= {\left\{ \begin{array}{ll} x_{j,{\rm best}}^{g}+r\cdot R_{j}^{g} &\quad \text {if}\; x_{j,{\rm best}}^{g}+r\cdot R_{j}^{g}\le x_{j,{\max}} \\ x_{j,{\max}} &\quad \text {otherwise} \end{array}\right. } \end{aligned}$$
(18)

where \(R_{j}^{g}\) expresses the jth dimension’s current value of search radius controlled by scale parameter SP which can be calculated via Eq. (14), and r is a random number within [0,1]. Formulas (15–16) introduce a renewal scheme which uses the current individual’s position as the center of the search space to determine the corresponding lower bound \(IL_{j,i}^{g}\) and upper bound \(IU_{j,i}^{g}\), and this scheme can randomly generate the new position of stagnant individual within \(\big [{\text {IL}}_{j,i}^{g},{\text {IU}}_{j,i}^{g}\big ]\) and is named as individual itself updated (IIU) strategy. Formulas (17–18) provide another scheme which takes the position of the current optimal individual as the center of the search space, hence this one is called optimal individual updated (OIU) strategy that generating a random position from \(\big [{\text {OL}}_{j,i}^{g},{\text {OU}}_{j,i}^{g}\big ]\) for the stagnant individual.

A weight parameter (WP) is used to control the cooperation relationship between IIU strategy and OIU strategy, and its computational formula can be expressed as follow.

$$\begin{aligned} {\text {WP}}=(1.0-IP)\cdot {\text {GP}}. \end{aligned}$$
(19)

Parameter WP determines the chance of selecting IIU strategy when the two renewal strategies are applied to generate a new individual. It is easy to see that the value of WP is decreasing during the optimization process, and the better the individual’s fitness value, the greater the probability that IIU strategy will be selected. For a stagnant individual, the specific selection mechanism is that if \(rand[0,1]\le {\text {WP}}\), then executing IIU renewal strategy, otherwise executing OIU renewal strategy.

Fig. 1
figure 1

The changing process of parameters RP, CR and WP for different individuals

Full size image

The three key parameters (RP, CR and WP) contained in ARSA framework that have been introduced and they are all controlled by parameters GP and IS. In order to visually reflect the influence of GP and IS on these three parameters, we use Fig. 1 to describe their changing processes with different individual states. In fact, the changing trend of these three adaptive parameters during the optimization process has a direct impact on the focus of exploration work and exploitation work.

figure a

4.3 The trigger condition

As mentioned above, if an individuals can not be updated for uninterrupted TC generations, the proposed ARSA framework will be triggered. Thus, the value of TC has an important influence on the application frequency of ARSA framework in the optimization procedure. In our work, we adjust the value of TC in an adaptive manner, and the calculation formula is expressed as follow.

$$\begin{aligned} {\text {TC}}=floor((1.0-IP)\cdot {\text {NP}}+1.0). \end{aligned}$$
(20)

The parameter NP means the population size and the function \(floor(\cdot )\) is used to return the maximum integer less than the input parameter. Figure 2 depicts the changing process of the parameter TC along with the individual status (IS) when the population size is set to 50, from which we can see the value of TC declines in a stepwise manner along with the increase of parameter IS. Therefore, individuals with better performance will be assigned a larger TC which means that it will be hard for these individuals to trigger in the regeneration framework. Conversely, individuals with poor performance will be given a smaller TC to have more chance to trigger the regeneration framework and escape from their poor situation. This setting mechanism of trigger condition is consistent with the objective reality of optimization algorithm, i.e., in general, the better the individual’s fitness value is, the harder it is to be optimized and updated. At the same time, this setting mechanism also helps to preserve the information of elite individuals, thereby balance the effectiveness of ARSA framework and the original algorithm.

What’s more, in order to evaluate the status of individuals in the process of optimization, we adopt a corresponding counter vector \(counter[i], i=1,2,\ldots ,\) NP to track the updates of each individual, and every counter[i] is set to 0 at the beginning of iterations. During the evolutionary process, if the ith individual is updated in the current generation, then reset \(counter[i]=0\), otherwise add 1 to the counter[i]. In addition, when the record variable counter[i] is increased to TC, i.e. the trigger condition is satisfied, it needs to determine whether the condition \((rand[0,1]<{\text {RP}})\) is satisfied, and if so, carry out the ARSA framework and reset the counter[i] to 0, otherwise the ARSA framework is not executed and the current value of counter[i] is calculated according to the following formula.

$$\begin{aligned} counter[i]=floor({\text {RP}}\cdot {\text {TC}}). \end{aligned}$$
(21)

This reset method of the counter implies a higher tolerance for stagnant individuals who do not implement ARSA framework in the early stage of the optimization process, but the tolerability will decrease in the later stage. In addition, the tolerance of individuals with better fitness value is greater than that of individuals with poor fitness value.

Fig. 2
figure 2

The changing process of parameter TC along with the individual status when the population size equals to 50

Full size image
Fig. 3
figure 3

Flowchart of DE algorithm with ARSA framework

Full size image

4.4 Interpretation of ARSA framework

So far, the new individual generation strategies and their related control parameters involved in ARSA framework have been described detailedly in the above subsections. To illustrate the proposed regeneration strategy more clearly, the pseudocode of ARSA framework is given in Algorithm 1. The workflow and pseudocode of basic DE algorithm with ARSA framework are presented in Fig. 3 and Algorithm 2 respectively. From the flowchart and pseudocode of ARSA framework, it is easy to draw two conclusions: one is that the ARSA framework is easily embedded into existing DE variants; the other is that ARSA framework has no parameters that need to be set in advance. Furthermore, the design philosophy of new location generation strategies for stagnant individuals and their related control parameters are based on the balance of exploration and exploitation. The basic idea is to focus on exploration work in the early stage of the optimization process or individuals with poor fitness value, and on exploitation work in the later stage or individuals with good fitness value. In addition, to improve the robustness of ARSA framework when used in different DE variants, the utilization rate of ARSA framework is relatively low that trying to play the role of the original DE algorithm in the early stage of the search process.

Table 1 Runtime complexity of algorithm (unit: s)
Full size table

4.5 Runtime complexity of ARSA-DE

The practical runtime test approach described in [60] is applied to analyse the difference in time consumption between DE algorithm with and without ARSA framework. The runtime of four DE algorithms used in comparison experiment and their ARSA version on 30D, 50D, and 100D are summarized in Table 1. The experiments were executed on a computer with AMD Radeon R7, 12 Compute Cores 4C+8G, 3.50 GHz CPU, 8.00 GB Memory and Visual C++ 6.0 on Windows 10 system. T0 is the runtime of the following test procedure.

$$\begin{aligned}&x= 0.55;\\&For\,\, i=1:1000000\\&x=x + x;\, x=x/2; x=x*x; x=sqrt(x); \\&x=\log (x);\, x=\exp (x);\, x=x/(x+2);\\&End. \end{aligned}$$

T1 is the runtime just for \(f_{18}\) with 200,000 valuations of a certain dimension. T2 is the complete runtime of the algorithm with 200,000 evaluations for \(f_{18}\) with the same dimension. For each adopted algorithm, the corresponding \(\hat{T}2\) is the mean value of five independent time consumption T2, and the difference of runtime can be reflected by \(\hat{T}2\) and \((\hat{T}2 - T1)/T0.\)

figure b

The results in Table 1 show that the runtime of ARSA-DEs are all less than the corresponding DE algorithm on all dimensions, and the higher the dimension, the greater the difference. The reason for this phenomenon is that fitness evaluation is the most time-consuming operation, and the ARSA framework essentially reduces the number of evaluations to some extent, owing to the existence of condition \((\text{rand}[0,1]<{\text {RP}})\). Moreover, the operations used in ARSA framework are all simple operations that their time consumption is very limited.

5 Simulation results and analysis

This section will briefly introduce the adopted experimental platform and the DE algorithms used for comparison, then provide the parameter system related to the experiment and the evaluation criterion used for result comparison, and finally present the final comparison results and analysis.

5.1 Test platform and compared algorithms

To validate the effectiveness of the ARSA framework, comparative experiments are conducted based on IEEE CEC 2017 [60] test suite among eight DE algorithms and their corresponding versions with ARSA framework. The test platform includes 30 benchmark functions which can be categorized into four groups with different characteristics: (1) unimodal functions: \(f_{1}\)–\(f_{3}\); (2) simple multimodal functions: \(f_{4}\)–\(f_{10}\); (3) hybrid functions: \(f_{11}\)–\(f_{20}\); (4) composition functions: \(f_{21}\)–\(f_{30}\). Moreover, every function can be tested on 30D, 50D and 100D, and more detailed description of them can refer to the literature [60]. Moreover, three real-life optimization problems, which are often used in evaluating the performance of various algorithms, are selected to enrich the comparative experiments. They are parameter optimization for polynomial fitting problem (denoted by \({rf}_1\)) [61], parameter estimation for frequency-modulated sound waves (denoted by \({rf}_2\)) [62] and spread spectrum radar poly-phase code design (denoted by \({rf}_3\)) [62]. In addition, we select two classic DE algorithms: DE/Rand/1 and DE/Best/1 and six outstanding DE algorithms: JADE [19], SHADE [23], SinDE [24], SADE [35], MGBDE [63] and JADE_sort [64] for the comparative experiments.

5.2 Parameter settings and evaluation criterion

For the sake of fairness and convenience, the population size NP of all DE algorithms are set to be the same as the dimension D when solving benchmark functions, while is reset to \(5\cdot D\) when optimizing real-life problems for the reason that the dimension of these optimization problems is small. Whether the benchmark functions or real-life problems are selected, the maximum evaluations are set to be \(10,000\cdot D\), which is also the termination condition of all algorithms. In other words, the allowed maximum generation is set to 10,000 and 2000 for benchmark functions and real-life problems, respectively. It should be pointed out that parameters setting \(F=0.7, {\text {CR}}=0.5\) are used in both DE/Rand/1 and DE/Best/1, and the other adopted parameters in the six state-of-the-art DE algorithms are in accordance with their original papers, respectively. Each used algorithm runs for 50 independent times to obtain the average (denoted by “Mean”) and standard deviation (denoted by “Std.”) of fitness error \(\big (f(\overrightarrow{X}_{\mathrm{best}})-f(\overrightarrow{X}^{*})\big )\), where \(\overrightarrow{X}^{*}\) is the global optima of optimized problem. To draw a statistical conclusion of the results, single-problem analysis by Wilcoxon’s rank-sum test is conducted on the experimental results at the 0.05 significance level to distinguish whether there is significant difference between the selected original algorithms and their corresponding ARSA versions. The markers “\(+/=/-\)” respectively represent that the ARSA variant is significantly better than, equal to or inferior to its corresponding original DE algorithm. Wilcoxon’s test based on the multiple-problem analysis is utilized at the 0.05 significance level to show the overall performance of ARSA framework when incorporating into original algorithms. Moreover, Friedman test which is a non-parametric test method is also chosen to calculate the average ranking of all the compared algorithms in our experiment, we use this method to observe the respective rankings of original algorithms and their corresponding ARSA versions.

Table 2 Optimization rate for ARSA versions
Full size table

5.3 Results and analysis

Eight pairwise comparisons of selected DE algorithms with or without ARSA framework are carried out on 30D, 50D and 100D, which are used to verify the effectiveness and stability of ARSA framework when the dimensions increasing. The comparison results of 30D, 50D and 100D are collected in Tables S1–S6 which are list in the supplementary document. The superior results are marked with boldface font for each pair of original DE algorithms and their ARSA versions. From the obtained comparison results, it is easy to see that the ARSA framework can markedly enhance the performance of every original DE algorithms. To show the comparison results more clearly, we apply a parameter named “optimization rate”, which is the proportion of the number of “\({+}\)” and the sum of “\({+}\)” and “\({-}\)”, to evaluate the validity of ARSA framework, and the corresponding results are summarized in Table 2. Parameter “optimization rate” can essentially reflect the value of adopting ARSA framework, that is, the probability ratio of getting better when the performance changes significantly. In other words, the higher the value, the safer it is to use ARSA framework. More detailed descriptions based on different dimensions will be provided in the following paragraphs.

Table 3 Results of the multiple-problem Wilcoxon test between eight compared original algorithms and their corresponding ARSA versions at the 0.05 significance level for 30D, 50D and 100D IEEE CEC 2017 functions
Full size table

From the pairwise comparison results collected in Tables S1–S2 listed in supplementary document between the original DE algorithms and their ARSA-DE versions on 30D, it is easy to see that the positive effect of ARSA framework is very noticeable when solving the optimization problems. The statistical results based on Wilcoxon’s rank-sum test show that the algorithm who gets the most “+” is ARSA-JADE_sort (28 out of 30) and the least is ARSA-SHADE (7 out of 30), the numbers of “+” for ARSA-DE/Rand/1, ARSA-DE/Best/1, ARSA-JADE, ARSA-SinDE, ARSA-MGBDE, ARSA-SADE are 25, 20, 19, 18, 17 and 11 in turn, and the numbers of “−” for ARSA-DE/Rand/1, ARSA-JADE, ARSA-MGBDE, ARSA-DE/Best/1, ARSA-SADE, ARSA-SinDE, ARSA-JADE_sort and ARSA-SHADE are 3, 3, 2, 1, 1, 1, 0 and 0 respectively. Moreover, compared to the corresponding original DE algorithms, ARSA-JADE_sort, ARSA-DE/Rand/1, ARSA-DE/Best/1, ARSA-JADE, ARSA-MGBDE, ARSA-SinDE, ARSA-SADE and ARSA-SHADE obtain better “Mean” value on 30, 27, 23, 23, 23, 22, 21 and 21 out of 30 functions. In addition, the results in Table 2 show that the minimum value of the optimization rate is as high as 86.36%.

Table 4 Average ranking of eight compared original algorithms and their corresponding ARSA versions by Friedman test for 30D, 50D and 100D IEEE CEC 2017 functions.
Full size table

Experiment results on solving the 50D problems are summarized in Tables S3–S4 listed in attachment, which show a similar performance to the results on 30D functions. To be more specific, compared to the statistical results obtained when \(D=30\), DE/Rand/1, DE/Best/1, SADE and SHADE with ARSA framework have better performance with obtaining more “+”. For JADE and MGBDE with ARSA framework, they get the same number of “+” but the performance of ARSA-JADE_sort and SinDE get little worse with obtaining less number of “+” compared with 30D problems. In view of “Mean” value, ARSA-DEs also have noticeable remarkable performance in which ARSA-JADE_sort (ranks first) obtain 30 superior results out of 30 functions, and the worst algorithms are ARSA-JADE and ARSA-SHADE which also get at least 22 superior results. The minimum value of the optimization rate is as high as 85.00%.

The comparison results of 100D summarized in Tables S5–S6 listed in attachment also indicate that ARSA framework can effectively enhance DE algorithms. The numbers of “+” versus “−” for those adopted algorithms can be listed as ARSA-DE/Rand/1 (26 vs 1), ARSA-DE/Best/1 (20 vs 1), ARSA-JADE (15 vs 3), ARSA-MGBDE (16 vs 3), ARSA-JADE_sort (22 vs 1), ARSA-SADE (12 vs 1), ARSA-SHADE (9 vs 2) and ARSA-SinDE (14 vs 0), and the corresponding optimization rates are 96.30%, 95.24%, 83.33%, 84.21%, 95.65%, 92.31%, 81.82% and 100%, respectively. Similar to 30D and 50D, ARSA-DEs also obtain much more superior results than the corresponding original DE algorithms.

In summary, the overall performance of ARSA-DEs are memorably better than that of the DE algorithms without ARSA framework at various dimensional levels, which can strongly prove that ARSA framework can effectively improve the performance of DE algorithms. Furthermore, the results of optimization rates collected in Table 2 can indicate that when DE algorithm handle a specific optimization problem, the probability of improvement by adopting ARSA framework is much greater than the probability of deterioration.

Fig. 4
figure 4

Convergence graphs for DE/Rand/1, MGBDE, JADE_sort and SinDE with or without ARSA framework on several typical 30D, 50D and 100D IEEE CEC 2017 benchmark functions

Full size image

To draw a comprehensive comparison based on statistical analysis, multiple-problem analysis by the Wilcoxon’s test is conducted at the 0.05 significance level. The statistical results are presented in Table 3, where \(R^{+}\) and \(R^{-}\) are the sum of ranks that ARSA versions are significantly better and worse than the corresponding original algorithms respectively, and the marker ‘Yes’ means that there is 95% probability to ensure that there is significant difference between ARSA-DEs and original algorithms and vice versa for ‘No’. From the results, we can see that ARSA versions obtains much larger \(R^{+}\) values than \(R^{-}\) values for all the eight comparisons when solving 30D, 50D and 100D problems, which indicates the overall performances of the original algorithms are greatly improved by embedding ARSA framework. The asymptotic p -values in different dimensions are all less than 0.05 except for ARSA-SHADE for solving 30D functions, which means there is significant difference between algorithms with the ARSA framework and their corresponding original algorithms for all the three dimensions except for ARSA-SHADE in 30D. In fact, the values of \(R^{+}\) and \(R^{-}\) for ARSA-SHADE can also show that ARSA-SHADE has a high probability of being better than SHADE.

Table 5 Comparison results between DE algorithms and their respective ARSA versions on three real-life problems
Full size table

Table 4 represents the average ranking of eight compared original algorithms and their corresponding ARSA versions by Friedman test based on the metric mean function errors for 30D, 50D and 100D benchmark functions. It should be noted that the test function \(f_2\) has been detached from IEEE CEC 2017 test suit for its unstable characteristic especially for higher dimensions according to the original reference [60]. Therefore, we treat the error value obtained by function \(f_2\) as an outlier and exclude it when performing Friedman analysis. From the results, we can draw two conclusions. On the one hand, whether it’s 30D, 50D or 100D functions, the ARSA versions all get better rankings than their corresponding original algorithms. On the other hand, taking 30D function as an example, we can see that the basic DE algorithms DE/Rand/1 and DE/best/1 show much poorer performance than other adopted state-of-the-art DE variants. However, when incorporating ARSA framework into these two basic algorithms, i.e., ARSA-DE/Rand/1 and ARSA-DE/Best/1, their average ranking is better than other original state-of-the-art DE variants except for the original SinDE. Similar conclusion can also be drawn for 50D and 100D functions. Both two conclusions can prove the effectiveness of our ARSA framework.

To analyze the influence of ARSA framework on the convergence speed of DE algorithm, we select four DE algorithms with or without ARSA framework to depict their convergence process of dealing with optimization problems, and the relevant information is shown in Fig. 4. To save space, the selected DE algorithms are DE/Rand/1, MGBDE, JADE_sort, SinDE and their ARSA versions, and the adopted test functions include 30D functions \(f_{8}\), \(f_{21}\), 50D functions \(f_{5}\), \(f_{16}\) and 100D functions \(f_{20}\), \(f_{26}\), which are with various features in different dimensions. Figure 4 demonstrates that ARSA-DEs have a similar or faster convergence speed than original DE algorithms in most cases. In addition, compared to the original DE algorithms, ARSA-DEs have a remarkable better convergence property in the later stage of the optimization process, which is in line with our original setting of ARSA framework.

In order to further analysis the usefulness of ARSA framework, we integrate the framework into original algorithms to solve three selected real-life optimization problems. The experimental results are shown in Table 5, and similarly, the better results are marked with bold face for each pair of original DE algorithms and their ARSA versions. Problem \({rf}_1\) is relatively simple and all the eight selected algorithms and their ARSA versions obtain the optimal solution. For \({rf}_2\) and \({rf}_3\), eight ARSA versions all show significantly better or equal performance than their corresponding original algorithms. From the perspective of mean error value, we can see that ARSA versions obtains smaller or equal error values than original variants except for ARSA-SinDE when solving problem \({rf}_2\). Therefore, we can draw a conclusion that proposed ARSA framework also demonstrates superior performance in optimizing some kinds of real-life problems.

6 Conclusion

To relieve the problems of premature convergence or stagnation problem faced by DE algorithms and enhance their performances, this paper proposes a dynamic space adjustment mechanism-based adaptive triggered regeneration framework which can be easily combined with many kinds of existing DE variants. When one individual is not improved for TC (one individual-dependent adaptive parameter) uninterrupted generations, the proposed ARSA framework is triggered which contains two search space dynamic adjustment-based generation strategies with different emphasis that one focuses on exploitation work and the other on exploration work. All of the related control parameters are adaptive and depend on an iteration process-based macroparameter and an individual status-based microparameter which can provide a well balance between the global exploration ability and local exploitation ability. To verify the proposed ARSA framework, the comparison experiment are conducted based on 30 test functions from IEEE CEC 2017 and three real-life problems by selecting eight DE algorithms. The simulation results indicate that the proposed framework can significantly improve the performance of the eight adopted DE algorithms with only simple modifications and no increase in time consumption.

In the future, the direction of our research work mainly consists of three aspects. Firstly, ARSA framework will be considered to combine with various DEs for solving some real-world optimization problem with more specific and complex features. Secondly, we will apply ARSA framework or similar variants to improve other intelligent algorithms, such as genetic algorithm, particle swarm optimization. Lastly, We will extend the design idea of ARSA framework to deal with multi-objective optimization problem and discrete optimization problem.