Keywords

1 Introduction

This research deals with the mutual intersection of the two softcomputing fields, which are the complex dynamics given by the chaotic systems driving the selection of indices in Differential Evolution (DE) algorithm and evolutionary computation techniques (ECT’s). Currently the DE [1] is known as powerful heuristic for many difficult and complex optimization problems.

A number of DE variants have been recently developed with the emphasis on adaptivity/selfadaptivity [2], ensemble approach [3] or other modern approaches [4, 5]. The importance of randomization within heuristics as a compensation of limited amount of search moves is stated in the survey paper [6]. This idea has been carried out in subsequent studies describing different techniques to modify the randomization process [7, 8] and especially in [9], where the sampling of the points is tested from modified distribution. The importance and influence of randomization operations was also deeply experimentally tested in simple control parameter adjustment jDE strategy [10]. Together with this persistent development in such mainstream research topics, the basic concept of chaos driven DE have been introduced. Recent research in chaotic approach for heuristics generally uses various chaotic maps in the place of pseudo random number generators (PRNG). The focus of this research is the direct embedding of chaotic dynamics in the form of chaos pseudo random number generator (CPRNG) for heuristic. The initial concept of embedding chaotic dynamics into the evolutionary/swarm algorithms is given in [11]. Later, the initial study [12] was focused on the simple embedding of chaotic systems for DE and Self Organizing Migration Algorithm (SOMA) [13]. Also the PSO (Particle Swarm Optimization) algorithm with elements of chaos was introduced as CPSO [14] followed by the introduction of chaos embedded PSO with inertia weigh strategy [15], further PSO strategy driven alternately by two chaotic systems [16] and finally PSO with ensemble of chaotic systems [17]. Recently the chaos driven heuristic concept has been utilized in ABC algorithm [18] and applications with DE [19].

The organization of this paper is following: Firstly, the motivation and novality for this research is proposed. The next sections are focused on the description of the concept of chaos driven DE, identification of chaotic series distribution and the experiment background. Results and conclusion follow afterwards.

2 Motivation

This research is an extension and continuation of the previous successful initial experiment with the single/multi-chaos driven DE (ChaosDE), where the positive influence of hidden complex dynamics for the heuristic performance has been experimentally shown. This research is also a follow up to previous initial experiments with time continuous chaotic systems and different sampling rates used [20].

Nevertheless, the questions remain, as to why it works, why it may be beneficial to use the correlated chaotic time series for generating pseudo random numbers driving the selection, mutation, crossover or other processes in particular heuristics.

The novality of the research is given by the experiment investigationg whether the chaos embedded heuristics concept belongs to the group of either “utilization of different PRNG with different distribution” or the unique chaos dynamics providing unique sequencing of pseudo random numbers is the key of performance improvements. The last point was also inspired by recent advances in connection of complexity and heuristic [21] together with the research focused on selection of indices in DE [22] where the indices (solutions) for mutation process were not selected randomly, but based on the complex behavior and neighborhood mechanisms.

To confirm or disprove the aforementioned hypothesis, a simple experiment was performed and presented here. Through the utilization of either chaotic systems or identical identified pseudo random number distribution, it is possible to fully keep or remove the hidden complex chaotic dynamics from the generated pseudo random data series for obtaining the pseudo random numbers for indices selection inside DE.

3 Differential Evolution

DE is a population-based optimization method that works on real-number-coded individuals [1]. DE is quite robust, fast, and effective, with global optimization ability. It does not require the objective function to be differentiable, and it works well even with noisy and time-dependent objective functions. There are essentially five inputs to the heuristic. Dim is the size of the problem, Gmax is the maximum number of generations, NP is the total number of solutions, F is the scaling factor of the solution and CR is the factor for crossover. F and CR together make the internal tuning parameters for the heuristic. Due to a limited space and the aims of this paper, the detailed description of well known canonical strategy of differential evolution algorithm basic principles is insignificant and hence omitted. Please refer to [1, 23] for the detailed description of the used DE/Rand/1/Bin strategy (both for ChaosDE and Canonical DE) as well as for the complete description of all other strategies.

4 Chaotic Systems and Identification of CPRNGs Distributions

Following two well known and frequently utilized discrete dissipative chaotic maps were used as the CPRNGS for DE: Burgers (1), and Lozi map (2).

The Burgers mapping is a discretization of a pair of coupled differential equations to illustrate the relevance of the concept of bifurcation to the study of hydrodynamics flows. The Lozi map is a simple discrete two-dimensional chaotic map. With the typical settings as in Table 1, systems exhibits typical chaotic behavior [24].

Table 1. Definition of chaotic systems used as CPRNGs
Full size table

For the comparisons of DE performance with indices selection driven either by CPRNG or identical PRNG distribution without chaotic dynamics, it was necessary to perform the CPRNGs distributions identification with 10000 samples and statistical distribution fit tests. Statistica and Wolfram Mathematica software were used for this task with following results (See also Figs. 1 and 2):

  • Burgers map based CPRNG was identified as Beta distribution (α, β) with α = 0.63 and β = 3.54.

  • Lozi map based CPRNG was identified as Beta distribution (α, β) with α = 1.05 and β = 1.57.

Fig. 1.
figure 1

Identification (blue line) of Burgers map based CPRNG (green line – smooth histogram)

Full size image
Fig. 2.
figure 2

Identification (blue line) of Lozi map based CPRNG (green line – smooth histogram)

Full size image

5 The Concept of ChaosDE with Discrete Chaotic System as Driving CPRNG

The general idea of CPRNG is to replace the default PRNG with the chaotic system. As the chaotic system is a set of equations with a static start position, we created a random start position of the system, in order to have different start position for different experiments. Thus we are utilizing the typical feature of chaotic systems, which is extreme sensitivity to the initial conditions, popularly known as “butterfly effect”. This random position is initialized with the default PRNG, as a one-off randomizer. Once the start position of the chaotic system has been obtained, the system generates the next sequence using its current position. Used approach is based on the following definition (3):

$$ rndreal = \bmod \left( {{\text{abs}}\left( {rndChaos} \right), 1.0} \right) $$
(3)

6 Experiment Design

For the purpose of ChaosDE performance comparison within this research, the Schwefel’s test function (4), shifted Grienwang function (5), and shifted Ackley’s original function in the form (6) and shifted Rastrigin’s function (7) were selected.

$$ f(x) = \sum\limits_{i = 1}^{\dim } { - x_{i} \sin \left( {\sqrt {\left| {x_{i} } \right|} } \right)} $$
(4)

Function minimum:

Position for \( {\text{E}}_{\text{n}} :\,\left( {x_{ 1} ,x_{ 2} \ldots x_{\text{n}} } \right) = \left( { 4 20. 9 6 9,\, 4 20. 9 6 9, \ldots ,\, 4 20. 9 6 9} \right) \)

Value for \( {\text{E}}_{\text{n}} :y = - 418.983 \cdot dim \); Function interval: \( < -500,\, 500 > \).

$$ f(x) = \sum\limits_{i = 1}^{\dim } {\frac{{(x_{i} - s_{i} )^{2} }}{4000} - \prod\limits_{i = 1}^{\dim } {\cos (\frac{{x_{i} - s_{i} }}{\sqrt i })} } + 1 $$
(5)

Function minimum: Position for \( {\text{E}}_{\text{n}} :\left( {x_{ 1} ,x_{ 2} \ldots x_{\text{n}} } \right) = \varvec{s} \); Value for \( {\text{E}}_{\text{n}} :y = 0 \)

Function interval: \( < -50,\, 50 > \).

$$ \begin{aligned} f(x) & = - 20\exp \left( { - 0.02\sqrt {\frac{1}{D}\sum\limits_{i = 1}^{\dim } {\left( {x_{i} - s_{i} } \right)^{2} } } } \right) - \exp \left( {\frac{1}{D}\sum\limits_{i = 1}^{\dim } {\cos 2\pi \left( {x_{i} - s_{i} } \right)} } \right) \\ & \quad + 20 + \exp \left( 1 \right) \\ \end{aligned} $$
(6)

Function minimum: Position for \( {\text{E}}_{\text{n}} :\left( {x_{ 1} ,x_{ 2} \ldots x_{\text{n}} } \right) = \varvec{s} \); Value for \( {\text{E}}_{\text{n}} :y = 0 \)

Function interval: \( < -30,\, 30 > \).

$$ f(x) = 10\dim \sum\limits_{i = 1}^{\dim } {(x_{i} - s_{i} )^{2} - 10\cos (2\pi x_{i} - s_{i} )} $$
(7)

Function minimum: Position for \( {\text{E}}_{\text{n}} :\left( {x_{ 1} ,x_{ 2} \ldots x_{\text{n}} } \right) = \varvec{s} \), Value for \( {\text{E}}_{\text{n}} :y = - 90000\left( {{ \dim }\, 30} \right) \)

Function interval: \( < - 5. 1 2,\, 5. 1 2> \).

Where s i is a random number from the 90% range of function interval; s vector is randomly generated before each run of the optimization process.

The parameter settings for both canonical DE and ChaosDE were obtained based on numerous experiments and simulations (see Table 2). It was experimentally determined, that ChaosDE requires lower values of Cr parameter [25] for any type of used CPRNG. Canonical DE is using the recommended settings [1]. The maximum number of generations was fixed at 1500 generations. This allowed the possibility to analyze the progress of DE within a limited number of generations and cost function evaluations. Experiments were performed in the environment of Wolfram Mathematica; canonical DE therefore has used the built-in Wolfram Mathematica pseudo random number generator Wolfram Cellular Automata representing traditional pseudorandom number generator in comparisons. All experiments used different initialization, i.e. different initial population was generated within the each run of Canonical or ChaosDE.

Table 2. Parameter set up for ChaosDE and Canonical DE
Full size table

7 Results

Statistical results for the Cost Function (CF) values are shown in comprehensive Tables 3–6 for all 50 repeated runs of DE/ChaosDE, four different benchmark functions and five randomization schemes.

Table 3. Simple results statistics for the Canonical DE and ChaosDE; Schwefel’s function
Full size table

The bold values within the all Tables 3–6 depict the best obtained results, italic values are considered to be similar. Statistical comparisons are based on the Wilcoxon signed-rank test with significance level 0.05; and performed for the pairs of ChaosDE with CPRNG and identified similar PRNG distribution. The graphical comparisons of the time evolution of average CF values for all 50 runs of five versions of DE/ChaosDE with different randomizations and two selected benchmark functions are depicted in Figs. 3 and 4. The notation in Tables and Figures is following: Burgers/Lozi Map represents the chaotic based CPRNG, whereas Burgers/Lozi Dist represents identified distribution PRNG.

Fig. 3.
figure 3

Comparison of the time evolution of avg. CF values for the all 50 runs of Canonical DE, and four versions of ChaosDE with different randomization. Schwefel’s function.

Full size image
Fig. 4.
figure 4

Comparison of the time evolution of avg. CF values for the all 50 runs of Canonical DE, and four versions of ChaosDE with different randomizations. Shifted Rastrigin’s function.

Full size image

8 Conclusions

The primary aim of this work is to experimentally investigate the utilization of the various discrete chaotic systems, as the chaotic pseudo random number generator embedded into DE. Experiments are focused on the extended investigation, whether the different randomization and pseudo random numbers distribution given by particular PRNG or hidden complex chaotic dynamics providing the unique sequencing are more beneficial to the heuristic performance. The findings can be summarized as:

  • Obtained graphical comparisons and data in Tables 3–6 and Figs. 3 and 4 support the claim that chaos driven heuristic is more sensitive to the hidden chaotic dynamics driving the selection, mutation, crossover or other processes through CPRNG. The influence of different PRNG randomization (distribution) type is strengthened by the presence of chaotic dynamics and sequencing in the pseudo random series given by the dynamics of discretized chaotic attractor/flow.

  • It is clear that (selection of) the best CPRNGs are problem-dependent. By keeping the information about the chaotic dynamics driving the selection/mutation processes inside heuristic, its performance is significantly different: either better or worse against other compared versions.

  • In the first two cases (Schwefel and shifted Rastrigin function – Tables 3 and 4), the performance of ChaosDE was significantly better in comparison with canonical DE. Furthermore the effect of different PRNG distribution became even stronger with the chaotic dynamics kept inside CPRNG sequences. Lozi map based CPRNG has given stable better performance than similar identified PRNG. Both Lozi map based PRNG/CPRNG have been outperformed by the utilization of Burgers map based PRNG/CPRNG. An interesting phenomenon has been revealed. The Burgers map based not-chaotic PRNG drives DE to the strong and fast progress towards function extreme (local) followed by premature population stagnation phase. Whereas Burgers map CPRNG with chaotic dynamics secured the continuous development of population towards global best solution without stagnation.

    Table 4. Simple results statistics for the Canonical DE and ChaosDE; shifted Rastrigin’s func.
    Full size table

  • The third and the fourth case study (Tables 5 and 6) have given absolutely reversed character of results. Performance of Lozi based CPRNG/PRNG is comparable even with canonical DE (slightly better results for Lozi map CPRNG and Ackley function), whereas the Burgers map based randomization has given worse results. As aforementioned in the previous point, the premature stagnation for PRNG has occurred also here (more considerable), whereas the Burgers map based CPRNG with chaotic dynamics has driven the DE more or less towards the function extreme.

    Table 5. Simple results statistics for the Canonical DE and ChaosDE; shifted Ackley’s func.
    Full size table
    Table 6. Simple results statistics for the Canonical DE and ChaosDE; shifted Grienwang func.
    Full size table

  • Since the aim was to investigate the randomization/sequencing of indices selection inside DE, only the simplest canonical DE/Rand/1/Bin strategy has been utilized in this research. The parameter adjustment/strategy adaptation or ensembles techniques in jDE, EPSDE, SHADE may significantly interact with the dynamics of sequencing (selection) of indices driven by particular not-uniform PRNG/CPRNG.

  • Sequencing of pseudo random numbers and chaotic dynamics hidden inside pseudo random series can be significantly changed by the selection of chaotic systems, thus to avoid the CF landscape dependency. The simplest way for changing the influence to the heuristic during the run is to swap currently used chaotic system for different one, or to change the internal parameters of chaotic systems (Table 1).

  • Furthermore many previous implementations of chaotic dynamics into the evolutionary/swarm based algorithms (not-adaptive/adaptive/ensemble based) showed that it is advantageous, since it can be easily implemented into any existing algorithm as a plug-in module.