Keywords

1 Introduction

Nonlinear dynamics, especially the study of chaotic systems (CSs), have been increasing interest in various fields, including science, engineering, communication, biomedical, finance, administration, and other areas [1]. The chaotic system, commonly known as the butterfly effect, means unpredictable, erratic motions, and a complex behavior governed by deterministic laws. The chaotic system that is extremely responsive to the initial conditions and parameters [2]. Chaotic systems with low-order deterministic models could enhance interpretation and description of phenomena (recorded data) of complicated real-world systems. The machinery condition, electrical circuits [3], and subjects of diagnosis of human health status [4] are some of the recent applications of chaotic systems. Many chaotic systems simulate real-world systems such as Chen, Lorenz, Newton–Leipnik, Volta, Rossler [5], and Lotka–Volterra [6].

In recent years, the control and synchronization of CSs have been intensively studied in multiple subjects [7]. Most of the recommended techniques only work according to the conditions of CSs that are defined in advance. Nevertheless, the parameters may be very difficult to identify in the actual world due to the CSs complexity. Parameter estimation of CSs has great value in nonlinear science. Control theory and signal processing have drawn great attention in different research subjects and can be investigated as a multidimensional optimization problem (multi-DOP) [2].

In the last decade, parameter identification of CSs has attracted a lot of attention in the literature [8]. Different types of traditional techniques have been improved to investigate these problems [9]. The most common approaches utilized to express the problem of parameters estimation of CSs as a multi-DOP are metaheuristic algorithms such as PSO [10], the genetic algorithm (GA) [11], and differential evolution algorithm (DE) [12]. They seem to be an efficient alternative to traditional methods because they do not require the gradient of the objective function, are not responsive to the starting point, and seldom get trapped in local optima [13].

The investigation of the parameter identification problems has a significant history, with a special focus on the Lorenz system. Lorenz introduced this system in 1963 while observing atmospheric convection [14], and it is considered the first chaotic attractor of a three-dimensional self-government system. The parameters are not easy to acquire due to the unstable dynamic behavior of CSs. So in recent years, metaheuristic algorithms have been popular and frequently used by the researchers for their efficient performance in a solution of optimization problems [15,16,17]; various metaheuristic algorithms employed for parameters identification of this system have been proposed by minimizing synchronization error such as GA [11], PSO [10], improved particle swarm optimization (IPSO) [18], a drifting particle swarm optimization (DPSO) [19], a quantum-behaved particle swarm optimization (QPSO) [20], PSO with dynamic inertia weight PSO (DIW-PSO) [21], and chaotic ant swarm (CAS) [5, 22, 23]. It is noted that PSO and its modifications are the most frequently used approaches for parameter estimation of CSs.

PSO was proposed in 1995 by Kennedy and Eberhart [24] as a population-based algorithm inspired by the social behavior of swarms in nature such as bird flock and fish school. The algorithm seeks out the best solution via sharing the historical and social knowledge between the candidates (particles) throughout the global solution space. PSO algorithm has been used as an important optimization technique in various applications due to its simple implementation, few parameters to adjust, and fast convergence rate. On the other hand, PSO demonstrates some shortages. It is possible to get stuck in a local optimum in another sense, premature convergence, and the convergence rate reduces significantly in the later stages of evolution [25,26,27]. Therefore, many variants of the standard PSO approach have been proposed for different applications to make the most of the merits, improve the search capability and overcome the deficiencies such as immune PSO (IPSO) [28], continuous trait-based PSO (CTB-PSO) [29], and the hybrid GA-PSO [30, 31].

In this paper, parameters estimation of chaotic Lorenz system is modeled as a multi-DOP and solved based on a modified version of PSO by adding two terms to the velocity update formula. The modified algorithm is called memory-based particle swarm optimization (MbPSO). This modification of its role diversifies and enhances search capability. This is the first research of MbPSO for parameters estimation of the Lorenz system that the authors are aware of. The comparisons of the results acquired by other techniques show this algorithm’s efficiency, effectiveness, and robustness.

The main contributions of this paper can be summarized as follows:

  • The chaotic system is modeled, generally, and the Lorenz system is illustrated clearly.

  • A modified version of PSO is proposed called MbPSO.

  • The proposed algorithm (MbPSO) is applied for the Lorenz system to estimate its parameters.

  • Comparisons are made between the proposed algorithm (MbPSO), modified PSO (PSO+), original PSO (programmed algorithms for this study), and other algorithms used in the literature.

The remainder of this paper is organized as follows: Section 2 introduces the problem formulation regarding the Lorenz system. Section 3 reviews the basic principles of PSO. Section 4 introduces the proposed MbPSO algorithm, and the parameter configuration for the proposed algorithm is illustrated in Sect. 5. The numerical simulation, and comparisons are presented in Sect. 6. Finally, this paper is concluded in Sect. 7.

2 Problem Formulation

Considering the n-dimensional chaotic system modeled by the ordinary nonlinear differential equation as below (ONLDE):

$$\dot{X} = f\left( {X,X_o ,\theta_o } \right)$$
(1)

where \(X{ } = { }\left( {x_1 ,x_2 ,{ }.{ }.{ }.{ },x_n } \right)^T\) \(\in R^n { }\) indicates the state vector, \(x_0 { }\) indicates the initial state and \(\theta_0 { } = { }\left( {\theta_{1o} ,\theta_{2o} ,{ }.{ }.{ }.{ },\theta_{do} } \right)^T\) is a set of original parameters. During calculating the parameters, assume the chaotic system structure is known in advance, and hence, the approximated system can be defined as follows:

$$\dot{Y} = f\left( {Y,X_o ,\theta } \right)$$
(2)

where \(Y{ } = { }\left( {y_1 ,y_2 ,{ }.{ }.{ }.{ },y_n } \right)^T\) \(\in R^n { }\) indicates the state vector, and \(\theta { } = { }\left( {\theta_1 ,\theta_2 ,{ }.{ }.{ }.{ },\theta_d } \right)^T\) is a set of calculated parameters. Thus, the problem of the parameter estimation can be described as the following optimization problem:

$${\text{Min}}\,J = \frac{{1}}{{M}}\sum_{{k = 1}}^{M} {{X}_{k} { - Y}_{k}^{2} \,{\rm{by\,searching\,suitable}}\,\theta^* }$$
(3)

where the length of data used for parameters estimation is defined by M. The state vectors of the original and the estimated systems at time k (k = 1,2, ..., M) are denoted by Xk and Yk, respectively.

The parameters estimation for CSs is obviously expressed as a problem of multi-DOP, where the vector of decision is \(\theta\), and minimization of J is the objective function of the optimization problem. The parameters estimation problem of CSs based on optimization techniques can be demonstrated by Fig. 1. The parameters are not easy to acquire due to the unstable dynamic behavior of CS. Furthermore, multiple variables always exist in the problem, and multiple local optima solutions mislead the algorithm in the search zone of J, so conventional techniques are very easy to trap in local optima, and the global optimal parameters are difficult to attain. Thus, Lorenz system was selected as a CS model to test the performance of the proposed algorithm. Lorenz system [14] has an unpredictable complex dynamic behavior and exhibits infinite erratic periodic motions with extremely dependent initial conditions and parameters. The Lorenz model is employed as an equivalent model about the behavior of the atmosphere because it replicates some of the features of large-scale weather patterns, such as the simulation of system behavior, variations in the predictability of local events, and various time scales. The equations of the Lorenz system are system of three ONLDE resulting from a simple form of the fluid convection between two horizontal plates that is called Rayleigh Bénard problem. Lorenz system designated three time-dependent variables: convection motion intensity is represented by x, the temperature differential between increasing and decreasing currents is denoted by y, and z is the deviation from linearity in the vertical temperature profile [32]. The following model is a mathematical representation of the Lorenz system:

$$\left\{ {\begin{array}{*{20}l} {\dot{x} = A\left( {y - x} \right)} \hfill \\ {\dot{y} = Bx - xy - y} \hfill \\ {\dot{z} = xy - Cz} \hfill \\ \end{array} } \right.$$
(4)
Fig. 1
figure 1

The overall scheme for optimization concept of parameters estimation in CSs

Full size image

where \(\dot{x} = {\text{d}}x/{\text{d}}t,\dot{y} = {\text{d}}y/{\text{d}}t,\dot{z} = {\text{d}}z/{\text{d}}t\), A is defined as the Prandtl number, B \(\left( {B = R_a /R_c } \right)\) is defined as the Rayleigh number over the critical Rayleigh number, and C gives the size approximated by the system of the region [32]. All the parameters are positive numbers; for example, Ref. [33] demonstrates the behavior of a chaotic attractor with initial conditions (X0 = 1, Y0 = 0, Z0 = 0) and A = 10, B = 28, and C = 8/3. For a complete study of this system, see [33] (Fig. 2).

Fig. 2
figure 2

The behavior of Lorenz system with A = 10, B = 28, C = 8/3 and initial conditions (X0 = 1, Y0 = 0, Z0 = 0) for evolving time = 100 s. a) the projection of Lorenz attractor into x–y plane; b) the Lorenz attractor in three-dimensional space

Full size image

3 Particle Swarm Optimization (PSO)

PSO is considered as an evolutionary algorithm that is based on individual improvement in addition to collaboration and competition in the population. It depends on the simulation of simplified social models, such as the swarm theory: fish schooling, bird flocking [24]. PSO has a highly obvious, straightforward, and easy-to-implement theoretical structure.

PSO initializes randomly with a swarm of birds (particles) over the searching space. The particles search with a certain velocity and find the global optimum after several iterations. The main idea is to adjust the trajectory of each particle to its own best position and to the best particle of the swarm at each iteration. At each iteration, the velocity vector of each particle is affected by its inertia, its best position, and the position of the best particle. Then, each particle moves to a new position. Assume that, the location and the velocity of particle i in the d-dimensional search space can be expressed as \(X_i = \left[ {x_{i,1} ,x_{i,2} , \ldots ,x_{i,d} } \right]^T\) and \(V_i = \left[ {v_{i,1} ,v_{i,2} , \ldots ,v_{i,d} } \right]^T\), respectively. The particle fitness can be estimated according to the objective function of the optimization problem. The best position visited previously of particle i is denoted by the personal best position (pbest). It can be expressed as \(P_i \left( t \right) = \left[ {p_{i,1} \left( t \right),p_{i,2} \left( t \right), \ldots ,p_{i,d} \left( t \right)} \right]^T\). The position of the best particle of the swarm is defined by the global best position (gbest) \(P_g \left( t \right) = \left[ {p_{g,1} \left( t \right),p_{g,2} \left( t \right), \ldots ,p_{g,d} \left( t \right)} \right]^T\). At each step, the velocity and position of each particle is determined as follows:

$$\begin{aligned} v_{i,j} \left( {t + 1} \right) & = v_{i,j} \left( t \right) + c_1 r_1 \left( {p_{i,j} \left( t \right) - x_{i,j} \left( t \right)} \right) \\ & \quad + c_2 r_2 \left( {p_{g,j} \left( t \right) - x_{i,j} \left( t \right)} \right), \quad j = 1,2, . . .,d \\ \end{aligned}$$
(5)
$$x_{i,j} \left( {t + 1} \right) = x_{i,j} \left( t \right) + v_{i,j} \left( {t + 1} \right),\quad j = 1,2, \ldots ,d$$
(6)

where \(\omega\), indicates to the inertia weight, controls the effect of previous particle velocity on its current one. C1 and C2 are the cognitive and collective acceleration coefficients, respectively, which are balance the effects of self-awareness and social awareness on particle movement toward the target and adjust the step size. r1 and r2 are two independent uniform distributed random constants within the range of [0, 1].

In PSO, Eq. (5) implies that the particle’s new velocity is updated based on its previous velocity and the distance of its present position from both the global best position of the entire swarm and its best historical position. A typical approach is to constrain the values of the elements of the vector vi to a range between [Vi, min, Vi, max] to control excessive wandering of particles outside of the search space [Xi, min, Xi, max]. The particle then moves toward a new position in accordance with Eq. (6). The procedure is repeated until a termination condition determined is met [34].

4 Memory-Based Particle Swarm Optimization (MbPSO)

This section presents a new modified version of PSO called memory-based particle swarm optimization (MbPSO). The suggested algorithm (MbPSO) is a proposed improvement for the particle velocity updating equation to determine the parameters of Lorenz system. Although the position updating equation in MbPSO is identical to the standard PSO, the equation of the velocity updating is improved by the addition of two new terms, as illustrated below:

$$\begin{aligned} & v_{i,j} \left( {t + 1} \right) = \Psi \left( {\omega v_{i,j} \left( t \right) + c_1 r_1 \left( {{\text{pbest}}_{i,j} \left( t \right) - x_{i,j} \left( t \right)} \right) + c_2 r_2 \left( {{\text{gbest}}_j \left( t \right) - x_{i,j} \left( t \right)} \right)} \right. \\ & \quad + \left. {c_3 r_3 \left( {{\text{pbest}}_{i,j} \left( {{\text{rand}}} \right) - x_{i,j} \left( t \right)} \right) + c_4 r_4 \left( {v_{i,j} \left( {{\text{rand}}} \right)} \right)} \right){ } \\ \end{aligned}$$
(7)
$$\begin{aligned} & x_{i,j} \left( {t + 1} \right) = x_{i,j} \left( t \right) + v_{i,j} \left( {t + 1} \right), \\ & \forall \,i \in {\text{number of particles,}}\forall \, j \in {\text{problem dimension, and }}\forall t\, \in \,{\text{number}}\,{\text{of}}\,{\text{iterations}} \\ \end{aligned}$$
(8)

where ω is defined by the inertia weight. r1, r2, r3, and r4 are random numbers within [0, 1]. C1, C2, C3, and C4 denote the acceleration constants that control the effect of each term in the update process. pbest denotes the particle’s best solution, while gbest denotes the best solution reached by all particles. Knowing that the acceleration constants pull each particle to its best individual and global locations. As a result, small values cause short movements toward target regions, whereas high values cause abrupt movements. High values of the inertia weight, on the other hand, result in a broad search across the search space, whereas low values result in a more focused search [34].

In the original PSO, the first term, “inertial part” is represented as \(\omega {{\varvec{v}}}_{{{\varvec{i}}},{{\varvec{j}}}} \left( {{\varvec{t}}} \right)\) which is utilized as a search skill factor by using \(\omega = \omega_{\max } - t\left( {\omega_{\max } - \omega_{\min } } \right)/t_{\max }\), where \(\omega_{\min }\) and \(\omega_{\max }\) are the values of minimum and maximum inertia weight, respectively, and \(t_{\max }\) refers to the maximum number of iterations [35]. The second term “cognitive part” is represented as \(c_1 r_1 \left( {{\text{pbest}}_{i,j} \left( t \right) - x_{i,j} \left( t \right)} \right)\) that is defined as the distance between the particle’s current position and the particle’s most well-known position, and it means that the particle’s decision will be influenced by its previous experiences. The third term “social part’’ is represented as \(c_2 r_2 \left( {{\text{gbest}}_j \left( t \right) - x_{i,j} \left( t \right)} \right)\) which is defined as the distance between the particle’s current position and its best neighborhood position, and it means that the particle’s decision is influenced by the rest of the particles [34]. From this idea, we added two new terms. The fourth term, “random self-cognition,” is represented as \(c_3 r_3 \left( {{\text{pbest}}_{i,j} \left( {{\text{rand}}} \right) - x_{i,j} \left( t \right)} \right)\) which relates the particle’s position to the best position of random particles, which in fact let each particle exploit the memory of other particles and allow the particles to randomly share their knowledge during the updating process; it results in a stronger interrelation between the whole swarm, higher chance in convergence toward good solutions, and higher diversity of the search process. The fifth term, which is represented as \(c_4 r_4 v_{i,j} \left( {{\text{rand}}} \right)\), includes a random velocity that leads to increasing the potential of space exploration and preventing the MbPSO from being stuck in local optima. The new terms are inspired by [36].

The researchers utilize many additional terminologies with different meanings (e.g., [23, 37]). Still, this paper incorporates modifications to diversify the population search direction and enhance the swarm’s search capability. In original PSO, gbest is utilized to enhance convergence characteristics, but the drawback of using gbest is decreasing the diversity of the population that results in local minima. Thus, adding two new terms increases search capability, providing new knowledge and more information to the population. It can guide the particles to a better position, and the attraction of gbest position to local optima in the search space is reduced. Figure 3 depicts a graphical view of the updating process for a particle’s position and velocity in the MbPSO algorithm. Also, the overall procedures of MbPSO are demonstrated in Fig. 4.

Fig. 3
figure 3

Illustration of the particle’s update equations in MbPSO

Full size image
Fig. 4
figure 4

Flowchart of the proposed algorithm (MbPSO)

Full size image

5 Parameter Configuration for the Proposed Algorithm (MbPSO)

To make the best use of the proposed algorithm, the range of the parameter values are taken from the literature [36], and the trial and error method was used through the implementation on Lorenz system for configuring the parameters of the proposed algorithm during the operation of minimizing J. Thus, a population of 100 particles (p) was used; an inertia weight ω was established as linearly reducing from the peak value of 0.9 to a minimum value of 0.4. The acceleration constants were used as follows C1 = 1.5, C2 = 2, and C3 = 3 [38]. As different values of parameter C4 have a significant impact on the proposed algorithm, the various effects of C4 were studied for values under one; based on the literature, C4 was studied without including the constriction factor, and Fig. 5 shows the various effect of C4. For this study, the constant C4 = 0.3 introduces a better influence on the convergence than the others (the procedures for C4 are based on the coefficient mentioned above with other terms in [23]); in another aspect, the constriction factor has an extremely great effect on the proposed algorithm; following C4 is set to 0.3, the effect of constriction factor was analyzed from 0.2 to 0.8. Figure 6 shows the various effect of the constriction factor on the convergence and demonstrates that the constant ѱ = 0.4 presents the best convergence. The maximum generation number was set to tmax = 100 and tmax = 200, and this is illustrated in the section of discussion and results. All simulations were implemented 20 times. Table 1 displays the parameters used in the compared algorithms (PSO, PSO+, and MbPSO).

Fig. 5
figure 5

Convergence graph. Specifying of the best value of constant C4

Full size image
Fig. 6
figure 6

Convergence graph. Specifying of the best value of constriction factor Ѱ

Full size image
Table 1 Parameter configuration for all algorithms used
Full size table

6 Results and Discussion

The proposed algorithm MbPSO is implemented to estimate the parameters of the Lorenz chaotic system (A, B, and C). Figure 7 and Fig. 8 show the convergence characteristic of the fitness function J for MbPSO, PSO, and PSO + . These figures display that the value of J reduces to zero rapidly, which means MbPSO can converge to the global optimum very fast.

Fig. 7
figure 7

Convergence characteristic of objective function (J) for three algorithms PSO, PSO+, and MbPSO (100 iterations)

Full size image
Fig. 8
figure 8

Convergence characteristic of the objective function (J) for three algorithms PSO, PSO+, and MbPSO (200 iterations)

Full size image

Furthermore, MbPSO is compared with a particle swarm optimization (PSO) [34] and a modified PSO (PSO+) [6] to test the performance of the proposed algorithm. The comparison was made for two cases: the first case, the maximum number of iterations was set to 100, the range of the estimated parameters is \((8 < A < 12,{ }20 < B < 30,{ }2 < C < 3)\).

Table 2 compared the error and standard deviation (St.D.) values found by PSO, PSO+, and MbPSO. As shown in Table 2, MbPSO has fast convergence and accurate performance. Figure 7 illustrated that MbPSO has high performance and fast convergence, but PSO and PSO+ had almost the same performance. The second case, when the maximum generation number was set to 200 and the range of the estimated parameters increased \((0 < A < 20,{ }0 < B < 50,{ }0 < C < 5)\), the performance of the modified algorithm (PSO+ and MbPSO) enhanced. Table 3 compared the error and St.D. values of PSO, PSO+ , and MbPSO. After the modifications of simulation conditions, MbPSO has the highest performance and the fastest convergence rate, and PSO+ has the second-best performance, as shown in Fig. 8.

Table 2 Comparison of convergence and statistical results of PSO, PSO+, and MbPSO (100 iterations)
Full size table
Table 3 Comparison of convergence and statistical results of PSO, PSO+, and MbPSO (200 iterations)
Full size table

Other comparisons can be made between the undefined parameter values A′, B′, and C′ found by the MbPSO and the values found by with PSO and PSO+ from fitness function performance. The convergence rates of the parameters for the three-dimensional Lorenz system using MbPSO, PSO, and PSO+ are shown in Fig. 9. All estimated parameters found by MbPSO are very similar to the real values in all simulations, as shown in Fig. 9. This result demonstrates that the trajectories of the estimated parameters converge to their real values asymptotically. Therefore, MbPSO converges much faster than PSO and PSO+ . To get a full picture of our estimates, the statistical results found in estimating the parameters A′, B′, and C′ using MbPSO, PSO, and PSO+ are summarized in Table 4. As shown in this table, the best results found by MbPSO are better than those obtained by PSO and the others. Additionally, the estimated parameter values found by MbPSO are still very similar to the real values of the original parameters.

Fig. 9
figure 9

Lorenz system searching process for A′, B′, and C′ using PSO, PSO+ , and MbPSO

Full size image
Table 4 Statistical results found by PSO, PSO+ , and MbPSO for the estimated parameters
Full size table

On the other side, the abilities of MbPSO are compared to other metaheuristic techniques solved the same problem. Table 4 shows the comparison between the proposed algorithm and: (DE) [12], PSO [2], and a hybrid swarm intelligence algorithm (PSO–ACO) [23] in parameters estimation of Lorenz system. In general, MbPSO has accurate results better than the best results determined by these metaheuristic techniques (Table 5).

Table 5 Results obtained by many parameter estimation algorithms and the proposed MbPSO
Full size table

7 Conclusion

In this work, parameters identification for chaotic systems is modeled as a multidimensional optimization problem. The problem is solved using a modified PSO version called a memory-based particle swarm optimization (MbPSO). Lorenz system is selected to test the performance of MbPSO. In the proposed algorithm, MbPSO, two new terms are added to the standard PSO to diversify and enhance search capability. Comparisons are made between the proposed algorithm MbPSO, the original PSO, a modified PSO (PSO+), and other algorithms published in the literature. The comparisons and results demonstrate that the suggested algorithm is an effective and useful arithmetic method for parameter identification of chaotic systems, especially the Lorenz system, with high efficiency, fast convergence process, and accurate performance.