1 Introduction

Identification of chaotic systems with unknown parameters is the subject of substantial and well-developed research issue in nonlinear science, and a deliberated parameter estimation for chaotic systems serves as the profound bases for chaotic control (Hübler 1989; Ott et al. 1990; Grebogi and Lai 1997; Liu et al. 2006) and synchronization of chaotic systems (Pecora and Carroll 1990; Kapitaniak 1995; Boccaletti et al. 2002; Liu et al. 2007; Coelho and Bernert 2009), wherein system parameter uncertainties can destroy or even break the control and synchronization in case the values of the parameters for dynamical systems are unknown or not exactly determined (Chen and Dong 1998; Fotsin and Woafo 2005; Fradkov and Evans 2005; Wang et al. 2011).

During the last two decades, both academia and industry in many fields not limited in nonlinear science have been witnessing the pressing needs for new efforts in dealing with the grand challenges arising from estimation of unknown parameters for chaotic systems. This is motivated, on the one hand by the fact that lots of natural and social systems (e.g., physics, meteorology, physiology, biology, sociology and economics) exhibit chaotic phenomena (Pecora et al. 1997; Sornette 2006), and to uncover the underlying chaotic dynamics which drive the dynamics of the aforementioned systems, the identification of a chaotic system with unknown parameters is indispensable; on the other hand, by the fact that the inherent complex characteristics of chaotic dynamics (e.g., sensitive dependence on initial conditions as well as on the variations of system parameter) bring about unique challenges to the problem of identification of the chaotic systems (He et al. 2007).

Over decades, the challenges brought by parameter estimation of nonlinear chaotic systems have led to the rising number of researches. Based on their estimating paradigms, we roughly divide them into two basic categories—control-based synchronization methods and optimization-based methods.

The fundamental concept of control-based synchronization method for parameter estimation is to feed certain control inputs to the slave (response) system so that the output of the slave system follows the output of the master (drive) system asymptotically (Pecora and Carroll 1990; Park 2005), meanwhile during the synchronizing process, the unknown parameters of the slave system are estimated via minimization of the synchronization errors (Chang 2006). A vast variety of control-based synchronization methods have been proposed for identifying chaotic systems. For instance, based on adaptive control method (Huberman and Lumer 1990; Liao 1998; Chen and Lu 2002; Elabbasy et al. 2004; Fotsin and Woafo 2005) designed an adaptive controller for synchronizing and identifying the modified Van der Pol–Duffing oscillators with large normalized parameters. Park (2005) addressed the problem of synchronization of Rossler systems with three uncertain parameters. Based on the Lyapunov stability theory, an adaptive control law was derived to make the states of two Rossler systems asymptotically synchronized. Yassen (2005) presented an adaptive control scheme for synchronization of Rossler and Lü systems when the parameters of the master system were fully uncertain or unknown and different with those of the slave system. Based on Lyapunov stability theory, the sufficient conditions for synchronization have been analyzed theoretically. Wang and Ge (2001) considered the problem of adaptive synchronization of uncertain nonlinear chaotic systems via adaptive backstepping technique with tuning functions. Bowong et al. (2006) proposed a robust adaptive observer -based response system to synchronize the given uncertain chaotic system, and to estimate unknown constants and uncertain parameters. Hyun et al. (2006) developed an adaptive fuzzy observer to estimate the unknown parameters, and the stability of the proposed system was guaranteed via Lyapunov stability theory. Maybhate and Amritkar (1999) introduced the combination of feedback-based synchronization method and an adaptive control method to estimate parameters for several chaotic systems. Simulation results demonstrated that the hybrid approach was effective and reasonably robust under noisy environment. Saha et al. (2004) adopted the hybrid approach proposed in Maybhate and Amritkar (1999) to estimate the parameter of the transmitter for chaotic secure communication.

Beyond the control-based synchronization method, some researchers are earnestly seeking for new ways to estimate the unknown parameters for chaotic systems. They noticed that parameter estimation for chaotic systems could be formulated as multi-modal constrained functional optimization problems with multi-dimensional decision variables. Generally, optimization-based methods attempt to find the best fit model to the time series data generated from chaotic dynamics with unknown parameters. A vast variety of optimization-based methods have been propounded for identifying chaotic systems and tackling the difficulties. For instance, based on the time series data from a chaotic dynamic model with unknown parameters, Parlitz et al. (1996, 1996) estimated its parameters by minimizing the average synchronization error. Moreover, for the past three decades, as universal problem solvers, population-based metaheuristics (PBMH), e.g., Genetic Algorithms (GAs) (Holland 1975; Goldberg 1989), and Evolutionary Programming (EP) (Fogel et al. 1966; Bäck 1996), have surfaced to become a mainstay of optimization (Chen and Ong 2012; Jiang et al. 2014). Population-based metaheuristics are characterized by iterative, random, population-based and often biologically or socially inspired features (Liu et al. 2011), and can be applicable to a wide range of optimization problems without being tailored and can find near- or global-optimal solutions with acceptable computational costs (Hart et al. 2004; Ong and Keane 2004; Ong et al. 2007, 2009). In this respect, Dai et al. (2002) adopted Genetic Algorithm (GA) (Holland 1975; Goldberg 1989) to estimate Lorenz system with one unknown parameter. Chang (2006) applied Differential Evolution Algorithm (DEA) (Storn and Price 1997) to identify parameters of Rossler system via mutation, crossover, and selection operations. Further, Ho et al. (2010) proposed a hybrid algorithm by combing DEA with Taguchi-sliding-level method to solve the problem of parameter identification for Chen, Lü and Rossler systems. Wang and Li (2010) enhanced the DEA by incorporating quantum-inspired operators for estimating parameters of the Lorenz system. He et al. (2007) propounded the Particle Swarm Optimization (PSO) algorithm (Kennedy et al. 2001; Wang and Liu 2008; Helwig et al. 2013; Lou et al. 2015) to estimate the parameters of Lorenz system. Numerical simulation and the comparisons demonstrated the effectiveness and robustness of PSO. Moreover, the effect of population size on the optimization performances was investigated as well. Chang et al. (2008) introduced the Evolutionary Programming (EP) (Bäck 1996) for solving the parameter identification problem for Lorenz, Lü, and Chen systems. Wang and Xu (2011) presented an improved Biogeography-Based Optimization (motivated by biogeography theory) for identifying chaotic system with unknown parameters.

As a special incarnation of population-based metaheuristic, teaching–learning-based optimization (TLBO), which is inspired by the teaching and learning processes in a class, has been proposed (Rao et al. 2011; Rao and Patel 2012, 2013a, b) as an alternative to GA, PSO and DEA for functional optimization. In TLBO, first, a population of solutions is initialized randomly, in which the most knowledgeable individual (solution with the best fitness value) is generally regarded as the teacher, while the remaining individuals are considered as students. Next, the population is evolved to find optimal solutions through (1) teaching phase in which the teacher helps the students to improve their grades, as well as (2) learning phase in which the students improve their grades through interactions among themselves. Compared with GA, PSO and DEA, TLBO has some attractive characteristics. First, it employs simple differential operation between teacher and students to create new candidate solutions, as well as to guide the search toward the most promising region. Second, TLBO works with real numbers in natural manner and avoid complicated generic searching operators in GA, and twofold updating strategy in PSO. Third, the conventional TLBO only contains one adjustable controlling parameter (i.e., population size), which facilitates easy tuning and implementation, while in GA, PSO and DEA more parameters need to be set in an appropriate manner so as to guarantee the searching performance. Nowadays, TLBO has attracted attention and applications in a few fields since its birth in 2011 (Patel and Savsani; Crepinsek et al. 2012; Waghmare 2013). For instance, Rao and Patel (2013c) enhanced the conventional TLBO to form M-TLBO (named by the authors of this paper) by incorporating three operations, i.e., group learning (number of teachers), adaptive teaching factor and self-motivated learning. I-TLBO (Patel and Savsani 2014a, b) was developed by introducing an additional operation, labeled as learning through tutorial operator into the above M-TLBO. Application areas cover dynamic economic emission dispatch (Niknam et al. 2012), structural optimization (Dede 2013), power system (García Ansola et al. 2012), heat exchangers (Rao and Patel 2013b; Patel and Savsani 2014b), thermoelectric cooler (Venkata Rao and Patel 2013) and engineering optimization problems (Yu et al. 2014), etc., which demonstrate the effectiveness and efficiency of the TLBO-based algorithms.

From our survey, to date, there has been a lack of research studies that concentrate on TLBO for system identification, let alone the parameter estimation for chaos systems. The objective of this investigation is explicitly set out to fulfill this role. We begin by introducing the parameter estimation problem for chaotic systems which is cast into a multi-dimensional numerical optimization problem. Subsequently, we give a brief review and the implementation details of TLBO. An analysis on the performance of TLBO on parameter estimation of chaotic systems is then conducted, which reveals TLBO often risks getting prematurely stuck in local optima despite its global fast coarse search capability. Based on the analysis, we incorporate Nelder–Mead simplex algorithm (Lagarias et al. 1998) into TLBO to enhance its local searching performance through reflecting, expanding, contracting, and shrinking operators that would enrich the searching modes and behaviors. It is worth noting that the empirical results with the well-known Lorenz system indicate the success of the proposed approach. Further, it is hoped that the proposed methodology will serve as a vital parameter estimation method for other systems, e.g., physical, social and economic systems.

2 Problem formulation and chaotic system

2.1 Optimization problem for parameter estimation

Without loss of generality, n-dimensional chaotic system can be stated as (He et al. 2007; Tien and Li 2012):

$$\begin{aligned} \dot{X}=F(X,X_0 ,\theta _0 ) \end{aligned}$$
(1)

where \(X=(x_1 ,x_2 ,\ldots ,x_n )^T\in R^n\) is the state vector, \(X_0 \) is the initial state, and \(\theta _0 =(\theta _{10} ,\theta _{20} ,\ldots ,\theta _{d0} )^T\) is a set of original parameters.

The estimated chaotic system can be stated as Eq. (2), given that the structure of the system is known in advance:

$$\begin{aligned} \dot{Y}=F(Y,X_0 ,\theta ) \end{aligned}$$
(2)

where \(Y=(y_1 ,y_2 ,\ldots ,y_n )^T\in R^n\) is the state vector, and \(\theta =(\theta _1 ,\theta _2 ,\ldots ,\theta _d )^T\) is the set of parameters to be estimated.

From the viewpoint of optimization, the problem of parameter estimation can be formulated as:

$$\begin{aligned} {\mathbf{min}}\;J=\frac{1}{M}\sum \limits _{k=1}^M {\left\| {X_k -Y_k } \right\| ^2} \;\text{ by } \text{ searching } \text{ suitable }\;\theta ^*\end{aligned}$$
(3)

where M is the length of time series data used for parameter estimation, \(X_k \) and \(Y_k \quad (k=1,2,\ldots ,M)\) are the state vectors of the original and the estimated systems at time k, respectively.

The principle of parameter estimation for chaotic systems in sense of optimization can be illustrated in Fig. 1, where the decision vector is \(\theta \) and the optimization objective is to minimize J.

Fig. 1
figure 1

The illustration of estimating unknown parameters for chaotic systems

Full size image

In the present investigation, the conventional and improved versions of TLBO are considered for solving the above minimization problem which involves searching suitable estimated parameters \(\theta ^*\) by minimizing objective function (3). Meanwhile, constraints arising from chaotic dynamics (1) and (2) should be obeyed. In the experiments, the chaotic dynamical system \({\mathbf{f}}:R^n\rightarrow R^n\) is then instantiated with the well-established Lorenz system.

2.2 Chaotic system used

Lorenz system is described as follows (He et al. 2007):

$$\begin{aligned} \left\{ {\begin{array}{l} \dot{x}_1 =a(x_2 -x_1 ) \\ \dot{x}_2 =bx_1 -x_1 x_3 -x_2 \\ \dot{x}_3 =x_1 x_2 -cx_3 \\ \end{array}} \right. \end{aligned}$$
(4)

where \(a=10\), \(b=28\), \(c=8/3\) are the original parameters.

3 Teaching–learning-based optimization (TLBO)

In this section, the TLBO approach is described. In the TLBO system, a population of solutions corresponding to a group of learners is initialized randomly. The most knowledgeable individual, which is analogous to the elite solution with the best fitness value in the search, then poses as the teacher, while the remaining individuals in population are considered as the learners or students. Each dimension of an individual solution in the TLBO represents the grade of different subject as attained by a teacher or learner. The population is then evolved to locate optimal solutions through a teaching phase in which the teacher helps the students to improve their grades as well as a learning phase where students improve their grades through interactions among themselves.

The ith individual in the d-dimensional search space at generation t can be represented as \(X_i (t)=[x_{i,1} ,x_{i,2} ,\ldots ,x_{i,d} ]\), (\(i=1,2,\ldots ,NP\), where NP denotes the size of the population). As the teacher is considered the most knowledgeable person, the best member \(X_\mathrm{best} (t)\) of the current population as defined by the objective function or fitness value is considered as the teacher. In minimization problem, the solution or individual with the smallest objective function value is thus regarded as the best member. At each generation t, the teaching andlearning operations are applied on the learners, and a new population arises. Then, comparison takes place, and the corresponding individuals from both populations compete to comprise the next generation.

For each learner \(X_i (t)\), according to the teaching operation, an updated learner \(V_i (t)=[v_{i,1} ,\ldots ,v_{i,j} ,\ldots ,v_{i,d} ]\) is generated by adding the weighted difference between the teacher and mean grade of learners to itself, which takes the following form:

$$\begin{aligned} V_i (t)=R\cdot *( {X_\mathrm{best} (t)-T\cdot *M(t)})+X_i (t) \end{aligned}$$
(5)

where the arithmetic operator \(\cdot *\) is element-by-element multiplication. \(R=[\mathrm{rand}_1 ,\ldots ,\mathrm{rand}_j ,\ldots ,\mathrm{rand}_d ]\) is d-dimensional random weight vector which controls amplification of the differential variation \(X_\mathrm{best} (t)-T\cdot *M(t)\), and each element \(\mathrm{rand}_j \) is the j-th independent random number which is uniformly distributed in the range of [0, 1]. As described previously, \(X_\mathrm{best} (t)\), the base vector to model after, is the best member of the current population so that the finest traits of the teacher can be passed to the learners. \(M(t)=[m_1,\ldots ,m_j ,\ldots ,m_d ]\) denotes the mean grade of the learners for each subject. T, known as the teaching factor which represents the aptitude of the teacher, is a d-dimensional random weight vector that controls the changes to the mean grades of learners. The value for each element of T is then either 1 or 2, as recommended in Rao et al. (Rao et al. 2012).

After all the learners have completed the teaching phase, the one to one selection operator is then applied on each individual to decide whether the updated learner \(V_i (t)\) or the original \(X_i (t)\) would become a member of the population that would subsequently undergo the learning phase. Thus, for each target individual, anew trial vector \(U_i (t)=[u_{i,1} (t),\ldots ,u_{i,d} (t)]\) is generated and assigned to the value \(X_i (t)\) if the target learner could not improve itself in the teaching process; otherwise, \(U_i (t)\) is set to be \(V_i (t)\).

For each trial vector \(U_i (t)\) of the ith learner, through the learning phase, learners improve themselves by learning from others in the group, which is described as following (for minimization problem):

$$\begin{aligned} W_i (t)\!=\!\left\{ \! {\begin{array}{l} U_i (t)+\mathrm{Rand} \cdot *( {U_i (t)\!-\!U_j (t)}),\quad \,\, \text {if}\ F( {U_i (t)})\!<\!F( {U_j (t)}), \\ U_i (t)+\mathrm{Rand} \cdot *( {U_j (t)-U_i (t)}),\quad \text {otherwise}. \\ \end{array}} \right. \end{aligned}$$
(6)

where \(W_i (t)\) is the trial vector of target individual i after learning. Rand is a d-dimensional random weight vector which controls amplification of the differential variation. The subscript j of \(U_j (t)\) denotes a randomly selected target individual j from the population \(\{1,2,\ldots ,NP\}\) and also different from the current index i. F( ) means the objective evaluation function, which is the value of Eq. (3) in our study.

Finally, the selection arises to decide whether the trial vector \(W_i (t)\) would be a member of the population of the next generation \(t+1\). For minimization problem, \(W_i (t)\) is compared with \(U_i (t)\) using the following one to one greedy based selection criterion:

$$\begin{aligned} X_i (t+1)=\left\{ {\begin{array}{l} W_i (t),\quad \text {if}\quad F(W_i (t))<F(U_i (t)), \\ U_i (t),\quad \text {otherwise}. \\ \end{array}} \right. \end{aligned}$$
(7)

where \(X_i (t+1)\) is the individual of the new population. The best individual of the new population is then determined, and then update \(X_\mathrm{best} (t)\) to \(X_\mathrm{best} (t+1)\) by selecting the better one between \(X_\mathrm{best} (t)\) and the best individual so far. The above operations are iterated until the stopping criterion is met, and \(X_\mathrm{best} \) is then the converged solution obtained. The procedure described above is considered as the standard version of TLBO. The key parameter in TLBO is NP (size of population). As described previously, the conventional TLBO only contains one adjustable controlling parameter (i.e., size of population) which facilitates easy tuning and implementation. This is in contrast to the GA, PSO and DEA, which have more parameters that are required to be appropriately defined to assert good and robust search performances.

4 Preliminary test on standard TLBO

This section investigates the feasibility of applying the recently proposed population-based metaheuristics TLBO for parameter estimation of chaotic systems. Working with the dynamics of the well-established Lorenz system, we conduct the analysis of the performance of TLBO on parameter estimation of chaotic system in terms of searching accuracy.

4.1 Optimization objective evaluation

To generate the initial state \(X_0 \) for both original and estimated systems, first let the original chaotic system with known parameters, incarnated as Lorenz system stated in Eq. (4), freely evolve from a random selected initial state. Through a period of transient process, a state vector is selected as the initial state \(X_0 \) for parameter estimation as shown in Fig. 1.

To generate the states \(X_k \) and \(Y_k \quad (k=1,2,\ldots ,M)\), the initial state \(X_0 \) is fed into both original and estimated systems simultaneously. By solving the ordinary differential equations, numbered as Eq. (1) (the original system with determined known parameters) and Eq. (2) (the estimated system with temporally estimated parameters by generate and test), respectively, the time series of successive M states (we take M=300 for Lorenz system) for both the original and the estimated systems are obtained which are used for computing estimation error J in Eq. (3), under the given set of estimated parameters.

4.2 Simulation on Lorenz system

Next, we make a systematic examination of the feasibility and effectiveness of TLBO for estimating the unknown parameters for chaotic systems. Preliminary test is conducted, meanwhile the results are examined to assess the relative performance of standard TLBO to other representative state-of-the-art metaheuristics for chaotic identification in literature, including particle swarm optimization (He et al. 2007), genetic algorithm (Dai et al. 2002), and quantum-inspired evolutionary algorithm (QEA) (Han and Kim 2002). It has been commented that QEA has superior performance to classical evolutionary algorithms in terms of convergence rate, searching quality, and robustness (Wang and Li 2010). Besides, as observed from previous researches that several variants of TLBO algorithms, e.g., M-TLBO (Rao and Patel 2013c), I-TLBO (Patel and Savsani 2014a, b) performed well, we also implement the two improved TLBO algorithms for estimating the unknown parameters for chaotic systems.

To allow for a fair comparison, we set the maximum function evaluation times which in this regard are the product of the maximum generation/iteration times and the population size to be same among all algorithms covered in this study. Specifically, we adopt from the above literature the values for the maximum generation number of 100 and population size of 20, 40 and 120 when the number of unknown parameter is 1, 2 and 3, respectively. In M-TLBO and I-TLBO, an additional control parameter, the number of teachers, requires to be specified. We adopt the values of 5 for population size of 20 and 40, and 10 for population size of 120. Furthermore, searching ranges of the estimated parameters are the same to all algorithms, that is, the estimated parameters a, b, and c are set in the range of [9, 11], [20, 30], and [2, 3], respectively.

4.2.1 One-dimensional parameter estimation

We first test the searching performance of TLBO for estimation of chaotic system with one unknown parameter, that is, each time only one of the three parameters a, b, and c needs to be identified. TLBO and its variants (I-TLBO and M-TLBO) are run 20 times independently for each case. The statistical results achieved by three TLBO-based algorithms for the three cases are listed in Table 1 together with the simulation results of other three algorithms in the literature, including PSO, GA, and QEA.

Table 1 Statistical result of different methods for one-dimensional parameter estimation
Full size table

From Table 1 (excluding the last three lines of TLBO-SM, which would be used in latter section), it can be seen that TLBO can achieve optimal estimated values in the statistical meaning of Best, which validates the feasibility of TLBO to identify the chaotic system with one unknown parameter. Next, we examine the statistical results of Average and Worst, and find that (1) for parameter a, TLBO is more or less as competent as PSO and GA, and superior to QEA; (2) for parameter b, TLBO is almost as competent as QEA, inferior to PSO, but superior to GA; and the same holds for parameter c. This phenomenon reveals TLBO suffers from the risks getting prematurely stuck in local optima. It is worth noting that both M-TLBO and I-TLBO perform best among all the algorithms for identifying the chaotic system with one unknown parameter.

4.2.2 Two-dimensional parameter estimation

Then, we examine the searching performance of TLBO for estimation of chaotic system with two unknown parameters, that is, each time two of the three parameters need to be identified. TLBO is run 20 times independently for each case. The statistical results obtained by TLBO-based algorithms for the three cases are listed in Tables 2, 3, 4 together with the simulation results of other three algorithms in the literature.

From Table 2 for estimating a and b (excluding the last column of TLBO-SM, which would be discussed in latter section), it can be seen that PSO performs best among the six algorithms, and TLBO can achieve almost the same search accuracy as PSO. The improved versions of TLBO, M-TLBO and I-TLBO perform better than standard TLBO. Besides, TLBO performs better than GA and QEA. The same conclusion holds for the results in Tables 3 and 4, which validates the feasibility of TLBO to identify the chaotic system with two unknown parameters. And by examining the statistical results of Average and Worst of TLBO, it could be found that TLBO still suffers from the risks of being trapped into local optima.

4.2.3 Three-dimensional parameter estimation

Last, we assess the searching performance of TLBO for estimating of all three unknown parameters simultaneously. The statistical results obtained by TLBO for running 20 times are listed in Tables 5.

Table 2 Statistical result of different methods for two-dimensional parameter estimation (a and b)
Full size table
Table 3 Statistical result of different methods for two-dimensional parameter estimation (a and c)
Full size table
Table 4 Statistical result of different methods for two-dimensional parameter estimation (b and c)
Full size table
Table 5 Statistical result of different methods for three-dimensional parameter estimation (a, b and c)
Full size table

From Table 5 for estimating all three unknown parameters (excluding the last column of TLBO-SM, which would be discussed in latter section), it can be seen that TLBO-based algorithms and PSO perform better than GA and QEA in the statistical meaning of Best, Average, and Worst criteria. Further, PSO is superior to TLBO in the criterion of Best, while PSO is inferior to TLBO in the criteria of Average and Worst. It is worth noting that M-TLBO and I-TLBO perform superiorly to PSO and standard TLBO in the criterion of Best, Average and Worst.

In summary, standard TLBO has received very interesting and notable results on the problem of parameter estimation for chaotic system, but generally TLBO lacks impetus in the search plateau, which suggests TLBO faces risks of getting prematurely stuck in local optima. In the following section, we will make an attempt to enhance the fine (local) searching ability of TLBO by incorporating Nelder–Mead simplex algorithm-based local search (Lagarias et al. 1998).

5 Hybrid TLBO with Nelder–Mead simplex algorithm (TLBO-SM)

To date, algorithm hybridization is one of the most important mainstays in the field of optimization, especially in the designing of memetic algorithms which is a union of a population-based global search and local improvements (Moscato 1989; Hart et al. 2004; Ong et al. 2007, 2009, 2010; Chen et al. 2011). It is well established that combining the features of distinct methods in a complementary manner may result in more effective and robust optimization approach (Ong et al. 2003; Liu et al. 2005; Zhu et al. 2007; Liu et al. 2011). Several studies have been focused on how to achieve a reasonable combination of global search and local search, and how to make a good balance between exploration and exploitation (Ishibuchi et al. 2003; Ong et al. 2006). Traditionally, most of the MAs rely on the use of one single population-based metaheuristic algorithm for globally rough exploration and one single local search for locally fine improvements. Some recent studies on the choice of local searches have shown that the choice significantly affects the searching efficiency (Ong and Keane 2004; Krasnogor and Smith 2005; Pan et al. 2008; Liu et al. 2010, 2011).

Inspired by the aforementioned research work in memetic algorithms which have materialized in the form of hybridization between the population-based search and refinement procedures (Liu et al. 2010; Chen et al. 2011; Le et al. 2012), we examine the effects of the incorporation of a local refinement method labeled here as Nelder–Mead simplex algorithm (Lagarias et al. 1998) into the population- based metaheuristic TLBO to form a hybrid optimization algorithm coined as TLBO-SM.

Fig. 2
figure 2

The framework of TLBO-SM

Full size image

5.1 Nelder–Mead simplex search

The Nelder–Mead simplex search is a direct search method for unconstrained optimization without relying on problem’s gradient information. Nelder–Mead simplex search algorithm uses a simplex of \(n+1\) points for \(n-\)dimensional vectors x. The algorithm first makes a simplex around the initial guess \(x_0 \) by adding \(\alpha \% \) of each component \(x_0 (i)\) to \(x_0 \), and using these n vectors as elements of the simplex in addition to \(x_0 \). Then, the algorithm modifies the simplex repeatedly through the reflection, expansion, contraction, and shrink operators, until certain stopping criteria are satisfied. The details of implementation of Nelder–Mead simplex algorithm can be found in Nelder and Mead (1965), Lagarias et al. (1998) and Avriel (2003), and to save space we omit the procedure of Nelder–Mead simplex search.

5.2 TLBO-SM

We materialize the TLBO-SM in the form of hybridization between the population-based TLBO search and Nelder–Mead simplex algorithm-based refinement procedure, whose framework is depicted in Fig. 2.

As illustrated in Fig. 2, the hybrid TLBO-SM represents a form of synergistic combination between population-based and local refinement heuristic, wherein first, via the multi-individual stochastic parallel search mechanism of TLBO, TLBO-SM provides better coverage of the searching domain (i.e., exploration), and provides a fast and reliable estimate of the global optimum; second, by taking advantage of initial guess point of high quality from TLBO (in this respect, \(X_\mathrm{best} (t))\), the Nelder–Mead simplex algorithm concentrates the searching effort in the neighborhood of the best solutions found so far (i.e., exploitation) to produce better solutions more efficiently. After one round of TLBO plus Nelder–Mead simplex algorithm search, the updated \(X_\mathrm{best} (t)\) generated via Nelder–Mead simplex algorithm would be fed back into TLBO system as the new best solution (i.e., teacher) for the next round search until certain termination criteria are reached.

6 Numerical simulation on TLBO-SM

We have investigated the feasibility and effectiveness of standard TLBO in Sect. 4 for the problem of identifying parameters of Lorenz system with one, two, and three unknown parameters, respectively. The preliminary test results demonstrate that as a viable method, standard TLBO has received very notable results, but TLBO still faces risks of getting prematurely stuck in local optima despite its global fast coarse search capability. In this section, we will make a systematic assessment of the proposed TLBO-SM algorithm in terms of searching accuracy and rate of convergence, so as to check whether the fine exploitation capability of TLBO could be enhanced by Nelder–Mead simplex algorithm-based local search procedure.

The results from TLBO-SM will be compared with those of standard TLBO, M-TLBO (Rao and Patel 2013d), I-TLBO (Patel and Savsani 2014a, b), particle swarm optimization (He et al. 2007), genetic algorithm (Dai et al. 2002), and quantum-inspired evolutionary algorithm (QEA) (Han and Kim 2002). In Sect. 4, the maximum function evaluation times are 2000 (one unknown parameter), 4000 (two unknown parameter), and 12,000 (three unknown parameters), respectively. To make fair comparisons, we set the maximum function evaluation times to be same among all algorithms involved in this research.

6.1 One-dimensional parameter estimation

We assess the searching performance of TLBO-SM for estimation of chaotic system with one unknown parameter. We return back to Table 1, whose last three rows list the statistical results obtained by running TLBO-SM 20 times independently. As shown in Table 1, TLBO-SM achieves the best values in the statistical meaning of Best, Average, and Worst, which not only validates the feasibility of TLBO-SM to identify the chaotic system with one unknown parameter, but also reveals that the local refinement ability of TLBO-SM has been improved greatly by incorporating Nelder–Mead simplex algorithm.

6.2 Two-dimensional parameter estimation

Next, we examine the searching performance of TLBO-SM for estimation of chaotic system with two unknown parameters. The statistical results obtained by TLBO-SM for the three cases are listed in the last column of Tables 2, 3, 4, respectively. As demonstrated in Tables 2, TLBO-SM performs best among all the algorithms. The same conclusion holds for the results in Tables 3 and 4, which suggests TLBO-SM could achieve notable results on the problem of parameter estimation for chaotic system, and could jump out of the local minima by being equipped with Nelder–Mead simplex algorithm.

Fig. 3
figure 3

Tuning trajectories of parameter a for Lorenz system

Full size image

6.3 Three-dimensional parameter estimation

Last, we assess the searching performance of TLBO-SM for estimating of all three unknown parameters simultaneously, which is more difficult. The statistical results obtained by TLBO-SM are listed in the last column of Table 5. From Table 5, it can be seen that TLBO-SM performs best in all of the statistical meanings.

For the case of estimating Lorenz system with three unknown parameters, we show the typical tuning trajectories of estimating parameters a, b and c in Figs. 3, 4, 5, respectively. As depicted in Fig. 3, after going through about 400 function evaluations, TLBO gets trapped into local minimum (the estimated value is 9) until it exhausts its entire allocated computation budget, which suggests TLBO lacks impetus in the search plateau. Nevertheless, TLBO-SM has a faster decline to estimated value of 9 around the number of function evaluation of 100, then TLBO-SM encounters the same situation as TLBO, that is, TLBO-SM gets prematurely stuck in local optima (the estimated value is also 9). Fortunately, TLBO-SM jumps out of the local optima to the global optima after hovering in the plateau about 400 function evaluations, which could be credited with the employment of Nelder–Mead simplex algorithm as the local refinement procedure, as well as the effective information exchange between TLBO and Nelder–Mead simplex algorithm.

The parameter tuning curves for b are illustrated in Fig. 4, from which it could be observed that TLBO-SM has a faster trend to global optima than TLBO; moreover, TLBO could be easily stuck in local optima and stagnates in the plateau for longer time. Even though TLBO jumps out of local minima for several times, it is pity that TLBO cannot find the global optima within the available time.

Fig. 4
figure 4

Tuning trajectories of parameter b for Lorenz system

Full size image
Fig. 5
figure 5

Tuning trajectories of parameter c for Lorenz system

Full size image

Further, we depict the turning curves for parameter c in Fig. 5. It is obvious that searching pattern of TLBO in Fig. 5 is more or less like that manifesting in Fig. 4. On the other side, TLBO-SM rapidly experiences its first premature convergence, whereas it promptly finds its best value after very short stagnation.

Finally, we demonstrate the convergence curves of objective value J for estimating Lorenz system with three unknown parameters by TLBO and TLBO-SM, respectively. As observed in Fig. 6, TLBO stagnates in different plateaus for several times, each time it needs to travel for a long time so as to jump from the current local minima. However, TLBO cannot find the global optima within the given computational budget. Yet, TLBO-SM rapidly meets with its first premature convergence, whereas it promptly jumps out of the plateau, and finds the global optima by going through about 500 function evaluation.

Fig. 6
figure 6

The convergence curve of objective value J for estimating Lorenz system with three unknown parameters

Full size image

From the viewpoint of accuracy (effectiveness) and rate of convergence (efficiency), TLBO-SM possesses high-performance searching competence, which could be ascribed to the fact that combining the different features from TLBO (more explorative) and Nelder–Mead simplex algorithm (more exploitative) in a complementary manner could bring about more effective and efficient optimization approach. In specific, the TLBO-SM hybrid could be materialized in the organized form of diversified searching operators, e.g., teaching, learning, and competing operations from TLBO, as well as the reflecting, expanding, contracting, and shrinking operators from Nelder–Mead simplex algorithm.

7 Statistical analysis

To show the statistical significance of the comparative results among the algorithms covered in this study, we conduct nonparametric pairwise comparisons (Sign test and Wilcoxon test) and multiple comparisons (Friedman’s rank test and Holm post hoc procedure) on the results achieved by TLBO, I-TLBO, M-TLBO, PSO, GA, QEA and our proposed TLBO-SM for parameter estimation of nonlinear chaotic system.

7.1 Pairwise comparisons

We first perform two prevailing pairwise statistical tests, i.e., Sign test and Wilcoxon test (Derrac et al. 2011; Rao and Patel 2013d) to demonstrate the performance difference between any pair of algorithms in an intuitive way. In the above statistical tests, the average performances of multiple independent running for each algorithm on every problem are used. In our study, seven algorithms (i.e., TLBO, PSO, GA, QEA, TLBO-SM, I-TLBO and M-TLBO) are utilized to solve seven-parameter estimation problems in total, including three one-dimensional parameter estimation problems, three two-dimensional parameter estimation problems, and one three-dimensional parameter estimation problem.

By means of Sign test, the average performance of TLBO-SM is compared separately with each algorithm, the results of which are summarized in Table 6. Wins (Loses) count the times when TLBO-SM performs superior (inferior) to its counterpart algorithm in the criterion of Average. The detected difference indicates that TLBO-SM outperforms TLBO, GA, and QEA with a significance level of 0.05.

Table 6 Result of sign test
Full size table

Furthermore, compared with Sign test, a more powerful pairwise test tool, Wilcoxon test (Derrac et al. 2011) is utilized which could take into consideration the degree of difference among searching performances. Specifically, the difference between two algorithms will be ranked according to its absolute value among all test instances (in our study, 7 instances). The sum of ranks that TLBO-SM is superior (inferior) to other algorithms is indicated by R\(^{+}\) (R\(^{-})\) in Table 7.

Table 7 Result of Wilcoxon’s test
Full size table

As observed from Table 7, the p values that TLBO-SM performs better than TLBO, GA, and QEA are at significance level of 0.05, and better than PSO at significance level of 0.1. Though the p values of Wilcoxon test cannot support that TLBO-SM outperforms I-TLBO or M-TLBO in statistical means, TLBO-SM performs better as the complexity of problem increases (with more parameters to be estimated, depicted from Table 5).

7.2 Multiple comparisons

To make up the imperfection resulting from simple pairwise test in which the evaluation result between one pair of algorithms is uncorrelated with other algorithms, we implement multiple comparisons tests (Derrac et al. 2011) to analyze the results achieved by various algorithms jointly. Friedman test is often used to identify the existence of differences in average performance among a group of algorithms. The null hypothesis H0 is that there is no significant difference among algorithms, while the alternative hypothesis H1 indicates the presence of significant differences. Table 8 indicates the results of Friedman test for all the algorithms involved in our study, including the rank of the performance for each algorithm, Friedman test statistic, critical value and p value. The null hypothesis is rejected since p value is less than significance level 0.05, which suggests that there exist significant differences among the algorithms.

Table 8 Friedman rank test for all parameter estimation problems
Full size table

To go a step further, we conduct Holm post hoc test (Patel and Savsani; Derrac et al. 2011) which considers comparison between a control method and a set of counterpart algorithms (\(1\times N\) comparisons) or between all the algorithms with each algorithm as a control method (\(N\times N\) comparisons). Table 9 lists the adjusted p values calculated by Holm procedure for multiple comparisons among all algorithms. As observed from Table 9 that the first 7 adjusted p values are \(<\)0.1, meanwhile the first 4 adjusted p values are \(<\)0.05, we conclude that the null hypothesis can be rejected at the corresponding significance levels which suggests that the performances between those algorithms are significantly different. Table 10 shows the adjusted p value with TLBO-SM as the control method, from which the improvement of TLBO-SM over GA, QEA and TLBO can be confirmed at the significance level of 0.05.

Table 9 Adjusted p values for multiple comparisons (\(N\times N\) comparison) among all algorithms
Full size table
Table 10 Adjusted p values for multiple comparisons (\(1\times N\) comparison) among all algorithms (TLBO-SM as the control method)
Full size table

8 Conclusions

Identification of chaotic systems with unknown parameters is the subject of substantial and active research field. Over the decades, the challenges brought about by this problem have led to the rising number of researches, as well as led to the pressing needs for new efforts in dealing with the grand challenges by estimation of nonlinear chaotic systems. After performing a concise review on the state-of-the-art methodology for estimating chaotic systems, we found there has been a lack of research studies that concentrate on teaching–learning-based optimization (TLBO) for system identification to date, let alone the parameter estimation for chaos systems. The objective of this investigate is explicitly set out for fulfilling this role.

The preliminary analysis was conducted by applying TLBO to the estimation of chaotic systems with unknown parameters, which revealed that TLBO often risked getting prematurely stuck in local optimum despite its global fast coarse search capability. Inspired by the research work in memetic algorithms which have materialized in the form of hybridization between the population-based search and refinement procedures, we proposed a new hybrid optimization algorithm coined as TLBO-SM by incorporating a local refinement method (Nelder–Mead simplex algorithm) into the population-based metaheuristic TLBO. As observed from simulation results together with the comparison with state-of-the-art metaheuristics in literature, including particle swarm optimization, genetic algorithm, and quantum-inspired evolutionary algorithm, TLBO-SM succeeded in solving the parameter identification problem. This could be credited, first to that multi-individual stochastic parallel search of TLBO provides better coverage and exploration of the searching domain; second to that the fine (local) searching ability has been greatly enhanced by incorporating the local search operators from Nelder–Mead simplex algorithm, e.g., reflecting, expanding, contracting, and shrinking; third to that the information has been efficiently exchanged and utilized by both counterpart algorithms. In this regard, TLBO provides high-quality initial guess points to Nelder–Mead simplex algorithm which is sensitive dependent on the initial starting points, meanwhile the solutions generated by Nelder–Mead simplex algorithm through fine exploitation would be fed back into TLBO system as the new best solution which offers the population more explorative stimulants.

We hope the proposed methodology could serve as a vital parameter estimation method for other systems, e.g., physical, social and economic systems (Ginsburgh and Keyzer 2002; Liu et al. 2012). Our future work will investigate the performances of TLBO-SM on such systems. Furthermore, we want to design TLBO-based algorithm for multi-objective optimization problems (Jiang et al. 2014), as well as to design TLBO-based feature selection algorithm to big data (Zhai et al. 2014).