1 Introduction

Strength of artificial intelligence algorithms (AIAs) has extensively been exploited to optimize the difficult nonlinear problems arising in a variety of fields [1–3]. Especially in computational and applied mathematics, AIAs based on unsupervised feed-forward artificial neural networks (ANNs) trained with global and local search methodologies have been applied to solve many linear and nonlinear systems. Few potential applications of AIA using ANNs, evolutionary and swarming intelligence, are highly stiff oscillatory systems based on Van der Pol-type nonlinear differential equations [4], mathematical model for fuel ignition model in combustion theory through one-dimensional Bratu-type equations [5], transformed third-order BVP of nonlinear two-dimensional Bratu-type equations [6], the maximum flow problem [7], similarity transformed system for thin film flow of third-grade fluids [8], nonlinear Lane–Emden-type equations [9], mathematical model for confinement of plasma represented with Troesch-type problems [10], magnetohydrodynamics (MHD) problem of fluid flow governed with Jeffery–Hamel-type equations [11], nonlinear Emden–Fowler type singular system [12], nonlinear first Painlevé-type equations [13, 14], nonlinear systems based on initial value problems [15], functional ordinary differential equations of pantograph types [16], fractional differential equations of nonlinear Riccati types [17], nanotechnology problems based on multi-wall carbon nanotubes [18] and fluid mechanic problem based on steady thin film flow of Johnson–Segalman fluid on vertical cylinder [19].

A number of numerical solvers have been proposed by the research community for solving problems based on nonlinear system of equations [20–22], but one of the simplest, oldest and, most commonly used numerical procedures is Newton–Raphson method (NRM) [23, 24], but its performance is sensitive to the initial weights as it is highly prone to getting stuck in the local minimum. Therefore, in order to utilize NRM algorithm effectively for the solution of system of nonlinear equations, it is supplied with biased initial guess by another global search solver. Besides NRM, many other iterative and recursive solvers have been introduced and used in applied mathematics, literature due to their unique advantages, limitations and applicability. For instance, Kelley, Campbell and Broyden’s provided numerical schemes for the solution of nonlinear equations and their systems [25–27]. Jacobian-free Newton–Krylov method has been used for deriving the lower triangular half of the sparse Jacobian matrix through automatic differentiation [28, 29]. Recently, Jafari and Gejji, Abbasbandy, Sharma and Vahidi have provided new solver [30–36] to effectively determine the solution of problems based on nonlinear equations. Most of the solutions available in the existing literature are based on deterministic procedure using iterative and recursive solvers [37–39], but well-established strength of the stochastic solvers, based on AIAs, has not been extensively explored for the solution of nonlinear equations. The aim of this study is to investigate the potential of AIA algorithms based on NIC technique for providing an alternate, accurate, convergent and efficient system for solving system of nonlinear equations.

In this study, NIC technique based on swarm intelligence aided with NMM is applied for solving nonlinear systems. Sixteen variants of particle swarm optimization (PSO) algorithm have been developed and applied to optimize the variables for five well-known nonlinear systems. The design parameters of these systems are further tuned with NMM for rapid local convergence. The salient features of the proposed methodologies are given briefly as:

  • Novel design of nature-inspired heuristic algorithms based on exploration and exploitation in standard PSO method hybrid with NMM to solve nonlinear systems.

  • Performance of the proposed schemes is evaluated on five benchmark models based on interval arithmetic benchmark, kinematic application, neurophysiology, combustion and chemical equilibrium models.

  • Reliability and effectiveness of the schemes are analyzed through the results of statistical analyses based on sufficiently large number of independent runs.

  • Computational complexity indices are used to measure the comparative efficiency.

  • Simple concept, effortlessness in implementations, consistent accuracy, steady convergence and wider applicability domain are the significant illustrative perks of the schemes.

The rest of the paper is organized as follows: In section two, design methodology for solving nonlinear equations and their systems is presented, i.e., formulation of the objective function and learning methodologies used for training. In section three, results of simulation are presented and compared with the studies based on statistical performance indices for accuracy and complexity. Conclusions and potential future research directions are submitted in the last section.

2 Methodology

Design scheme presented here for finding the solution of systems of nonlinear equations consists of two parts: In the first part, formulation of objective function based on the residual error in absolute or the mean square sense is formulated, while in the second part, details of PSO and its variants are outlined. The graphical abstract of the proposed methodology is shown in Fig. 1.

Fig. 1
figure 1

Graphical view of proposed design methodology

Full size image

2.1 Fitness function for systems nonlinear of equations

General form of system of nonlinear equations is written as:

$${\mathbf{X}}({\mathbf{y}}) = {\mathbf{0}}.$$
(1)

Equation (1) can be written in the expanded form as:

$${\mathbf{X}}({\mathbf{y}}) = 0, \Rightarrow \left[ {\begin{array}{*{20}c} {x_{1} ({\mathbf{y}})} \\ {x_{2} ({\mathbf{y}})} \\ \vdots \\ {x_{n} ({\mathbf{y}})} \\ \end{array} } \right] = {\mathbf{0}}, \Rightarrow \left\{ {\begin{array}{*{20}c} {x_{1} (y_{1} ,y_{2} ,y_{3} , \ldots ,y_{n} ) = 0,} \\ {x_{2} (y_{1} ,y_{2} ,y_{3} , \ldots ,y_{n} ) = 0,} \\ \vdots \\ {x_{n} (y_{1} ,y_{2} ,y_{3} , \ldots ,y_{n} ) = 0.} \\ \end{array} } \right.$$
(2)

To determine the solution for a system (1) with n number of unknowns and n number of equations, objective function based on the residual error in mean absolute or square sense can be written as [37, 38]:

$${\text{Oir}} = \left\{ \begin{aligned} \frac{1}{n}\sum\limits_{i = 1}^{n} {\left| {r_{i} } \right|,\quad {\text{or}}} \hfill \\ \frac{1}{n}\sum\limits_{i = 1}^{n} {r_{i}^{2} .} \hfill \\ \end{aligned} \right.$$
(3)

where (r 1, r 2, …, r n ) are the residual error functions associated with (x1, x 1, …, x n ) equations, respectively, and can be written as:

$${\mathbf{r}} = \left[ {\begin{array}{*{20}c} {r_{1} } \\ {r_{2} } \\ \vdots \\ {r_{n} } \\ \end{array} } \right] = \left[ {\begin{array}{*{20}c} {x_{1} (y_{1} ,y_{2} ,y_{3} , \ldots ,y_{n} )} \\ {x_{2} (y_{1} ,y_{2} ,y_{3} , \ldots ,y_{n} )} \\ \vdots \\ {x_{n} (y_{1} ,y_{2} ,y_{3} , \ldots ,y_{n} )} \\ \end{array} } \right].$$
(4)

Now objective is to determine the variables of system (1), such that the residual error function (3) \({\text{Oir}} \to 0\), which is only possible when each of r = [r 1, r 2,…,r n ] → 0, consequently \({\mathbf{X}}({\mathbf{y}}) \to {\mathbf{0}}\), i.e., values of the vector \({\mathbf{y}} = [y_{1} ,y_{2} ,y_{3} , \ldots ,y_{n} ]\) will be the approximate solution for the system.

2.2 Optimization of variables

The procedure to use NIC technique based on variants of PSO for the optimization of residual error is elaborated in this section. PSO algorithms, introduced by Kennedy and Eberhart [40], belong to the class of NIC and have been developed using mathematical model of bird flocking and fish schooling. Reliable and efficient exploration and exploitation of candidate solutions is a well-established strength of PSO algorithm, and its convergence and stability are well-proven facts on number of benchmark problems; please see [41–45] and references therein. Numerous discrete and continuous types of the PSO algorithms have been used for optimization in many applications such as adaptive IIR system identification [46], text feature selection [47], product quality estimation [48], nonlinear parameter estimation [49] and optimization of fractional order PID controller [50].

Following the standard procedure of PSO algorithm, a candidate solution of the problem-specific objective function, known as a particle, is globally searched in the entire space. The population of these candidate solutions or particles is named as a ‘swarm’. All particles of the swarm have their own role to find a better solution of the problem-specific objective function. The particles are initialized randomly, and then, they participate in the process of optimization as a set of particle, i.e., population or swarm. Update in the position and velocity of every particle is made during each flight or iteration taking into account of their previous local P t−1Lbest and global P t−1Gbest best positions. The mathematical structure of standard PSO in terms of velocity and updated positions’ equations is written as:

$${\mathbf{v}}_{i}^{t} = w{\mathbf{v}}_{i}^{t - 1} + a_{1} {\mathbf{rand}}_{1} \left( {{\mathbf{P}}_{\text{Lbest}}^{t - 1} - {\mathbf{z}}_{i}^{t - 1} } \right) + a_{2} {\mathbf{rand}}_{2} \left( {{\mathbf{P}}_{\text{Gbest}}^{t - 1} - {\mathbf{z}}_{i}^{t - 1} } \right),$$
(5)
$${\mathbf{z}}_{i}^{t} = {\mathbf{z}}_{i}^{t - 1} + {\mathbf{v}}_{i}^{t} ,$$
(6)

where the vector z i is the ith particle of the swarm Z, i.e., Z = [z 1, z 2 ,…,z m ], and m integer denotes the total number of particles. Velocity vector (v i ) is associated with respective ith z i particle, w is the inertia weight, a 1 is a local acceleration factor, a 2 is the global acceleration constant, while rand 1 and rand 2 are random vectors with elements bounded between 0 and 1. The values of constants w, a 1 and a 2 are linearly increasing or decreasing over the course of the search between 0 and 1. The elements of velocity are limited to its minimum (MIN) and maximum (MAX) values, i.e., v i ∈ [−v max, v max].

Power of updating weights with PSO decreases considerably with the increase in the number of flights of the swarm; therefore, the real strength of this global optimizer can be seen by the process of hybridization with the local search algorithm. Nelder–Mead method (NMM) is an efficient local search algorithm incorporated for further refinement of the results by taking the global best particles of PSO as the starting point of the NMM algorithm. The flow diagram of the proposed mechanism based on PSO-NMM for optimization of residual error function is shown in Fig. 1, while the details about the intermediate steps are as follows:

  • Step 1: Initialization of PSO Initialize the particle with randomly assigned bounded real entries equal to the number of unknown variables of nonlinear systems. Mathematical notation of the particle is given as:

    $${\mathbf{z}} = [y_{1} ,y_{2} ,y_{3} , \ldots ,y_{n} ]$$

    where member of the particle represents the solution for the system. The set of particles m, i.e., swarm Z, is written as:

    $${\mathbf{Z}} = \left[ {\begin{array}{*{20}c} {{\mathbf{z}}_{1} } \\ {{\mathbf{z}}_{2} } \\ \vdots \\ {{\mathbf{z}}_{m} } \\ \end{array} } \right] = \left[ {\begin{array}{*{20}c} {y_{1} ,y_{2} ,y_{3} , \ldots ,y_{n} } \\ {y_{1} ,y_{2} ,y_{3} , \ldots ,y_{n} } \\ \vdots \\ {y_{1} ,y_{2} ,y_{3} , \ldots ,y_{n} } \\ \end{array} } \right].$$

    Sixteen variants of PSO algorithm are formulated for different values assigned to m, i.e., number of particles in the swarm and the number of flights a swarm executed. The variants of PSO algorithms are listed in Table 1. The fixed parameters initialized to each variant of PSO algorithm are listed in Table 2. These parameters are set carefully after extensive numerical experimentation to avoid premature convergence.

    Table 1 List of PSO variants hybrid with NMMs
    Full size table
    Table 2 Parameters’ settings for the optimization algorithms
    Full size table

  • Step 2: Fitness evaluation Calculate the value of residual error for each particle by using the relationship given in Eq. (3).

  • Step 3: Ranking Rank each particle through the MIN values of the residual error function, and the particle with smaller values is ranked higher and vice versa.

  • Step 4: Termination Terminate the executions of PSO variants for the following conditions:

    • Predefined fitness value is achieved, e.g., Oir ≤ 10−15

    • A MAX number of cycles or flights are executed. If above mentioned criteria are fulfilled, then proceed to step 6; otherwise, continue.

  • Step 5: Renewal Renew the velocity and position of the particles using Eqs. (5) and (6), respectively. Proceed to step 2 for the next flight of the swarm Z.

  • Step 6: Fine tuning: MATLAB built-in routine for optimization through ‘fminsearch’ function is utilized for simplex approach based on Nelder–Meads method. The refinement of the parameters is attained by taking best particle of PSO variants as initial weights, while other values of parameters are set as per values given in Table 2. The theory, underlying concepts, mathematical description and applications of NMM can be seen in [51, 52].

  • Step 7: Storage Values of the best global of PSO and PSO-NMM along with their corresponding residual error functions and time consumed are stored for the execution of the algorithm.

  • Step 8: Statistical analysis To generate a large set of data for reliable statistical analysis, repeat the steps 1–7 of the PSO-NMM procedure to form a sufficiently large number of independent executions.

3 Numerical experimentation

Results of sixteen variants of PSO and PSO-NMMs to find the solution of systems of nonlinear equations for five different problems are presented here. Comprehensive numerical experimentation for each case study is carried out to check the effectiveness of the proposed schemes in comparison with the results of statistical analysis.

3.1 Problem 1: interval arithmetic benchmark model

In this study, the performance of the design solver is examined for nonlinear system of equations of interval arithmetic benchmark (IAB) model. Mathematical expressions of nonlinear equations of the IAB model [53–56] can be written as:

$$\left[ {\begin{array}{*{20}c} {x_{1} ({\mathbf{y}})} \\ {x_{2} ({\mathbf{y}})} \\ {x_{3} ({\mathbf{y}})} \\ {x_{4} ({\mathbf{y}})} \\ {x_{5} ({\mathbf{y}})} \\ {x_{6} ({\mathbf{y}})} \\ {x_{7} ({\mathbf{y}})} \\ {x_{8} ({\mathbf{y}})} \\ {x_{9} ({\mathbf{y}})} \\ {x_{10} ({\mathbf{y}})} \\ \end{array} } \right] = {\mathbf{0}} \Rightarrow \left\{ \begin{aligned} y_{1} - 0.25428722 - 0.18324757y_{4} y_{3} y_{9} = 0, \hfill \\ y_{2} - 0.37842197 - 0.16275449y_{1} y_{10} y_{6} = 0, \hfill \\ y_{3} - 0.27162577 - 0.16955071y_{1} y_{2} y_{10} = 0, \hfill \\ y_{4} - 0.19807914 - 0.15585316y_{7} y_{1} y_{6} = 0, \hfill \\ y_{5} - 0.44166728 - 0.19950920y_{7} y_{6} y_{3} = 0, \hfill \\ y_{6} - 0.14654113 - 0.18922793y_{8} y_{5} y_{10} = 0, \hfill \\ y_{7} - 0.42937161 - 0.21180486y_{2} y_{5} y_{8} = 0, \hfill \\ y_{8} - 0.07056438 - 0.17081208y_{1} y_{7} y_{6} = 0, \hfill \\ y_{9} - 0.34504906 - 0.19612740y_{10} y_{6} y_{8} = 0, \hfill \\ y_{10} - 0.42651102 - 0.21466544y_{4} y_{8} y_{1} = 0. \hfill \\ \end{aligned} \right.$$
(7)

Here the system model (7) consists of 10 equations and the unknown vector \({\mathbf{y}} = [y_{1} ,y_{2} ,y_{3} , \ldots ,y_{n} ]\) comprises of n = 10 elements. The proposed hybrid schemes are applied to solve a nonlinear equation as per the procedure outlined in the last section, while the residual error as fitness function for the system is constructed as:

$$\begin{aligned} {\text{Oir}} & = \frac{1}{10}\sum\limits_{i = 1}^{10} {r_{i}^{2} } \\ & = \frac{1}{10}\sum\limits_{i = 1}^{10} {\left( {\begin{array}{*{20}l} {\left( {y_{1} - 0.25428722 - 0.18324757y_{4} y_{3} y_{9} } \right)^{2} +\, \left( {y_{2} - 0.37842197 - 0.16275449y_{1} y_{10} y_{6} } \right)^{2} + } \hfill \\ {\left( {y_{3} - 0.27162577 - 0.16955071y_{1} y_{2} y_{10} } \right)^{2} +\, \left( {y_{4} - 0.19807914 - 0.15585316y_{7} y_{1} y_{6} } \right)^{2} + } \hfill \\ {\left( {y_{5} - 0.44166728 - 0.19950920y_{7} y_{6} y_{3} } \right)^{2} +\, \left( {y_{6} - 0.14654113 - 0.18922793y_{8} y_{5} y_{10} } \right)^{2} + } \hfill \\ {\left( {y_{7} - 0.42937161 - 0.21180486y_{2} y_{5} y_{8} } \right)^{2} +\, \left( {y_{8} - 0.07056438 - 0.17081208y_{1} y_{7} y_{6} } \right)^{2} + } \hfill \\ {\left( {y_{9} - 0.34504906 - 0.19612740y_{10} y_{6} y_{8} } \right)^{2} +\, \left( {y_{10} - 0.42651102 - 0.21466544y_{4} y_{8} y_{1} } \right)^{2} } \hfill \\ \end{array} } \right)} . \\ \end{aligned}$$
(8)

Optimization of residual error function (8) is conducted with sixteen variants of PSO and PSO-NMMs for 100 independent executions. The approximate solutions and their absolute errors (AEs) calculated for each equation of nonlinear system (7) are shown graphically in Fig. 2 for both algorithms. Results are also calculated for statistical performance indices in terms of mean and standard deviation (STD) values and are listed in Table 3 for both PSO and PSO-NMMs. It can be seen that average residual error for PSO-1 to PSO-4, PSO-5 to PSO-8, PSO-9 to PSO-12 and PSO-13 to PSO-16 lies in the range 10−05–10−07, 10−07–10−10, 10−15–10−22, and 10−12–10−34, respectively, while for PSO-NMMs, these results are around 10−31–10−34. It is observed that the increase in number of flights from 50 to 500 or swarm size from 25 to 200 particles contributes toward the improvement of results, which is more conspicuous for PSO-1 to PSO-12. Generally, the results obtained by hybrid approaches PSO-NMM have higher precision than that of PSO variants, while comparing the hybrid approaches, no significant difference is seen in the results.

Fig. 2
figure 2

Proposed results of PSO and PSO-NMMs for interval arithmetic benchmark model presented in problem 1. a Approximate solution for PSO-1. b Absolute errors (AEs) for constituents equations of the system. c AEs for PSO-5 to PSO-8. d AEs for PSO-9 to PSO-12. e AEs for PSO-13 to PSO-16. f Approximate solution for PSO-NMM-1. g Absolute errors (AEs) for constituents equations of the system. h AEs for PSO-NMM-5 to PSO-NMM-8. i AEs for PSO-NMM-9 to PSO-NMM-12. j AEs for PSO-NMM-13 to PSO-NMM-16

Full size image
Table 3 Results of statistical performance indices for 100 independent runs of PSO and PSO-NMMs in case of problem 1
Full size table

3.2 Problem 2: chemical equilibrium application model (CEAM)

To observe the performance of proposed solver, a second case of nonlinear equation systems has been taken from CEAM [53, 54, 57]. Mathematical relations for CEAM can be written as follows:

$$\left[ {\begin{array}{*{20}c} {x_{1} ({\mathbf{y}})} \\ {x_{1} ({\mathbf{y}})} \\ {x_{1} ({\mathbf{y}})} \\ {x_{1} ({\mathbf{y}})} \\ {x_{1} ({\mathbf{y}})} \\ \end{array} } \right] = {\mathbf{0}}, \Rightarrow \left\{ {\begin{array}{*{20}l} {y_{1} y_{2} + y_{1} - 3y_{5} = 0,} \hfill \\ {2y_{1} y_{2} + y_{1} + y_{2} y_{3}^{2} + c{}_{8}y_{2} - cy_{5} + 2c_{10} y_{2}^{2} + c_{7} y_{2} y_{3} + c_{9} t_{2} t_{4} = 0,} \hfill \\ {2y_{2} y_{3}^{2} + 2c_{5} y_{3}^{2} - 8y_{5} + c_{6} y_{3} + c_{7} y_{2} y_{3} = 0,} \hfill \\ {c_{9} y_{2} y_{4} + 2y_{4}^{2} - 4cy_{5} = 0,} \hfill \\ {y_{1} (y_{2} + 1)\, + \,c_{10} y_{2}^{2} + y_{2} y_{3}^{2} + c_{8} y_{2} + c_{5} y_{3}^{2} + y_{4}^{2} - 1 + c_{6} y_{3} + c_{7} t_{2} t_{3} + c_{9} t_{2} t_{4} = 0.} \hfill \\ \end{array} } \right.$$
(9)

here the values of the constants are \(c = 10,\) \(\,c_{5} = 0.193,\) \(\,c_{6} = 0.000410,\,\) \(c_{7} = 0.000545,\) \(\,c_{8} = 0.000000449,\) \(c_{9} = 0.0000340,\) and \(c_{10} = 0.000000961\). The unknown vector, \({\mathbf{y}} = [y_{1} ,y_{2} ,y_{3} ,y_{4} ,y_{5} ]\), comprises of n = 5 elements. Similar approach, adopted in the last problem, is followed to solve the equations of the system (9), but the residual error function for this case is developed as given below:

$$\begin{aligned} {\text{Oir}} & = \frac{1}{5}\sum\limits_{i = 1}^{5} {r_{i}^{2} } \\ & = \frac{1}{5}\sum\limits_{i = 1}^{5} {\left( {\begin{array}{*{20}l} {\left( {y_{1} y_{2} + y_{1} - 3y_{5} } \right)^{2} + \left( {2y_{1} y_{2} + y_{1} + y_{2} y_{3}^{2} + c{}_{8}y_{2} - cy_{5} + 2c_{10} y_{2}^{2} + c_{7} y_{2} y_{3} + c_{9} t_{2} t_{4} } \right)^{2} + } \hfill \\ {\left( {2y_{2} y_{3}^{2} + 2c_{5} y_{3}^{2} - 8y_{5} + c_{6} y_{3} + c_{7} y_{2} y_{3} } \right)^{2} + \left( {c_{9} y_{2} y_{4} + 2y_{4}^{2} - 4cy_{5} } \right)^{2} + } \hfill \\ {\left( {y_{1} (y_{2} + 1)\, + \,c_{10} y_{2}^{2} + y_{2} y_{3}^{2} + c_{8} y_{2} + c_{5} y_{3}^{2} + y_{4}^{2} - 1 + c_{6} y_{3} + c_{7} t_{2} t_{3} + c_{9} t_{2} t_{4} } \right)^{2} } \hfill \\ \end{array} } \right)} . \\ \end{aligned}$$
(10)

Approximate solutions and their AEs calculated for each equation of the system (9) are given in Fig. 2 for both variants of PSO and PSO-NMMs, while the results for a statistical indicator of mean and STD values are listed in Table 4. It is observed that MIN values of residual error for PSO-1 to PSO-16 range from 10−04 to 10−07, while these values for PSO-NMMs range from 10−31 to 10−34. The residual error on the basis of mean values of both the algorithm is around 10−03; this is due to the fact that a single bad run can drastically affect the mean values. It is also observed that the increase in number of flights or particles in PSO does not contribute toward a decrease in the residual error for this problem (Fig. 3).

Table 4 Results of statistical performance indices for 100 independent runs of PSO and PSO-NMMs in case of problem 2
Full size table
Fig. 3
figure 3

Proposed results of PSO and PSO-NMMs for chemical equilibrium application model presented in problem 2. a Approximate solution for PSO-1. b Absolute errors (AEs) for constituents equations of the system. c AEs for PSO-5 to PSO-8. d AEs for PSO-9 to PSO-12. e AEs for PSO-13 to PSO-16. f Approximate solution for PSO-NMM-1. g Absolute errors (AEs) for constituents equations of the system. h AEs for PSO-NMM-5 to PSO-NMM-8. i AEs for PSO-NMM-9 to PSO-NMM-12. j AEs for PSO-NMM-13 to PSO-NMM-16

Full size image

3.3 Problem 3: neurophysiology application model

The performance of the proposed scheme is examined for nonlinear systems representing the neurophysiology application model (NPAM). The NPAM is written in terms of simultaneous equations as follows: [53, 58]

$$\left[ {\begin{array}{*{20}c} {x_{1} ({\mathbf{y}})} \\ {x_{2} ({\mathbf{y}})} \\ {x_{3} ({\mathbf{y}})} \\ {x_{4} ({\mathbf{y}})} \\ {x_{5} ({\mathbf{y}})} \\ {x_{6} ({\mathbf{y}})} \\ \end{array} } \right] = {\mathbf{0}}, \Rightarrow \left\{ {\begin{array}{*{20}l} {y_{1}^{2} + y_{3}^{2} - 1 = 0,} \hfill \\ {y_{2}^{2} + y_{4}^{2} - 1 = 0,} \hfill \\ {y_{5} y_{3}^{2} + y_{6} y_{4}^{2} - c_{1} = 0,} \hfill \\ {y_{5} y_{1}^{3} + y_{6} y_{2}^{3} - c_{2} = 0,} \hfill \\ {y_{5} y_{1} y_{3}^{2} + y_{6} y_{4}^{2} y_{2} - c_{3} = 0,} \hfill \\ {y_{5} y_{1}^{2} y_{3} + y_{6} y_{2}^{2} y_{4} - c_{4} = 0.} \hfill \\ \end{array} } \right.$$
(11)

Here the values of constant c 1, c 2, c 3 and c 4 are zeros. System (10) consists of six unknown and six numbers of equations; therefore, the residual error function is formulated as:

$$\begin{aligned} {\text{Oir}} & = \frac{1}{6}\sum\limits_{i = 1}^{6} {r_{i}^{2} } \\ & = \frac{1}{5}\sum\limits_{i = 1}^{5} {\left( {\begin{array}{*{20}l} {\left( {y_{1}^{2} + y_{3}^{2} - 1} \right)^{2} + \left( {y_{2}^{2} + y_{4}^{2} - 1} \right)^{2} + \left( {y_{5} y_{3}^{2} + y_{6} y_{4}^{2} - c_{1} } \right)^{2} + \left( {y_{5} y_{1}^{3} + y_{6} y_{2}^{3} - c_{2} } \right)^{2} } \hfill \\ { + \left( {y_{5} y_{1} y_{3}^{2} + y_{6} y_{4}^{2} y_{2} - c_{3} } \right)^{2} + \left( {y_{5} y_{1}^{2} y_{3} + y_{6} y_{2}^{2} y_{4} - c_{4} } \right)^{2} } \hfill \\ \end{array} } \right)} . \\ \end{aligned}$$
(12)

Optimization of error function (11) is performed with PSO and PSO-NMMs, and results are listed in Table 5 in terms of statistical operator of the mean and standard deviation, while the approximate solutions and their AEs for the constituting equations are plotted in Fig. 4. It is seen that average residual error for PSO-1 to PSO-4, PSO-5 to PSO-8, PSO-9 to PSO-12 and PSO-13 to PSO-16 lies in the range 10−06–10−07, 10−07–10−09, 10−09–10−11 and 10−10–10−11, respectively, while for PSO-NMMs, these results are around 10−21–10−32. Additionally, desired minimum values of residual errors are achieved for each PSO-NMM, while these results are only obtained from PSO-13 to PSO-16. It is observed that highly accurate results are determined consistently with hybrid algorithms for this problem.

Table 5 Results of statistical performance indices for 100 independent runs of PSO and PSO-NMMs in case of problem 3
Full size table
Fig. 4
figure 4

Proposed results of PSO and PSO-NMMs for neurophysiology application model presented in problem 3. a Approximate solution for PSO-1. b Absolute errors (AEs) for constituents equations of the system. c AEs for PSO-5 to PSO-8. d AEs for PSO-9 to PSO-12. e AEs for PSO-13 to PSO-16. f Approximate solution for PSO-NMM-1 to PSO-NMM-4. g Approximate solution for PSO-NMM-5 to PSO-NMM-8. h Approximate solution for PSO-NMM-9 to PSO-NMM-12. i Approximate solution for PSO-NMM-13 to PSO-NMM-16

Full size image

3.4 Problem 4: combustion theory application model

In this case, performance of the proposed solver is evaluated for the well-known system of nonlinear equations representing combustion process with temperature around 3000 °C [53, 59]. Governing mathematical expressions for this problem are given below:

$$\left[ {\begin{array}{*{20}c} {x_{1} ({\mathbf{y}})} \\ {x_{2} ({\mathbf{y}})} \\ {x_{3} ({\mathbf{y}})} \\ {x_{4} ({\mathbf{y}})} \\ {x_{5} ({\mathbf{y}})} \\ {x_{6} ({\mathbf{y}})} \\ {x_{7} ({\mathbf{y}})} \\ {x_{8} ({\mathbf{y}})} \\ {x_{9} ({\mathbf{y}})} \\ {x_{10} ({\mathbf{y}})} \\ \end{array} } \right] = {\mathbf{0}} \Rightarrow \left\{ {\begin{array}{*{20}l} {y_{2} + 2y_{6} + y_{9} + 2y_{10} - 10^{ - 5} = 0,} \hfill \\ {y_{3} + y_{8} - 3 \times 10^{ - 5} = 0,} \hfill \\ {y_{1} + y_{3} + 2y_{5} + 2y_{8} + y_{9} + y_{10} - 5 \times 10^{ - 5} = 0,} \hfill \\ {y_{4} + 2y_{7} - 10^{ - 5} = 0,} \hfill \\ {0.5140437\, \times 10^{ - 7} y_{5} - y_{1}^{2} = 0,} \hfill \\ {0.1006932 \times 10^{ - 6} y_{6} - 2y_{2}^{2} = 0,} \hfill \\ {0.7816278 \times 10^{ - 15} y_{7} - y_{4}^{2} = 0,} \hfill \\ {0.1496236 \times 10^{ - 6} y_{8} - y_{1} y_{3} = 0,} \hfill \\ {0.6194411 \times 10^{ - 7} y_{9} - y_{1} y_{2} = 0,} \hfill \\ {0.2089296 \times 10^{ - 14} y_{10} - y_{1} y_{2}^{2} = 0.} \hfill \\ \end{array} } \right.$$
(13)

Similar approach, as followed in the previous cases, is adopted to solve system (13) by constructing a residual error function for the number of unknown n = 10 as:

$$\begin{aligned} Oir & = \frac{1}{10}\sum\limits_{i = 1}^{10} {r_{i}^{2} } \\ & = \frac{1}{10}\sum\limits_{i = 1}^{10} {\left( {\begin{array}{*{20}l} {\left( {y_{2} + 2y_{6} + y_{9} + 2y_{10} - 10^{ - 5} } \right)^{2} + \left( {y_{3} + y_{8} - 3 \times 10^{ - 5} } \right)^{2} + } \hfill \\ {\left( {y_{1} + y_{3} + 2y_{5} + 2y_{8} + y_{9} + y_{10} - 5 \times 10^{ - 5} } \right)^{2} + \left( {y_{4} + 2y_{7} - 10^{ - 5} } \right)^{2} + } \hfill \\ {\left( {0.5140437 \times 10^{ - 7} y_{5} - y_{1}^{2} } \right)^{2} + \left( {0.1006932 \times 10^{ - 6} y_{6} - 2y_{2}^{2} } \right)^{2} + } \hfill \\ {\left( {0.7816278 \times 10^{ - 15} y_{7} - y_{4}^{2} } \right)^{2} + \left( {0.1496236 \times 10^{ - 6} y_{8} - y_{1} y_{3} } \right)^{2} + } \hfill \\ {\left( {0.6194411 \times 10^{ - 7} y_{9} - y_{1} y_{2} } \right)^{2} + \left( {0.2089296 \times 10^{ - 14} y_{10} - y_{1} y_{2}^{2} } \right)^{2} } \hfill \\ \end{array} } \right)} . \\ \end{aligned}$$
(14)

Optimization of residual error function (13) is conducted, and approximate solutions are plotted graphically in Fig. 5 for variants of PSO and PSO-NMMs. The values of mean and STD based on 100 independent runs of the algorithms are listed in Table 6. It can be seen that average residual error for PSO-1 to PSO-4, PSO-5 to PSO-8, PSO-9 to PSO-12 and PSO-13 to PSO-16 ranges from 10−04 to 10−05, 10−05–10−06, 10−07–10−08 and 10−07–10−09, respectively, while for PSO-NMMs these results are around 10−17. Minimum values of PSO and PSO-NMMs range from 10−05 to 10−12 and 10−33–10−35, respectively. The dominance of hybrid schemes is generally observed for this case study as well.

Fig. 5
figure 5

Proposed results of PSO and PSO-NMMs for combustion theory application model presented in problem 4. a Approximate solution for PSO-1. b Absolute errors (AEs) for constituents equations of the system. c AEs for PSO-5 to PSO-8. d AEs for PSO-9 to PSO-12. e AEs for PSO-13 to PSO-16. f Approximate solution for PSO-NMM-1. g Absolute errors (AEs) for constituents equations of the system. h AEs for PSO-NMM-5 to PSO-NMM-8. i AEs for PSO-NMM-9 to PSO-NMM-12. j AEs for PSO-NMM-13 to PSO-NMM-16

Full size image
Table 6 Results of statistical performance indices for 100 independent runs of PSO and PSO-NMMs in case of problem 4
Full size table

3.5 Problem 5: economics modeling system

The econometric model of arbitrary dimensions is studied in this case [53, 59]. The problem in term of mathematical equations is stated as:

$$\left\{ {\begin{array}{*{20}l} {\left( {y_{k} + \sum\limits_{i = 1}^{n - k - 1} {y_{i} y_{i} + k} } \right)y_{n} - c_{k} = 0;\quad 1 \le k \le n - 1} \hfill \\ {\sum\limits_{i = 1}^{n - 1} {y_{i} + 1 = 0.} } \hfill \\ \end{array} } \right.$$
(15)

Here the value of constant c k is chosen randomly. Proposed schemes are examined in a special case of the problem (15) by taking the values of n = 5 and k = [1–4] as:

$$\left[ {\begin{array}{*{20}c} {x_{1} ({\mathbf{y}})} \\ {x_{2} ({\mathbf{y}})} \\ {x_{3} ({\mathbf{y}})} \\ {x_{4} ({\mathbf{y}})} \\ {x_{5} ({\mathbf{y}})} \\ \end{array} } \right] = {\mathbf{0}}, \Rightarrow \left\{ {\begin{array}{*{20}l} {y_{1} + y_{2} + y_{3} + y_{4} + 1 = 0,} \hfill \\ {y_{1} y_{5} + y_{1} y_{2} y_{5} + y_{2} y_{3} y_{5} + y_{3} y_{4} y_{5} - 1 = 0,} \hfill \\ {y_{2} y_{5} + y_{1} y_{3} y_{5} + y_{2} y_{4} y_{5} - 1 = 0,} \hfill \\ {y_{3} y_{5} + y_{1} y_{4} y_{5} - 1 = 0,} \hfill \\ {y_{4} y_{5} - 1\, = 0.} \hfill \\ \end{array} } \right.$$
(16)

The residual error function for the system (16) is developed as:

$$\begin{aligned} {\text{Oir}} & = \frac{1}{5}\sum\limits_{i = 1}^{5} {r_{i}^{2} } \\ & = \frac{1}{5}\sum\limits_{i = 1}^{5} {\left( \begin{aligned} \left( {y_{1} + y_{2} + y_{3} + y_{4} + 1} \right)^{2} + \left( {y_{1} y_{5} + y_{1} y_{2} y_{5} + y_{2} y_{3} y_{5} + y_{3} y_{4} y_{5} - 1} \right)^{2} + \hfill \\ \left( {\,y_{2} y_{5} + y_{1} y_{3} y_{5} + y_{2} y_{4} y_{5} - 1} \right)^{2} + \left( {y_{3} y_{5} + y_{1} y_{4} y_{5} - 1} \right)^{2} + \left( {y_{4} y_{5} - 1} \right)^{2} \hfill \\ \end{aligned} \right)} . \\ \end{aligned}$$
(17)

Results of 16 variants of the proposed adaptive schemes are listed in Table 7 for the values of mean and STD calculated for 100 independent runs of the algorithms. Additionally, approximate solutions of the system (16) are shown in Fig. 6. It can be seen that average residual error for PSO-1 to PSO-4, PSO-5 to PSO-8, PSO-9 to PSO-12 and PSO-13 to PSO-16 ranges from 10−04 to 10−07, 10−07–10−12, 10−17–10−27 and 10−29–10−33, respectively, while for PSO-NMMs, these results are around 10−32.

Table 7 Results of statistical performance indices for 100 independent runs of PSO and PSO-NMMs in case of problem 5
Full size table
Fig. 6
figure 6

Proposed results of PSO and PSO-NMMs for economics modeling system presented in problem 5. a Approximate solution for PSO-1. b Absolute errors (AEs) for constituents equations of the system. c AEs for PSO-5 to PSO-8. d AEs for PSO-9 to PSO-12. e AEs for PSO-13 to PSO-16. f Approximate solution for PSO-NMM-1. g Absolute errors (AEs) for constituents equations of the system. h AEs for PSO-NMM-5 to PSO-NMM-8. i AEs for PSO-NMM-9 to PSO-NMM-12. j AEs for PSO-NMM-13 to PSO-NMM-16

Full size image

4 Comparative studies

In this section, comparison between variants of PSO and PSO-NMM is presented through results of statistical analysis based on 100 independent runs for all five nonlinear systems.

The accuracy of the proposed schemes is examined for 100 independent runs of algorithms. An independent run of the algorithms is defined as a run with a randomly generated initial swarm of the particles with different seeds. Results on the basis of value of fitness, i.e., the values of Oir, for a number of independent runs are plotted in case of PSO and PSO-NMMs in Fig. 7a–h, respectively, for problem 1. These plots are given on semilog scale and fitness, sorted in ascending manner, in order to clearly highlight small variations in the results. On the similar pattern, results for the problems 2, 3, 4 and 5 are shown in Figs. 8, 9, 10 and 11, respectively, for each variation. It can be seen that generally for all five problems, with the increase in the number of particles in the swarm from 25 to 200 for the fixed cycles of the algorithm, the results of the PSO variants improve, which is more evident in case of PSO-1 to PSO-4 for 50 cycles, PSO-5 to PSO-8 for 100 cycles and PSO-9 to PSO-12 for 150 cycles. However, for the cases PSO-13 to PSO-16 for 500 cycles, no noticeable change is observed in the results. Whereas in cases of hybrid PSO-NMM approach by taking the best particle of PSO as the starting point of NMM, precision enhanced considerably. Generally, results of PSO-NMMs are improved as compared to respective PSO variants and this improvement is more apparent in the case of PSO-NMM-1 to PSO-NMM-12 and for the cases for which PSO algorithm achieved fitness around 10−30; however, no significant contribution of hybridization with local search is observed.

Fig. 7
figure 7

Plot of sorted fitness, Oir, values for 100 independent runs of algorithms in the case of the interval arithmetic benchmark model presented in problem 1. a–h Are for PSO and PSO-NMMs, respectively

Full size image
Fig. 8
figure 8

Plot of sorted fitness, Oir, values for 100 independent runs of algorithms in the case of the chemical equilibrium application model presented in problem 2. a–h Are for PSO and PSO-NMMs, respectively

Full size image
Fig. 9
figure 9

Plot of sorted fitness, Oir, values for 100 independent runs of algorithms in the case of the neurophysiology application model presented in problem 3. a–h Are for PSO and PSO-NMMs, respectively

Full size image
Fig. 10
figure 10

Plot of sorted fitness, Oir, values for 100 independent runs of algorithms in the case of combustion theory application model presented in problem 4. a–h Are for PSO and PSO-NMMs, respectively

Full size image
Fig. 11
figure 11

Plot of sorted fitness, Oir, values for 100 independent runs of algorithms in the case of economics modeling system presented in problem 5. a–h Are for PSO and PSO-NMMs, respectively

Full size image

Next accuracy and convergence are examined for 100 independent runs of PSO and PSO-NMMs achieving different levels of accuracy. Results of convergence analyses, based on different criterion, are listed in Table 6 for variants of PSO for all five problems, while for variants of PSO-NMMs, these results are given in Table 7. It can be seen that criteria based on fitness values around 10−07–10−09, almost 100 % of the runs for all hybrid approaches PSO-NMM-1 to PSO-NMM-16 and algorithms PSO-13 to PSO-16, have fulfilled the criteria, while PSO-1 to PSO-12 gave relatively degraded convergence rate. Generally, in all five cases, the rate of convergence of the hybrid PSO-NMMs approaches is relatively better than that of PSO variants. Additionally, it can be seen that the performance of the proposed algorithms based on both PSO and PSO-NMMs is relatively superior for the third problem as compared to the other four problems and more accurate results are obtained for second problem as compared to the remaining problems (Tables 8, 9).

Table 8 Comparison of results on the basis of convergence analysis for 100 independent runs of PSO variants
Full size table
Table 9 Comparison of results on the basis of convergence analysis for 100 independent runs of PSO-NMMs
Full size table

Complexity analysis of the variants of PSO and PSO-NMMs is performed on the basis of average time consumed by the algorithm to find the approximate solution of a given nonlinear system. The values of mean execution time along with STD for 100 independent runs of each solver for all five problems are listed in Table 10. It is seen from the results presented that with the increase in the number of particles of the swarm or number of flights, the values of mean execution time also increase. The values of complexity operator for PSO variants are relatively smaller than that of PSO-NMMs in case of all five case studies, i.e., hybrid approaches take 3–4 times more time as compared to the execution time of PSO variants. The optimization process of the proposed algorithms relatively takes longer execution time in case of problem 2 as compared to other problems. Simulations are carried out in the present study on ACER Laptop Model V3-471G with Core i5-3210 M 2.6 GHz processor with turbo boost up to 3.1 GHz, 4.00 GB DDR3 memory and running MATLAB software package version 2012a in a Microsoft Windows 7 operating environment.

Table 10 Complexity analysis based on average execution time consumed by the algorithms during optimization process for PSO and PSO-NMMs
Full size table

The true comparison among between stochastic and deterministic numerical methods is not possible due to their inherent different optimization procedures; however, in order to establish the effectiveness of proposed design algorithms, a generic comparison among standard solvers is given here. The solution of all five benchmark studies are determined by using ‘fsolve’ routine, a built-in function of MATLAB optimization toolbox for solving nonlinear system of equations with known initial start point based on trust-region-dogleg, trust-region-reflective and Levenberg–Marquardt algorithms. Default setting of the parameters of all three algorithms is used for finding the solution. It is found that the results obtained by all three algorithms lie in the range of 10−03–10−09 for arithmetic benchmark model, chemical equilibrium, neurophysiology, combustion theory and economics models, whereas the proposed methods based on PSO hybrid with NMM achieved the accuracy of order 10−30. The complexity of the deterministic solver is superior from stochastic methods, which is understandable due to the stochastic solver based on complex global search procedures. However, this aspect of stochastic numerical scheme is overshadowed due to consistently accurate results without known initial bias guess.

5 Conclusions

On the basis of simulation performed in this study, following conclusions are drawn:

  1. 1.

    Sixteen variants PSO-1 to PSO-16 of swarm intelligence algorithms hybridized with Nelder–Mead method PSO-NMM-1 to PSO-NMM-16 are developed effectively to solve the nonlinear system of equations by taking different scenarios based on the number of particles in the swarm and the flights of the swarm.

  2. 2.

    Validation of the performance of the proposed schemes based on 100 independent runs is established through consistently getting the small values of statistically operators in terms of mean and standard deviation in case of all five nonlinear systems.

  3. 3.

    The accuracy and convergence of the given schemes are further evaluated through calculation of percentage convergent runs based on different precision levels, and the results show that hybrid approaches are almost 100 % convergent on the basis of fitness ≤10−03 in case of all five nonlinear systems. For stiff criteria, i.e., fitness ≤10−09, problems 1, 3 and 5 are still around 100 % convergent, while for problems 2 and 5, convergence rates degraded but remained in close vicinity of 85 %.

  4. 4.

    Computational complexity in terms of mean and standard deviation values shows that variants of PSO with more number of particles in the swarm or number of flights of the swarm have long execution time and vice versa, whereas the complexity of the hybrid computing approaches PSO-NMMs is always on the higher side than that of PSO variants, but this limitation can be afforded due to their better performance on accuracy and convergence from the rest.

One may explore to apply the design variants of PSO and PSO-NMMs to solve stiff nonlinear systems, differential–algebraic system, ordinary, partial, fractional differential equations and their systems to different physical science application which still remained unsolved by conventional or classical techniques. Recently introduced fractional variants of PSO [60–64] look as a promising area to be exploited to study the dynamics of nonlinear systems. Additionally, the better performance in terms of accuracy and convergence may achieved by exploring neural network model for solving these benchmark systems.