Abstract
In the present study, nature-inspired computing technique has been designed for the solution of nonlinear systems by exploiting the strength of particle swarm optimization (PSO) hybrid with Nelder–Mead method (NMM). Fitness function based on least square approximation theory is developed for the systems, while optimization of the design variables is performed with PSO, an efficient global search method, refined with NMM for rapid local convergence. Sixteen variants of the proposed hybrid scheme PSO-NMM have been evaluated on five benchmark nonlinear systems, namely interval arithmetic benchmark model, kinematic application model, neurophysiology problem, combustion model and chemical equilibrium system. Reliability and effectiveness of the proposed solver have been validated after comparison with the results of statistical analysis based on massive data generated for sufficiently large number of independent executions.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
1 Introduction
Strength of artificial intelligence algorithms (AIAs) has extensively been exploited to optimize the difficult nonlinear problems arising in a variety of fields [1–3]. Especially in computational and applied mathematics, AIAs based on unsupervised feed-forward artificial neural networks (ANNs) trained with global and local search methodologies have been applied to solve many linear and nonlinear systems. Few potential applications of AIA using ANNs, evolutionary and swarming intelligence, are highly stiff oscillatory systems based on Van der Pol-type nonlinear differential equations [4], mathematical model for fuel ignition model in combustion theory through one-dimensional Bratu-type equations [5], transformed third-order BVP of nonlinear two-dimensional Bratu-type equations [6], the maximum flow problem [7], similarity transformed system for thin film flow of third-grade fluids [8], nonlinear Lane–Emden-type equations [9], mathematical model for confinement of plasma represented with Troesch-type problems [10], magnetohydrodynamics (MHD) problem of fluid flow governed with Jeffery–Hamel-type equations [11], nonlinear Emden–Fowler type singular system [12], nonlinear first Painlevé-type equations [13, 14], nonlinear systems based on initial value problems [15], functional ordinary differential equations of pantograph types [16], fractional differential equations of nonlinear Riccati types [17], nanotechnology problems based on multi-wall carbon nanotubes [18] and fluid mechanic problem based on steady thin film flow of Johnson–Segalman fluid on vertical cylinder [19].
A number of numerical solvers have been proposed by the research community for solving problems based on nonlinear system of equations [20–22], but one of the simplest, oldest and, most commonly used numerical procedures is Newton–Raphson method (NRM) [23, 24], but its performance is sensitive to the initial weights as it is highly prone to getting stuck in the local minimum. Therefore, in order to utilize NRM algorithm effectively for the solution of system of nonlinear equations, it is supplied with biased initial guess by another global search solver. Besides NRM, many other iterative and recursive solvers have been introduced and used in applied mathematics, literature due to their unique advantages, limitations and applicability. For instance, Kelley, Campbell and Broyden’s provided numerical schemes for the solution of nonlinear equations and their systems [25–27]. Jacobian-free Newton–Krylov method has been used for deriving the lower triangular half of the sparse Jacobian matrix through automatic differentiation [28, 29]. Recently, Jafari and Gejji, Abbasbandy, Sharma and Vahidi have provided new solver [30–36] to effectively determine the solution of problems based on nonlinear equations. Most of the solutions available in the existing literature are based on deterministic procedure using iterative and recursive solvers [37–39], but well-established strength of the stochastic solvers, based on AIAs, has not been extensively explored for the solution of nonlinear equations. The aim of this study is to investigate the potential of AIA algorithms based on NIC technique for providing an alternate, accurate, convergent and efficient system for solving system of nonlinear equations.
In this study, NIC technique based on swarm intelligence aided with NMM is applied for solving nonlinear systems. Sixteen variants of particle swarm optimization (PSO) algorithm have been developed and applied to optimize the variables for five well-known nonlinear systems. The design parameters of these systems are further tuned with NMM for rapid local convergence. The salient features of the proposed methodologies are given briefly as:
-
Novel design of nature-inspired heuristic algorithms based on exploration and exploitation in standard PSO method hybrid with NMM to solve nonlinear systems.
-
Performance of the proposed schemes is evaluated on five benchmark models based on interval arithmetic benchmark, kinematic application, neurophysiology, combustion and chemical equilibrium models.
-
Reliability and effectiveness of the schemes are analyzed through the results of statistical analyses based on sufficiently large number of independent runs.
-
Computational complexity indices are used to measure the comparative efficiency.
-
Simple concept, effortlessness in implementations, consistent accuracy, steady convergence and wider applicability domain are the significant illustrative perks of the schemes.
The rest of the paper is organized as follows: In section two, design methodology for solving nonlinear equations and their systems is presented, i.e., formulation of the objective function and learning methodologies used for training. In section three, results of simulation are presented and compared with the studies based on statistical performance indices for accuracy and complexity. Conclusions and potential future research directions are submitted in the last section.
2 Methodology
Design scheme presented here for finding the solution of systems of nonlinear equations consists of two parts: In the first part, formulation of objective function based on the residual error in absolute or the mean square sense is formulated, while in the second part, details of PSO and its variants are outlined. The graphical abstract of the proposed methodology is shown in Fig. 1.
2.1 Fitness function for systems nonlinear of equations
General form of system of nonlinear equations is written as:
Equation (1) can be written in the expanded form as:
To determine the solution for a system (1) with n number of unknowns and n number of equations, objective function based on the residual error in mean absolute or square sense can be written as [37, 38]:
where (r 1, r 2, …, r n ) are the residual error functions associated with (x1, x 1, …, x n ) equations, respectively, and can be written as:
Now objective is to determine the variables of system (1), such that the residual error function (3) \({\text{Oir}} \to 0\), which is only possible when each of r = [r 1, r 2,…,r n ] → 0, consequently \({\mathbf{X}}({\mathbf{y}}) \to {\mathbf{0}}\), i.e., values of the vector \({\mathbf{y}} = [y_{1} ,y_{2} ,y_{3} , \ldots ,y_{n} ]\) will be the approximate solution for the system.
2.2 Optimization of variables
The procedure to use NIC technique based on variants of PSO for the optimization of residual error is elaborated in this section. PSO algorithms, introduced by Kennedy and Eberhart [40], belong to the class of NIC and have been developed using mathematical model of bird flocking and fish schooling. Reliable and efficient exploration and exploitation of candidate solutions is a well-established strength of PSO algorithm, and its convergence and stability are well-proven facts on number of benchmark problems; please see [41–45] and references therein. Numerous discrete and continuous types of the PSO algorithms have been used for optimization in many applications such as adaptive IIR system identification [46], text feature selection [47], product quality estimation [48], nonlinear parameter estimation [49] and optimization of fractional order PID controller [50].
Following the standard procedure of PSO algorithm, a candidate solution of the problem-specific objective function, known as a particle, is globally searched in the entire space. The population of these candidate solutions or particles is named as a ‘swarm’. All particles of the swarm have their own role to find a better solution of the problem-specific objective function. The particles are initialized randomly, and then, they participate in the process of optimization as a set of particle, i.e., population or swarm. Update in the position and velocity of every particle is made during each flight or iteration taking into account of their previous local P t−1Lbest and global P t−1Gbest best positions. The mathematical structure of standard PSO in terms of velocity and updated positions’ equations is written as:
where the vector z i is the ith particle of the swarm Z, i.e., Z = [z 1, z 2 ,…,z m ], and m integer denotes the total number of particles. Velocity vector (v i ) is associated with respective ith z i particle, w is the inertia weight, a 1 is a local acceleration factor, a 2 is the global acceleration constant, while rand 1 and rand 2 are random vectors with elements bounded between 0 and 1. The values of constants w, a 1 and a 2 are linearly increasing or decreasing over the course of the search between 0 and 1. The elements of velocity are limited to its minimum (MIN) and maximum (MAX) values, i.e., v i ∈ [−v max, v max].
Power of updating weights with PSO decreases considerably with the increase in the number of flights of the swarm; therefore, the real strength of this global optimizer can be seen by the process of hybridization with the local search algorithm. Nelder–Mead method (NMM) is an efficient local search algorithm incorporated for further refinement of the results by taking the global best particles of PSO as the starting point of the NMM algorithm. The flow diagram of the proposed mechanism based on PSO-NMM for optimization of residual error function is shown in Fig. 1, while the details about the intermediate steps are as follows:
-
Step 1: Initialization of PSO Initialize the particle with randomly assigned bounded real entries equal to the number of unknown variables of nonlinear systems. Mathematical notation of the particle is given as:
$${\mathbf{z}} = [y_{1} ,y_{2} ,y_{3} , \ldots ,y_{n} ]$$where member of the particle represents the solution for the system. The set of particles m, i.e., swarm Z, is written as:
$${\mathbf{Z}} = \left[ {\begin{array}{*{20}c} {{\mathbf{z}}_{1} } \\ {{\mathbf{z}}_{2} } \\ \vdots \\ {{\mathbf{z}}_{m} } \\ \end{array} } \right] = \left[ {\begin{array}{*{20}c} {y_{1} ,y_{2} ,y_{3} , \ldots ,y_{n} } \\ {y_{1} ,y_{2} ,y_{3} , \ldots ,y_{n} } \\ \vdots \\ {y_{1} ,y_{2} ,y_{3} , \ldots ,y_{n} } \\ \end{array} } \right].$$Sixteen variants of PSO algorithm are formulated for different values assigned to m, i.e., number of particles in the swarm and the number of flights a swarm executed. The variants of PSO algorithms are listed in Table 1. The fixed parameters initialized to each variant of PSO algorithm are listed in Table 2. These parameters are set carefully after extensive numerical experimentation to avoid premature convergence.
-
Step 2: Fitness evaluation Calculate the value of residual error for each particle by using the relationship given in Eq. (3).
-
Step 3: Ranking Rank each particle through the MIN values of the residual error function, and the particle with smaller values is ranked higher and vice versa.
-
Step 4: Termination Terminate the executions of PSO variants for the following conditions:
-
Predefined fitness value is achieved, e.g., Oir ≤ 10−15
-
A MAX number of cycles or flights are executed. If above mentioned criteria are fulfilled, then proceed to step 6; otherwise, continue.
-
-
Step 5: Renewal Renew the velocity and position of the particles using Eqs. (5) and (6), respectively. Proceed to step 2 for the next flight of the swarm Z.
-
Step 6: Fine tuning: MATLAB built-in routine for optimization through ‘fminsearch’ function is utilized for simplex approach based on Nelder–Meads method. The refinement of the parameters is attained by taking best particle of PSO variants as initial weights, while other values of parameters are set as per values given in Table 2. The theory, underlying concepts, mathematical description and applications of NMM can be seen in [51, 52].
-
Step 7: Storage Values of the best global of PSO and PSO-NMM along with their corresponding residual error functions and time consumed are stored for the execution of the algorithm.
-
Step 8: Statistical analysis To generate a large set of data for reliable statistical analysis, repeat the steps 1–7 of the PSO-NMM procedure to form a sufficiently large number of independent executions.
3 Numerical experimentation
Results of sixteen variants of PSO and PSO-NMMs to find the solution of systems of nonlinear equations for five different problems are presented here. Comprehensive numerical experimentation for each case study is carried out to check the effectiveness of the proposed schemes in comparison with the results of statistical analysis.
3.1 Problem 1: interval arithmetic benchmark model
In this study, the performance of the design solver is examined for nonlinear system of equations of interval arithmetic benchmark (IAB) model. Mathematical expressions of nonlinear equations of the IAB model [53–56] can be written as:
Here the system model (7) consists of 10 equations and the unknown vector \({\mathbf{y}} = [y_{1} ,y_{2} ,y_{3} , \ldots ,y_{n} ]\) comprises of n = 10 elements. The proposed hybrid schemes are applied to solve a nonlinear equation as per the procedure outlined in the last section, while the residual error as fitness function for the system is constructed as:
Optimization of residual error function (8) is conducted with sixteen variants of PSO and PSO-NMMs for 100 independent executions. The approximate solutions and their absolute errors (AEs) calculated for each equation of nonlinear system (7) are shown graphically in Fig. 2 for both algorithms. Results are also calculated for statistical performance indices in terms of mean and standard deviation (STD) values and are listed in Table 3 for both PSO and PSO-NMMs. It can be seen that average residual error for PSO-1 to PSO-4, PSO-5 to PSO-8, PSO-9 to PSO-12 and PSO-13 to PSO-16 lies in the range 10−05–10−07, 10−07–10−10, 10−15–10−22, and 10−12–10−34, respectively, while for PSO-NMMs, these results are around 10−31–10−34. It is observed that the increase in number of flights from 50 to 500 or swarm size from 25 to 200 particles contributes toward the improvement of results, which is more conspicuous for PSO-1 to PSO-12. Generally, the results obtained by hybrid approaches PSO-NMM have higher precision than that of PSO variants, while comparing the hybrid approaches, no significant difference is seen in the results.
3.2 Problem 2: chemical equilibrium application model (CEAM)
To observe the performance of proposed solver, a second case of nonlinear equation systems has been taken from CEAM [53, 54, 57]. Mathematical relations for CEAM can be written as follows:
here the values of the constants are \(c = 10,\) \(\,c_{5} = 0.193,\) \(\,c_{6} = 0.000410,\,\) \(c_{7} = 0.000545,\) \(\,c_{8} = 0.000000449,\) \(c_{9} = 0.0000340,\) and \(c_{10} = 0.000000961\). The unknown vector, \({\mathbf{y}} = [y_{1} ,y_{2} ,y_{3} ,y_{4} ,y_{5} ]\), comprises of n = 5 elements. Similar approach, adopted in the last problem, is followed to solve the equations of the system (9), but the residual error function for this case is developed as given below:
Approximate solutions and their AEs calculated for each equation of the system (9) are given in Fig. 2 for both variants of PSO and PSO-NMMs, while the results for a statistical indicator of mean and STD values are listed in Table 4. It is observed that MIN values of residual error for PSO-1 to PSO-16 range from 10−04 to 10−07, while these values for PSO-NMMs range from 10−31 to 10−34. The residual error on the basis of mean values of both the algorithm is around 10−03; this is due to the fact that a single bad run can drastically affect the mean values. It is also observed that the increase in number of flights or particles in PSO does not contribute toward a decrease in the residual error for this problem (Fig. 3).
3.3 Problem 3: neurophysiology application model
The performance of the proposed scheme is examined for nonlinear systems representing the neurophysiology application model (NPAM). The NPAM is written in terms of simultaneous equations as follows: [53, 58]
Here the values of constant c 1, c 2, c 3 and c 4 are zeros. System (10) consists of six unknown and six numbers of equations; therefore, the residual error function is formulated as:
Optimization of error function (11) is performed with PSO and PSO-NMMs, and results are listed in Table 5 in terms of statistical operator of the mean and standard deviation, while the approximate solutions and their AEs for the constituting equations are plotted in Fig. 4. It is seen that average residual error for PSO-1 to PSO-4, PSO-5 to PSO-8, PSO-9 to PSO-12 and PSO-13 to PSO-16 lies in the range 10−06–10−07, 10−07–10−09, 10−09–10−11 and 10−10–10−11, respectively, while for PSO-NMMs, these results are around 10−21–10−32. Additionally, desired minimum values of residual errors are achieved for each PSO-NMM, while these results are only obtained from PSO-13 to PSO-16. It is observed that highly accurate results are determined consistently with hybrid algorithms for this problem.
3.4 Problem 4: combustion theory application model
In this case, performance of the proposed solver is evaluated for the well-known system of nonlinear equations representing combustion process with temperature around 3000 °C [53, 59]. Governing mathematical expressions for this problem are given below:
Similar approach, as followed in the previous cases, is adopted to solve system (13) by constructing a residual error function for the number of unknown n = 10 as:
Optimization of residual error function (13) is conducted, and approximate solutions are plotted graphically in Fig. 5 for variants of PSO and PSO-NMMs. The values of mean and STD based on 100 independent runs of the algorithms are listed in Table 6. It can be seen that average residual error for PSO-1 to PSO-4, PSO-5 to PSO-8, PSO-9 to PSO-12 and PSO-13 to PSO-16 ranges from 10−04 to 10−05, 10−05–10−06, 10−07–10−08 and 10−07–10−09, respectively, while for PSO-NMMs these results are around 10−17. Minimum values of PSO and PSO-NMMs range from 10−05 to 10−12 and 10−33–10−35, respectively. The dominance of hybrid schemes is generally observed for this case study as well.
3.5 Problem 5: economics modeling system
The econometric model of arbitrary dimensions is studied in this case [53, 59]. The problem in term of mathematical equations is stated as:
Here the value of constant c k is chosen randomly. Proposed schemes are examined in a special case of the problem (15) by taking the values of n = 5 and k = [1–4] as:
The residual error function for the system (16) is developed as:
Results of 16 variants of the proposed adaptive schemes are listed in Table 7 for the values of mean and STD calculated for 100 independent runs of the algorithms. Additionally, approximate solutions of the system (16) are shown in Fig. 6. It can be seen that average residual error for PSO-1 to PSO-4, PSO-5 to PSO-8, PSO-9 to PSO-12 and PSO-13 to PSO-16 ranges from 10−04 to 10−07, 10−07–10−12, 10−17–10−27 and 10−29–10−33, respectively, while for PSO-NMMs, these results are around 10−32.
4 Comparative studies
In this section, comparison between variants of PSO and PSO-NMM is presented through results of statistical analysis based on 100 independent runs for all five nonlinear systems.
The accuracy of the proposed schemes is examined for 100 independent runs of algorithms. An independent run of the algorithms is defined as a run with a randomly generated initial swarm of the particles with different seeds. Results on the basis of value of fitness, i.e., the values of Oir, for a number of independent runs are plotted in case of PSO and PSO-NMMs in Fig. 7a–h, respectively, for problem 1. These plots are given on semilog scale and fitness, sorted in ascending manner, in order to clearly highlight small variations in the results. On the similar pattern, results for the problems 2, 3, 4 and 5 are shown in Figs. 8, 9, 10 and 11, respectively, for each variation. It can be seen that generally for all five problems, with the increase in the number of particles in the swarm from 25 to 200 for the fixed cycles of the algorithm, the results of the PSO variants improve, which is more evident in case of PSO-1 to PSO-4 for 50 cycles, PSO-5 to PSO-8 for 100 cycles and PSO-9 to PSO-12 for 150 cycles. However, for the cases PSO-13 to PSO-16 for 500 cycles, no noticeable change is observed in the results. Whereas in cases of hybrid PSO-NMM approach by taking the best particle of PSO as the starting point of NMM, precision enhanced considerably. Generally, results of PSO-NMMs are improved as compared to respective PSO variants and this improvement is more apparent in the case of PSO-NMM-1 to PSO-NMM-12 and for the cases for which PSO algorithm achieved fitness around 10−30; however, no significant contribution of hybridization with local search is observed.
Next accuracy and convergence are examined for 100 independent runs of PSO and PSO-NMMs achieving different levels of accuracy. Results of convergence analyses, based on different criterion, are listed in Table 6 for variants of PSO for all five problems, while for variants of PSO-NMMs, these results are given in Table 7. It can be seen that criteria based on fitness values around 10−07–10−09, almost 100 % of the runs for all hybrid approaches PSO-NMM-1 to PSO-NMM-16 and algorithms PSO-13 to PSO-16, have fulfilled the criteria, while PSO-1 to PSO-12 gave relatively degraded convergence rate. Generally, in all five cases, the rate of convergence of the hybrid PSO-NMMs approaches is relatively better than that of PSO variants. Additionally, it can be seen that the performance of the proposed algorithms based on both PSO and PSO-NMMs is relatively superior for the third problem as compared to the other four problems and more accurate results are obtained for second problem as compared to the remaining problems (Tables 8, 9).
Complexity analysis of the variants of PSO and PSO-NMMs is performed on the basis of average time consumed by the algorithm to find the approximate solution of a given nonlinear system. The values of mean execution time along with STD for 100 independent runs of each solver for all five problems are listed in Table 10. It is seen from the results presented that with the increase in the number of particles of the swarm or number of flights, the values of mean execution time also increase. The values of complexity operator for PSO variants are relatively smaller than that of PSO-NMMs in case of all five case studies, i.e., hybrid approaches take 3–4 times more time as compared to the execution time of PSO variants. The optimization process of the proposed algorithms relatively takes longer execution time in case of problem 2 as compared to other problems. Simulations are carried out in the present study on ACER Laptop Model V3-471G with Core i5-3210 M 2.6 GHz processor with turbo boost up to 3.1 GHz, 4.00 GB DDR3 memory and running MATLAB software package version 2012a in a Microsoft Windows 7 operating environment.
The true comparison among between stochastic and deterministic numerical methods is not possible due to their inherent different optimization procedures; however, in order to establish the effectiveness of proposed design algorithms, a generic comparison among standard solvers is given here. The solution of all five benchmark studies are determined by using ‘fsolve’ routine, a built-in function of MATLAB optimization toolbox for solving nonlinear system of equations with known initial start point based on trust-region-dogleg, trust-region-reflective and Levenberg–Marquardt algorithms. Default setting of the parameters of all three algorithms is used for finding the solution. It is found that the results obtained by all three algorithms lie in the range of 10−03–10−09 for arithmetic benchmark model, chemical equilibrium, neurophysiology, combustion theory and economics models, whereas the proposed methods based on PSO hybrid with NMM achieved the accuracy of order 10−30. The complexity of the deterministic solver is superior from stochastic methods, which is understandable due to the stochastic solver based on complex global search procedures. However, this aspect of stochastic numerical scheme is overshadowed due to consistently accurate results without known initial bias guess.
5 Conclusions
On the basis of simulation performed in this study, following conclusions are drawn:
-
1.
Sixteen variants PSO-1 to PSO-16 of swarm intelligence algorithms hybridized with Nelder–Mead method PSO-NMM-1 to PSO-NMM-16 are developed effectively to solve the nonlinear system of equations by taking different scenarios based on the number of particles in the swarm and the flights of the swarm.
-
2.
Validation of the performance of the proposed schemes based on 100 independent runs is established through consistently getting the small values of statistically operators in terms of mean and standard deviation in case of all five nonlinear systems.
-
3.
The accuracy and convergence of the given schemes are further evaluated through calculation of percentage convergent runs based on different precision levels, and the results show that hybrid approaches are almost 100 % convergent on the basis of fitness ≤10−03 in case of all five nonlinear systems. For stiff criteria, i.e., fitness ≤10−09, problems 1, 3 and 5 are still around 100 % convergent, while for problems 2 and 5, convergence rates degraded but remained in close vicinity of 85 %.
-
4.
Computational complexity in terms of mean and standard deviation values shows that variants of PSO with more number of particles in the swarm or number of flights of the swarm have long execution time and vice versa, whereas the complexity of the hybrid computing approaches PSO-NMMs is always on the higher side than that of PSO variants, but this limitation can be afforded due to their better performance on accuracy and convergence from the rest.
One may explore to apply the design variants of PSO and PSO-NMMs to solve stiff nonlinear systems, differential–algebraic system, ordinary, partial, fractional differential equations and their systems to different physical science application which still remained unsolved by conventional or classical techniques. Recently introduced fractional variants of PSO [60–64] look as a promising area to be exploited to study the dynamics of nonlinear systems. Additionally, the better performance in terms of accuracy and convergence may achieved by exploring neural network model for solving these benchmark systems.
References
Nazemi AR (2013) Solving general convex nonlinear optimization problems by an efficient neurodynamic model. Eng Appl Artif Intell 26:685–696
Nazemi AR (2014) A neural network model for solving convex quadratic programming problems with some applications. Eng Appl Artif Intell 32:54–62
Nazemi AR, Nazemi M (2014) A gradient-based neural network method for solving strictly convex quadratic programming problems. Cogn Comput 6:484–495
Khan JA, Raja MAZ, Syam MI, Tanoli SAK, Awan SE (2015) Design and application of nature inspired computing approach for non-linear stiff oscillatory problems. Neural Comput Appl 26(7):1763–1780. doi:10.1007/s00521-015-1841-z
Raja MAZ (2014) Solution of the one-dimensional Bratu equation arising in the fuel ignition model using ANN optimised with PSO and SQP. Connect Sci 26(3):195–214. doi:10.1080/09540091.2014.907555
Raja MAZ, Ahmad SUI, Samar R (2014) Solution of the 2-dimensional Bratu problem using neural network, swarm intelligence and sequential quadratic programming. Neural Comput Appl 25:1723–1739. doi:10.1007/s00521-014-1664-3
Nazemi AR, Omidi F (2012) A capable neural network model for solving the maximum flow problem. J Comput Appl Math 236:3498–3513
Raja MAZ, Khan JA, Haroon T (2014) Stochastic numerical treatment for thin film flow of third grade fluid using unsupervised neural networks. J Chem Inst Taiwan 48:26–39. doi:10.1016/j.jtice.2014.10.018
Mall S, Chakraverty S (2014) Chebyshev neural network based model for solving Lane–Emden type equations. Appl Math Comput 247:100–114
Raja MAZ (2014) Stochastic numerical techniques for solving Troesch’s Problem. Inf Sci 279:860–873. doi:10.1016/j.ins.2014.04.036
Raja MAZ, Samar R, Haroon T, Shah SM (2015) Unsupervised neural network model optimized with evolutionary computations for solving variants of nonlinear MHD Jeffery–Hamel problem. Appl Math Mech 36(12):1611–1638
Mall Susmita, Chakraverty S (2015) Numerical solution of nonlinear singular initial value problems of Emden–Fowler type using Chebyshev Neural Network method. Neurocomputing 149:975–982
Raja MAZ, Khan JA, Behloul D, Haroon T, Siddiqui AM, Samar R (2015) Exactly satisfying initial conditions neural network models for numerical treatment of first Painlevé equation. Appl Soft Comput 26:244–256. doi:10.1016/j.asoc.2014.10.009
Raja MAZ, Khan JA, Shah SM, Bhahoal D, Samar R (2015) Comparison of three unsupervised neural network models for first Painlevé Transcendent. Neural Comput Appl 26(5):1055–1071. doi:10.1007/s00521-014-1774-y
Mall S, Chakraverty S (2015) Multi layer versus functional link single layer neural network for solving nonlinear singular initial value problems. In: Proceedings of the third international symposium on women in computing and informatics (pp 678–683). ACM
Raja MAZ (2014) Numerical treatment for boundary value problems of pantograph functional differential equation using computational intelligence algorithms. Appl Soft Comput 24:806–821. doi:10.1016/j.asoc.2014.08.055
Raja MAZ, Manzar MA, Samar R (2015) An efficient computational intelligence approach for solving fractional order Riccati equations using ANN and SQP”. Appl Math Model 39(10):3075–3093. doi:10.1016/j.apm.2014.11.024
Raja MAZ, Farooq U, Chaudhary NI, Wazwaz AM (2016) Stochastic numerical solver for nanofluidic problems containing multi-walled carbon nanotubes. Appl Soft Comput 38:561–586
Raja MAZ, Shah FH, Khan AA, Khan NA (2016) Design of bio-inspired computational intelligence technique for solving steady thin film flow of Johnson-Segalman fluid on vertical cylinder for drainage problems. J Taiwan Inst Chem Eng 60:59–75
Li Min (2014) A Polak–Ribière–Polyak method for solving large-scale nonlinear systems of equations and its global convergence. Appl Math Comput 248:314–322
Amat S, Hernández-Verón MA, Rubio MJ (2014) Improving the applicability of the secant method to solve nonlinear systems of equations. Appl Math Comput 247:741–752
Sharma JR, Arora H, Petković MS (2014) An efficient derivative free family of fourth order methods for solving systems of nonlinear equations. Appl Math Comput 235:383–393
Ortega JM, Rheinboldt WC (1970) Iterative solution of nonlinear equations in several variables, vol 30. Siam, Philadelphia
Kelley CT (2003) Solving nonlinear equations with Newton’s method, vol 1. Siam, Philadelphia
Kelley CT (1999) Iterative methods for optimization, vol 18. Siam, Philadelphia
Campbell SL et al (1996) GMRES and the minimal polynomial. BIT Numer Math 36(4):664–675
Darvishi MT, Barati A (2007) A fourth order method from quadrature formulae to solve the system of nonlinear equations. Appl Math Comput 188:257–261
Knoll DA, Keyes DE (2004) Jacobian free Newton–Krylov methods: a survey of approaches and applications. J Comput Phys 193:357–397
Saad Y, van der Vorst HA (2000) Iterative solution of non linear system in the 20th century. J Comput Appl Math 123:1–33
Jafari H, Gejji VD (2006) Revised Adomian decomposition method for solving a system of nonlinear equation. J Appl Math Comput 175:1–7
Abbasbandy S (2005) Extended Newton method for a system on non linear equations by modified domain decomposition method. J Appl Math Comput 170:648–656
Vahidi AR, Javadi S, Khorasani SM (2012) Solving system of nonlinear equations by restarted Adomain’s method. J Appl Math Comput 6:509–516
Sharma JR, Guha RK, Gupta P (2013) Improved King’s methods with optimal order of convergence based on rational approximations. Appl Math Lett 26(4):473–480
Sharma JR, Gupta P (2014) An efficient fifth order method for solving systems of nonlinear equations. Comput Math Appl 67(3):591–601
Sharma JR, Gupta P (2013) On some efficient techniques for solving systems of nonlinear equations. Adv Numer Anal 2013:11, Art ID 252798. doi:10.1155/2013/252798
Sharma JR, Arora H (2014) An efficient family of weighted-Newton methods with optimal eighth order convergence. Appl Math Lett 29:1–6
Arqub OA, Al-Smadi M, Momani S, Hayat T (2015) Numerical solutions of fuzzy differential equations using reproducing kernel Hilbert space method. Soft Comput. doi:10.1007/s00500-015-1707-4
Abu O (2015) Arqub, adaptation of reproducing kernel algorithm for solving fuzzy Fredholm–Volterra integrodifferential equations. Neural Comput Appl. doi:10.1007/s00521-015-2110-x
Arqub OA, Abo-Hammour Z (2014) Numerical solution of systems of second-order boundary value problems using continuous genetic algorithm. Inf Sci 279:396–415
Kennedy J, Eberhart R (1995) Particle swarm optimization. In: Proceedings of IEEE international conference on neural networks, Perth, Australia, vol 4. IEEE Service Center, Piscataway, pp 1942–1948
Trelea IC (2003) The particle swarm optimization algorithm: convergence analysis and parameter selection. Inf Process Lett 85(6):317–325
Ping Tian D (2013) A review of convergence analysis of particle swarm optimization. Int J Grid Distrib Comput 6(6):117–128
Schmitt M, Wanka R (2015) Particle swarm optimization almost surely finds local optima. Theor Comput Sci 561:57–72
García-Gonzalo E, Fernández-Martínez JL (2012) A brief historical review of particle swarm optimization (PSO). J Bioinform Intell Control 1(1):3–16
Zhang Y, Wang S, Ji G (2015) A comprehensive survey on particle swarm optimization algorithm and its applications. Math Probl Eng 2015:38, Art ID 931256. doi:10.1155/2015/931256
Jiang S, Wang Y, Ji Z (2015) A new design method for adaptive IIR system identification using hybrid particle swarm optimization and gravitational search algorithm. Nonlinear Dyn 79(4):2553–2576
Lu Y et al (2015) Improved particle swarm optimization algorithm and its application in text feature selection. Appl Soft Comput 35:629–636
Saeid S, Sadeghi MT, Marvas MA (2014) High reliability estimation of product quality using support vector regression and hybrid meta-heuristic algorithms. J Taiwan Inst Chem Eng 45:2225–2232
Prawin J, Rao ARM, Lakshmi K (2016) Nonlinear parametric identification strategy combining reverse path and hybrid dynamic quantum particle swarm optimization. Nonlinear Dyn 84(2):797–815
Liu X (2016) Optimization design on fractional order PID controller based on adaptive particle swarm optimization algorithm. Nonlinear Dyn 84(1):379–386
Nelder JA, Mead R (1965) A simplex method for function minimization. Comput J 7(4):308–313
Dennis JE, Woods DJ (1987) Optimization on microcomputers: the Nelder-Mead simplex algorithm. In: Wouk A (ed) New computing environments: microcomputers in large-scale computing. SIAM, Philadelphia, pp 116–122
Grosan Crina, Abraham Ajith (2008) A new approach for solving nonlinear equations systems. IEEE Trans Syst Man Cybern Part A Syst Hum 38(3):698–714
Grosan Crina, Abraham Ajith (2008) Multiple solutions for a system of nonlinear equations. Int J Innov Comput Inf Control 4(9):2161–2170
Van Hentenryck P, McAllester D, Kapur D (1997) Solving polynomial systems using a branch and prune approach. SIAM J Numer Anal 34(2):797–827
Hong H, Stahl V (1994) Safe starting regions by fixed points and tightening. Computing 53(3/4):323–335
Meintjes K, Morgan AP (1990) Chemical equilibrium systems as numerical test problems. ACM Trans Math Softw 16(2):143–151
Verschelde J, Verlinden P, Cools R (1994) Homotopies exploiting Newton polytopes for solving sparse polynomial systems. SIAM J Numer Anal 31(3):915–930
Morgan AP (1987) Solving polynomial system using continuation for scientific and engineering problems. Prentice-Hall, Englewood Cliffs
Pires ES, Machado JT, de Moura Oliveira PB, Cunha JB, Mendes L (2010) Particle swarm optimization with fractional-order velocity. Nonlinear Dyn 61(1–2):295–301
Couceiro MS, Rocha RP, Ferreira NF, Machado JT (2012) Introducing the fractional-order Darwinian PSO. Signal Image Video Process 6(3):343–350
Couceiro MS, Martins FM, Rocha RP, Ferreira NM, Sivasundaram S (2012) Introducing the fractional order robotic Darwinian PSO. In: AIP conference proceedings, American institute of physics, vol 1493, no 1, p 242
Pires EJS, Machado JAT, de Moura Oliveira PB (2013) Entropy diversity in multi-objective particle swarm optimization. Entropy 15(12):5475–5491
Couceiro M, Ghamisi P (2016) Fractional-order Darwinian PSO. In: Merkle D (ed) Fractional order Darwinian particle swarm optimization. Springer, Berlin, pp 11–20
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Raja, M.A.Z., Zameer, A., Kiani, A.K. et al. Nature-inspired computational intelligence integration with Nelder–Mead method to solve nonlinear benchmark models. Neural Comput & Applic 29, 1169–1193 (2018). https://doi.org/10.1007/s00521-016-2523-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00521-016-2523-1
Keywords
- Nonlinear system of equations
- Hybrid computing
- Particle swarm optimization
- Nelder–Mead method
- Nature-inspired computing
- Benchmark models