Abstract
Nature inspired meta-heuristic algorithms provide derivative-free solution to optimize complex problems. Cuckoo Search (CS) algorithm is one of the most modern addition to the group of nature inspired optimization meta-heuristics. The Simple Recurrent Networks (SRN) were initially trained by Elman with the standard back propagation (SBP) learning algorithm which is less capable and often takes enormous amount of time to train a network of even a moderate size. And the complex error surface of the SBP makes many training algorithms are prone to being trapped in local minima. This paper proposed a new meta-heuristic based Cuckoo Search Back Propagation Recurrent Neural Network (CSBPRNN) algorithm. The CSBPRNN is based on Cuckoo Search to train BPRNN in order to achieve fast convergence rate and to avoid local minima problem. The performance of the proposed CSBPRNN is compared with Artificial Bee Colony using BP algorithm, and other hybrid variants. Specifically OR and XOR datasets are used. The simulation results show that the computational efficiency of BP training process is highly enhanced when coupled with the proposed hybrid method.
Access provided by Autonomous University of Puebla. Download to read the full chapter text
Chapter PDF
Similar content being viewed by others
Keywords
References:
Elman, J. L.: Finding structure in time, J. Cognitive Science, Vol.14 (2), pp. 179–211, (1990)
Barbounis, T.G., Theocharis, J.B., et al., Long-term wind speed and power forecasting using local recurrent neural network models, J. IEEE Transactions on Energy Conversion, vol. 21,(1,) pp. 273–284, (2006)
Goedtel, A., Dasilva, I.N., and Amaral Semi, P.J., Recurrent Neural Network for Induction Motor Speed Estimation in Industry Application, In: IEEE MELECON, pp. 1134–1137, August (2006)
Xiao, P., Venayagamoorthy, G. K., and Corzine, K. A.: Combined Training of Recurrent Neural Networks with Particle Swarm Optimization and Back propagation Algorithms for Impedance Identification, In: Proceedings of the IEEE Swarm Intelligence Symposium (2007)
Hecht-Nielsen, R.: Neurocomputing, Addison-Wesley, Reading, PA, (1990)
Giles, C. L., Lawrence, S., and Tsoi, A. C.: Rule inference for financial prediction using recurrent neural networks, In: IEEE Conference on Computational Intelligence for Financial Engineering, IEEE Press. (1997)
Li, S., Wunsch II, D. C., O’Hair, E., and Giesselmann, M. G.: Wind turbine power estimation by neural networks with Kalman filter training on a SIMD parallel machine, In: International Joint Conference on Neural Networks, (1999)
Coulibay, P., Anctil, F., and Rousselle, J.: Real-time short-term water inflows forecasting using recurrent neural networks, In: International Joint Conference on Neural Networks, (1999)
Gudise, V.G. and Venayagamoorthy, G.K.: Comparison of particle swarm optimization and backpropagation as training algorithms for neural networks, In: IEEE Swarm Intelligence Symposium, pp. 110–117. April (2003)
Janson, D.J. and Frenzel, J.F.: Training product unit neural networks with genetic algorithms, J. IEEE Intelligent Systems and Their Applications, vol. 8 (5), pp. 26–33, Oct. (1993)
Salerno, J., Using the particle swarm optimization technique to train a recurrent neural model, In: Ninth IEEE International Conference on Tools with Artificial Intelligence, pp. 45–49, Nov. (1997)
Werbos, P.: Back propagation through time: what it does and how to do it, In: Proceedings of the IEEE, vol. 78 (10), pp. 1550–1560. (1990)
Temurtas, F., Yumusak, N., Gunturkun, R., Temurtas, H., and Cerezci, O.: Elman’s Recurrent Neural Networks Using Resilient Back Propagation for Harmonic Detection, In: 8th Pacific Rim International Conference on Artificial Intelligent Proceedings, vol. 3157, pp. 422–428. (2004)
Rumelhart D.E. Hinton G.E., and Williams R.J.: Learning Representations by Back– Propagating Errors Nature, vol. 323, pp. 533-536. (1986)
Nawi, N. M., Rehman, M. Z., and Khan, A.: A New Bat-Based Back-propagation (BAT-BP) Algorithm, In: ICSS 2013, Wrocław, Poland, September 10-12. (2013)
Nawi, N. M., Khan, A., and Rehman, M. Z.: A New Back-propagation Neural Network optimized with Cuckoo Search Algorithm, In: ICCSA-2013, pp. 413–426. (2013)
Nawi, N. M., Khan, A., and Rehman, M. Z.: A New Cuckoo Search based Levenberg-Marquardt (CSLM) Algorithm, In: ICCSA-2013, pp. 438–451. (2013)
Nawi, N. M., Khan, A., and Rehman, M. Z.: A New Levenberg-Marquardt based Back-propagation Algorithm trained with Cuckoo Search, In: ICEEI-2013, pp.18–24. (2013)
Lahmiri, S.: A comparative study of backpropagation algorithms in financial prediction. J. International Journal of Computer Science, Engineering and Applications (IJCSEA), vol.1 (4). (2011)
Nawi, N. M., Ransing, R. S., Salleh, M.N.M., Ghazali,R., and Hamid, N.A, An improved back propagation neural network algorithm on classification problems, J. Communications in Computer and Information Science vol. 118, pp. 177–188. (2010)
Nawi, N.M., Ghazali, R., Salleh, M.N.M. The development of improved back-propagation neural networks algorithm for predicting patients with heart disease, J. LNCS, vol. 6377 (4), pp. 317–324. (2010)
Yang. XS., and Deb. S.: Cuckoo search via Lévy flights, In: World Congress on Nature & Biologically Inspired Computing, India, pp. 210–214. (2009)
Yang, X. S., and Deb, S.: Engineering Optimisation by Cuckoo Search, Int. J. of Mathematical Modelling and Numerical Optimisation, vol. 1 (4), pp. 330– 343. (2010)
Tuba, M., Subotic, M., Stanarevic, N.: Modified cuckoo search algorithm for unconstrained optimization problems, In: European Computing Conference, pp. 263–268. (2011)
Tuba, M., Subotic, M., Stanarevic, N.: Performance of a Modified Cuckoo Search Algorithm for Unconstrained Optimization Problems, J. Faculty of Computer Science, vol. 11 (2), pp. 62–74. (2012)
Chaowanawate, K., and Heednacram, A.: Implementation of Cuckoo Search in RBF Neural Network for Flood Forecasting, In: Fourth International Conference on Computational Intelligence, Communication Systems and Networks, pp. 22–26. (2012)
Pavlyukevich, I.: Levy flights, non-local search and simulated annealing, J. Journal of Computational Physics, vol. 226 (2), pp. 1830–1844. (2007)
Walton, S., Hassan, O., Morgan, K., and Brown, M.: Modified cuckoo search: A new gradient free optimisation algorithm. J. Chaos, Solitons& Fractals, vol. 44 (9), pp. 710–718. (2011)
Williams, R. J. and Peng, J.: An efficient gradient-based algorithm for on-line training ofrecurrent network trajectories, J. Neural Computation, vol.2, pp. 490–501. (1990)
Nandy, S., Sarkar, P. P., and Das, A.: Training a Feed-forward Neural Network with Artificial Bee Colony Based Backpropagation Method, J. International Journal of Computer Science & Information Technology (IJCSIT), vol. 4 (4), pp. 33–46. (2012)
Ozturk, C., and Karaboga, D.: Hybrid Artificial Bee Colony algorithm for neural network training, In: IEEE Congress of Evolutionary Computation (CEC), pp. 84–88. (2011)
Karaboga, D., and Ozturk, C..: Neural networks training by artificial bee colony algorithm on pattern classification, J. Neural Network World, vol. 19, no. 10, pp. 279–292, (2009).
Acknowledgments
The Authors would like to thank Office of Research, Innovation, Commercialization and Consultancy Office (ORICC), Universiti Tun Hussein Onn Malaysia (UTHM) and Ministry of Higher Education (MOHE) Malaysia for financially supporting this Research under Fundamental Research Grant Scheme (FRGS) vote no. 1236.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer Science+Business Media Singapore
About this paper
Cite this paper
Nawi, N.M., khan, A., Rehman, M.Z. (2014). CSBPRNN: A New Hybridization Technique Using Cuckoo Search to Train Back Propagation Recurrent Neural Network. In: Herawan, T., Deris, M., Abawajy, J. (eds) Proceedings of the First International Conference on Advanced Data and Information Engineering (DaEng-2013). Lecture Notes in Electrical Engineering, vol 285. Springer, Singapore. https://doi.org/10.1007/978-981-4585-18-7_13
Download citation
DOI: https://doi.org/10.1007/978-981-4585-18-7_13
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-4585-17-0
Online ISBN: 978-981-4585-18-7
eBook Packages: EngineeringEngineering (R0)