Abstract
A possible alternative to topology fine-tuning for Neural Network (NN) optimization is to use Echo State Networks (ESNs), recurrent NNs built upon a large reservoir of sparsely randomly connected neurons. The promises of ESNs have been fulfilled for supervised learning tasks, but unsupervised ones, e.g. control problems, require more flexible optimization methods – such as Evolutionary Algorithms. This paper proposes to apply CMA-ES, the state-of-the-art method in evolutionary continuous parameter optimization, to the evolutionary learning of ESN parameters. First, a standard supervised learning problem is used to validate the approach and compare it to the standard one. But the flexibility of Evolutionary optimization allows us to optimize not only the outgoing weights but also, or alternatively, other ESN parameters, sometimes leading to improved results. The classical double pole balancing control problem is then used to demonstrate the feasibility of evolutionary (i.e. reinforcement) learning of ESNs. We show that the evolutionary ESN obtain results that are comparable with those of the best topology-learning methods.
Access provided by Autonomous University of Puebla. Download to read the full chapter text
Chapter PDF
Similar content being viewed by others
References
Auger, A., Hansen, N.: Performance evaluation of an advanced local search evolutionary algorithm. In: Proc. CEC 2005 (2005)
Banzhaf, W.: Artificial Regulatory Networks and Genetic Programming. In: Riolo, R., Worzel, B. (eds.) Genetic Programming Theory and Practice, ch. 4, pp. 43–62. Kluwer Academic Publishers, Dordrecht (2003)
Devert, A., Bredeche, N., Schoenauer, M.: Robust multi-cellular developmental design. In: GECCO 2007: Proceedings of the 9th annual conference on Genetic and evolutionary computation, pp. 982–989. ACM Press, New York (2007)
Dürr, P., Mattiussi, C., Floreano, D.: Neuroevolution with Analog Genetic Encoding. In: Runarsson, T.P., Beyer, H.-G., Burke, E.K., Merelo-Guervós, J.J., Whitley, L.D., Yao, X. (eds.) PPSN 2006. LNCS, vol. 4193, pp. 671–680. Springer, Heidelberg (2006)
Gomez, F.J., Miikkulainen, R.: Solving non-markovian control tasks with neuro-evolution. In: IJCAI, pp. 1356–1361 (1999)
Gruau, F., Whitley, D., Pyeatt, L.: A comparison between cellular encoding and direct encoding for genetic neural networks. In: Koza, J.R., et al. (eds.) Proc. GP 1996, pp. 28–31. MIT Press, Cambridge (1996)
Hansen, N., Kern, S.: Evaluating the CMA evolution strategy on multimodal test functions. In: Yao, X., et al. (eds.) PPSN 2004. LNCS, vol. 3242, pp. 282–291. Springer, Heidelberg (2004)
Hansen, N., Ostermeier, A.: Adapting arbitrary normal mutation distributions in evolutionstrategies: the covariance matrix adaptation. In: Proc. CEC 1996, pp. 312–317. IEEE Press, Los Alamitos (1996)
Hansen, N., Ostermeier, A.: Completely derandomized self-adaptation in evolution strategies. Evolutionary Computation 9(2), 159–195 (2001)
Igel, C.: Neuroevolution for reinforcement learning using evolution strategies. In: Proc. CEC 2003, pp. 2588–2595. IEEE Press, Los Alamitos (2003)
Ishu, K., van der Zant, T., Becanovic, V., Ploger, P.: Identification of motion with echo state network. In: Proc. OCEANS 2004. MTTS/IEEE TECHNO-OCEAN 2004, vol. 3, pp. 1205–1210 (2004)
Jaeger, H.: The Echo State Approach to Analysing and Training Recurrent Neural Networks. Technical Report GMD Report 148, German National Research Center for Information Technology (2001)
Jaeger, H.: Tutorial on training recurrent neural networks. Technical report, GMD Report 159, Fraunhofer Institute AIS (2002)
Jaeger, H., Haas, H., Principe, J.C. (eds.): NIPS 2006 Workshop on Echo State Networks and Liquid State Machines (2006)
Schmidhuber, J., Wierstra, D., Gagliolo, M., Gomez, F.: Training recurrent networks by evolino. Neural Comput. 19(3), 757–779 (2007)
Stanley, K.: Compositional Pattern Producing Networks: A Novel Abstraction of Development. Genetic Programming and Evolvable Machines 8(2), 131–162 (2007)
Stanley, K.O., Miikkulainen, R.: Efficient reinforcement learning through evolving neural network topologies. In: Langdon, W.B., et al. (eds.) Proc. GECCO 2002, pp. 569–577. Morgan Kaufmann, San Francisco (2002)
Xu, D., Lan, J., Principe, J.: Direct adaptive control: an echo state network and genetic algorithm approach. In: Proc. IEEE International Joint Conference on Neural Networks IJCNN 2005, 31 July–4 August 2005, vol. 3, pp. 1483–1486 (2005)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Jiang, F., Berry, H., Schoenauer, M. (2008). Supervised and Evolutionary Learning of Echo State Networks. In: Rudolph, G., Jansen, T., Beume, N., Lucas, S., Poloni, C. (eds) Parallel Problem Solving from Nature – PPSN X. PPSN 2008. Lecture Notes in Computer Science, vol 5199. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-87700-4_22
Download citation
DOI: https://doi.org/10.1007/978-3-540-87700-4_22
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-87699-1
Online ISBN: 978-3-540-87700-4
eBook Packages: Computer ScienceComputer Science (R0)