Skip to main content
SpringerLink
Log in

Neuroevolution

  • Living reference work entry
  • Latest version View entry history
  • First Online:
Encyclopedia of Machine Learning and Data Science

  • 109 Accesses

Abstract

Neuroevolution is a method for modifying aspects of neural network design in order to learn a specific task. Evolutionary computation is used to discover designs that maximize a fitness function that measures performance in the task. Compared to other neural network learning methods, neuroevolution is highly general, allowing learning without explicit targets and modifying both differentiable and nondifferentiable aspects of the design, such as the architecture, weights, activation and loss functions, and learning algorithms. Neuroevolution thus serves three roles: First, it is a policy search method for reinforcement learning problems, where it is well suited to continuous domains and to domains where the state is only partially observable. Second, it is an automatic method for discovering effective deep learning architectures. Third, especially when combined with learning in individual networks, it is a model of biological adaptation.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

Similar content being viewed by others

Recommended Reading

  • Ackley D, Littman M (1992) Interactions between learning and evolution. In: Langton CG, Taylor C, Farmer JD, Rasmussen S (eds) Artificial life II. Addison-Wesley, Reading, pp 487–509

    Google Scholar 

  • Angeline PJ, Saunders GM, Pollack JB (1994) An evolutionary algorithm that constructs recurrent neural networks. IEEE Trans Neural Netw 5:54–65

    Article  Google Scholar 

  • Bingham G, Macke W, Miikkulainen R (2020) Evolutionary optimization of deep learning activation functions. In: Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2020), Cancun

    Google Scholar 

  • Bongard J (2011) Morphological change in machines accelerates the evolution of robust behavior. Proc Natl Acad Sci U S A 108:1234–1239

    Article  Google Scholar 

  • Bryant BD, Miikkulainen R (2007) Acquiring visibly intelligent behavior with example-guided neuroevolution. In: Proceedings of the Twenty-Second National Conference on Artificial Intelligence. AAAI, Menlo Park

    Google Scholar 

  • Chellapilla K, Fogel DB (1999) Evolution, neural networks, games, and intelligence. Proc IEEE 87:1471–1496

    Article  Google Scholar 

  • Clune J, Mouret J-B, Lipson H (2013) The evolutionary origins of modularity. Proc R Soc B Biol Sci 280(1755):20122863

    Article  Google Scholar 

  • Elsken T, Metzen JH, Hutter F (2019) Neural architecture search: a survey. J Mach Learn Res 20:1–21

    MathSciNet  MATH  Google Scholar 

  • Floreano D, Dürr P, Mattiussi C (2008) Neuroevolution: from architectures to learning. Evol Intell 1:47–62

    Article  Google Scholar 

  • Francon O, Gonzalez S, Hodjat B, Meyerson E, Miikkulainen R, Qiu X, Shahrzad H (2020) Effective reinforcement learning through evolutionary surrogate-assisted prescription. In: Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2020), Cancun

    Google Scholar 

  • Gomez F, Miikkulainen R (2003) Active guidance for a finless rocket using neuroevolution. In: Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2003), Chicago

    Google Scholar 

  • Gomez F, Schmidhuber J, Miikkulainen R (2008) Accelerated neural evolution through cooperatively coevolved synapses. J Mach Learn Res 9:937–965

    MathSciNet  MATH  Google Scholar 

  • Gonzalez S, Miikkulainen R (2020) Improved training speed, accuracy, and data utilization via loss function optimization. In: Proceedings of the 2020 IEEE Congress on Evolutionary Computation (CEC-2020), Glasgow

    Google Scholar 

  • Gruau F, Whitley D (1993) Adding learning to the cellular development of neural networks: evolution and the Baldwin effect. Evol Comput 1:213–233

    Article  Google Scholar 

  • Igel C (2003) Neuroevolution for reinforcement learning using evolution strategies. In: Sarker R, Reynolds R, Abbass H, Tan KC, McKay B, Essam D, Gedeon T (eds) Proceedings of the 2003 Congress on Evolutionary Computation. IEEE, Piscataway, pp 2588–2595

    Google Scholar 

  • Keinan A, Sandbank B, Hilgetag CC, Meilijson I, Ruppin E (2006) Axiomatic scalable neurocontroller analysis via the Shapley value. Artif Life 12:333–352

    Article  Google Scholar 

  • Lehman J, Stanley KO (2010) Abandoning objectives: evolution through the search for novelty alone. Evol Comput 2011:189–223

    Google Scholar 

  • Lessin D, Fussell D, Miikkulainen R (2013) Open-ended behavioral complexity for evolved virtual creatures. In: Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2013), Amsterdam

    Google Scholar 

  • Li A, Spyra O, Perel S, Dalibard V, Jaderberg M, Gu C, Budden D, Harley T, Gupta P (2019) A generalized framework for population based training. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp 1791–1799

    Google Scholar 

  • Liang J, Meyerson E, Miikkulainen R (2018) Evolutionary architecture search for deep multitask networks. In: Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2018), Kyoto

    Google Scholar 

  • Liang J, Meyerson E, Fink D, Mutch K, Miikkulainen R (2019) Evolutionary neural AutoML for deep learning. In: Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2019), Prague

    Google Scholar 

  • Liang J, Gonzalez S, Miikkulainen R (2020) Population-based training for loss function optimization. arXiv:2002.04225

    Google Scholar 

  • Liu Y, Yao X, Higuchi T (2000) Evolutionary ensembles with negative correlation learning. IEEE Trans Evol Comput 4:380–387

    Article  Google Scholar 

  • Loshchilov I, Hutter F (2016) CMA-ES for hyperparameter optimization of deep neural networks. In: International Conference on Learning Representations (ICLR-2016) Workshop Track

    Google Scholar 

  • Meyerson E, Miikkulainen R (2018) Pseudo-task augmentation: from deep multitask learning to intratask sharing—and back. In: Proceedings of the 35th International Conference on Machine Learning (ICML-2018), Stockholm

    Google Scholar 

  • Meyerson E, Miikkulainen R (2019) Modular universal reparameterization: deep multi-task learning across diverse domains. In: Proceedings of the 33rd Annual Conference on Neural Information Processing Systems (NeurIPS-2019), Vancouver

    Google Scholar 

  • Miikkulainen R (2019) Creative AI through evolutionary computation. In: Banzhaf W et al (eds) Evolution in action: past, present and future. Springer, New York

    Google Scholar 

  • Miikkulainen R, Bryant BD, Cornelius R, Karpov IV, Stanley KO, Yong CH (2006) Computational intelligence in games. In: Yen GY, Fogel DB (eds) Computational intelligence: principles and practice, Piscataway. IEEE Computational Intelligence Society

    Google Scholar 

  • Miikkulainen R, Meyerson E, Liang J, Rawal A, Shahrzad H, Fink D, Francon O, Raju B, Navruzyan A, Hodjat B, Duffy N (2018) Evolving deep neural networks. In: Morabito CF, Alippi C, Choe Y, Kozma R (eds) Artificial intelligence in the age of neural networks and brain computing. Elsevier, New York

    Google Scholar 

  • Moriarty DE, Schultz AC, Grefenstette JJ (1999) Evolutionary algorithms for reinforcement learning. J Artif Intell Res 11:199–229

    Article  MATH  Google Scholar 

  • Mouret J-B, Doncieux S (2012) Encouraging behavioral diversity in evolutionary robotics: an empirical study. Evol Comput 20:91–133

    Article  Google Scholar 

  • Nolfi S, Floreano D (2000) Evolutionary robotics. MIT Press, Cambridge

    Google Scholar 

  • Potter MA, Jong KAD (2000) Cooperative coevolution: an architecture for evolving coadapted subcomponents. Evol Comput 8:1–29

    Article  Google Scholar 

  • Rajagopalan P, Holekamp KE, Miikkulainen R (2020) Evolution of complex coordinated behavior. In: Proceedings of the 2020 IEEE Congress on Evolutionary Computation (CEC-2020), Glasgow

    Google Scholar 

  • Rawal A, Liang J, Miikkulainen R (2020) Discovering gated recurrent neural network architectures. In: Iab H, Noman N (eds) Deep neural evolution – deep learning with evolutionary computation. Springer, New York

    Google Scholar 

  • Real E, Aggarwal A, Huang Y, Le QV (2019) Regularized evolution for image classifier architecture search. In: Proceedings of the AAAI Conference on Artificial Intelligence

    Book  Google Scholar 

  • Real E, Liang C, So DR, Le QV (2020) AutoML-zero: evolving machine learning algorithms from scratch. arXiv:2003.03384

    Google Scholar 

  • Risi S, Togelius J (2017) Neuroevolution in games: state of the art and open challenges. IEEE Trans Comput Intell AI Games 9:25–41

    Article  Google Scholar 

  • Schaffer JD, Whitley D, Eshelman LJ (1992) Combinations of genetic algorithms and neural networks: a survey of the state of the art. In: Whitley D, Schaffer J (eds) Proceedings of the International Workshop on Combinations of Genetic Algorithms and Neural Networks. IEEE Computer Society Press, Los Alamitos, pp 1–37

    Google Scholar 

  • Schrum J, Miikkulainen R (2016) Discovering multimodal behavior in Ms. Pac-Man through evolution of modular neural networks. IEEE Trans Comput Intell AI Games 8:67–81

    Article  Google Scholar 

  • Stanley KO, D’Ambrosio DB, Gauci J (2009) A hypercube-based encoding for evolving large-scale neural networks. Artif Life 15(2):185–212

    Article  Google Scholar 

  • Stanley KO, Miikkulainen R (2003) A taxonomy for artificial embryogeny. Artif Life 9(2):93–130

    Article  Google Scholar 

  • Stanley KO, Miikkulainen R (2004) Competitive coevolution through evolutionary complexification. J Artif Intell Res 21:63–100

    Article  Google Scholar 

  • Stanley KO, Lehman J (2015) Why greatness cannot be planned: the Myth of the objective. Springer, New York

    Book  Google Scholar 

  • Stanley KO, Clune J, Lehman J, Miikkulainen R (2019) Designing neural networks through evolutionary algorithms. Nat Mach Intell 1:24–35

    Article  Google Scholar 

  • Such FP, Madhavan V, Conti E, Lehman J, Stanley KO, Clune J (2017) Deep neuroevolution: genetic algorithms are a competitive alternative for training deep neural networks for reinforcement learning. arXiv:1712.06567

    Google Scholar 

  • Togelius J, Lucas SM (2006) Evolving robust and specialized car racing skills. In: IEEE congress on evolutionary computation. IEEE, Piscataway, pp 1187–1194

    Google Scholar 

  • Valsalam V, Hiller J, MacCurdy R, Lipson H, Miikkulainen R (2013) Constructing controllers for physical multilegged robots using the enso neuroevolution approach. Evol Intell 14:303–331

    Google Scholar 

  • Werner GM, Dyer MG (1992) Evolution of communication in artificial organisms. In: Langton CG, Taylor C, Farmer JD, Rasmussen S (eds) Proceedings of the Workshop on Artificial Life (ALIFE’90). Addison-Wesley, Reading, pp 659 –687

    Google Scholar 

  • Yao X (1999) Evolving artificial neural networks. Proc IEEE 87(9):1423–1447

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. The University of Texas at Austin, Austin, TX, USA

    Risto Miikkulainen

  2. Cognizant AI Labs, San Francisco, CA, USA

    Risto Miikkulainen

Authors
  1. Risto Miikkulainen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Risto Miikkulainen .

Editor information

Editors and Affiliations

  1. Clayton, VIC, Australia

    Dinh Phung

  2. Software Engineering, Monash University School of Computer Science &, Melbourne, VIC, Australia

    Geoffrey I. Webb

  3. Engineering (CSE), University of New South Wales School of Computer Science &, Sydney, NSW, Australia

    Claude Sammut

Rights and permissions

Reprints and permissions

Copyright information

© 2023 Springer Science+Business Media, LLC, part of Springer Nature

About this entry

Check for updates. Verify currency and authenticity via CrossMark

Cite this entry

Miikkulainen, R. (2023). Neuroevolution. In: Phung, D., Webb, G.I., Sammut, C. (eds) Encyclopedia of Machine Learning and Data Science. Springer, New York, NY. https://doi.org/10.1007/978-1-4899-7502-7_594-2

Download citation

  • DOI: https://doi.org/10.1007/978-1-4899-7502-7_594-2

  • Published:

  • Publisher Name: Springer, New York, NY

  • Print ISBN: 978-1-4899-7502-7

  • Online ISBN: 978-1-4899-7502-7

  • eBook Packages: Springer Reference Computer SciencesReference Module Computer Science and Engineering

Publish with us

Policies and ethics

Chapter history

  1. Latest

    Neuroevolution
    Published:
    07 December 2022

    DOI: https://doi.org/10.1007/978-1-4899-7502-7_594-2

  2. Original

    Neuroevolution
    Published:
    29 April 2015

    DOI: https://doi.org/10.1007/978-1-4899-7502-7_594-1

Search

Navigation