Abstract
Approaches combining genetic algorithms and neural networks have received a great deal of attention in recent years. As a result, much work has been reported in two major areas of neural network design: training and topology optimisation. This paper focuses on the key issues associated with the problem of pruning a multilayer perceptron using genetic algorithms and simulated annealing. The study presented considers a number of aspects associated with network training that may alter the behaviour of a stochastic topology optimiser. Enhancements are discussed that can improve topology searches. Simulation results for the two mentioned stochastic optimisation methods applied to non-linear system identification are presented and compared with a simple random search.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
References
Bhat VH, Minderman AP, McAvoy T, Wang NS. Modeling chemical process systems via neural computation. IEEE Control Systems Magazine 1990; April, 24–29
Evans JT, Gomm JB, Williams D, Lisboa PJB, To QS. A practical application of neural modelling and predictive control. In: Application of Neural Networks to Modelling and Control, Page GF, Gomm JB, Williams D (eds.). Chapman & Hall, 1994
Hunt KJ, Sbarbaro D, Tbikowski R, Gawthrop PJ. Neural networks for control systems — a survey. Automatica 1992; 28 (6): 1083–1112
Narendra KS, Parthasarathy K. Identification and control of dynamical systems using neural networks. IEEE Trans on Neural Networks 1990; 1 (1): 4–27
Beaufays F, Abdel-Mogid Y, Widrow B. Application of neural networks to load-frequency control in power systems. Neural Networks 1994; 7 (1): 183–194
Wang D, Chai T. Multivariable adaptive control of unknown nonlinear dynamic systems using neural networks. Proc 33rd Conf on Decision and Control, Lake Buena Vista, FL, 1994, 2500–2505
Willis MJ, Massimo C, Montague GA, Tham MT, Morris AJ. Artificial neural networks in process engineering. IEEE Proc Pt D 1991; 138 (3): 256–266
Naidu RS, Zafiriou E, McAvoy TJ. Use of neural networks for sensor failure detection in a control system. IEEE Control Systems Magazine 1990; April, 49–55
Haykin S. Neural Networks — A Comprehensive Foundation. Macmillan College Publishing, 1994
Reed R. Pruning algorithms — a survey. IEEE Trans on Neural Networks 1993; 4 (5)
Eigel-Danielson V, Augustejin MF. Neural network pruning and its effect on generalization — some experimental results. Neural Parallel & Scientific Computation 1993; 1: 59–70
Hassibi B, Stork DG, Wolff GJ. Optimal brain surgeon and general network pruning. IEEE International Conference on Neural Networks, 1993; 1, 293–299
Fahlman SE, Lebiere C. The cascade-correlation learning architecture. Technical Report CMU-CS-90-100, Carnegie Mellon University, 1990
Kuscu I, Thornton C. Design of Artificial Neural Networks Using Genetic Algorithms: review and prospect. Cognitive and Computing Sciences, University of Sussex, 1994
Billings SA, Jamaluddin HB, Chen S. Properties of neural networks with applications to modeling non-linear dynamical systems. Int J Control 1992, 55 (1): 193–224
Jutten C. Learning in evolutive neural architectures: an ill-posed problem? From natural to artificial neural computations. Int Workshop on ANN, Springer-Verlag, 1995, 361–371
Miller GF, Todd PM, Hegde SU. Designing neural networks using genetic algorithms. Proc 3rd Int Conf on Genetic Algorithms. Morgan Kaufmann, 1989, 379–384
Whitley D, Starkweather T, Bogart C. Genetic algorithms and neural networks: optimizing connections and connectivity. Parallel Computing 1990; 14: 347–361
Yaw-Terng Su, Yuh-Tay Sheen, Neural network for system identification. Int J Systems Sci 1992; 23 (12): 2171–2186
Schiffmann W, Joost M, Werner R. Optimization of the Backpropagation Algorithm for Training Multilayer Perceptrons. Technical Report, University of Koblenz, 1992
Cichocki A, Unbehauen R. Neural Networks for Optimization and Signal Processing, 4th ed. Wiley, 1994
Stepniewski SW, Keane AJ. Topology design of feed-forward neural networks by genetic algorithms. PPSN IV, Berlin, 1996
Bishop CM. Novelty detection and neural network validation. IEE Proc Vision, Image and Signal Processing 19??; 141 (4): 217–222
Goldberg, DE. Genetic Algorithms in Search, Optimization and Machine Learning. Addison-Wesley, 1989
BÄck T, Hoffmeister F. Extended selection mechanisms in genetic algorithms. Proc 4th Int Conf on Genetic Algorithms, Morgan Kaufmann, 1991, 92–99
Keane AJ. The Options Design Exploration System, Reference Manual and User Guide, 1994 (available by Internet, http:www.soton.ac.uk/najk/options.ps)
Keane AJ. Experiences with optimizers in structural design. Proc Conf on Adaptive Computing in Engineering Design and Control, 1994, 14–27
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Stepniewski, S.W., Keane, A.J. Pruning backpropagation neural networks using modern stochastic optimisation techniques. Neural Comput & Applic 5, 76–98 (1997). https://doi.org/10.1007/BF01501173
Issue Date:
DOI: https://doi.org/10.1007/BF01501173