Abstract
In the system identification context, neural networks are black-box models, meaning that both their parameters and structure need to be determined from data. Their identification is often done iteratively in an ad-hoc fashion focusing the first aspect. Frequently the selection of inputs, model structure, and model order are underlooked subjects by practitioners, because the number of possibilities is commonly huge, thus leaving the designer at the hands of the curse of dimensionality. Moreover, the design criteria may include multiple conflicting objectives, which gives to the model identification problem a multiobjective combinatorial optimisation character. Evolutionary multiobjective optimisation algorithms are particularly well suited to address this problem because they can evolve optimised model structures that meet pre-specified design criteria in acceptable computing time. In this article the subject is reviewed, the authors present their approach to the problem in the context of identifying neural network models for time-series prediction and for classification purposes, and two application case studies are described, one in each of these fields.
Access provided by Autonomous University of Puebla. Download to read the full chapter text
Chapter PDF
Similar content being viewed by others
Keywords
- Pareto Front
- Radial Basis Function Neural Network
- Radial Basis Function Network
- Prediction Horizon
- Multiobjective Evolutionary Algorithm
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Amari, S., Murata, N., Müller, K.R., Finke, M., Yang, H.: Asymptotic statistical theory of overtraining and cross-validation. IEEE Transactions on Neural Networks 8(5), 985–996 (1997), doi:10.1109/72.623200
Bauer, M., Buchtala, O., Horeis, T., Kern, R., Sick, B., Wagner, R.: Technical data mining with evolutionary radial basis function classifiers. Applied Soft Computing 9, 765–774 (2009), doi:10.1016/j.asoc.2008.07.007
Billings, S.A., Zheng, G.L.: Radial basis function network configuration using genetic algorithms. Neural Networks 8(6), 877–890 (1995)
Branke, J.: Evolutionary algorithms for neural network design and training. Tech. Rep. 322, University of Karlsruhe, Institute AIFB, Karlsruhe, Germany (1995)
Carse, B., Pipe, A.G., Fogarty, T.C., Hill, T.: Evolving radial basis function neural networks using a genetic algorithm. In: IEEE International Conference on Evolutionary Computation, vol. 1, pp. 300–305 (1995), doi:10.1109/ICEC.1995.489163
Chen, S., Wu, Y., Luk, B.: Combined genetic algorithm optimization and regularized orthogonal least squares learning for radial basis function networks. IEEE Transactions on Neural Networks 10(5), 1239–1243 (1999)
Coello Coello, C.: Recent trends in evolutionary multiobjective optimization. In: Jain, L., Wu, X., Abraham, A., Jain, L., Goldberg, R. (eds.) Evolutionary Multiobjective Optimization, Advanced Information and Knowledge Processing, pp. 7–32. Springer, Heidelberg (2005), http://dx.doi.org/10.1007/1-84628-137-7_2 , doi:10.1007/1-84628-137-7-2
Coello Coello, C.: Evolutionary multi-objective optimization: a historical view of the field. IEEE Computational Intelligence Magazine 1(1), 28–36 (2006), http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=1597059&tag=1 , doi:10.1109/MCI.2006.1597059
Crispim, E.M., Ferreira, P.M., Ruano, A.E.: Prediction of the solar radiation evolution using computational intelligence techniques and cloudiness indices. International Journal of Innovative Computing, Information and Control 4(5), 1121–1133 (2008)
Deb, K.: Multi-Objective Optimization using Evolutionary Algorithms. John Wiley & Sons., Ltd, Chichester (2001)
Ferreira, P.M., Faria, E., Ruano, A.E.: Neural network models in greenhouse air temperature prediction. Neurocomputing 43(1-4), 51–75 (2002)
Ferreira, P.M., Martins, I.A., Ruano, A.E.: Cloud and clear sky pixel classification in ground-based all-sky hemispherical digital images. In: Ferreira, P.M. (ed.) Proceedings of CMTEE 2010, the IFAC Conference on Control Methodologies and Technology for Energy Efficiency. International Federation of Automatic Control, Vilamoura, Portugal (2010)
Ferreira, P.M., Ruano, A.E.: Exploiting the separability of linear and non-linear parameters in radial basis function neural networks. In: IEEE Symposium 2000: Adaptive Systems for Signal Processing, Communications, and Control, Canada, pp. 321–326 (2000), http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=882493 , doi:10.1109/ASSPCC.2000.882493
Ferreira, P.M., Ruano, A.E.: Application of computational intelligence methods to greenhouse environmental modelling. In: (WCCI 2008) IJCNN 2008 (IEEE World Congress on Computational Intelligence). 2008 IEEE International Joint Conference on Neural Networks, pp. 3582–3589 (2008), doi:10.1109/IJCNN.2008.4634310
Ferreira, P.M., Ruano, A.E.: On-line sliding-window methods for process model adaptation. IEEE Transactions on Instrumentation and Measurement 58(9), 3012–3020 (2009), doi:10.1109/tim.2009.2016818
Ferreira, P.M., Ruano, A.E., Fonseca, C.: Genetic assisted selection of rbf model structures for greenhouse inside air temperature prediction. In: IEEE Conference on Control Applications, Turkey, pp. 576–581 (2003)
Ferreira, P.M., Ruano, A.E., Fonseca, C.: Evolutionary multi-objective design of radial basis function networks for greenhouse environmental control. In: IFAC World Congress on Automatic Control 16th, Czech Republic (2005)
Ferreira, P.M., Ruano, A.E., Pestana, R.: Improving the identification of rbf predictive models to forecast the portuguese electricity consumption. In: Ferreira, P.M. (ed.) Proceedings of CMTEE 2010, the IFAC Conference on Control Methodologies and Technology for Energy Efficiency. International Federation of Automatic Control, Vilamoura, Portugal (2010)
Ferreira, P.M., Ruano, A.E., Pestana, R., Kóczy, L.T.: Evolving rbf predictive models to forecast the portuguese electricity consumption. In: ICONS 2009: The 2nd IFAC Int. Conference on Intelligent Control Systems and Signal Processing, Istanbul, Turkey (2009)
Fletcher, R.: Practical Methods of Optimization, 2nd edn. Wiley Interscience, Hoboken (2000)
Fonseca, C., Fleming, P.: Non-linear system identification with multiobjective genetic algorithms. In: Proceedings of the 13 IFAC World Congress, vol. C, pp. 187–192 (1996)
Fonseca, C., Fleming, P.: Multiobjective optimization and multiple constraint handling with evolutionary algorithms i: A unified formulation. IEEE Transactions on Systems, Man and Cybernetics - Part A: Systems and Humans 28(1), 26–37 (1998), doi:10.1109/3468.650319
Gill, P.E., Murray, W., Wright, M.H.: Practical Optimization. Academic Press, Inc., London (1981)
Griffin, I., Fleming, P.: An overview of non-linear identification and control with neural networks. In: Ruano, A.E. (ed.) Intelligent Control Using Soft-Computing Methodologies. Control Series, pp. 89–118. IEE Publishing (2005)
Guillén, A., Pomares, H., González, J., Rojas, I., Valenzuela, O., Prieto, B.: Parallel multiobjective memetic rbfnns design and feature selection for function approximation problems. Neurocomputing 72(16-18), 3541–3555 (2009), doi:10.1016/j.neucom.2008.12.037
Haykin, S.: Neural Networks: a Comprehensive Foundation, 2nd edn. Prentice Hall, Inc., Englewood Cliffs (1999)
Jung, J., Reggia, J.: Evolutionary design of neural network architectures using a descriptive encoding language. IEEE Transactions on Evolutionary Computation 10(6), 676–688 (2006), doi:10.1109/TEVC.2006.872346
Kaylani, A., Georgiopoulos, M., Mollaghasemi, M., Anagnostopoulos, G.C., Sentelle, C., Zhong, M.: An adaptive multiobjective approach to evolving art architectures. IEEE Transactions on Neural Networks 21(4), 529–550 (2010)
Lee, C.W., Shin, Y.C.: Growing radial basis function networks using genetic algorithm and orthogonalization. International Journal of Innovative Computing, Information and Control 5(11(A)), 3933–3948 (2009)
Leung, C., Lam, F.: Performance analysis for a class of iterative image thresholding algorithms. Pattern Recognition 29(9), 1523–1530 (1996)
Marquardt, D.W.: An algorithm for least-squares estimation of nonlinear parameters. Journal of the Society for Industrial and Applied Mathematics 11(2), 431–441 (1963)
McDonnell, J., Waagen, D.: Determining neural network hidden layer size using evolutionary programming. In: Proceedings of the 1993 World Congress on Neural Networks, vol. III, pp. 564–657 (1993)
Otsu, N.: A threshold selection method from gray-level histograms. IEEE Transactions on Systems, Man and Cybernetics SMC-9(1), 62–66 (1979)
Peck, C., Dhawan, A., Meyer, C.: Genetic algorithm based input selection for a neural network function approximator with applications to ssme health monitoring. In: IEEE International Conference on Neural Networks, vol. 2, pp. 1115–1122 (1993), doi:10.1109/ICNN.1993.298714
Ridler, T., Calvard, S.: Picture thresholding using an iterative selection method. IEEE Transactions on Systems, Man and Cybernetics SMC-8(8), 630–632 (1978)
Rodríguez-Vázquez, K., Fonseca, C., Fleming, P.: Identifying the structure of nonlinear dynamic systems using multiobjective genetic programming. IEEE Transactions on Systems, Man and Cybernetics - Part A: Systems and Humans 34(4), 531–545 (2004), doi:10.1109/TSMCA.2004.826299
Ruano, A., Crispim, E., Frazão, P.: Moga design of neural network predictors of inside temperature in public buildings. In: Balas, V., Fodor, J., Várkonyi-Kóczy, A. (eds.) Soft Computing Based Modeling in Intelligent Systems. SCI, vol. 196, pp. 35–61. Springer, Heidelberg (2009), doi:10.1007/978-3-642-00448-3-3
Ruano, A., Fleming, P., Jones, D.: Connectionist approach to pid autotuning. IEE Proceedings (part D), 139(3), 279–285 (1992), http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=141517
Ruano, A., Jones, D., Fleming, P.: A new formulation of the learning problem of a neural network controller. In: Proceedings of the 30th IEEE Conference on Decision and Control, vol. 1, pp. 865–866 (1991), http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=261439 , doi:10.1109/CDC.1991.261439
Ruano, A.E., Ferreira, P.M., Cabrita, C., Matos, S.: Training neural networks and neuro-fuzzy systems: A unified view. In: Proceedings of the 15th IFAC World Congress, vol. 15 (2002)
Ruano, A.E., Ferreira, P.M., Fonseca, C.: An overview of non-linear identification and control with neural networks. In: Ruano, A.E. (ed.) Intelligent Control Using Soft-Computing Methodologies. Control Series, pp. 37–87. IEE Publishing (2005)
Ruano, A.E., Ferreira, P.M., Mendes, H.: Moga design of temperature and relative humidity models for predictive thermal comfort. In: Ferreira, P.M. (ed.) Proceedings of CMTEE 2010, the IFAC Conference on Control Methodologies and Technology for Energy Efficiency. International Federation of Automatic Control, Vilamoura, Portugal (2010)
Ruano, A.E., Fleming, P., Teixeira, C., Rodríguez-Vázquez, K., Fonseca, C.: Nonlinear identification of aircraft gas-turbine dynamics. Neurocomputing 55(3-4), 551–579 (2003), doi:10.1016/S0925-2312(03)00393-X
Sezgin, M., Sankur, B.: Survey over image thresholding techniques and quantitative performance evaluation. Journal of Electronic Imaging 13(1), 146–165 (2004)
Sjöberg, J., Ljung, L.: Overtraining, regularization, and searching for minimum with application to neural networks. In: Preprint IFAC Symposium on Adaptive Systems in Control and Signal Processing, pp. 669–674 (1994)
Trussel, H.: Comments on picture thresholding using an iterative selection method. IEEE Transactions on Systems, Man and Cybernetics SMC-9(5), 311 (1979)
Yen, G.: Multi-objective evolutionary algorithm for radial basis function neural network design. In: Jin, Y. (ed.) Multi-objective machine learning. SCI, vol. 16, pp. 221–239. Springer, Heidelberg (2006), doi:10.1007/3-540-33019-4-10
Zitzler, E., Laumanns, M., Bleuler, S.: A tutorial on evolutionary multiobjective optimization. In: Gandibleux, X., et al. (eds.) Metaheuristics for Multiobjective Optimisation. Lecture Notes in Economics and Mathematical Systems, vol. 535, pp. 3–37. Springer, Heidelberg (2004)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Ferreira, P.M., Ruano, A.E. (2011). Evolutionary Multiobjective Neural Network Models Identification: Evolving Task-Optimised Models. In: Ruano, A.E., Várkonyi-Kóczy, A.R. (eds) New Advances in Intelligent Signal Processing. Studies in Computational Intelligence, vol 372. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-11739-8_2
Download citation
DOI: https://doi.org/10.1007/978-3-642-11739-8_2
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-11738-1
Online ISBN: 978-3-642-11739-8
eBook Packages: EngineeringEngineering (R0)