Abstract
The chapter presents a new method of rule extraction from trained neural networks, based on a hierarchical multiobjective genetic algorithm. The problems associated with rule extraction, especially its multiobjective nature, are described in detail, and techniques used when approaching them with genetic algorithms are presented. The main part of the chapter contains a thorough description of the proposed method. It is followed by a discussion of the results of experimental study performed on popular benchmark datasets that confirm the method’s effectiveness.
Access provided by Autonomous University of Puebla. Download to read the full chapter text
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Andrews, R., J. Diedrich, and A. Tickle: 1995, ‘Survey and critique of techniques for extracting rules from trained artificial neural networks’. Knowledge-Based Systems 8(6), 373–389.
Andrews, R. and S. Geva: 1994, ‘Rule extraction from constrained error back propagation MLP’. In: Proceedings of 5th Australian Conference on Neural Networks, Brisbane, Quinsland. pp. 9–12.
Blake, C. C. and C. Merz: 1998, ‘UCI Repository of Machine Learning Databases’. University of California, Irvine, Dept. of Information and Computer Sciences.
Coello, Carlos A. Van Veldhuizen, D. A. and G. B. Lamont: 2002, Evolutionary Algorithms for Solving Multi-Objective Problems. Kluwer Academic Pblisher. New York.
Fu, X. and L. Wang: 2001, ‘Rule extraction by genetic algorithms based on a simplified RBF neural network’. In: Proceedings Congress on Evolutionary Computation. pp. 753–758.
Goldberg, D. E.: 1989, Genetic Algorithms in Search, Optimization and machine Learning. Addison-Wesley Publishing Company, Reading, Massachusetts.
Hayashi, Y., R. Setiono, and K. Yoshida: 2000, ‘Learning M of N concepts for medical diagnosis using neural networks’. Journal of Advanced Computational Intelligence 4(4), 294–301.
Hayward, R., C. Ho-Stuart, J. Diedrich, and P. E.: 1996, ‘RULENEG: Extracting rules from trained ANN by stepwise negation’. Technical report, Neurocomputing Research Centre Queensland University Technology Brisbane, Old.
Jin, Y. and B. Sendhoff: 2003, ‘Extracting interpretable fuzzy rules from RBF networks’. Neural Processing Letters 17(2), 149–164.
Jin, Y., B. Sendhoff, and E. Koerner: 2005, ‘Evolutionary multi-objective optimization for simultaneous generation of signal-type and symbol-type representations’. In: The Third International Conference on Evolutionary Multi-Criterion Optimization. pp. 752–766.
Markowska-Kaczmar, U. and P. Wnuk-Lipinski: 2004, ‘Rule Extraction from Neural Network by Genetic Algorithm with Pareto Optimization’. In: L. Rutkowski (ed.): Artificial Intelligence and Soft Computing. pp. 450–455.
Markowska-Kaczmar, U. and R. Zagorski: 2004, ‘Hierarchical Evolutionary Algorithms in the Rule Extraction from neural Networks’. Fourth International Symposium on Engineering of Intelligent Systems, EIS. Funchal, Portugal.
Michalewicz, Z.: 1996, Genetic Algorithms + Data Structures = Evolution Programs. Springer Verlag Berlin Heidelberg.
Mitra, S. and H. Yoichi: 2000, ‘Neuro-Fuzzy Rule Generation: Survey in Soft Computing Framework’. IEEE Transactions on Neural Networks 11(3), 748–768.
Palade, V., D. C. Neagu, and R. J. Patton: 2001, ‘Interpretation of Trained Neural Networks by Rule Extraction’. Fuzzy Days 2001, LNC 2206 pp. 152–161.
Santos, R. Nievola, J. and A. Freitas: 2000, ‘Extracting comprehensible rules from neural networks via genetic algorithms’. Symp. on Combinations of Evolutionary Computation and Neural Network 1, 130–139.
Setiono, R., W. K. Leow, and J. Zurada: 2002, ‘Extraction of Rules from artificial Neural Networks for Nonlinear Regression’. IEEE Transactions on Neural Networks 13(3), 564–577.
Siponen, M., J. Vesanto, O. Simula, and P. Vasara: 2001, ‘An Approach to Automated Interpretation of SOM’. In: In Proceedings of Workshop on Self-Organizing Map 2001(WSOM2001). pp. 89–94.
Srinivas, N. and K. Deb: 1993, ‘Multiobjective optimizatio using nondominated sorting in genetic algorithms’. Technical report, Department of Mechanical Engineering, Indian Institute of Technology, Kanpur, India.
Taha, I. and J. Ghosh: 1999, ‘Symbolic Interpretation of artificial Neural Networks’. IEEE Transactions on Knowledge and Data Engineering 11(3), 448–463.
Thrun, S. B.: 1995, ‘Extracting rules from artificial neural networks with distributed representation’. In: G. Tesauro, D. Touretzky, and T. Leen (eds.): Advances in Neural Information Processing Systems.
Weijters, A. and J. Paredis: 2000, ‘Rule Induction with Genetic Sequential Covering Algorithm (GeSeCo)’. In: Proceedings of the second ICSC Symposium on Engineering of Intelligent Systems, (EIS 2000). pp. 245–251.
Zitzler, E. and L. Thiele: 1999, ‘Multiobjective Evolutionary Algorithms: A Comparative Case Study and Strength Pareto Approach’. IEEE Transaction on Evolutionary Computation 3(4), 257–271.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer
About this chapter
Cite this chapter
Markowska-Kaczmar, U., Mularczyk, K. (2006). GA-Based Pareto Optimization for Rule Extraction from Neural Networks. In: Jin, Y. (eds) Multi-Objective Machine Learning. Studies in Computational Intelligence, vol 16. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-33019-4_14
Download citation
DOI: https://doi.org/10.1007/3-540-33019-4_14
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-30676-4
Online ISBN: 978-3-540-33019-6
eBook Packages: EngineeringEngineering (R0)