Abstract
Multilayer feedforward neural networks have been successfully applied in different domains. Defining an interpretable architecture of a multilayer perceptron (MLP) for a given problem is still challenging. We propose a novel approach based on concept lattices to automatically design a neural network architecture. The designed architecture can then be trained with the backpropagation algorithm. We report experimental results obtained on different datasets, and then discuss our contribution as a means to provide semantics to each neuron in order to build an interpretable neural network.
Access provided by Autonomous University of Puebla. Download to read the full chapter text
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Andrews, R., Diederich, J., Tickle, A.: Surevy and critique of techniques for extracting rules from trained artificial neural networks. Knowledge-Based Systems 8(6), 373–389 (1995)
Bastide, Y., Pasquier, N., Taouil, R., Stumme, G., Lakhal, L.: Mining minimal non-redundant association rules using frequent closed itemsets. In: Palamidessi, C., Moniz Pereira, L., Lloyd, J.W., Dahl, V., Furbach, U., Kerber, M., Lau, K.-K., Sagiv, Y., Stuckey, P.J. (eds.) CL 2000. LNCS (LNAI), vol. 1861, pp. 972–986. Springer, Heidelberg (2000)
Bertini Jr., J.R., do Carmo Nicoletti, M.: MBabCoNN - A Multiclass Version of a Constructive Neural Network Algorithm Based on Linear Separability and Convex Hull. In: Kůrková, V., Neruda, R., Koutník, J. (eds.) ICANN 2008, Part II. LNCS, vol. 5164, pp. 723–733. Springer, Heidelberg (2008)
Carpineto, C., Romano, G.: Concept Data Analysis: Theory and Applications. John Wiley and Sons, Chichester (2004)
Cibas, T., Fogelman, F., Gallinari, P., Raudys, S.: Variable Selection with Optimal Cell Damage. In: International conference on Artificial Neural Network (ICANN 1994), Part I, pp. 727–730 (1994)
Cornuéjols, A., Miclet, L.: Apprentissage Artificiel: Concepts et algorithmes, Eyrolles (2002)
Mitchell., T.M.: Machine Learning. McGraw-Hill, New York (1997)
Curran, D., O’Riordan, C.: Applying Evolutionary Computation to designing Neural Networks: A study of the State of the art department of Information Technology, Technical report, NUI Galway (2002)
Darbari, A.: Rule extraction from trained ANN: A Survey, Technical report, Institute of Artificial Intelligence, Dept. of Computer Science, TU Dresden, Germany (2001)
Dreyfus, G., Samuelides, M., Martinez, J.M., Gordon, M., Badran, F., Thiria, S., Hérault, L.: Réseaux de Neurones: Méthodologie et applications. Eyrolles (2002)
Duch, W., Setiono, R., Zurada, J.M.: Computational intelligence methods for understanding of data. Proceedings of the IEEE 92(5), 771–805 (2004)
Endres, D., Fldiák, P.: An application of formal concept analysis to Neural Decoding. In: Belohlavek, E.R., Kuznetsov, S.O. (eds.) proceedings of sixth Intl. Conf. on Concept Lattices and Applications (CLA), pp. 181–192 (2008)
Frean, M.: The Upstart algorithm: A method for constructing and training feed forward neural networks. Neural computation (4), 198–209 (1992)
Fu, H., Fu, H., Njiwoua, P., Nguifo, E.M.: A comparative study of FCA-based supervised classification algorithms. In: Eklund, P. (ed.) ICFCA 2004. LNCS (LNAI), vol. 2961, pp. 313–320. Springer, Heidelberg (2004)
Gallant, S.: Perceptron based learning algorithms. IEEE Transactions Neural Networks 1, 179–191 (1990)
Ganter, B., Wille, R.: Formal Concepts Analysis: Mathematical foundations. Springer, Heidelberg (1999)
Garcez d’Avila, A.S., Broda, K., Gabbay, D.M.: Symbolic knowledge extraction from trained neural networks: a sound approach. Artificial Intelligence 125, 155–207 (2001)
Gasmi, G., Ben Yahia, S., Mephu Nguifo, E., Slimani, Y.: A new informative generic base of association rules. In: Ho, T.-B., Cheung, D., Liu, H. (eds.) PAKDD 2005. LNCS (LNAI), vol. 3518, pp. 81–90. Springer, Heidelberg (2005)
Guigues, J.L., Duquenne, V.: Familles minimales d’implications informatives resultant d’un tableau de donnes binaires. Mathmatiques et sciences sociales 95, 5–18 (1986)
Han, J., Kamber, M.: Datamining: Concepts and Techniques. Morgan Kauffman Publishers, San Francisco (2001)
Hertz, J., Krogh, A., Palmer, R.G.: Introduction to the theory of neural computation. Lecture Notes, Santa Fe Institute. Addison Wesley Publishing, Reading (1991)
Kuznetsov, S., Obiedkov, S.: Comparing Performance of Algorithms for Generating Concept Lattices. JETAI 14(2/3), 189–216 (2002)
Le Cun, Y., Denker, J.S., Solla, S.A.: Optimal Brain Damage. In: Advances in Neural Information Processing Systems, vol. 2, pp. 598–605. Morgan Kaufmann Publishers, San Francisco (1990)
Mephu Nguifo, E.: Une nouvelle approche base sur le treillis de Galois pour l’apprentissage de concepts. Mathématiques, Informatique, Sciences Humaines 134, 19–38 (1994)
Mephu Nguifo, E., Tsopzé, N., Tindo, G.: M-CLANN: Multi-class concept lattice-based artificial neural network for supervised classification. In: Kůrková, V., Neruda, R., Koutník, J. (eds.) ICANN 2008,, Part II. LNCS, vol. 5164, pp. 812–821. Springer, Heidelberg (2008)
Newmann, D.J., Hettich, S., Blake, C.L., Merz, C.J.: (UCI)Repository of machine learning databases, Dept. Inform. Comput. Sci. Univ. California, Irvine, CA (1998), http://www.ics.uci.edu/AI/ML/MLDBRepository.html
Parekh, R., Yang, J., Honavar, V.: Constructive Neural Networks Learning Algorithms for Multi-Category Classification. Department of Computer Science Lowa State University Tech. Report ISU CS TR 95-15 (1995)
Parekh, R., Yang, J., Honavar, V.: Constructive Neural-Network Learning Algorithms for Pattern Classification. IEEE Transactions on neural networks 11(2), 436–451 (2000)
Piccinini, G.: Some neural networks compute, others dont. Neural Network 21 (special issue), 311–321 (2008)
Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning representations by backpropagating errors. Nature (323), 318–362 (1986)
Rudolph, S.: Using FCA for Encoding Closure Operators into Neural Networks. In: Priss, U., Polovina, S., Hill, R. (eds.) ICCS 2007. LNCS (LNAI), vol. 4604, pp. 321–332. Springer, Heidelberg (2007)
Shavlik, W.J., Towell, G.G.: Kbann: Knowledge based articial neural networks. Artificial Intelligence (70), 119–165 (1994)
Stumme, G., Taouil, R., Bastide, Y., Pasquier, N., Lakhal, L.: Computing Iceberg concept lattices with TITANIC. Journal on Knowledge and Data Engineering (KDE) 2(42), 189–222 (2002)
Subirats, J.L., Franco, L., Molina Conde, I., Jerez, J.M.: Active learning using a constructive neural network algorithm. In: Kůrková, V., Neruda, R., Koutník, J. (eds.) ICANN 2008,, Part II. LNCS, vol. 5164, pp. 803–811. Springer, Heidelberg (2008)
Tsopze, N., Mephu Nguifo, E., Tindo, G.: CLANN: Concept-Lattices-based Artificial Neural Networks. In: Diatta, J., Eklund, P., Liquire, M. (eds.) Proceedings of fifth Intl. Conf. on Concept Lattices and Applications (CLA 2007), Montpellier, France, October 24-26, 2007, pp. 157–168 (2007)
Witten, I.H., Frank, E.: Data Mining: Practical machine learning tools and techniques. Morgan Kaufmann, San Francisco (2005)
Yacoub, M., Bennani, Y.: Architecture Optimisation in Feedforward Connectionist Models. In: Gerstner, W., Hasler, M., Germond, A., Nicoud, J.-D. (eds.) ICANN 1997. LNCS, vol. 1327. Springer, Heidelberg (1997)
Yang, J., Parekh, R., Honavar, V.: Distal: An Inter-pattern Distance-based Constructive Learning Algorithm: Intell. Data Anal. 3, 55–73 (1999)
Werbos, P.J.: Why neural networks? In: Fiesler, E., Beale, R. (eds.) Handbook of Neural Computation, pp. A2.1:1–A2.3:6. IOP Pub., Oxford University Press, Oxford (1997)
Wille, R.: Restructuring Lattice Theory: An Approach Based on Hierarchies of Concepts. In: Rival, I. (ed.) Ordered Sets, pp. 445–470 (1982)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Nguifo, E.M., Tsopze, N., Tindo, G. (2009). M-CLANN: Multiclass Concept Lattice-Based Artificial Neural Network. In: Franco, L., Elizondo, D.A., Jerez, J.M. (eds) Constructive Neural Networks. Studies in Computational Intelligence, vol 258. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-04512-7_6
Download citation
DOI: https://doi.org/10.1007/978-3-642-04512-7_6
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-04511-0
Online ISBN: 978-3-642-04512-7
eBook Packages: EngineeringEngineering (R0)