Abstract
Modal learning in neural computing [33] refers to the strategic combination of modes of adaptation and learning within a single artificial neural network structure. Modes, in this context, are learning methods that are transferable from one learning architecture to another, such as weight update equations. In modal learning two or more modes may proceed in parallel in different parts of the neural computing structure (layers and neurons), or they occupy the same part of the structure, and there is a mechanism for allowing the neural network to switch between modes.
From a theoretical perspective any individual mode has inherent limitations because it is trying to optimise a particular objective function. Since we cannot in general know a priori the most effective learning method or combination of methods for solving a given problem, we should equip the system (the neural network) with the means to discover the optimal combination of learning modes during the learning process. There is potential to furnish a neural system with numerous modes. Most of the work conducted so far concentrates on the effectiveness of two to four modes. The modal learning approach applies equally to supervised and unsupervised (including self organisational) methods. In this chapter, we focus on modal self organisation.
Examples of modal learning methods include the Snap-Drift Neural Network (SDNN) [5, 25, 28, 33, 32] which toggles its learning between two modes, an adaptive function neural network, in which adaptation applies simultaneously to both the weights and to the shape of the individual neuron activation functions, and the combination of four learning modes, in the form of Snap-drift ADaptive FUnction Neural Network [17, 18, 33]. In this chapter, after reviewing modal learning in general, we present some examples methods of modal self organisation. Self organisation is taken in the broadest context to include unsupervised methods.We review the simple unsupervised modalmethod called snap-drift [5, 25, 28, 32], which combines Learning Vector Quantization [21, 22, 23, 37] with a ’Min’ or Fuzzy AND method. Snap-drift is then applied to the Self-Organising Map [34]. The methods are utilised in numerous real-world problems such as grouping learners’ responses to multiple choice questions, natural language phrase recognition and pattern classification on well known datasets. Algorithms, dataset descriptions, pseudocode and Matlab code are presented.
Access provided by Autonomous University of Puebla. Download to read the full chapter text
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Alpaydin, E., Kaynak, C.: Cascading Classifiers. Kybernetika 34, 369–374 (1998)
Burge, P., Shawe-Taylor, J.: An Unsupervised Neural Network Approach to Profiling the Behavior of Mobile Phone Users for Use in Fraud Detection. Journal of Parallel and Distributed Computing 61(7), 915–925 (2001)
Carpenter, G.A., Grossberg, S.: Adaptive resonance theory. In: Arbib, M.A. (ed.) The Handbook of Brain Theory and Neural Networks, Second Edition, 2nd edn., pp. 87–90. MIT Press, Cambridge (2003)
Dafoulas, G.A.: The role of feedback in online learning communities. In: Fifth IEEE International Conference on Advanced Learning Technologies, pp. 827–831 (2005)
Donelan, H., Pattinson, C., Palmer-Brown, D.: The Analysis of User Behaviour of a Network Management Training Tool using a Neural Network. Journal of Systemics, Cybernetics and Informatics 3(5) (2006)
Dotan, Y., Intrator, N.: Multimodality exploration by an unsupervised projection pursuit neural network. IEEE Transactions on Neural Networks 9(3), 464–472 (1998)
Duda, R.O., Hart, P.E.: Pattern Classification and Scene Analysis, p. 218. John Wiley & Sons, New York (1973)
Ekpenyong, F., Palmer-Brown, D., Brimicombe, A.: Extracting road information from recorded GPS data using snap-drift neural network. Neurocomputing 73, 24–36 (2009)
Fernandez Aleman, J.L., Palmer-Brown, D., Jayne, C.: Effects of Response Driven Feedback in Computer Science Learning. IEEE Transactions on Education 99 (2010), doi:10.1109/TE.2010.2087761
Fisher, R.A.: The use of multiple measurements in taxonomic problems. Annual Eugenics 7, Part II, 179–188 (1936); also in Contributions to Mathematical Statistics. John Wiley, NY (1950)
Forina, M., Lanteri, S., Armanino, C., et al.: PARVUS–an extendible package for data exploration, classification and correlation. Institute of Pharmaceutical and Food Analysis and Technologies, Via Brigata Salerno, 16147 Genoa, Italy (1991)
Garside, R., Leech, G., Varadi, T.: Manual of Information to Accompany the Lancaster Parsed Corpus. University of Oslo (1987)
Gupta, L., McAvoy, M.: Investigating the prediction capabilities of the simple recurrent neural network on real temporal sequences. Pattern Recognition 33(i12), 2075–2081 (2000)
Hebb, D.O.: The organization of behavior. Wiley & Sons, New York (1949)
Higgins, E., Tatham, L.: Exploring the potential of Multiple Choice Questions in Assessment. Learning & Teaching in Action 2(1) (2003)
Horton, P., Nakai, K.: A Probablistic Classification System for Predicting the Cellular Localization Sites of Proteins. Intelligent Systems in Molecular Biology, 109–115 (1996)
Kang, M., Palmer-Brown, D.: A Multilayer ADaptive FUnction Neural Network (MADFUNN) for Analytical Function Recognition. IJCNN (2006); part of the IEEE World Congress on Computational Intelligence, WCCI 2006, Vancouver, BC, Canada, pp. 1784–1789 (2006)
Kang, M., Palmer-Brown, D.: A Modal Learning Adaptive Function Neural Network Applied to Handwritten Digit Recognition. Information Sciences 178(20), 3802–3812 (2008)
Kaynak, C.: Methods of Combining Multiple Classifiers and Their Applications to Handwritten Digit Recognition. MSc Thesis, Institute of Graduate Studies in Science and Engineering, Bogazici University (1995)
Kohonen, T.: Self-organised formation of topologically correct feature maps. Biological Cybernetics 43 (1982)
Kohonen, T.: Learning Vector Quantisation. Helsinki University of Technology, Laboratory of Computer and Information Science, Report TKK-F-A-601 (1986)
Kohonen, T.: Learning Vector Quantisation. Neural Networks 1, 303 (1988)
Kohonen, T.: Self-Organisation and Asssociative Memory, 3rd edn. Springer, Heilderberg (1989)
Kohonen, T.: Improved Versions of Learning Vector Quantization. In: Proc. of IJCNN 1990, Washington, DC, vol. 1, pp. 545–550 (1990)
Lee, S.W., Palmer-Brown, D., Roadknight, C.M.: Performance guided Neural Network for Rapidly Self Organising Active Network Management. Neurocomputing 61C, 5–20 (2004a)
Lee, S.W., Palmer-Brown, D., Roadknight, C.M.: Reinforced Snap Drift Learning for Proxylet Selection in Active Computer Networks. In: Proc. of IJCNN 2004, Budapest, Hungary, vol. 2, pp. 1545–1550 (2004b)
Lee, S.W., Palmer-Brown, D.: Snap-drift learning for phrase recognition. In: Proc. IEEE IJCNN 2005, Montréal, Québec, Canada, vol. 1, pp. 588–592 (2005)
Lee, S.W., Palmer-Brown, D.: Phonetic Feature Discovery in Speech Using Snap-Drift Learning. In: Kollias, S.D., Stafylopatis, A., Duch, W., Oja, E. (eds.) ICANN 2006, Part II. LNCS, vol. 4132, pp. 952–962. Springer, Heidelberg (2006)
Mayberry III, M.R., Miikkulainen, R.: SARDSRN: A Neural Network ShiftReduce Parser. In: Proc. of the 16th IJCAI, Stockholm, Sweden, pp. 820–825 (1999)
MacKay, D.J.C.: Bayesian methods for supervised neural networks. In: Arbib, M.A. (ed.) The Handbook of Brain Theory and Neural Networks, pp. 144–149. MIT Press, Cambridge (1998)
Palmer-Brown, D., Tepper, J., Powell, H.: Connectionist Natural Language Parsing. Trends in Cognitive Sciences 6(10), 437–442 (2002)
Palmer-Brown, D., Draganova, C., Lee, S.W.: Snap-Drift Neural Network for Selecting Student Feedback. In: Proc. IJCNN 2009, Atlanta, USA, pp. 391–398 (2009)
Palmer-Brown, D., Lee, S.W., Draganova, C., Kang, M.: Modal Learning Neural Networks. WSEAS Transactions on Computers 8(2), 222–236 (2009)
Palmer-Brown, D., Draganova, C.: Snap-Drift Self Organising Map. In: Diamantaras, K., Duch, W., Iliadis, L.S. (eds.) ICANN 2010, Part II. LNCS, vol. 6353, pp. 402–409. Springer, Heidelberg (2010)
Palmer-Brown, D., Draganova, C.: Recurrent Snap Drift Neural Network for Phrase Recognition. In: WCCI 2010 IEEE World Congress on Computational Intelligence, IJCNN 2010, Barcelona, Spain, pp. 3445–3449 (2010)
Payne, A., Brinkman, W.-P., Wilson, F.: Towards Effective Feedback in e-Learning Packages: The Design of a Package to Support Literature Searching, Referencing and Avoiding Plagiarism. In: Proceedings of HCI 2007 Workshop: Design, Use and Experience of e-Learning Systems, pp. 71–75 (2007)
Ritter, H., Kohonen, T.: Self-Organizing Semantic Maps. Biological Cybernetics 61, 241–254 (1989)
Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning representations by back-propagation errors. Nature 323, 533–536 (1986)
Rushton, J.N.: Natural Language Parsing using Simple Neural Networks. In: Proc. of MLMTA, Las Vegas, Nevada, pp. 137–141 (2003)
Tepper, J., Powell, H.M., Palmer-Brown, D.: A Corpus based Connectionist Architecture for Large scale Natural Language Parsing. Connection Science 14(2), 93–114 (2002)
Vapnik, V.: The Nature of Statistical Learning Theory. Springer, New York (1995)
Webb, A.R., Lowe, D.: A hybrid optimisation strategy for adaptive feed-forward layered newtorks. RSRE Memorandum 4193, Royal Signals and Radar Establishemnt, St Andrews Road, Malvern, UK (1988)
Widrow, B., Hoff, M.E.: Institute of Radio Engineers WESCON Convention Record, Adaptive switching circuits, pp. 96–104. Institute of Radio Engineers, New York (1960)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Palmer-Brown, D., Jayne, C. (2013). Self Organisation and Modal Learning: Algorithms and Applications. In: Bianchini, M., Maggini, M., Jain, L. (eds) Handbook on Neural Information Processing. Intelligent Systems Reference Library, vol 49. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-36657-4_11
Download citation
DOI: https://doi.org/10.1007/978-3-642-36657-4_11
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-36656-7
Online ISBN: 978-3-642-36657-4
eBook Packages: EngineeringEngineering (R0)