Abstract
The inclusion of a macroscopic adaptive threshold is studied for the retrieval dynamics of layered feedforward neural network models with synaptic noise. It is shown that if the threshold is chosen appropriately as a function of the cross-talk noise and of the activity of the stored patterns, adapting itself automatically in the course of the recall process, an autonomous functioning of the network is guaranteed. This self-control mechanism considerably improves the quality of retrieval, in particular the storage capacity, the basins of attraction and the mutual information content.
Access provided by Autonomous University of Puebla. Download to read the full chapter text
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Willshaw, D.J., Buneman, O.P., Longuet-Higgins, H.C.: Nonholographic associative memory. Nature 222, 960 (1969)
Palm, G.: On the storage capacity of an associative memory with random distributed storage elements. Biol. Cyber. 39, 125 (1981)
Gardner, E.: The space of interactions in neural network models. J. Phys. A: Math. Gen. 21, 257 (1988)
Okada, M.: Notions of associative memory and sparse coding. Neural Networks 9, 1429 (1996)
Dominguez, D.R.C., Bollé, D.: Self-control in sparsely coded networks. Phys. Rev. Lett. 80, 2961 (1998)
Bollé, D., Dominguez, D.R.C., Amari, S.: Mutual information of sparsely coded associative memory with self-control and ternary neurons. Neural Networks 13, 455 (2000)
Bollé, D., Heylen, R.: Self-control dynamics for sparsely coded networks with synaptic noise. In: 2004 Proceedings of the IEEE International Joint Conference on Neural Networks, p. 3195 (2004)
Dominguez, D.R.C., Korutcheva, E., Theumann, W.K., Erichsen Jr., R.: Flow diagrams of the quadratic neural network. In: Dorronsoro, J.R. (ed.) ICANN 2002. LNCS, vol. 2415, p. 129. Springer, Heidelberg (2002)
Bollé, D., Massolo, G.: Thresholds in layered neural networks with variable activity. J. Phys. A: Math. Gen. 33, 2597 (2000)
Hertz, J., Krogh, A., Palmer, R.G.: Introduction to the Theory of Neural Computation. Addison-Wesley, Redwood City (1991)
Domany, E., Kinzel, W., Meir, R.: Layered Neural Networks. J.Phys. A: Math. Gen. 22, 2081 (1989)
Bollé, D.: Multi-state neural networks based upon spin-glasses: a biased overview. In: Korutcheva, E., Cuerno, R. (eds.) Advances in Condensed Matter and Statistical Mechanics, pp. 321–349. Nova Science Publishers, New-York (2004)
Nadal, J.-P., Brunel, N., Parga, N.: Nonlinear feedforward networks with stochastic outputs: infomax implies redundancy reduction. Network: Computation in Neural Systems 9, 207 (1998)
Schultz, S., Treves, A.: Stability of the replica-symmetric solution for the information conveyed by a neural network. Phys. Rev. E 57, 3302 (1998)
Blahut, R.E.: Principles and Practice of Information Theory. Addison-Wesley, Reading (1990)
Shannon, C.E.: A mathematical theory for communication. Bell Systems Technical Journal 27, 379 (1948)
Schwenker, F., Sommer, F.T., Palm, G.: Iterative retrieval of sparsely coded associative memory patterns. Neural Networks 9, 445 (1996)
Amari, S.: Neural theory and association of concept information. Biol. Cyber. 26, 175 (1977)
Amari, S., Maginu, K.: Statistical neurodynamics of associative memory. Neural Networks 1, 63 (1988)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Bollé, D., Heylen, R. (2006). Adaptive Thresholds for Layered Neural Networks with Synaptic Noise. In: Kollias, S.D., Stafylopatis, A., Duch, W., Oja, E. (eds) Artificial Neural Networks – ICANN 2006. ICANN 2006. Lecture Notes in Computer Science, vol 4131. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11840817_71
Download citation
DOI: https://doi.org/10.1007/11840817_71
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-38625-4
Online ISBN: 978-3-540-38627-8
eBook Packages: Computer ScienceComputer Science (R0)