Abstract
The human brain processes information showing learning and prediction abilities but the underlying neuronal mechanisms still remain unknown. Recently, many studies prove that neuronal networks are able of both generalizations and associations of sensory inputs.
In this paper, following a set of neurophysiological evidences, we propose a learning framework with a strong biological plausibility that mimics prominent functions of cortical circuitries. We developed the Inductive Conceptual Network (ICN), that is a hierarchical bio-inspired network, able to learn invariant patterns by Variable-order Markov Models implemented in its nodes. The outputs of the top-most node of ICN hierarchy, representing the highest input generalization, allow for automatic classification of inputs. We found that the ICN clusterized MNIST images with an error of 5.73% and USPS images with an error of 12.56%.
Access provided by Autonomous University of Puebla. Download to read the full chapter text
Chapter PDF
Similar content being viewed by others
References
Bassett, D.S., Greenfield, D.L., Meyer-Lindenberg, A., Weinberger, D.R., Moore, S.W., Bullmore, E.T.: Efficient physical embedding of topologically complex information processing networks in brains and computer circuits. PLoS Computational Biology 6(4), e1000748 (2010); Friston, K.J. (ed.), Public Library of Science, doi:10.1371/journal.pcbi.1000748
DiCarlo, J., Zoccolan, D., Rust, N.: How Does the Brain Solve Visual Object Recognition? Neuron 73, 415–434 (2012)
Takahashi, N., Kitamura, K., Matsuo, N., Mayford, M., Kano, M., Matsuki, N., Ikegaya, Y.: Locally Synchronized Synaptic Inputs. Science 335, 353–356 (2012)
Kleindienst, T., Winnubst, J., Roth-Alpermann, C., Bonhoeffer, T., Lohmann, C.: Activity-Dependent Clustering of Functional Synaptic Inputs on Developing Hippocampal Dendrites. Neuron 72(6), 1012–1024 (2011)
Makino, H., Malinow, R.: Compartmentalized versus Global Synaptic Plasticity on Dendrites Controlled by Experience. Neuron 72(6), 1001–1011 (2011)
Meyers, E.M., Qi, X.L., Constantinidis, C.: Incorporation of new information into prefrontal cortical activity after learning working memory tasks. Proc. Natl. Acad. Sci. USA (2012), doi:10.1073/pnas.1201022109
Branco, T., Clark, B.A., Hausser, M.: Dendritic Discrimination of Temporal Input Sequences in Cortical Neurons. Science 329(5999), 1671–1675 (2010)
Bühlmann, P., Wyner, A.J.: Variable Length Markov Chains. The Annals of Statistics 27(2), 480–513 (1999)
Begleiter, R., El-Yaniv, R., Yona, G.: On Prediction Using Variable Order Markov Models. Journal of Artificial Intelligence Research 22, 385–421 (2004)
Hawkins, J., Blackeslee, S.: On Intellingence. Times Books, New York (2004)
Dileep, G., Hawkins, J.: Towards a Mathematical Theory of Cortical Micro-circuits. PLoS Comput. Biol. 5(10), e1000532 (2009)
Baum, L.E., Petrie, T.: Statistical Inference for Probabilistic Functions of Finite State Markov Chains. The Annals of Mathematical Statistics 37(6), 1554–1563 (1966)
Baum, L.E., Petrie, T., Soules, G., Weiss, N.: A Maximization Technique Occurring in the Statistical Analysis of Probabilistic Functions of Markov Chains. The Annals of Mathematical Statistics 41, 164 (1970)
Rissanen, J.: Universal coding, information, prediction, and estimation. IEEE Transactions on Information Theory 30(4), 629–636 (1984)
Cleary, J., Witten, I.: Data Compression Using Adaptive Coding and Partial String Matching. IEEE Transactions on Communications 32(4), 396–402 (1984)
Teahan, W.: Probability estimation for PPM. In: Proceedings of the New Zealand Computer Science Research Students’ Conference. University of Waikato, Hamilton (1995)
Hamming, R.W.: Error detecting and error correcting codes. Bell System Technical Journal 29(2), 147–160 (1950)
Hebb, D.O.: The organization of behavior; a neuropsychological theory. Wiley, New York (1949)
Hubel, D., Wiesel, T.: Receptive fields and functional architecture of monkey striate cortex. Journal of Physiology 195, 215–243 (1968)
LeCun, Y., Boser, B., Denker, J.S., et al.: Handwritten Digit Recognition with a Back-Propagation Network. In: Touretzky, D. (ed.) Advances in Neural Information Processing Systems 2 (NIPS 1989). Morgan Kaufman, Denver (1990)
LeCun, Y., Bengio, Y.: Convolutional networks for images, speech, and time-series. In: Arbib, M.A. (ed.) The Handbook of Brain Theory and Neural Networks. MIT Press (1995)
Powell, W.B.: Approximate Dynamic Programming: Solving the Curse of Dimensionality. Wiley, New York (2007)
Kotsiantis, S., Kanellopoulos, D., Pintelas, P.: Data Preprocessing for Supervised Leaning. International Journal of Computer Science 1(2), 111–117 (2006)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Zippo, A.G., Gelsomino, G., Nencini, S., Biella, G.E.M. (2013). Handwritten Digits Recognition by Bio-inspired Hierarchical Networks. In: Apolloni, B., Bassis, S., Esposito, A., Morabito, F. (eds) Neural Nets and Surroundings. Smart Innovation, Systems and Technologies, vol 19. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-35467-0_20
Download citation
DOI: https://doi.org/10.1007/978-3-642-35467-0_20
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-35466-3
Online ISBN: 978-3-642-35467-0
eBook Packages: EngineeringEngineering (R0)