Abstract
Animals anticipate the future in a variety of ways. For instance: (a) they make motor actions that are timed to a reference stimulus and motor actions that anticipate future movement dynamics; (b) they learn to make choices that will maximize reward they receive in the future; and (c) they form memories of behavioral episodes such that the animal’s future actions can be predicted by current neural activity associated with those memories. Although these effects are clearly observable at the behavioral level, research into the mechanisms of such anticipatory learning are still largely in the early stages. This review, intended for those who have a computational background and are less familiar with neuroscience, addresses neural mechanisms found in the mammalian cerebellum, basal ganglia, and the hippocampus that give rise to such adaptive anticipatory behavior.
Access provided by Autonomous University of Puebla. Download to read the full chapter text
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Aldridge, J.W., Berridge, K.C.: Coding of serial order by neostriatal neurons: a natural action approach to movement sequence. J Neurosci 18, 2777–2787 (1998)
Bastian, A.J.: Learning to predict the future: the cerebellum adapts feedforward movement control. Curr. Opin. Neurobiol 16, 645–649 (2006)
Buhusi, C.V., Meck, W.H.: What makes us tick? functional and neural mechanisms of interval timing. Nat. Rev. Neurosci 6, 755–765 (2005)
Butz, M.V., Siguad, O., Gerard, P.: Anticipatory behavior: Exploiting knowledge about the future to improve current behavior. In: Butz, M.V., Sigaud, O., Gérard, P. (eds.) Anticipatory Behavior in Adaptive Learning Systems. LNCS (LNAI), vol. 2684, pp. 1–10. Springer, Heidelberg (2003)
Clayton, N.S., Bussey, T.J., Dickinson, A.: Can animals recall the past and plan for the future? Nat. Rev. Neurosci 4, 685–691 (2003)
Doya, K.: Complementary roles of basal ganglia and cerebellum in learning and motor control. Curr. Opin. Neurobiol 10, 732–739 (2000)
Ferbinteanu, J., Shapiro, M.L.: Prospective and retrospective memory coding in the hippocampus. Neuron 40, 1227–1239 (2003)
Fleischer, J., Marsland, S., Shapiro, J.: Sensory anticipation for autonomous selection of robot landmarks. In: Butz, M.V., Sigaud, O., Gérard, P. (eds.) Anticipatory Behavior in Adaptive Learning Systems.Foundations,Theories, and Systems. LNCS (LNAI), vol. 2684, Springer, Heidelberg (2003)
Fleischer, J.G., Gally, J.A., Edelman, G.M., Krichmar, J.L.: Retrospective and prospective responses arising in a modeled hippocampus during maze navigation by a brain-based device. Proc. Natl. Acad. Sci. USA 104, 3556–3561 (2007)
Frank, L.M., Brown, E.N., Wilson, M.A.: Trajectory encoding in the hippocampus and entorhinal cortex. Neuron 27, 169–178 (2000)
Griffiths, D., Dickenson, A., Clayton, N.: Episodic memory: what can animals remember about their past? Trends in Cognitive Science 3, 74–80 (1999)
Gurney, K., Prescott, T.J., Wickens, J.R., Redgrave, P.: Computational models of the basal ganglia: from robots to membranes. Trends in Neurosciences 27, 453–459 (2004)
Houk, J., Adams, J., Barto, A.: A model of how the basal ganglia generate and use neural signals that predict reinforcement. In: Models of Information Processing in the Basal Ganglia, MIT Press, Cambridge (1995)
Ivry, R.B., Keele, S.W., Diener, H.C.: Dissociation of the lateral and medial cerebellum in movement timing and movement execution. Exp. Brain Res. 73, 167–180 (1988)
Ivry, R.B., Spencer, R.M.C.: The neural representation of time. Curr. Opin. Neurobiol 14, 225–232 (2004)
Joel, D., Niv, Y., Ruppin, E.: Actor-critic models of the basal ganglia: new anatomical and computational perspectives. Neural Networks 15, 535–547 (2002)
Kandel, E., Schwartz, J., Jessell, T. (eds.): Principles of Neural Science, 4th edn. McGraw-Hill, New York (2000)
Kaplan, F., Oudeyer, P.Y.: Maximizing learning progress: An internal reward system for development. In: Iida, F., Pfeifer, R., Steels, L., Kuniyoshi, Y. (eds.) Embodied Artificial Intelligence. LNCS, vol. 3139, pp. 259–270. Springer, Heidelberg (2004)
Krichmar, J., Seth, A., Nitz, D., Fleischer, J., Edelman, G.: Spatial navigation and causal analysis in a brain-based device modeling cortical-hippocampal interactions. Neuroinformatics 3, 197–221 (2005)
Lisman, J.: The theta/gamma discrete phase code occuring during the hippocampal phase precession may be a more general brain coding scheme. Hippocampus 15, 913–922 (2005)
Maguire, E.A., Woollett, K., Spiers, H.J.: London taxi drivers and bus drivers: A structural MRI and neuropsychological analysis. Hippocampus 16, 1091–1101 (2006)
Marr, D.: A theory of cerebellar cortex. J. Phsysiol. 202, 437–470 (1969)
Mauk, M.D., Buonomano, D.V.: The neural basis of temporal processing. Annu. Rev. Neurosci 27, 307–340 (2004)
McCormick, D.A., Thompson, R.F.: Neuronal responses of the rabbit cerebellum during acquisition and performance of a classically conditioned nictitating membrane-eyelid response. J Neurosci 4, 2811–2822 (1984)
McEchron, M.D., Tseng, W., Disterhoft, J.F.: Single neurons in ca1 hippocampus encode trace interval duration during trace heart rate (fear) conditioning in rabbit. Journal of Neuroscience 23, 1535–1547 (2003)
McHaffie, J.G., Stanford, T.R., Stein, B.E., Coizet, V., Redgrave, P.: Subcortical loops through the basal ganglia. Trends Neurosci 28, 401–407 (2005)
Miall, R.C., Reckess, G.Z.: The cerebellum and the timing of coordinated eye and hand tracking. Brain Cogn. 48, 212–226 (2002)
Miall, R.C., Reckess, G.Z., Imamizu, H.: The cerebellum coordinates eye and hand tracking movements. Nat. Neurosci 4, 638–644 (2001)
Montague, P.R., Berns, G.S.: Neural economics and the biological substrates of valuation. Neuron 36, 265–284 (2002)
Moskovitch, M., Nadel, L., Winocur, G., Gilboa, A., Rosenbaum, R.S.: The cognitive neuroscience of remote episodic, semantic and spatial memory. Current Opinion in Neurobiology 16, 179–190 (2006)
O’Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Friston, K., Dolan, R.J.: Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science 304, 452–454 (2004)
O’Keefe, J., Dostrovsky, J.: The hippocampus as a spatial map. preliminary evidence from unit activity in the freely-moving rat. Brain Research 34, 171–175 (1971)
O’Keefe, J., Nadel, L.: The Hippocampus as a Cognitive Map. Clarendon Press, Oxford (1978)
O’Keefe, J., Recce, M.L.: Phase relationship between hippocampal place units and the eeg theta rhythm. Hippocampus 3, 317–330 (1993)
O’Reilly, R.C., Frank, M.J.: Making working memory work: a computational model of learning in the prefrontal cortex and basal ganglia. Neural Computation 18, 283–328 (2006)
Redgrave, P., Prescott, T.J., Gurney, K.: Is the short-latency dopamine response too short to signal reward error? Trends in Neurosciences 22, 146–151 (1999)
Redish, A.D.: Beyond the cognitive map: From place cells to episodic memory. MIT Press, Cambridge (1999)
Rescorla, R., Wagner, A.: A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. In: Classical Conditioning II. Appleton-Century-Crofts (1972)
Roberts, A., Robbins, T., Weiskrantz, L. (eds.): The Prefrontal Cortex: Executive and Cognitive Functions. Oxford University Press, Oxford, UK (1998)
Schmidhuber, J.: Curious model-building control systems. In: IEEE Intl Joint Conf. on Neural Networks. vol.2, pp. 1458–1463 (1991)
Schultz, W., Dayan, P., Montague, P.R.: A neural substrate of prediction and reward. Science 275, 1593–1599 (1997)
Schultz, W.: Getting formal with dopamine and reward. Neuron 36, 241–263 (2002)
Schultz, W.: Neural coding of basic reward terms of animal learning theory, game theory, microeconomics and behavioural ecology. Curr. Opin. Neurobiol 14, 139–147 (2004)
Scoville, W.B., Milner, B.: Loss of recent memory after bilateral hippocampal lesions. J. Neurochem. 20 (1957)
Shadmer, R., Wise, S.: The computational neurobiology of reaching and pointing: a foundation for motor learning. MIT Press, Cambridge (2005)
Squire, L.R., Stark, C.E.L., Clark, R.E.: The medial temporal lobe. Annu. Rev. Neurosci 27, 279–306 (2004)
Sutton, R.S., Barto, A.G.: Reinforcement learning: an introduction. MIT Press, Cambridge (1998)
Thompson, R.F.: In search of memory traces. Annual Review of Psychology 56, 1–23 (2005)
Thorndike, E.: Animal Intelligence. Macmillan Press (1911)
Waelti, P., Dickinson, A., Schultz, W.: Dopamine responses comply with basic assumptions of formal learning theory. Nature 412, 43–48 (2001)
Wolpert, D.M., Ghahramani, Z.: Computational principles of movement neuroscience. Nat. Neurosci 3, 1212–1217 (2000)
Wood, E.R., Dudchenko, P.A., Robitsek, R.J., Eichenbaum, H.: Hippocampal neurons encode information about different types of memory episodes occurring in the same location. Neuron 27, 623–633 (2000)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Fleischer, J.G. (2007). Neural Correlates of Anticipation in Cerebellum, Basal Ganglia, and Hippocampus. In: Butz, M.V., Sigaud, O., Pezzulo, G., Baldassarre, G. (eds) Anticipatory Behavior in Adaptive Learning Systems. ABiALS 2006. Lecture Notes in Computer Science(), vol 4520. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-74262-3_2
Download citation
DOI: https://doi.org/10.1007/978-3-540-74262-3_2
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-74261-6
Online ISBN: 978-3-540-74262-3
eBook Packages: Computer ScienceComputer Science (R0)