Abstract
The Reinforcement Learning theory is a powerful tool for building recognition systems. This theory has long been used in the construction of computational models of neural networks of the brain. However, the validity of its use for these purposes is not unequivocally recognized. One of the reasons for this is the significant differences between the variables used in the theory and the characteristics of the brain that determine behavioral choice. In this paper, the possibility of applying the theory of reinforcement learning for modeling the process of behavioral choice is evaluated. It is argued that insular cortex may be considered displaying state value of RL theory. Such an attempt turns out to be useful, as it allows us to formulate new questions concerning the way control structures interact and the nature of control in the brain, which in turn will allow us to make further progress in understanding the mechanisms of its work.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction, 2nd edn. MIT Press, Cambridge (2018)
Schultz, W.: Predictive reward signal of dopamine neurons. J. Neurophysiol. 80, 1–27 (1998)
Schultz, W.: Updating dopamine reward signals. Curr. Opin. Neurobiol. 23(2), 229–238 (2013)
Balleine, B.W., Daw, N.D., O’Doherty, J.P.: Multiple forms of value learning and the function of dopamine. In: Glimcher, P.W., Camerer, C.F., Poldrack, R.A., Fehr, E. (eds.) Neuroeconomics: Decision Making and the Brain. Academic Press, New York (2008)
Berridge, K.C.: From prediction error to incentive salience: mesolimbic computation of reward motivation. Eur. J. Neurosci. 35(7), 1124–1143 (2012)
Kusumoto-Yoshidaa, I., Liub, H., Chena, B.T., Fontaninib, A., Boncia, A.: Central role for the insular cortex in mediating conditioned responses to anticipatory cue. PNAS 112(4), 1190–1195 (2015)
Schiff, H.C., et al.: An insula–central amygdala circuit for guiding tastant-reinforced choice behavior. J. Neurosci. 38(6), 1418–1429 (2018)
Nomi, J.S., Schettini, E., Broce, I., Dick, A.S., Uddin, L.Q.: Structural connections of functionally defined human insular subdivisions. Cereb. Cortex 28, 3445–3456 (2018)
Sridharan, D., Levitin, D.J., Menon, V.: A critical role for the right fronto-insular cortex in switching between central-executive and default-mode networks. PNAS 105(34), 12569–12574 (2008)
Keay, K.A., Badler, R.: Periaqiaductal gray. In: The Rat Nervous System, 4th edn. (2015)
Quirk, G.J., Repa, C., LeDoux, J.E.: Fear conditioning enhances short-latency auditory responses of lateral amygdala neurons: parallel recordings in the freely behaving rat. Neuron 15, 1029–1039 (1995)
Berret, M., et al.: Insular cortex processes aversive somatosensory information and is crucial for threat learning. Science 364(6443) (2019)
Kim, J., Zhang, X., Muralidhar, S., LeBlanc, S.A., Tonegawa, S.: Basolateral to central amygdala neural circuits for appetitive behaviors. Neuron 93(6), 1464–1479 (2017)
Parkes, Sh.L., Bradfield, L.A., Balleine, B.W.: Interaction of insular cortex and ventral striatum mediates the effect of incentive memory on choice between goal-directed actions. J. Neurosci. 35(16), 6464–6471 (2015)
LeDoux, J.E.: Emotion circuits in the brain. Annu. Rev. Neurosci. 23, 155–184 (2000)
Davis, M.: The role of the amygdala in fear and anxiety. Annu. Rev. Neurosci. 15, 353–375 (1992)
Shi, C.-J., Cassell, M.D.: Cortical, thalamic, and amygdaloid connections of the anterior and posterior insular cortices. J. Comp. Neurol. 399, 440–468 (1998)
Allen, G.V., Saf’Er, C.B., Hurley, K.M., Cechetto, D.F.: Organization of visceral and limbic connections in the insular cortex of the rat. J. Comp. Neurol. 311, 1–16 (1991)
Cerliani, L., et al.: Probabilistic tractography recovers a rostrocaudal trajectory of connectivity variability in the human insular cortex. Hum. Brain Mapp. 33, 2005–2034 (2012)
Cai, W., Chen, T., Ryali, S., Kochalka, J., Li, C.-S., Menon, V.: Causal interactions within a frontal-cingulate-parietal network during cognitive control: convergent evidence from a multisite-multitask investigation. Cereb. Cortex 26, 2140–2153 (2016)
Funding
The research was done within the 2022 state task FNEF-2022-0003 Research into Neuromorphic Big-Data Processing Systems and Technologies of Their Creation.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Smirnitskaya, I.A. (2023). The Reinforcement Learning Theory, Value Function, and the Nature of Value Function Calculation by the Insular Cortex. In: Kryzhanovsky, B., Dunin-Barkowski, W., Redko, V., Tiumentsev, Y. (eds) Advances in Neural Computation, Machine Learning, and Cognitive Research VI. NEUROINFORMATICS 2022. Studies in Computational Intelligence, vol 1064. Springer, Cham. https://doi.org/10.1007/978-3-031-19032-2_25
Download citation
DOI: https://doi.org/10.1007/978-3-031-19032-2_25
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-19031-5
Online ISBN: 978-3-031-19032-2
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)