Abstract
Real-world decisions are rarely as straightforward as choosing between clearly “good” vs. “bad” options. More often, options must be evaluated carefully because they differ in relative value. For example, we typically learn about (and make decisions between) options in comparison, where one outcome may be more costly or risky than the other. Several neuropsychiatric conditions are characterized by atypical evaluation of effort and risk costs, including major depression, schizophrenia, autism, obsessive-compulsive disorder, and substance use disorders. Aberrant value learning and decision-making have long been considered a cognitive-behavioral endophenotype of these disorders and can be modeled in rodents. This chapter presents two general methodological domains that the experimenter can manipulate in animal decision-making tasks: risk and effort. Here, we present detailed methods of rodent tasks frequently employed within these domains: probabilistic reversal learning (PRL) and effort choice. These tasks recruit regions within rodent frontal cortex, the amygdala, and the striatum, and performance is heavily modulated by dopamine, making these assays highly valid measures in the study of behavioral and substance addictions, in particular.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Deisseroth K (2015) Optogenetics: 10 years of microbial opsins in neuroscience. Nat Neurosci 18:1213–1225
English JG, Roth BL (2015) Chemogenetics-a transformational and translational platform. JAMA Neurol 72:1361–1366
Cai DJ, Aharoni D, Shuman T, Shobe J, Biane J, Song W, Wei B, Veshkini M, La-Vu M, Lou J, Flores SE, Kim I, Sano Y, Zhou M, Baumgaertel K, Lavi A, Kamata M, Tuszynski M, Mayford M, Golshani P, Silva AJ (2016) A shared neural ensemble links distinct contextual memories encoded close in time. Nature 534:115–118
Izquierdo A, Brigman JL, Radke AK, Rudebeck PH, Holmes A (2017) The neural basis of reversal learning: an updated perspective. Neuroscience 345:12–26
Wassum KM, Izquierdo A (2015) The basolateral amygdala in reward learning and addiction. Neurosci Biobehav Rev 57:271–283
Dayan P, Daw ND (2008) Decision theory, reinforcement learning, and the brain. Cogn Affect Behav Neurosci 8:429–453
Addicott MA, Pearson JM, Sweitzer MM, Barack DL, Platt ML (2017) A primer on foraging and the explore/exploit trade-off for psychiatry research. Neuropsychopharmacology 42:1931–1939
McNamara JM, Fawcett TW, Houston AI (2013) An adaptive response to uncertainty generates positive and negative contrast effects. Science 340:1084–1086
Rangel A, Camerer C, Montague PR (2008) A framework for studying the neurobiology of value-based decision making. Nat Rev Neurosci 9:545–556
O’Leary JD, O’Leary OF, Cryan JF, Nolan YM (2018) A low-cost touchscreen operant chamber using a Raspberry Pi. Behav Res Methods 50(6):2523–2530
Izquierdo A, Belcher AM, Scott L, Cazares VA, Chen J, O’Dell SJ, Malvaez M, Wu T, Marshall JF (2010) Reversal-specific learning impairments after a binge regimen of methamphetamine in rats: possible involvement of striatal dopamine. Neuropsychopharmacology 35:505–514
Stolyarova A, Izquierdo A (2017) Complementary contributions of basolateral amygdala and orbitofrontal cortex to value learning under uncertainty. elife 6:e27483
Izquierdo A (2017) Functional heterogeneity within rat orbitofrontal cortex in reward learning and decision making. J Neurosci 37:10529–10540
Izquierdo A, Wiedholz LM, Millstein RA, Yang RJ, Bussey TJ, Saksida LM, Holmes A (2006) Genetic and dopaminergic modulation of reversal learning in a touchscreen-based operant procedure for mice. Behav Brain Res 171:181–188
Morton AJ, Skillings E, Bussey TJ, Saksida LM (2006) Measuring cognitive deficits in disabled mice using an automated interactive touchscreen system. Nat Methods 3:767
Salamone JD, Steinpreis RE, McCullough LD, Smith P, Grebel D, Mahan K (1991) Haloperidol and nucleus accumbens dopamine depletion suppress lever pressing for food but increase free food consumption in a novel food choice procedure. Psychopharmacology 104:515–521
Nowend KL, Arizzi M, Carlson BB, Salamone JD (2001) D1 or D2 antagonism in nucleus accumbens core or dorsomedial shell suppresses lever pressing for food but leads to compensatory increases in chow consumption. Pharmacol Biochem Behav 69:373–382
Salamone JD, Wisniecki A, Carlson BB, Correa M (2001) Nucleus accumbens dopamine depletions make animals highly sensitive to high fixed ratio requirements but do not impair primary food reinforcement. Neuroscience 105:863–870
Salamone JD, Correa M, Farrar A, Mingote SM (2007) Effort-related functions of nucleus accumbens dopamine and associated forebrain circuits. Psychopharmacology 191:461–482
Nunes EJ, Randall PA, Hart EE, Freeland C, Yohn SE, Baqi Y, Muller CE, Lopez-Cruz L, Correa M, Salamone JD (2013) Effort-related motivational effects of the VMAT-2 inhibitor tetrabenazine: implications for animal models of the motivational symptoms of depression. J Neurosci 33:19120–19130
Floresco SB, Ghods-Sharifi S (2007) Amygdala-prefrontal cortical circuitry regulates effort-based decision making. Cereb Cortex 17:251–260
Rudebeck PH, Walton ME, Smyth AN, Bannerman DM, Rushworth MF (2006) Separate neural pathways process different decision costs. Nat Neurosci 9:1161–1168
Walton ME, Bannerman DM, Alterescu K, Rushworth MF (2003) Functional specialization within medial frontal cortex of the anterior cingulate for evaluating effort-related decisions. J Neurosci 23:6475–6479
Walton ME, Bannerman DM, Rushworth MF (2002) The role of rat medial frontal cortex in effort-based decision making. J Neurosci 22:10996–11003
Walton ME, Rudebeck PH, Bannerman DM, Rushworth MF (2007) Calculating the cost of acting in frontal cortex. Ann N Y Acad Sci 1104:340–356
Hart EE, Gerson JO, Zoken Y, Garcia M, Izquierdo A (2017) Anterior cingulate cortex supports effort allocation toward a qualitatively preferred option. Eur J Neurosci 46(1):1682–1688
Hart EE, Izquierdo A (2017) Basolateral amygdala supports the maintenance of value and effortful choice of a preferred option. Eur J Neurosci 45:388–397
Ostrander S, Cazares VA, Kim C, Cheung S, Gonzalez I, Izquierdo A (2011) Orbitofrontal cortex and basolateral amygdala lesions result in suboptimal and dissociable reward choices on cue-guided effort in rats. Behav Neurosci 125:350–359
Stolyarova A, Thompson AB, Barrientos RM, Izquierdo A (2015) Reductions in frontocortical cytokine levels are associated with long-lasting alterations in reward valuation after methamphetamine. Neuropsychopharmacology 40:1234–1242
Winstanley CA, Floresco SB (2016) Deciphering decision making: variation in animal models of effort- and uncertainty-based choice reveals distinct neural circuitries underlying core cognitive processes. J Neurosci 36:12069–12079
Izquierdo A, Pozos H, Torre Ade L, DeShields S, Cevallos J, Rodriguez J, Stolyarova A (2016) Sex differences, learning flexibility, and striatal dopamine D1 and D2 following adolescent drug exposure in rats. Behav Brain Res 308:104–114
Stolyarova A, Izquierdo A (2015) Distinct patterns of outcome valuation and amygdala-prefrontal cortex synaptic remodeling in adolescence and adulthood. Front Behav Neurosci 9:115
Kosheleff AR, Rodriguez D, O’Dell SJ, Marshall JF, Izquierdo A (2012) Comparison of single-dose and extended methamphetamine administration on reversal learning in rats. Psychopharmacology 224:459–467
Randall PA, Pardo M, Nunes EJ, Lopez Cruz L, Vemuri VK, Makriyannis A, Baqi Y, Muller CE, Correa M, Salamone JD (2012) Dopaminergic modulation of effort-related choice behavior as assessed by a progressive ratio chow feeding choice task: pharmacological studies and the role of individual differences. PLoS One 7:e47934
Soltani A, Noudoost B, Moore T (2013) Dissociable dopaminergic control of saccadic target selection and its implications for reward modulation. Proc Natl Acad Sci U S A 110:3579–3584
Hosking JG, Cocker PJ, Winstanley CA (2014) Dissociable contributions of anterior cingulate cortex and basolateral amygdala on a rodent cost/benefit decision-making task of cognitive effort. Neuropsychopharmacology 39:1558–1567
Hosking JG, Floresco SB, Winstanley CA (2015) Dopamine antagonism decreases willingness to expend physical, but not cognitive, effort: a comparison of two rodent cost/benefit decision-making tasks. Neuropsychopharmacology 40:1005–1015
Hosking JG, Lam FC, Winstanley CA (2014) Nicotine increases impulsivity and decreases willingness to exert cognitive effort despite improving attention in “slacker” rats: insights into cholinergic regulation of cost/benefit decision making. PLoS One 9:e111580
Randall PA, Pardo M, Nunes EJ, López Cruz L, Vemuri VK, Makriyannis A, Baqi Y, Müller CE, Correa M, Salamone JD (2012) Dopaminergic modulation of effort-related choice behavior as assessed by a progressive ratio chow feeding choice task: pharmacological studies and the role of individual differences. PLoS One 7:e47934
Thompson AB, Gerson J, Stolyarova A, Bugarin A, Hart EE, Jentsch JD, Izquierdo A (2017) Steep effort discounting of a preferred reward over a freely-available option in prolonged methamphetamine withdrawal in male rats. Psychopharmacology 234:2697–2705
Acknowledgments
This research was supported by the UCLA Academic Senate Grant and the UCLA Division of Life Sciences Recruitment and Retention Fund.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Science+Business Media, LLC, part of Springer Nature
About this protocol
Cite this protocol
Izquierdo, A., Aguirre, C., Hart, E.E., Stolyarova, A. (2019). Rodent Models of Adaptive Value Learning and Decision-Making. In: Kobeissy, F. (eds) Psychiatric Disorders. Methods in Molecular Biology, vol 2011. Humana, New York, NY. https://doi.org/10.1007/978-1-4939-9554-7_7
Download citation
DOI: https://doi.org/10.1007/978-1-4939-9554-7_7
Published:
Publisher Name: Humana, New York, NY
Print ISBN: 978-1-4939-9553-0
Online ISBN: 978-1-4939-9554-7
eBook Packages: Springer Protocols