Keywords

1 Introduction

It has been known for some time that mesolimbic dopamine (DA) and its main projection target, the nucleus accumbens , are involved in aspects of motivation . Motivation has been defined in many ways, but for the purposes of this chapter, it is defined as the processes that enable organisms to regulate the proximity, probability, and availability of stimuli (Salamone 1992). Mogenson et al. (1980) considered nucleus accumbens to be a limbic–motor interface, at which brain areas processing information related to motivation and emotion can gain access to the basal ganglia motor system circuitry (Floresco 2015). In addition, mesolimbic DA appears to be a key part of the neural circuitry regulating phenomena such as intracranial self-stimulation and self-administration of drugs, especially stimulants (Wise 2008), though the specific processes being affected by dopaminergic manipulations are still a matter of some debate (e.g., Hernandez et al. 2010; Venugopalan et al. 2011).

Similarly, with natural reinforcers such as food, the question is not whether accumbens DA is involved in motivation , but how. In other words—which aspects of motivation require intact DA transmission? Mesolimbic DA neurons are activated during motivationally relevant situations, and interference with accumbens DA transmission can affect aspects of motivated behavior. Nevertheless, it should be recognized that motivation is a complex process involving multiple interacting functions mediated by various neural circuits. Classically, motivational theory has emphasized two distinct aspects of motivation : directional and activational aspects (Duffy 1963; Cofer and Apley 1964; Salamone 1988). Thus, behavior is said to be directed toward some stimuli (food, water, sex) and away from others (painful stimuli, predators). In addition to being directed toward or away from significant stimuli (see Chapter by Cornwell, Franks & Higgins), motivated behavior also is characterized by a high degree of behavioral activation , as demonstrated by the speed, vigor or persistence seen in both the instigation and maintenance of instrumental responding (Salamone 1988, 1992; Salamone and Correa 2002, 2012; Robbins and Everitt 2007; Croxson et al. 2009; Kurniawan et al. 2010; Nicola 2010; McGinty et al. 2013; Floresco 2015). As discussed below, considerable evidence indicates that interference with accumbens DA transmission can substantially affect activational aspects of motivation for food and other natural reinforcers, while leaving fundamental features of primary motivation intact (e.g., appetite, primary food motivation; see Salamone and Correa 2002, 2012). These findings have implications for understanding the neural circuitry underlying both normal and pathological aspects of motivation.

2 Dynamic Activity of DA Neurons: Multiple Modes of Responding

One way of studying the behavioral functions of DA systems is to investigate the responsiveness of these systems across an array of behaviorally relevant conditions. The dynamic activity of the mesolimbic DA system has been studied using a variety of methods, employing different neural markers and covering distinct time scales. Although mesolimbic DA neurons are sometimes referred to under the blanket term “reward neurons,” and it is common to see the mesolimbic DA pathway referred to as “the reward system,” it is clear from the literature that this is a gross oversimplification. For example, in experienced animals, the dopaminergic response in accumbens to primary food reinforcement or simple access to food is minimal or absent; this is seen as the lack of population response of ventral tegmental area (VTA) DA neurons in monkeys trained on a discrete trial discrimination task (Schultz et al. 1993), the lack of response to sucrose reinforcement in trained animals as measured by voltammetry (Roitman et al. 2004), the habituation of the extracellular DA response to palatable food (Bassareo and DiChiarra 1999a, b; Bassareo et al. 2002), and the lack of responsiveness to food in experienced animals as measured by microdialysis (Salamone et al. 1994b; Segovia et al. 2012). There appears to be general agreement that mesolimbic DA neurons respond to conditioned stimuli (Schultz et al. 1993; Schultz 2010). Moreover, microdialysis studies show that DA release and DA-related signal transduction is increased in nucleus accumbens during the performance of food-reinforced instrumental behavior (McCullough et al. 1993a; Salamone et al. 1994b; Sokolowski et al. 1998; Cousins et al. 1999; Roitman et al. 2004; Ostlund et al. 2011; Segovia et al. 2011, 2012). Finally, there is the frequently cited finding that the response of DA neurons provides a teaching signal that allows for the determination of reward prediction errors (Schultz et al. 1997; Bayer and Glimcher 2002; Niv 2009; Schultz 2010; Steinberg et al. 2013). The latter effect has led to a large number of physiological, computational, and theoretical papers.

Nevertheless, it has been evident for some time that DA neuron activity and accumbens DA release respond in a variety of different modes to a wide array of stimuli (Salamone and Correa 2012; Marinelli and McCutcheon 2014; Lammel et al. 2014). For example, electrophysiological and voltammetric studies have shown that putative VTA DA neurons can show increased responsiveness to stressful or aversive stimuli such as restraint stress (Anstrom and Woodward 2005), footshock (Brischoux et al. 2009), social defeat stress (Anstrom et al. 2009), and tail pinch (Zweifel et al. 2011; Budygin et al. 2012). In terms of responsiveness to aversive stimuli, there is heterogeneity based upon anatomical loci of the cell somata and the terminal projections (Brischoux et al. 2009; Roeper 2013; Lammel et al. 2014). Nevertheless, these electrophysiology and voltammetry studies are broadly consistent with the array of microdialysis reports showing increased extracellular DA in response to stressful or aversive conditions (McCullough and Salamone 1992; McCullough et al. 1993b; Salamone 1994, 1996; Tidey and Mizcek 1996; Young 2004). Thus, it is clear that one should not assume that all DA neurons are simply “reward neurons,” nor should one link in a simple way enhanced DA release to subjective pleasure. Rather, it appears that activation of VTA DA neurons in specific subcircuits, and the release of DA in specific accumbens subregions, participates in a variety of motivational and learning processes in a rather complex manner that has yet to be fully characterized. For example, recent evidence indicates that phasic DA release in nucleus accumbens was related to preference for specific outcomes in choice procedures, but only when it was imbedded into the context of action selection (Saddoris et al. 2015). It seems likely that activation of mesolimbic DA transmission promotes the instigation of instrumental actions, modulates the vigor of these responses, increases energy expenditure, guides value-based action selection, and provides signals that are important for aspects of neuroplasticity involved in Pavlovian and instrumental learning.

3 Behavioral Manifestations of Interference with Accumbens DA Transmission: Dissociation of Distinct Components of Motivation and Reinforcement

Important though they are, studies of the dynamic activity of DA neurons only tell part of the story. Another critical part, which is the focus of the present chapter, is the behavioral manifestations of impaired DA transmission. Several decades ago, it was recognized that high doses of DA antagonists, or whole forebrain DA depletions, could reduce food intake (Ungerstedt 1971; Zigmond and Stricker 1972). However, evidence gradually accumulated demonstrating that this effect was due largely to the motoric effects of impaired DA transmission in the neostriatum (i.e., caudate/putamen, or dorsal striatum), and in particular, the lateral or ventrolateral neostriatum (Dunnett and Iversen 1982; Salamone et al. 1990, 1993; Bakshi and Kelley 1991). In terms of nucleus accumbens DA , several lines of evidence indicate that the impact of DA depletions or antagonism in that terminal region have minimal impact on food intake (Salamone and Correa 2002, 2009; Baldo and Kelley 2007). Ungerstedt (1971) and Koob et al. (1978) showed that selective depletions of nucleus accumbens DA did not reduce food intake. Salamone et al. (1993) showed that accumbens DA depletions induced by 6-hydroxydopamine did not reduce food intake, nor did they alter food handling or feeding rate. In DA-deficient mice, restoration of feeding behavior occurred after viral rescue of DA transmission in neostriatum, but not nucleus accumbens (Szczpka et al. 2001). Baldo et al. (2002) injected D1 and D2 family antagonists into core and shell subregions of the accumbens and found that doses of these drugs that suppressed locomotion failed to alter food intake.

Several lines of evidence also show that nucleus accumbens DA does not directly mediate hedonic reactivity to food stimuli. An enormous collection of studies from Berridge and colleagues has demonstrated that systemic administration of DA antagonists, as well DA depletions in whole forebrain or nucleus accumbens , does not blunt appetitive taste reactivity for food, which is a widely used measure of hedonic reactivity to sweet solutions (Berridge and Robinson 1998, 2003; Berridge 2007; Berridge and Kringelbach 2015; Robinson et al., this volume). DA D2 receptors in the shell subregion of the accumbens regulate aversive taste reactivity, and brainstem D2 receptor stimulation was shown to suppress sucrose consumption, but neither group of receptors mediated the hedonic display of taste (Sederholm et al. 2002). Furthermore, knockdown of the DA transporter (Peciña et al. 2003) and microinjections of amphetamine into nucleus accumbens (Smith et al. 2011), both of which elevate extracellular DA , failed to enhance appetitive taste reactivity for sucrose. In fact, some of the non-dopaminergic neurobiological processes that underlie hedonic reactions to appetitive rewards have been identified (for more information, see Robinson et al., this volume).

Another process that is sometimes assumed to be associated with nucleus accumbens DA transmission is reinforcement learning. But here again, the literature has yielded a complex set of findings. Reports indicating that intra-accumbens injections of stimulant drugs such as amphetamine can support self-administration, or that optogenetic stimulation of DA neurons can be reinforcing (Ilango et al. 2014; Steinberg et al. 2014), do not necessarily mean that accumbens DA release acts primarily as a “reinforcement system” that stamps in instrumental learning related to natural stimuli. As discussed previously, such a reinforcing effect could be an emergent property that results from the modulation of the various channels of information passing through the nucleus accumbens , including the enhancement of the impact of environmental cues (Salamone et al. 2005; Everitt and Robbins 2005; Steinberg et al. 2014). As noted by Ilango et al. (2014), a major outcome resulting from the phasic activation of DA neurons by optogenetic stimulation is the transient instigation of conditioned approach behavior, which is consistent with an earlier optogenetic study (Adamantidis et al. 2011). Selective genetic inactivation of NMDA receptors, which blunted burst firing in VTA DA neurons, impaired the acquisition of cue-dependent appetitive learning but did not disrupt the acquisition of responding on a progressive ratio schedule (Zweifel et al. 2009). Moreover, the research that has specifically focused on the potential role of striatal areas in mediating action–outcome associations suggests that neostriatum, rather than nucleus accumbens , is more critically involved (Yin et al. 2005; Corbit and Janak 2010; Corbit et al. 2013). In their comprehensive review of this literature, Yin et al. (2008) stated that “the accumbens is neither necessary nor sufficient for instrumental learning” (p. 1439). Although there are many studies showing that cell body lesions, DA antagonists, or DA depletions can affect the learning-related outcomes in procedures such as place preference, acquisition of lever pressing, or other procedures, this does not directly demonstrate that accumbens neurons or mesolimbic DA transmission is essential for the action–outcome associations that are the basis of instrumental learning (Yin et al. 2008; Belin et al. 2009; Salamone and Correa 2012). Specific processes related to associative aspects of instrumental learning can be demonstrated by assessments of the effects of reinforcer devaluation or contingency degradation, which often are not conducted in pharmacology or lesion studies. These procedures assess the behavioral effects of reducing reinforcement value (e.g., pre-feeding or lacing food with quinine), or disrupting the response-reinforcer contingency [e.g., non-contingent presentation of the reinforcer; see Yin et al. (2008) for more details]. Thus, it is important to note that cell body lesions in either core or shell of the accumbens did not alter sensitivity to contingency degradation (Corbit et al. 2001). Furthermore, Lex and Hauber (2010) found that rats with nucleus accumbens DA depletions were still sensitive to reinforcer devaluation, indicating that accumbens core DA does not appear to be crucial for encoding action-outcome associations.

Considerable evidence indicates that accumbens DA is important for Pavlovian approach and Pavlovian-to-instrumental transfer (PIT; Parkinson et al. 2002; Wyvell and Berridge 2000; Dalley et al. 2005; Lex and Hauber 2008, 2010; Yin et al. 2008). PIT is a behavioral process that reflects the impact of Pavlovian-conditioned stimuli (CS) on instrumental responding. For example, presentation of a Pavlovian CS paired with food can increase output of food-reinforced instrumental behaviors, such as lever pressing. Outcome-specific PIT occurs when the Pavlovian unconditioned stimulus (US) and the instrumental reinforcer are the same stimulus, whereas general PIT is said to occur when the Pavlovian US and the reinforcer are different. Lex and Hauber (2008) showed that D1 or D2 family antagonists locally injected into either the core or shell subregions of nucleus accumbens reduced general PIT, which is consistent with the report of Corbit et al. (2007), who demonstrated that inactivation of the VTA suppressed both general and outcome-specific PIT. More recent evidence indicates that accumbens core and shell appear to mediate different aspects of PIT; shell lesions and inactivation reduced outcome-specific PIT, while core lesions and inactivation suppressed general PIT (Corbit and Balleine 2011). These core versus shell differences are likely due to the different anatomical inputs and pallidal outputs associated with these accumbens subregions (Root et al. 2015). These results led Corbit and Balleine (2011) to suggest that accumbens core mediates the general excitatory effects of reward-related cues. PIT provides a fundamental behavioral process by which conditioned stimuli can exert activating effects upon instrumental responding (Robbins and Everitt 2007; Salamone et al. 2007). The activating or arousing effects of conditioned stimuli can be a factor in amplifying an instrumental response that has already been acquired, but also could facilitate acquisition of instrumental learning by increasing response output and behavioral variability, thus setting the occasion for more opportunities to pair a response with reinforcement [Salamone and Correa 2012; see Rick et al. (2006) for a discussion of behavioral variability]. Further dissection and discussion of the behavioral and neural processes contributing to PIT are provided by Corbit and Balleine in this volume.

This discussion of the activating effects of conditioned stimuli, and the role of accumbens DA in these processes, leads one back to the distinction between directional and activational aspects of motivation that was discussed above. Indeed, an overwhelming body of evidence indicates that accumbens DA is involved in behavioral activation and energy expenditure (Salamone 1988, 1992; Salamone and Correa 2002, 2012; Robbins and Everitt 2005, 2007; Beeler et al. 2012). Accumbens DA depletions or antagonism suppress novelty-induced locomotion (Koob et al. 1978; Cousins et al. 1993; Baldo et al. 2002; Correa et al. 2002). Scheduled presentation of food pellets to food-restricted rats can induce various types of activity, including excessive drinking, wheel running, and locomotion; these schedule-induced activities have been shown to be accompanied by increases in accumbens DA release (McCullough and Salamone 1992) and suppressed by accumbens DA depletions (Robbins and Koob 1980; Wallace et al. 1983; McCullough and Salamone 1992). Knab et al. (2009) reported altered expression of dopaminergic genes in mice with high versus low levels of running wheel activity, which was independent of their previous wheel exposure. Recent studies have shown that inactivation of VTA DA neurons by DREADD (designer receptors exclusively activated by designer drugs) methods significantly suppressed locomotor activity (Marchant et al. 2015). Recent data from our laboratory have demonstrated that locomotor activity can be instigated by the presentation of cues associated with sucrose and that this effect was blocked by DA antagonism (Fig. 1).

Fig. 1
figure 1

Presentation of a CS + paired with sucrose instigates locomotor activity, and this effect is blocked by a very low dose of haloperidol (Correa et al. 2012). Mice (N = 66) were trained to associate an olfactory stimulus (CS+) with 10 % sucrose and another stimulus (CS−) with water during 30 min per day, 4-week training sessions. On the test day, after the conditioning period, one of the odors (CS+ or CS−) was present in the upper part of the locomotion chamber. The effect of haloperidol (0.0 or 0.05 mg/kg) on locomotion induced by CS+ or CS− presentation on horizontal locomotion (A and C) and vertical locomotion (B and D), in the quadrant where the CS (black dot) was located (upper panels, A and B; see blue quadrant) and in the other 3 quadrants (lower panels, C and D; blue shading indicates these quadrants). Mean (±SEM) number of counts in the open field during the 15 min test session. **p < 0.01 significantly different between doses in the same group; ##p < 0.01 significant difference between CS+ and CS− at the same dose. In the CS quadrant, the two-way factorial ANOVA for the horizontal locomotion showed a significant interaction (F(1, 62) = 4.08, p < 0.05), and for the vertical locomotion (F(1, 62) = 13.05, p < 0.01). In the quadrants with no CS, a two-way factorial ANOVA for horizontal locomotion showed a significant interaction (F(1, 62) = 4.18, p < 0.05), but not for the vertical locomotion (F(1, 62) = 2.46, n.s.). In a parallel study, vehicle-treated mice drank 0.94 ± 0.20 ml of 10 % sucrose in 60 min, and when they received 0.05 mg/kg haloperidol the amount of sucrose consumed was 1.07 ± 0.16 ml. The t-test for related samples yielded no significant differences (t(20) = 0.97; n.s.). Thus, the effect of haloperidol on the activational properties of the CS+ was not dependent upon an effect on sucrose consumption

Full size image

Consistent with the known involvement of accumbens DA in behavioral activation and energy expenditure, several studies have demonstrated that the effects of nucleus accumbens DA depletions interact with the work requirements presented across various instrumental tasks. One way of varying the response requirements of instrumental behavior is to vary the ratio requirements of operant schedules. Studies have shown that accumbens DA depletions suppress operant lever pressing on ratio schedules in a manner that is directly related to the size of the ratio requirement. Fixed ratio (FR) 1 responding is only marginally and transiently affected by DA depletion, while rats responding on moderate size ratio schedules (FR 5, 16, 20) showed modest reductions in response rates, and animals tested on schedules with high ratios (e.g., FR16, 64, 300) were severely impaired (McCullough et al. 1993a; Aberman et al. 1998; Aberman and Salamone 1999; Salamone et al. 2001; Ishiwari et al. 2004). These ratio-related effects of accumbens DA depletion differed substantially from the pattern produced by pre-feeding to devalue the food reinforcement (Aberman and Salamone 1999). Moreover, these effects did not depend upon the time intervals or intermittency of reinforcement; rats responding on conventional variable interval schedules (e.g., VI 30, 60 or 120 s) were not affected by accumbens DA depletions that substantially suppressed responding when a ratio requirement (FR5 or 10) was attached to the interval requirement (Correa et al. 2002; Mingote et al. 2005). Thus, accumbens DA depletions appear to blunt the response-enhancing effects of moderate-sized ratios, and enhance the response suppressing effects of very large ratio requirements (Salamone and Correa 2002).

4 Manifestations of Interference with Accumbens DA Transmission: Behavioral Activation, Behavioral Economics, and Effort-Related Choice

As evidence was emerging on the role of accumbens DA in behavioral activation , it was suggested several years ago that interference with DA transmission should affect cost/benefit decision making on tasks involving varying effort requirements (Salamone 1987, 1991, 1992). In complex environments, organisms continually make effort-related decisions based upon cost/benefit analyses, allocating behavioral resources into goal-directed actions based upon assessments of motivational value and response costs. It was hypothesized that behavioral processes related to activational aspects of motivation could be engaged to facilitate the ability of organisms to overcome work-related response costs that separate them from significant stimuli and that interference with DA transmission would alter the relative allocation of responses, biasing animals toward low-cost alternative (Salamone 1987, 1991, 1992). Such a function would be extremely important for animals foraging in the wild (e.g., for efficient investment of behavioral resources and energy expenditure) and could also be studied in laboratory experiments. Within the next few years, research was undertaken to study the role of DA in effort-related choice behavior (also called effort-related or effort-based decision making).

Effort-based decision making is generally studied using tasks that offer a choice between high-effort instrumental actions leading to more highly valued reinforcers versus low-effort options leading to less valued reinforcers. An operant concurrent choice task was developed, which offers rats a choice between fixed ratio 5 (FR5) lever pressing to obtain a relatively preferred food (high-carbohydrate pellets), versus approaching and consuming a less preferred food (lab chow) that is concurrently available (Salamone et al. 1991). Under control conditions, rats pressing on the FR5 schedule typically get most of their food by lever pressing and eat only small amounts of chow. Pre-feeding to devalue food reinforcement suppressed both lever pressing and chow intake (Salamone et al. 1991). In contrast, low doses of DA antagonists and depletions or antagonism of accumbens DA shift choice behavior, decreasing lever pressing but substantially increasing intake of the concurrently available chow (Salamone et al. 1991, 2002; Koch et al. 2000; Nowend et al. 2001; Sink et al. 2008; Farrar et al. 2010). Thus, despite the fact that lever pressing is decreased by accumbens DA antagonism or depletions, the rats show a compensatory reallocation of behavior and select a new path to an alternative food source. The use of this task as a measure of effort-related choice behavior has been validated in several ways. Drug treatment conditions that produced the shift in choice behavior did not alter food intake or preference in free-feeding choice tests (Salamone et al. 1991; Koch et al. 2000; Farrar et al. 2008; Nunes et al. 2013a, b; Pardo et al. 2015), indicating that DA -ergic manipulations were not simply changing food preference. Increasing the lever pressing work requirement (i.e., larger fixed ratios) resulted in a shift from lever pressing to chow intake (Salamone et al. 1997). Although DA antagonists reduce FR5 lever pressing and increase chow intake, appetite suppressants such as fenfluramine and cannabinoid CB1 antagonists do not increase chow intake at doses that suppress lever pressing (Salamone et al. 2002; Sink et al. 2008; Randall et al. 2012, 2015). Thus, interference with DA transmission does not simply reduce appetite for food (Salamone and Correa 2009, 2012).

A T-maze choice procedure also was developed to assess effort-related choice behavior (Salamone et al. 1994a). The two choice arms of the maze can have different reinforcement densities (e.g., 4 vs. 2 food pellets, or 4 vs. 0), and under some conditions, a vertical barrier is placed in the arm with the higher density of food to provide an effort-related challenge. DA antagonism and accumbens DA depletions bias animals toward the low-effort alternative, decreasing selection of the high-reward /high-cost arm with the barrier, but increasing selection of the low-reward/low-cost arm with no barrier (Salamone et al. 1994a; Cousins et al. 1996; Mott et al. 2009; Mai et al. 2012; Pardo et al. 2012). When no barrier is present in the arm with the high-reward density, or when both arms have a barrier, neither DA antagonism nor DA depletion alters response choice (Salamone et al. 1994a; Pardo et al. 2012). When the arm with the barrier contained 4 pellets, but the other arm contained no pellets, rats with impaired accumbens DA transmission still chose the high-density arm, climbed the barrier, and consumed the pellets (Cousins et al. 1996; Yohn et al. 2015a, b).

Effort-discounting tasks also have been developed, in which the relation between required effort and reinforcement value is systematically varied within a test session, and the animal is offered a variety of choices with different effort/reward trade-offs. Bardgett et al. (2009) developed an effort-discounting task based upon the T-maze barrier procedure described above and showed that D1 or D2 antagonism reduced selection of the high-effort arm with the barrier. Floresco and colleagues have established effort-discounting procedures based upon the choices between ratio schedules with different response requirements. Administration of the DA antagonist flupenthixol altered effort-related decision making and biased selection toward the lower ratio option, even if the time to completion of the ratio components was controlled for (Floresco et al. 2008). Moreover, local blockade of GABAA/B receptors in the core region of the accumbens , but not the shell, reduced selection of the higher effort alternative under both the standard and equivalent delay conditions (Ghods-Sharifi and Floresco 2010). Interestingly, the effects of dopaminergic receptor blockade on ratio discounting, which involves physical effort, are dissociable from the effects of DA antagonism on a cognitive effort-discounting task. Hosking et al. (2015) compared the effects of the DA D1 and DA D2 family antagonists on a ratio-discounting task that assesses physical effort versus a cognitive effort-discounting task in which animals can choose to allocate greater visuospatial attention to obtain a higher reward level. While DA antagonism altered decision making based upon physical effort, it had no effect on discounting based upon cognitive effort. These studies, together with the results of a recently developed progressive ratio (PROG)/chow feeding choice task (Randall et al. 2012, 2015), demonstrate that DA antagonism and accumbens DA depletions cause animals to reallocate their instrumental response selection based upon the response requirements of the task, and select lower cost alternatives (Salamone et al. 2007, 2012; Salamone and Correa 2012).

There also is evidence that DA systems exert a bidirectional influence over response output in tasks involving effort-related choice behavior. Cagniard et al. (2006) reported that DA transporter (DAT) knockdown mice showed increased lever pressing and decreased chow intake compared to wild-type mice. Consistent with this observation, Randall et al. (2015) recently reported that administration of the catecholamine uptake inhibitor bupropion, which elevates extracellular DA and increases expression of phosphorylated DARPP-32 in a manner consistent with increased D1 and D2 signaling, substantially increased selection of the high-effort (progressive ratio) option in rats tested on a concurrent choice procedure. Trifilieff et al. (2013) reported that selective overexpression of D2 receptors in the nucleus accumbens of adult mice also led to an increase in selection of high-effort alternatives in choice tasks. Interestingly, it also appears that individual differences in behavioral output on effort-related tasks can be correlated with neural markers of DA transmission. The progressive ratio/chow feeding choice task generates enormous individual variability in behavioral output, and Randall et al. (2012) found that rats with high lever pressing output showed greater expression of accumbens core DARPP-32 phosphorylated at the threonine 34 site compared to low performers.

Of course, nucleus accumbens DA is just one part of the broader neural circuitry involved in effort-related processes. DA interacts with other transmitters and neuromodulators to regulate effort-related functions. Systemic or local intra-accumbens core administration of adenosine A2A antagonists can reverse the effort-related effects of DA antagonists (Farrar et al. 2007, 2010; Worden et al. 2009; Mott et al. 2009; Salamone et al. 2009; Nunes et al. 2010; Pardo et al. 2012; Randall et al. 2015; Yohn et al. 2015a). Intra-accumbens injections of adenosine A2A agonists can induce effects on effort-related choice that resemble those induced by DA antagonism or depletion (Font et al. 2008; Mingote et al. 2008), and systemic administration of the adenosine A2A antagonist to rats was shown to increase work output on the lever pressing component of the progressive ratio/chow feeding choice procedure (Randall et al. 2012). A number of papers, including some that have employed disconnection methods (i.e., combined contralateral manipulation of two different parts of the circuit), have shown that there is a distributed neural circuitry that regulates effort-based decision making, which involves basolateral amygdala, prefrontal/anterior cingulate cortex, nucleus accumbens, and ventral pallidal GABA (Salamone et al. 1994a, 1997, 2007; Walton et al. 2003; Floresco and Ghods-Sharifi 2007; Farrar et al. 2008; Mingote et al. 2008; Hauber and Sommer 2009; see Fig. 2). (Further discussion of these and related circuits is found in the O’Doherty chapter and the Bissonette and Roesch chapter in this volume.)

Fig. 2
figure 2

Schematic showing anatomical connections between structures involved in effort-related choice behavior in the rodent brain. Acb nucleus accumbens; ACg anterior cingulate gyrus; VP ventral pallidum; Amg amygdala; VTA ventral tegmental area; DA dopamine; GABA gamma aminobutyric acid; Glut glutamate

Full size image

5 Clinical Significance of Effort-Related Functions

There are widespread reports in the clinical literature of human pathologies involving activational or psychomotor impairments. Motivational dysfunctions are reported to be some of the most common psychiatric symptoms seen in general medicine (Demyttenaere et al. 2005). In addition, motivational/psychomotor symptoms variously labeled as anergia , fatigue , psychomotor retardation, or lassitude are frequently observed in patients with major depression and related disorders (Stahl 2002; Caligiuri et al. 2003; Demyttenaere et al. 2005; Salamone et al. 2006, 2014; Bella et al. 2010; Treadway and Zald 2011; Fava et al. 2014; Soskin et al. 2013). Effort-related symptom severity in depressed people is correlated with problems involving social function, employment, and response to treatment (Tylee et al. 1999; Stahl 2002). Gullion and Rush (1998) conducted a correlational and factor analytic study of data from depressed patients and identified a “lack of energy” factor that was related to problems such as low energy/increased fatigability, inability to work, and psychomotor retardation; this was also the factor that loaded most strongly onto a second-order general depression factor. Depressed patients can have core impairments in exertion of effort during reward seeking that do not simply depend upon any problems that they may have with experiencing pleasure in response to a primary motivational stimulus (Treadway and Zald 2011; Treadway et al. 2012; Argyropoulos and Nutt 2013). (More information on motivation -related phenotypes and biomarkers in depression can be found in Treadway et al. in this volume.) Because motivational symptoms in depression and other disorders can be highly resistant to treatment (Stahl 2002; Fava et al. 2014), this is an important unmet need in psychiatry.

Within the last few years, human tasks assessing effort-related decision making have been developed (Treadway et al. 2009; Gold et al. 2013), and there has been a wave of publications reporting on the presence of altered effort-based function across various clinical populations. People with major depression show reduced selection of high-effort alternatives (Treadway et al. 2012; Yang et al. 2014). Several reports involving various behavioral procedures have shown that schizophrenic patients also show “effort shyness,” and tend to select low-effort alternatives when tested on choice tasks (Gold et al. 2013, 2015; Fervaha et al. 2013; Green and Horan 2015; Green et al. 2015; Hartmann et al. 2015). The clinical assessment of motivational deficits in schizophrenia is described in detail in this volume by Reddy et al. In addition, an elegant series of experiments that have identified some of the behavioral components of motivational deficits in schizophrenia are provided by Waltz and Gold in this volume. Patients with Parkinson’s disease also showed reduced selection of high-effort alternatives compared to control subjects, and these deficits were significantly reduced when patients were on their dopaminergic medication (Chong et al. 2015). In contrast, autistic patients showed increased selection of high-effort choices (Damiano et al. 2012).

Because of the clinical significance of deficits in behavioral activation and exertion of effort, another line of work has emerged in which effort-related dysfunctions are being studied using explicit animal models that are related to psychopathology. Thus, the rodent tasks described above can be employed to study the effects of conditions associated with depression , and this type of task can be used to assess the effects of putative and well-established therapeutic agents. Recent studies have shown that conditions associated with depression in humans can shift effort-related choice behavior and reduce selection of high-effort choices in rats. These conditions include stress (Shafiei et al. 2012), the proinflammatory cytokine interleukin 1-β (IL1-β; Nunes et al. 2014), and administration of tetrabenazine (TBZ). TBZ inhibits the vesicular monoamine transporter type 2 (VMAT-2), which leads to a blockade of vesicular storage and a depletion of monoamines, with its greatest effects at low doses being on striatal DA (Pettibone et al. 1984; Tanra et al. 1995). TBZ is used to treat Huntington’s disease and other movement disorders, but major side effects include depressive symptoms (Frank 2009, 2010; Guay 2010; Chen et al. 2012). Moreover, TBZ has been employed in studies involving traditional animal models of depression (Preskorn et al. 1984; Kent et al. 1986; Wang et al. 2010). Recent research has demonstrated that low doses of TBZ that decreased DA release and DA-related signal transduction in the accumbens could alter effort-related choice behavior as assessed by concurrent lever pressing/chow feeding choice procedures (Nunes et al. 2013b; Randall et al. 2015), as well as the T-maze barrier choice task (Yohn et al. 2015b). The low doses of TBZ that decreased selection of FR5 or progressive ratio lever pressing did not alter relative preference for high-carbohydrate pellets (the reinforcer for the high-effort option) versus chow intake (Nunes et al. 2013b) and did not produce effects similar to pre-feeding or appetite suppressant drugs (Randall et al. 2012, 2015). These effects of systemic TBZ were also shown after local injections of the drug into nucleus accumbens core, but not overlying medial dorsal striatum (Nunes et al. 2013b). Recently, a version of the concurrent lever pressing/chow intake task was developed in which different sucrose concentrations were used (Pardo et al. 2015). TBZ reduced lever pressing for the strongly preferred higher concentration of sucrose, but actually increased selection of the low concentration of sucrose that was obtained with low effort. Nevertheless, the same doses of TBZ had no effect on sucrose preference or appetitive taste reactivity (Pardo et al. 2015). In another recent study (Yohn et al. 2015c), TBZ altered effort-related decision making in rats responding on the T-maze barrier task, as marked by a reduction in the selection of the barrier arm that contained the high density of food reinforcement (4 pellets), and increased selection of the arm with 2 pellets but no barrier. In the dose range tested (0.25–0.75 mg/kg), TBZ did not affect arm selection when there was no barrier in either arm, or when the arm with the barrier had 4 reinforcement pellets but the other arm had no pellets. This pattern of evidence indicates that TBZ was not reducing selection of the high-effort alternative (i.e., the barrier arm in the 4–2 barrier condition) because it was impairing sensitivity to reinforcement density, preference for 4 pellets versus 2, reference memory, left/right discrimination, or because of an absolute inability to climb the barrier or a ceiling level of barrier crossings (Yohn et al. 2015c).

Several drugs have been evaluated for their ability to reverse the deficits in effort-related choice described above. Adenosine A2A antagonists have antiparkinsonian effects in animal models and human clinical studies and can produce behavioral effects in rodents that are consistent with antidepressant actions as assessed by classical behavioral models such as the forced swimming and tail suspension tests (Hodgson et al. 2009; Hanff et al. 2010; Yamada et al. 2013, 2014).The adenosine A2A antagonist MSX-3 has been shown to reverse the effects of IL-1β on concurrent FR5/chow feeding choice performance (Nunes et al. 2014). This drug also reversed the effort-related effects of TBZ in rats responding across multiple tasks (Nunes et al. 2013b; Randall et al. 2015; Yohn et al. 2015a). These findings are consistent with previous research showing that adenosine A2A antagonists can reverse the effort-related effects of DA D2 family antagonists (Farrar et al. 2007; Salamone et al. 2009; Worden et al. 2009; Mott et al. 2009; Nunes et al. 2010; Santerre et al. 2012; Pardo et al. 2012). Anatomical studies have shown that adenosine A2A receptors are co-localized with DA D2 family receptors on enkephalin-positive medium spiny neurons in both neostriatum and accumbens (Rosin et al. 1998; Svenningson et al. 1999). Adenosine A2A and DA D2 receptors can form heteromeric complexes, and they also converge onto the same c-AMP/protein kinase A signal transduction pathway (Ferré et al. 2008; Santerre et al. 2012). Recently, it was shown that 0.75 mg/kg TBZ reduced DA -related signal transduction mediated by D1 and D2 receptors (e.g., changes in cFos and DARPP-32 expression) and that 2.0 mg/kg MSX-3 could reverse the cellular effects of diminished D2 transmission (Nunes et al. 2013b).

Bupropion (Wellbutrin) is a widely prescribed antidepressant (Milea et al. 2010), which has been shown to produce antidepressant-like effects in traditional rodent models such as the forced swim and tail suspension tests (Cryan et al. 2004; Bourin et al. 2005; Kitamura et al. 2010). Bupropion inhibits catecholamine uptake and has been shown to occupy DA transporters in humans at doses that are clinically useful for treating depression (Learned-Coughlin et al. 2003). This drug also elevates extracellular DA and norepinephrine (NE) in rats as measured by microdialysis methods (Hudson et al. 2012; Randall et al. 2015). Bupropion fully reversed the effects of TBZ in rats tested on the T-maze barrier choice task, increasing selection of the barrier arm in TBZ-treated rats (Yohn et al. 2015a). These results are consistent with previous research showing that bupropion increased lever pressing performance in TBZ-treated rats tested on the fixed ratio 5/chow feeding choice (Nunes et al. 2013b) and progressive ratio/chow feeding choice tasks (Randall et al. 2015). Furthermore, doses of bupropion that increase extracellular DA and DARPP-32 expression in nucleus accumbens core also increased progressive ratio output in rats responding on the progressive ratio/chow feeding choice task (Randall et al. 2015). This is consistent with previous reports showing that the novel DA uptake inhibitor MRZ-9547 increased progressive ratio choice lever pressing output (Sommer et al. 2014) and that amphetamine increased selection of the high-effort alternative in humans responding on an effort-related decision-making task (Wardle et al. 2011). Considering the known antidepressant actions of catecholamine uptake inhibitors such as bupropion in humans, these results serve to validate the hypothesis that tests of effort-related choice behavior can be used to assess some of the motivational effects of well-known or putative therapeutic agents.

Clinical data indicate that catecholamine uptake inhibitors are moderately efficacious for treating psychomotor retardation and fatigue symptoms of depression (Fabre et al. 1983; Rampello et al. 1991; Pae et al. 2007; Cooper et al. 2014) and can be more effective than 5-HT uptake blockers for treating motivational dysfunction in depressed people (Papakostas et al. 2006; Cooper et al. 2014). Recent research has studied the effects of various monoamine uptake inhibitors for their ability to reverse the effects of TBZ (Yohn et al. 2015c). The effort-related effects of TBZ in rats tested on the concurrent FR5/chow feeding choice task were attenuated by bupropion, and this effect of bupropion was reversed by either D1 or D2 family antagonism. The effort-related effects of TBZ also were attenuated by the selective DA transport inhibitor GBR12909. However, the 5-HT uptake inhibitor fluoxetine and the norepinephrine uptake inhibitor desipramine failed to reverse the effects of TBZ, and higher doses of these drugs, when given alone or in combination with TBZ, led to further behavioral impairments (Yohn et al. 2015c). Thus, drugs acting on DA transmission appear to be relatively effective at reversing the effort-related effects of TBZ and for enhancing work-related behavioral output. These findings are consistent with the hypothesis that drugs that enhance DA transmission may be effective at treating effort-related psychiatric symptoms in humans.

Tests of effort-related decision making also have been used to model features of schizophrenia. Although local overexpression of DA D2 receptors in adult animals leads to increased behavioral activation and effort expenditure (Trifilieff et al. 2013), several studies have shown that overexpression of D2 receptors in striatal medium spiny neurons throughout development leads to the opposite effect, reducing behavioral activation and exertion of effort in motivated behavior (Ward et al. 2012). These D2 overexpressing mice show reductions in progressive ratio break responding (Drew et al. 2007; Simpson et al. 2011) and reduced selection of the high-effort alternative in a test of effort-based choice (Ward et al. 2012). However, these animals do not show alterations in hedonic reactivity to food rewards , or changes in appetite or food preference. It has been hypothesized that these motivational impairments in D2 receptor overexpressing mice could be useful for modeling some of the negative symptoms of schizophrenia (i.e., avolition, amotivation, apathy; Simpson et al. 2012; see also the chapter by Ward in this volume).

6 Conclusions

As reviewed above, considerable evidence indicates that mesolimbic DA is a critical component of the neural circuitry that regulates behavioral activation , energy expenditure, and effort-related processes. This research is important for understanding the brain mechanisms involved in regulating important aspects of motivation , but it also has considerable clinical significance. Tests of effort-related decision making can be used for the preclinical assessment of novel drugs that are targeted for the treatment of effort-related motivational symptoms, and future research should assess the effects of a broad range of potential therapeutic agents. Although potentially useful for drug development, and for identifying specific neurochemical mechanisms regulating activational aspects of motivation , tests of effort-based decision making in rodents do not represent animal models of depression or any other specific disorder, and their potential utility is not limited to the assessment of antidepressant drugs. Impairments in behavioral activation and effort-related functions appear across multiple disorders and psychiatric conditions (Winograd-Gurvich et al. 2006; Treadway and Zald 2011; Treadway et al. 2012; Gold et al. 2013; Markou et al. 2013). There are some known similarities, and also some differences, in the mechanisms that underlie motivational deficits in depression and schizophrenia (see Barch et al., this volume). Therefore, rodent tests of effort-related dysfunction are likely to reflect a component of depression , rather than a global measure, and represent models of a set of motivational symptoms that cross multiple diagnostic categories, rather than being a model of a specific disorder. This view is consistent with the Research Domain Criterion (RDoC) approach, which suggests that researchers should focus on psychiatric symptoms and their associated neural circuits, in addition to the traditional emphasis on specific disorders (Cuthbert and Insel 2013).