Introduction

Economic decision-making refers to the complex cost-benefit analyses that guide value-based choices (Rangel et al. 2008; Kalenscher and van Wingerden 2011). During this process, each reinforcer must be integrated with factors that detract from, or “discount”, the subjective economic value of that option (Montague and Berns 2002; Bechara 2005; Rangel et al. 2008). A recurring decision-making scenario involves the conflict between high effort and high risk options (Winstanley and Floresco 2016). For example, appraising methods of paying for food and housing may necessitate choice between steady work for a reliable paycheck vs. a trip to a nearby casino. While each option may have a comparable ideal outcome (sufficient earnings to pay bills), overall value will differ according to evaluation of each option’s discounting factors, in this case, the effort expenditure of a job vs the risk of losing one’s savings at the casino. Comparison between choices involving effortful and probabilistic costs is commonly disrupted in pathologies of decision-making, such as the bias in substance use disorder toward seeking high risk/low effort reinforcement (Brevers et al. 2014; Saddoris et al. 2016; Verdejo-García et al. 2018).

Animal models provide a necessary avenue toward deconstructing the economic decision-making process (Floresco et al. 2008a, b; Orsini et al. 2015; Winstanley and Floresco 2016). Standard paradigms study economic decision-making by measuring choice between a small and large reinforcer associated with a dynamic discounting factor such as delay, risk, or effort (Evenden and Ryan 1996; Cardinal and Howes 2005; Floresco et al. 2008b; Winstanley and Floresco 2016). However, these designs fail to capture situations in which each reinforcer is associated with a distinct cost, which is often the case in real-world decision-making. To this end, we developed the effort vs probability (EvP) economic conflict task to model complex choices between high-effort and high-risk options. In the EvP, two reinforcers of equal magnitude are each associated with a distinct discounting factor: (1) an effort requirement that increases throughout the session (effortful choice), or (2) a low probability of reinforcement (probabilistic choice). The EvP begins with the effortful choice requiring only a single lever press for reinforcement vs the probabilistic choice yielding a 25% chance of reinforcement, rendering the effortful choice the objectively superior option. However, as the task advances, the effortful choice requires progressively more effort (5, 10, and 20 lever presses), whereas the probabilistic choice maintains a single lever press at a fixed delivery probability of 25%. Critically, the reinforcer associated with each choice is comparable in value (one pellet), which eliminates the influence of magnitude discrimination on the decision-making process. We predicted that rats would be able to evaluate both discounting factors to guide decision-making, initially preferring effortful choice, then shifting preference toward the probabilistic option as effort requirements increased.

The neurotransmitter dopamine is particularly critical for mediating value-based decision-making (Salamone et al. 2016; Schultz 2016; Schultz et al. 2017; Burke et al. 2018), and plays a significant role in the evaluation of both effort and probability. Cocaine and amphetamine, drugs of abuse that both enhance dopamine transmission, have contrasting and sometimes complex effects on decision-making. Cocaine reduces willingness to expend high levels of effort, increases tolerance for risk in pursuit of reinforcers, and causes perseverative behavior in decision-making with risk of punishment (van Haaren and Anderson 1994; Verdejo-García et al. 2007; Simon et al. 2009; Peña-Oliver et al. 2014). Conversely, elevated dopamine signaling caused by amphetamine can increase or decrease risk tolerance contingent on the identity of punishment (risk vs reward omission) and the manner in which the cost of uncertainty is manipulated throughout a task (Simon et al. 2009; St. Onge and Floresco 2009; St. Onge et al. 2010; Yang et al. 2018). Amphetamine also exerts dose dependent effects on effortful behavior, with high doses attenuating effort expenditure in pursuit of reinforcers, whereas lower doses or the slow release prodrug lisdexamfetamine increase effort expenditure during easier effort requirements (Floresco et al. 2008b; Yohn et al. 2016).

Manipulation of specific dopamine receptor subtypes has also been shown to influence risky and effortful decision-making. Blockade of D1-like dopamine receptors reduces probabilistic choice, while D1 stimulation increases preference for uncertain reinforcement (St. Onge and Floresco 2009). Similarly, probabilistic choice is reduced by D2 antagonists and increased by D2 agonists (St. Onge and Floresco 2009; Grall-Bronnec et al. 2016). Manipulation of specific receptor subtypes produces similar effects on effortful choice. Effortful choice is reduced by D1 blockade (Nunes et al. 2010; Yohn et al. 2015; Salamone et al. 2016). Conversely, D1 activation increases effortful choice, in addition to deterring the effort aversion elicited by D1 antagonism (Yohn et al. 2015; Soutschek et al. 2020). D2 antagonists also reduce effortful choice (Yohn et al. 2015), though their stimulation does not affect effort expenditure for food (Zhang et al. 2010).

The contribution of both D1- and D2-like receptors to the evaluation of both effort and probability suggests a role for these receptors when both factors are evaluated prior to a decision. To this end, we tested the role of dopaminergic activity during effort vs risk economic conflict. First, we measured behavior in EvP to confirm that subjects were able to accurately discern between effortful and risky options. We then used acute exposure to the psychostimulant cocaine, a dopamine transporter inhibitor that increases synaptic dopamine concentration, to test impact of a general increase in synaptic dopamine on EvP. Finally, we assessed the acute effects of systemic D1- and D2-like dopamine receptor activation or blockade on effortful vs probabilistic choice.

Materials and methods

Subjects

Male Long-Evans rats (n = 23, Envigo Corp) were obtained at approximately three months of age. Subjects arrived pair-housed with ad libitum access to food and water and were kept on a 12-h light/dark cycle beginning with lights off at 7:30 am. One week after arrival, subjects were food restricted to 90% of their free feeding baseline weight to increase motivation in behavioral tasks. Baseline weight was increased in accordance with the growth projection provided by Envigo to allow for growth with age, resulting in approximately 5 g increases per week until 15 weeks of age. Pairs were separated if aggression or food dominance was observed. All protocols were approved by the University of Memphis Institutional Animal Care and Use Committee.

Behavioral apparatus

Behavior and decision-making processes were measured in MedAssociates (FairFax, VA) operant conditioning chambers in soundproof cubicles. Each were equipped with one retractable lever on each side of an illuminable food trough with 0.635 cm recessed photobeams to track entries, pellet dispenser, metal floor grate, and locomotion tracking photobeams. Sugar pellets weighing 45 mg (Bio-Serv, Flemington, NJ) were used as a behavioral reinforcer.

Instrumental shaping

Initial shaping procedures followed previous protocols (Gabriel et al. 2019; Orsini and Simon 2020). To reduce neophobia, rats were placed in their operant chamber for 5 min and had sugar pellets place in their home cages the day prior to shaping. Shaping began with magazine training in which 38 food pellets were delivered at a rate of 1 every 30 ± 10 s to teach subjects to associate the food trough with pellet delivery. Pellet delivery was accompanied by illumination of the food trough, which remained on until subjects collected the pellets. After collecting all food reinforcers, subjects were trained to exert a single lever press for pellet delivery. In a counterbalanced order, each lever was extended by itself until subjects obtained 50 reinforcers, after which the opposite lever was extended until delivering 50 reinforcers.

Finally, rats learned to nose poke into the trough upon simultaneous illumination of the house light and trough. This response extinguished the food trough light and extended a single lever, which rats pressed for reinforcer delivery (1 pellet) and food trough illumination. Levers were presented in pseudorandom order, with neither extended more than twice sequentially, and retracted immediately when pressed. Lights were extinguished 10 s after reinforcer delivery or upon reinforcer collection and an intertrial interval (ITI) of 10 ± 4 s preceded the next trial. Failure to respond within 10 s of food trough illumination or lever extension resulted in the trial being marked an omission, termination of all stimuli, and ITI initiation. Upon achieving 35 presses on each lever in a session, subjects began training in the EvP.

The effort vs probability economic conflict task

The EvP measured decision-making between two reinforcers that were equivalent in magnitude, but associated with distinct costs: either effort or probability (Fig. 1a). The EvP consisted of 4 blocks of 18 trials each. Each trial began with illumination of the food trough, after which a head entry into the trough extinguished the light and evoked extension of either a single lever (forced choice trial) or two levers (free choice trial). A press on either of these levers was associated with receipt of a single sugar pellet as well as a unique discounting factor. One lever was associated with low probability of pellet delivery (25%), and the other a dynamic effort requirement (fixed ratio 1, 5, 10, or 20). The identity of these levers was counterbalanced between subjects and held consistent throughout training. After a single press of the probabilistic lever, both levers were retracted until the next trial. After an effortful choice, the probabilistic lever was retracted, and the effortful lever remained extended until the subject completed the requisite response sequence. Each block began with 8 forced choice trials with a single lever extended to establish changes in action-outcome contingencies throughout the session, followed by 10 free choice (dual lever) trials in which rats chose between the probabilistic and effortful choice levers. After pellet delivery (or after the lever press in probabilistic trials in which no reinforcer was given), trials proceeded to a 10 ± 4 s ITI in which all lights were extinguished, and levers retracted. Trial omissions resulted from failure to either initiate a trial or select a lever within 10 s resulted in immediate procession to the ITI. Failure to complete the effort requirement within 2 min also resulted in the trial being marked as an omission and progressing to the ITI.

Fig. 1
figure 1

Effort vs probability economic conflict task (EvP). a Schematic of the EvP. During each trial, rats make either a probabilistic or effortful choice. A single sugar pellet is delivered on 25% of probabilistic choice trials and after every completion of the lever press requirement of effortful choice. The probabilistic choice is fixed throughout the session, whereas effort requirement increases across four blocks. After each choice sequence is completed, the trial progresses to the ITI. b Rats shifted away from effortful and toward probabilistic choice as effort requirements increased/became more difficult. (N = 23, mean ± SEM). \( \%\mathrm{Effortful}\ \mathrm{over}\ \mathrm{probabilistic}\ \mathrm{choice}=\frac{\mathrm{Effortful}\ \mathrm{choice}\ \mathrm{trials}}{\mathrm{Effortful}+\mathrm{probabilistic}\ \mathrm{choice}\ \mathrm{trials}}\times 100\% \) c Percent effortful choice trials of individual subjects during EvP. Each line represents a single rat. d Scatterplot visualizing that effortful choice in EvP (AUC for blocks 2–4) was consistent across time

Full size image

Diminishing probability vs fixed effort economic conflict task

The EvP held the probabilistic choice at a constant rate of reinforcer delivery (25%) and increased the effortful choice’s lever press requirement (1, 5, 10, or 20). To confirm behavior was not solely a product of the variable effort/fixed probability design, a subset of rats performed in an inverted task wherein the effortful choice was fixed at 10 lever presses throughout the session, and the rate of delivery for the probabilistic choice decreased throughout the session (100%, 75%, 50%, 25%). Forced and free choice trials were identical to the standard EvP in every respect other than these changes to the discounting factors.

Economic conflict between probability and effort-equivalent delays

Completion of effort requirements caused a delay between the decision and reinforcement equal to the time required to complete the effortful fixed ratio. To confirm that discounting of the high-effort option in EvP was not solely caused by this delay, a subset of subjects performed a modified task wherein the effortful choice was replaced with a single lever press requirement that delivered a pellet after a delay (0 s, 1.6 s, 3.5 s, or 7.5 s). These delays were determined by taking an average of the time in seconds required to complete each effort requirement in the standard EvP across the final 5 days of training.

Pharmacological manipulations

All acute pharmacological dopamine manipulations were performed using an 8-day drug administration protocol (Simon et al. 2011). On days 1, 3, 5, and 7, subjects received an intraperitoneal injection of the appropriate drug solution prior to testing. Dose sequence was counterbalanced across subjects for every drug regimen. Days 2, 4, 6, and 8 consisted only of behavior in the EvP, serving as controls to ensure no carryover effects of drug administration. All drugs were suspended in .9% saline solution.

The dopamine transporter inhibitor cocaine hydrochloride (0, 2, 5, 10 mg/kg, Sigma-Aldrich), D1 agonist SKF81297 (0, .1, .3, 1.0 mg/kg, Cayman Chemicals), and D2 agonist Quinpirole hydrochloride (0, 0.0375, 0.125, 0.25 mg/kg, Sigma-Aldrich) were all administered 5 min prior to behavior in the EvP. The D1 antagonist SCH23390 hydrobromide (0, 0.005, 0.01, 0.03 mg/kg, Sigma-Aldrich) and D2 antagonist Eticlopride hydrochloride (0, 0.01, 0.03, 0.05 mg/kg, Sigma-Aldrich) were administered 20 min prior to behavior. Drug doses and timing of injections were based on previous studies with these drugs in similar decision-making tasks (Simon et al. 2009, 2011; Winstanley et al. 2011). Drug conditions were tested in the following order: Cocaine, SCH23390, Eticlopride, Quinpirole, SKF81297, with a minimum of 1 week between each new drug regimen. After completion of all drug treatments (55.43 ± 7.67 days for two cohorts), baseline behavior was again measured over a 3-day span, and compared to predrug baseline levels to assess long-term stability of EvP decision-making.

Experimental timeline

Subjects were divided into two cohorts and trained in the EvP until preference between effortful and probabilistic choice was stable. Upon reaching stability, each cohort progressed along a distinct experiment series (Table 1). In brief, cohort 1 received D1/D2 antagonists and agonists, with a minimum of 1-week separating administration of each drug. Cohort 1’s behavior concluded with a no-drug retest in EvP to test if choice preference in the EvP was stable. In cohort 2, effects of cocaine on EvP performance were assessed, followed by a no-drug test of EvP stability. Cohort 2 then trained in variations of the EvP to test the effect of manipulating delay in place of effort and making the probabilistic cost variable and effort fixed. Finally, the effects of D2 activation were measured on EvP with variable probability and fixed effort.

Table 1 Order and duration of experiments conducted with experimental cohorts 1 and 2
Full size table

Data analysis

Subjects performed each task until stable over a 3-day period, determined as a lack of interaction between day and task block in a day × block repeated measures ANOVA. Stability was achieved after an average of 16.25 sessions for all three tasks (EvP, fixed effort vs diminishing probability, and fixed probability vs effort-equivalent delays). Behavior was averaged over the stable period to provide a representative baseline of pre-treatment behavior. Decision preference in EvP was quantified using (1) percent choice of the effortful option across all blocks, and (2) area under the curve (AUC) of percent effortful choice for blocks 2–4. AUC was calculated geometrically as the summed area of a 2 trapezoids created by drawing vertical lines from the x-axis to percent choice of the effortful option in blocks 2, 3, and 4 of the task, i.e., AUC = (B2 % effort + B3 % effort)/2 + (B3 % effort + B4 % effort)/2. This approach has previously been utilized for dose-response curves, and these data were found to be highly correlated with approaches used in classical behavioral economic studies (Myerson et al. 2001; Amlung et al. 2015).

Block 1 was omitted in calculating AUC to maintain a focus on conditions of true economic conflict, as this is the only block with no effort vs probability conflict (single lever press for a reinforcer vs a single lever press with a 25% chance of a reinforcer) and an objectively superior choice. AUC was used in both Pearson’s correlations and paired-samples t tests to compare decision-making between tasks. The indifference point in EvP was defined as the combination of effort and probability where choice was most evenly split between the two options. The delay associated with each effort requirement in the EvP was determined by averaging the time in seconds that subjects required to complete each fixed ratio.

Effects of pharmacological manipulation were analyzed using dose × block (4 × 4) repeated measures ANOVAs. Significant main effects were investigated using pairwise comparisons to identify which doses differed from saline using the EMMEANS and Compare subcommands within the IBM® SPSS® Statistics 26 syntax. Significance was computed based on estimated marginal means as predicted for each dose (saline, low, mid, and high) collapsed across all 4 blocks of the EvP. Because these comparisons collapsed each dose across all effort requirements, each significant saline-dose pair was further analyzed with a follow-up dose × block (2 × 4) repeated measures ANOVA to identify under which effort requirement behavior was changed compared to saline. Dose × effort requirement interactions were investigated through separate repeated measures ANOVAs for each effort requirement where dose served as the within-subjects variable and pairwise comparisons were used to identify which doses differed from saline at each effort requirement.

In cases where drug administration led to omissions of entire blocks of a task, missing data were interpolated using slope between adjacent blocks. In sessions where a subject fully omitted at least one block of trials, data from that subject was not included in repeated-measures ANOVA analyses due to the listwise deletion of asymmetrical data. To confirm that interpretation of dopaminergic manipulations was not skewed by subjects excluded from ANOVAs, linear mixed effects models were created in Matlab 2020b (Mathworks; Natick, MA) for the effects of each. Fixed factors were defined as drug concentration, effort requirement, and the dose × effort interaction, subject served as the random factor, and the fit of these models was then compared to a degenerate model consisting of a single constant term. The Greenhouse-Geisser correction is reported for all statistics in which the assumption of sphericity was violated.

Results

Experiment 1: the economic conflict task

The economic conflict task (Fig. 1a) offers rats a choice between two equal-sized pellet rewards, each associated with either a low probability of delivery (25%) or a dynamic effort requirement that increased throughout the session (1, 5, 10, and 20 lever presses). In the first block of the task, when both options were associated with a single lever press, subjects displayed definite preference for “effortful” over probabilistic reinforcement (M = 86.40, SEM = 3.01, Fig. 1b). However, as the effort requirement increased across blocks rats shifted preference (F (1.869, 41.121) = 53.608, p < .001, Fig. 1b), ultimately preferring reinforcers associated with probabilistic over effortful costs (M = 23.27, SEM = 5.93, Fig. 1b). This suggests that rats can compare and distinguish between multiple discounting factors, in this case probability and effort requirement. There was a substantial degree of individual variability between subjects (Fig. 1c), which is commonly observed in other cost-benefit decision-making tasks (Simon et al. 2011; Freels et al. 2020). Importantly, AUC for choice of the effortful over probabilistic option was strongly correlated at the initial predrug baseline period and after culmination of all drug treatments (55.4 ± 7.7 days later, r = 0.856, p < .001, Fig. 1d), indicating that preference for reinforcement associated with effortful vs probabilistic costs was highly stable over long periods of time. Furthermore, baseline effortful vs probabilistic choice was consistent across all drug administration schedules, as indicated by a lack of difference within saline sessions across all schedules (F (3, 27) = 1.718, p = 0.187) and no schedule by effort requirement interaction (F (4.933, 44.398) = 1.129, p = 0.359).

Next, we confirmed that behavior during the economic conflict task was due to true economic conflict and not to one discounting factor remaining constant while another changed. This was accomplished by training a subset of subjects in a modified task measuring preference between a fixed effort requirement of 10 lever presses and reinforcement delivered with decreasing probability throughout the session (100%, 75%, 50%, 25%). As expected, choice in block 1 favored the objectively advantageous option, such that rats preferred the reinforcer associated with 100% “probabilistic” cost over the reinforcer associated with a 10 press effort requirement (M = 82.99, SEM = 7.28, Fig. 2a). Critically, preference shifted toward the effortful option as the probability of reinforcement was reduced (F (3, 33) = 8.582, p < 0.001, Fig. 2a). We estimated the “indifference point” (approximately 50% choice of both options) to be block 3 of the standard EvP (10 lever press vs 25% delivery rate, M = 51.03, SEM = 5.85, Fig. 1b), and determined that preference between options during this block point was comparable between the EvP and the modified economic conflict task (t (11) = 0.387, p = 0.706, Fig. 2b). Furthermore, there were no differences in overall preference for effortful vs probabilistic choice (t (11) = 0.063, p = 0.951) as well as a near significant correlation in AUC between tasks (r (11) = 0.509, p = 0.091, Fig. 2c), suggesting that behavior during economic conflict is consistent regardless of the varying discounting factor.

Fig. 2
figure 2

Preference between effortful and probabilistic choice is consistent regardless of which cost increases throughout the task. a Percent effortful choice in EvP (N = 12, mean ± SEM). b In reversed EvP wherein effortful choice was held constant at 10 lever presses and probabilistic choice became increasingly risky through the session, preference shifted from probabilistic to effortful choice across the session (n = 12, mean ± SEM). c An indifference point (~ 50% choice of both rewards) was identified in block 2 of the EvP (25% reinforcement rate vs 10 lever press effort; mean ± SEM). Percent effortful choice trials at this indifference point did not differ between the standard and reversed EvP. d Effortful vs probabilistic choice across the session did not differ between the standard and reversed EvP (AUC for task blocks 2-4; mean ± SEM)

Full size image

Selection of the effortful choice causes a delay before reinforcement equal to the time necessary to finish the lever press requirement. It is possible that aversion to this delay, rather than the physical effort requirement, drove the shift from effortful to probabilistic choice. Therefore, we designed a modified EvP in which choice was between a probabilistic (25%) reinforcer vs. reinforcement delivered after a delay that increased throughout the session. Each delay corresponded to the average time subjects required between the first lever press and completion of the effort requirement in the original task. These mean delays were calculated as 0 s, 1.6 s, 3.5 s, and 7.5 s corresponding respectively with the 1, 5, 10, and 20 lever press effort requirements. Rats previously trained in the EvP (Fig. 3a) displayed preference for the immediate over probabilistic reinforcement in the first block of the effort equivalent delay vs fixed probability task (M = 81.39, SEM = 9.11, Fig. 3b), then shifted choice preference away from the delayed toward the probabilistic option as delay preceding reward delivery increased (F (1.578, 17.353) = 4.271, p = 0.039, Fig. 3b). Preference for probabilistic choice, as measured by AUC, was significantly greater in this delay-based task than the standard EvP (t (11) = 5.250, p < 0.001, Fig. 3c), and there was no correlation between the effort and delay versions of the economic conflict task (r (12) = 0.239, p = 0.454). Thus, replacing effort with an equivalent delay substantially changed choice behavior, suggesting that the shift away from rewards with effortful costs was not solely caused by delay. Rather, the influence of effortful and delayed costs is evaluated differently when being compared to probabilistic costs, with rats demonstrating enhanced discounting of effort compared to delay.

Fig. 3
figure 3

Comparison between effort and delay costs in EvP. a Percent effortful choice during the standard EvP (n = 12, mean ± SEM). b When effort-requirement was replaced by delay, choice preference shifted from effort-equivalent delays to probabilistic reinforcement as delays increased (n = 12, mean ± SEM). c Probabilistic vs effortful choice differs from probabilistic vs effort-equivalent delays (AUC for task blocks 2–4; mean ± SEM)

Full size image

Experiment 2: effect of cocaine on economic conflict

After establishing the economic conflict task, we tested the acute effects of multiple dopaminergic drugs on effort vs probability discounting. First, rats were administered cocaine, a highly reinforcing drug that increases synaptic dopamine (as well as norepinephrine and serotonin) via reuptake inhibition. Overall, cocaine significantly shifted preference from probabilistic to effortful reinforcement (F (3, 33) = 3.626, p = 0.023, Fig. 4a). There was also a dose by effort requirement interaction (F (9, 99) = 2.648, p = 0.009, Fig. 4a). Post hoc tests revealed that, compared to saline, preference for effortful over probabilistic choice was increased by the low and mid doses of cocaine but unaffected by the high dose (low: p = 0.017; mid: p = 0.003; high: p = 0.370). Further analyses found that this shift from probabilistic to effortful choice induced by low and mid doses compared to saline was manifested only in the initial blocks (1 and 5 press requirements, p < 0.015), but was not evident in higher effort requirement blocks (10 and 20 press requirements, p > 0.30). Notably, a small group of subjects that displayed relatively low selection (< 60%) of effortful choice in block one with saline, which may have accounted for the effects of cocaine. However, after removing these subjects, the significant effects of cocaine at the low and mid doses in these blocks persisted, with low and mid doses of cocaine increasing effortful choice compared to saline during the 1 press (low: p = 0.007; mid: p = 0.007) and the 5 press conditions (low: p = 0.047; mid: p = 0.007). No omitted trials occurred at any dose, and AUC for conditions involving true economic conflict (blocks 2–4) did not differ with dose (F (3, 33) = 1.104, p = 0.361).

Fig. 4
figure 4

Cocaine and D1 receptor modulation had limited effects on choice in the EvP. a Acute administration of low and mid, but not high, doses of cocaine shifted preference from probabilistic toward effortful choice in low effort (1 and 5 lever press) blocks of EvP. b The D1 antagonist SCH23390 did not alter effortful vs probabilistic choice. c High dose D1 agonist SKF81297 shifted preference from effortful to probabilistic choice during the highest (20 lever press) effort requirement. All panels depict mean ± SEM

Full size image

Experiment 3: role of D1 dopamine receptor subtype during economic conflict

While the effects of cocaine suggest a role for dopamine in the economic conflict task, specific dopamine receptor subtypes have distinct and sometimes opposite effects on economic decision-making (Floresco et al. 2008a). To delineate the role of dopamine receptor subtypes during economic conflict between rewards with effortful and probabilistic costs, agonists and antagonists for D1- and D2-like receptors were systemically administered prior to task performance.

Blockade of D1 receptors with D1 antagonist SCH23390 had no effect on preference between effortful and probabilistic reinforcement (F (3, 30) = 0.950, p = 0.429; dose by block interaction: F (3.831, 38.314) = 0.963, p = 0.436, Fig. 4b). Furthermore, D1 blockade did not modulate AUC (F (3, 30) = 1.331, p = 0.283), and did not affect the number of omitted trials in a session (F (1.222, 12.220) = 3.754, p = 0.070). The D1 agonist SKF81297 hydrobromide also had no effect on choice between the effortful and probabilistic options (F (3, 24) = 0.632, p = 0.601, Fig. 4c) or AUC (F (3, 24) = 0.934, p = 0.440) and did not significantly increase omitted trials (F (1.631, 16.312) = 0.684, p = 0.490) although two subjects were excluded from ANOVA analyses due to omissions. However, D1 activation did elicit a block × dose interaction (F (4.029, 32.235) = 2.853, p = 0.039, Fig. 4c). A significant block by dose interaction was also observed using a linear mixed effects model (LMEM), which is less vulnerable to omissions than an individual ANOVA (Yates 2018) (Table 2). Post hoc analyses revealed that D1 activation shifted preference from effortful to probabilistic choices only at the highest effort requirement (F (3, 24) = 3.047, p = 0.048) and only at the highest dose of the D1 agonist compared to saline (p = 0.034). This suggests that high dose D1 agonist increases the preference for probabilistic over effortful costs, but only during relatively high effort requirements. Thus, systemic D1 receptor activation, but not blockade, has selective effects on decision-making during economic conflict.

Table 2 Linear mixed effects model for drugs that induced omissions. All drug schedules in which at least one subject was excluded from ANOVA analyses at any dose due to omitted trials were also analyzed using linear mixed model, which is more effective than standard ANOVA for processing missing data
Full size table

Experiment 4: role of D2 dopamine receptor subtype during economic conflict

The D2 antagonist Eticlopride hydrochloride caused a significant increase in trial omissions (F (1.802, 18.021) = 6.930, p = .007; saline: M = 0.45 ± 0.28; low: M = 0.64 ± 0.54; mid: M = 10.55 ± 4.15; high: M = 17.10 ± 4.24) which was most evident with higher doses of drug. D2 blockade did not alter choice between probabilistic and effortful options (F (3, 15) = 2.173, p = 0.134, Fig. 5a), and there was no dose by effort requirement interaction (F (9, 45) = 1.100, p = 0.382, Fig. 5a). As detailed in the “Methods” section, percent choice of the effortful option during blocks in which rats omitted all trials was interpolated using the slope between completed blocks. However, five subjects required exclusion from the ANOVA due to omissions beyond the scope of interpolating missing data. An LMEM constructed to control for omissions replicated the null effects of D2 blockade on EvP (Table 2), though the effect of dose did trend toward significance (p = 0.095). However, D2 antagonism significantly reduced AUC across blocks 2–4, the blocks in which effort and risk were in conflict (F (3, 24) = 4.640, p = 0.011). Paired-samples t tests were used to compare AUC of each dose to saline, and the shift toward the risky option became more pronounced as dosage increased (low dose D2 antagonist: t (9) = 2.269, p = 0.049; mid: t (9) = 2.666, p = 0.026; high: t (9) = 4.251, p = 0.002).

Fig. 5
figure 5

Modulation of D2 receptors unilaterally shifts preference toward probabilistic reinforcement. a The D2 antagonist Eticlopride dose-dependently reduced effortful choice during economic conflict (percent effortful choice AUC during EvP blocks 2–4). b All doses of the D2 agonist quinpirole shifted preference from effortful to probabilistic choice. c During the reversed EvP (10 lever press effort vs increasing risk), preference was again shifted from effortful to probabilistic choice by all doses of the D2 agonist. All panels depict mean ± SEM

Full size image

Activation of D2 receptors via quinpirole elicited a significant shift in preference from those entailing effortful costs to those with probabilistic costs throughout the entire session (F (3, 27) = 21.315, p < 0.001, Fig. 5b), further illustrated by a reduction of AUC (F (3, 27) = 28.329, p < 0.001). This shift was comparable across all doses compared to saline (low: F (1, 10) = 71.328, p < 0.001; mid: F (1, 9) = 100.761, p < 0.001; High: F (1, 10) = 42.067, p < 0.001, Fig. 5b). This shift toward probabilistic from effortful choice was even observed in the first block (1 press vs 25%) during which there is no economic conflict and the “probabilistic” option is objectively disadvantageous (F (3, 27) = 3.640, p = 0.025, Fig. 5b). There was no dose by block interaction (F (9, 81) = 1.516, p = 0.156), and D2 activation dose dependently increased omitted trials (F (3, 30) = 8.802, p < 0.001). One subject was removed from post hoc comparisons between saline and mid dose sessions due to excessive omissions during behavior after mid dose administration; to account for this, we used LMEM to reanalyze the data, which replicated the significant effect of drug treatment and the lack of interaction (Table 2).

Experiment 5: role of D2 activation during modified economic conflict task

In experiment 4, activating D2 receptors markedly reduced preference for options entailing effort in favor of easier options involving probabilistic risk. During the EvP, the effort requirement became increasingly difficult across the session, while probabilistic choice maintained a consistent 25% rate of reinforcement. This difference in contingencies between options raises the concern that D2 activation shifted preference toward probabilistic over effortful costs by increasing preference for stable (probability) over dynamic (effort) parameters. To address this, we trained subjects in the modified task previously used in experiment 1 in which the effort requirement was fixed at 10 presses throughout the session, and the probabilistic choice became increasingly risky (100%, 75%, 50%, 25% chance of reinforcement), then again tested all doses of the D2 agonist. D2 activation again shifted preference toward the probabilistic option away from the effortful one (F (1.836, 16.522) = 4.812, p = 0.025, Fig. 5c), and elicited a dose by effort requirement interaction (F (9, 81) = 2.243, p = 0.027, Fig. 5c). During true economic conflict (blocks 2–4), AUC was also shifted from the effortful to the probabilistic option (F (1.861, 16.751) = 6.604, p = 0.009). Thus, D2 activation during economic conflict selectively increases preference for probabilistic over effortful costs rather than causing a general preference for consistent costs. As in experiment 4, D2 activation caused a dose-dependent increase in omissions (F (1.921, 21.135) = 5.960, p = 0.009), although they remained low even at the highest dose (M = 2.94, SEM = .85), and 2 subjects were removed from ANOVA analyses due to excessive omissions. The LMEM constructed to control for excluded subjects did not produce a dose effect, though a trend toward dose × effort interaction was identified (Table 2).

Discussion

Conflicts between high effort and high-risk reward seeking (such as a steady job vs. playing the lottery) are a common occurrence in everyday decision-making. We developed the effort vs probability economic conflict task (EvP) to measure choice between rewards of comparable magnitude associated with distinct costs, either physical effort or risk of omission. We observed that rats demonstrate flexible decision-making that shifts toward the other option when either effort or risk is increased, and that intrinsic preference for high effort vs high risk options remains stable over long periods of time. Increasing synaptic dopamine with the psychostimulant cocaine shifted preference from probabilistic to effortful choice, but this was limited to conditions with low effort requirements. D1 dopamine receptors played a limited role in EvP, with blockade eliciting no effect, and D1 agonist dose-dependently increasing probabilistic choice, though this effect was limited to trials with a high effort requirement. D2 receptor blockade shifted preferences away from effort and toward probabilistic costs, but also caused a substantial increase in omissions. D2 activation produced a robust shift in choice of risky over effortful costs across all doses and effort requirements.

Validation of EvP task

Rats were able to detect changes in effortful and probabilistic costs and adjust behavior accordingly, shifting choice toward probabilistic costs as effortful costs became increasingly difficult. This provided a novel translational assessment of complex, real-world analogous decision-making in which all available options include some measure of either effort or risk. Furthermore, most cost-benefit decision-making tasks involve choice between rewards of different magnitude (Wade et al. 2000; Floresco et al. 2008a; Salamone et al. 2016; Orsini et al. 2018), whereas choice in the EvP is unaffected by potential differences in reward magnitude discrimination or preference, as the outcome of all rewarded trials is identical (one pellet). This enables detection of effects of experimental manipulation that are less vulnerable to gross motivational factors.

In the EvP, rewards associated with the probabilistic choice were delivered at a fixed rate of 25% of trials, and the effort requirement increased throughout the session. To confirm that patterns of decision-making were not driven by which factor was fixed/dynamic, subjects trained in an inverted version of the task in which effort-requirement was fixed at 10 lever presses and probability of reinforcement decreased throughout the session. Subjects shifted preference from the probabilistic toward the effortful option as probability decreased, thus demonstrating the propensity to shift away from less valuable options regardless of which variable is dynamic. Moreover, subjects’ decision-making was unchanged by this reversal of effort and probability parameters, demonstrating comparable preference between effortful vs probabilistic costs regardless of which factor was fixed, as well as similar performance at the determined “indifference point”, defined as the block in EvP with the least preference for one option over the other (10 lever presses effort vs 25% probability). This indicates that decision-making behavior in the EvP reflects intrinsic preference between effortful and probabilistic costs that is not strongly confounded by task design.

Choice of the effortful option required time for the subjects to complete all lever presses demanded by the effort requirement, and this time increased throughout the session as effort requirement increased. Therefore, it was difficult to discern if subjects were directly weighing effort vs probability or devaluing the effort as a function of the delay required to complete multiple lever presses (i.e., delay discounting). To address this, subjects trained in another modified EvP which substituted delay for effort. Rats shifted preference toward the probabilistic choice as delay increased; however, choice preference differed from the standard EvP, with rats demonstrating greater discounting of effortful than delayed options. This confirmed that preference for probabilistic over effortful costs in EvP was not solely accounted for by delay discounting, while also revealing that effort is tabulated as a greater cost than delay during comparison with uncertainty.

Finally, we observed that preference between effort and probabilistic costs was highly consistent across 2–3 months, suggesting that preference between effort and risk is a stable behavioral phenotype that is unchanged by the passage of time or transient alterations in neurochemistry. Subjects in each cohort received drug schedules in the same order (although doses were counterbalanced within each schedule); thus, it is possible that there were carryover effects that influenced subsequent drug treatments. However, EvP choice preference between baseline saline sessions was consistent between drug protocols, suggesting that the earlier drug schedules did not exert enduring influences on decision-making. Furthermore, the drastic shift from effortful to probabilistic choice induced by D2 receptor activation was observed in two distinct cohorts that had different drug histories, suggesting that prior exposure to a specific regimen of drugs did not drive this effect. This consistency over time suggests that EvP provides a reliable measurement of individual differences in choice during economic conflict, with some subjects potentially capturing the suboptimal preference for high risk over high effort options observed in many psychiatric populations (Brevers et al. 2014; Saddoris et al. 2016; Verdejo-García et al. 2018).

Effects of cocaine on EvP

The dopamine system regulates value-based decision-making (Killcross et al. 1997; Stopper et al. 2014). Psychostimulants such as cocaine increase mesolimbic dopamine activity via inhibition of the dopamine transporter (Mcelvain and Schenk 1992; Allain et al. 2015). We tested the acute effects of cocaine on behavior in the EvP to measure the effects of a dopaminergic drug with strong abuse potential on preference between effort and probability. The effects of cocaine were highly selective: only low and medium (but not high) dose cocaine affected decision-making, manifesting as a shift in preference from probabilistic to effortful options that was only evident with low effort requirements (1 and 5 LP). This subtle shift in preference from probabilistic toward low effort costs was somewhat surprising, as cocaine has been shown to reduce effort expenditure in high fixed ratio tasks (van Haaren and Anderson 1994), and acute cocaine shifts preference toward risky options in a probabilistic discounting task (Mai et al. 2015). However, it is likely that the effects of cocaine (and possibly other psychostimulants) differ between situations with a single discounting factor (effort or probability) vs. situations with multiple factors in conflict. This dissociation suggests that choosing between effortful and probabilistic options may recruit different neuronal circuitry than decision-making requiring evaluation of only one of these factors.

Interestingly, despite increasing synaptic dopamine activity, effects of cocaine contrast with the shift from probabilistic to effortful options produced by both D1 and D2 activation. Cocaine elevates synaptic norepinephrine and serotonin as well as dopamine (Pierce and Kalivas 1997). Therefore, cocaine effects on EvP may be influenced by these other monoamines, or by interactions between dopamine and these other neurotransmitter systems. Indeed, the noradrenergic system is implicated in the integration of effort with reward (Varazzani et al. 2015), and simultaneous blockade of either dopamine and norepinephrine transporters or norepinephrine and serotonin transporters has been shown to alter decision-making in risky decision-making tasks involving probabilistic reward (Baarendse et al. 2013; Cui et al. 2018). It will be of interest for future studies to test the acute effects of amphetamine on EvP, as amphetamine influences both effortful and risky decision-making and has less impact on serotonin transmission than cocaine (Pierce and Kalivas 1997; Andrews and Lucki 2001; Floresco et al. 2008b; St. Onge and Floresco 2009).

Parsing the roles of dopamine receptor subtypes in EvP

The D1- and D2-like dopamine receptor subtypes have dissociable roles in both effortful and probabilistic choices (Floresco et al. 2008b; St. Onge and Floresco 2009; Orsini et al. 2015; Grall-Bronnec et al. 2016; Salamone et al. 2016). However, the role of specific dopamine receptor subtypes when effortful and probabilistic options are in conflict is undetermined. To address this, we tested the effects of D1- and D2-like agonists and antagonists on choice behavior in the EvP.

D1-like receptor modulation

Previous studies have reported that preference for both probabilistic and effortful reinforcement is reduced by D1 antagonists (St. Onge and Floresco 2009; Yohn et al. 2015). However, when risk and effort costs are weighed against each other in EvP, D1 blockade did not affect decision-making. It is possible that D1 blockade simultaneously increases the discounting of both risky and effortful costs in EvP, thus eliciting no behavioral change in either direction. This also provides further evidence that effort discounting in EvP is evaluated differently than delay discounting, as D1 blockade increased preference for delayed over immediate rewards (Koffarnus et al. 2011; Tian et al. 2019), but did not influence choice in EvP.

D1 activation biased choice toward high risk options and away from effortful ones, though this was selective for trials with the most difficult effort requirement (20 LP) and was only evident at the highest dose (0.3 mg/kg). Previous studies have shown that D1 activation increases both effort and probabilistic choice in separate effort and probabilistic discounting tasks (St. Onge and Floresco 2009; Nunes et al. 2010; Yohn et al. 2015). Rather than these preferences canceling out (as with the D1 antagonist), the D1 agonist causes a selective shift toward risk, suggesting that increased preference for risk is more tightly linked to D1 receptors than preference for effort. Interestingly, high dose systemic D1 agonists in humans cause decreased preference for larger, risky rewards and increased effort expenditure for rewards (Soutschek et al. 2020), which contrasts with the results here. This is likely related to task differences (the human tasks did not involve choice between effortful and probabilistic costs in the same session) or differences in decision-making processes between rodents and humans. In sum, the effects of D1 receptor manipulation on behavior in the EvP suggest that sensitivity to effort may be more tightly linked to D1 receptors than sensitivity to risk in rat models.

D2-like receptor modulation

D2 receptor blockade shifted behavior away from the effortful choice toward the probabilistic option. This is consistent with a report that D2 antagonists reduce overall effortful decision-making (Yohn et al. 2015), though it contrasts with evoked reductions in risky choice (St. Onge and Floresco 2009; Grall-Bronnec et al. 2016). Thus, D2 blockade’s reduction in effortful choice supersedes reductions in risk preference. These effects were dose dependent, with higher doses causing an increased shift away from the effortful choice/toward the risky choice. Critically, the observed effect of D2 blockade in the EvP should be interpreted with caution due to the drastic increase in omitted trials, especially at the higher doses.

The most notable change in decision-making during EvP was elicited by D2 activation, which shifted preference away from effort toward probabilistic costs across all conditions of effort requirement across all doses. This suggests that decision-making in the face of conflicting effortful and probabilistic choices may rely heavily on activation of D2 dopamine receptors. While the effects of systemic D2 agonists on effortful choice have not been clearly identified, stimulating D2 receptors in both the shell and core subregions of the nucleus accumbens reduces effortful choice (Bryce and Floresco 2019). Additionally, D2 receptor activation is closely tied to gambling and preference for uncertain reinforcement in both humans and rats (St. Onge and Floresco 2009; Cocker et al. 2016; Grall-Bronnec et al. 2016; Tremblay et al. 2017). Therefore, it is likely that D2 activation produces a twofold effect on decision-making in EvP, making the explicit cost associated with effortful choice more aversive and reducing the cost of possible reward omission. This summation of effects may explain the extremely robust, dose-independent risk preference evoked by the D2 agonist in EvP.

The D2 agonist-evoked shift from effortful to probabilistic choice persisted in the modified EvP, in which effort requirement was fixed, and the probabilistic cost became increasingly risky. This confirmed that D2 activation makes uncertain/risky choice extremely compelling regardless of whether effort or probability remained fixed throughout the session. The D2 activation–evoked preference for risky choice may arise from summative effects on the discounting of both effortful and risky reinforcement.

Interestingly, D2 activation elicited probabilistic preference in the initial block of EvP, during which the probabilistic option was objectively inferior to the effortful option. In this block, the effortful lever press requirement was only a single action, but the probabilistic option was only reinforced on 25% of trials. Regardless, D2 activation induced a significant shift toward this disadvantageous option, suggesting that D2 activation increases the value of uncertain reinforcement independent of effort level.

Another potential explanation for the observed shift from effortful to probabilistic choice is the role of presynaptic D2 autoreceptors in reducing phasic dopamine release (Ford 2014) and motivation for food rewards (Bello et al. 2012). By attenuating phasic release, D2 autoreceptor activation consequentially reduces postsynaptic D1 activation, which attenuates motivated behavior (Depoortere et al. 1993; Wall et al. 2011; Sugam et al. 2014; Saddoris et al. 2015). This suggests that the shift away from effortful choices may arise from reduced overall motivation to perform instrumental responding due to secondary effects on D1 receptors. This is an unlikely explanation, however, as direct D1 blockade did not affect choice in the EvP. Furthermore, reduced instrumental motivation does not account for the shift from effortful to probabilistic options when effort requirements were identical. It is therefore likely that D2 receptor activation drastically shifts preference from effortful to probabilistic choice by making uncertain rewards highly compelling.

While dose sequence was counterbalanced within each drug schedule, the order of different drug treatments was the same for all subjects in a cohort (Table 1). It is thus possible that previous drug exposure influenced the effect of subsequent manipulations, or that subtle baseline shifts accounted for drug effects. However, it is unlikely that carryover effects accounted for the observed changes in EvP, as drastic D2 agonist-induced shifts from effortful to probabilistic choice preference were observed in both cohorts despite having been previously exposed to different drugs and different lengths of training. In addition, there were no statistical differences between saline administration sessions for all dopamine-specific drug schedules.

Notably, both blockade and activation of D2 receptors evoked comparable effects on decision-making, with any disruption of baseline D2 receptor transmission causing a shift toward risky reinforcement and away from effortful reinforcement. This contrasts with risky decision-making tasks that do not include an effort component, in which D2 activation increases risky choice while D2 antagonism reduces risky choice (St. Onge and Floresco 2009). Thus, adding an effort component to a risky decision-making task appears to alter effects of dopamine receptor manipulation, manifested as a nonlinear relationship between decision-making and D2 receptor blockade/activation. This is consistent with other aspects of cognition that are regulated in nonlinear fashion by dopamine transmission (Robbins 2005), including punishment-based risky decision-making, wherein either high or low levels of D2 receptor abundance in medial prefrontal cortex is associated with inflexible decision-making (Simon et al. 2011).

Conclusions and implications

To our knowledge, this study marks the first instance in which decision-making between two equivalent rewards associated with either high effort or high risk was assessed in rodents. Rats demonstrated the ability to compare risk and effort, then adapt choice in congruence with changing contingencies. The EvP extends insight from tasks involving a single discounting factor (Evenden and Ryan 1996; Cardinal and Howes 2005; Floresco et al. 2008b; Winstanley and Floresco 2016) to more complex situations, demonstrating that, in some cases, the presence of multiple discounting factors modifies how dopaminergic drugs influence decision-making. Critically, this study suggests that dopamine manipulation, particularly via D2-like receptors, may serve as a potential therapeutic avenue for correcting pathologically disordered preference for high risk over high effort outcomes seen in substance use disorder, ADHD, and gambling disorder (Green and Myerson 2004; Crowley et al. 2006; Floresco et al. 2008a; Cocker et al. 2016; Koffarnus and Kaplan 2018).