Abstract
Rationale
The schedule of drug availability may enhance choice of a drug. In non-human subjects, reinforcers are chosen more often when available under variable schedules of reinforcement relative to fixed schedules.
Objective
To determine whether variable-drug access is an important determinant of cocaine choice by manipulating the schedule, drug dose, and combination of schedule + dose.
Method
Four male rhesus monkeys chose between cocaine doses (0.025–0.4 mg/kg/injection). In control conditions, the schedule and dose of each drug delivery were fixed. In other conditions, the reinforcement schedule (i.e., variable-ratio schedule), dose of each cocaine delivery, or both were variable on one lever while all aspects on the other lever remained fixed.
Results
When cocaine dose was equal on average (0.1 mg/kg/injection), 2 of 4 subjects chose cocaine associated with the variable schedule more than the fixed schedule. All subjects chose the variable dose that was equal on average to the fixed dose, and this difference was statistically significant. Three of 4 subjects chose cocaine associated with the variable combination over the fixed option (when the dose was equal on average). During dose-response determinations (when dose on the variable and fixed options were not equal), making the schedule, dose, or both variable generally did not alter cocaine’s potency as a reinforcer.
Conclusion
While many factors contribute to drug choice, unpredictable drug access is a feature that may be common in the natural environment and could play a key role in the allocation of behavior to drug alternatives by patients with substance-use disorders.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
Introduction
Drug choice can be affected by amount, probability, delay, and cost of drug and non-drug reinforcers (e.g., Hart et al. 2000; Higgins et al. 1994; Nader and Woolverton 1991, 1992; Negus 2003; Stoops et al. 2012; Woolverton and Anderson 2006; Woolverton and Rowlett 1998). Drug-choice studies have informed behavioral treatments like contingency management (CM), where tangible reinforcers, or the chance to receive tangible reinforcers, are delivered contingent on drug-free urine samples (e.g., Packer et al. 2012; Petry et al. 2000; Silverman et al. 1999). However, the extent to which unpredictable drug access impacts drug choice is relatively unexplored.
In the laboratory, effects of environmental manipulations on drug choice usually apply similar response-reinforcer relations for both options (e.g., fixed-ratio (FR) schedules). However, individuals with substance use disorders (SUDs) likely receive drug and non-drug reinforcers under different schedules. A non-drug reinforcer such as a paycheck generally occurs at predictable points in time, in exchange for specific amounts of behavior. Other non-drug reinforcers like food or hobbies are available in predictable locations, at relatively predictable prices. However, illicit drugs are less predictable in terms of availability, quality, location, time, and price.
Predictability can be manipulated in the laboratory. Under fixed-ratio (FR) schedules, reinforcers are delivered after a fixed and predictable number of responses. With variable-ratio (VR) schedules, an unpredictable number of responses are required. Behavior maintained by VR schedules occurs at high, steady rates and is more resistant to extinction and satiation (Catania 1972; Mellon and Shull 1986). The observation that more responding and reinforcers occur with a variable (i.e., random-ratio) schedule relative to a fixed one (Madden et al. 2005) was recently extended to drug self-administration in rhesus monkeys (Lagorio and Winger 2014). Moreover, human and non-human subjects choose non-drug reinforcers available under variable schedules more often than fixed schedules that require, on average, the same number of responses or same delay to reinforcer delivery (e.g., Fantino 1967; Field et al. 1996; Lagorio and Hackenberg 2010; Locey et al. 2009). Choice of a non-drug reinforcer available under a VR schedule occurs even when the average requirement is greater than that of the FR schedule (Ahearn et al. 1992; Johnson et al. 2011, 2012).
Predictability of illicit-drug reinforcers may mirror variable schedules in that drug seeking sometimes results in relatively immediate reinforcement such as the high associated with drug taking or relief from withdrawal. Other times, it requires a large amount of time and behavior, resulting in delayed reinforcement. In addition to schedule, illicit drugs may be less predictable in terms of quality or potency, factors that can be modeled in the laboratory. To date, there are no studies examining choice between fixed and variable doses of a drug reinforcer. When choice is between fixed and variable food amounts, results are mixed with different studies showing greater choice of a variable food amount and others showing aversion or indifference (e.g., Bateson and Kacelnik 1995; Essock and Reese 1974; Lagorio and Hackenberg 2012; McSweeney et al. 2003; see Kacelnik and Bateson 1996 for a review).
The purpose of this experiment was to determine if variability impacts cocaine choice. We examined choice between variable and fixed cocaine availability by manipulating the schedule of reinforcement, dose, and their combination. We hypothesized that (1) subjects would choose cocaine available under variable conditions over fixed conditions, (2) variable cocaine conditions would increase its potency, and (3) combining variability in schedule and dose would have greater effects on choice than would single conditions of variability relative to fixed cocaine availability.
Materials and methods
All procedures were approved by the University of Mississippi Medical Center’s (UMMC) Institutional Animal Care and Use Committee and were conducted in accordance with the National Research Council’s Guide for Care and Use of Laboratory Animals (8th edition, 2011).
Subjects, apparatus, and surgery
Four male rhesus monkeys (Macaca mulatta) were subjects. Prior to catheterization, subjects were trained to lever press with food pellets. Other than training, subjects 314206 and 274-2009 were experimentally naïve, subject 5274 had experience choosing between cocaine and M&M candies, and 314172 had a history of delay discounting with drug and non-drug reinforcers (unpublished data). Subjects received unlimited access to water and were fed standard biscuits (Teklad 25% Monkey Diet, Harlan/Teklad, Madison, WI) to maintain healthy body weights. Fruit, foraging materials, multivitamins, light cycle, jacket/tether system, home cage, and implantation of catheters were as described previously (Huskinson et al. 2016). Presurgery, subjects received preoperative antibiotics (cefazolin; 20–25 mg/kg, i.m.) and analgesics (carprofen, 2–4 mg/kg, s.c. and/or buprenorphine SR, 0.05 mg/kg, s.c.), along with postoperative analgesics (carprofen, 4 mg/kg, p.o) daily for at least 3 days and antibiotics (usually Keflex, 22.2 mg/kg, p.o. or i.m.; Eli Lilly & Company, Indianapolis, IN) as needed. If a catheter became non-functional, it was removed, and a new catheter was implanted once health was verified by veterinary staff.
General procedure
Sessions were conducted daily at the same time and consisted of 6 samples and 12 choice trials. During samples, one lever was active, signaled by illumination of its white lights; its consequence was delivered after the response requirement was completed. Active levers were randomly determined at the start of each session and alternated thereafter. Sample trials were followed by choice trials, during which both sets of white lights were illuminated, and consequences of associated with both levers were available. For choice trials, a single response on either lever darkened the lights on the opposite lever to prevent switching (e.g., Rider 1983), and responses to that lever were recorded but had no other programmed consequences.
During cocaine delivery, the white lights associated with the lever that was pressed were darkened, and the associated red lights were illuminated during the injection. Trials were separated by a 20-min timeout, during which all lights were darkened, and lever presses were recorded but had no programmed consequences. Conditions were in effect for at least 3 sessions and until choice was stable, though more than 3 sessions were typically needed to meet criteria (group mean = 7.9 sessions, subject range = 6.5–9.2 sessions). Stability required choice of one lever to be within 20% of the mean for 3 consecutive sessions, no upward/downward trends over 3 consecutive sessions, and completion of all trials in a session. Once choice was stable, the schedule and reinforcer associated with each lever was reversed, and choice was re-determined. Each data point represents three stable sessions of a lever-injection pairing and its reversal.
Fixed-control condition
Cocaine choice was examined using the procedure described above when 0.1 mg/kg/injection of cocaine, delivered over a 10-s injection period, was available under concurrent and identical FR 30 schedules. During this condition, 30 responses were required to receive 0.1 mg/kg/injection of cocaine associated with either the left or right lever (see Table 1). This condition served as a baseline against which variable conditions could be compared.
Variable-schedule condition
Cocaine choice was examined under concurrent FR and VR schedules. In this condition, 0.1 mg/kg/injection of cocaine was available on both options; one lever was associated with an FR 30 and the other lever with a VR 30 (see Table 1). The VR required, on average, an equal number of responses as the FR option. Three values constituted the VR requirement: 1, 30, and 59 responses to allow for small, equal, or large requirement on the VR relative to the constant FR value (e.g., Fantino 1967; Johnson et al. 2011, 2012; Rider 1983). During sample trials on the VR lever, each value was presented once in a random order. During choice trials, if the alternative associated with the VR lever was chosen, values were presented pseudo-randomly, using an order that ensured the same value could not be presented on more than 4 trials in a row, and if the variable option was chosen on all 12 trials, all values were presented the same number of times. All subjects experienced the variable-schedule condition twice.
Variable-dose condition
Cocaine choice was examined under concurrent FR 30 schedules. However, one lever was associated with a 10-s injection of cocaine (0.1 mg/kg/injection). The other option produced a 2- (0.02 mg/kg/injection), 10- (0.1 mg/kg/injection), or 18-s (0.18 mg/kg/injection) injection of cocaine that on average equaled 10 s and 0.1 mg/kg/injection (see Table 1). The total intake possible on the fixed option was the same as the total intake possible on the variable option. Variable injection durations were presented in sample and choice trials as described for the variable-schedule condition. All subjects experienced this condition twice.
Variable schedule + dose condition
This condition was similar to the separate variable-schedule and dose conditions described above, except that the variable option consisted of a combination of the VR 30 schedule and the variable doses. Each response requirement (1, 30, 59) and injection duration (2, 10, 18 s) were presented in a random order during sample trials. Within a sample or choice trial, it was possible for any combination of the 3 requirements and 3 injection durations to occur together on the same trial. The fixed lever remained constant at an FR 30 with a 10-s injection of 0.1 mg/kg/injection of cocaine (see Table 1). The variable schedule + dose condition occurred twice for 3 subjects and once for subject 314172.
Dose-effect determinations
Dose-effect determinations occurred under fixed-control conditions to ensure a larger cocaine dose was chosen over a smaller dose and to determine a baseline to test for potency shifts under conditions of variability (described below). Dose on one lever was constant (0.1 mg/kg/injection), while the other changed to 0.05 and 0.2 mg/kg/injection in two conditions. Data from the fixed-control condition with 0.1 vs. 0.1 mg/kg/injection was used as an intermediate point for the fixed-control dose-effect curve (see Table 2).
For each subject, if an effect was obtained in a variable condition (Table 1) with the same average cocaine dose, at least one additional dose was determined for that dimension of variability. If no effect occurred in a condition in Table 1 with the same average cocaine dose, additional doses were not tested. For example, if a subject chose the variable dose over the fixed dose in the variable-dose condition, additional doses were determined by changing average cocaine dose on the variable option and holding cocaine dose constant on the fixed option, and vice versa by changing cocaine dose on the fixed option and holding average cocaine dose constant on the variable option (see Table 2). Data from the corresponding variable condition with 0.1 vs. 0.1 mg/kg/injection were used as an intermediate data point for each dose-effect curve.
Initially, exposure to all conditions were counterbalanced. Over time, we presented fixed control with equal doses first, followed by variable schedule, variable dose, and variable combination with 0.1 mg/kg/injection in an irregular order across subjects (Table 1). Additional doses were determined only if an effect was obtained with equal doses, and conditions in Table 2 were presented in an irregular order across subjects and interspersed with Table 1 conditions.
Data analysis
In all conditions, mean choice data from the stable sessions of each lever-injection pairing and its reversal were plotted for each condition and were used to evaluate data from individual subjects and the group. When a condition was experienced twice, the average of both determinations was used in analysis, because the first determination and replication of each were not statistically different. A one-sample t test compared the mean of each condition to 50% to determine whether each condition was statistically different from indifference. A repeated-measures analysis of variance (ANOVA) followed by Dunnett’s multiple-comparisons tests were used to compare choice in each variable condition to the fixed control. Geisser-Greenhouse method corrected for violations of sphericity. Finally, obtained schedule values and injection durations were calculated to ensure they did not differ widely from programmed values, and within-session analyses determined whether small, intermediate, or large values associated with the variable option were likely to result in selecting the variable or fixed option on the subsequent trial.
For conditions in Table 2, parallel and linear dose-effect curves were not consistently obtained during dose-effect determinations so ED50 values were not calculated, and conclusions were drawn on visual inspection of individual-subject data. Obtained values and within-session responding for each condition were not reported because an overall effect of variability was not observed. When the variable option was the smaller dose, it was chosen on a small number of trials, resulting in skewed values. Within-session patterns were not reported for these conditions, because choice was nearly exclusive for one option, and thus, switching rarely occurred.
Drugs
Cocaine hydrochloride was provided by the National Institute on Drug Abuse (Rockville, MD). Solutions were prepared using 0.9% sterile saline. Doses are expressed as the salt form.
Results
Figure 1 shows percent choice of the fixed option as a function of the condition available on the opposite lever for Table 1 conditions when choice was between the same average cocaine dose (0.1 mg/kg/injection). In the fixed control (hashed bars), choice was approximately 50% for all subjects, and the group mean (rightmost panel) indicated indifference between the same cocaine dose under the same fixed parameters of delivery. In the variable-schedule condition (gray bars), 2 subjects (314206 and 247-2009) chose cocaine associated with the VR schedule more than the FR schedule, one subject (314172) chose cocaine associated with the VR schedule more than the FR schedule during the first determination but tended to choose the opposite during the second determination, and one subject (5274) was indifferent between cocaine associated with the FR or VR schedule. The programmed ratio for the VR schedule was 30, and for the first determination, the group mean obtained ratio was 30.6 (subject range = 29.3–31.9). The group mean for the second determination was 27.8 (subject range = 22.8–31.2). The latter mean was largely influenced by subject 5274 who did not choose cocaine associated with the VR schedule over the FR schedule. For other subjects, obtained ratios ranged from 28.1 to 31.2.
In the variable-dose condition (Fig. 1, black bars), all subjects chose the variable dose over the fixed dose, though this effect was small for subject 5274. For other subjects, choice of the variable-dose option tended to be more extreme during the first determination relative to the second determination; however, the first and second determinations were not statistically different in the group analysis. The programmed injection duration for the variable dose was 10 s, and for the first determination, the group mean obtained duration was 10.2 s (subject range = 9.8–10.8 s), and for the second determination was 9.8 s (subject range = 8.9–10.1 s).
In the variable schedule + dose condition (Fig. 1, white bars), subject 314172 was indifferent while the other three subjects chose cocaine associated with the variable option over the fixed option. Again, this effect was generally larger for subjects 314206 and 274-2009 relative to 5274 and during the first determination relative to the second, but the first and second determinations were not statistically different. The average obtained ratio on the variable option was 27.0 (subject range = 22.3–30.6) and was largely influenced by the subject (314172) who did not choose cocaine associated with the variable option over the fixed option (obtained ratio = 22.3). The mean obtained injection duration in the first determination was 10.7 s (subject range = 9.9–11.8 s). For the second determination, the mean obtained ratio was 30.9 (subject range = 30.2–31.5), and the mean obtained duration was 10.4 s (subject range = 10.1–10.8 s).
Percent-choice analyses for the group were generally consistent with individual-subject data. The fixed-control and variable-schedule conditions were not significantly different from 50% [p’s > .05]. The variable-dose cocaine was chosen significantly more than 50% [t(3) = 6.3, p < .01], and the variable schedule + dose condition narrowly missed significance [t(3) = 3.1, p = .053]. A repeated-measures ANOVA indicated a main effect of conditions of variability [F(1.7, 5.2) = 9.2, p = .02], and the variable-dose condition was significantly different from fixed control [p < .01]. The variable schedule + dose effect approached significance [p = .056].
The within-session analyses for Table 1 conditions were conducted to determine if small, intermediate, or large values associated with the variable option were more or less likely to be followed by selecting cocaine associated with the variable option again or switching to the fixed option on the subsequent trial was largely unsystematic across subjects, replications, or conditions. Subjects with similar choice profiles (e.g., 314206 and 274-2009) did not show similar within-session choice patterns. In the combination condition, subjects were more likely to switch between response alternatives on a within-session basis during the second determination relative to the first determination. However, the effect was not systematic in that a small, intermediate, or large value did not predict when a subject was likely to switch between response options. The former effect was not observed in the variable schedule or dose conditions. Therefore, it is unlikely within-session behavior contributed to the observed patterns of choice behavior.
Figure 2 shows dose-effect determinations for the variable-schedule condition for the two subjects who chose cocaine associated with the variable schedule in Fig. 1. Under fixed-control conditions (triangles), choice was indifferent when the test dose was the same as the constant dose (0.1 mg/kg/injection). When the test dose was raised to 0.2 mg/kg/injection, choice was almost exclusive for the larger dose, and vice versa; when the test dose was lowered to 0.05 mg/kg/injection, choice of the test dose was less than 20%. If our hypothesis that making cocaine variable would increase its potency as a reinforcer is true, this would be demonstrated by parallel shifts in the dose-effect determinations. When the dose was the same on both options, 2 subjects (314206 and 274-2009) chose cocaine associated with the VR schedule over the FR schedule. However, when the test dose was lowered on the VR lever (circles), subjects chose cocaine associated with the FR schedule over the VR schedule, and similarly, when the test dose was raised on the FR lever (squares), subjects chose cocaine associated with the FR schedule over the VR schedule. In other words, subjects chose the higher doses regardless of schedule.
Figure 3 shows dose-effect determinations for the variable-dose condition. Data shown for the fixed control are the same as in Fig. 2 for subjects 314172 and 274-2009. Under the fixed control, all subjects chose the larger dose over the smaller dose as doses were adjusted. For the variable-dose conditions, all subjects chose the variable dose over the fixed one when the dose on both options was equal on average (filled symbols; 0.1 mg/kg/injection). For subject 314206, a leftward shift was observed when the test dose on the variable lever was lowered (circles), indicating that a lower, variable cocaine dose (0.05 mg/kg/injection) was chosen more often than a larger, constant dose (0.1 mg/kg/injection). A similar effect was obtained for this subject when the test dose was raised on the fixed option and remained at 0.1 mg/kg/injection on the variable option (squares). However, when the test dose was lowered on the variable lever (circles), the other subjects chose the fixed and larger dose over the variable one. Similarly, when the test dose was raised on the fixed option (squares), these subjects chose that option over the variable one. Thus, parallel shifts in the dose-effect determinations were not observed for 3 of 4 subjects.
Figure 4 shows dose-effect determinations for the variable schedule + dose condition for the three subjects who chose cocaine associated with this variable option in Fig. 1. Data for the fixed control are the same as in Fig. 3. For the variable conditions, these subjects chose cocaine associated with the variable option over the fixed one when the schedule and dose on both options was, on average, equal. For subject 5274, a leftward shift was obtained when the test dose on the variable option was decreased (circles). That is, a lower, variable dose (0.05 mg/kg/injection) was chosen as often as a larger, constant dose (0.1 mg/kg/injection). A similar effect was not observed for this subject when the test dose was raised on the fixed option and remained at 0.1 mg/kg/injection on the variable option (squares). For other subjects, when the test dose was lowered on the variable option (circles), subjects chose cocaine associated with the fixed option over the variable one. Similarly, when the test dose was raised on the fixed option (squares), subjects chose cocaine associated with the fixed option over the variable one.
Discussion
This study demonstrates that rhesus monkeys chose cocaine associated with a variable option more often than cocaine associated with a fixed option when average dose was equal (in 9 of 12 conditions across subjects, determinations, and dimensions of variability). Results are somewhat consistent with the literature with rats and pigeons (e.g., Fantino 1967; Herrnstein 1964; Rider 1983), in which animals choose food associated with a VR over an FR schedule, even when the choice resulted in net reinforcer loss (Johnson et al. 2011, 2012). In this study, individual differences occurred. Two subjects who chose cocaine associated with a VR schedule were experimentally naïve at the beginning of the experiment, while the two who did not had different histories, making it impossible to determine which factors resulted in a lack of cocaine choice associated with the VR schedule. Individual differences might be overcome with larger requirements (e.g., FR and VR 100) as larger average ratio values typically result in more extreme choices with VR versus FR schedules (e.g., Fantino 1967). Our results in the variable-dose condition were more robust relative to the variable-schedule or combination of schedule + dose condition, and this pattern of results is somewhat different from that of rats and pigeons where choice between variable vs. fixed food amounts is mixed (Lagorio and Hackenberg 2012; McSweeney et al. 2003; see Kacelnik and Bateson 1996 for a review). Monkeys may be more sensitive to variability in the dose of a drug than rats and pigeons are to variability in food amounts. However, procedural differences also occurred, and additional studies would be necessary to determine potential species, reinforcer types, or procedural differences.
Our hypothesis that the combination of variability in dose + schedule would result in greater effects on choice than either condition of variability alone was not supported. Variability in dose may have exerted control over choice behavior in the combination condition for subject 5274, in which choice shifted toward the variable option in the variable-dose and variable-combination condition but not the variable-schedule condition. For subjects 314206 and 274-2009, cocaine associated with the single conditions was chosen to a similar extent as the combination. Perhaps a floor effect prevented more extreme choices, or combining conditions of variability are not additive. Subject 314172, who chose cocaine associated with the variable-dose option but not the variable-schedule option, did not choose cocaine associated with the combination over the fixed option, suggesting that, for this subject, variability in dose during the combination condition was overshadowed by variability in schedule. Finally, it is possible that the combination condition was so complex that choice did not come under control of the condition of variability.
When the dose of one cocaine option was raised or lowered, subjects chose the dose that was larger on average. In other words, cocaine choice associated with a variable option was overcome by raising the dose on the fixed option. Parallel shifts in the dose-response determinations were not observed in most cases, indicating that variability did not alter cocaine’s potency as a reinforcer. Conversely, variable cocaine may function as a more effective reinforcer. Lagorio and Winger (2014) evaluated the reinforcing effectiveness of remifentanil, cocaine, and ketamine in rhesus monkeys by varying the cost of drug injections using an FR or random-ratio (RR) schedule. Demand was less elastic with the RR schedule than the FR, indicating that variable response requirements increased the effectiveness of the drugs as reinforcers. These effects were larger with remifentanil and cocaine than ketamine, at larger requirements, and with lower doses, suggesting that this phenomenon exerted more control over behavior under conditions of lean reinforcement (Lagorio and Winger 2014). This supports our prediction that larger average requirements may have resulted in more extreme choice of cocaine associated with a VR schedule, and potency effects may have emerged during dose-response determinations. Although cocaine’s potency as a reinforcer was not affected under the parameters of this study, variable conditions of cocaine access may act as a “tie breaker” when choice is between equal average doses.
Many hypotheses attempt to explain behavioral mechanisms that could account for disproportionate choice of reinforcers associated with variable schedules over fixed ones. Choice of reinforcers associated with VR schedules is most robust when the smallest possible ratio value is 1, and the effect goes away as the smallest possible ratio value approaches that of the FR schedule (e.g., Fantino 1967; Field et al. 1996). A quick payoff may have more value than the larger requirements that result in longer delays to reinforcer delivery (e.g., Soreth and Hineline 2009). In the present study, a similar relation may have occurred, where the occasional delivery of larger doses had a disproportionately greater effect on choice than did the occasional delivery of smaller doses in the variable-dose condition. Conversely, the number of component ratios does not appear to affect choice of reinforcers associated with variable schedules (i.e., similar findings are reported with two-value VR schedules and with four- or nine-value VR schedules; Fantino 1967; Johnson et al. 2011; Sherman and Thomas 1968). To our knowledge, one study has directly compared choice between food associated with fixed- and variable-interval schedules, and the number of component intervals (2, 3, or 7 values) did not affect choice (Davison 1972).
Madden et al. (2007, 2011) have proposed that choice of reinforcers associated with VR schedules can be explained within a delay-discounting framework (i.e., the subjective value of a delayed reinforcer is a hyperbolic function of the delay to its receipt). According to this hypothesis, the combined value of a reinforcer under the different requirements of a VR schedule is subjectively greater than the value of the fixed but always delayed reinforcer, especially with relatively large response requirements. Choice of reinforcers associated with VR schedules should positively correlate with an individual’s degree of delay discounting, such that more rapid loss of reinforcer value with increasing delay is correlated with greater choice of reinforcers associated with VR schedules (Madden et al. 2011). In the current study, there were individual differences with the variable schedule. We also have observed individual differences in delay discounting in rhesus monkeys (Freeman et al. 2009, 2012; Huskinson et al. 2015, 2016; Woolverton et al. 2007). Future work could examine whether delay discounting and degree of choice between reinforcers associated with FR and VR schedules are correlated.
Accordingly, drug-dependent individuals discount delayed outcomes more steeply than do their non-dependent counterparts (for meta-analyses, see Amlung et al. 2016; MacKillop et al. 2011) and discount hypothetical drug outcomes more steeply than hypothetical money (e.g., Madden et al. 1997). If drug reinforcers in the natural environment are unpredictable relative to non-drug reinforcers, and if degree of discounting can account for choice of unpredictable outcomes, unpredictable drug availability may play a facilitative role in the allocation of behavior by drug-dependent individuals in the acquisition of drug reinforcers. However, additional research is needed to determine the relation between delay discounting and unpredictable outcomes.
In addition to potential behavioral mechanisms, there is neurochemical evidence that unpredictable reinforcer deliveries result in greater dopamine release in the nucleus accumbens (NAc) core in rats (Sugam et al. 2012) and increased activity in dopamine neurons of ventral midbrain areas in monkeys (Fiorillo et al. 2003). Conversely, omitted reinforcers on unpredictable trials resulted in decreases in dopamine release in the NAc core compared to predictable reinforcer deliveries (Sugam et al. 2012). Reinforcers in the present study were never omitted, but it is possible that their unpredictable delivery resulted in greater dopamine release compared to fixed reinforcer deliveries. Perhaps parallel shifts in the dose-response functions were not obtained because increasing the dose of cocaine may have been sufficient to increase levels of dopamine following fixed reinforcer delivery to a greater extent than the variable, but lower-dose option.
The current results raise important questions. For example, long-term exposure to variable drug availability may increase impulsive choice, resistance to extinction or punishment, escalation of drug-taking, and the reinforcing effectiveness of drugs. Variable exposure to drugs conceivably might hamper an individual’s response to treatment, while a variable schedule of non-drug delivery also may be an effective means of treatment (i.e., contingency management (CM)).
Importantly, CM is the most effective psychosocial treatment for substance abuse (Dutra et al. 2008), and clinical research in CM has examined effects of a probabilistic schedule of non-drug reinforcer delivery under a “prize” schedule as well as “standard” CM (e.g., Olmstead and Petry 2009; Petry and Alessi 2010; Petry and Martin 2002; Petry et al. 2005a, 2005b). In standard CM, participants earn escalating vouchers contingent on drug-free urine samples, exchangeable for non-drug reinforcers (see Higgins and Silverman 2008). A voucher of some monetary value is delivered for every drug-free urine sample, and the exact value of the vouchers escalate in a known manner with consecutive drug-free samples. In contrast, in prize CM, participants earn increasing numbers of draws from a prize bowl for each consecutive negative urine sample submitted. The probability of winning a prize for each draw is usually 50%, and higher-magnitude prizes (e.g., $100) are always available but at low probabilities (e.g., 0.02%). Prize CM is at least equally efficacious to standard voucher CM, even when the expected amount of reinforcement for perfect performance is lower with prizes than vouchers (Petry et al. 2005a, 2015), and this prize system is more cost effective (Olmstead et al. 2007). Future research with non-human subjects could determine parameters of reinforcement that are most effective at reducing drug choice and increasing non-drug choice. Variable schedules of non-drug delivery may more effectively compete with and reduce drug choices than fixed schedules.
We have known for several decades that behavior maintained by variable schedules occurs at a high rate, with little pausing between response bouts, and is more resistant to extinction (e.g., Catania 1992; Mellon and Shull 1986). While many factors contribute to the excessive amount of behavior spent by individuals with SUDs in the acquisition of drugs, results observed in the present and previous studies (e.g., Lagorio and Winger 2014; Madden et al. 2005, 2007, 2011) suggest that variability in at least some dimensions of drug delivery (e.g., quality, cost, delay) could be a contributing factor to the amount of behavior allocated toward procuring drug reinforcers in a population known to show steep discounting of delayed outcomes. In other words, scarce and uncertain conditions of drug availability and quality in the natural environment could be a contributing factor to drug choice and demand. Behavioral interventions may benefit by capitalizing on the use of variable schedules for non-drug reinforcers to counteract drug use.
References
Ahearn W, Hineline PN, David FG (1992) Relative preferences for various bivalued ratio schedules. Anim Learn Behav 20:407–415. doi:10.3758/BF03197964
Amlung M, Vedelago L, Acker J, Balodis I, MacKillop J (2016) Steep delay discounting and addictive behavior: a meta-analysis of continuous associations. Addiction 112:51–62. doi:10.1111/add.13535
Bateson M, Kacelnik A (1995) Preferences for fixed and variable food sources: variability in amount and delay. J Exp Anal Behav 63:313–329. doi:10.1901/jeab.1995.63-313
Catania AC (1992). Learning. Englewood Cliffs, New Jersey: Prentice-Hall, Inc.
Davison MC (1972) Preference for mixed-interval versus fixed-interval schedules: number of component intervals. J Exp Anal Behav 17:169–176. doi:10.1901/jeab.1972.17-169
Dutra L, Stathopoulou G, Basden SL, Leyro TM, Powers MB, Otto MW (2008) A meta-analytic review of psychosocial interventions for substance use disorders. Am J Psychiatry 165:179–187. doi:10.1176/appi.ajp.2007.06111851
Essock SM, Reese EP (1974) Preference for and effects of variable- as opposed to fixed- reinforcer duration. J Exp Anal Behav 21:89–97. doi:10.1901/jeab.1974.21-89
Fantino E (1967) Preference for mixed- versus fixed-ratio schedules. J Exp Anal Behav 10:35–43. doi:10.1901/jeab.1967.10-35
Field DP, Tonneau F, Ahearn W, Hineline PN (1996) Preference between variable-ratio and fixed-ratio schedules: local and extended relations. J Exp Anal Behav 66:283–295. doi:10.1901/jeab.1996.66-283
Fiorillo CD, Tobler PN, Schultz W (2003) Discrete coding of reward probability and uncertainty by dopamine neurons. Science 299:1898–1902. doi:10.1126/science.1077349
Freeman KB, Green L, Myerson J, Woolverton WL (2009) Delay discounting of saccharin in rhesus monkeys. Behav Process 82:214–218. doi:10.1016/j.beproc.2009.06.002
Freeman KB, Nonnemacher JE, Green L, Myerson J, Woolverton WL (2012) Delay discounting in rhesus monkeys: equivalent discounting of more and less preferred sucrose concentrations. Learn Behav 40:54–60. doi:10.3758/s13420-011-0045-3
Hart CL, Haney M, Foltin RW, Fischman MW (2000) Alternative reinforcers differentially modify cocaine self-administration in humans. Behav Pharmacol 11:87–91
Herrnstein RJ (1964) Aperiodicity as a factor in choice. J Exp Anal Behav 7:179–128. doi:10.1901/jeab.1964.7-179
Higgins ST, Bickel WK, Hughes JR (1994) Influence of an alternative reinforcer on human cocaine self-administration. Life Sci 55:179–187. doi:10.1016/0024- 3205(94)00878-7
Higgins ST, Silverman K (2008) Contingency management. In The American Psychiatric Publishing Textbook of Substance Abuse Treatment, 4th Edition. American Psychiatric Publishing, Inc.
Huskinson SL, Woolverton WL, Green L, Myerson J, Freeman KB (2015) Delay discounting of food by rhesus monkeys: cocaine and food choice in isomorphic and allomorphic situations. Exp Clin Psychopharmacol 23:184–193. doi:10.1037/pha0000015
Huskinson SL, Myerson J, Green L, Rowlett JK, Woolverton WL, Freeman KB (2016) Shallow discounting of delayed cocaine by male rhesus monkeys when immediate food is the choice alternative. Exp Clin Psychopharmacol 24:456–463. doi:10.1037/pha0000098
Johnson PS, Madden GJ, Brewer AT, Pinkston JW, Fowler SC (2011) Effects of acute pramipexole on preference for gambling-like schedules of reinforcement in rats. Psychopharmacol 213:11–18. doi:10.1007/s00213-010-2006-5
Johnson PS, Madden GJ, Stein JS (2012) Effects of acute pramipexole on male rats’ preference for gambling-like rewards II. Exp Clin Psychopharmacol 20:167–172. doi:10.1037/a0027117
Kacelnik A, Bateson M (1996) Risky theories—the effects of variance on foraging decisions. Am Zool 36:402–434. doi:10.1093/icb/36.4.402
Lagorio CH, Hackenberg TD (2010) Risky choice in pigeons and humans: a cross-species comparison. J Exp Anal Behav 93:27–44. doi:10.1901/jeab.2010.93-27
Lagorio CH, Hackenberg TD (2012) Risky choice in pigeons: preference for amount variability using a token-reinforcement system. J Exp Anal Behav 98:139–154. doi:10.1901/jeab.2012.98-139
Lagorio CH, Winger G (2014) Random-ratio schedules produce greater demand for iv drug administration than fixed-ratio schedules in rhesus monkeys. Psychopharmacol 231:2981–2988. doi:10.1007/s00213-014-3477-6
Locey ML, Pietras CJ, Hackenberg TD (2009) Human risky choice: delay sensitivity depends on reinforcer type. J Exp Psychol 35:15–22. doi:10.1037/a0012378
MacKillop J, Amlung MT, Few LR, Ray LA, Sweet LH, Munafò MR (2011) Delayed reward discounting and addictive behavior: a meta-analysis. Psychopharmacol 216:305–321. doi:10.1007/s00213-011-2229-0
Madden GJ, Dake JM, Mauel EC, Rowe RR (2005) Labor supply and consumption of food in a closed economy under a range of fixed- and random-ratio schedules: tests of unit price. J Exp Anal Behav 83:99–118. doi:10.1901/jeab.2005.32-04
Madden GJ, Ewwan EE, Lagorio CH (2007) Toward an animal model of gambling: delay discounting and the allure of unpredictable outcomes. J Gambl Stud 23:63–83. doi:10.1007/s10899-006-9041-5
Madden GJ, Francisco MT, Brewer AT, Stein JS (2011) Delay discounting and gambling. Behav Process 87:43–49. doi:10.1016/j.beproc.2011.01.012
Madden GJ, Petry NM, Badger GJ, Bickel WK (1997) Impulsive and self-control choices in opioid-dependent patients and non-drug-using control patients: drug and monetary rewards. Exp Clin Psychopharmacol 5:256–262. doi:10.1037/1064-1297.5.3.256
McSweeney FK, Kowal BP, Murphy ES (2003) The effect of rate of reinforcement and time in session on preference for variability. Anim Learn Behav 31:225–241. doi:10.3758/BF03195985
Mellon RC, Shull RL (1986) Resistance to change produced by access to fixed-delay versus variable-delay terminal links. J Exp Anal Behav 46:79–92. doi:10.1901/jeab.1986.46-79
Nader MA, Woolverton WL (1991) Effects of increasing the magnitude of an alternative reinforcer on drug choice in a discrete-trials choice procedure. Psychopharmacol 105:169–174. doi:10.1007/BF02244304
Nader MA, Woolverton WL (1992) Effects of increasing response requirement on choice between cocaine and food in rhesus monkeys. Psychopharmacol 108:295–300. doi:10.1007/BF02245115
Negus SS (2003) Rapid assessment of choice between cocaine and food in rhesus monkeys: effects of environmental manipulations and treatment with d-amphetamine and flupenthixol. Neuropsychopharmacology 28:919–931. doi:10.1038/sj.npp.1300096
Olmstead TA, Petry NM (2009) The cost-effectiveness of prize-based and voucher-based contingency management in a population of cocaine- or opioid-dependent outpatients. Drug Alcohol Depend 102:108–115. doi:10.1016/j.drugalcdep.2009.02.005
Packer RR, Howell DN, McPherson S, Roll JM (2012) Investigating reinforcer magnitude and reinforcer delay: a contingency management analog study. Exp Clin Psychopharmacol 20:287–292. doi:10.1037/a0027802
Petry NM, Alessi SM (2010) Prize-based contingency management is efficacious in cocaine abusing patients with and without recent gambling participation. J Subst Abus Treat 39:282–288. doi:10.1016/j.jsat.2010.06.011
Petry NM, Alessi SM, Barry D, Carroll KM (2015) Standard magnitude prize reinforcers can be as efficacious as larger magnitude reinforcers in cocaine-dependent methadone patients. J Consult Clin Psychol 83:464–472. doi:10.1037/a0037888
Petry NM, Alessi SM, Marx J, Austin M, Tardif M (2005a) Vouchers versus prizes: contingency management treatment of substance abusers in community settings. J Consult Clin Psychol 73:1005–1014. doi:10.1037/0022-006X.73.6.1005
Petry NM, Martin B (2002) Low-cost contingency management for treating cocaine- and opioid-abusing methadone patients. J Consult Clin Psychol 70:398–405. doi:10.1037/0022-006X.70.2.398
Petry NM, Martin B, Cooney JL, Kranzler HR (2000) Give them prizes, and they will come: contingency management for treatment of alcohol dependence. J Consult Clin Psychol 68:250–257. doi:10.1037/0022-006X.68.2.250
Petry NM, Peirce JM, Stitzer ML, Blaine J, Roll JM, Cohen A, Obert J, Killeen T, Saladin ME, Cowell M, Kirby KC (2005b) Effect of prize-based incentives on outcomes in stimulant abusers in outpatient psychosocial treatment programs: a national drug abuse treatment clinical trials network study. Arch Gen Psychiatry 62:1148–1156. doi:10.1001/archpsyc.62.10.1148
Rider DP (1983) Choice for aperiodic versus periodic ratio schedules: a comparison of concurrent and concurrent-chains procedures. J Exp Anal Behav 40:225–237. doi:10.1901/jeab.1983.40-225
Sherman JA, Thomas JR (1968) Some factors controlling preference between fixed-ratio and variable-ratio schedules of reinforcement. J Exp Anal Behav 11:689–702. doi:10.1901/jeab.1968.11-689
Silverman K, Chutuape M, Bigelow GE, Stitzer ML (1999) Voucher-based reinforcement of cocaine abstinence in treatment-resistant methadone patients: effects of reinforcement magnitude. Psychopharmacol 146:128–138. doi:10.1007/s002130051098
Soreth ME, Hineline PN (2009) The probability of small schedule values and preference for random-interval schedules. J Exp Anal Behav 91:89–103. doi:10.1901/jeab.2009.91-89
Stoops WW, Lile JA, Glaser PEA, Hays LR, Rush CR (2012) Alternative reinforcer response costs impacts cocaine choice in humans. Progress in Neuro-Psychopharmacol & Biological Psychiatry 36:189–193. doi:10.1016/j.pnpbp.2011.10.003
Sugam JA, Day JJ, Wightman RM, Carelli RM (2012) Phasic nucleus accumbens dopamine encodes risk-based decision-making behavior. Biol Psychiatry 71:19–205. doi:10.1016/j.biopsych.2011.09.029
Woolverton WL, Anderson KG (2006) Effects of delay to reinforcement on the choice between cocaine and food in rhesus monkeys. Psychopharmacol 186:99–106. doi:10.1007/s00213-006-0355-x
Woolverton WL, Myerson J, Green L (2007) Delay discounting of cocaine by rhesus monkeys. Exp Clin Psychopharmacol 15:238–244. doi:10.1037/1064-1297.15.3.238
Woolverton WL, Rowlett JK (1998) Choice maintained by cocaine or food in monkeys: effects of varying probability of reinforcement. Psychopharmacol 138:102–106. doi:10.1007/s002130050651
Acknowledgements
This research and manuscript preparation were supported by the National Institute on Drug Abuse (NIDA) grants F32 DA037619 to S.L.H, R01 DA027666, and R01 DA039167 to K.B.F. and R01 DA033795 and R01 DA011792 to J.K.R. The authors would like to thank Jacob Smith, Talal Ahmed, Georganna Nutt, and Hunter Bruce for their technical assistance.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Rights and permissions
About this article
Cite this article
Huskinson, S., Freeman, K., Petry, N. et al. Choice between variable and fixed cocaine injections in male rhesus monkeys. Psychopharmacology 234, 2353–2364 (2017). https://doi.org/10.1007/s00213-017-4659-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00213-017-4659-9