Introduction

Drug choice can be affected by amount, probability, delay, and cost of drug and non-drug reinforcers (e.g., Hart et al. 2000; Higgins et al. 1994; Nader and Woolverton 1991, 1992; Negus 2003; Stoops et al. 2012; Woolverton and Anderson 2006; Woolverton and Rowlett 1998). Drug-choice studies have informed behavioral treatments like contingency management (CM), where tangible reinforcers, or the chance to receive tangible reinforcers, are delivered contingent on drug-free urine samples (e.g., Packer et al. 2012; Petry et al. 2000; Silverman et al. 1999). However, the extent to which unpredictable drug access impacts drug choice is relatively unexplored.

In the laboratory, effects of environmental manipulations on drug choice usually apply similar response-reinforcer relations for both options (e.g., fixed-ratio (FR) schedules). However, individuals with substance use disorders (SUDs) likely receive drug and non-drug reinforcers under different schedules. A non-drug reinforcer such as a paycheck generally occurs at predictable points in time, in exchange for specific amounts of behavior. Other non-drug reinforcers like food or hobbies are available in predictable locations, at relatively predictable prices. However, illicit drugs are less predictable in terms of availability, quality, location, time, and price.

Predictability can be manipulated in the laboratory. Under fixed-ratio (FR) schedules, reinforcers are delivered after a fixed and predictable number of responses. With variable-ratio (VR) schedules, an unpredictable number of responses are required. Behavior maintained by VR schedules occurs at high, steady rates and is more resistant to extinction and satiation (Catania 1972; Mellon and Shull 1986). The observation that more responding and reinforcers occur with a variable (i.e., random-ratio) schedule relative to a fixed one (Madden et al. 2005) was recently extended to drug self-administration in rhesus monkeys (Lagorio and Winger 2014). Moreover, human and non-human subjects choose non-drug reinforcers available under variable schedules more often than fixed schedules that require, on average, the same number of responses or same delay to reinforcer delivery (e.g., Fantino 1967; Field et al. 1996; Lagorio and Hackenberg 2010; Locey et al. 2009). Choice of a non-drug reinforcer available under a VR schedule occurs even when the average requirement is greater than that of the FR schedule (Ahearn et al. 1992; Johnson et al. 2011, 2012).

Predictability of illicit-drug reinforcers may mirror variable schedules in that drug seeking sometimes results in relatively immediate reinforcement such as the high associated with drug taking or relief from withdrawal. Other times, it requires a large amount of time and behavior, resulting in delayed reinforcement. In addition to schedule, illicit drugs may be less predictable in terms of quality or potency, factors that can be modeled in the laboratory. To date, there are no studies examining choice between fixed and variable doses of a drug reinforcer. When choice is between fixed and variable food amounts, results are mixed with different studies showing greater choice of a variable food amount and others showing aversion or indifference (e.g., Bateson and Kacelnik 1995; Essock and Reese 1974; Lagorio and Hackenberg 2012; McSweeney et al. 2003; see Kacelnik and Bateson 1996 for a review).

The purpose of this experiment was to determine if variability impacts cocaine choice. We examined choice between variable and fixed cocaine availability by manipulating the schedule of reinforcement, dose, and their combination. We hypothesized that (1) subjects would choose cocaine available under variable conditions over fixed conditions, (2) variable cocaine conditions would increase its potency, and (3) combining variability in schedule and dose would have greater effects on choice than would single conditions of variability relative to fixed cocaine availability.

Materials and methods

All procedures were approved by the University of Mississippi Medical Center’s (UMMC) Institutional Animal Care and Use Committee and were conducted in accordance with the National Research Council’s Guide for Care and Use of Laboratory Animals (8th edition, 2011).

Subjects, apparatus, and surgery

Four male rhesus monkeys (Macaca mulatta) were subjects. Prior to catheterization, subjects were trained to lever press with food pellets. Other than training, subjects 314206 and 274-2009 were experimentally naïve, subject 5274 had experience choosing between cocaine and M&M candies, and 314172 had a history of delay discounting with drug and non-drug reinforcers (unpublished data). Subjects received unlimited access to water and were fed standard biscuits (Teklad 25% Monkey Diet, Harlan/Teklad, Madison, WI) to maintain healthy body weights. Fruit, foraging materials, multivitamins, light cycle, jacket/tether system, home cage, and implantation of catheters were as described previously (Huskinson et al. 2016). Presurgery, subjects received preoperative antibiotics (cefazolin; 20–25 mg/kg, i.m.) and analgesics (carprofen, 2–4 mg/kg, s.c. and/or buprenorphine SR, 0.05 mg/kg, s.c.), along with postoperative analgesics (carprofen, 4 mg/kg, p.o) daily for at least 3 days and antibiotics (usually Keflex, 22.2 mg/kg, p.o. or i.m.; Eli Lilly & Company, Indianapolis, IN) as needed. If a catheter became non-functional, it was removed, and a new catheter was implanted once health was verified by veterinary staff.

General procedure

Sessions were conducted daily at the same time and consisted of 6 samples and 12 choice trials. During samples, one lever was active, signaled by illumination of its white lights; its consequence was delivered after the response requirement was completed. Active levers were randomly determined at the start of each session and alternated thereafter. Sample trials were followed by choice trials, during which both sets of white lights were illuminated, and consequences of associated with both levers were available. For choice trials, a single response on either lever darkened the lights on the opposite lever to prevent switching (e.g., Rider 1983), and responses to that lever were recorded but had no other programmed consequences.

During cocaine delivery, the white lights associated with the lever that was pressed were darkened, and the associated red lights were illuminated during the injection. Trials were separated by a 20-min timeout, during which all lights were darkened, and lever presses were recorded but had no programmed consequences. Conditions were in effect for at least 3 sessions and until choice was stable, though more than 3 sessions were typically needed to meet criteria (group mean = 7.9 sessions, subject range = 6.5–9.2 sessions). Stability required choice of one lever to be within 20% of the mean for 3 consecutive sessions, no upward/downward trends over 3 consecutive sessions, and completion of all trials in a session. Once choice was stable, the schedule and reinforcer associated with each lever was reversed, and choice was re-determined. Each data point represents three stable sessions of a lever-injection pairing and its reversal.

Fixed-control condition

Cocaine choice was examined using the procedure described above when 0.1 mg/kg/injection of cocaine, delivered over a 10-s injection period, was available under concurrent and identical FR 30 schedules. During this condition, 30 responses were required to receive 0.1 mg/kg/injection of cocaine associated with either the left or right lever (see Table 1). This condition served as a baseline against which variable conditions could be compared.

Table 1 Parameters associated with each lever during conditions comparing the same average dose of cocaine on each lever (0.1 mg/kg/injection)
Full size table

Variable-schedule condition

Cocaine choice was examined under concurrent FR and VR schedules. In this condition, 0.1 mg/kg/injection of cocaine was available on both options; one lever was associated with an FR 30 and the other lever with a VR 30 (see Table 1). The VR required, on average, an equal number of responses as the FR option. Three values constituted the VR requirement: 1, 30, and 59 responses to allow for small, equal, or large requirement on the VR relative to the constant FR value (e.g., Fantino 1967; Johnson et al. 2011, 2012; Rider 1983). During sample trials on the VR lever, each value was presented once in a random order. During choice trials, if the alternative associated with the VR lever was chosen, values were presented pseudo-randomly, using an order that ensured the same value could not be presented on more than 4 trials in a row, and if the variable option was chosen on all 12 trials, all values were presented the same number of times. All subjects experienced the variable-schedule condition twice.

Variable-dose condition

Cocaine choice was examined under concurrent FR 30 schedules. However, one lever was associated with a 10-s injection of cocaine (0.1 mg/kg/injection). The other option produced a 2- (0.02 mg/kg/injection), 10- (0.1 mg/kg/injection), or 18-s (0.18 mg/kg/injection) injection of cocaine that on average equaled 10 s and 0.1 mg/kg/injection (see Table 1). The total intake possible on the fixed option was the same as the total intake possible on the variable option. Variable injection durations were presented in sample and choice trials as described for the variable-schedule condition. All subjects experienced this condition twice.

Variable schedule + dose condition

This condition was similar to the separate variable-schedule and dose conditions described above, except that the variable option consisted of a combination of the VR 30 schedule and the variable doses. Each response requirement (1, 30, 59) and injection duration (2, 10, 18 s) were presented in a random order during sample trials. Within a sample or choice trial, it was possible for any combination of the 3 requirements and 3 injection durations to occur together on the same trial. The fixed lever remained constant at an FR 30 with a 10-s injection of 0.1 mg/kg/injection of cocaine (see Table 1). The variable schedule + dose condition occurred twice for 3 subjects and once for subject 314172.

Dose-effect determinations

Dose-effect determinations occurred under fixed-control conditions to ensure a larger cocaine dose was chosen over a smaller dose and to determine a baseline to test for potency shifts under conditions of variability (described below). Dose on one lever was constant (0.1 mg/kg/injection), while the other changed to 0.05 and 0.2 mg/kg/injection in two conditions. Data from the fixed-control condition with 0.1 vs. 0.1 mg/kg/injection was used as an intermediate point for the fixed-control dose-effect curve (see Table 2).

Table 2 Parameters associated with each lever during dose-effect determinations across conditions of variability
Full size table

For each subject, if an effect was obtained in a variable condition (Table 1) with the same average cocaine dose, at least one additional dose was determined for that dimension of variability. If no effect occurred in a condition in Table 1 with the same average cocaine dose, additional doses were not tested. For example, if a subject chose the variable dose over the fixed dose in the variable-dose condition, additional doses were determined by changing average cocaine dose on the variable option and holding cocaine dose constant on the fixed option, and vice versa by changing cocaine dose on the fixed option and holding average cocaine dose constant on the variable option (see Table 2). Data from the corresponding variable condition with 0.1 vs. 0.1 mg/kg/injection were used as an intermediate data point for each dose-effect curve.

Initially, exposure to all conditions were counterbalanced. Over time, we presented fixed control with equal doses first, followed by variable schedule, variable dose, and variable combination with 0.1 mg/kg/injection in an irregular order across subjects (Table 1). Additional doses were determined only if an effect was obtained with equal doses, and conditions in Table 2 were presented in an irregular order across subjects and interspersed with Table 1 conditions.

Data analysis

In all conditions, mean choice data from the stable sessions of each lever-injection pairing and its reversal were plotted for each condition and were used to evaluate data from individual subjects and the group. When a condition was experienced twice, the average of both determinations was used in analysis, because the first determination and replication of each were not statistically different. A one-sample t test compared the mean of each condition to 50% to determine whether each condition was statistically different from indifference. A repeated-measures analysis of variance (ANOVA) followed by Dunnett’s multiple-comparisons tests were used to compare choice in each variable condition to the fixed control. Geisser-Greenhouse method corrected for violations of sphericity. Finally, obtained schedule values and injection durations were calculated to ensure they did not differ widely from programmed values, and within-session analyses determined whether small, intermediate, or large values associated with the variable option were likely to result in selecting the variable or fixed option on the subsequent trial.

For conditions in Table 2, parallel and linear dose-effect curves were not consistently obtained during dose-effect determinations so ED50 values were not calculated, and conclusions were drawn on visual inspection of individual-subject data. Obtained values and within-session responding for each condition were not reported because an overall effect of variability was not observed. When the variable option was the smaller dose, it was chosen on a small number of trials, resulting in skewed values. Within-session patterns were not reported for these conditions, because choice was nearly exclusive for one option, and thus, switching rarely occurred.

Drugs

Cocaine hydrochloride was provided by the National Institute on Drug Abuse (Rockville, MD). Solutions were prepared using 0.9% sterile saline. Doses are expressed as the salt form.

Results

Figure 1 shows percent choice of the fixed option as a function of the condition available on the opposite lever for Table 1 conditions when choice was between the same average cocaine dose (0.1 mg/kg/injection). In the fixed control (hashed bars), choice was approximately 50% for all subjects, and the group mean (rightmost panel) indicated indifference between the same cocaine dose under the same fixed parameters of delivery. In the variable-schedule condition (gray bars), 2 subjects (314206 and 247-2009) chose cocaine associated with the VR schedule more than the FR schedule, one subject (314172) chose cocaine associated with the VR schedule more than the FR schedule during the first determination but tended to choose the opposite during the second determination, and one subject (5274) was indifferent between cocaine associated with the FR or VR schedule. The programmed ratio for the VR schedule was 30, and for the first determination, the group mean obtained ratio was 30.6 (subject range = 29.3–31.9). The group mean for the second determination was 27.8 (subject range = 22.8–31.2). The latter mean was largely influenced by subject 5274 who did not choose cocaine associated with the VR schedule over the FR schedule. For other subjects, obtained ratios ranged from 28.1 to 31.2.

Fig. 1
figure 1

Mean percent choice of cocaine associated with the fixed lever as a function of the condition available on the opposite lever when the average dose of cocaine (0.1 mg/kg/injection) was the same on both options. Data are shown for individual monkeys in each panel (subject number is indicated in each panel) and the group mean in the bottom right panel. Each bar is the average of the final 3 sessions of the initial lever-injection pairing and the final 3 sessions of the reversal. The first bar among each pair is the first time the condition was determined, and the second bar among each pair is the replication. Replications did not occur for the fixed-control condition for any subject or for the variable schedule + dose condition for subject 314172. The dashed line indicates 50%, and error bars are one standard error of the mean (SEM). Double asterisks indicate significance (p < .01) compared to fixed control

Full size image

In the variable-dose condition (Fig. 1, black bars), all subjects chose the variable dose over the fixed dose, though this effect was small for subject 5274. For other subjects, choice of the variable-dose option tended to be more extreme during the first determination relative to the second determination; however, the first and second determinations were not statistically different in the group analysis. The programmed injection duration for the variable dose was 10 s, and for the first determination, the group mean obtained duration was 10.2 s (subject range = 9.8–10.8 s), and for the second determination was 9.8 s (subject range = 8.9–10.1 s).

In the variable schedule + dose condition (Fig. 1, white bars), subject 314172 was indifferent while the other three subjects chose cocaine associated with the variable option over the fixed option. Again, this effect was generally larger for subjects 314206 and 274-2009 relative to 5274 and during the first determination relative to the second, but the first and second determinations were not statistically different. The average obtained ratio on the variable option was 27.0 (subject range = 22.3–30.6) and was largely influenced by the subject (314172) who did not choose cocaine associated with the variable option over the fixed option (obtained ratio = 22.3). The mean obtained injection duration in the first determination was 10.7 s (subject range = 9.9–11.8 s). For the second determination, the mean obtained ratio was 30.9 (subject range = 30.2–31.5), and the mean obtained duration was 10.4 s (subject range = 10.1–10.8 s).

Percent-choice analyses for the group were generally consistent with individual-subject data. The fixed-control and variable-schedule conditions were not significantly different from 50% [p’s > .05]. The variable-dose cocaine was chosen significantly more than 50% [t(3) = 6.3, p < .01], and the variable schedule + dose condition narrowly missed significance [t(3) = 3.1, p = .053]. A repeated-measures ANOVA indicated a main effect of conditions of variability [F(1.7, 5.2) = 9.2, p = .02], and the variable-dose condition was significantly different from fixed control [p < .01]. The variable schedule + dose effect approached significance [p = .056].

The within-session analyses for Table 1 conditions were conducted to determine if small, intermediate, or large values associated with the variable option were more or less likely to be followed by selecting cocaine associated with the variable option again or switching to the fixed option on the subsequent trial was largely unsystematic across subjects, replications, or conditions. Subjects with similar choice profiles (e.g., 314206 and 274-2009) did not show similar within-session choice patterns. In the combination condition, subjects were more likely to switch between response alternatives on a within-session basis during the second determination relative to the first determination. However, the effect was not systematic in that a small, intermediate, or large value did not predict when a subject was likely to switch between response options. The former effect was not observed in the variable schedule or dose conditions. Therefore, it is unlikely within-session behavior contributed to the observed patterns of choice behavior.

Figure 2 shows dose-effect determinations for the variable-schedule condition for the two subjects who chose cocaine associated with the variable schedule in Fig. 1. Under fixed-control conditions (triangles), choice was indifferent when the test dose was the same as the constant dose (0.1 mg/kg/injection). When the test dose was raised to 0.2 mg/kg/injection, choice was almost exclusive for the larger dose, and vice versa; when the test dose was lowered to 0.05 mg/kg/injection, choice of the test dose was less than 20%. If our hypothesis that making cocaine variable would increase its potency as a reinforcer is true, this would be demonstrated by parallel shifts in the dose-effect determinations. When the dose was the same on both options, 2 subjects (314206 and 274-2009) chose cocaine associated with the VR schedule over the FR schedule. However, when the test dose was lowered on the VR lever (circles), subjects chose cocaine associated with the FR schedule over the VR schedule, and similarly, when the test dose was raised on the FR lever (squares), subjects chose cocaine associated with the FR schedule over the VR schedule. In other words, subjects chose the higher doses regardless of schedule.

Fig. 2
figure 2

Mean percent choice of the test dose as a function of the amount (mg/kg/injection) of the test dose for the 2 (of 4) monkeys (subject number is indicated in each panel) who experienced conditions when the test dose (0.05–0.2 mg/kg/injection) was available under a VR schedule vs. a comparator 0.1 mg/kg/injection available under an FR schedule (circles) or vice versa, when the test dose (0.05–0.2 mg/kg/injection) was available under an FR schedule vs. a comparator 0.1 mg/kg/injection available under a VR schedule (squares). Triangles indicate fixed conditions when the test dose (0.05–0.2 mg/kg/injection) and comparator 0.1 mg/kg/injection were both available under FR schedules. The dashed line indicates 50%, and error bars are one SEM

Full size image

Figure 3 shows dose-effect determinations for the variable-dose condition. Data shown for the fixed control are the same as in Fig. 2 for subjects 314172 and 274-2009. Under the fixed control, all subjects chose the larger dose over the smaller dose as doses were adjusted. For the variable-dose conditions, all subjects chose the variable dose over the fixed one when the dose on both options was equal on average (filled symbols; 0.1 mg/kg/injection). For subject 314206, a leftward shift was observed when the test dose on the variable lever was lowered (circles), indicating that a lower, variable cocaine dose (0.05 mg/kg/injection) was chosen more often than a larger, constant dose (0.1 mg/kg/injection). A similar effect was obtained for this subject when the test dose was raised on the fixed option and remained at 0.1 mg/kg/injection on the variable option (squares). However, when the test dose was lowered on the variable lever (circles), the other subjects chose the fixed and larger dose over the variable one. Similarly, when the test dose was raised on the fixed option (squares), these subjects chose that option over the variable one. Thus, parallel shifts in the dose-effect determinations were not observed for 3 of 4 subjects.

Fig. 3
figure 3

Mean percent choice of the test dose as a function of the amount (mg/kg/injection) of the test dose for all monkeys (subject number is indicated in each panel) during conditions when the test dose (0.025–0.2 mg/kg/injection) was variable vs. a comparator 0.1 mg/kg/injection (circles) or vice versa, when the test dose (0.05–0.4 mg/kg/injection) was available under a fixed dose and a comparator 0.1 mg/kg/injection was variable (squares). Triangles indicate fixed conditions when the test dose (0.05–0.2 mg/kg/injection) and comparator 0.1 mg/kg/injection were both fixed. The dashed line indicates 50%, and error bars are one SEM

Full size image

Figure 4 shows dose-effect determinations for the variable schedule + dose condition for the three subjects who chose cocaine associated with this variable option in Fig. 1. Data for the fixed control are the same as in Fig. 3. For the variable conditions, these subjects chose cocaine associated with the variable option over the fixed one when the schedule and dose on both options was, on average, equal. For subject 5274, a leftward shift was obtained when the test dose on the variable option was decreased (circles). That is, a lower, variable dose (0.05 mg/kg/injection) was chosen as often as a larger, constant dose (0.1 mg/kg/injection). A similar effect was not observed for this subject when the test dose was raised on the fixed option and remained at 0.1 mg/kg/injection on the variable option (squares). For other subjects, when the test dose was lowered on the variable option (circles), subjects chose cocaine associated with the fixed option over the variable one. Similarly, when the test dose was raised on the fixed option (squares), subjects chose cocaine associated with the fixed option over the variable one.

Fig. 4
figure 4

Mean percent choice of the test dose as a function of the amount (mg/kg/injection) of the test dose for the 3 (of 4) monkeys (subject number is indicated in each panel) who experienced conditions when the test dose (0.025–0.2 mg/kg/injection) was available under a variable schedule + dose vs. a comparator 0.1 mg/kg/injection (circles) or vice versa, when the test dose (0.05–0.2 mg/kg/injection) was available under a fixed schedule + dose and a comparator 0.1 mg/kg/injection was available under a variable schedule + dose (squares). Triangles indicate fixed conditions when the test dose (0.05–0.2 mg/kg/injection) and comparator 0.1 mg/kg/injection were both fixed. The dashed line indicates 50%, and error bars are one SEM

Full size image

Discussion

This study demonstrates that rhesus monkeys chose cocaine associated with a variable option more often than cocaine associated with a fixed option when average dose was equal (in 9 of 12 conditions across subjects, determinations, and dimensions of variability). Results are somewhat consistent with the literature with rats and pigeons (e.g., Fantino 1967; Herrnstein 1964; Rider 1983), in which animals choose food associated with a VR over an FR schedule, even when the choice resulted in net reinforcer loss (Johnson et al. 2011, 2012). In this study, individual differences occurred. Two subjects who chose cocaine associated with a VR schedule were experimentally naïve at the beginning of the experiment, while the two who did not had different histories, making it impossible to determine which factors resulted in a lack of cocaine choice associated with the VR schedule. Individual differences might be overcome with larger requirements (e.g., FR and VR 100) as larger average ratio values typically result in more extreme choices with VR versus FR schedules (e.g., Fantino 1967). Our results in the variable-dose condition were more robust relative to the variable-schedule or combination of schedule + dose condition, and this pattern of results is somewhat different from that of rats and pigeons where choice between variable vs. fixed food amounts is mixed (Lagorio and Hackenberg 2012; McSweeney et al. 2003; see Kacelnik and Bateson 1996 for a review). Monkeys may be more sensitive to variability in the dose of a drug than rats and pigeons are to variability in food amounts. However, procedural differences also occurred, and additional studies would be necessary to determine potential species, reinforcer types, or procedural differences.

Our hypothesis that the combination of variability in dose + schedule would result in greater effects on choice than either condition of variability alone was not supported. Variability in dose may have exerted control over choice behavior in the combination condition for subject 5274, in which choice shifted toward the variable option in the variable-dose and variable-combination condition but not the variable-schedule condition. For subjects 314206 and 274-2009, cocaine associated with the single conditions was chosen to a similar extent as the combination. Perhaps a floor effect prevented more extreme choices, or combining conditions of variability are not additive. Subject 314172, who chose cocaine associated with the variable-dose option but not the variable-schedule option, did not choose cocaine associated with the combination over the fixed option, suggesting that, for this subject, variability in dose during the combination condition was overshadowed by variability in schedule. Finally, it is possible that the combination condition was so complex that choice did not come under control of the condition of variability.

When the dose of one cocaine option was raised or lowered, subjects chose the dose that was larger on average. In other words, cocaine choice associated with a variable option was overcome by raising the dose on the fixed option. Parallel shifts in the dose-response determinations were not observed in most cases, indicating that variability did not alter cocaine’s potency as a reinforcer. Conversely, variable cocaine may function as a more effective reinforcer. Lagorio and Winger (2014) evaluated the reinforcing effectiveness of remifentanil, cocaine, and ketamine in rhesus monkeys by varying the cost of drug injections using an FR or random-ratio (RR) schedule. Demand was less elastic with the RR schedule than the FR, indicating that variable response requirements increased the effectiveness of the drugs as reinforcers. These effects were larger with remifentanil and cocaine than ketamine, at larger requirements, and with lower doses, suggesting that this phenomenon exerted more control over behavior under conditions of lean reinforcement (Lagorio and Winger 2014). This supports our prediction that larger average requirements may have resulted in more extreme choice of cocaine associated with a VR schedule, and potency effects may have emerged during dose-response determinations. Although cocaine’s potency as a reinforcer was not affected under the parameters of this study, variable conditions of cocaine access may act as a “tie breaker” when choice is between equal average doses.

Many hypotheses attempt to explain behavioral mechanisms that could account for disproportionate choice of reinforcers associated with variable schedules over fixed ones. Choice of reinforcers associated with VR schedules is most robust when the smallest possible ratio value is 1, and the effect goes away as the smallest possible ratio value approaches that of the FR schedule (e.g., Fantino 1967; Field et al. 1996). A quick payoff may have more value than the larger requirements that result in longer delays to reinforcer delivery (e.g., Soreth and Hineline 2009). In the present study, a similar relation may have occurred, where the occasional delivery of larger doses had a disproportionately greater effect on choice than did the occasional delivery of smaller doses in the variable-dose condition. Conversely, the number of component ratios does not appear to affect choice of reinforcers associated with variable schedules (i.e., similar findings are reported with two-value VR schedules and with four- or nine-value VR schedules; Fantino 1967; Johnson et al. 2011; Sherman and Thomas 1968). To our knowledge, one study has directly compared choice between food associated with fixed- and variable-interval schedules, and the number of component intervals (2, 3, or 7 values) did not affect choice (Davison 1972).

Madden et al. (2007, 2011) have proposed that choice of reinforcers associated with VR schedules can be explained within a delay-discounting framework (i.e., the subjective value of a delayed reinforcer is a hyperbolic function of the delay to its receipt). According to this hypothesis, the combined value of a reinforcer under the different requirements of a VR schedule is subjectively greater than the value of the fixed but always delayed reinforcer, especially with relatively large response requirements. Choice of reinforcers associated with VR schedules should positively correlate with an individual’s degree of delay discounting, such that more rapid loss of reinforcer value with increasing delay is correlated with greater choice of reinforcers associated with VR schedules (Madden et al. 2011). In the current study, there were individual differences with the variable schedule. We also have observed individual differences in delay discounting in rhesus monkeys (Freeman et al. 2009, 2012; Huskinson et al. 2015, 2016; Woolverton et al. 2007). Future work could examine whether delay discounting and degree of choice between reinforcers associated with FR and VR schedules are correlated.

Accordingly, drug-dependent individuals discount delayed outcomes more steeply than do their non-dependent counterparts (for meta-analyses, see Amlung et al. 2016; MacKillop et al. 2011) and discount hypothetical drug outcomes more steeply than hypothetical money (e.g., Madden et al. 1997). If drug reinforcers in the natural environment are unpredictable relative to non-drug reinforcers, and if degree of discounting can account for choice of unpredictable outcomes, unpredictable drug availability may play a facilitative role in the allocation of behavior by drug-dependent individuals in the acquisition of drug reinforcers. However, additional research is needed to determine the relation between delay discounting and unpredictable outcomes.

In addition to potential behavioral mechanisms, there is neurochemical evidence that unpredictable reinforcer deliveries result in greater dopamine release in the nucleus accumbens (NAc) core in rats (Sugam et al. 2012) and increased activity in dopamine neurons of ventral midbrain areas in monkeys (Fiorillo et al. 2003). Conversely, omitted reinforcers on unpredictable trials resulted in decreases in dopamine release in the NAc core compared to predictable reinforcer deliveries (Sugam et al. 2012). Reinforcers in the present study were never omitted, but it is possible that their unpredictable delivery resulted in greater dopamine release compared to fixed reinforcer deliveries. Perhaps parallel shifts in the dose-response functions were not obtained because increasing the dose of cocaine may have been sufficient to increase levels of dopamine following fixed reinforcer delivery to a greater extent than the variable, but lower-dose option.

The current results raise important questions. For example, long-term exposure to variable drug availability may increase impulsive choice, resistance to extinction or punishment, escalation of drug-taking, and the reinforcing effectiveness of drugs. Variable exposure to drugs conceivably might hamper an individual’s response to treatment, while a variable schedule of non-drug delivery also may be an effective means of treatment (i.e., contingency management (CM)).

Importantly, CM is the most effective psychosocial treatment for substance abuse (Dutra et al. 2008), and clinical research in CM has examined effects of a probabilistic schedule of non-drug reinforcer delivery under a “prize” schedule as well as “standard” CM (e.g., Olmstead and Petry 2009; Petry and Alessi 2010; Petry and Martin 2002; Petry et al. 2005a, 2005b). In standard CM, participants earn escalating vouchers contingent on drug-free urine samples, exchangeable for non-drug reinforcers (see Higgins and Silverman 2008). A voucher of some monetary value is delivered for every drug-free urine sample, and the exact value of the vouchers escalate in a known manner with consecutive drug-free samples. In contrast, in prize CM, participants earn increasing numbers of draws from a prize bowl for each consecutive negative urine sample submitted. The probability of winning a prize for each draw is usually 50%, and higher-magnitude prizes (e.g., $100) are always available but at low probabilities (e.g., 0.02%). Prize CM is at least equally efficacious to standard voucher CM, even when the expected amount of reinforcement for perfect performance is lower with prizes than vouchers (Petry et al. 2005a, 2015), and this prize system is more cost effective (Olmstead et al. 2007). Future research with non-human subjects could determine parameters of reinforcement that are most effective at reducing drug choice and increasing non-drug choice. Variable schedules of non-drug delivery may more effectively compete with and reduce drug choices than fixed schedules.

We have known for several decades that behavior maintained by variable schedules occurs at a high rate, with little pausing between response bouts, and is more resistant to extinction (e.g., Catania 1992; Mellon and Shull 1986). While many factors contribute to the excessive amount of behavior spent by individuals with SUDs in the acquisition of drugs, results observed in the present and previous studies (e.g., Lagorio and Winger 2014; Madden et al. 2005, 2007, 2011) suggest that variability in at least some dimensions of drug delivery (e.g., quality, cost, delay) could be a contributing factor to the amount of behavior allocated toward procuring drug reinforcers in a population known to show steep discounting of delayed outcomes. In other words, scarce and uncertain conditions of drug availability and quality in the natural environment could be a contributing factor to drug choice and demand. Behavioral interventions may benefit by capitalizing on the use of variable schedules for non-drug reinforcers to counteract drug use.