Despite smokers’ awareness of the health consequences (DHHS, 1989), cigarette smoking remains the leading cause of preventable morbidity and mortality in the USA (CDC 2014). Reasons for continued or relapse to smoking frequently include avoidance of immediate consequences such as cravings (Killen et al. 1991), withdrawal (West et al. 1989), stress (Cohen & Lichtenstein, 1990), and negative effects (Shiffman et al. 1996). This relative bias for immediacy, in lieu of the delayed benefits of not smoking (e.g., health), can be conceptualized as steep delay discounting—an exaggerated loss of the value of future outcomes.

The research examining cigarette smoking and delay discounting bears out the relation, with cigarette smokers exhibiting steeper delay discounting compared to non-smokers and those who have successfully quit (Bickel, Odum, & Madden 1999; Secades-Villa et al. 2014; see reviews in MacKillop et al. 2011; Reynolds 2006; Yi, Mitchell, & Bickel 2009). Though the connection between pre-treatment delay discounting and treatment outcomes is equivocal for non-tobacco drugs of abuse, showing both significant (Passetti et al. 2011; Stanger et al. 2012; Washio et al. 2011) and non-significant relations (De Wilde et al. 2013; Passetti et al. 2008; Heinz et al. 2013; Peters et al. 2013; see review in Stevens et al. 2014), a developing literature indicates that steep delay discounting is associated with relapse to smoking in a human laboratory model (Dallery & Raiff 2007; Mueller et al. 2009) as well as in real-world clinical settings (MacKillop & Kahler 2009; Sheffer et al. 2012; 2014; Yoon et al. 2007).

While greater likelihood of relapse to smoking by individuals with greater bias for immediate rewards is intuitive, the hyperbolic model (as well as other non-normative modelsFootnote 1) of delay discounting explicitly predicts reversals of preference that are thought to model failures of self-control such as relapse.

Figure 1 illustrates how hyperbolic discounting predicts preference reversal. Assuming that a delayed smaller–sooner (SS) reward has an objective value, as indicated by the length of the vertical line marked SS, that a more-delayed larger–later (LL) reward has an objective value as indicated by the length of the vertical line marked LL (shown here with both shown as a percentage of the LL), and that the delay between the SS and LL rewards remains constant, the panels represent four hypothetical individuals with different delay discount rates: very high (i.e., steep) to very low (i.e., shallow), going left to right. Moving from right to left within each panel as time passes, preference is for the smaller–sooner (SS) or larger–later (LL) alternative with the higher discounted value (y-axis) at any given point in time. When both SS and LL rewards are very distal (before time A), the very high discounter (far left panel) exhibits a switch in preference very early, resulting in behavior that appears to always prefer SS. As time passes (between times A and B), the moderate high discounter (center left panel) now prefers SS. As more time passes (between times B and C), the moderate low discounter (center right panel) prefers SS. It is important to note that preference reversals in these examples occur simply due to the passage of time, as the objective values of the SS and LL rewards, and the duration of the delay between them (the additional delay associated with waiting for the LL reward) remain constant. Finally, the very low discounter (far right panel) consistently prefers LL.

Fig. 1
figure 1

The x-axis represents time, starting at the right and going left as time passes. SS (“smaller–sooner”) represents a small reward that is available relatively sooner, and LL (“larger–later”) represents a larger reward that is available relatively later. The y-axis indicates subjective value. The far left panel depicts a very high discounter, such that the intersection of discounted utility functions occurs at some point prior to point A and is not visible in this depiction. The middle-left panel depicts a moderate high discounter, such that the intersection of discounted utility functions occurs relatively early, between points A and B. The middle-right panel depicts a moderate low discounter, such that the intersection of discounted utility functions occurs relatively late, between points B and C. The far right panel depicts a very low discounter, such that the LL reward always has higher discounted utility (i.e., exhibits no preference reversal)

Full size image

Despite the significance of relapse as a defining characteristic of addiction, very little research has directly explored the presumptive relationship between delay discounting and preference reversals. Though a number of human and non-human animal studies have illustrated that preference reversals do occur (Ainslie & Herrnstein 1981; Green & Estle 2003; Green, Fisher, Perlow, & Sherman, 1981; Holt, Green, Myerson & Estle 2008; Kirby & Herrnstein 1995; Luhmann 2013; Millar & Navarick 1984), no explicit connection to rate of delay discounting has been made. For example, while Green, Fristoe, and Myerson (1994) demonstrated that preference reversals occur in a predictable manner as a function of both delay to the sooner reward and delay between sooner and later rewards, no attempt was made to examine the relation between preference reversals and delay discounting. Thus, while non-normative models of delay discounting predict preference reversals, and the literature indicates that preference reversals occur, we are aware of no research that is able to speak to the direct relation between delay discounting rate and timing of preference reversals. The purpose of the present study was to address this gap in the literature by examining the relation between delay discounting and preference reversals for hypothetical money in a sample population of cigarette smokers.

Method

Participants

Forty-seven (47) adult, non-treatment-seeking smokers from the Washington, D.C. metropolitan area met two or more of the following smoking criteria: 1. currently smoking ≥ 10 cigarettes per day for ≥ 1 year (M [SD] cigarettes/day = 18.7 [7.84]); 2. score 5 ≥ on the Fagerstrom tolerance questionnaire (FTQ; Fagerstrom & Schneider 1989; M [SD] = 7.25 [1.82]); 3. DSM-IV-TR diagnostic criteria for nicotine dependence. Due to technical problems, three participants had missing data on relevant variables and were excluded from analyses, yielding a final sample of 44. Individuals with major medical illnesses, psychiatric disorders, or dependence other than nicotine were excluded. Smoking status was verified by an expired carbon monoxide level ≥ 8 parts per million.

Materials

Delay Discounting Task

Delay discounting was assessed using a computerized binary choice procedure, where participants indicated preference between two amounts of hypothetical money by using the mouse to click on the preferred alternative. The smaller–sooner (SS) alternative was an amount of money that was available immediately, and was adjusted from trial to trial in the task. The larger–later (LL) alternative was an amount of money that was available following a delay, and remained fixed from trial to trial within a magnitude condition. The LL amount was $50 or $1000 depending on the magnitude condition. The delays for the LL alternatives were: 1 day, 1 week, 1 month, 6 months, 1 year, 5 years, and 25 years.

Using the algorithm of Du, Green, and Myerson (2002; see also Holt, Green & Myerson 2003), the SS alternative was titrated across six trials to determine an indifference point for each unique magnitude/delay pairing of the LL alternative. In the first of six trials within each magnitude/delay pairing, the SS alternative was 50 % of the LL alternative. If the LL alternative was selected, the SS alternative was increased on the subsequent trial to 75 % of the LL alternative; if SS was selected, the SS alternative was decreased to 25 % of the LL alternative. Over the remaining trials within each magnitude/delay pairing, the SS alternative was increased or decreased in this manner, by half of the previous adjustment (e.g., 12.5 % increase/decrease for trial 3). The indifference point (i.e., the present subjective value of the delayed LL amount) was calculated as the resulting SS alternative following the sixth trial.

Preference Reversal Task

Preference reversals were assessed using four variations of a novel computerized choice procedure partially informed by established procedures to assess delay discounting (Du et al. 2002) and preference reversals (Green et al. 1994; Holt et al. 2008), with the purpose of allowing a higher degree of temporal specificity than previous studies of preference reversal. In each trial of the choice procedure, two hypothetical money rewards (SS and LL) were presented where the SS reward was delayed (by a front-end delay) and the LL reward was more delayed (by the same front-end delay plus back-end delay). The four variations of the preference reversal task incorporated each combination of a magnitude condition of the LL reward from the delay discounting task ($50, $1000) and a back-end delay condition (7 days, 30 days). To wit, the preference reversal conditions were: (1) LL = $50 with back-end delay = 7 days, (2) LL = $50 with back-end delay = 30 days, (3) LL = $1000 with back-end delay = 7 days, (4) LL = $1000 with back-end delay = 30 days.

In order to determine the appropriate SS value for each preference reversal task condition, the SS values at which a significant majority of participants were expected to exhibit a preference reversal were calculated using archival hyperbolic discount rates from a similar population, within each combination of LL magnitude and back-end delay. Based on these calculations, the SS reward was set at 95 % of the LL reward at back-end delay = 7 days (i.e., $47.50 and $950 for $50 and $1000 magnitude conditions, respectively) and 65 % of the LL reward at back-end delay = 30 days (i.e., $32.50 and $650 for $50 and $1000 magnitude conditions, respectively).

A two-step algorithm was applied in order to determine the front-end delay (i.e., preference reversal point) at which participants exhibit a preference reversal. Algorithm step 1 sought to identify an initial temporal window in which a preference reversal occurred by working backwards in time (i.e., left to right in each panel of Fig. 1). On the first trial, the SS amount was 95 % or 65 % of the LL amount (depending on the back-end delay condition) with front-end delay = 0; the LL amount was $50 or $1000 (magnitude condition), delayed by 7 or 30 days (back-end delay condition). This first trial was similar to a now versus later trial of a conventional delay discounting task. If the participant indicated preference for the LL alternative on this first trial, the program was terminated and the participant was scored as ‘larger–later in first trial’ for that combination of magnitude and back-end delay conditions, indicating no preference reversal was possible given study parameters (i.e., the far right panel of Fig. 1). For participants that indicated preference for the SS alternative in this initial trial, the front-end delay was increased by 4-month (when back-end delay = 7 days) or 8-year (when back-end delay = 30 days) increments until the participant switched preference toward the LL alternative. For example, in the $50, 7-day back-end delay condition, the second trial was a choice between $47.50 delayed by 4 months (SS) and $50 delayed by 4 months plus 7 days (LL). If no switch to preference for the LL alternative was observed across five trials of increasing front-end delay while back-end delay remained constant, the program was terminated and the participant was scored as ‘smaller–sooner for all trials’ for that combination of magnitude and back-end delay conditions, indicating no preference reversal was observed given the study parameters (i.e., the far left panel of Fig. 1).

For those participants who did exhibit a switch in preference from the SS to LL alternatives during algorithm step 1, the switch point defined the initial preference reversal window as between the longest front-end delay where the SS alternative was preferred and the shortest front-end delay where the LL alternative was preferred. Within these lower and upper boundaries, algorithm step 2 sought to more focally define the preference reversal point. For the first of six trials in this second step, the two alternatives were: (1) the SS alternative with a front-end delay halfway between the lower and upper boundaries, and (2) the LL alternative with the same front-end delay plus back-end delay (7 or 30 days, depending on the condition). If the LL alternative was selected, the front-end delay for both SS and LL alternatives was increased on the subsequent trial by 25 % of the preference reversal window. If the SS alternative was selected, the front-end delay for both SS and LL alternatives was decreased on the subsequent trial by 25 % of the preference reversal window. Over the remaining trials, the front-end delay for both SS and LL alternatives was increased or decreased in this manner, by half of the previous adjustment (e.g., 12.5 % increase/decrease for trial 3). An example series of trials is shown in Fig. 2.

Fig. 2
figure 2

Diagram of hypothetical sequence of trials in the preference reversal task for a participant who exhibits a preference reversal in the $50-, 7-day, back-end delay condition. The left and right columns represent the smaller–sooner (SS) and larger–later (LL) alternatives, respectively. Each row represents a single trial, and the bolded alternative represents the selected alternative in the hypothetical sequence

Full size image

Procedure

As part of an institutional review board (IRB)-approved 2-hour session, the computerized delay discounting task for both magnitude conditions were completed prior to the computerized preference reversal tasks. In the delay discounting tasks, the magnitude condition order was counterbalanced between subjects, and the delay order was fixed (increasing). In the preference reversal tasks, the order of conditions was counterbalanced across participants. Results of a questionnaire battery completed following these assessments are not reported here, and participants were financially compensated for participation.

Results

Using individual indifferent points at each delay in each magnitude condition, delay discounting rate (k) was estimated separately for each participant using non-linear regression based on the hyperbolic decay function (Mazur 1987): \( {V}_d=\frac{V}{1+kd} \), where V d is the discounted value (i.e., the indifference point) of a reward at delay d, and V is the undiscounted value of a reward (i.e., the magnitude of the LL alternative). High values of discounting rate k indicate steep discounting, where the subjective value of the LL alternative quickly loses value as a function of delay. In instances where delay discounting data for only one reward magnitude condition was available (eight participants, due to technical problems), participant data were considered only for the available magnitude condition. The discounting rate of two participants were outliers in the $1000 condition (>3 standard deviations from the mean; Ratcliff 1993) and excluded from the analyses.

The model provided a good fit to individual data, with low root mean squared error (RMSE; M $50 = 0.141 and M $1000 = 0.123; RMSE is a more appropriate measure of fit with non-linear regression than R2; Johnson & Bickel 2008). For demonstration purposes, the model is fit to median indifference points in Fig. 3 at each magnitude. Natural logarithm transformations of discounting rate k (ln-k) were conducted in order to normalize the distribution and allow for parametric analyses.

Fig. 3
figure 3

Mazur’s (1987) hyperbolic decay function fit to median indifference points as a function of delay, represented as a proportion of the delayed amount in the $50- (top panel) and $1000-magnitude (bottom panel) conditions. Classifications (predicted low, moderate, high discounter) were determined by the pattern of responding in the preference reversal tasks

Full size image

Age, sex, and income were not significantly associated with any of the variables of interest, and are not considered further. A paired t test compared delay discounting rate (ln-k) between $50 (M = -4.40) and $1000 (M = -5.61) reward outcomes (t[33] = 2.57, p = 0.015). These means were consistent with previous work using identical methods and magnitude conditions with smokers (e.g., Yi & Landes 2012), and replicated the established magnitude effect in which high-magnitude amounts are discounted less steeply than low-magnitude amounts (Kirby 1997; see review in Madden & Johnson 2010).

The relation between delay discounting and preference reversals was first explored by determining if rates of delay discounting differed in the predicted manner among individuals who were classified within each magnitude as high discounters (left panel of Fig. 1), as moderate discounters (two center panels), or as low discounters (right panel). Classification was based on the pattern of responding in the preference reversal tasks within each magnitude condition, resulting in a separate classification for each participant in the $50- and $1000-magnitude conditions. A participant was classified within a magnitude condition as a (a) predicted low discounter if s/he preferred the LL reward on the first trial for both preference reversal tasks (7-day and 30-day back-end delay conditions), (b) predicted high discounter if s/he preferred the SS reward on all trials for both preference reversal tasks, and (c) predicted moderate discounter in all other instances (e.g., exhibited a preference reversal on both preference reversal tasks within a magnitude condition; preferred the LL reward on the first trial in one preference reversal task and preferred the SS reward on all trials in the other preference reversal task).

The frequency of each classification (Table 1) indicates an appropriate degree of coherence. Consistent with the magnitude effect (increasing discounting with decreasing magnitude), the percentage of low discounting classifications was higher in the $1000- than $50-magnitude conditions, while the percentage of high discounting classifications was higher in the $50- than $1000-magnitude conditions.

Table 1 Percent and number of participants in each classification resulting from the preference reversal task, as a function of reward magnitude
Full size table

Following classification based on preference reversal tasks, analysis of variance (ANOVA) and chi-square analyses were conducted to explore possible group differences in sociodemographic or smoking characteristics, and no significant differences were observed (all p values > 0.09). The rates of delay discounting for the three groups (predicted high, moderate, low discounters) were then compared at each reward magnitude ($50, $1000).

ANOVA was conducted (Fig. 4) with rate of delay discounting for the corresponding magnitude ($50, $1000) as the dependent variable. A significant overall difference was observed in the $50-magnitude condition (F[2, 38] = 3.92, p = 0.03), with Tukey’s post-hoc comparisons indicating that predicted low discounters had significantly lower observed discount rates than predicted high discounters (p = 0.023); no significant differences were observed in the other pairwise comparisons. A significant overall difference was also observed in the $1000-magnitude condition (F[2,39] = 7.38, p < 0.01), with Tukey’s post-hoc analyses indicating that predicted high discounters had significantly higher discounting rates than predicted low discounters (p < 0.01) and predicted moderate discounters (p < 0.01).

Fig. 4
figure 4

The mean rate of delay discounting is shown as a function of classification (predicted low, moderate, high discounter) within each magnitude condition. All means are in the predicted direction, with statistically significant differences between predicted low/high discounters in the $50-condition, and between predicted low/high and moderate/high discounters in the $1000-condition

Full size image

For the purpose of a correlational analysis of delay discounting and preference reversals, moderate discounters were subclassified as low moderate (LL on the first trial in one task and preference reversal on the other task), mid-moderate (LL on the first trial in one task and SS on all trials in the other task; preference reversal on both tasks), and high moderate (SS on all trials in one task and preference reversal on the other task). Spearman correlations conducted with this ordinal preference reversal sub-classification and rate of delay discounting revealed non-significant correlations (r s  = +0.26, p = 0.28 and r s  = +0.34, p = 0.25 in the $50- and $1000-magnitude conditions) in the predicted direction.

Discussion

The reversal of preference from larger–later to smaller–sooner outcomes as a function of the passage of time is an initial decision to quit smoking followed by relapse. Such preference reversals are predicted by hyperbolic and other non-exponential models of delay discounting, and studies of intertemporal choice have frequently assumed this relation without formally determining that the relation exists. We believe that the present study is the first to explicitly examine whether rate of delay discounting is associated with preference reversals in a human sample. We examined this population because smokers have elevated delay discounting and may be particularly vulnerable to preference reversals.

Based on the pattern of responding in the preference reversal task, participants were predicted to fall into one of the following categories: high, moderate, and low discounters. Consistent with prediction, the predicted high discounters exhibited significantly higher rates of delay discounting than predicted low discounters in both ($50 and $1000) magnitude conditions. A significantly higher rate of delay discounting was also observed in predicted high discounters relative to predicted moderate discounters in the $1000-magnitude condition. Though some pairwise comparisons were not statistically significant, the pattern across predicted high, moderate, and low discounters was identical in the two magnitude conditions. A more precise examination of this relation between delay discounting and preference reversal via correlational analysis revealed predicted relations that did not reach statistical significance, and, in this respect, the present preference reversal task failed to provide the high degree of temporal resolution we had hoped for during task development. Nonetheless, we believe this consistent overall pattern of results in support of the delay discounting and preference reversal relation is compelling.

The lack of statistically significant findings in some of the analyses highlights insufficient statistical power as the primary limitation of the present research. Given the clear pattern of results when comparing predicted high, moderate, and low discounters, it appears likely that non-significant contrasts were due to the small sample size. One obvious solution would be to increase the sample size in future research. Based on the most conservative (i.e., smallest) effect size obtained in the present research when making binary contrasts comparing predicted discounter classifications (η p 2 = 0.02), a sample size of 144 in each classification would have been necessary for a power of 0.80 when conducting a two-tailed (p = 0.05) test to detect statistically significant differences in all pairwise contrasts. This is assuming that there is a true difference between participants classified as low and moderate discounters using the present study paradigm, which may or may not be the case.

Modification of the preference reversal task (e.g., larger SS reward or longer back-end delay conditions; different titrating algorithm) could also have resulted in a higher number of observed preference reversals, which would have enhanced the correlational analysis of delay discounting and preference reversal. Using the present paradigm, we observed few preference reversals (46 % and 32 % in $50- and $1000-magnitude conditions, respectively) which was likely insufficient to adequately power such an analysis. Previous studies (Green et al. 1994; Holt et al. 2008) used a variety of inter-reward delays (i.e., back-end delay of the present study) in a preference reversal task, and a similar procedure could have been used to personalize the back-end delay such that preference for the SS reward on the first preference reversal trial was guaranteed (i.e., preference reversal was possible). We elected not to implement such a personalized task because back-end delays that allow for preference reversals at the individual level (which are theoretically influenced by delay discounting rate) would have then been confounded with the timing of preference reversals (also theoretically influenced by delay discounting rate). Given that we elected to make the SS reward and back-end delay constant across all participants, a closer examination of the data indicates that an SS = $49.79 would have been appropriate to obtain 90 % of participants choosing the SS reward in the first trial (thereby allowing the possibility of a preference reversal in the $50-, 7-day, back-end delay conditionFootnote 2).

Despite the failure to reach conventional thresholds for statistical significance in some analyses, we wish to note the high degree of coherence in the classification distribution (Table 1) that is consistent with theory and, as such, not likely due to chance. For example, the observation that a higher percentage of participants were classified as low discounters in the $1000-magnitude conditions (compared to the $50-magnitude conditions), and a higher percentage of participants were classified as high discounters in the $50-magnitude conditions (compared to the $1000-magnitude conditions), is consistent with the well-established magnitude effect (Kirby 1997) that was replicated in the present study.

A minor limitation related to an insufficient portion of the sample exhibiting a preference reversal is that the present study is unable to differentiate between various models of delay discounting that also predict preference reversals. We examined data from the present study using an alternative single-parameter delay discounting model (exponential-power; Yi, Landes & Bickel 2009), and no differences were observed in the pattern of results when compared to results with the hyperbolic model. This is partially due to the fact that indices from different models of delay discounting are highly correlated, so that scoring of delay discounting using an alternative to Mazur’s (1987) hyperbolic model does not typically change the results. While alternative indices of delay discounting might have made a small difference when examining the possible continuum of the delay discounting and preference reversal relation, the insufficient power for that analysis in the present study made model comparison unfeasible.

Another minor limitation of the present research is that the outcomes for both delay discounting and preference reversal tasks were hypothetical money. This is partially addressed by previous research using real money outcomes that have exhibited elevated delay discounting by smokers (e.g., Bickel, Odum & Madden 1999; Mitchell 1999), and recent evidence indicating that delay discounting metrics for outcomes that are hypothetical and real are statistically equivalent (Matusiewicz et al. 2013).

Despite these limitations, we believe the present results provide basic support for the concept that rate of delay discounting can predict preference reversals. To the extent that preference reversals model smoking relapse, assessment of smoking-related delay discounting (e.g., discounting of cigarettes, withdrawal symptoms) in future research may provide additional insight into when a smoker who has or will quit is vulnerable to relapse. Establishment of the predictive utility of delay discounting on smoking relapse could inform the appropriate temporal targeting of interventions to help prevent relapse and maintain attempts at quitting.