Introduction

I spent 13 years at NIMH really pushing on the neuroscience and genetics of mental disorders, and when I look back on that I realize that while I think I succeeded at getting lots of really cool papers published by cool scientists at fairly large costs—I think $20 billion—I don’t think we moved the needle in reducing suicide, reducing hospitalizations, improving recovery for the tens of millions of people who have mental illness. I hold myself accountable for that.

  • Former NIMH Director, Tom Insel

Similar arguments can be made in the field of addiction science. First, substance use disorders are diagnosed via symptoms rather than by mechanism. As former NIMH Director Insel stated: “Unlike our definitions of ischemic heart disease, lymphoma, or AIDS, the DSM diagnoses are based on a consensus about clusters of clinical symptoms, not any objective laboratory measure. In the rest of medicine, this would be equivalent to creating diagnostic systems based on the nature of chest pain or the quality of fever” (Insel, 2013). Second, although a variety of treatments have been developed that improve rates of successful quit attempts, the efficacy for these treatments leaves considerable room for improvement. For example, only 1 in 9 individuals with alcohol-use disorder benefit from treatment with medication, and brief psychotherapeutic interventions (Foxcroft et al., 2016; Klimas et al., 2012; Rösner, Hackl-Herrwerth, Leucht, Lehert et al., 2010a; Rösner, Hackl-Herrwerth, Leucht, Vecchi et al., 2010b) produce only small reductions in alcohol consumption. Therefore, even with today’s best treatments, failure is the expected result and a modest reduction in drinking is considered a good outcome. Third, consider that the opioid overdose epidemic has resulted in over 60,000 deaths, whereas alcohol is associated with approximately 80,000 per year and tobacco is associated with 480,000 deaths per year (Substance Abuse & Mental Health Services Administration [SAMHSA], 2014). We argue that our understanding and effectiveness in treating addiction is less than fully adequate and, in turn, supports the need to develop new approaches to the understanding of the disorder and the development of potential interventions.

What should this new approach entail? We could develop it top down, using some hypothetico-deductive method or by combining many existing theories of addiction into some über-theory (e.g., West & Brown, 2013). However, such methods may lead to approaches that, although internally consistent (hypothetico-deductive), are not effective and may not specify interventions that address mechanistic causes of disease. As an alternative, we could atheoretically try intervention after intervention to improve therapeutic outcomes in addiction. Finally, we could use methods to discover potential determinants of addiction by identifying new targets of intervention that are closely aligned with the underlying mechanisms of the disorder. This has been referred to as the experimental medicine approach and is more consistent with the inductive method often supported within behavior analysis. Using this approach, novel determinants are developed by identifying novel targets; that is, identify and measure a process closely aligned with a disorder. For example, high blood pressure is associated with stroke. The next step is to develop an intervention that will modify that target, and then examine whether that intervention, in turn, has any salutary effects on other processes or features of the disorder. For example, antihypertensive therapies reduce blood pressure, and indeed reduce the incidence of stroke. Such a program of research if successful may lead to identifying novel determinants contributing to the disorder as well as suggest new determinants. Below we briefly outline the history of experimental medicine, the application of the experimental medicine approach, and conclude with a theoretical perspective on addiction derived from this application.

Experimental Medicine Approach

Claude Bernard (1813–1878) was a French physiologist best known for his discoveries involving the functions of the pancreas, liver, and vasomotor system, and developing the concept of milieu interieur, which eventually became the underlying principle of homeostasis (Bernard, 1957). As important, Bernard incorporated the scientific method into medicine and is recognized as the father of the experimental medicine approach (McCance, 1951). His account of how science should be conducted and best translated to medicine was codified in his classic text An Introduction to the Study of Experimental Medicine, published in 1865.

In his work, Bernard begins by differentiating observation and experimentation. He proposes that within the realm of experimental medicine, “mere observation is not enough” (Bernard, 1957, p. 5). Instead, Bernard insists that experimentation is necessary for the observer to develop scientific knowledge. Experiments may be informed by previously gathered observation but must be carried out to expand our knowledge around a particular cluster of facts.

Bernard’s proposal does not exist in isolation. In fact, the experimental medicine approach shares common features with other important scientific perspectives. For example, Thompson (1984) points out the commonalities between experimental medicine and the experimental analysis of behavior. He notes that the process of “experimental” reasoning per Bernard (i.e., generating new hypotheses from observing something new) could also be interpreted as identifying the “proximate cause” of a particular phenomenon. In the vocabulary of experimental analysis of behavior, the proximate cause is equivalent to “the controlling independent variable of the response” (Thompson, 1984). In other words, both approaches are interested in the underlying mechanism, the “why” for observing a particular outcome. In turn, this understanding can then help inform the experimenter on the appropriate target for an intervention.

The uniqueness of the experimental medicine approach is its emphasis on testing the ability of an intervention to change the hypothesized target (analogous to blood pressure above) in order to affect the behavioral outcome (analogous to stroke above). That is, can we target the underlying mechanism in order to change behavior (Nielsen et al., 2018; Riddle & Science of Behavior Change Working Group, 2015)? This mediation-like perspective is arguably more efficient in promoting behavior change, intervening at the root instead of chasing symptoms. In fact, the NIH Common Fund supports the Science of Behavior Change (SoBC) program, which is dedicated to integrating scientific disciplines to understanding behavior change from a mechanistic, experimental medicine perspective to develop viable potential therapeutic interventions. The SoBC program summarizes this approach as the following steps (Riddle & Science of Behavior Change Working Group, 2015):

  1. 1)

    identify an intervention target;

  2. 2)

    develop assays to verify target measurement;

  3. 3)

    engage the target via experiment or intervention; and

  4. 4)

    test the degree to which target engagement produces the desired behavior change (Nielsen et al., 2018).

Step 1. Identify an Intervention Target

Target identification requires the early work of reliably observing a particular phenomenon, prior to any experimentation. Mowrer and Ullman (1945) showed that persistent maladaptive behavior may have more to do with time than consequences. In particular, the authors wrote that, “a given action to be perpetuated or inhibited is influenced not only by the nature of the consequences ('effects') of that action but also by the temporal order, or timing of these consequences” (Mowrer & Ullman, 1945). Their review is one of the first to note that time is associated with human and animal behavior that may seem irrational or impulsive. Later research identified that, in the case of addiction, heroin-dependent individuals are less likely to predict or organize events in the distant future (Petry, Bickel, & Arnett, 1998), supporting the role of a shortened temporal window in maladaptive behavior. In a laboratory study, individuals with heroin-use disorder were asked to finish a hypothetical story about a man waking up in bed and thinking about his future. The median time frame of the remaining story from the participants spanned a total of 9 days, significantly less compared to the median time frame from control participants of 4.7 years (Petry et al., 1998). These results are supported by others demonstrating that individuals with alcohol (Smart, 1968) and heroin-use disorder scored significantly lower on future time perspective scales compared to controls (Henik & Domino, 1975; Manganiello, 1978; Petry et al., 1998). Although these are just a few examples, decades of literature have demonstrated the relationship between maladaptive behaviors, especially addiction, and shortened temporal windows. Therefore, the target for intervention for these behaviors, based on the experimental medicine approach, is hypothesized to be a shortened temporal window. In the experimental medicine approach the next step is to develop assays to best measure this target.

Step 2. Develop Assays

In order to proceed to experimentation from the observation of shortened temporal windows coinciding with addiction, the temporal window must be directly assayed. The measurement of short temporal windows leading to maladaptive behavior can be accomplished numerous ways. For example, self-report questionnaires such as the future time perspective questionnaire (Stouthard & Peetsma, 1999) and the Zimbardo Time Perspective Inventory (Zimbardo & Boyd, 1999) have been developed. It is important to note that future time perspective may also be measured by an organism’s preference for larger delayed outcomes (i.e., “delay discounting”; Klineberg, 1968).

The process of delay discounting is the decline in value of a delayed outcome as a function of length of the delay. Ainslie (1975) identified that “impulsiveness,” the choice between a smaller reward now over a larger reward after a delay, can be defined by reliable hyperbolic curves predicting the decline in value of a delayed outcome. Delay discounting can be measured using questionnaires (e.g., assessing preference between $50 now and $100 in 3 weeks (Kirby & Maraković, 1996; Kirby, Petry, & Bickel, 1999) as well as titrating tasks (Du, Green, & Myerson, 2002; Koffarnus & Bickel, 2014; see Madden & Johnson, 2010 for review). Over the years, the measurement of delay discounting has evolved into an efficient and effective assay for an individual's temporal window. For example, while many iterations of the delay-discounting task are widely used, a relatively new task has been adapted to be completed by a participant in less than 1 minute (Koffarnus & Bickel, 2014). Moreover, several delay-discounting tasks have been adapted to be used in an MRI scanner in order to assess an individual’s brain activation while making choices (Amlung, Sweet, Acker, Brown, & MacKillop, 2014; Koffarnus et al., 2017; McClure, Laibson, Loewenstein, & Cohen, 2004). Others have also demonstrated effectiveness of delay-discounting tasks by showing that individuals respond similarly to the task whether the outcomes are hypothetical or real (Johnson & Bickel, 2002; Madden, Begotka, Raiff, & Kastern, 2003; Madden et al., 2004). Finally, the development of metric guidelines for identifying inconsistent participant responding improves the validity of task data (Johnson & Bickel, 2008), and therefore the efficacy of the assay.

Applications of these assays have demonstrated the role of delay discounting in addictive behavior. It is important to note that the rate of decline in reward value has been found to be positively correlated with a shortened temporal window of individuals and predictive of maladaptive behaviors, including addiction severity. In the first human demonstration of this, Ainslie and Schafer (1981) found greater impulsive choices among alcohol-dependent patients compared to the staff providing alcohol treatment. In addition, Madden et al. (1997) extended that observation by demonstrating that opioid dependent individuals discounted the future significantly more than matched controls. The relation between delay discounting and addictive behavior has been supported by many subsequent studies and meta-analyses (Amlung, Petker, Jackson, Balodis, & MacKillop, 2016; MacKillop et al., 2011). Given a robust assay for our target (shortened temporal windows), the next step is to engage the target with potential interventions and determine the degree of engagement using delay discounting.

Step 3. Engage the Target

The development of a valid assay for the temporal window set the stage for subsequent research identifying the conditions that may alter the temporal window. A variety of studies have shown that different circumstances are associated with changes in discounting, such as withdrawal (Giordano et al., 2002). Subsequent research began to specifically engage delay discounting as the target of the intervention (Koffarnus, Jarmolowicz, Mueller, & Bickel, 2013). For example, Bickel Yi, Landes, Hill, and Baxter (2011b) demonstrated that working memory training significantly attenuated delay-discounting rates (lengthened temporal windows), whereas other cognitive measures were not changed. More recent interventions have been developed from the new field of prospection (Gilbert & Wilson, 2007). Employing episodic future thinking (the mental simulation of future events; Atance & O’Neill, 2001), by cueing participants with vivid and personal future events, may engage the underlying processes that dictate temporal windows, and therefore delay discounting. In particular, participants first develop cues by describing potential future events that correspond to several future time frames (e.g., 1 week, 1 month, 3 months). For each of these time frames participants are asked to concretize the events (e.g., What are you doing? Who will be there? What will you see, hear, smell, and feel?). Future cues are then presented on the screen while completing the delay discounting task. Episodic future thinking relies on multiple neural substrates (Schacter, Benoit, & Szpunar, 2017) also recruited during delay discounting (Benoit & Schacter, 2015; Koffarnus et al., 2017; Wesley & Bickel, 2014), and episodic future thinking may attenuate delay discounting (lengthen temporal windows) through recruitment of these substrates (Peters & Büchel, 2010). What is critical is that episodic future thinking has been observed to produce substantial reductions in discounting in a meta-analysis of 10 studies (Rung & Madden, 2018), with comparable effects across healthy adult (Lin & Epstein, 2014), obese (Daniel, Stanton, & Epstein, 2013a), alcohol using (Snider et al., 2018; Snider, LaConte, & Bickel, 2016), and smoking (Stein et al., 2016) populations. Among the multitude of methods to engage delay discounting as a target for intervention, episodic future thinking may have substantial clinical potential in an experimental medicine approach. However, the most powerful demonstration of engagement of a particular target should show the ability to move the assayed process symmetrically. That is, subsequent testing of the effects of target engagement is best supported by a suite of experimental tools offering not only the ability to expand, but also to constrict the temporal window.

Some manipulations have been shown to increase delay discounting (shorten temporal windows). For example, natural experiments assessing delay discounting after periods of hardship have indicated that negative income shocks shortened temporal windows (Haushofer & Fehr, 2014). In particular, subsistence farmers demonstrate shortened temporal windows, and steeper discounting rates, after periods of drought (Haushofer, Schunk, & Fehr, 2013). This same effect has been observed after hypothetical simulation of negative income shock (Bickel, George Wilson, Chen, Koffarnus, & Franck, 2016; cf. Haushofer et al., 2013; Mellis, Snider, & Bickel, 2018a), as well as by asking participants to imagine a job loss scenario. Symmetric engagement supports the temporal window as a manipulable target. The fourth step in the experimental medicine approach is to determine the degree to which target engagement produces the desired behavior change.

Step 4. Test the Degree to Which Target Engagement Produces the Desired Behavior Change

Episodic future thinking and scarcity manipulations provide robust methods for engaging the target of the temporal window in experimental work. In the experimental medicine approach, the next and final step is to determine whether target engagement (i.e., lengthening or constricting temporal windows) engenders a subsequent health behavior change in addiction. Initial evidence that changing delay discounting with episodic future thinking may also change behaviors correlated with discounting emerged from a study among obese individuals (Daniel, Stanton, & Epstein, 2013b). In a subsequent study of individuals with alcohol-use disorder, Snider et al. (2016) demonstrated a reduction in delay discounting as well as purchasing of alcoholic drinks after episodic future thinking, compared to control episodic thinking of recent past events (see Figure 1). This same pattern has been observed among smokers, with episodic future thinking engendering reductions in both delay discounting and the number of cigarette puffs consumed by smokers (Stein et al., 2016). In these experimental contexts, reducing delay discounting has been shown to coincide with reduced valuation and self-administration of substances of abuse, suggesting that target engagement does produce the desired behavior change.

Fig. 1
figure 1

Episodic Future Thinking (EFT) reduces delay discounting rates and alcohol demand compared to control thinking. Panel A: Indicates participant indifference points at each of 5 delays from the delay discounting task. EFT condition was vivid future cues and Control condition was vivid recent past cues. Points are group means (± SEM). *p < 0.05 between group comparison. Panel B: illustrates alcohol demand by plotting total number of drinks hypothetically purchased as a function of increasing price. Points indicate group means (± SEM)

Full size image

On the other hand, experimental manipulations constricting the temporal window may increase the valuation of substances of abuse. In support of this, the same hypothetical negative income shock manipulations that increase delay discounting have also been demonstrated to increase valuation of unhealthy foods among overweight and obese individuals (Mellis, Athamneh et al., 2018b; Stein et al., 2017). This effect has not been fully explored in addiction, however other manipulations increasing delay discounting rate may increase drug valuation (cf. Athamneh et al., in press). If increasing delay discounting also subsequently increases drug valuation, it would provide a symmetric demonstration that target engagement may alter components of health behavior bidirectionally.

The Resulting Hypothesis: Reinforcer Pathology 2.0

Future research is needed to truly demonstrate that altering the temporal window produces long-term changes in health behavior. However, taken together, these results have led to the development of a new hypothesis regarding reinforcer pathology (Bickel et al., 2017). This hypothesized model formalizes a relationship between the temporal patterns of benefits of abstinence versus consumption of drugs (e.g., Rachlin, 2000). Central to this model is a hypothesis that the temporal window represents the period of time over which the value of reinforcers may be integrated. Addictive substances offer brief, intense, reliable reinforcement directly following consumption, and offer diffuse, uncertain consequences after a delay. Individuals will tend to select reinforcers with the highest subjective value, which is assessed by integrating the value offered moment-by-moment over the temporal window being considered. As delay discounting increases, the temporal window constricts, and the domain over which the subjective value is considered is reduced, favoring drug reinforcement. In contrast, many positive health behaviors offer temporally extended, low-intensity, and variable reinforcement (e.g., work, relationships). The benefits of abstinence, for example, are only realized when integrating over a substantially longer time frame (i.e., weeks, months, and years) than that required to realize the benefits of alcohol consumption (i.e., minutes and hours). In a constricted temporal window, brief reinforcement will be preferred over temporally extended reinforcement; when the window is expanded, these valuations reverse (see Figure 2). In this model of reinforcer valuation, delay discounting is an index of the temporal window and steep rates of delay discounting underlie high valuation of unhealthy reinforcers. Thus, Reinforcer Pathology 2.0 observes not only the cooccurrence of two risk factors,Footnote 1 but identifies one risk factor (a constricted temporal window, indexed by high delay discounting) as promoting another (high valuation of substances). No other theoretical perspective of addiction (Bickel, Crabbe, & Sher, 2019) addresses the temporal window as a relevant independent variable in addiction science (Bickel et al., 2018). This statement of reinforcer pathology may provide a new hypothesis for the development of novel treatments of addiction. In particular, episodic future thinking may provide a method to engage the target of the temporal window, moving individuals with addiction from a shortened to a lengthened temporal window, decreasing self-administration. The greatest test of this hypothesis would be in its application in the broader quest to treat addiction, through randomized controlled trials of the effects of episodic future thinking on addiction itself. Additional support for this hypothesis could come from the investigation of other interventions that engage the target of delay discounting (as reviewed in Koffarnus et al., 2013) to identify whether these also modulate drug valuation and addiction.

Fig. 2
figure 2

Hypothesized relationship between reinforcer valuation and the temporal window. The area under the curves depicts the subjective value (y axis) of drug (orange) and prosocial (blue) reinforcers as a function of time (x axis), integrated over different temporal windows. The integrated value of these reinforcers may reverse depending on the temporal window, or domain of the x axis over which the area under the curves is integrated. In a short temporal window, drugs (offering brief, intense, immediate, and reliable reinforcement) are more highly valued than prosocial reinforcers (offering temporally extended, delayed, lower-intensity, variable reinforcement); in a longer temporal window, these valuations reverse

Full size image

Conclusions

Given the health crisis of addiction and the relatively modest effects of existing treatments, efforts to identify new mechanisms as well as our theoretical understanding of how these models work is a worthwhile endeavor. In this article, we describe an approach, experimental medicine, that has guided our research program. This approach has allowed us to identify a target related to the disorder, employed methods to engage and change that target, and observed changes in other components of the disorder (e.g., drug valuation as measured by craving, purchase task, and self-administration). In doing so, the approach allowed us to develop a theoretical account of the results: reinforcer pathology. Therefore, the adoption of an experimental medicine approach may allow us to move the needle to assist the recovery of millions of people suffering from addiction.