Keywords

1 Overview

Anhedonia has long been considered a core clinical feature of schizophrenia [1–4], which has been shown to predict important clinical outcomes, such as social and vocational functioning, recovery, and disease liability [5–7]. The most common definition of anhedonia is that it reflects a diminished capacity to experience pleasure. In an early theory of anhedonia, Rado [4] proposed that individuals with schizophrenia had an “integrative pleasure deficiency”. This deficiency was thought to be all-encompassing, impacting the frequency and intensity with which patients both expressed and experienced positive emotions. Although such views have guided diagnostic and treatment practices for over years now, modern research suggests that they may not be fully accurate. For example, although individuals with schizophrenia express less positive emotion in facial and vocal channels than healthy controls in response to evocative stimuli, they report experiencing levels of positive emotion that are equivalent to healthy controls [8, 9]. This disjunction between outward expression and subjective experience suggests that the common-sense notion that people who do not display emotion are not experiencing much emotion may not apply to people with schizophrenia.

Indeed, individuals with schizophrenia on average appear to report feeling as much positive emotion as healthy individuals in response to a range of pleasurable stimuli, such as film clips, complex photographs, food, drinks, social interactions, faces, and words (see [10] for a review). However, not every study has found fully normal reports of positive emotion in response to evocative stimuli (see [11–13]), particularly olfactory stimuli (e.g., [14, 15]). To gain greater clarity regarding whether individuals with schizophrenia do in fact have a diminished capacity for pleasure, Cohen and Minor [16] conducted a meta-analysis of 26 laboratory-based studies where people with schizophrenia and healthy controls were asked to indicate their self-reported level of positive emotion to evocative stimuli. Meta-analytic results indicated that schizophrenia patients reported levels of positive emotion that were comparable to controls in response to pleasant, neutral, and unpleasant stimuli. This finding held true regardless of stimulus type and rating scale procedure (i.e., unipolar or bipolar rating scales). Cohen and Minor’s [16] meta-analytic results also indicated that people with schizophrenia reported greater negative emotion than controls in response to unpleasant, neutral, and pleasant stimuli, consistent with an abnormality in state negative, rather than positive emotional experience. Although these findings on valence are informative, they provide only a partial answer to the question of whether in-the-moment positive emotional experience is intact in schizophrenia. To further explore hedonic capacity in schizophrenia, Llerena, Strauss, and Cohen [17] conducted a meta-analysis on 26 laboratory-based studies of self-reported subjective arousal to emotional stimuli. Arousal is the second major component of prominent models of emotional experience, which is thought to reflect the intensity of motivational activation of the positive valence system. If individuals with schizophrenia do indeed evidence reduced hedonic capacity, one might expect that their self-reports of arousal would be diminished relative to controls. However, meta-analytic results indicated that this was in fact not the case- schizophrenia patients and controls evidenced similar levels of subjective arousal to pleasant stimuli, providing additional support for intact hedonic experience in schizophrenia. These findings appear to suggest that hedonic capacity may be intact in schizophrenia, and have lead some to propose that individuals with schizophrenia should no longer be considered “anhedonic” in the strictest sense of the word [18].

Based on the self-report literature reviewed above, there are several potential problems with the conclusion that individuals with schizophrenia do not evidence a reduced capacity for pleasurable experiences. One potential problem, or caveat, could be that reductions in hedonic capacity are not characteristic of the majority of individuals with schizophrenia, but only a small subgroup. If true, this could result in a masking of hedonic deficits when data are analyzed at the group level. Suspecting that hedonic normality may be a by-product of clinical heterogeneity in schizophrenia, since only approximately 25 % of patients display clinically elevated negative symptoms [19], Strauss and Herbener [20] used a data-driven statistical approach to determine whether a subset of schizophrenia patients could be identified who displayed a diminished capacity for in-the-moment pleasure. Self-reported valence and arousal reports were obtained in response to photographs in a sample of schizophrenia patients and controls, and these scores were submitted to cluster analysis to examine whether meaningful sub-groups of patients could be identified based upon patients’ in-the-moment affective self-report. Consistent with hypotheses, results supported the existence of two affective sub-groups within the patient sample: one that was affectively normal with self-reports of valence and arousal that were indistinguishable from controls, consisting of 60 % of patients, and a second sub-group that was affectively abnormal (40 %). Discriminant function analysis confirmed that these two groups were indeed reliable and highly separable. However, contrary to expectations, the affectively abnormal sub-group did not evidence lower valence or arousal values for pleasant stimuli. Instead, they reported increased negative emotion and arousal in response to unpleasant stimuli, with reports of valence and arousal that were comparable to controls in response to pleasant stimuli. Thus, “state” affective abnormalities may indeed characterize only a minority of schizophrenia patients, and these abnormalities primarily involve negative, rather than positive emotions.

A second potential problem with the conclusion that schizophrenia patients do not display a hedonic deficit based upon the self-report literature alone is that it is possible for the neural response to pleasurable stimuli to be abnormal, even when subjective report is intact. It is possible that neural response provides a more objective estimate of hedonic capacity, which is less influenced by demand characteristics that influence self-reports. Numerous functional magnetic resonance imaging (fMRI) and Positron Emission Tomography (PET) studies have examined neural response while patients and controls reported their subjective positive emotional experience to evocative stimuli. Results of these studies have been somewhat inconsistent. Some studies have indicated that individuals with schizophrenia have diminished activation in response to pleasant affective stimuli relative to controls (e.g., [21]), whereas others have indicated that individuals with schizophrenia have levels of neural activation that are comparable to controls (e.g., [22]). Discrepancies in group-differences may in part reflect methodological differences such as stimulus type, whether activation contrasts are calculated in relation to neutral stimuli or baseline, and whether subjects are asked to rate their feelings in response to the stimulus or rate the stimulus itself [23–25]. Psychophysiological studies measuring affective modulation of startle response are also consistent with intact hedonics (see [10]). In these studies, startling noises are presented at various times while participants view emotional or neutral stimuli. Startle stimuli reliably induce a reflexive eye-blink response, and the magnitude of this response is modulated depending upon whether the startle probe is presented in the presence of pleasant, neutral, or unpleasant stimuli. Unpleasant stimuli potentiate the startle response moreso than neutral stimuli, which result in greater startle than pleasant stimuli. Studies examining affect modulated startle in schizophrenia have found that both healthy controls and people with schizophrenia evidence similar patterns of startle potentiation to pleasant, unpleasant, and neutral stimuli (i.e., unpleasant > neutral > pleasant) [10]. Thus, studies examining neural and psychophysiological response to laboratory-based stimuli are generally consistent with laboratory-based self-report studies suggesting intact hedonic responses to pleasant stimuli.

A third potential criticism of the notion that hedonic capacity is normal in schizophrenia based upon the literature described thus far is that laboratory-based studies may lack ecological validity. Is it possible that patients have intact hedonic responses in the laboratory, yet report markedly different experiences during real-world activities? Several experience-sampling studies have explored the self-report of positive and negative emotions during daily activities. Early studies concluded that patients reported less intense and less variable experiences of positive emotions [26, 27]. However, these early studies failed to take into account that individuals with schizophrenia engage in fewer activities, and that averaging across all time points may complicate interpretations regarding capacity because patients simply have fewer opportunities for pleasurable events in their lives. In two studies where in-the-moment pleasure was examined in relation to instances when patients were engaged in activities, it was found that people with schizophrenia reported increases in positive emotion that were comparable to controls [28, 29]. Thus, contrary to the notion that schizophrenia patients are anhedonic, in-the-moment positive emotion has been found to be intact when patients report their level of positive emotion when engaged in activities during everyday life.

The aforementioned empirical studies therefore appear to support the conclusion that individuals with schizophrenia do not in fact have a diminished capacity for pleasure, as has long been assumed. The notion that hedonic capacity is reduced in schizophrenia primarily originated from interpretations of self-reports that patients would provide during clinical interviews. For decades, self-reports of anhedonia gathered through clinical interviews of negative symptoms have been taken as irrefutable evidence that people with schizophrenia have a diminished capacity for pleasure (see [30] for review of assessment strategies). Indeed, when such interviews are administered, the majority of schizophrenia patients are rated as having clinically significant anhedonia. For example, in a large sample of archival data from our research group on 385 patients who had been rated using the Scale for the Assessment of Negative Symptoms (SANS: [31]), 82 % met criteria for at least mild severity of anhedonia and 58 % for moderate or higher (i.e., the majority were rated as having clinically significant anhedonia). However, do the reports obtained from these scales reflect a diminished capacity for pleasure, or an impairment in some other aspect of affective functioning? To clarify this matter, it is helpful to carefully examine the nature of questions that are asked during a clinical interview, as well as the anchors used to make the determination that a patient is anhedonic. On clinical interviews such as the SANS [31], it is common place to ask patients to provide “retrospective” reports of how often they engaged in different pleasurable activities over the past week, past 2 weeks, or past month. Interviewers are then tasked with translating the information gleaned from their interview into a rating of anhedonia on the clinical rating scale. This involves trying to match the patient’s report to several levels of anhedonia denoted by anchors on the item being rated. Examination of the individual anchors on the SANS anhedonia items provides valuable information about whether scores on these scales can actually be taken as evidence for a diminished capacity for pleasure. On the SANS, which is perhaps the most widely used clinical rating scale, anchors require the interviewer to rate the frequency with which the patient reports having recently engaged in pleasurable activities, such as social interactions, sexual activity, and recreational pursuits. They do not require an evaluation of whether the patient reports feeling maximally good when exposed to potentially pleasurable activities, which would evaluate capacity for pleasure. This may suggest that anhedonia in fact reflects a behavioral, rather than experiential abnormality in schizophrenia. Recognizing this possibility, newer next-generation negative symptom scales like the Brief Negative Symptom Scale (BNSS: [32]) and Clinical Assessment Interview for Negative Symptoms (CAINS: [33]), include items examining the frequency with which patients engage in pleasurable activities. Additionally, data from real-world experience sampling studies supports the notion that a substantial proportion of schizophrenia patients engage in fewer pleasurable behaviors than controls, but do not experience reductions in pleasure when they are in fact engaged in activities [28, 29]. Thus, although self-reports obtained via clinical interview are commonly interpreted as reflecting a reduction in the capacity to experience pleasure, this interpretation may be incorrect; a more appropriate interpretation may be that schizophrenia patients display a behavioral deficit characterized by reductions in seeking out pleasurable activities.

In summary, there is increasing consensus that individuals with schizophrenia are not anhedonic in the traditional sense of the term. That is, they do not appear to have a diminished capacity for pleasure. Evidence supporting this claim comes from laboratory-based self-report studies of valence and arousal, functional neuroimaging and psychophysiology studies indicating intact neurophysiological response to pleasant stimuli, and experience-sampling studies indicating that patients report increases in positive emotion that are comparable to controls when they are engaged in activities. Instead, anhedonia appears to at least in part reflect a behavioral abnormality, whereby patients initiate fewer instances of goal-directed behavior aimed at obtaining rewards. Although this revised view of anhedonia as a behavioral, rather than experiential deficit (see also [10, 18, 25, 34] for similar suggestions), provides meaningful advances regarding the nature of anhedonia, it does not shed light onto the mechanisms that contribute to this behavioral abnormality. The remainder of this chapter is devoted to providing a mechanistic account for this behavioral component of anhedonia, capitalizing on recent advances in the field of affective neuroscience and the application of neuroscience frameworks to studying reward processing in schizophrenia.

2 The Behavioral Component of Anhedonia: A Deficit in Translating Reward Information into Pleasure-Seeking Behavior

The simplest understanding of why individuals with schizophrenia do not initiate pleasurable activities as often as controls would be that they do not find such activities enjoyable. However, since this explanation does not appear to be correct, an important question therefore emerges: “Why do apparently normal hedonic experiences not translate into actions aimed at obtaining rewards?”

One explanation for why normal hedonic responses do not translate into behaviors aimed at obtaining rewards is that patients have deficits in various reward-related processes that are needed to promote decision-making and action selection (see [35]). The basic neuroscience literature has identified core neural systems that are involved with processing and integrating rewards, as well as translating them into value signals that can be used to guide action selection. Several of these systems and their corresponding reward-related processes have been studied in schizophrenia, including: (1) Reward prediction; (2) Value representation; (3) Uncertainty-driven exploration; and (4) Effort-value computations. Barch and Dowd [34] proposed that deficits in translating reward information into motivated behavior are subsumed by abnormalities in frontal-striatal circuitry. The sections that follow describe the neural mechanisms responsible for the aforementioned aspects of reward processing and review the relevant literature on how these reward components are affected in individuals with schizophrenia to evaluate the possibility that dysfunctional frontal-striatal circuitry contributes to deficits in appetitive behavior.

3 Reinforcement Learning and Reward Prediction

Two interactive and complementary neural systems have been shown to be involved with reinforcement learning and reward prediction [36]. The first system is mediated by the prefrontal cortex (PFC), especially the orbitofrontal cortex (OFC), and involves rapid learning. The rapid learning system updates mental representations of value for stimuli and response alternatives on a trial-by-trial basis, and guides decision-making by allowing individuals to flexibly respond to changes in reinforcement contingency. The second system is a slower learning system, which is mediated by the basal ganglia (BG). Learning achieved through this system occurs gradually across a number of trials [36]. Both of these systems have been shown to utilize prediction error signals to guide learning. Prediction errors occur in the presence of mismatches between expected and obtained outcomes, and can be either positive or negative. Positive prediction errors are signaled by phasic increases in dopamine activity when individuals receive outcomes that are better than expected. In contrast, negative prediction errors are associated with transient decreases in dopamine cell activity in response to outcomes that were worse than expected. From a functional standpoint, positive and negative prediction errors serve a critical role in informing motivated behavior by signaling which actions have resulted in outcomes that should be repeated or avoided.

Several behavioral and neuroimaging studies have investigated the integrity of the fast and slow learning systems, as well as prediction error signaling, in people with schizophrenia. There is consistent evidence that patients have deficits in rapid learning and making trial-by-trial adjustments in response to positive and negative feedback [37, 38]. Additionally, some studies suggest that higher levels of clinically rated negative symptoms, including anhedonia [37, 38], are associated with impairments in rapid learning and making adjustments to behavior in an adaptive manner. Functional neuroimaging studies indicate that deficits in rapid learning are associated with aberrant activation in the prefrontal cortex, especially the orbitofrontal cortex [39, 40].

Several studies have also investigated the integrity of the gradual, basal ganglia-driven, learning system using a variety of tasks, such as motor learning, serial reaction time, and cognitive skill-based paradigms [41, 42]. Results from these studies are somewhat inconsistent (see [43, 44]); however, the majority of studies suggest that gradual learning may be relatively intact in schizophrenia [45]. Discrepancies among these gradual learning studies may reflect a combination of differences in task properties, as well as subject-related characteristics. In particular, antipsychotic medications may affect gradual learning, as chlorpromazine equivalent dosage has been linked to procedural learning [46] and procedural learning impairments are more mild in antipsychotic naïve patients [47]. Very high levels of D2 blockade may therefore significantly impair gradual learning. Given that the majority of studies examining the gradual learning system appear to suggest that patient performance is relatively spared, one might be tempted to infer that basal ganglia activation is relatively normal during procedural learning. However, several neuroimaging studies suggest that normal learning may in fact be accompanied by abnormal neural activation in many areas, including the basal ganglia [48, 49]. It may therefore be the case that patients achieve normal gradual learning through use of a number of cognitive processes, as well as neural substrates outside of the neostriatum.

In many reinforcement-learning paradigms, it is also possible to make a dissociation between reward-driven “Go” learning and punishment-driven “NoGo” learning. Several studies indicate that schizophrenia patients display intact NoGo learning, but impaired Go learning. Waltz et al. [50] administered the probabilistic stimulus selection task, which includes an initial learning phase for pairs of probabilistically reinforced stimuli (e.g., AB: 80/20 %; CD: 70/30 %; EF: 60/40 %) and a subsequent test phase where the stimuli presented in the initial phase are paired with each other and novel stimuli. Go learning can be assessed in test phase performance by examining the extent to which a subject selects the most highly rewarded stimulus (A) when it is paired with novel stimuli that were not paired with (A) during the acquisition phase (i.e., CDEF). NoGo learning is assessed by evaluating the number of times a subject avoids the least rewarding stimulus (B) when it is paired with novel stimuli not paired with (B) during the acquisition phase (i.e., CDEF). Consistent with spared NoGo learning, and impaired Go learning, Waltz et al. [50] found that patients had a selective deficit in choosing A at test, but no impairment in avoiding B. Importantly, these Go learning impairments were most profound in patients with a greater severity of clinically rated negative symptoms. Using different paradigms, Strauss et al. [51] and Waltz et al. [38], also found that patients had selective deficits in “Go” learning, which were associated with greater severity of negative symptoms. This pattern of performance can be considered a perfect neurobehavioral recipe for the behavioral component of anhedonia, i.e., patients can adequately learn to avoid outcomes that lead to aversive outcomes, yet have deficits in learning to select actions that had previously yielded reward.

Although these studies indicate that there is a link between negative symptoms and Go learning, they do not provide a clear indication of the cognitive and neural mechanisms that underlie this deficit. Studies using computational modeling and functional neuroimaging have offered valuable insight into these potential mechanisms. One explanation for the Go-learning deficit is that it could result from a failure to generate or learn from positive prediction errors that occur during positive outcomes. Such a deficit would likely implicate aberrant dopaminergic signaling during prediction errors. Alternatively, orbitofrontal cortex driven deficits in value representation could keep patients from precisely representing the value of response alternatives during decision-making. To explore these two alternative explanations, Gold et al. [52] administered a probabilistic reinforcement learning task that allowed for dissociation between value representation and prediction error abnormalities. Participants were presented with four stimulus pairs: in two of the pairs, the correct choice led to a monetary reward on either 90 or 80 % of trials with incorrect choices leading to no reward; in the other two pairs, the correct choice led to the avoidance of a monetary loss on 90 or 80 % of trials. Using this design, selection of the correct response is associated with the generation of a positive prediction error (and phasic dopamine burst) in both the gain and loss avoidance pairs. Behavioral results indicated that patients with more severe avolition and anhedonia showed impaired acquisition of the gain pairs, but intact performance on the loss avoidance pairs. These findings indicate that patients are able to use prediction errors to guide learning, at least when the positive prediction error is associated with successful loss avoidance and intact learning from negative prediction errors. A second important finding was that in the transfer phase, when stimuli learned during acquisition were presented in novel pairings, only the patients with elevated avolition and anhedonia failed to prefer the stimuli associated with rewarding outcomes over those that had been associated with loss avoidance (i.e., those associated with positive prediction errors that did not have positive expected value). In essence high negative symptom patients primarily made choices based on the history of prediction errors, not by their expected value. Computational modeling confirmed this interpretation, providing separate estimates of whether prediction error signaling in the basal ganglia (actor-critic model) or prediction errors used to update value representations of actions in the OFC (Q-learning) were most representative of behavioral performance. The modeling results were very clear: performance of avolitional/anhedonic patients was well fit by a pure actor-critic model, whereas healthy controls and patients with low avolition/anhedonia were best fit by the model where the actor-critic was supplemented by the contribution of Q-learning. These modeling results provide further support for the interpretation that the deficit observed in avolitional/anhedonic patients reflects impairments in value representation, not learning from prediction errors. Thus, behavioral and modeling data indicated that prediction error signaling is largely spared in schizophrenia.

However, the functional neuroimaging literature paints a picture that is not entirely consistent with the behavioral and computational modeling data regarding prediction errors. On the one hand is imaging data indicating intact activation in the ventral striatum in relation to negative prediction errors [39, 53, 54]. These findings are consistent with the behavioral and modeling evidence. However, on the other hand, data from several imaging studies indicates that positive prediction errors are accompanied by reduced neural response in the ventral striatum, as well as other regions such as the insula, frontal cortex, amygdala, hippocampus, putamen, and cingulate [39, 53–59], although, reduced striatal response has not been universally found (see [39, 60, 61]). Discrepancies across studies may to some extent reflect characteristics of the patient samples that were studied since individual differences in clinically rated negative symptoms predicted striatal response [39, 54, 60, 61]. Thus, the literature on the integrity of positive prediction error signaling is unclear; however, one interpretation of these imaging findings is that poor learning from positive feedback is driven by aberrant positive prediction errors and dopamine signaling in the midbrain.

Reward prediction, which refers to the ability to anticipate a reward when a predictive cue is presented, is another factor that drives pleasure-seeking behavior. Dopaminergic activity in the striatum is thought to play a key role in this process, allowing affective salience to become linked to predictive cues. The monetary incentive delay paradigm has been used to study the neural substrates of reward anticipation in several schizophrenia studies. In this task, different colored shapes predict gains, losses, and neutral outcomes and it is possible to differentiate neural response during the anticipation of rewards (ventral striatum) from neural response during the receipt of rewards (medial prefrontal cortex). Monetary incentive delay results have indicated that individuals with schizophrenia have reduced activation in the ventral striatum in response to cues predicting upcoming rewards [62–64]. These findings hold true in patients who are unmedicated or taking first generation, but not second-generation antipsychotics [63, 64]. Several studies also report that blunted striatal activation during reward anticipation is associated with greater severity of negative symptoms [39, 54, 61], and these relationships hold true in patients taking second-generation antipsychotics [39, 61]. One complication of interpreting results from the monetary incentive delay or other instrumental learning paradigms is that reward anticipation is dependent on the subject’s ability to earn rewards via appropriate responding and therefore relies on several cognitive processes that are known to be impaired in schizophrenia other than prediction errors. Clarifying this matter somewhat, Pavlovian conditioning paradigms evaluate reward anticipation and prediction error signaling independent of factors like action selection and response execution. Waltz et al. [54] used a passive conditioning paradigm which presented subjects with a light cue and then a squirt of juice. To allow for an examination of neural response to positive and negative prediction errors, on 75 % of trials, juice receipt occurred exactly 6 s following light cue, whereas receipt was delayed by a further 4–7 s on 25 % of trials. Imaging results indicated reduced neural response to positive prediction errors in several brain regions, but largely intact neural response to negative prediction errors. Dowd and Barch [60] administered a Pavlovian reward conditioning paradigm with no response requirements, where subjects passively viewed cues (colored shapes) that predicted subsequent monetary reward or non-reward. Imaging results indicated that at the group level, neural response to reward receipt and anticipation were comparable between patients and controls; however, individual differences in self-reported anhedonia were associated with reduced activation in the left ventral striatum and ventromedial prefrontal cortex during reward anticipation. Thus, findings from conditioning paradigms without response demands and instrumental paradigms with significant response demands are largely consistent- patients with higher levels of anhedonia evidence reduced activation in the ventral striatum and ventromedial prefrontal cortex during reward anticipation. Thus, reward anticipation may be impaired in schizophrenia, whereas reward receipt may not.

4 Value Representation

Several research groups have proposed that abnormalities in “value representation” may be critically linked to anhedonia and avolition in schizophrenia [25, 34, 35]. In particular, reduced reward-seeking and goal-directed behavior is thought to be associated with impairments in generating, maintaining, and updating mental representations of value. The orbitofrontal cortex (OFC) plays a critical role in several aspects of value representation [65]. For example, the OFC is responsible for calculating the value of an outcome, evaluating how much an outcome satisfies current motivational needs, and comparing the value of an outcome with other possible outcomes [65]. Like other regions of the PFC, the OFC serves the purpose of holding information about reward value in working memory, which in turn facilitates goal-directed behavior by indicating when outcomes have changed and action plans need to be updated.

Compared to other aspects of reward processing, relatively few studies have examined the integrity of OFC function as it relates to value representation in schizophrenia. The two tasks associated with lateral and medial OFC function that have been most frequently used to study value representation in schizophrenia are Probabilistic Reversal Learning and the Iowa Gambling Task. In probabilistic reversal learning, participants are presented with pairs of stimuli that are probabilistically reinforced (e.g., selection of stimulus A reinforced 80 % of the time; Selection of stimulus B reinforced 20 % of the time) and asked to learn which is the correct stimulus. Instructions stipulate that subjects should continue selecting the stimulus they think is correct until they determine that the correct stimulus has changed. Once subjects meet some predetermined criteria for demonstrating adequate learning of the most frequently reinforced stimulus, the contingencies are reversed (e.g., Stimulus A reinforced 20 % of the time; Stimulus B reinforced 80 % of the time). In this reversal phase, the number of errors made by the subject and the number of trials needed to reach criterion have been linked to OFC function, and reflect how well and individual can integrate positive and negative feedback across trials to update value representations that are used to guide action selection. When individuals with schizophrenia have completed probabilistic reversal learning tasks, or Intra-dimensional/extra-dimensional set-shifting tasks, it has been found that they are more impaired than controls at the reversal stage of this task [37, 66–70]. Neuroimaging evidence indicates that impairments in the reversal phase are associated with reduced deactivation of the medial prefrontal cortex [71]. Additionally, elevated clinical ratings of anhedonia and avolition are associated with the magnitude of patients’ deactivations in the ventromedial prefrontal cortex and ventral striatum [71]. Thus, findings confirm the role of ventrolateral prefrontal cortex and dorsomedial prefrontal cortex in updating mental representations of value, as well as a link between these regions and reduced pleasure-seeking behavior.

Individuals with schizophrenia have also demonstrated impairments on the Iowa Gambling Task ([71–78]; however see [79–81]). This task requires subjects to draw one card at a time from four decks (A-D). Each selection either results in winning or losing money, with the frequency and magnitudes of gains and losses differing across decks. Two of the decks are disadvantageous and result in high immediate gains as well as even higher losses, such that selecting from these decks on average leads to more overall loss. The other two decks are more advantageous, with selections resulting in low immediate gains and infrequent low-value losses. Choosing these advantageous decks results in more net gains on average. Neurological patients with OFC lesions are more likely to select from the disadvantageous decks [82]. Although individuals with schizophrenia evidence volumetric reductions in the OFC, these reductions are not predictive of Iowa Gambling Test performance in patients like they are in controls [76, 83]. Thus, impaired Iowa Gambling Test performance has been noted in schizophrenia, but it is unclear whether these deficits reflect abnormalities in the OFC or other structures; it is therefore possible that aspects of cognition other than value representation may contribute to deficits observed on this task.

Another task that has been associated with OFC dysfunction is a simple preference judgment task that evaluates “relative” value assignments. In this task, participants are presented with a set of like items (e.g., pictures of cute puppies) and asked to select the stimulus that they prefer [84]. There are no correct or incorrect answers and no outcome occurs in relation to choices. All stimuli within the set are presented in conjunction with every other stimulus, making it possible to examine the hierarchy of preferred stimuli and the consistency of selections relative to preferences. For example, if a subject prefers stimulus A over B and B over C, they should also prefer A over C. Failures to maintain transitivity of preferences have been linked to the ventromedial prefrontal cortex (defined as the region encompassing both medial OFC and adjacent ventral medial PFC) in lesion studies [84]. One study administered this preference task to a sample of schizophrenia outpatients and demographically matched healthy controls [85]. Results indicated that schizophrenia patients were both less consistent in their selections (i.e., more errors in transitivity) and more likely to have larger magnitudes of discrepant responses than controls. Furthermore, whereas controls showed clear differentiation between degrees of valence in a condition that presented a set of pleasant and unpleasant stimuli selected for normative gradations in valence (i.e., highly positive > mildly positive > mildly negative > highly negative), patients showed no preference for highly positive over mildly positive items or mildly negative over highly negative items (despite preferring positive to negative stimuli). Abnormal preference judgments were also correlated with self-reported anhedonia on the Chapman scales and general working memory impairments. When viewed in relation to the broader neuroscience and lesion literature on value representation using the preference task, these behavioral results are consistent with the notion that OFC dysfunction is linked to deficits in developing or maintaining nuanced representations of value that occur in schizophrenia.

The delayed discounting paradigm has also been suggested to involve value representation. This task examines the degree to which individuals prefer smaller rewards sooner or larger rewards later. When the slope of a delayed discounting function increases, this indicates a preference for more proximal rewards. Steeper discounting rates have been linked to abnormalities in both the nucleus accumbens and ventromedial cortex, suggesting that discounting abnormalities may reflect both dopaminergic dysfunction and deficits in value representation. Such deficits have also been found in multiple forms of psychopathology [86, 87]. In delayed discounting experiments examining individuals with schizophrenia, where participants were presented with an option for smaller immediate rewards or larger delayed rewards, it has been found that schizophrenia patients evidence steeper discounting rates than controls, i.e., they prefer smaller immediate rewards over larger delayed rewards [88, 89]. A functional neuroimaging study of delay discounting in schizophrenia patients and controls matched on behavioral performance indicated that patients had less activation in inferior frontal, dorsal anterior cingulate, and posterior parietal cortices, as well as the ventral striatum [90]. It is easy to see how deficits in representing the value of future outcomes might contribute to impairments in reward-seeking behavior in schizophrenia. Simply put, when value cannot be represented precisely, rewards that cannot be obtained immediately may not have enough pull to motivate patients to produce the actions needed to obtain them.

In addition to deficits in generating and updating value representations, there is also some evidence that schizophrenia patients have impairments in maintaining value representations. Gard et al. [91] had patients and controls perform an emotional maintenance task, where subjects were presented with two stimuli of similar valence (e.g., both pleasant) that were separated by a short delay (3 s). Participants were instructed to determine whether the first or second image was stronger in intensity, and these evaluations were compared to ratings of intensity that the subjects made later in a separate task to determine the presence of emotional maintenance errors. Results indicated that patients made more errors in maintaining intensity judgments, suggesting that they had a deficit in maintaining value representations and using them to appropriately guide decision-making. Similarly, in a psychophysiological study by Kring et al. [92], startle probes were presented during stimulus presentations of affective and neutral photographs, as well as during the delay period between stimulus presentations. Similar to prior startle studies examining startle potentiation during stimulus viewing, schizophrenia patients and controls demonstrated comparable affect modulated startle potentiation when images were on screen. However, whereas controls continued to display affect modulated startle during the delay period, schizophrenia patients did not, consistent with a deficit in maintaining value representations. A functional neuroimaging study utilized a similar paradigm, where neural activation to affective and neutral images was examined while stimuli were on screen, as well as during the delay period following stimulus offset [22]. Results indicated that schizophrenia patients had comparable neural response to controls in the presence of emotional stimuli, but reduced neural activation during the delay period in several areas, including the dorsolateral and ventromedial/orbitofrontal cortices. Furthermore, delay period activity in the dorsolateral prefrontal cortex for pleasant stimuli was correlated with individual differences in clinically rated anhedonia. Thus, schizophrenia patients may have deficits in maintaining mental representations of value and using them to guide decision-making- a problem that stems from reduced OFC activation.

Collectively, these studies provide evidence that a distributed network of regions is involved in deficits in generating, updating, and maintaining mental representations of value. Given that some of these abnormalities occur in tasks that require simple preference judgment in the absence of learning and feedback processing, impairments in value representation do not appear to be merely byproducts of reinforcement learning abnormalities. That is not to say that value representation is not influenced by general cognitive impairments or working memory specifically. Indeed, there is strong evidence for such associations, supporting the notion that working memory deficits may underlie the ability to couple affective value and behavior [93]. The ability to seek out pleasurable activities and perform goal-directed behavior may be highly influenced by a patient’s ability to generate and maintain value representations in working memory. When value representations are not sufficiently salient or not adequately sustained, it is unlikely that they will be salient enough to adequately motivate behavior. Thus, value representation impairments may be a key contributor to the behavioral component of anhedonia in schizophrenia.

5 Uncertainty-Driven Exploration

In everyday life, we are constantly forced to make decisions between actions that have resulted in positive outcomes in the past, versus trying out new actions that could yield even better results. For example, when at a favorite local restaurant, do you order your tried and true favorite dish? Or do you go with the special of the day which you have never tried in hopes that it is even better than your old reliable? This decision-making process, termed the exploration-exploitation dilemma, is experienced at all levels of behavior and influences decisions ranging from how to plan one’s day to which job to apply for. How an individual approaches this exploration-exploitation dilemma therefore critically impacts the frequency with which they engage in behaviors aimed at obtaining rewards, as well as the variety of pleasurable activities that they are exposed to. Given that current conceptual frameworks (e.g., [18]) and newer negative symptom rating scales [32, 33] emphasize the frequency and variety of pleasurable activities as core aspects of anhedonia, it is possible that exploration and exploitation may offer hope for a mechanistic account of anhedonia in people with schizophrenia.

Sometimes it is adaptive to repeat actions that have previously lead to reward (i.e., exploit). This is particularly true when individuals encounter “stationary” environments, where reinforcement contingencies are stable. In such circumstances, individuals can make decisions based upon expected value and exploit to maximize rewards [94, 95]. In stationary environments, exploitation is heavily influenced by dopamine nuclei and target areas in the basal ganglia and prefrontal cortex [96, 97].

However, many real-life situations involve environments that are “non-stationary”, where reinforcement contingencies are not stable. In such circumstances, it may be more valuable to explore the value resulting from actions with uncertain outcomes, in hopes of obtaining rewards that are greater than those previously experienced. Exploration can be achieved through several strategies. One strategy is to repeat behaviors that have best lead to reward (i.e., exploit), while also discovering over time whether there are better options by occasionally choosing a different action at random [98]. Another strategy is more systematic and involves selecting actions based upon their level of uncertainty relative to the status quo (i.e., the exploited option). By continuously tracking both the frequency and magnitude of potential options, as well as the degree of uncertainty associated with them, individuals using this strategy maximize the amount of information learned about potentially rewarding outcomes. Uncertainty-driven exploration may therefore be a more ideal strategy for enhancing the probability of obtaining maximal rewards.

Several neurobiological processes are involved with uncertainty-driven exploration. At the neuroanatomical level, human neuroimaging evidence indicates that the rostrolateral prefrontal cortex is responsible for tracking uncertainty in an ongoing manner to promote exploratory behavior [97, 99]. Individual differences in uncertainty-driven exploration have also been linked to genes associated with prefrontal dopamine function (COMT), while individual differences in exploitation are associated with genes controlling striatal dopamine function (DARPP-32 and DRD2) [100]. Exploration may depend on one’s ability to engage more dorsal and anterior regions of the prefrontal cortex that drive top-down control and limit prepotent behavioral responses in favor of selecting new actions aimed at obtaining maximal reward [94]. A second explanation is that exploratory behavior is influenced by neuromodulatory control of cortical norepinephrine [94, 101–103]. In particular, it is thought that phasic and tonic norepniephrine release serves to differentially promote exploration and exploitation as a function of ongoing utility estimates that are governed by frontal and medial regions of the prefrontal cortex. These prefrontal control regions, which are known to be impaired in schizophrenia, are critical for regulating the balance between decisions to explore or exploit under conditions of uncertainty [94, 103]. Based upon the basic and cognitive neuroscience literature, one could therefore imagine that multiple mechanisms could contribute to reduced exploration and these have been implicated in schizophrenia.

To date, few studies have examined exploration and exploitation in schizophrenia. Strauss et al. [51] administered the Temporal Utility Integration Task [104] in which participants observe a moving hand rotate throughout a clock face over a five second period. Subjects were asked to press a button to stop the clock hand at any point on the clock in order to earn a reward, with the goal of winning the most points possible throughout the task. Reward magnitude and probability was manipulated in relation to response time, such that expected value increased, decreased, or remained constant at different levels of response time. Across several conditions, denoted by blocks where clock faces appeared over different colored backgrounds, participants were required to learn the optimal strategy for maximizing rewards (e.g., responding more quickly or waiting until the hand reached the end of the clock). Via computational modeling, it was possible to examine trial-by-trial dynamics in response time adjustments to estimate a subject’s degree of uncertainty-driven exploration. Modeling results indicated that patients as a whole were less likely than controls to explore response alternatives when the values of those alternatives were uncertain. Furthermore, reduced exploration predicted individual differences in clinically rated anhedonia on the Scale for the Assessment of Negative Symptoms (but not other aspects of negative symptoms) in schizophrenia patients. The specificity of this association with anhedonia but not other aspects of negative symptoms may be meaningful because anhedonia on this scale reflects a behavioral abnormality characterized by reductions in the frequency of pleasurable activities. When a similar computational model was applied to behavioral data from a probability matching task administered in Kasanova et al. [105], a similar relationship between clinically rated negative symptoms and reduced exploration was found. Thus, prior schizophrenia findings provide preliminary support for a novel mechanistic understanding of anhedonia as a deficit in exploring new actions that could lead to a greater magnitude, frequency, or variety of rewarding outcomes compared to rewards gained from actions generated in the past.

Several neurobiological mechanisms may serve to link anhedonia and reduced uncertainty-driven exploration in people with schizophrenia. One possibility, as proposed by Strauss et al., is that reduced exploration results from degredations in prefrontal cortical dopamine function, an abnormality that has been implicated in the etiology of negative symptoms multiple times [106–108]. This interpretation is supported by functional neuroimaging studies indicating that the prefrontal cortex is involved in tracking uncertainty [97, 99], as well as a gene-dose effect of the val/met polymorphism of the COMT gene in healthy individuals performing the same task as Strauss et al. [51] [100]. Impaired prefrontal mechanisms may therefore reduce top-down control needed to inhibit a prepotent exploitative behavior and facilitate exploratory actions under conditions of uncertainty. Although degredation in prefrontal cortical dopamine appears to be the most likely explanation for reduced uncertainty-driven exploation, several additional mechanisms could also be involved.

Huys and Dayan [109] have suggested that major depressive disorder is associated with a deficit in processing uncertainty itself, such that depressed patients assign a negative expected value to uncertain outcomes. Since a sizeable proportion of people with schizophrenia carry a comorbid diagnosis of major depressive disorder and depression contributes to some portion of variance associated with anhedonia in schizophrenia patients, this interpretation seems plausible. However, Strauss et al. did not find an association between exploration and depression, potentially suggesting that the mechanisms underlying reduced exploration may differ between schizophrenia and depression.

Another potential explanation is that schizophrenia patients have a deficit in processing uncertainty itself, and that such deficits contribute to reductions in exploration. Yu and Dayan [101] proposed that expected and unexpected forms of uncertainty exist, and that two neuromodulatory processes may be involved with these processes: acetylcholine and norepinephrine. Decisions to explore and exploit may be critically linked to the processing of expected and unexpected uncertainty. Unexpected uncertainty, which is signaled by norepinephrine, may be particularly important for indicating the need to explore. According to Yu and Dayan’s model [101], we should persist in our current behavior (i.e., exploit) when the extent that we expect an outcome to vary tracks with what we observe in the environment. In contrast, we should select a new course of action (i.e., explore) when there are large discrepancies between our expectations and how often (or to which magnitude) an action yields the expected outcome. Perhaps individuals with schizophrenia have deficits in formulating expectations about how often outcomes should vary, and/or updating their representations of how often outcomes do vary when changes in the environment occur. Such abnormalities in tracking unexpected uncertainty could thus contribute to reductions in exploratory behavior, preventing patients from modifying their actions in non-stationary environments where reinforcement contingencies are changing.

Related to the Yu and Dayan model, [101] Aston-Jones and Cohen [103] proposed that decisions to explore or exploit are critically linked to ongoing utility estimates, which are executed by frontal structures that regulate norepinephrine release. Utility estimates are thought to be fundamental to the decision of whether to give up or persist in instances when task performance might be poor and thus not leading to adequate reward attainment. For example, in a situation where an individual has generally been performing well on a task that yields rewards, but they occasionally make errors on single trials (i.e., transient decreases in utility), it would be to their benefit to persist in the task (i.e., exploit) and try to restore their performance to a high level following errors. In contrast, when an individual is performing poorly on a task and making many errors over consecutive trials (i.e., long-term utility is low and progressively declining), the individual should be encouraged to give up and try out other actions (i.e., explore) that could result in alternative outcomes and potentially better rewards. Based on this model, one possibility is that schizophrenia patients have deficits in tracking long-term utility and using utility signals to promote exploratory behaviors aimed at obtaining rewards in contexts where they have engaged in unsuccessful behaviors that have failed to yield sufficient rewards.

Social context could be yet another important factor driving the exploration-exploitation dilemma in schizophrenia patients. In particular, healthy people may be more likely to explore potential rewards within an environment when they have access to information about the behavior of others, or when they are faced with competition from others for resources that lead to rewards [94]. It is possible that deficits in social cognition, social drive, social skills and asociality may render individuals with schizophrenia less likely to explore based upon interpersonal interactions. Studies examining exploration and exploitation in schizophrenia to date have not manipulated social context; however, this could be an important future direction.

Overall, studies examining uncertainty-driven exploration in schizophrenia have indicated an important association with anhedonia and a novel mechanistic account for reduced reward-seeking behavior. Future studies on exploration are needed to evaluate some of the alternative cognitive and neurobiological explanations posed here.

6 Effort-Value Computations

Another potential mechanism for why normal hedonic experiences do not translate into reward-seeking behavior is that schizophrenia patients have deficits in “effort-value computation” that prevent them from making an accurate estimation of whether the benefits associated with an action outweigh the “costs” related to obtaining them (e.g., physical effort, mental effort). Several behavioral neuroscience paradigms have been used to study the neural substrates of effort-value computation (see [110] for review). One widely used method, the progressive-ratio paradigm, requires animals to exert physical effort (e.g., pressing a lever) to obtain differing magnitudes of reward (e.g., food) [111]. In this paradigm, a reward is initially delivered after the animal has exerted a low number of physical responses, and the threshold for reward receipt is then progressively increased until the animal’s “breakpoint” is determined (i.e., the number of effortful responses at which the animal will no longer work to receive a reward). Another paradigm involves offering the animal a choice between multiple rewards, where one of the rewards (which is either greater in quantity or value) requires greater expenditure of effort to obtain it (e.g., climbing a wall) [112]. This paradigm therefore forces the animal to choose between expending a high degree of effort for a large reward or less effort for a lower reward. The willingness to exert effort aimed at obtaining rewards of differing value has been critically linked to dopaminergic function. Specifically, studies have shown that willingness to work for reward has been affected by focal depletion of dopamine in the nucleus accumbens [111, 112]. Increasing dopamine levels via administration of amphetamine also enhances willingness to exert effortful behavior [113]. In humans, stimulation of dopamine release via administration of d-amphetamine has also been linked to increases in effortful behavior, and individual differences in dopamine release have been found to predict how willing an individual is to work for higher rewards [113, 114]. Striatal dopamine release and dopamine receptor availability may therefore play a critical role in whether high amounts of effort will be exerted to obtain a reward.

Although it is well-documented that schizophrenia patients have dopaminergic abnormalities, these abnormalities are not consistent with what one would expect in a disorder characterized by decreased motivation. For example, the basic neuroscience literature suggests that reduced effortful behavior is associated with reduced striatal dopamine receptor availability and release. However, schizophrenia patients exhibit tonic increases in dopamine levels and greater dopamine release in response to dopamine enhancing agents like d-amphetamine. It therefore seems likely that another mechanism must be contributing to the reductions in effortful behavior that are characteristic of schizophrenia. A recent animal model of motivational impairments in schizophrenia provides one viable explanation for this apparent inconsistency. Ward et al. [115] found that developing mice which are genetically altered to have an overexpression of postsynaptic D2 receptors are less willing to work to receive rewards, despite having normal hedonic reactions [115]. Given that schizophrenia patients do in fact display an increase in D2 receptor availability [116, 117], it seems plausible that reductions in effortful behavior result from an overexpression of postsynaptic D2 receptors rather than reduced striatal dopamine release.

Effort computation has also been neuroanatomically linked to the anterior cingulate cortex. This association has been demonstrated both via animal lesion studies [118–120] and positron emission tomography studies of rats indicating that effortful behavior is predicted by ACC activation [121]. In humans, ACC activation also predicts decisions to expend effort [122, 123]. Consistent with a potential role of the ACC in motivational abnormalities in schizophrenia, several structural MRI studies have indicated that patients have volumetric reductions in the ACC [124, 125]. Functional neuroimaging studies also indicate that schizophrenia patients have aberrant activation in the ACC during tasks requiring conflict or error monitoring (e.g., [126]), providing indirect support for a potential role of the ACC in effort-value calculation.

There is also reason to suspect that reductions in effortful behavior may reflect a circuit-level dysfunction, rather than the ACC and nucleus accumbens making separate contributions in parallel. In a study that lesioned the connection between the ACC and nucleus accumbens, it was found that effortful behavior was reduced equivalently to when the nucleus accumbens alone was lesioned [127]. This may suggest that striatal dopamine abnormalities and the ACC function in concert to contribute to effort-based decision-making.

To date, only two published studies have examined effort-value computations in schizophrenia. In the first such study, Gold et al. [128] administered a computerized behavioral task to a sample of outpatients with schizophrenia and demographically matched healthy controls. Participants were presented with a decision-making task where they could chose between making 20 button presses to obtain $1 (low effort/low reward condition) or 100 presses to obtain rewards ranging from $3 to $7. The probability of reward receipt was also manipulated to determine whether certain (100 % probability) or uncertain (50 % probability) outcomes influenced effort-based decision-making. Results indicated that schizophrenia patients were less likely than controls to select the high effort option in the 100 % probability condition when the potential reward value that could be earned was at its highest ($5, $6, $7). Additionally, the deficit in how willing patients were to work for higher value rewards was uniquely linked to individual differences in negative symptom severity on the Brief Negative Symptom Scale [32, 129, 130]. Patients with high negative symptoms were also less willing than controls to select a high effort option in the 50 % (uncertain) condition, when they had selected a high effort option on the previous trial and been rewarded. Effort-value computation abnormalities were also accompanied by general evidence of appetitive behavioral deficits as indicated by reduced response vigor and increased time needed for task completion (despite selecting more low effort options).

The second study, conducted by Fervaha et al. [131], obtained results similar to Gold et al. [128]. Fervaha et al. [131] administered the Effort Expenditure for Reward Task (EEfRT: [132]) to a sample of schizophrenia patients and controls. This decision-making task asks participants to make either a low effort/low reward choice that requires making a set number of button presses with their dominant hand index finger within 7 s to earn $1, or a high effort/high value choice where they must make a greater number of button presses within 21 s using their non-dominant hand pinky finger to earn higher values ranging from $1.24 to $4.30. Probability of reward receipt is also manipulated to estimate the role of certainty, with probabilities corresponding to either 12, 50, or 88 % on each trial. Importantly, Fervaha also modified the task to account for motoric abnormalities known to impact people with schizophrenia (e.g., finger tapping deficits identified on neuropsychological tests) by individually tailoring the maximum number of button presses for the easy and hard conditions based on a pre-test of the participant’s finger-tapping speed. Results indicated that schizophrenia patients were less willing to expend effort to receive high value rewards, and that these deficits were correlated with clinically rated avolition.

Although few studies have directly examined effort-value computation in schizophrenia, the results of the two studies conducted to date point to an association between negative symptoms and reductions in willingness to put forth effortful responses to obtain high-value rewards. One explanation for these results is that the high negative symptom patients did not find the high-value rewards worth the effort needed to obtain them. Alternatively, deficits in value representation could undermine the decision to engage in effortful behavior, such that the cost associated with the action required to receive a reward seems prohibitively high when value is not represented precisely. Functional neuroimaging studies are needed to examine the neural factors contributing to this effort-value computation dysfunction; however, based upon the pre-clinical and human neuroimaging literature, there is reason to suspect that effort computation deficits are linked to abnormalities in the mesolimbic dopaminergic system and the ACC, and potentially the connectivity between these regions. The human schizophrenia findings are also consistent with data supporting the D2 over-expression animal model of schizophrenia, which provides evidence for intact hedonics in the context of impaired effortful behavior to obtain rewards. Further research is needed to explore the role of antipsychotic medications in unmedicated patients, as D2 antagonists have been found to reduce the extent to which rats are willing to work for rewards.

7 Conclusions and Future Directions

In the past decade, there have been important advances in the conceptualization of anhedonia in individuals with schizophrenia. These developments have at least in part stemmed from the application of frameworks and methods from the fields of affective science and affective neuroscience to study various aspects of reward processing and their association with clinical symptomatology. There is now compelling evidence that individuals with schizophrenia do not have a reduced capacity to experience pleasure when exposed to potentially rewarding activities. Instead, individuals with schizophrenia appear to display a behavioral deficit that manifests as a reduced frequency of engaging in pleasurable activities. The current chapter reviewed several aspects of reward processing that are disrupted in schizophrenia, and evaluated evidence suggesting that this behavioral component of anhedonia is related to an impairment in translating reward information into motivated behavior. Aberrant cortical-striatal interactions may be associated with multiple aspects of reward processing that contribute to reductions in the frequency of pleasurable behavior in schizophrenia, including: (1) dopamine-mediated basal ganglia systems that support reinforcement learning and the ability to predict cues that lead to rewarding outcomes; (2) orbitofrontal cortex-driven deficits in generating, updating, and maintaining value representations; (3) aberrant effort-value computations, which may be mediated by disrupted anterior cingulate cortex and midbrain dopamine functioning; and (4) altered activation of the prefrontal cortex, which is important for generating exploratory behaviors in environments where reward outcomes are uncertain.

Although this mechanistic account provides clarity regarding the cognitive and neural basis of the behavioral component of anhedonia in schizophrenia, there are still several important issues left to be resolved in this area. First, there is some inconsistency among findings within the different areas of reward processing. For example, there are discrepant results among neuroimaging studies examining the integrity of positive prediction error signaling and reward anticipation. A meta-analysis would help clarify whether the neural processes underlying these functions are abnormal, and potentially identify mediators of prediction error signaling that could explain discrepancies in the literature (e.g., individual differences in anhedonia/avolition, first vs. second generation antipsychotics, D2 blockade, general cognitive impairments). There is also a need for a meta-analysis examining neuroimaging studies where subjects are exposed to pleasant stimuli in the laboratory and asked to indicate how positive they feel in response to those stimuli. Results from imaging studies to date do seem to be consistent with the self-report literature indicating intact hedonic responses, but a meta-analysis is needed to support this interpretation. Anticevic et al. [23] provided meta-analytic evidence that imaging contrast methods are a critical factor in determining whether patient neural response is intact for unpleasant stimuli, and it would be necessary to consider these variables for pleasant stimuli as well.

Several factors may contribute to inconsistent findings across neuroimaging studies examining the neural signature of prediction errors and self-reported positive emotional experiences. One factor is clearly clinical heterogeneity. Individual differences in the severity of negative symptoms have been linked to reward processing in many studies. Given that not all studies recruit samples that are enriched for negative symptoms, and only a subset of patients do in fact evidence clinically significant elevations in negative symptoms, it is possible that clinical heterogeneity hinders accurate comparisons across studies. Second, many of the tasks described in this chapter have only been explored at the behavioral level, leaving much in the way of inference to make conclusions regarding the neural circuits involved with behavioral task performance. This is particularly true of studies examining different components of value representation, effort-cost computation, and uncertainty-driven exploration. It will be critically important to conduct neuroimaging studies with some of the tasks reviewed in these sections of this book chapter to determine whether the neurobiological processes inferred to play a role in behavioral performance are in fact correct. Third, some studies suggest differential effects of first and second generation antipsychotics on reward processing, and it may be the case that D2 antagonists explain inconsistency among findings. Few studies have systematically examined the role of antipsychotics in reinforcement learning, and there is a need to randomly assign patients to antipsychotics to disentangle the influences of patient characteristics and medication-specific effects on reward processing.

Another important point of consideration is that few studies have examined more than one aspect of reward processing in the same sample, making it difficult to gage the extent to which these processes interact to contribute to reductions in pleasure-seeking behavior. There are several reasons to think that deficits in one process may contribute to abnormalities in another. For example, prediction error signaling and value representation may be critically linked – one would not expect patients with impaired prediction error signaling to be able to represent value precisely. Similarly, aberrant value representations may contribute to a number of other reward processing deficits, such as computing whether an action is worth the effort needed to obtain it, making the decision to exploit actions that have lead to prior rewards or to explore new actions, and learning to make rapid trial-by-trial adjustments in response to probabilistic feedback. Impairments in reinforcement learning and tracking uncertainty may also interact with several other reward processes. For example, decision-making in stationary environments is highly influenced by learning rate. Schizophrenia patients have consistently been found to have deficits in rapid learning, and these deficits may influence the extent to which they can update value representations and use them to exploit actions that will consistently yield reward. In non-stationary environments, where reward contingencies are not consistent, the ability to track uncertainty may be paramount in determining whether individuals engage in reward-seeking behavior. It is possible that deficits in value and uncertainty representations may have a combined influence on decision-making and behavior in non-stationary environments, limiting the extent to which individuals learn about and explore alternative actions that lead to reward. Finally, effort-cost computations may interact with other processes, such as exploration/exploitation. In some circumstances, such as those occurring in the exploration-exploitation dilemma, there can be “costs” associated with switching from one behavior to another. If patients have difficulty in judging these costs and whether the effort needed to switch to a new action is worth it, they may be less likely to try out new actions (i.e., explore) that can yield more frequent or more maximal rewards. Thus, it is clear that individuals with schizophrenia have deficits in multiple aspects of reward processing, and these may have important interactions that influence how reward information is integrated and translated into behaviors aimed at obtaining rewards.

Although it is clear that various reward processes have important interactions, there may also be some common underlying mechanisms for these deficits. Abnormalities in cortical and subcortical dopamine may be associated with impairments in all of the aspects of reward processing described in this chapter and impede the translation of reward information into pleasure-seeking behavior. Given the role of DA in influencing different components of reward processing, and that the majority of schizophrenia patients are treated with D2 antagonists, it will be important to systematically examine the role of antipsychotics in each of the reward processing domains reviewed in this chapter. This can be done by examining medicated and unmedicated patients, comparing first and second generation antipsychotics, and evaluating individuals at high-risk for psychosis who have not been exposed to antipsychotics. PET studies may also be very helpful in isolating the role of dopamine in different components of reward processing. Furthermore, new animal models of schizophrenia, such as the D2 post-synaptic over-expression model of negative symptoms, have significant potential to clarify the role of dopamine in different aspects of reward processing. Translating these models directly into human studies of medicated and unmedicated patients is an important next step. Additionally, the reward processes described here also place high demands on cognitive control circuits. It is possible that the reward-based deficits described here represent another means by which cognitive control impairments are manifested in the affective domain. Cognitive control circuits may also be influenced by dopaminergic function, and it will be important to explore interactions between cortical and subcortical structures using newer functional imaging connectivity methods.

It is also important to consider that the multiple reward-related processes described in the current chapter only capture some of the mechanisms that may contribute to reduced pleasure-seeking behavior. For example, Kring, Gard, and colleagues [25, 28] have proposed that while schizophrenia patients do not have deficits in experiencing pleasure in-the-moment (i.e., consumatory pleasure), they do have reductions in “anticipatory” pleasure (i.e., a between-groups difference where patients expect less pleasure in the future than healthy controls). Deficits in “affective forecasting” may underlie these impairments in anticipatory pleasure and contribute to reduced pleasure-seeking behavior. For example several cognitive and psychological mechanisms contribute to the anticipation of future pleasure, including retrieving prior pleasurable experiences from episodic memory and generating mental representations of future events that include relevant contextual details and essential features of potential situations [133]. Generating mental representations of future pleasurable events is thought to rely heavily on the ventromedial prefrontal cortex and the medial temporal lobe, as well as midbrain dopamine neurons in the ventral striatum and nucleus accumbens [133]. Abnormalities in cortical-striatal circuitry may therefore contribute to reduced anticipatory pleasure and reduced pleasure-seeking behavior. Extending this anticipatory pleasure deficit model, we recently proposed that patients not only have reductions in anticipating future pleasure, but also remembering past pleasure [18]. Specifically, we proposed that working memory and long-term memory may be critical in determining the extent to which individuals “over-estimate” future and past pleasure, respectively [18]. There is consistent evidence that healthy individuals typically expect more pleasure in the future and remember more pleasure from the past compared to what they actually experience in the moment [134]. Individuals with schizophrenia do not display this normative tendency for over-estimating past and future positive emotions [28, 135] Over-estimation of pleasure in the future and past is adaptive, as it promotes the initiation of behaviors aimed at obtaining rewards. Based upon the affective science literature, one might expect that the within-subjects comparison of future or past relative to current positive emotion is more critical than the overall mean level of subjective future or past positive emotion alone in determining motivation. As suggested in Strauss and Gold [18], cognitive impairments may influence whether patients display the normative tendency to over-estimate future and past relative to current positive emotion. Additionally, while individuals with schizophrenia do not appear to have a reduced capacity for pleasure, there is consistent evidence that they have elevations in negative emotionality that are associated with poor functional outcome and elevated positive and negative symptoms. There has been recent interest in the link between negative emotion and anhedonia, and it has been found that increases in state and trait negative emotion are linked to difficulty down-regulating the neural response to unpleasant stimuli when patients attempt to apply various emotion regulation strategies (e.g., reappraisal) [137]. Much like the components of reward processing reviewed here, these abnormalities in regulating negative emotion appear to involve impairments in prefrontal cognitive control circuitry [136]. Abnormalities in down-regulating negative emotion may result in a chronically elevated negative emotional state, and contribute to anhedonia by limiting the extent to which individuals seek out rewarding activities [137, 138]. Finally, factor-analytic studies typically indicate that anhedonia and avolition travel together [130, 139, 140], potentially signifying overlapping neural substrates as well as cognitive and psychological processes. Reduced goal-directed and pleasure-seeking behavior may in fact go hand-in hand; however, it would be important to determine whether they do indeed have shared or separable mechanisms as these may necessitate different treatments targets.

Finally, deficits in seeking out pleasurable activities may only be one aspect of anhedonia. For example, as proposed by Strauss and Gold [18], Grant et al. [141], and Beck et al. [142], anhedonia in schizophrenia may also reflect a psychological abnormality, which can best be described as “low pleasure beliefs”. For example, many individuals with schizophrenia appear to have the belief that they generally do not experience pleasure or that specific situations are unlikely to be enjoyable. These psychological processes may be critically involved with reductions in pleasure-seeking behavior. For example, if a patient believes that certain activities are not enjoyable (social interactions), then they are unlikely to engage in them regardless of whether their capacity for pleasure is intact. These low-pleasure beliefs may be associated with impairments in reward processing. For example, a patient who is impaired at learning from and integrating positive feedback may not be able to update value representations needed to change beliefs that certain types of experiences are not enjoyable (e.g., social interactions), despite having a normal hedonic reaction when exposed to a potentially pleasurable event. Similarly, patients who have reduced uncertainty-driven exploration may not evaluate a large enough number of response alternatives to determine whether certain experiences could be pleasurable, which would limit their exposure to experiences that could provide evidence contrary to the belief that certain activities are not enjoyable. Thus, there may be several processes that contribute to reductions in pleasure-seeking behavior in schizophrenia, as well as multiple components of anhedonia in addition to the behavioral deficit that was the focus of this chapter.

8 Treatment Implications

Current psychosocial and pharmacological treatments for negative symptoms of schizophrenia have been minimally effective, especially in terms of improving anhedonia and avolition (see [143]). This is likely due in part to the fact that the cognitive, psychological, and neural mechanisms involved with these symptoms have not been well-delineated. The literature on reward in schizophrenia has provided important advances in this regard, lending some hints for how novel behavioral intervention strategies could be developed or adapted to enhance reward-seeking behavior in schizophrenia. For example, it is clear that patients have deficits in generating, updating, and maintaining mental representations of value. To account for these impairments, it may be necessary to incorporate external cues and reinforcers into standard behavioral therapy approaches. A new Cognitive Behavioral Therapy approach developed by Grant, Beck, [144] and colleagues at the University of Pennsylvania has incorporated some of these procedures. For example, this program has therapists adopt an engaging style (e.g., direct and crisp speaking, energetic, commanding, and confident) and aims to reduce patient lapses in engagement by having them engage in activities during the therapy session (e.g., playing cards, listening to music), as well as by using frequent and intense reinforcement of goal-directed behavior (e.g., verbal praise, tokens, stickers). Interestingly, in an extended randomized psychosocial treatment trial, Grant et al. [144] found evidence that this CBT approach significantly improved avolition more so than treatment as usual. These results are very promising, as few interventions have been found to improve negative symptoms. There was no effect on anhedonia in this trial, suggesting that additional methods may be needed to enhance pleasure-seeking behavior. Over the past decade, there have been significant advances in using mobile technology (e.g., smart phones) in the context of psychosocial treatment and these may be useful in delivering cues, reinforcers, and reminders that can enhance behavioral activation. For example, clinicians could set weekly goals with their patients for engaging in rewarding activities and have apps deliver prompts for the patient to initiate these activities and report their feelings while completing them. The “data” resulting from these reports can then be used in the therapy sessions following that week to review how successful the patient was in fulfilling their goals, as well as the diversity of pleasurable activities experienced and the strength of experiences. The mobile technology approach may therefore enable therapists to systematically shape a patient’s frequency of engaging in pleasurable activities, as well as providing a means to prompt patients to think of future experiences and remember recent past pleasurable experiences. With frequent reviewing of such data, and modifying the behavioral activation program to continuously increase the frequency of pleasurable experiences, it may be possible to shift patients’ beliefs that little is enjoyable and that some activities are not worth the effort.