Abstract
Reward learning is a ubiquitous cognitive mechanism guiding adaptive choices and behaviors, and when impaired, can lead to considerable mental health consequences. Reward-related functional neuroimaging studies have begun to implicate networks of brain regions essential for processing various peripheral influences (e.g., risk, subjective preference, delay, social context) involved in the multifaceted reward processing construct. To provide a more complete neurocognitive perspective on reward processing that synthesizes findings across the literature while also appreciating these peripheral influences, we used emerging meta-analytic techniques to elucidate brain regions, and in turn networks, consistently engaged in distinct aspects of reward processing. Using a data-driven, meta-analytic, k-means clustering approach, we dissociated seven meta-analytic groupings (MAGs) of neuroimaging results (i.e., brain activity maps) from 749 experimental contrasts across 176 reward processing studies involving 13,358 healthy participants. We then performed an exploratory functional decoding approach to gain insight into the putative functions associated with each MAG. We identified a seven-MAG clustering solution that represented dissociable patterns of convergent brain activity across reward processing tasks. Additionally, our functional decoding analyses revealed that each of these MAGs mapped onto discrete behavior profiles that suggested specialized roles in predicting value (MAG-1 & MAG-2) and processing a variety of emotional (MAG-3), external (MAG-4 & MAG-5), and internal (MAG-6 & MAG-7) influences across reward processing paradigms. These findings support and extend aspects of well-accepted reward learning theories and highlight large-scale brain network activity associated with distinct aspects of reward processing.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
Reward learning is a fundamental cognitive process vital for adaptive functioning that consequently, when disrupted, has a significant impact on mental health. Widely accepted and commonly adopted neurocognitive models of reward learning propose that value predictions are formed and updated based on dopaminergic reward prediction error (RPE) signals (Glimcher, 2011; Hollerman & Schultz, 1998; Knutson, Fong, Adams, Varner, & Hommer, 2001; Schultz, 1998; Schultz, 2016; Schultz, Dayan, & Montague, 1997). RPE signals represent the discrepancy between expected outcomes and actual outcomes (Rescorla & Wagner, 1972). In support of these models, functional neuroimaging studies have consistently demonstrated striatal and midbrain activity associated with unexpected positive or negative cues/outcomes (i.e., RPE task events) across a variety of paradigms (Cooper, Hollon, Wimmer, & Knutson, 2009; Garrison, Erdeniz, & Done, 2013; Knutson, Bhanji, Cooney, Atlas, & Gotlib, 2008; O’Doherty, Dayan, Friston, Critchley, & Dolan, 2003). As neuroimaging investigations of value prediction have advanced, scientists continue to construct progressively more complex paradigms involving reward contingencies within increasingly externally valid contexts involving additional influences on valuation, and ultimately reward learning (Daw, Gershman, Seymour, Dayan, & Dolan, 2011; Kahnt, 2018; Porcelli & Delgado, 2009; Seaman et al., 2018).
For example, many reward processing tasks involve paradigms that mirror gambling activities played regularly in casinos or watched on game shows, like blackjack (Miedl, Fehr, Meyer, & Herrmann, 2010), wheel of fortune (Ernst et al., 2005), and slot machines (Daw et al., 2006). Many tasks also include psychosocial processes, in which participants make altruistic decisions about how to spend money (Hare, Camerer, Knoepfle, & Rangel, 2010), or choose to reward/punish themselves over others (Kramer, Jansma, Tempelmann, & Munte, 2007). Other tasks provoke preferences about culturally embedded food brands or luxury items, craft pseudostock markets in which participants can make investment decisions and respond to the consequences (Kuhnen & Knutson, 2005), or require participants to move through virtual spaces to collect rewards (Marsh et al., 2010) or escape punishments (Mobbs et al., 2007). Overall, nearly all contemporary reward processing tasks in neuroimaging experiments engage multiple mental operations (Kahnt, 2018), such as working memory (Pochon et al., 2002), rule switching (Remijnse et al., 2009), probability estimation (Hampton, Bossaerts, & O’Doherty, 2006), spatial navigation (Marsh et al., 2010), episodic memory (de Greck et al., 2008), emotion regulation (Delgado, Gillis, & Phelps, 2008), and/or inhibition (Mengotti, Foroni, & Rumiati, 2019).
These research paradigms model a broader array of real-world reward contingencies and have demonstrated that aspects of reward contexts, such as social factors, temporal factors, riskiness, cost or effort required, and the probabilistic nature of contingencies all influence choices and behaviors (Rangel, Camerer, & Montague, 2008; Rushworth & Behrens, 2008; Seaman et al., 2018). Internal states, memories of past experiences, and subjective preferences also influence subjective value and choice decisions (Engelmann & Tamir, 2009; Koeneke, Pedroni, Dieckmann, Bosch, & Jancke, 2008; Lopez-Persem, Domenech, & Pessiglione, 2016). Accordingly, neuroimaging reward processing studies have not only demonstrated RPE-related striatal and midbrain responsivity (Cooper et al., 2009; Garrison et al., 2013; Knutson et al., 2008), but also implicate a wider network of brain activity linked with processing these various peripheral influences (Acikalin, Gorgolewski, & Poldrack, 2017; Behrens, Woolrich, Walton, & Rushworth, 2007; Gläscher, Daw, Dayan, & O’Doherty, 2010; Juechems et al., 2019; Seaman et al., 2018; Sugrue, Corrado, & Newsome, 2004). As such, neurocognitive models of reward learning should endeavor to more fully characterize these additional contextual influences.
Given that reward processing studies use a variety of complex tasks, task modeling approaches, and theoretical frameworks, it has been difficult to synthesize a quantitative summary of consistent associations between cognitive constructs and brain function. Emerging meta-analytic techniques have proven helpful in synthesizing neuroimaging findings across disperse experimental sites and diverse scientific approaches (Bottenhorn et al., 2019; Bzdok et al., 2015; Bzdok et al., 2013a; Bzdok et al., 2013b; Clos, Amunts, Laird, Fox, & Eickhoff, 2013; Eickhoff, Laird, Fox, Bzdok, & Hensel, 2016; Laird et al., 2015; Ray et al., 2015; Riedel et al., 2018). Using these techniques, we aimed to elucidate brain regions, and in turn networks, consistently linked with different aspects of reward processing.
We employed a data-driven, meta-analytic clustering approach to an extensive body of reward processing neuroimaging results archived in the BrainMap database (www.brainmap.org; Fox & Lancaster, 2002; Laird, Lancaster, & Fox, 2005) to characterize meta-analytic groupings (MAGs) of reward processing experiments based on the spatial similarity of brain activity patterns. These groupings were formed under the assumption that spatially similar task-based brain activity patterns can be categorized as functionally similar, while spatially different patterns can be classified as distinct. To provide insight into the mental processes associated with each of these MAGs, we then conducted functional decoding, a data-driven method for inferring mental processes from observed patterns of brain activity.
While previous reward-related meta-analyses have focused on specific aspects of reward processing (e.g., subjective valuation; Bartra, McGuire, & Kable, 2013; Diekhof, Kaps, Falkai, & Gruber, 2012; Sescousse, Caldu, Segura, & Dreher, 2013; van der Laan, de Ridder, Viergever, & Smeets, 2011), the corpus of neuroimaging results included in our meta-analysis adds to prior work by including brain activity coordinates from a diverse range of tasks and experimental contrasts. The relatively liberal inclusion criteria for our corpus provides a more complete and agnostic investigation of the reward processing construct that arguably better captures its multifaceted nature. Further, while existing reward processing meta-analyses manually grouped published neuroimaging results according to reward-related cognitive constructs of interest to characterize associated brain activity patterns (e.g., Liu, Hairston, Schrier, & Fan, 2011), our meta-analytic, data-driven parsing of experimental contrasts allowed for distinct patterns of brain activity to emerge organically and unbiased by prior assumptions. This k-means clustering approach differs from existing meta-analytic strategies as it can identify dissociable patterns of brain activity present within subgroups of studies within the corpus that might otherwise be obscured when focusing only on activity convergence across the entire corpus.
Overall our goals were to (1) provide a data-driven, functionally relevant summary of brain activity patterns commonly observed across diverse reward-related neuroimaging paradigms; (2) perform functional decoding analyses linking these activity patterns with distinct cognitive-behavioral processes; and (3) to integrate summative interpretations of empirically derived meta-analytic results within existing reward processing theories.
Method
To organize reward-related neuroimaging results into subgroups based on the spatial profile of their modeled activity maps, we performed k-means clustering across multiple model orders. We identified viable model orders operationalized as those that maximized between-group differences and minimized within-group differences of between-experiment correlation distributions using four information-theoretic criterion metrics. After comparing and contrasting viable solutions, we ultimately selected the solution with the greatest number of neurocognitively plausible MAGs. Subsequently, convergent brain activity within each grouping of experiments in this selected solution was quantified via separate neuroimaging meta-analyses. Functional decoding assessments using the NeuroSynth database (Neurosynth.org; Yarkoni, Poldrack, Nichols, Van Essen, & Wager, 2011) were then performed on the resulting MAG’s unthresholded activation likelihood estimation (ALE) maps. The following subsections elaborate the methodological details of these steps in our meta-analytic clustering approach.
Identification of included studies
To compile a large corpus of neuroimaging results across reward processing paradigms, we extracted activation coordinates reported in published studies archived in the BrainMap Database as of April 22, 2016, under the paradigm class metadata labels Reward, Delay Discounting, and Gambling (www.brainmap.org; Fox et al., 2005; Fox & Lancaster, 2002; Laird et al., 2011). The vast majority (94.9%) of the identified studies were archived under the Reward label with most Delay Discounting and Gambling studies being additionally archived under Reward. The Reward label denotes that the reported activation coordinates were identified in a task where a stimulus served to reinforce a desired response (e.g., monetary reward after a correct response; www.brainmap.org/taxonomy/paradigms).
We considered only activation coordinates from published neuroimaging studies, among healthy participants, that were reported in standard Talairach (Talairach & Tournoux, 1988) or Montreal Neurological Institute (MNI; Collins, Neelin, Peters, & Evans, 1994) space and derived from whole-brain statistical tests. Brain coordinates derived through behavioral correlations or a priori region of interest (ROI) analyses were excluded. As this meta-analysis aimed to investigate brain activity linked with typical reward processing, coordinates from groups of individuals diagnosed with neuropsychiatric disorders (e.g., addictive disorders) were excluded from the corpus. Each study included provided at least one experimental contrast that statistically identified brain activity associated with a certain task event defined by the original authors (e.g., a brain activity map). These experimental contrasts were summarized and curated in the BrainMap database as a set of brain activity foci/coordinates linked either with phases of the original task (i.e., task response, anticipation of outcome, outcome delivery) or stimuli presented in the task (e.g., positive outcome, negative outcome, high reward, low reward). Foci from experimental contrasts could also reflect brain activity locations linked with more abstract and computationally derived constructs of interest in the original study (e.g., learning rate, subjective value).
Modeled activation (MA) images of experimental contrasts
Reward-related activation coordinates reported in Talairach space were first transformed into MNI space (Lancaster et al., 2007). Subsequently, modeled activation (MA) images for each experimental contrast (2mm3 resolution) were generated by modeling coordinate foci as Gaussian probability distributions to account for spatial uncertainty due to template and between-subject variance (Eickhoff et al., 2009; see Fig. 1, Step 1). This algorithm eliminates the effect of within-experiment number of foci and within-experiment foci proximity by determining the voxel-wise MA values via the maximum probability associated with any one location reported by the experiment instead of taking the union of probabilities associated with all the foci reported by an experiment (Turkeltaub et al., 2012). This ensured that multiple foci from a single experiment did not jointly influence the MA value of a single voxel. Each experimental contrast’s MA image was collapsed to a single dimension vector composed of v voxels, and then each experiment’s (e) vector was concatenated to create an e × v matrix. The Pearson’s product-moment correlation (r) was then computed for each pair of MA maps, resulting in a symmetric e × e cross-correlation matrix reflecting the extent of similarity (correlation) between the activity locations of different experiments while considering the experiments sample size (see Fig. 1, Step 2).
Correlation matrix-based k-means clustering
To group experimental contrasts with similar brain activity patterns, we applied an adapted meta-analytic k-means clustering procedure previously applied to large collections of experimental contrasts (Bottenhorn et al., 2019; see Fig. 1, Step 3). The distribution of experiment-to-experiment distance values, defined as the additive inverse of the Pearson’s correlation coefficient (i.e., 1 − r), was the spatial measure used to determine an experimental contrast’s group assignment. We applied an implementation of k-means clustering in the MATLAB environment (Version 2014b; The MathWorks, Inc., Natick, MA) and classified experiments into k groupings. We generated 14 clustering solutions (k = 2 through k = 15) similar to previous work using k-means clustering tehniques on meta-anayltic functional neuroimaging data (Eickhoff, Bzdok, Laird, Kurth, & Fox, 2012; Zangemeister et al., 2016). K-means clustering is a nonhierarchical clustering approach that segregates observations into a prespecified number of groups (k) (Forgy, 1965; Hartigan & Wong, 1979; Nanetti, Cerliani, Gazzola, Renken, & Keysers, 2009). This is accomplished by randomly selecting a starting centroid for each group and iterativitly reassigning observations to groups such that within-group variance is minimized and between-group variance is maximized. The random selection of an initial grouping centroid and the subsequent reassignment of experiments was repeated 1,000 times to identify viable solutions (Nanetti et al., 2009).
Selection of an optimal number of clusters (k)
We identified viable model orders based on four information-theoretic criterion metrics that characterized the group separation and group stability properties of each model order (Bzdok et al., 2015; Eickhoff et al., 2016; see Fig. 1, Step 4). These metrics have previously been used in coactivation-based parcellation analyses to determine optimal clustering solutions (Bzdok et al., 2013b; Chang, Yarkoni, Khaw, & Sanfey, 2013) for brain regions across task-independent (Bottenhorn et al., 2019; Kahnt, Chang, Park, Heinzle, & Haynes, 2012; Kelly et al., 2010) and task-dependent (Bzdok et al., 2015; Chase et al., 2015a; Clos et al., 2013; Ray et al., 2015) brain states. We compared and contrasted the outcomes from viable clustering solutions and ultimately selected the solution identifying the greatest number of neurocognitively plausible MAGs for presentation in the main text. The outcomes of other viable clustering solutions are presented in the supplemental materials.
Regarding between-group separation, solutions indicating greater separability were desired as they possess a lower likelihood of experimental contrast misclassification into neighboring groupings. Two metrics were used to assess group separation. The first was the average silhouette coefficient (ASC; Kaufman & Rousseeuw, 2009) that quantified a given experiment’s similarity to the others within its designated grouping relative to the experiments in other groupings. This criterion metric identifies viable solutions (e.g., k) as those with significantly higher coefficients than the subsequent solution (k + 1) or as solutions without significantly lower coefficients than the preceding solution (k − 1). The second separability metric was the hierarchy index (HI), which compared the percentage of experiments assigned to a given grouping that were lost from that grouping when moving from the k to the k − 1 solution for all solutions greater than k = 2 (Kahnt et al., 2012). Experiments are generally thought to be lost from a grouping because of incorrect group assignment. Viable solution candidates were defined as those with a percentage of lost experiments below the median value for all possible solutions.
Regarding within-group stability, solutions demonstrating higher stability were desired as high stability indicates consistent and robust grouping. Two metrics were employed to assess group stability. The first was the variation of information (VI) statistic (Meilă, 2007), which described the amount of information lost or gained when moving to a subsequent clustering solution. Viable solutions were defined as those where the VI increased from the k to k + 1 solution (primary criterion) and where the VI decreased from the immediately preceding k − 1 solution to the k solution (secondary criterion; Clos et al., 2013). The second metric used to assess group stability was the minimum number of experimental contrasts consistently assigned to a grouping, relative to the mean number of experimental contrasts consistently assigned to that grouping across the 1,000 iterations (Nickl-Jockschat et al., 2015; Poldrack, 2006). Viable solution candidates were defined as those with a minimum number of consistently assigned experiments that was greater than half the mean number of consistently assigned experiments.
Statistical convergence within meta-analytic groupings
The k-means clustering algorithm assigns all the experimental contrasts within the corpus to separate collections/groupings. We refer to the collections of experimental contrasts in the k-means clustering solutions as meta-analytic groupings (MAGs). After identifying a clustering solution, convergent brain activity maps across all experimental contrasts within each of the k MAGs were produced using a revised activation likelihood estimation (ALE) algorithm implemented in the MATLAB environment (see Fig. 1, Step 5; Eickhoff et al., 2012; Eickhoff et al., 2009; Turkeltaub et al., 2012). Voxel-wise ALE scores were computed as the union of all MA images within each MAG and served as a quantitative representation of convergent activity within a grouping. Each MAG’s distribution of ALE scores was then subjected to significance testing using a null distribution analytically derived from the random spatial association between experiments (Eickhoff et al., 2012). The resulting p values were thresholded at pcluster-level < .05 (family-wise-error corrected for multiple comparisons; cluster-forming threshold pvoxel-level < .001) to identify significant clusters within each MAG. Convergent activity maps were exported to MANGO (http://ric.uthscsa.edu/mango/) and rendered for visualization using Nilearn 0.5.0 (Abraham et al., 2014).
Functional decoding of meta-analytic groupings
To gain insight into cognitive and behavioral processes associated with convergent brain activity captured within each MAG, we performed functional decoding using NeuroVault (Gorgolewski et al., 2015) and NeuroSynth (Yarkoni et al., 2011). NeuroVault is a Web-based repository allowing researchers to store, share, and visualize statistical neuroimaging maps. NeuroVault also facilitates functional decoding by interfacing with NeuroSynth, a platform that produces term-specific meta-analytic maps based on its database containing activation coordinates and automatically extracted terms from over 14,000 published neuroimaging studies (abstracts). Functional decoding was then performed by uploading each MAG’s unthresholded ALE map to NeuroVault, which then correlated these uploaded maps with term-specific meta-analytic maps extracted from NeuroSynth’s database. This yielded a ranked list of maximally related NeuroSynth terms and their respective Pearson correlation values. These correlation values represented how well the spatial distribution of activity associated with each term in the database matched the activity pattern of the uploaded MAG’s ALE map. This functional decoding approach allowed for an objective description of the NeuroSynth terms linked with each MAG, as well as a comparison of our MAGs with the broader neuroimaging literature.
Results
The search criteria yielded a corpus of brain activation foci from a total of 749 experimental contrasts across 176 studies involving 13,358 healthy participants. Published articles with experimental contrasts included in the corpus are listed in Table S1 in the Supplemental Information. Of these 749 contrasts, 711 (94.9%) were archived under BrainMap’s Reward paradigm class label, 42 (5.6%) under the Delay Discounting paradigm class label, and 20 (2.7%) under the Gambling paradigm class label. Almost all studies included in the corpus were also archived under a variety of other meta-data labels (e.g., Task Switching [6.4%], Go/No-Go [2.9%], Visuospatial Attention [2.9%], Wisconsin Card Sorting Test [2.6%], Reasoning/Problem Solving [1.3%]), which is unsurprising as reward processing is a multifaceted construct, connecting elements of sensation, perception, cognitive control, and other mental operations. As studies included in the corpus could be dually archived under multiple paradigm classes, the percentages of corpus experiments in each paradigm class was not expected to sum to 100%. A complete list of the percentage of experiments achieved under each paradigm class is presented in Table S2 in the Supplemental Information.
The foci included in our corpus were from 749 experimental contrasts that probed specific events from a variety of tasks including, but not limited to monetary incentive delay (MID) tasks; gambling simulation tasks (e.g., wheel of fortune, four-arm bandit/slot machine, blackjack tasks); stock market simulation tasks; delay discounting tasks; face, product, or food preference tasks; punishment avoidance tasks (i.e., predator, prey paradigm); reversal learning tasks (e.g., Wisconsin card sort tasks); Iowa gambling tasks; tower of London tasks; go/no-go tasks; and balloon analogue risk tasks (BART). However, not all studies used identical versions of these well-known paradigms and even studies applying the same task often reported brain activity from different experimental contrasts.
Clustering solutions
To identify viable clustering solutions, we quantitatively evaluated solutions k = 2 through k = 15 across four metrics: the average silhouette coefficient (ASC), the hierarchy index (HI), the variation of information (VI) statistic, and a cluster-size metric (see Fig. 2). The ASC metric identified the k = 4 model order as a viable solution, as it had a higher ASC than the subsequent k = 5 model order, but not a lower ASC than the preceding k = 3 model order (see Fig. 2a). Regarding HI, model orders k = 3, 4, 5, 7, 8, and 9 met criterion, having a percentage of lost experiments below the median with the k = 7 model order having the lowest percentage of lost experiments (see Fig. 2b). Regarding VI, the k = 5, 7, and 13 model orders all met primary and secondary metric criteria. Specifically, the subsequent solution and the immediately preceding solution both displayed an increased VI statistic (see Fig. 2C). Regarding the cluster consistency metric, all solutions except k = 14 and k = 15 had a minimum number of consistently assigned experiments that was greater than half the mean number of consistently assigned experiments indicating high cluster consistency (see Fig. 2D).
Based on these metrics, we identified model orders k = 4, 5, and 7 as viable solutions, as they all met criteria for three of the four metrics. Ultimately, we selected the k = 7 solution for presentation in the main text, as it identified the greatest number of neurocognitively plausible MAGs. However, we also performed supplementary analyses and examination of the k = 4 and k = 5 solutions. Clustering (see Figs. S1 and S3) and functional decoding results (Figs. S2 and S4, Tables S4 and S5) for these two other solutions are presented in the Supplemental Information and discussed in relation to the k = 7 outcomes. We suggest that examination and functional decoding of these three viable model orders provides additional information regarding the integration and segregation of functional brain networks and cognitive-behavioral constructs across varying levels of parcellation.
Meta-analytic groupings
As depicted in the experiment (e) × experiment (e) cross-correlation matrix ordered by the seven groupings (see Fig. S5 in the Supplemental Information), MAGs from the k = 7 clustering solution displayed between-MAG differences and within-MAG similarities when considering pairwise experiment–experiment correlation distributions. Further, each of the seven MAGs contained a reasonably high number of experiments and foci from studies involving a large pool of participants (see Table 1). The generated ALE maps identified regions of convergent activity for each of the seven MAGs (pcluster-level < .05; family-wise-error corrected; pvoxel-level < .001; see Fig. 3, Table S3). The order in which the seven derived MAGs were labeled does not reflect any aspect of the k-means clustering analysis.
The ALE map for MAG-1 (see Fig. 3, red) displayed bilateral convergent activity in a large cluster extending through most of the striatum and thalamus and into the midbrain and anterior insula with the center of mass in the ventral striatum (VS). MAG-1 also included smaller clusters in the subgenual anterior cingulate, posterior cingulate cortex (PCC), and ventromedial prefrontal cortex (vmPFC). MAG-2 (Fig. 3, yellow) also exhibited convergent activity in the striatum, thalamus, medial prefrontal cortex (mPFC), cingulate, and anterior insula. Whereas MAG-1 included activity in the VS, vmPFC, anterior cingulate cortex (ACC) and PCC, MAG-2 displayed convergent activity centered on more dorsal regions (see Fig. S6a), specifically in the dorsal striatum (DS), dorsal medial prefrontal cortex (dmPFC), pre-supplementary motor area (pre-SMA), and dorsal medial cingulate cortex (dmCC). MAG-3 (Fig. 3, green) exhibited bilateral convergent activity in the amygdala and hippocampus, as well as in the SMA and left posterior insula. MAG-4 (Fig. 3, pink) displayed convergence in the bilateral insulae and the dmPFC extending into the dorsal ACC. MAG-4 also included smaller bilateral clusters in the posterior thalamus encompassing the epithalamic habenular nuclei and in the lateral PFC (LPFC), PCC, and the bilateral intraparietal sulcus (IPS). MAG-5 (Fig. 3, purple) also displayed convergence in the bilateral IPS extending into the inferior and superior parietal lobules, dmPFC, and the bilateral LPFC including clusters in the dorsal middle frontal gyrus and frontal pole. MAG-6 (Fig. 3, aqua) displayed convergence in the central medial PFC (cmPFC), subgenual ACC, right middle temporal gyrus, bilateral inferior frontal gyrus, precuneus, and bilateral temporoparietal junction (TPJ) extending into the right middle temporal gyrus and left supramarginal gyrus. MAG-7 (Fig. 3, navy) also displayed convergence in the medial frontal cortex, but ventral to that for MAG-6 (Fig. S6b in the Supplemental Information). Specifically, MAG-7 displayed a large cluster extending across the entire medial orbital frontal cortex (OFC) and subgenual ACC in addition to convergence in the right amygdala, right hippocampus, PCC, and left middle frontal gyrus.
Functional decoding
NeuroSynth functional decoding results were exported as term correlation values quantifying the similarity between an input map (i.e., each MAG’s ALE map) and maps associated with the NeuroSynth term. To facilitate interpretation, the top 10 NeuroSynth terms with the highest correlation values for each MAG were used to generate behavior profiles (see Fig. 4, Table 2). Anatomical NeuroSynth terms, such as “insula” or “anterior,” were excluded from these lists. NeuroSynth terms that were near duplicates of those already on the list were condensed into the one term that was highest on the list. For example, the terms “emotional” and “emotions” would be condensed if the term “emotion” was already in the top 10 (see Table 2, Near Duplicates section).
Functional interpretations of MAGs
We provide an interpretation of each MAG’s presumed functional role based on formal decoding outcomes and previous studies involving the brain regions displaying convergent activity. These functional interpretations speak to probabilistic co-occurrence of activity patterns with certain cognitive-behavioral functions across the literature; but they do not define the function of specified brain locations, per se.
MAG-1 displayed convergent activity in the mid-brain, vmPFC, and VS (see Fig. 3, red). The VS and its dopaminergic connections are known to respond to reward cues and unexpected outcomes or reward prediction errors (RPEs; Hollerman & Schultz, 1998; O’Doherty et al., 2003; Parker et al., 2016; Schultz et al., 1997). Functional decoding results indicated that MAG-1 was associated with the terms prediction error, outcome, anticipation, reward, gains, and incentive (see Fig. 4, Table 2). These terms, in conjunction with the convergent activity observed in the VS and vmPFC, suggest that MAG-1 may play a role in processing reward cues, unexpected outcomes, and encoding RPEs.
MAG-2 also showed convergent activity in the striatum and mPFC, albeit in more dorsal regions than those observed for MAG-1. Specifically, MAG-2 displayed activity in the dmPFC/pre-SMA, DS, dmCC, and anterior insula (see Fig. 3, yellow). Behaviorally, MAG-2 was associated with many of the same terms as MAG-1, suggesting RPE encoding and reward cue processing, specifically, anticipation, reward, gains, incentive, incentive delay, monetary, and motivation (see Fig. 4, Table 2). In addition, MAG-2 was also associated with three other terms: losses, behavior, and task. Evidence suggests that more dorsal regions of the striatum, mPFC, cingulate, and the pre-SMA may specialize in processing response–outcome contingencies (Atallah, Lopez-Paniagua, Rudy, & O’Reilly, 2007; McNamee, Liljeholm, Zika, & O’Doherty, 2015; O’Doherty et al., 2004; Parker et al., 2016) and negative outcomes (Bartra et al., 2013; Tops & Boksem, 2012). MAG-2’s brain activity profile and behavioral profile were consistent with this view and suggest involvement in RPE encoding, but perhaps with a specialization for response–outcome contingencies and/or involving negative outcomes.
MAG-3 exhibited convergent activity in the amygdala, hippocampus, left posterior insula, and SMA (see Fig. 3, green). Previous research has implicated these brain regions in the recognition and regulation of emotions (Adolphs, Baron-Cohen, & Tranel, 2002; Grecucci, Giorgetta, Van’t Wout, Bonini, & Sanfey, 2013; Phan et al., 2003; Phan et al., 2004; Schnell, Bluschke, Konradt, & Walter, 2011; Seymour & Dolan, 2008). Our functional decoding results were consistent with these previous findings as MAG-3’s behavior profile included affective terms such as reactivity, emotion, emotional stimuli, anxiety, fearful, and happy (see Fig. 4, Table 2). In conjunction with the brain activity profile, one functional interpretation of MAG-3 is a role in emotional processing.
MAG-4 included convergent activity in the bilateral anterior insulae, dACC, dmPFC, and thalamus with clusters encompassing the epithalamic habenular nuclei (see Fig. 3, pink). The anterior insula, dACC, and thalamus represent core nodes of the salience network (SN) (Seeley et al., 2007), a commonly observed large-scale brain network thought to be involved in detecting salient stimuli and, in turn, adjusting attentional focus by modulating the relative activity level of other networks (Menon & Uddin, 2010; Sridharan, Levitin, & Menon, 2008; Sutherland, McHugh, Pariyadath, & Stein, 2012). MAG-4’s behavior profile included terms indicating a role in generalized “task-on” processes (e.g., cognitive control, decision, task), and both MAG-4 and MAG-5 were jointly associated with terms related to the specific executive function working memory (working memory, maintenance, load; see Fig. 4, Table 2). Further, MAG-4’s behavior profile also included the term stop, suggestive of another executive function—specifically, inhibitory control/response inhibition. MAG-4 convergent activity also encompassed the habenula, an epithalamic nucleus implicated in conflict monitoring, error processing, reward, and consequently decision-making (Baker et al., 2016; Ide & Li, 2011; Kawai, Yamada, Sato, Takada, & Matsumoto, 2015). MAG-4’s pattern of convergent activity resembled previous accounts of human habenula connectivity (Ely, Stern, Kim, Gabbay, & Xu, 2019; Ely et al., 2016), particularly during error processing (Ide & Li, 2011), and thus it is noteworthy that MAG-4’s behavior profile also includes terms related to performance monitoring (conflict, monitoring, cognitive control). In conjunction with the observed convergent activity in the anterior insula, dACC, and habenula, the collection of terms in MAG-4’s behavior profile suggested a role in externally focused attention particularly in the context of executive functions such as working memory, response inhibition, and performance monitoring.
MAG-5 presented convergent activity in the LPFC and parietal cortex (see Fig. 3, purple), core nodes of the executive control network (ECN; Seeley et al., 2007). The ECN is thought to play a prominent role in attention, inhibition, cognitive flexibility, working memory, and planning (Brown, Schmitt, Smith, & Gold, 2019; Niendam et al., 2012) and has also been linked with reward motivated decision-making (Hobkirk, Bell, Utevsky, Huettel, & Meade, 2019; Tanji & Hoshi, 2008). As noted above, MAG-5 and MAG-4 were jointly associated with terms related to working memory (working memory, maintenance, load; see Fig. 4, Table 2), and similar to MAG-4, MAG-5’s behavior profile also included terms that indicated a role in generalized “task-on” processes (executive, attentional, performance). Additionally, MAG-5 was associated with terms indicative of mathematical calculating (calculation, arithmetic). Overall, this collection of terms suggested a role in externally focused attention particularly in the context of higher order mental operations such as mathematical calculation.
MAG-6 displayed convergent activity in the cmPFC, parietal lobe, TPJ, middle temporal gyrus, PCC, and precuneus (see Fig. 3, aqua), regions resembling the default-mode network (DMN). The DMN is a commonly observed collection of brain regions showing decreased activity during overt task performance (Raichle et al., 2001) and increased activity during “rest” (Spreng, 2012; Spreng & Grady, 2009) or internally focused tasks involving theory of mind, autobiographical memory retrieval, or other forms of self-referential thought (Andrews-Hanna, 2012; Mars et al., 2012; Qin & Northoff, 2011; Spreng & Grady, 2009). MAG-6’s behavioral profile was consistent with these functions previously linked to the DMN and included terms suggesting self-referential thought (autobiographical, self-referential, beliefs), mentalizing (beliefs, mental states, mentalizing), and theory of mind (theory mind, social, moral; see Fig. 4, Table 2). The convergent activity in DMN regions, in conjunction with this collection of terms suggested involvement of this MAG in internally-focused thought specifically regarding abstract mental states.
MAG-7 also displayed convergent activity in DMN regions (see Fig. 3, navy), including the medial OFC, ACC, and PCC (Acikalin et al., 2017; Raichle et al., 2001; Spreng, 2012; Spreng & Grady, 2009). However, the convergent activity in the medial OFC and PCC was located more ventrally than that from MAG-6. Additionally, MAG-7 did not display convergent activity in the TPJ or temporal gyrus, as in MAG-6, but instead displayed convergence in the right amygdala. Interestingly, the vmPFC, medial OFC, amygdala, and PCC have been implicated in subjective valuation and have been previously characterized as constituting a subjective valuation network (Bartra et al., 2013; Levy & Glimcher, 2012; Peters & Buchel, 2010). While both MAG-7 and MAG-6 were jointly associated with terms suggesting intrapersonal and interpersonal mentalizing (e.g., autobiographical, social, referential), MAG-7 was also associated with terms related to subjective valuation and arousal (i.e., value, valence, arousal, reward, neutral, and emotion; see Fig. 4, Table 2). Overall, the localization of convergent activity in conjunction with the functional decoding of MAG-7 suggested a role in internally focused thought specifically in the context of subjective valuation.
Discussion
Using a data-driven k-means clustering approach, we classified 749 reward processing experiments into seven meta-analytic groupings (MAGs). We then computed and visualized the convergent brain activity for each MAG in the ALE framework. Our results suggested that the underlying organization of brain activity during reward processing tasks could be dissociated into separate networks, each with distinct functional roles. By performing exploratory functional decoding analyses, we constructed a behavior profile for each MAG comprised of cognitive-behavioral terms that captured common and unique mental operations that may be critically linked with each MAG’s brain activity profile. We note that while our functional decoding results suggest differential functioning across brain activity patterns, they do not definitively prescribe functional specializations to these patterns.
MAG functions within a reward processing framework
To serve as a roadmap for the discussion below, we conceptualized these outcomes in a reward processing heuristic framework. This framework, combining the summative interpretations of all MAGs, captures cognitive mechanisms involved with various potential influences on reward processing present in the complex contexts generated by various reward processing tasks (see Fig. 5). In the more complex situations generated by human neuroimaging paradigms, we propose that value predictions, based on RPE signals generated through stimuli–outcome and response–outcome pairings, are further influenced by external contextual factors (Behrens et al., 2007; Fiorillo, Tobler, & Schultz, 2007; Hariri et al., 2006; Porcelli & Delgado, 2009; Preuschoff & Bossaerts, 2007; Rangel et al., 2008; Seaman et al., 2018), and/or by subjective internal factors (Hetherington, Pirie, & Nabb, 2002; Juechems et al., 2019; Lopez-Persem et al., 2016; Singer, Critchley, & Preuschoff, 2009).
While MAG-1 and MAG-2 identified convergent brain activity in a network of regions implicated in value prediction through RPE signals during classical and instrumental conditioning (an interpretation further supported by quantitatively defined behavior profiles), the other five MAGs possessed behavior profiles suggesting putative roles in processing more peripheral, external, and internal influences present in the complex reward contingencies inherent to more externally valid reward learning circumstances. Based on these results, our heuristic framework proposes that emotional, external, and internal influences in a given context play a role in modulating the valuation of outcomes formed and updated by RPE signals (see Fig. 5). The following subsections describe each MAG’s interpretation in more detail within this conceptualization.
Value prediction through RPE encoding
The most basic elements of value prediction are often studied within conditioning paradigms in which a reward or punishment is paired with a neutral stimulus (O’Doherty, 2004; Porcelli & Delgado, 2009). After this association has been learned, the neutral stimulus acquires a predictive value (Hollerman & Schultz, 1998; Schultz et al., 1997). Neuroimaging findings have provided robust evidence of dopaminergic signals in the striatum encoding RPEs (O’Doherty, 2004; O’Doherty et al., 2003; Porcelli & Delgado, 2009; Wang, Smith, & Delgado, 2016). These signals are thought to form value predictions and learned stimulus–outcome and response–outcome associations (Delgado, Nystrom, Fissell, Noll, & Fiez, 2000; Gottfried, O’Doherty, & Dolan, 2003; Hollerman & Schultz, 1998; Knutson et al., 2001; Schultz, 2016; Zald et al., 2004). Additionally, blood oxygen-level dependent (BOLD) activity in the mPFC, thalamus, midbrain, amygdala, insula, and inferior frontal cortex has been associated with RPE-related task events (Chase et al., 2015b; Fouragnan, Retzler, & Philiastides, 2018; Garrison et al., 2013; Rutledge, Dean, Caplin, & Glimcher, 2010). Both of our MAGs that included striatal activity (i.e., MAG-1, MAG-2; in conjunction with other regions) exhibited behavior profiles consistent with RPE task event processing.
That said, accumulating evidence suggests that there may be dissociable ventral and dorsal RPE-encoding pathways with distinct functional roles (García-García, Zeighami, & Dagher, 2017; O’Doherty et al., 2004; Parker et al., 2016) and that these roles can be mapped to a well-studied computational framework termed the actor-critic model (García-García et al., 2017; Niv, 2009). Specifically, the pathway from the ventral tegmental area (VTA) to the VS to limbic and cortical regions (i.e., the mesocorticolimbic pathway) is thought to use RPEs to update stimuli values, corresponding to the role of the critic in actor-critic models. Whereas the pathway from the substantia nigra (SN) to the DS to the motor and premotor cortex (i.e., the nigrostriatal pathway) is thought to assign this value to actions and be involved in action selection, corresponding to the actor role in actor-critic models (Barto, 1995; García-García et al., 2017).
The distinction between regions displaying convergent activity in MAG-1 and MAG-2 is in line with such actor-critic models distinguishing between ventral and dorsal pathways. MAG-1 displayed convergent activity in the striatum, thalamus, midbrain, amygdala, anterior insula, and ventral aspects of the mPFC and the associated behavioral profile included the terms prediction error, anticipation, reward, outcome, and incentive supporting previous accounts of these regions responding to reward cues and RPEs (Bray & O’Doherty, 2007; Kirsch et al., 2003; McClure, Berns, & Montague, 2003). MAG-2, while also displaying convergent activity in the striatum and mPFC, was localized to more dorsal regions and included activity in the dmCC and pre-SMA (see Fig. S6a in the Supplemental Information). MAG-2 was also associated with terms indicative of RPE encoding, specifically incentive, reward, losses, motivation, and anticipation. However, unlike MAG-1’s behavioral profile, that for MAG-2 included the terms behavior and task suggestive of learning more instrumental, response-outcome contingencies. Evidence suggests that DS function is also linked to RPE encoding and particularly so during instrumental conditioning paradigms (Atallah et al., 2007; Elliott, Newman, Longe, & Deakin, 2004; García-García et al., 2017; Hart, Leung & Balleine, 2014; Haruno et al., 2004; O’Doherty et al., 2004). In these paradigms, outcomes are contingent on the execution of an action. As instrumental contingencies are learned, actions that lead to desired outcomes are more likely to be repeated. Neuroimaging studies investigating this instrumental component of conditioning support the notion that DS activity may reflect dopaminergic signaling underlying response–outcome contingency learning similar to the way the ventral striatum supports stimulus–outcome learning (Atallah et al., 2007; Haruno et al., 2004; O’Doherty et al., 2004). Further, the dmCC and pre-SMA have been shown to regulate performance of instrumental, goal-directed responses (Liljeholm et al., 2015; Mannella, Mirolli, & Baldassarre, 2016; McNamee et al., 2015). Thus, the convergent activity observed in the dmCC, pre-SMA, and DS in MAG-2 corresponds well with regions previously implicated in instrumental conditioning. In addition to positive outcomes, this network is also thought to specialize in processing negative outcomes (Bartra et al., 2013). Overall, our data-driven, meta-analytic groupings support the contemporary notion of functionally dissociable ventral and dorsal RPE encoding networks. Specifically, MAG-1 captured a network including VS and vmPFC activity that previous research has implicated in representing and updating value in stimuli–outcome associations, while MAG-2 captured a network including the DS, dmCC, pre-SMA, and the anterior insula implicated in processing response–outcome contingencies and negative outcomes (Bartra et al., 2013; O’Doherty et al., 2004).
Value modulation by dissociable influences
The behavior profiles associated with MAG-3-7 suggested additional cognitive mechanisms and brain regions linked with reward processing. We propose that in complex reward contexts, valuation may be influenced by emotional factors, probabilistic external factors, and/or other subjective internal factors. The behavior profiles for MAGs 3-7 included terms representing cognitive mechanisms involved in processing a variety of the peripheral aspects of reward contingencies that often influence (augment or discount) valuation. Based on the results of our functional decoding analyses, we interpreted the functional dissociation of MAGs-3-7 to reflect the processing of differential value modulators. Depending on the type of task being employed, value could be modulated by internally focused processes (e.g., emotion, physiological/affective states, episodic memory of prior experiences, and personal beliefs or preferences), and/or externally focused processes (e.g., performance monitoring, executive attention, risk and probability calculation).
Emotional influences on valuation
MAG-3 displayed convergent activity in the amygdala, hippocampus, left posterior insula, and SMA and was associated with affect-related terms such as reactivity, emotion, emotional stimuli, anxiety, fearful, and happy. Reward-related contexts often engender acute emotional states necessitating emotion identification and/or regulation (Quartz, 2009; Seymour & Dolan, 2008). Affective states such as fear, stress, anger, and excitement, and the capacity to regulate these emotions, impact reward processing and goal-directed decision-making (Coricelli, Dolan, & Sirigu, 2007; Lawrence, Allen & Chanen, 2010; Paulus & Yu, 2012). External reward contexts may involve emotional or arousing stimuli (e.g., happy or angry facial expressions; deprecating or praising feedback), which in turn may evoke internal emotional reactions (reactivity, anxiety, fearful, emotion) or require internal processes (emotion regulation). Therefore, in the context of our meta-analytic corpus, it is plausible that convergent activity patterns linked with emotional processing would not discretely cluster into an internal or external subgrouping of studies, but rather may represent a distinct neurocomputational influence on reward processing as observed in MAG-3.
External influences on valuation
Both MAG-4 and MAG-5 exhibited convergent activity in the IPS and LPFC, major hubs of the ECN (Seeley et al., 2007). The ECN is thought to underlie “task-on attention” when tasks involve executive attention, working memory, inhibition, cognitive flexibility, and planning (Brown et al., 2019; Niendam et al., 2012) and has been implicated in reward-motivated cognitive control and decision-making (Hobkirk et al., 2019; Krmpotich et al., 2013; Tanji & Hoshi, 2008). Additionally, the IPS and LPFC have been associated with cue-based attention orienting (Anderson, 2017) and monitoring peripheral aspects of reward tasks such as stimuli presentation patterns and a stimulus’s associated probability or risk levels (Blankenstein, Schreuders, Peper, Crone, & van Duijvenvoorde, 2018; Gläscher et al., 2010; Guo, Zhang, Da, Sheng, & Zhang, 2018; Huettel, Mack, & McCarthy, 2002; Rushworth & Behrens, 2008; Sela, Kilim, & Lavidor, 2012). The correspondence between MAG-4 and MAG-5 and the ECN is intuitive considering the terms associated with these MAGs suggest a role in processing external aspects of the task context (MAG-4: monitoring, difficulty, decision, conflict; MAG-5: performance, calculation, interference, arithmetic), employing executive control (MAG-4: cognitive control, MAG-5: executive, attentional), and maintaining information in working memory (MAG-4 & 5: working memory, maintenance, load).
Notably, MAG-4 also displayed robust convergent activity in core nodes of the SN (Seeley et al., 2007) including the bilateral anterior insulae, dACC, and thalamic nuclei, which were not observed in MAG-5. The SN is another large-scale brain network thought to be involved in detecting salient stimuli and accordingly orienting attentional resources (Menon & Uddin, 2010; Sridharan et al., 2008; Sutherland et al., 2012). Additionally, MAG-4 included two bilateral clusters centered over the habenular nuclei that extended into the thalamus and midbrain. MAG-4’s pattern of convergent activation also resembled previous accounts of human habenula connectivity (Ely et al., 2019; Ely et al., 2016), particularly during error processing (Ide & Li, 2011). The habenula has been implicated in processing negative outcomes and error monitoring (Baker et al., 2016; Flannery et al., 2019; Li et al., 2008; Ullsperger & Cramon, 2003) and is of emerging interest in the context of reward processing and pathologies characterized by disruptions thereof, such as addiction and depression (Batalla et al., 2017; Flannery et al., 2019; Loonen & Ivanova, 2016; Mathis & Kenny, 2018). Thus, we found it particularly noteworthy that MAG-4 was uniquely associated with terms suggesting a potential role in performance monitoring (difficulty, stop, conflict, monitoring, cognitive control). In line with the bottom-up, salience detecting, and attention-orienting roles of the SN, preclinical research has demonstrated a habenula-ACC circuit in which errors are detected in the habenula and maintained in the ACC (Kawai et al., 2015). Then, through interactions with the ACC, the anterior insula is thought to use this information to appropriately mediate dynamic switching between DMN and ECN activity (Kerns et al., 2004; Menon & Uddin, 2010; Sridharan et al., 2008; Sutherland et al., 2012).
Overall, our functional decoding of MAG-4 and MAG-5 suggest that they may be involved in executive functions needed during reward-processing tasks, such as monitoring task performance or task difficulty, assessing associated risk levels, or calculating the probabilistic nature of reward contingencies. Further, the dissociated convergent activity of these two MAGs may highlight discrete bottom-up, cognitive control (MAG-4) and higher order, calculation-based (MAG-5) executive functions during reward processing.
Internal influences on valuation
In addition to external influences, valuation could be influenced by a variety of internal factors specific to an individual’s current hedonic state and past experiences; both of which are modulated by physiological, social, and/or cultural influences (Bartra et al., 2013; McClure et al., 2004; Rangel et al., 2008; Yoder & Decety, 2018). Both MAG-6 and MAG-7 were associated with terms suggesting such an internally oriented focus such that MAG-6’s behavior profile was indicative of self-referential abstract mentalizing (self-referential, mentalizing, mental states, beliefs, moral) and MAG-7’s profile was indicative of using personal preferences and prior experience to form subjective values (value, valence, arousal, autobiographical, emotion, neutral, referential).
Both MAGs were characterized by topologic resemblance to the DMN, which further supported such an interpretation, given the known link between DMN regions and internally focused tasks involving theory of mind, autobiographical memory retrieval, and other forms of self-referential thought (Hamilton, Farmer, Fogelman, & Gotlib, 2015; Mars et al., 2012; Philippi, Tranel, Duff, & Rudrauf, 2015; Qin & Northoff, 2011; Spreng & Grady, 2009). However, despite some similarities, dissociable areas of activity convergence were noteworthy between these MAGs, such that MAG-6 displayed convergence in more dorsal regions of the mPFC and PCC, whereas MAG-7 displayed more ventral convergence in the mPFC, medial OFC, and PCC. Further, MAG-6 included convergent activity in the TPJ, precuneus, lateral inferior frontal gyrus, and middle temporal gyrus that was not seen in MAG-7. Instead, MAG-7 displayed convergent activity in a cluster encompassing the right amygdala and hippocampus.
Regions included specifically in MAG-6 have been broadly associated with tasks evoking theory of mind and self-referential thought (Mars et al., 2012; Qin & Northoff, 2011). Additionally, MAG-7 includes brain regions (vmPFC, medial OFC, amygdala, PCC) that have been implicated in subjective valuation networks defined by previous work (Bartra et al., 2013; Levy & Glimcher, 2012; Peters & Buchel, 2010). For example, Acikalin et al. (2017) demonstrated a meta-analytic, functional overlap between the DMN and what they termed the subjective valuation network (SVN). Key nodes displaying DMN and SVN overlap were also reflected in MAG-7 of our meta-analytic k-means clustering analysis. Overall, our functional decoding of MAG-6 and MAG-7 demonstrated how two aspects of internally focused attention may be functionally dissociated into similar networks that differ in their recruitment of the mPFC (ventral or dorsal) and other regions (right amygdala or TPJ). The dissociated convergent activity across these two MAGs corresponds with previous work exploring the multifaceted functions associated with the DMN (Acikalin et al., 2017; Mars et al., 2012; Qin & Northoff, 2011).
Comprehensive meta-analysis: Integration of understudied functions
The inclusion criteria for our corpus of experiments encompassed a wider range of reward-related tasks than previous meta-analyses have considered. This allowed us to provide a unique meta-analytic account of brain regions that are not as commonly linked to reward processing, but are nonetheless vital for its realization. Previous neuroimaging meta-analyses of reward processing have focused on specific aspects such as food processing (van der Laan et al., 2011) and subjective valuation (Bartra et al., 2013), or specific comparisons such as passive anticipation versus consumption of rewards (Diekhof et al., 2012) and receipt of primary versus secondary rewards (Sescousse et al., 2013). While our sample of reward processing results was constrained to the perhaps disproportionate frequency distribution of certain topics and tasks characterized in the extant neuroimaging literature, our sample did not exclude experimental contrasts based on outcome valance (i.e., positive, negative), outcome type (i.e., monetary, food, verbal), or processing stage (i.e., decision, anticipation, outcome delivery). Additionally, while existing meta-analyses manually classified data sets into aspects of interest to subsequently characterize associated brain activation patterns (e.g., Liu et al., 2011), our meta-analytic, data-driven parsing of experimental contrasts allowed for groups of activity patterns to emerge organically and unbiased by prior assumptions.
Methodological considerations
Our clustering results may be influenced by characteristics of the k-means clustering algorithm, specifically its sensitivity to spherical clusters and its assumption that input data are linearly separable (Bottenhorn et al., 2019). Further, while this approach is, for the most part, data driven, it requires specification of certain parameters. Specifically, we chose 1,000 as the number of iterations performed to ensure that each solution minimized the point-to-centroid distance. This decision was made based on prior work (Bottenhorn et al., 2019; Kanungo et al., 2004). We also selected the additive inverse of the Pearson’s correlation coefficient as the metric for calculating pairwise distances between variables as it has been previously recommended to maximize distances between spatial patterns of activity maps (Laird et al., 2015). Exploring the impact of these parameters on clustering results is beyond the scope of the current work.
While we used metrics to determine viable clustering solutions, the optimal solution is not always an obvious selection. Our metrics identified three viable model orders and we ultimately selected the solution that identified the greatest number of neurocognitively plausible MAGs. However, we also report assessments of the two other viable solutions in the supplemental material and believe that examination of all three viable model orders provides increased transparency regarding the integration and segregation of our data across degrees of meta-analytic, k-means clustering parcellation. Additionally, as with all meta-analyses, our results are potentially influenced by any reporting biases present in the extant literature. Finally, while our functional decoding yielded outcomes reflective of the broader literature, more corpus-specific and domain-specific insights might have been achieved with a functional decoding approach based on annotation of experiments in our corpus (see the Supplemental Information for further discussion on manual corpus-specific versus automated generalized annotation strategies). Approaches allowing for unbiased, objective, and yet detailed annotation of corpus-specific experiments may benefit future research.
Conclusion
We dissociated meta-analytic groupings of reward processing experiments based on the similarity between each experiment’s pattern of activation using a meta-analytic k-means clustering approach. The resulting seven meta-analytic groupings represented patterns of activity consistently occurring across reward processing tasks. Our functional decoding analyses revealed that these dissociable brain activity patterns could be mapped onto discrete mental processes within a heuristic framework of valuation. Specifically, we observed two frontal-striatal pathway MAGs, one displaying convergent activity in a VS–vmPFC network (MAG-1) and another displaying convergent activity in a DS–dmPFC network (MAG-2). While our functional decoding linked both MAGs to anticipating outcomes and encoding RPEs, these MAGs appeared to be consistent with specialized roles described in actor-critic models. Further, we identified five additional MAGs that correspond to recognized limbic (MAG-3), SN (MAG-4), ECN (MAG-5), DMN (MAG-6), and SVN (MAG-7) activity patterns. Functional decoding linked these MAGs to processing distinct emotional, internal, and external influences present across complex reward contexts. These findings demonstrate the extensive variety of activity patterns involved in aspects of valuation and highlight the role of commonly observed large-scale brain networks in certain aspects of reward processing tasks.
References
Abraham, A., Pedregosa, F., Eickenberg, M., Gervais, P., Mueller, A., Kossaifi, J., . . . Varoquaux, G. (2014). Machine learning for neuroimaging with scikit-learn. Frontiers in Neuroinformatics, 8, 14. doi:https://doi.org/10.3389/fninf.2014.00014
Acikalin, M. Y., Gorgolewski, K. J., & Poldrack, R. A. (2017). A coordinate-based meta-analysis of overlaps in regional specialization and functional connectivity across subjective value and default mode networks. Frontiers in Neuroscience, 11, 1. doi:https://doi.org/10.3389/fnins.2017.00001
Adolphs, R., Baron-Cohen, S., & Tranel, D. (2002). Impaired recognition of social emotions following amygdala damage. Journal of Cognitive Neuroscience, 14, 1264–1274. doi:https://doi.org/10.1162/089892902760807258
Anderson, B. A. (2017). Reward processing in the value-driven attention network: Reward signals tracking cue identity and location. Social Cognitive Affective Neuroscience, 12, 461–467. doi:https://doi.org/10.1093/scan/nsw141
Andrews-Hanna, J. R. (2012). The brain’s default network and its adaptive role in internal mentation. Neuroscientist, 18(3), 251–270. doi:https://doi.org/10.1177/1073858411403316
Atallah, H. E., Lopez-Paniagua, D., Rudy, J. W., & O’Reilly, R. C. (2007). Separate neural substrates for skill learning and performance in the ventral and dorsal striatum. Nature Neuroscience, 10, 126–131.
Baker, P. M., Jhou, T., Li, B., Matsumoto, M., Mizumori, S. J., Stephenson-Jones, M., & Vicentic, A. (2016). The lateral habenula circuitry: Reward processing and cognitive control. The Journal of Neuroscience, 36, 11482–11488. doi:https://doi.org/10.1523/JNEUROSCI.2350-16.2016
Barto, A. G. (1995) Adaptive critics and the basal ganglia. In J. C. Houk, J. Davis, & D. Beiser (Eds.), Models of information processing in the basal ganglia ( pp. 215–232). Cambridge, MA: MIT Press.
Bartra, O., McGuire, J. T., & Kable, J. W. (2013). The valuation system: A coordinate-based meta-analysis of bold fmri experiments examining neural correlates of subjective value. NeuroImage, 76, 412–427. doi:https://doi.org/10.1016/j.neuroimage.2013.02.063
Batalla, A., Homberg, J. R., Lipina, T. V., Sescousse, G., Luijten, M., Ivanova, S. A., . . . Loonen, A. J. M. (2017). The role of the habenula in the transition from reward to misery in substance use and mood disorders. Neuroscience and Biobehavioral Reviews, 80, 276–285. doi:https://doi.org/10.1016/j.neubiorev.2017.03.019
Behrens, T. E., Woolrich, M. W., Walton, M. E., & Rushworth, M. F. (2007). Learning the value of information in an uncertain world. Nature Neuroscience, 10, 1214–1221. doi:https://doi.org/10.1038/nn1954
Blankenstein, N. E., Schreuders, E., Peper, J. S., Crone, E. A., & van Duijvenvoorde, A. C. K. (2018). Individual differences in risk-taking tendencies modulate the neural processing of risky and ambiguous decision-making in adolescence. NeuroImage, 172, 663–673. doi:https://doi.org/10.1016/j.neuroimage.2018.01.085
Bottenhorn, K. L., Flannery, J. S., Boeving, E. R., Riedel, M. C., Eickhoff, S. B., Sutherland, M. T., & Laird, A. R. (2019). Cooperating yet distinct brain networks engaged during naturalistic paradigms: A meta-analysis of functional mri results. Network Neuroscience, 3, 27–48. doi:https://doi.org/10.1162/netn_a_00050
Bray, S., & O’Doherty, J. (2007). Neural coding of reward-prediction error signals during classical conditioning with attractive faces. Journal of Neurophysiology, 97, 3036–3045. doi:https://doi.org/10.1152/jn.01211.2006
Brown, C. A., Schmitt, F. A., Smith, C. D., & Gold, B. T. (2019). Distinct patterns of default mode and executive control network circuitry contribute to present and future executive function in older adults. NeuroImage, 195, 320–332. doi:https://doi.org/10.1016/j.neuroimage.2019.03.073
Bzdok, D., Heeger, A., Langner, R., Laird, A. R., Fox, P. T., Palomero-Gallagher, N., . . . Eikhoff, S. B. (2015). Subspecialization in the human posterior medial cortex. NeuroImage, 106, 55–71. doi:https://doi.org/10.1016/j.neuroimage.2014.11.009
Bzdok, D., Laird, A. R., Zilles, K., Fox, P. T., Eickhoff, S. B. (2013a). An investigation of the structural, connectional, and functional subspecialization in the human amygdala. Human Brain Mapping, 34, 3247–3266. doi:https://doi.org/10.1002/hbm.22138
Bzdok, D., Langner, R., Schilbach, L., Jakobs, O., Roski, C., Caspers, S., . . . Eikhoff, S. B. (2013b). Characterization of the temporo-parietal junction by combining data-driven parcellation, complementary connectivity analyses, and functional decoding. NeuroImage, 81, 381–392. doi:https://doi.org/10.1016/j.neuroimage.2013.05.046
Chang, L. J., Yarkoni, T., Khaw, M. W., & Sanfey, A. G. (2013). Decoding the role of the insula in human cognition: Functional parcellation and large-scale reverse inference. Cerebral Cortex, 23, 739–749. doi:https://doi.org/10.1093/cercor/bhs065
Chase, H. W., Clos, M., Dibble, S., Fox, P., Grace, A. A., Phillips, M. L., & Eickhoff, S. B. (2015a). Evidence for an anterior-posterior differentiation in the human hippocampal formation revealed by meta-analytic parcellation of fmri coordinate maps: Focus on the subiculum. NeuroImage, 113, 44–60. doi:https://doi.org/10.1016/j.neuroimage.2015.02.069
Chase, H. W., Kumar, P., Eickhoff, S. B., & Dombrovski, A. Y. (2015b). Reinforcement learning models and their neural correlates: An activation likelihood estimation meta-analysis. Cognitive, Affective, & Behavioral Neuroscience, 15, 435–459. doi:https://doi.org/10.3758/s13415-015-0338-7
Clos, M., Amunts, K., Laird, A. R., Fox, P. T., & Eickhoff, S. B. (2013). Tackling the multifunctional nature of Broca’s region meta-analytically: Co-activation-based parcellation of area 44. NeuroImage, 83, 174–188. doi:https://doi.org/10.1016/j.neuroimage.2013.06.041
Collins, D. L., Neelin, P., Peters, T. M., & Evans, A. C. (1994). Automatic 3D intersubject registration of MR volumetric data in standardized Talairach space. Journal of Computer Assisted Tomography, 18(2), 192–205. doi:https://doi.org/10.1097/00004728-199403000-00005
Cooper, J. C., Hollon, N. G., Wimmer, G. E., & Knutson, B. (2009). Available alternative incentives modulate anticipatory nucleus accumbens activation. Social Cognitive and Affective Neuroscience, 4, 409–416. doi:https://doi.org/10.1093/scan/nsp031
Coricelli, G., Dolan, R. J., & Sirigu, A. (2007). Brain, emotion and decision making: The paradigmatic example of regret. Trends in Cognitive Sciences, 11, 258–265. doi:https://doi.org/10.1016/j.tics.2007.04.003
Daw, N. D., Gershman, S. J., Seymour, B., Dayan, P., & Dolan, R. J. (2011). Model-based influences on humans’ choices and striatal prediction errors. Neuron, 69, 1204–1215. doi:https://doi.org/10.1016/j.neuron.2011.02.027
Daw, N. D., O'Doherty, J. P., Dayan, P., Seymour, B., Dolan, R. J. (2006). Cortical substrates for exploratory decisions in humans. Nature 441:876-879.
de Greck, M., Rotte, M., Paus, R., Moritz, D., Thiemann, R., Proesch, U., . . .Northoff, G. (2008). Is our self based on reward? Self-relatedness recruits neural activity in the reward system. NeuroImage, 39, 2066–2075. doi:https://doi.org/10.1016/j.neuroimage.2007.11.006
Delgado, M. R., Gillis, M. M., & Phelps, E. A. (2008). Regulating the expectation of reward via cognitive strategies. Nature Neuroscience, 11, 880–881.
Delgado, M. R., Nystrom, L. E., Fissell, C., Noll, D. C., & Fiez, J. A. (2000). Tracking the hemodynamic responses to reward and punishment in the striatum. Journal of Neurophysiology, 84, 3072–3077. doi:https://doi.org/10.1152/jn.2000.84.6.3072
Diekhof, E. K., Kaps, L., Falkai, P., & Gruber, O. (2012). The role of the human ventral striatum and the medial orbitofrontal cortex in the representation of reward magnitude—An activation likelihood estimation meta-analysis of neuroimaging studies of passive reward expectancy and outcome processing. Neuropsychologia, 50, 1252–1266. doi:https://doi.org/10.1016/j.neuropsychologia.2012.02.007
Eickhoff, S. B., Bzdok, D., Laird, A. R., Kurth, F., & Fox, P. T. (2012). Activation likelihood estimation meta-analysis revisited. NeuroImage, 59, 2349–2361. doi:https://doi.org/10.1016/j.neuroimage.2011.09.017
Eickhoff, S. B., Laird, A. R., Fox, P. T., Bzdok, D., & Hensel, L. (2016). Functional segregation of the human dorsomedial prefrontal cortex. Cerebral Cortex, 26, 304–321. doi:https://doi.org/10.1093/cercor/bhu250
Eickhoff, S. B., Laird, A. R., Grefkes, C., Wang, L. E., Zilles, K., & Fox, P. T. (2009). Coordinate-based activation likelihood estimation meta-analysis of neuroimaging data: A random-effects approach based on empirical estimates of spatial uncertainty. Human Brain Mapping, 30, 2907–2926. doi:https://doi.org/10.1002/hbm.20718
Elliott, R., Newman, J. L., Longe, O. A., & Deakin, J. W. (2004). Instrumental responding for rewards is associated with enhanced neuronal response in subcortical reward systems. NeuroImage, 21, 984–990. doi:https://doi.org/10.1016/j.neuroimage.2003.10.010
Ely, B. A., Stern, E. R., Kim, J. W., Gabbay, V., & Xu, J. (2019). Detailed mapping of human habenula resting-state functional connectivity. NeuroImage, 200, 621–634. doi:https://doi.org/10.1016/j.neuroimage.2019.06.015
Ely, B. A., Xu, J., Goodman, W. K., Lapidus, K. A., Gabbay, V., & Stern, E. R. (2016). Resting-state functional connectivity of the human habenula in healthy individuals: Associations with subclinical depression. Human Brain Mapping, 37, 2369–2384. doi:https://doi.org/10.1002/hbm.23179
Engelmann, J. B., & Tamir, D. (2009). Individual differences in risk preference predict neural responses during financial decision-making. Brain Research, 1290, 28–51. doi:https://doi.org/10.1016/j.brainres.2009.06.078
Ernst, M., Nelson, E. E., Jazbec, S., McClure, E. B., Monk, C. S., Leibenluft, . . . Pine, D. S. (2005). Amygdala and nucleus accumbens in responses to receipt and omission of gains in adults and adolescents. NeuroImage, 25, 1279–1291. doi:https://doi.org/10.1016/j.neuroimage.2004.12.038
Fiorillo, C. D., Tobler, P. N., & Schultz, W. (2007). Discrete coding of reward probability and uncertainty by dopamine neurons. Science, 299, 1898–1902. doi:https://doi.org/10.1126/science.1077349
Flannery, J., Riedel, M., Poudel, R., Laird, A., Ross, T., Salmeron, B., . . . Sutherland, M. (2019). Habenular and striatal activity during performance feedback is differentially linked with state-like and trait-like aspects of tobacco use disorder. Science Advances, 10. doi:https://doi.org/10.1126/sciadv.aax2084
Forgy, E. W. (1965). Cluster analysis of multivariate data: Efficiency vs interpretability of classifications. Biometrics, 21, 768–769.
Fouragnan, E., Retzler, C., & Philiastides, M. G. (2018). Separate neural representations of prediction error valence and surprise: Evidence from an fmri meta-analysis. Human Brain Mapping, 39(7), 2887–2906. doi:https://doi.org/10.1002/hbm.24047
Fox, P. T., Laird, A. R., Fox, S. P., Fox, P. M., Uecker, A. M., Crank, M., . . . Lancaster, J. L. (2005). Brainmap taxonomy of experimental design: Description and evaluation. Human Brain Mapping, 25, 185–198. doi:https://doi.org/10.1002/hbm.20141
Fox, P. T., Lancaster, J. L. (2002). Mapping context and content: The brainmap model. Nature Reviews Neurosciencem, 3, 319–321. doi:https://doi.org/10.1038/nrn789
García-García, I., Zeighami, Y., & Dagher, A. (2017). Reward prediction errors in drug addiction and Parkinson’s disease: From neurophysiology to neuroimaging. Current Neurology and Neuroscience Reports, 17(6), 46. doi:https://doi.org/10.1007/s11910-017-0755-9
Garrison, J., Erdeniz, B., & Done, J. (2013). Prediction error in reinforcement learning: A meta-analysis of neuroimaging studies. Neuroscience and Biobehavioral Reviews, 37, 1297–1310. doi:https://doi.org/10.1016/j.neubiorev.2013.03.023
Gläscher, J., Daw, N., Dayan, P., & O’Doherty, J. P. (2010). States versus rewards: Dissociable neural prediction error signals underlying model-based and model-free reinforcement learning. Neuron, 66(4), 585–595. doi:https://doi.org/10.1016/j.neuron.2010.04.016
Glimcher, P. W. (2011). Understanding dopamine and reinforcement learning: The dopamine reward prediction error hypothesis. Proceedings of the National Academy of Sciences of the United States of America, 108, 15647–15654. doi:https://doi.org/10.1073/pnas.1014269108
Gottfried, J. A., O’Doherty, J., & Dolan, R. J. (2003). Encoding predictive reward value in human amygdala and orbitofrontal cortex. Science, 301(5636), 1104–1107. doi:https://doi.org/10.1126/science.1087919
Grecucci, A., Giorgetta, C., Van’t Wout, M., Bonini, N., & Sanfey, A. G. (2013). Reappraising the ultimatum: An fMRI study of emotion regulation and decision making. Cerebral Cortex, 23(2), 399–410. doi:https://doi.org/10.1093/cercor/bhs02
Guo, H., Zhang, Z., Da, S., Sheng, X., & Zhang, X. (2018). High-definition transcranial direct current stimulation (HD-tDCS) of left dorsolateral prefrontal cortex affects performance in balloon analogue risk task (BART). Brain and Behavior, 8(2), e00884. doi:https://doi.org/10.1002/brb3.884
Hamilton, J. P., Farmer, M., Fogelman, P., & Gotlib, I. H. (2015). Depressive rumination, the default-mode network, and the dark matter of clinical neuroscience. Biological Psychiatry, 78, 224–230. doi:https://doi.org/10.1016/j.biopsych.2015.02.020
Hampton, A. N., Bossaerts, P., & O’Doherty, J. P. (2006). The role of the ventromedial prefrontal cortex in abstract state-based inference during decision making in humans. The Journal of Neuroscience, 26, 8360–8367. doi:https://doi.org/10.1523/JNEUROSCI.1010-06.2006
Hare, T. A., Camerer, C. F., Knoepfle, D. T., & Rangel, A. (2010). Value computations in ventral medial prefrontal cortex during charitable decision making incorporate input from regions involved in social cognition. The Journal of Neuroscience, 30, 583–590. doi:https://doi.org/10.1523/JNEUROSCI.4089-09.2010
Hariri, A. R., Brown, S. M., Williamson, D. E., Flory, J. D., de Wit, H., & Manuck, S. B. (2006). Preference for immediate over delayed rewards is associated with magnitude of ventral striatal activity. The Journal of Neuroscience, 26, 13213–13217. doi:https://doi.org/10.1523/JNEUROSCI.3446-06.2006
Hart, G., Leung, B. K., & Balleine, B. W. (2014). Dorsal and ventral streams: The distinct role of striatal subregions in the acquisition and performance of goal-directed actions. Neurobiology of Learning and Memory, 108, 104–118. doi:https://doi.org/10.1016/j.nlm.2013.11.003
Hartigan, J. A., & Wong, M. A. (1979). Algorithm as 136: A k-means clustering algorithm. Journal of the Royal Statistical Society, 28, 100–108. doi:https://doi.org/10.2307/2346830
Haruno, M., Kuroda, T., Doya, K., Toyama, K., Kimura, M., Samejima, K., . . . Kawato, M. (2004). A neural correlate of reward-based behavioral learning in caudate nucleus: A functional magnetic resonance imaging study of a stochastic decision task. The Journal of Neuroscience, 24, 1660–1665. doi:https://doi.org/10.1523/JNEUROSCI.3417-03.2004
Hetherington, M. M., Pirie, L. M., & Nabb, S. (2002). Stimulus satiation: Effects of repeated exposure to foods on pleasantness and intake. Appetite, 38(1), 19–28. doi:https://doi.org/10.1006/appe.2001.0442
Hobkirk, A. L., Bell, R. P., Utevsky, A. V., Huettel, S., & Meade, C. S. (2019). Reward and executive control network resting-state functional connectivity is associated with impulsivity during reward-based decision making for cocaine users. Drug and Alcohol Dependence, 194, 32–39. doi:https://doi.org/10.1016/j.drugalcdep.2018.09.013
Hollerman, J. R., & Schultz, W. (1998). Dopamine neurons report an error in the temporal prediction of reward during learning. Nature Neuroscience, 1, 304–309. doi:https://doi.org/10.1038/1124
Huettel, S. A., Mack, P. B., & McCarthy, G. (2002). Perceiving patterns in random series: Dynamic processing of sequence in prefrontal cortex. Nature Neuroscience, 5(5), 485–490. doi:https://doi.org/10.1038/nn841
Ide, J. S., & Li, C. S. (2011). Error-related functional connectivity of the habenula in humans. Frontiers in Human Neuroscience, 5, 25. doi:https://doi.org/10.3389/fnhum.2011.00025
Juechems, K., Balaguer, J., Castañón, S. H., Ruz, M., O’Reilly, J. X., & Summerfield, C. (2019). A network for computing value equilibrium in the human medial prefrontal cortex. Neuron, 101, 977–987. doi:https://doi.org/10.1016/j.neuron.2018.12.029
Kahnt, T. (2018). A decade of decoding reward-related fmri signals and where we go from here. NeuroImage, 180, 324–333. doi:https://doi.org/10.1016/j.neuroimage.2017.03.067
Kahnt, T., Chang, L. J., Park, S. Q., Heinzle, J., & Haynes, J. D. (2012). Connectivity-based parcellation of the human orbitofrontal cortex. The Journal of Neuroscience, 32(18), 6240–6250. doi:https://doi.org/10.1523/JNEUROSCI.0257-12.2012
Kanungo, T., Mount, D. M., Netanyahu, N. S., Piatko, C. D., Silverman, R., & Wu, A. Y. (2004). A local search approximation algorithm for k-means clustering. Computational Geometry, 28, 89–112. doi:https://doi.org/10.1016/j.comgeo.2004.03.003
Kaufman, L., & Rousseeuw, P. J. (2009) Finding groups in data: An introduction to cluster analysis. Hoboken, NJ: John Wiley & Sons. doi:https://doi.org/10.1002/9780470316801
Kawai, T., Yamada, H., Sato, N., Takada, M., & Matsumoto, M. (2015). Roles of the lateral habenula and anterior cingulate cortex in negative outcome monitoring and behavioral adjustment in nonhuman primates. Neuron, 88(4), 792–804. doi:https://doi.org/10.1016/j.neuron.2015.09.030
Kelly, A. K., McGee, M., Crews, D. H., Sweeney, T., Boland, T. M., & Kenny, D. A. (2010). Repeatability of feed efficiency, carcass ultrasound, feeding behavior, and blood metabolic variables in finishing heifers divergently selected for residual feed intake. Journal of Animal Science, 88(10), 3214–3225. doi:https://doi.org/10.2527/jas.2009-2700
Kerns, J., Cohen, J., MacDonald, A., Cho, R., Stenger, V., & Carter, C. (2004). Anterior cingulate conflict monitoring and adjustments in control. Science, 303(5660), 1023–1026. doi:https://doi.org/10.1126/science.1089910
Kirsch, P., Schienle, A., Stark, R., Sammer, G., Blecker, C., Walter, B., Ott, U., . . . Vaitl, D. (2003). Anticipation of reward in a nonaversive differential conditioning paradigm and the brain reward system: An event-related fMRI study. NeuroImage, 20(2), 1086–1095. doi:https://doi.org/10.1016/S1053-8119(03)00381-1
Knutson, B., Bhanji, J. P., Cooney, R. E., Atlas, L. Y., & Gotlib, I. H. (2008). Neural responses to monetary incentives in major depression. Biological Psychiatry, 63(7), 686–692. doi:https://doi.org/10.1016/j.biopsych.2007.07.023
Knutson, B., Fong, G. W., Adams, C. M., Varner, J. L., & Hommer, D. (2001). Dissociation of reward anticipation and outcome with event-related fMRI. NeuroReport, 12, 3683–3687. doi:https://doi.org/10.1097/00001756-200112040-00016
Koeneke, S., Pedroni, A. F., Dieckmann, A., Bosch, V., & Jancke, L. (2008). Individual preferences modulate incentive values: Evidence from functional MRI. Behavioral and Brain Functions, 4, 55. doi:https://doi.org/10.1186/1744-9081-4-55
Kramer, U. M., Jansma, H., Tempelmann, C., & Munte, T. F. (2007). Tit-for-tat: The neural basis of reactive aggression. NeuroImage, 38(1), 203–211. doi:https://doi.org/10.1016/j.neuroimage.2007.07.029
Krmpotich, T. D., Tregellas, J. R., Thompson, L. L., Banich, M. T., Klenk, A. M., & Tanabe, J. L. (2013). Resting-state activity in the left executive control network is associated with behavioral approach and is increased in substance dependence. Drug and Alcohol Dependence, 129(1/2), 1–7. doi:https://doi.org/10.1016/j.drugalcdep.2013.01.021
Kuhnen, C. M., & Knutson, B. (2005). The neural basis of financial risk taking. Neuron, 47(5), 763–770. doi:https://doi.org/10.1016/j.neuron.2005.08.008
Laird, A. R., Eickhoff, S. B., Fox, P. M., Uecker, A. M., Ray, K. L., Saenz, J. J., . . . Fox, P. T. (2011). The BrainMap strategy for standardization, sharing, and meta-analysis of neuroimaging data. Biomedical Central Research Notes, 349, 1–9. doi:https://doi.org/10.1186/1756-0500-4-349
Laird, A. R., Lancaster, J. L., & Fox, P. T. (2005). BrainMap the social evolution of a human brain mapping database. Neuroinformatics, 3, 65–78. doi:https://doi.org/10.1385/ni:3:1:065
Laird, A. R., Riedel, M. C., Sutherland, M. T., Eickhoff, S. B., Ray, K. L., Uecker, A. M., . . . Fox, P. T. (2015). Neural architecture underlying classification of face perception paradigms. NeuroImage, 119, 70–80. doi:https://doi.org/10.1016/j.neuroimage.2015.06.044
Lancaster, J. L., Tordesillas-Gutierrez, D., Martinez, M., Salinas, F., Evans, A., Zilles, K., . . . Fox, P. T. (2007). Bias between MNI and Talairach coordinates analyzed using the ICBM-152 brain template. Human Brain Mapping, 28, 1194–1205. doi:https://doi.org/10.1002/hbm.20345
Lawrence, K. A., Allen, J. S., & Chanen, A. M. (2010). Impulsivity in borderline personality disorder: Reward-based decision-making and its relationship to emotional distress. Journal of Personality Disorders, 24, 785–799. doi:https://doi.org/10.1521/pedi.2010.24.6.785
Levy, D. J., & Glimcher, P. W. (2012). The root of all value: A neural common currency for choice. Current Opinion in Neurobiology, 22, 1027–1038. doi:https://doi.org/10.1016/j.conb.2012.06.001
Li, C. S., Yan, P., Chao, H. H., Sinha, R., Paliwal, P., Constable, R. T., . . . Lee, T. W. (2008). Error-specific medial cortical and subcortical activity during the stop signal task: A functional magnetic resonance imaging study. Neuroscience, 155, 1142–1151. doi:https://doi.org/10.1016/j.neuroscience.2008.06.062
Liljeholm, M., Dunne, S., & O’Doherty, J. P. (2015). Differentiating neural systems mediating the acquisition vs. expression of goal-directed and habitual behavioral control. The European Journal of Neuroscience, 41, 1358–1371. doi:https://doi.org/10.1111/ejn.12897
Liu, X., Hairston, J., Schrier, M., & Fan, J. (2011). Common and distinct networks underlying reward valence and processing stages: A meta-analysis of functional neuroimaging studies. Neuroscience & Biobehavioral Reviews, 35, 1219–1236. doi:https://doi.org/10.1016/j.neubiorev.2010.12.012
Loonen, A. J., & Ivanova, S. A. (2016). Circuits regulating pleasure and happiness-mechanisms of depression. Frontiers in Human Neuroscience, 10, 571. doi:https://doi.org/10.3389/fnhum.2016.00571
Lopez-Persem, A., Domenech, P., & Pessiglione, M. (2016). How prior preferences determine decision-making frames and biases in the human brain. Elife, 5, e20317. doi:https://doi.org/10.7554/eLife.20317
Mannella, F., Mirolli, M., & Baldassarre, G. (2016). Goal-directed behavior and instrumental devaluation: A neural system-level computational model. Frontiers in Behavioral Neuroscience, 10, 181. doi:https://doi.org/10.3389/fnbeh.2016.00181
Mars, R. B., Neubert, F. X., Noonan, M. P., Sallet, J., Toni, I., & Rushworth, M. F. (2012). On the relationship between the “default mode network” and the “social brain”. Frontiers in Human Neuroscience, 6, 189. doi:https://doi.org/10.3389/fnhum.2012.00189
Marsh, R., Hao, X., Xu, D., Wang, Z., Duan, Y., Liu, J., . . . Peterson, B. S. (2010). A virtual reality-based fmri study of reward-based spatial learning. Neuropsychologia, 48, 2912–2921. doi:https://doi.org/10.1016/j.neuropsychologia.2010.05.033
Mathis, V., & Kenny, P. J. (2018). From controlled to compulsive drug-taking: The role of the habenula in addiction. Neuroscience & Biobehavioral Reviews, 106, 102–111. doi:https://doi.org/10.1016/j.neubiorev.2018.06.018
McClure, S. M., Berns, G. S., & Montague, P. R. (2003). Temporal prediction errors in a passive learning task activate human striatum. Neuron, 38, 339–346. doi:https://doi.org/10.1016/s0896-6273(03)00154-5
McClure, S. M., Li, J., Tomlin, D., Cypert, K. S., Montague, L. M., & Montague, P. R. (2004). Neural correlates of behavioral preference for culturally familiar drinks. Neuron, 44, 379–387. doi:https://doi.org/10.1016/j.neuron.2004.09.019
McNamee, D., Liljeholm, M., Zika, O., & O’Doherty, J. P. (2015). Characterizing the associative content of brain structures involved in habitual and goal-directed actions in humans: A multivariate fMRI study. The Journal of Neuroscience, 35, 3764–3771. doi:https://doi.org/10.1523/JNEUROSCI.4677-14.2015
Meilă, M. (2007). Comparing clusterings—An information based distance. Journal of Multivariate Analysis, 98, 873–895. doi:https://doi.org/10.1016/j.jmva.2006.11.013
Mengotti, P., Foroni, F., & Rumiati, R. I. (2019). Neural correlates of the energetic value of food during visual processing and response inhibition. NeuroImage, 184, 130–139. doi:https://doi.org/10.1016/j.neuroimage.2018.09.017
Menon, V., & Uddin, L. Q. (2010). Saliency, switching, attention and control: A network model of insula function. Brain Structure and Function, 214, 655–667. doi:https://doi.org/10.1007/s00429-010-0262-0
Miedl, S. F., Fehr, T., Meyer, G., & Herrmann, M. (2010). Neurobiological correlates of problem gambling in a quasi-realistic blackjack scenario as revealed by fMRI. Psychiatry Research, 181, 165–173. doi:https://doi.org/10.1016/j.pscychresns.2009.11.008
Mobbs, D., Petrovic, P., Marchant, J. L., Hassabis, D., Weiskopf, N., Seymour, B., . . . Frith, C. D. (2007). When fear is near: Threat imminence elicits prefrontal–periaqueductal gray shifts in humans. Science, 317(5841), 1079–1083. doi:https://doi.org/10.1126/science.1144298
Nanetti, L., Cerliani, L., Gazzola, V., Renken, R., & Keysers, C. (2009). Group analyses of connectivity-based cortical parcellation using repeated k-means clustering. NeuroImage, 47, 1666–1677. doi:https://doi.org/10.1016/j.neuroimage.2009.06.014
Nickl-Jockschat, T., Rottschy, C., Thommes, J., Schneider, F., Laird, A. R., Fox, P. T., & Eickhoff, S. B. (2015). Neural networks related to dysfunctional face processing in autism spectrum disorder. Brain Structure and Function, 220, 2355–2371. doi:https://doi.org/10.1007/s00429-014-0791-z
Niendam, T. A., Laird, A. R., Ray, K. L., Dean, Y. M., Glahn, D. C., & Carter, C. S. (2012). Meta-analytic evidence for a superordinate cognitive control network subserving diverse executive functions. Cognitive, Affective, & Behavioral Neuroscience, 12, 241–268. doi:https://doi.org/10.3758/s13415-011-0083-5
Niv, Y. (2009). Reinforcement learning in the brain. Journal of Mathematical Psychology, 53, 139–154. doi:https://doi.org/10.1016/j.jmp.2008.12.005
O’Doherty, J. P. (2004). Reward representations and reward-related learning in the human brain: Insights from neuroimaging. Current Opinions in Neurobiology, 14, 769–776. doi:https://doi.org/10.1016/j.conb.2004.10.016
O’Doherty, J., Dayan, P., Friston, K., Critchley, H., & Dolan, R. J. (2003). Temporal difference models and reward-related learning in the human brain. Neuron, 28, 329–337. doi:https://doi.org/10.1016/s0896-6273(03)00169-7
O’Doherty, J., Dayan, P., Schultz, J., Deichmann, R., Friston, K., & Dolan, R. J. (2004). Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science, 304, 452–454. doi:https://doi.org/10.1126/science.1094285
Parker, N. F., Cameron, C. M., Taliaferro, J. P., Lee, J., Choi, J. Y., Davidson, T. J., . . . Witten, I. B. (2016). Reward and choice encoding in terminals of midbrain dopamine neurons depends on striatal target. Nature Neuroscience, 19, 845–854. doi:https://doi.org/10.1038/nn.4287
Paulus, M. P., & Yu, A. J. (2012). Emotion and decision-making: Affect-driven belief systems in anxiety and depression. Trends in Cognitive Science, 16, 476–483. doi:https://doi.org/10.1016/j.tics.2012.07.009
Peters, J., & Buchel, C. (2010). Neural representations of subjective reward value. Behavioural Brain Research, 213, 135–141. doi:https://doi.org/10.1016/j.bbr.2010.04.031
Phan, K. L., Taylor, S. F., Welsh, R. C., Decker, L. R., Noll, D. C., Nichols, T. E., . . . Liberzon, I. (2003). Activation of the medial prefrontal cortex and extended amygdala by individual ratings of emotional arousal: A fMRI study. Biological Psychiatry, 53, 211–215. doi:https://doi.org/10.1016/s0006-3223(02)01485-3
Phan, K. L., Taylor, S. F., Welsh, R. C., Ho, S. H., Britton, J. C., & Liberzon, I. (2004). Neural correlates of individual ratings of emotional salience: A trial-related fmri study. NeuroImage, 21, 768–780. doi:https://doi.org/10.1016/j.neuroimage.2003.09.072
Philippi, C. L., Tranel, D., Duff, M., & Rudrauf, D. (2015). Damage to the default mode network disrupts autobiographical memory retrieval. Social Cognitive and Affective Neuroscience, 10, 318–326. doi:https://doi.org/10.1093/scan/nsu070
Pochon, J. B., Levy, R., Fossati, P., Lehericy, S., Poline, J. B., Pillon, B., . . . Dubois, B. (2002). The neural system that bridges reward and cognition in humans: An fMRI study. Proceedings of the National Academy of Sciences of the United States of America, 8, 5669–5674. doi:https://doi.org/10.1073/pnas.082111099
Poldrack, R. A. (2006). Can cognitive processes be inferred from neuroimaging data? Trends in Cognitive Sciences, 10, 59–63. doi:https://doi.org/10.1016/j.tics.2005.12.004
Porcelli, A. J., & Delgado, M. R. (2009). Reward processing in the human brain: Insights from fMRI. In J.-C. Dreher & L. Tremblay (Eds.), Handbook of reward and decision making (pp. 165–184). New York, NY: Academic Press. doi:https://doi.org/10.1016/B978-0-12-374620-7.00007-8
Preuschoff, K., & Bossaerts, P. (2007). Adding prediction risk to the theory of reward learning. Annals of the New York Academy of Sciences, 1104, 135–146. doi:https://doi.org/10.1196/annals.1390.005
Qin, P., & Northoff, G. (2011). How is our self related to midline regions and the default-mode network? NeuroImage, 57, 1221–1233.
Quartz, S. R. (2009). Reason, emotion and decision-making: Risk and reward computation with feeling. Trends in Cognitive Sciences, 13, 209–215. doi:https://doi.org/10.1016/j.tics.2009.02.003
Raichle, M. E., MacLeod, A. M., Snyder, A. Z., Powers, W. J., Gusnard, D. A., & Shulman, G. L. (2001). A default mode of brain function. Proceedings of the National Academy of Sciences of the United States of America, 98, 676–682. doi:https://doi.org/10.1073/pnas.98.2.676
Rangel, A., Camerer, C., & Montague, P. R. (2008). Neuroeconomics: The neurobiology of value-based decision-making. Nature Reviews Neuroscience, 9, 545. doi:https://doi.org/10.1038/nrn2357
Ray, K. L., Zald, D. H., Bludau, S., Riedel, M. C., Bzdok, D., Yanes, J., . . . Laird, A. R. (2015). Co-activation based parcellation of the human frontal pole. NeuroImage, 123, 200–211. doi:https://doi.org/10.1016/j.neuroimage.2015.07.072
Remijnse, P. L., Nielen, M. M., van Balkom, A. J., Hendriks, G. J., Hoogendijk, W. J., Uylings, H. B., & Veltman, D. J. (2009). Differential frontal-striatal and paralimbic activity during reversal learning in major depressive disorder and obsessive-compulsive disorder. Psychological Medicine, 39, 1503–1518. doi:https://doi.org/10.1017/S0033291708005072
Rescorla, R. A., & Wagner, A. R. (1972) A theory of pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. In W. Black & A. H. B. W. F. Prokosy (Eds.), Classical conditioning II: Current theory and research (pp. 64–99). New York, NY: Appleton-Century-Crofts.
Riedel, M. C., Yanes, J. A., Ray, K. L., Eickhoff, S. B., Fox, P. T., Sutherland, M. T., & Laird, A. R. (2018). Dissociable meta-analytic brain networks contribute to coordinated emotional processing. Human Brain Mapping, 39, 2514–2531. doi:https://doi.org/10.1002/hbm.24018
Rushworth, M. F., & Behrens, T. E. (2008). Choice, uncertainty and value in prefrontal and cingulate cortex. Nature Neuroscience, 11, 389–397. doi:https://doi.org/10.1038/nn2066
Rutledge, R. B., Dean, M., Caplin, A., & Glimcher, P. W. (2010). Testing the reward prediction error hypothesis with an axiomatic model. The Journal of Neuroscience, 30, 13525–13536. doi:https://doi.org/10.1523/JNEUROSCI.1747-10.2010
Schnell, K., Bluschke, S., Konradt, B., & Walter, H. (2011). Functional relations of empathy and mentalizing: An fMRI study on the neural basis of cognitive empathy. NeuroImage, 54, 1743–1754. doi:https://doi.org/10.1016/j.neuroimage.2010.08.024
Schultz, W. (1998). Predictive reward signal of dopamine neurons. Journal of Neurophysiology, 80, 1–27. doi:https://doi.org/10.1152/jn.1998.80.1.1
Schultz, W. (2016). Reward functions of the basal ganglia. Journal of Neural Transmission (Vienna), 123, 679–693. doi:https://doi.org/10.1007/s00702-016-1510-0
Schultz, W., Dayan, P., & Montague, P. R. (1997). A neural substrate of prediction and reward. Science, 275, 1593–1599. doi:https://doi.org/10.1126/science.275.5306.1593
Seaman, K. L., Brooks, N., Karrer, T. M., Castrellon, J. J., Perkins, S. F., Dang, L. C., . . . Samanez-Larkin, G. R. (2018). Subjective value representations during effort, probability and time discounting across adulthood. Social Cognitive and Affective Neuroscience, 13, 449–459. doi:https://doi.org/10.1093/scan/nsy021
Seeley, W. W., Menon, V., Schatzberg, A. F., Keller, J., Glover, G. H., Kenna, H., . . . Greicius, M. D. (2007). Dissociable intrinsic connectivity networks for salience processing and executive control. The Journal of Neuroscience, 27, 2349–2356. doi:https://doi.org/10.1523/JNEUROSCI.5587-06.2007
Sela, T., Kilim, A., & Lavidor, M. (2012). Transcranial alternating current stimulation increases risk-taking behavior in the balloon analog risk task. Frontiers in Neuroscience, 6, 22. doi:https://doi.org/10.3389/fnins.2012.00022
Sescousse, G., Caldu, X., Segura, B., & Dreher, J. C. (2013). Processing of primary and secondary rewards: A quantitative meta-analysis and review of human functional neuroimaging studies. Neuroscience and Biobehavioral Reviews, 37, 681–696. doi:https://doi.org/10.1016/j.neubiorev.2013.02.002
Seymour, B., & Dolan, R. (2008). Emotion, decision making, and the amygdala. Neuron, 58, 662–671. doi:https://doi.org/10.1016/j.neuron.2008.05.020
Singer, T., Critchley, H. D., & Preuschoff, K. (2009). A common role of insula in feelings, empathy and uncertainty. Trends in Cognitive Sciences, 13, 334–340. doi:https://doi.org/10.1016/j.tics.2009.05.001
Spreng, R. N. (2012). The fallacy of a “task-negative” network. Frontiers in Psychology, 3, 145. doi:https://doi.org/10.3389/fpsyg.2012.00145
Spreng, R. N., Grady, C. L. (2009). Patterns of brain activity supporting autobiographical memory, prospection, and theory of mind, and their relationship to the default mode network. Journal of Cognitive Neuroscience, 22, 1112–1123. doi:https://doi.org/10.1162/jocn.2009.21282
Sridharan, D., Levitin, D. J., & Menon, V. (2008). A critical role for the right fronto-insular cortex in switching between central-executive and default-mode networks. Proceedings of the National Academy of Sciences of the United States of America, 105, 12569–12574. doi:https://doi.org/10.1073/pnas.0800005105
Sugrue, L. P., Corrado, G. S., & Newsome, W. T. (2004). Matching behavior and the representation of value in the parietal cortex. Science, 304, 1782–1787. doi:https://doi.org/10.1126/science.1094765
Sutherland, M. T., McHugh, M. J., Pariyadath, V., & Stein, E. A. (2012). Resting state functional connectivity in addiction: Lessons learned and a road ahead. NeuroImage, 62, 2281–2295. doi:https://doi.org/10.1016/j.neuroimage.2012.01.117
Talairach, J., & Tournoux, P. (1988). Co-planar stereotaxic atlas of the human brain: 3-dimensional proportional system: An approach to cerebral imaging. New York, NY: Thieme.
Tanji, J., & Hoshi, E. (2008). Role of the lateral prefrontal cortex in executive behavioral control. Physiological Reviews, 88, 37–57. doi:https://doi.org/10.1152/physrev.00014.2007
Tops, M., & Boksem, M. A. (2012). “What’s that?” “What went wrong?” Positive and negative surprise and the rostral-ventral to caudal-dorsal functional gradient in the brain. Frontiers in Psychology, 3, 21. doi:https://doi.org/10.3389/fpsyg.2012.00021
Turkeltaub, P. E., Eickhoff, S. B., Laird, A. R., Fox, M., Wiener, M., & Fox, P. (2012). Minimizing within-experiment and within-group effects in activation likelihood estimation meta-analyses. Human Brain Mapping, 33, 1–3. doi:https://doi.org/10.1002/hbm.21186
Ullsperger, M., & Cramon, D. Y. V. (2003). Error monitoring using external feedback: Specific roles of the habenular complex, the reward system, and the cingulate motor area revealed by functional magnetic resonance imaging. The Journal of Neuroscience, 23, 4308–4314. doi:https://doi.org/10.1523/JNEUROSCI.23-10-04308.2003
van der Laan, L. N., de Ridder, D. T., Viergever, M. A., & Smeets, P. A. (2011). The first taste is always with the eyes: A meta-analysis on the neural correlates of processing visual food cues. NeuroImage, 55, 296–303. doi:https://doi.org/10.1016/j.neuroimage.2010.11.055
Wang, K. S., Smith, D. V., & Delgado, M. R. (2016). Using fMRI to study reward processing in humans: Past, present, and future. Journal of Neurophysiology, 115, 1664–1678. doi:https://doi.org/10.1152/jn.00333.2015
Yarkoni, T., Poldrack, R. A., Nichols, T. E., Van Essen, D. C., & Wager, T. D. (2011). Large-scale automated synthesis of human functional neuroimaging data. Nature Methods, 8, 665. doi:https://doi.org/10.1038/nmeth.1635
Yoder, K. J., & Decety, J. (2018). The neuroscience of morality and social decision-making. Psychology, Crime & Law, 24, 279–295. doi:https://doi.org/10.1080/1068316X.2017.1414817
Zald, D. H., Boileau, I., El-Dearedy, W., Gunn, R., McGlone, F., Dichter, G. S., & Dagher, A. (2004). Dopamine transmission in the human striatum during monetary reward tasks. The Journal of Neuroscience, 24, 4105–4112. doi:https://doi.org/10.1523/JNEUROSCI.4643-03.2004
Zangemeister, L., Grabenhorst, F., & Schultz, W. (2016). Neural basis for economic saving strategies in human amygdala-prefrontal reward circuits. Current Biology, 26, 3004–3013. doi:https://doi.org/10.1016/j.cub.2016.09.016
Acknowledgements
Primary support for this project was provided by NIH R01DA041353 (M.T.S., A.R.L., M.C.R., R.P.) and NIH U54MD012393 (J.S.F., M.T.S.). Contributions from coauthors were also provided with support from NSF 1631325 (A.R.L., M.C.R., T.S.), NIH K01DA037819 (M.T.S.), NIH U01DA041156 (A.R.L., M.T.S., M.C.R., K.L.B., L.D.H.B.), NSF REAL DRL-1420627 (A.R.L.), and NSF CNS 1532061 (A.R.L.). We thank the FIU Instructional & Research Computing Center (IRCC, http://ircc.fiu.edu) for providing access to the HPC computing resources that contributed to the generation of the research results reported within this paper.
Author note
This manuscript’s content is original research. All authors have contributed substantially to this manuscript. There are no financial or other relations associated with this study that represent a conflict of interest.
Open Practices Statement
The authors have released all code and data associated with this manuscript. The code and tabular data are available on GitHub (https://github.com/NBCLab/reward-processing-meta-analysis), and the unthresholded maps of each MAG are available on NeuroVault (https://neurovault.org/collections/5070/).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic supplementary material
ESM 1
(PDF 9485 kb)
Rights and permissions
About this article
Cite this article
Flannery, J.S., Riedel, M.C., Bottenhorn, K.L. et al. Meta-analytic clustering dissociates brain activity and behavior profiles across reward processing paradigms. Cogn Affect Behav Neurosci 20, 215–235 (2020). https://doi.org/10.3758/s13415-019-00763-7
Published:
Issue Date:
DOI: https://doi.org/10.3758/s13415-019-00763-7