Keywords

Introduction

Our memories reflect the accumulation of our past experiences. They shape future decisions and determine what and how we learn over time. It should not be surprising, then, that many fundamental elements of memory (e.g., different associative algorithms, motivation, sensory, motor, attention, memory updating, and response selection) must work together to continuously guide experience-dependent and adaptive behaviors regardless of the nature of the type of currently active memory. Study of a variety of amnesic populations has illustrated that not only many regions of the brain play these important roles in memory, but also different brain areas do so for different reasons. Temporal lobe patients (the most famous of which is patient H. M.) show severe but select anterograde episodic memory impairment, while procedural memory remained intact (Bayley et al. 2005; Milner 2005). Patients suffering from basal ganglia dysfunction show selective impairment in habit learning and procedural memory (Knowlton et al. 1996; Yin and Knowlton 2006). Amygdala damage results in poor emotional regulation of memory (Adolphs et al. 2005; Paz and Pare 2013). Frontal patients suffer from inadequate working memory (Baddeley and Della Sala 1996; Goldman-Rakic 1996). These classic distinctions of the mnemonic consequences of damage to different brain areas in humans have been replicated in rodents by many, and the Kesner laboratory has been particularly successful at demonstrating not only double, but also often triple dissociations of functions of structures like the hippocampus, striatum, amygdala, and prefrontal cortex (e.g., Chiba et al. 2002; Gilbert and Kesner 2002; Kametani and Kesner 1989; Kesner et al. 1993; Kesner et al. 1989; Kesner and Williams 1995). Moreover, the often clever behavioral paradigms created in the Kesner laboratory over the years have been inspirational for generating more specific hypotheses about memory function that could be tested in human subjects (e.g., Hopkins et al. 1995, 2004; Kesner and Hopkins 2001, 2006).

Decades ago, Ray Kesner was perhaps the first to develop a broad theoretical model of memory that sought to explain many of its complexities. His attribute model of memory posits the existence of three basic types of memory (event, knowledge, and rule-based memories), each one of which incorporates similar and fundamental memory operations to establish and use their particular type of memory. Kesner has written many elegant reviews of his work (Kesner 1980, 1998, 2009; Kesner and DiMattia 1987; Kesner and Rogers 2004), and readers, if they have not done so already, should seek out those reviews to gain an appreciation for his most impressive programmatic, timely, and innovative research program. Kesner initially proposed his attribute model at a time when most studies on the neurobiological models of memory focused on simpler memory functions of a small number of brain regions. However, more recently, the development of new behavioral and neuroscience technologies has sparked the current, widespread, and strong interest in studying the multiple neural systems of the brain during complex memory function that involve learning and decision-making mechanisms. Thus, it is clear that the attribute model was and continues to be a visionary theoretical framework for studying brain mechanisms of memory, learning, and decision-making. Indeed, it is now generally accepted that as espoused by the attribute model, hierarchical sets of parallel and distributed neural networks mediate the complex and dynamic processes of simple and complex memories in the brain. Current challenges are to figure out how different networks interact, how behaviors come to guide memory operations, and how existing memories guide future learning and decisions.

Neurophysiological investigations of these memory-related brain regions both confirmed and challenged the multiple memory system view. Spatial and conjunctive context-dependent coding were identified in the hippocampus (O’Keefe and Dostrovsky 1971; O’Keefe and Nadel 1978) and this was consistent with the view that hippocampus mediates episodic memory (Tulving 2002). Response-related codes were found in the striatum (Eschenko and Mizumori 2007; Jog et al. 1999; Yeshenko et al. 2004), supporting the hypothesis that striatum mediate habit or response learning (Knowlton et al. 1996). Frontal cortical neurons remain active during delay periods (Goldman-Rakic 1995), a finding that one might expect from a brain region that is importantly involved in working memory (Fuster 2006, 2008, 2009). With time, however, additional studies began to show that these striking neural correlates of behavior were not so unique to the hippocampus, striatum, and frontal cortex. Egocentric movement-related firing by hippocampal interneurons was reported long ago (e.g., Vanderwolf 1969) but was largely unstudied until recently in favor of studying what was at the time the more intriguing place cells. Parietal cortical neurons also showed strong representations of behavioral responses (e.g., Fogassi et al. 2005; McNaughton et al. 1994). Delay cells were found in regions of the cortex other than the prefrontal cortex, for example, in somatosensory cortex (Meftah et al. 2009), parietal cortex (Snyder et al. 1997), frontal eye fields (Curtis et al. 2004), and less so in temporal cortex (Kurkin et al. 2011). The fact that the single unit evidence did not align directly with the lesion literature suggested that many regions of the brain use similar types of information during their mnemonic computations. However, since the single-unit data came from studies of rodents and primates that used different recording methods while subjects performed a diverse set of tasks, it became important to record from multiple memory-related brain structures as an animal performed a single task that required animals to switch between different memories to continue adaptive decision-making.

In the following section, we describe our efforts to address the issue of whether different brain regions mediate different memories because they represent different kinds of information. The last section of this chapter explores the hypothesis that the relative contribution of different brain areas to memory is driven by homeostatic neural mechanisms that insures proper self-regulation of a behavioral adaptation system that depends on the memory functions described in the attribute model.

Memory Specialization and the Brain

Given that different memory capacities exist across different brain structures, a major challenge has been to understand why those different brain areas make such specialized contributions to memory. The following describes investigations that tested the hypothesis that different brain areas represent different types of information and that this is responsible for their different memory capacities. Their specific focus here will be on a comparison of hippocampal and striatal neural representations as rats performed a hippocampal-dependent (spatial) task or a striatal-dependent (response) task. Importantly, these are tasks that show dissociable mnemonic involvement by the hippocampus and striatum in lesion and clinical investigations. Also, it is noted that the principle conclusions from these results should apply more broadly to an understanding of the relationship between other memory-related brain areas such as the amygdala and prefrontal cortex.

Are Memory Specializations Due to the Nature of the Information Represented by Neurons?

Dorsal striatal and dorsal hippocampal single-unit activity were recorded as rats performed a T-Maze task (Yeshenko et al. 2004; Eschenko and Mizumori 2007). One group of rats was trained to solve the first 10 trials of a recording session according to a spatial strategy and then the next 10 trials according to a response strategy. According to a spatial strategy, rats seek a location that had been previously associated with reward. A response strategy, on the other hand, requires rats to use the same egocentric response (i.e., right turn from the start location) to find food. Another group of rats ran 10 response trials followed by 10 spatial trials. Since these 20 trials were performed within one recording session, the same striatal and hippocampal neurons were recorded before, during, and after an experimenter-controlled switch in cognitive strategy (or memory). Also, since striatal neurons were often recorded simultaneously with hippocampal neurons, it was possible to compare directly activity in the two brain structures relative to the currently active memory, the accuracy of the choices made, and relative to the type of cognitive switch. Such switches included, not only changes from spatial to response strategy use (or vice versa) but also changes from one spatial memory (e.g., food is in the north location) to another spatial memory (e.g., food is in the south location), or from one response strategy (always turn right to find food) to another response strategy (always turn left to find food). Also, in some tests, the visual cues were altered to present another type of strategy (or memory) shift. Together, the inclusion of these different types of manipulations made it possible to see if neural responses in hippocampus or striatum were specific to a particular type of activated memory or cognitive change, or just cognitive change in general.

As had been reported in previous studies (e.g., McNaughton et al. 1983; O’Keefe and Dostrovsky 1971; Olton et al. 1978; Muller and Kubie 1987; Ranck 1973; Redish 1999; summarized in Mizumori 2008b; O’Keefe and Nadel 1978), hippocampal pyramidal cell discharge showed strong correlations with the location of an animal on the maze, while hippocampal (presumed) interneurons showed firing that was correlated with an animal’s movement velocity (Eschenko and Mizumori 2007; Yeshenko et al. 2004). As expected from the results of striatal lesion studies, dorsal striatal neurons showed strong correlations with behavioral response parameters such as the rat’s movement velocity and acceleration. Unexpectedly, location-selective neurons were also found in both medial and lateral sectors of dorsal striatum. (This pattern contrasts with an earlier report that a different type of neural representation of space, a rat’s directional heading, is found in only the medial, not lateral, dorsal striatum, Ragozzino et al. 2001.) Most of the details of the properties of both the movement and location correlates of hippocampal and striatal neurons did not differ as a function of whether the rat was solving the task with a (hippocampal-dependent) spatial or (striatal-dependent) response strategy. This result indicates that both structures continuously and actively process similar types of information (although some of the details may vary) regardless of whether hippocampal-dependent or striatal-dependent memories are engaged.

Given that both spatial and response-related information are represented in hippocampus and striatum, it is possible that the distinct mnemonic contributions of these areas derive from differential sensitivities to changes in memory or contextual information. This hypothesis was also tested by Eschenko and Mizumori (2007) and Yeshenko et al. (2004) and the clear result was that both hippocampal and striatal spatial, and movement neural codes were sensitive to changes in the memory demands of the task regardless of how the change in memory was brought about (e.g., by changing choice strategies, the available cues, from spatial to response strategy, or vice versa). For example, velocity correlated firing by either striatal or hippocampal neurons showed significant changes in the magnitude of the velocity correlation after the cognitive demands of a task shifted. This was the case even though the actual behaviors and velocities exhibited by the rats were consistent in all phases of the test session. Interestingly, hippocampal movement codes responded most dramatically when rats shifted from one type of spatial strategy to another, whereas striatal movement codes responded similarly across all types of context shifts. Thus, details of a given context shift may differentially impact particular hippocampal neurons whereas the same could not be said for striatal networks. Perhaps striatum responds more generally than hippocampus to any type of context change, a conclusion that is consistent with the view that striatum is primarily responsive to changes in reinforcement conditions more generally. Striatal and hippocampal velocity codes per se then are not solely determined by the ongoing behavior of the animals, but rather it is determined, or gated, by memory.

Place-specific firing by striatum and hippocampus also showed sensitivity to changes in cognitive strategy (or activated memory) and this was evidenced by significant changes in place field location or in-field firing rates. In the future then it would be of interest to determine if the context-regulation of striatal neural codes derives from hippocampus by disabling hippocampus, while testing the striatal neural responses to a context change. Since the hippocampus is hypothesized to detect changes in context (e.g., Mizumori et al. 1999; Mizumori et al. 2007a; Mizumori and Jo 2013), one prediction is that striatal neurons will not respond to context manipulations without a proper functioning hippocampus. If, however, striatal context-sensitivity is not affected when hippocampal input goes off-line, then the context-dependent striatal codes may emerge from neocortical memory systems.

Do the Memory Specializations Within the Brain Reflect Compensation After Brain Damage?

Another consideration that can be used to explain the different memory consequences of hippocampal and striatal lesions is that their effects reflect the extent to which remaining brain areas can compensate for a particular type of lesion. If the intrinsic processing by the structure of interest is unique and essential for learning to take place, then no behavioral impairment should be observed if other neural circuits are compromised. Indeed, there is abundant evidence that under most conditions, stimulus-response learning is not impaired following hippocampal lesions, presumably since striatal computations are sufficient to support such learning (e.g., Knowlton et al. 1996; McDonald and White 1993; Packard et al. 1989; Packard and McGaugh 1996; Yin and Knowlton 2006). This does not mean that the hippocampus does not normally play a role in stimulus-response performance; if hippocampus defines the context of the learning so that animals can quickly adapt when test conditions change, deficits in stimulus-response performance may be observed after hippocampal damage only if a context change is involved. In contrast, context-dependent learning is by definition complex and dynamic—a situation that the hippocampus seems uniquely qualified to handle. Thus, as shown in the literature, hippocampal, but not striatal, lesions result in selective context memory deficits.

Are Memory Specializations in the Brain Defined by the Coordination of Neural Networks Across Brain Structures?

It is possible that brain structures play different roles in memory because of task-dependent co-modulated neural activity across different brain structures at strategic times during task performance. Although much work remains to thoroughly test this hypothesis, there are initial indications that support this view. Ragozzino et al. (2001) recorded striatal head direction cells simultaneously with hippocampal place cells, and then compared their responses to different types of context shifts. It was found that when familiar cues were shifted, head directional preferences and place field locations shifted in a comparable fashion. In contrast, when rats were placed in new environments, the shift in head direction preferences and place fields appeared random relative to each other. This result suggests that memory (i.e., past experiences) can bias striatal and hippocampal neural responses in a coordinated fashion, and that without such memory guidance, the two structures function more independently.

Numerous laboratories report that specific neural population-based rhythms can be detected both within and between memory structures such as the hippocampus, striatum, or prefrontal cortex (DeCoteau et al. 2007a; Engel et al. 2001; Fell et al. 2001; Siapas et al 2005; Tabuchi et al. 2000; Varela et al. 2001; Womelsdorf et al. 2007). Hippocampal theta activity seems to coordinate the timing of spatial coding by hippocampal neurons (Gengler et al. 2005; O’Keefe and Recce 1993). Striatal theta oscillations have been shown to become entrained to the hippocampal theta rhythm (Berke et al. 2004; DeCoteau et al. 2007a) in a behaviorally-dependent fashion. Also, directly stimulating the striatum can induce hippocampal high frequency theta activity (Sabatino et al. 1985). When neural activity is disrupted in the striatum via D2 receptor antagonism, striatal modulation of high frequency hippocampal theta activity is reduced, motor and spatial/contextual information is not integrated, and task performance is impaired (Gengler et al. 2005). This is consistent with the idea that theta is important for sensory-motor integration (Hallworth and Bland 2004). It appears then that during goal directed navigation, hippocampal and striatal activity becomes increasingly coherent, and this pattern is dopamine dependent. Given its putative role in assessing the value of behavioral outcomes (e.g., Schultz and Dickinson 2000), dopamine may play an important role in biasing the relative strengths of hippocampal and striatal output signals according to the most effective mnemonic strategy (e.g., Mizumori et al. 2004).

Particularly intriguing is a finding common to both the hippocampus and striatum, and that is that synchronous neural activity occurs in specific task-relevant ways (e.g., Hyman et al. 2005; Jones and Wilson 2005), especially when rats engage in decision-making (e.g., Benchenane et al. 2010). For example, striatal theta is modified over the course of learning an egocentric T-maze task, increasing as the rat comes to regularly choose and initiate turn behavior (DeCoteau et al. 2007a, 2007b). Rats that learned the task developed an antiphase relationship between hippocampal and striatal theta oscillations, while rats that did not learn the task did not show this coherent theta relationship. This coherence has also been observed during striatal-dependent classical conditioning (Kropf and Kuschinsky 1993).

The possibility that dopamine contributes to the regulation of memory efficiency and strategies by coordinating ensemble neural activity in distant brain structures is intriguing given its role in decision-making: Coherent theta oscillations across distant brain structures can be enhanced with application of dopamine (Benchenane et al. 2010) and impaired by dopamine antagonism (Gengler et al. 2005). Functionally, this type of control by dopamine suggests that information about the saliency of reward may determine which brain systems become synchronized (and desynchronized), and this in turn informs the decision process about what information is used to update memories and which behaviors are selected.

The functional importance of neural oscillations in the gamma range (30–80 Hz) remains debated. However, since gamma oscillations tend to occur intermittently (i.e., “gamma bursts” of about 150–250 ms are followed by periods of desynchronous activity), information carried by the cells that participate in a gamma burst effectively become a clear and punctuate signal against a background of disorganized neural activity. For this reason, it has been suggested that gamma bursts represent a fundamental mechanism by which information becomes segmented and/or filtered within a structure, as well as a way to coordinate information across structures (Buzsaki 2006). Although theta and gamma frequencies are quite different (perhaps reflecting the type of information that each rhythm coordinates), there are many common physiological and behavioral relationships that suggest they are components of a coordinated and larger scale oscillatory network. For example, similar to theta rhythms, single unit responses that are recorded simultaneously with gamma oscillations have been found to have specific phase relationships to the gamma rhythm (e.g., Berke 2009; Kalenscher et al. 2010; van der Meer and Redish 2009). Also, it is hypothesized that gamma oscillations may effectively select salient information that can come to impact decisions, learning, and behavioral responses (e.g., Kalenscher et al. 2010; van der Meer and Redish 2009), since their appearance is often in relation to task-relevant events. Another similarity with the theta system is that the occurrence gamma oscillations appear to be at least in part regulated by the dopamine system (Berke 2009).

Are Memory Specializations in the Brain Defined by the Functional Significance of the Output Messages of Populations of Cells?

From the above discussion it is clear that there are widespread neural codes for spatial and response aspects of task performance across different brain areas, and that these are context- (or memory-) dependent. It is possible that neuromodulators such as dopamine bias the strengths of the output messages according to recent behavioral success. What then is the meaning of the efferent neural code at the population level? What neural mechanisms control this meaning, and how are the outputs of different memory processing areas of the brain coordinated to result in continuously adaptive behaviors? These are some of the big challenges that remain to be addressed before we can fully understand the neural systems basis of multiple memory function. What follows is a suggested approach to future investigations of these big challenges.

Predictive Memories and Adaptive Decisions

Known patterns of intrinsic neural connectivity indicate that each memory-related brain structure undoubtedly processes information in a somewhat unique way, and yet it is unclear if these differences are sufficiently unique to account for the documented specialized memory capacities of each brain area. There is, however, growing evidence that the output of different brain structures has a common goal for different kinds of information, and that is to relay the extent to which experience-based predictions relevant to optimal task performance are born out. In fact, an emerging view is that the brain evolved in large part to allow organisms to accurately predict the outcomes of events and behaviors (e.g., Buzsaki 2013; Buzsaki and Moser 2013; Llinas and Roy 2009; Mizumori and Jo 2013). In this way, organisms have been able to adapt to environments and societies of increasing complexity—a condition that required sophisticated mechanisms to make decisions and predictions in a dynamic and conditional environment. According to this view, the underlying neural mechanisms of predictions (and the assessment of their accuracy) are likely to be highly conserved across species (Adams et al. 2013; Watson and Platt 2008). This includes the ability to retain information over times of varying scales depending on the desired goal. Indeed, different brain areas are known to generate and retain sequences of information, an ability that can be accounted for by state-dependent changes in network dynamics (Mauk and Buonomano 2004), internally-generated oscillatory activity (Pastalkova et al. 2008), and/or dedicated “time cells” (Kraus et al. 2013).

Hippocampal Evaluation of the Accuracy of Predictions About Contextual Information

A context discrimination hypothesis (CDH) postulates that single hippocampal neurons provide multidimensional (context-defining) data to population-based network computations that ultimately determine whether expected contextual features of a situation have changed (e.g., Mizumori et al. 1999, 2000a, 2007a; Mizumori 2008a, b; Smith and Mizumori 2006a, b). Specifically, these hippocampal representations of spatial context information (O’Keefe and Nadel 138; Nadel and Payne 2002; Nadel and Wilner 1980) may contribute to a match-mismatch type of analysis that evaluates the present context according to how similar it is to the context that an animal expects to encounter based on past experiences (e.g., Anderson and Jeffery 2003; Gray 1982, 2000; Hasselmo 2005b; Hasselmo et al. 2002; Jeffery et al. 2004, Lisman and Otmakhova 2001; Manns et al. 2007a; Mizumori et al. 1999, 2000a; Smith and Mizumori 2006a, b; Nadel 2008; Vinogradova 1995). Detected mismatches can be used to identify novel situations, initiate learning-related neural plasticity mechanisms, and to distinguish different contexts in memory—functions that are necessary to define significant events or episodes. When a match is computed, the effect of hippocampal output could be to strengthen currently active memory networks located elsewhere in the brain (e.g., neocortex). Thus, hippocampus may play different mnemonic roles depending on whether or not contexts change.

In support of the CDH, disconnecting hippocampus by fornix lesions impairs context discrimination (Smith et al. 2004), and hippocampal lesions reduce animals’ ability to respond to changes in a familiar environment (Good and Honey 1991; Save et al. 1992a, 1992b). Spatial novelty detection corresponds to selective elevation of the immediate early gene c-fos in hippocampus, and not in surrounding parahippocampal cortical regions (Jenkins et al. 2004). Also, as described above, hippocampal neurons show significantly altered firing patterns when rats experience spatial or nonspatial changes in a familiar environment (Eschenko and Mizumori 2007; Ferbinteanu and Shapiro 2003; Fyhn et al. 2002; Leutgeb et al. 2005a, 2005b; Moita et al. 2004; Muller and Kubie 1987; O’Keefe 1976; Puryear et al. 2006; Smith and Mizumori 2006b; Wood et al. 1999; Yeshenko et al. 2004). As an example, Smith and Mizumori (2006b) showed that hippocampal neurons develop context-specific responses, but only when rats were required to discriminate contexts. Discriminating neural responses were not observed when rats were allowed to randomly forage for the same amount of time. Further, Manns et al. (2007b) demonstrated that relative to match trials in an odor cue or object recognition task, CA1 neurons were preferentially discharged when animals experienced a nonmatch situation in these same tasks. Also consistent with the CDH, neuroimaging studies of human performance shows that hippocampus becomes differentially active during match and mismatch trials (Chen et al. 2011; Dickerson et al. 2011; Duncan et al. 2012a; Duncan et al. 2012b; Foerde and Shohamy 2011; Kuhl et al. 2010; Kumaran and Maguire 2007).

The detection of changes in context is fundamentally important for the continual selection of appropriate behaviors that optimize performance and learning in a variety of tasks (e.g., navigation-based learning, instrumental conditioning, or classical conditioning). Context discrimination engages and prepares cellular mechanisms for rapid and new learning at potentially important times (Paulsen and Moser 1998), as it is generally known that novelty detection increases attention and exploratory behaviors in a variety of tasks. Interestingly, hippocampal cell firing tends to occur during the “encoding phase” of the ongoing theta rhythm (Hasselmo 2005a), which is increased during exploratory and investigatory behaviors (Vanderwolf 1969). Thus, detection of a nonmatch situation can change the relationship between cell discharge and the local theta rhythm such that encoding functions are enhanced. Detection of matches, on the other hand, does not cause changes in the hippocampal neural activity profile, resulting in efferent messages that continue to retrieve/utilize the currently active memory network that drove the execution of recently successful responses. Context discrimination, then, can be viewed as being critical for the formation of new episodic memories because it leads to the separation in time and space, one meaningful event from the next. Such division of memories could facilitate long-term information storage according to memory schemas (Bethus et al. 2010; Tse et al. 2007).

Context discrimination, or the detection of a mismatch between expected and experienced context-specific information, is considered an example of an error in predicting the contextual details of the current situation, referred to as a context prediction error. Transmission of a context prediction error signal from hippocampus should inform distal brain areas that a change in the context has occurred. In this case, upon receipt of the context prediction error message, efferent midbrain structures may respond with changes in excitation or inhibition that are needed to evaluate the subjective value of the context prediction error signal (e.g., Humphries and Prescott 2010; Lisman and Grace 2005; Mizumori et al. 2004; Penner and Mizumori 2012a). On the other hand, a hippocampal signal indicating that there was no prediction error may enable plasticity mechanisms that facilitate the incorporation of new information into existing memory schemas (e.g., Bethus et al. 2010; Mizumori et al. 2007a, b; Tse et al. 2007). Thus, hippocampal context analysis become critical for the formation of new episodic memories not only because prediction signals provide a mechanism that separates in time and space one meaningful event from the next, but also because the outcome of the prediction error computation engages appropriate neuroplasticity mechanisms in efferent structures that promote subsequent adaptive decisions and memory.

Striatal Evaluation of the Accuracy of Predictions About Response Outcomes

Analogous to hippocampus, the midbrain dopaminergic system also generates prediction error signals, but in this case the focus is on whether the outcome of goal-directed behaviors occur as predicted based on past experience (Bayer and Glimcher 2005; Hollerman et al. 1998; Hollerman and Schultz 1998; Mizumori et al. 2009; Stalnaker et al. 2012). In particular, it is thought that dopamine neurons transmit information about the subjective value of rewards in terms of reward prediction error signals (RPEs). RPEs are thought to initiate three distinct and parallel loops of information processing between striatum and neocortex as new associations become learned sufficiently to habitually drive behaviors (e.g., Alexander et al. 1986; Alexander and Crutcher 1990a, b; Haber 2003). Penner and Mizumori (2012b) recently summarized this vast literature: Information within the limbic loop flows between ventromedial prefrontal cortex with the ventral striatum (Alexander and Crutcher 1990a, b; Graybiel 2008; Graybiel et al. 1994; Pennartz et al. 2009; Voorn et al. 2004; Yin and Knowlton 2006) to mediate learning about the significance of previously neutral stimuli (i.e., as it occurs in Pavlovian learning). The associative loop involves the medial prefrontal cortex and the dorsomedial striatum to support action-outcome learning. The sensorimotor loop involves transmission between somatosensory and motor cortical areas with the dorsolateral striatum. The latter loop is suited for incremental sensory-motor learning that happens when new procedural memories are formed. It is hypothesized that the transformation of newly learned behaviors to habits occurs as a result of multiple iterations of information flow through these three information loops starting with the limbic loop, the associative loop, and then finally the sensorimotor loop. Importantly, information flow through these systems is thought to be continually informed about the expected values of goals via dopamine signaling from the ventral tegmental area (VTA) and/or the substantia nigra (SN; Horvitz 2002; Nicola et al. 2004; Schultz 2010). When performing well-learned habits, the striatum is particularly suitable to rapidly control behavior or to provide feedback about behaviors that led to prediction errors (Stalnaker et al. 2012) because of its rather unique pattern of reciprocal connections with sensory and motor cortical regions (Alexander and Crutcher 1990a; Groenewegen et al. 1999; Haber 2003), and because striatum can receive immediate feedback when goal outcomes are not what was expected. In this way, midbrain signals of errors in predicting rewards may initiate adjustments to future planned behaviors (Penner and Mizumori 2012b).

Sensory and Motor Predictions

In addition to hippocampus and striatum, various sensory-motor cortical and cerebellar areas have been reported to generate prediction errors when expected sensory or motor-related input does not match expectations (e.g., Scheidt et al. 2012; Tanaka et al. 2009). This sort of feedback permits temporally and spatially precise behavior adjustments based on past outcomes. Also, information about expected sensory and motor events can be used to plan future sensory expectations and specific anticipatory movements (e.g., Duhamel et al. 1992). Such prediction error mechanisms are thought to fine tune actions to optimize the chances of securing a desired goal.

Summary: Error Signaling in the Brain

The above description illustrates that the generation of neural responses that signal times when actual events or information do not match those expected based on past experiences (i.e., prediction error signals) is often observed across many brain areas. In fact, it has been suggested that the ability to predict behavioral outcomes has essentially driven the evolution of the entire brain (Llinas and Roy 2009). Such error signals allow organisms to appropriately refine movements and choices relative to their perceived utility or value, and thus ultimately determine future decisions and behavior (e.g., Doll et al. 2012; Schultz and Dickinson 2000; Walsh and Anderson 2012). At a cellular level, prediction error signals may elevate the level of excitability of efferent neurons such that they become more responsive to outcome signals. This greater neural responsiveness may enhance the temporal and spatial resolution of future neural responses, and this in turn may ultimately result in improved accuracy of future predictions. For example, if hippocampus detects a mismatch between expected and actual contextual features, it may generate an error signal that “alerts” striatal efferent structures so that they become more responsive to future rewards (Lisman and Grace 2005; Mizumori et al. 2000a, 2004, 2009; Penner and Mizumori 2012a, b; Schultz and Dickinson 2000). Midbrain-generated reward prediction error signals may destabilize cortical neural (memory) networks so that they become more readily updated with new information (Mizumori 2008a; Penner and Mizumori 2012b). The updated memory information can then be passed on to hippocampus in the form of the most up-to-date context expectations. This view of how error signals can inform future processing in other prediction regions of the brain is consistent with the view that there is a high degree of interdependence across mnemonic structures regardless of the task (Mizumori et al. 2004; Yeshenko et al. 2004).

Homeostatic Regulation of Predictive Memory Systems

An interesting and often described feature of memory functions is the rapid and seemingly automatic nature of its processes, or changes in processing, when a significant feature of a memory task changes. A challenge for future research then is to understand the neural mechanisms of the apparent automaticity and high accuracy of prediction analyses. An intriguing possibility is that the seemingly automatic nature derives from self-regulatory, intrinsic synaptic mechanisms rather than (only) responses to external information. Such mechanisms may align with the principles of homeostasis similar to those described for self-regulation at synaptic and neural circuit levels (e.g., Marder and Goaillard 2006; Marder and Prinz 2003; Mizumori and Jo 2013; Turrigiano 1999, 2008, 2011; Turrigiano and Nelson 2004). That is, homeostatic regulation could drive the automatic and continuous maintenance of the balance between stable and flexible processing that neural networks need to retain learned (stable or expectancy) information that can to be (flexibly and adaptively) updated as needed.

Marder and Goaillard (2006) suggested that homeostatic neural plasticity may be nested: Calcium sensors may monitor neural firing rates, then up or down regulate the availability of glutamate receptors to ramp up or down firing rates toward an optimal firing rate set point. Groups of neurons or neural networks may sense changes in firing collectively to regulate experience-dependent population activity levels and patterns of activation. In this way homeostatic synaptic plasticity enables groups of neural circuits to find a balance between flexible and stable processing as needed to learn from experiences, and to be responsive to future changed inputs. While details of how networks of cells or their connections engage in homeostatic regulation remain to be discovered it is worth noting that homeostatic regulation at the neural systems level indeed occurs, as is evident from studies of brain development, as well as from studies of reactive or compensatory neuroplasticity mechanisms that occur in response to experience (e.g., sensorimotor learning; Froemke et al. 2007) or brain injury (e.g., brain trauma or addiction; Nudo 2011; Robinson and Kolb 2004). Although homeostatic neural plasticity mechanisms have not been used to account for complex learning, current theories of reinforcement- and context-based learning and memory commonly rely on the autoregulation of feedback loops between systems that assess the outcomes of choices and existing (episodic) memory systems.

If homeostatic regulation pertains to neural networks that underlie adaptive memory, it should be possible to identify key components including variables that are being monitored by sensors and then regulated by controllers. For complex memory systems, such a model likely contains multiple hierarchically organized loops of control similar to what was described by Buzsaki (2013). For example, iterative updating via interactions between hippocampus, the dopamine striatal system, and memory networks would allow context prediction errors to guide future adaptive behaviors. This is somewhat similar to the micro- macro-agent distinction recently described by Kurth-Nelson and Redish (2009). This process may be enabled by sensors which monitor cell excitability within each structure. In this case, changes in calcium flux appear to be an important part of the sensing system that determines the current level of firing. A change in firing rates (either higher or lower than the optimal level) is taken as an indicator of a mismatch between optimal and actual rates, and a controller mechanism becomes engaged to bring the firing rates back to the optimal levels. Such a natural restorative mechanism that responds to cellular detection of errors in prediction (as reflected in firing rate deviations) seems essential since it would be unhealthy for neurons to exist in a chronically excited or inhibited state.

It has been suggested that the prefrontal cortex serves in a controller role that maintains the optimal excitatory level in prediction regions of the brain (more details below). In particular prefrontal cortex is strategically situated to enable mechanisms that restore afferent firing rates to a predetermined ‘set point’ via its detailed excitatory and inhibitory extrinsic connections (as reviewed in Arnsten et al. 2012). In this way, prefrontal cortex may orchestrate and coordinate the level of neural excitability in different prediction error brain areas (e.g., hippocampus and striatum) according to homeostatic principles. Thus, prefrontal cortex biases the nature of the outputs of connected brain areas according to experience and recent outcomes of decisions (Fig. 9.1).

Fig. 9.1
figure 1

a A homeostatic model of memory processing suggests that the primary goal of prefrontal cortex interactions with prediction centers of the brain (e.g., hippocampus and the midbrain-striatal area) is to maintain the baseline (tonic) firing rate of neurons within these centers at a set point level that is optimal for detecting future prediction errors. Modulating factors such as one’s motivation or emotional state can elevate or reduce the baseline firing rates. It is postulated that the prefrontal cortex continually receives information from the prediction areas regarding the current population firing rates. If the baseline rates become elevated (e.g., due to stress) the prefrontal cortex is equipped to anatomically and physiologically restore firing rates (red straight arrows) to their optimal (baseline) levels. If the rates become too low (e.g., in depression), the prefrontal cortex should engage mechanisms to elevate firing to optimal levels over time. When a prediction error signal arrives in, or is generated by, a given prediction structure, firing rates can increase (in cases when prediction errors are positive) or decrease (when prediction errors are negative). The degree of rate increases or decreases scales to the degree of mismatch that is detected, and the slope of the increase or decrease in firing may vary between individuals and/or as a function of experience (blue arrow). b The prefrontal cortex is responsible for restoring the firing rates back to optimal levels and this reduces the uncertainty that was generated by the prediction error signals

Full size image

It should be noted that although it is reasonable to assume that the prefrontal cortex is a major controller of the impact of prediction error signaling in the brain, other sources of control of cell excitability may arise via direct interconnections between the multiple prediction detection areas of the brain. For example, a prediction error from the hippocampus could be transmitted to midbrain–striatal neurons along pathways that do not necessarily include the prefrontal cortex. Indirect support for this idea comes from observations that conditions that produce error messages in the hippocampus change reward responses of dopamine neurons (Jo et al. 2013; Puryear et al. 2010), phasic theta comodulation is observed between hippocampus and striatum (DeCoteau et al. 2007a) during decision tasks, and comodulation of neural activity has been reported between prefrontal cortex and parietal cortex (Diwadkar et al. 2000). Perhaps neuromodulators such as dopamine strengthen or weaken associations that led to the last correct or incorrect, respectively, decision or behavior.

In sum, homeostatic regulatory processes may contribute to the automatic and continuous self-regulatory nature of prediction error analysis, decision-making, and learning. Such a naturally adaptive mechanism may optimize the relative contribution of different types of prediction error signals to future decisions and actions according to the pattern of recent successes and failures in prediction.

Setting the Baseline from Which Prediction Error Signals Emerge: A Role for Hypothalamus, Amygdala, and the Prefrontal Cortex

Individual neurons face a continual barrage of excitatory inputs across tens of thousands of synaptic connections. Yet, neurons cannot maintain high levels of excitability and remain viable in the long term. Fortunately, individual neurons appear to be able to naturally and automatically engage mechanisms that control their level of excitability. This may occur by sensing and controlling the flow of various ions across cell membranes (e.g., Burrone et al. 2002; Turrigiano 2008; Turrigiano et al. 1998, see more detailed description below). Optimal levels of neuronal activity can be maintained also by achieving a relatively constant balance of excitatory and inhibitory synaptic input (e.g., Burrone et al. 2002). Together these factors define the baseline level of tonic activity from which phasic error signals are generated. Interestingly, the tonic level of cell excitability can be set according to the motivational state of an animal (Cagniard et al. 2006; Pecina et al. 2003; Puryear et al. 2010) suggesting that ones motivational state may play a significant role in determining the threshold for phasic neural and behavioral responding to unexpected events.

Motivational state information (e.g., signals of hunger or thirst) may arrive in prediction error structures (e.g. the hippocampus or striatum) and/or their controller (prefrontal cortex) via hypothalamic afferent systems. For example, lateral hypothalamus signals of hunger that reach brain areas that evaluate predictions may increase subsequent reward-responsiveness of efferent target neurons. Elevated responses to reward may reflect higher subjective values of the reward, an interpretation that is consistent with the biological needs of an animal. The amygdala, on the other hand, is thought to mediate a different motivational variable, and that is the emotional state of animals (Johansen et al. 2011). A message reflecting the current emotional state may emerge from the amygdala’s role in associating cues with their aversive consequences (e.g., Chau and Galvez 2012; Paz and Pare 2013). Amygdala likely alters its neural activity in response to fear (Ciocchi et al. 2010; Haubensak et al. 2010; Li et al. 2013). Since the amygdala has direct excitatory effects on SN or VTA neurons (Lee et al. 2005; Zahm et al. 2011), fear-induced amygdala activation may increase the likelihood that dopamine neurons transition to a more excitable “up-state” (Wilson 1993; Wilson and Kawaguchi 1996) when hippocampal messages arrive. In this way, in urgent situations, animals can more readily assess the value of a changed context since transitioning to an “up-state” could make the dopamine cells respond more quickly to an input. This could be adaptive since responses can be implemented more quickly.

In addition to generally biasing the levels of neural excitability (which may translate to biasing the threshold for prediction error signaling), the amygdala may modulate prediction error-based learning efficiency on a trial by trial basis. For example, it is known that there is increased attention to cues or rewards that are unexpected or surprising based on past experiences (Pearce and Hall 1980; Rescorla and Wagner 1972). The dopamine system clearly plays a role in surprise-induced enhancement of learning (e.g., Schultz et al. 1997; Schultz and Dickinson 2000), and this may relate to transient influences of the amygdala on dopamine neurons since this prediction error-based learning effect is abolished in rats with amygdala disruption (Holland and Gallagher 2006) with no effects on the subsequent expression of surprise-induced enhanced learning (Lee et al. 2008). The amygdala and hypothalamus, then, may orchestrate information processing circuits/systems by ultimately setting the threshold for future error detection via direct connections to prediction error structures such as the hippocampus, striatum, sensory and motor cortex, and the cerebellum.

The prefrontal cortex can also be thought of as regulating the excitability state of neurons in predictive centers of the brain but for different reasons than both amygdala and hypothalamus. The prefrontal cortex is commonly thought to be important for holding information on-line in a working memory buffer (e.g., Arnsten et al. 2012; Fuster 2008). This function is considered essential to be able to select appropriate responses and/or for switching behavioral strategies (Ragozzino et al. 1999a; Ragozzino et al. 1999b; Young and Shapiro 2009), and this interpretation is consistent with findings that transient functional connections exist between the prefrontal cortex and the hippocampus or striatum especially when working memory is helpful for optimal behaviors. For example, hippocampal and prefrontal theta become co-modulated at times when animals make choices (e.g., Hyman et al. 2005; Shirvalkar et al. 2010), but not at other times during task performance. Co-activation of striatal and prefrontal activity has also been observed when working memory is required for accurate response selection (Levy et al. 1997; Scimeca and Badre 2012). Thus, the functional connections between striatum and prefrontal cortex, or between hippocampus and prefrontal cortex, can vary in strength and impact depending on the current task demands. Presumably this variation reflects the phasic task-dependent coordination of patterns of excitation and inhibition between prefrontal cortex and its efferent targets. Since the prefrontal cortex is thought to play a role in prediction analysis (e.g., Holyroyd et al. 2002), we suggest the possibility that its major contribution is to regulate efferent cell excitability according to recent behavioral outcomes. Indeed, Karlsson et al. (2012) recently showed that prefrontal cortical representations switch states of stability when conditions of greater uncertainty arise, that is, when response outcomes do not occur as predicted. Also, Merchant et al. (2011) suggest that prefrontal cortex exerts “top-down” control over parietal cortical responses in a match-to-sample task.

Prefrontal modulation is made possible due to the rather complex pattern of inhibitory and excitatory control over multiple types of efferent neurons (i.e., both interneurons and projection neurons) in efferent prediction brain areas (as reviewed in Arnsten 2011, Arnsten et al. 2012; Khan and Muly 2011), neurons that then could return information back to prefrontal cortex about their current activity state. Neocortex has indeed been shown to regulate the excitability states of subcortical neurons (e.g., Calhoon and O’Donnell 2013; Plenz and Arnsten 1996; Plenz and Kitai 1998). During baseline conditions, prefrontal cortex in particular may continually receive information about the current level of neural activity in target regions, and then use this afferent data to determine the extent and type of excitatory and inhibitory control needed to achieve optimal tonic activity within each of the multiple efferent prediction error systems. If the tonic activity becomes too low, for example, at times when there are no prediction error signals, prefrontal cortex may elevate the state of neural excitability so that the prediction cells are more responsive to future error signals, a feature that should increase the speed and accuracy of the error signaling. If, on the other hand, the baseline activity of a target region is higher than the optimal for the detection of prediction errors, further increasing the excitability of the cells may be detrimental for the cell’s health and ability to produce clear error signals. In this case, it would be most adaptive if the prefrontal cortex lowered the level of excitability of its target cells so that optimal responsivity can be restored in the target region.

Recurrent neurocircuitry within the prefrontal cortex is thought to contribute to its working memory capacity (e.g., Arnsten et al. 2012), and as such this circuit is a clear candidate system to not only integrate error signals arriving from the different prediction error brain regions, but to also bias the thresholds and strengths of subsequent error-related signals from the brain regions that originally produced the error signal. The particular constellation of excitatory and inhibitory biases presumably will result in the most desired behavioral outcome.

In summary, at specific times when working memory is needed, the intrinsic recurrent neural circuits of the prefrontal cortex (Arnsten et al. 2012) may selectively and strategically exploit (differentially or in concert) its rich array of excitatory and inhibitory efferent connections to regulate the probability of neural firing in different prediction areas of the brain such that the relative responsiveness of different prediction brain regions changes in task-dependent ways. When prediction errors are detected and firing rates change, the prefrontal cortex may not only integrate the signal within its recurrent intrinsic circuitry, but it may have a key restorative function in efferent structures such that the firing rates return to a baseline tonic level that optimizes subsequent responsiveness to input. Thus, the prefrontal cortex may bias efferent neurons’ ability to engage in, or efficiently use, prediction error analysis and hence their ability to adaptively guide future behaviors. This process may be a key factor that determines which prediction error-generating (i.e., memory) system ultimately controls the selection of future responses.

Final Comments

Memory research continues to evolve in complexity as new technologies become available. Ray Kesner’s conceptualization of the cognitive and neurobiological underpinnings of memory was visionary in that it laid out a multidimensional neural systems view of memory that has provided guidance for a number of decades. Current challenges are to understand why different brain structures have select roles in memory, the nature of the interaction between brain structures, the control mechanisms for the interactions between brain areas, and the mechanisms by which memory functions, in all of its complexities, appear to be self-regulated and automatic. We offer a new hypothesis that the different memory regions of the brain make special contributions to memory because they assess the validity of different types of predictions that are needed for one to continually make adaptive choices and engage in adaptive and goal-directed behaviors. A homeostatic model of memory regulation is described to at least in part account for the seemingly automatic nature of memory function. Thank you Ray, for guiding the field to this very important and crucial time in memory research, one that promises exponential growth in the near future.