Keywords

Sleep is vital for the long-term consolidation of newly acquired information [1–3]. The exact mechanisms behind this consolidation are yet unknown and often debated. The two most accepted theories for a mechanistic role of sleep in memory consolidation are the synaptic homeostasis hypothesis [4] and the active systems consolidation hypothesis [5]. As the names suggest, these theories differ in their view of whether consolidation is a passive or an active process.

The synaptic homeostasis hypothesis claims that slow oscillations induce synaptic downscaling, which nullifies the weakly potentiated connections and thereby improves the signal-to-noise ratio for the more strongly potentiated synapses. This de-potentiation of synapses also “resets” (refreshes) the cells, facilitating the efficient processing of new information upon waking. At frequencies below 1 Hz, de-potentiation of synapses through long-term depression (LTD) does appear to take place [6]. Homeostasis may be very important for reducing the signal-to-noise ratio of existing memories, by “pruning” inessential connections [7], and may reestablish a convenient working point [8] for proper circuitry function and for the acquisition of new memories. Reinforcing this hypothesis, specific slow-wave sleep (SWS) deprivation in humans impairs subsequent encoding of new memories [9].

However, during these periods of slow oscillations, activity is characterized by hippocampal sharp wave-ripple complexes and cortical spindles. These high-frequency events are known to facilitate long-term potentiation [10–12], indicating that there is more to memory enhancement than de-potentiation alone. Sharp waves are fast depolarizations with a large amplitude (1–3 mV) that occur aperiodically in the stratum radiatum of the CA1 area of the hippocampus during SWS and appear to be generated in CA3, an area with abundant recurrent connectivity [13–15]. Fast field potentials of 100–300 Hz, known as ripples, are often found riding on the waves of this depolarization. These high-frequency oscillations are the result of extremely synchronous neuronal activity [16, 17]. In fact, these events are often referred to as the most synchronous activity observed in the brain [18].

Active systems consolidation theory proposes instead that stable memory representations are formed through the repeated reactivation during sleep of networks that were active during waking behavior in both the hippocampus and neocortex [19–21]. Here, the emphasis is not at the level of single synapses, rebalancing their weight, but at the level of brain-wide networks, in which information is reorganized. The hippocampus is thought to be the primary encoder for declarative memories, embedding them in episodes, spatiotemporally organized representations of all the components of life occurrences [22–24]. The hippocampus is also thought to initiate the consolidation process [25], by which the memorized information is gradually embedded in cortical networks. This is of great importance for the fate of memories, because, whether or not the hippocampus is ever completely freed from memory maintenance duties [26, 27], cortical networks are thought to have a greater storage capacity, and their slower learning rates prevent catastrophic interference, which could result in the destruction of all memory content [19]. Thus, systems consolidation seems crucial for the long-term maintenance of a large body of memories. According to the standard view of systems consolidation, the hippocampus supplies information at this reduced rate by “replaying” information about previous experience (see [5, 28] for reviews) in off-line periods when the memory is neither encoded nor retrieved. Because of the relative absence of sensory interference and the rich, peculiar patterns of spontaneous activity, sleep poses itself as the ideal state for the expression of replay. Collective activity events such as sharp wave/ripple (SWR) complexes in the hippocampus and slow oscillations and spindles in the cortex facilitate the transfer of information from the hippocampus to the neocortex. As we will see below, slow oscillations from the neocortex synchronize hippocampal SWRs. As a result, hippocampal replay and cortical spindle events are concentrated around the same time points, providing a temporal frame for the consolidation process.

It is to be noted that homeostasis and active consolidation do not need to be mutually exclusive, rather active processes may take place on a background of passive, noise-reducing rescaling, thereby avoiding synaptic saturation [7, 29].

In this chapter, we will briefly summarize recent findings on hippocampal and cortical replay, in the context of the neurophysiology of cortico-hippocampal interactions and the theory of systems consolidation.

The Neurophysiology of the Consolidation Processes

While the idea about the underlying computational processes goes back to the 1970s and 1980s [30–32], in 1989 Buzsaki [33] proposed a model for the physiological processes underlying the forming and consolidation of memory traces. This hypothesis consists of two distinct stages. The first stage describes the initial formation of the memory trace. This is then followed by a second stage in which the memory trace is consolidated. In the first stage, an unstable memory trace is formed during exploratory behavior, characterized by theta activity. Theta in rodents is a 6–10 Hz oscillation that dominates electrical activity in the hippocampus when subjects are moving around and attentive. During theta, select CA3 pyramidal cells are potentiated by input from granule cells in the dentate gyrus. This selectivity is determined by the response of the granule cells. Those granule cells that respond most to the environmental events will have a high firing rate; thereby, the CA3 pyramidal cells on which the mossy fibers of these active granule cells converge will be potentiated more relative to the surrounding pyramidal cells. This process ensures the specificity of information storage [34]. The second stage is characterized by the formation of a stable, more permanent form of the memory trace, which forms once the exploratory theta-dominated behavior subsides and slower oscillations surface, dominated by highly synchronized CA3 population bursts. This CA3 excitation converges onto a subset of CA1 cells and potentiates these, resulting in sharp wave-ripple complexes in the LFP signal. The potent depolarization resulting from this highly synchronous activity enhances the synaptic efficiency of both the activated CA3 and CA1 cells. This process allows the past state of the network to determine the exact combination of cells that fire during the bursts, ensuring that the most recent relevant information is strengthened.

Segmentation in to two physiological regimes, separating acquisition from consolidation, provides a way to acquire new information without jeopardizing the stability of older memory (the “stability-plasticity” dilemma described by [19] as well as earlier neural network literature—see [35]).

The switch between the two states is dictated by the neuromodulatory state: while during wakefulness a high cholinergic tone favors encoding [36], the most conducive condition for such replay is an idle or sleeping brain state [29]. When the brain is off-line, the normally continuous stream of external stimuli is absent, leaving room for the undisturbed reorganization and strengthening of recent memories. Furthermore, during SWS sleep, acetylcholine levels in the hippocampus decrease, facilitating endogenous activity through an increase in recurrent connectivity [36].

Buzsaki’s model focused on the hippocampus; however, as mentioned previously, stable, long-term memory consolidation requires a dialogue between hippocampus and cerebral cortex. This latter structure also underlies significant dynamical changes between wakefulness and sleep; it may therefore be useful to define a “2-stage” model that includes the neocortex [37] and characterizes the functional and computational consequences of the intricate interplay that has been described in physiological studies.

Cortico-Hippocampal Interactions During Sleep

Sleep is characterized by the reoccurrence of two alternating phases: slow-wave sleep (SWS) and rapid eye movement (REM) sleep. During REM sleep, theta oscillations of 4–8 Hz are often observed (albeit not reliably in humans [38, 39]). During SWS a slow oscillation of 0.1–1 Hz dominates, interspersed by cortical spindles and hippocampal SWRs. SWS and REM sleep phases are discerned by clearly dissimilar neuromodulatory activity. The low cholinergic activity during SWS reduces tonic inhibition of incoming information in cortex, thereby facilitating communication with the hippocampus.

In contrast REM sleep is characterized by high cholinergic levels, as is the awake state. During REM sleep, the coupling between prefrontal and hippocampal area is greatly reduced [38–41], which could be used as an argument against its possible role in memory consolidation. The dominant activity pattern during SWS is a slow oscillation present throughout the neocortex and synchronizing large cortical areas [42, 43]. This oscillation is characterized by neural activity fluctuating from widespread depolarization to hyperpolarization, referred to as UP and DOWN states. The sudden transitions between these states can exercise a powerful influence on co-occurring oscillatory activity both within the cortex and other regions such as the hippocampus [44].

UP states are maintained by intrinsic intracortical excitatory feedback mediated by glutamatergic synapses [45]. They are therefore a quintessential cortical network phenomenon. DOWN states represent the termination of this excitatory feedback by inactivation of persistent Na+ currents and the activation of Ca2+-dependent K+ currents [46–48] or by synaptic depression [49]. Importantly, this termination is not due to increased inhibitory activity. Rather, inhibitory interneurons oscillate in phase with principal cells, maintaining a balance between excitation and inhibition throughout the oscillation cycle [50]. As the circuit recovers from the influences that terminated the preceding UP state, a new UP state may be initiated, triggered by miniature EPSPs, or by T-type Ca2+ and persistent Na+ current, and resuming activity in a pattern reflecting previously stored information [51] or intrinsic excitability properties of single neurons [52].

Slow oscillations can be observed in widespread cortical areas. It is possible that an area that appears to participate in this slow, synchronous activity actually acts as the pacemaker driving the other areas involved. Possible areas that could provide such a drive include the basal forebrain [53–55] and thalamus [45, 56]. However, the sparsity of thalamic projections to the entorhinal cortex, subiculum, and hippocampus [57] could limit the potential involvement of the thalamus in slow oscillations. Moreover, the fronto-caudal spread of the slow oscillations [43] and the time shifts between the neocortical and dentate gyrus oscillations [58] indicate that the dynamics underlying slow oscillations may be more intricate.

Slow oscillations and in particular the transitions between UP and DOWN states synchronize neural activity between distant cortical modules [59, 60]. This long range synchronization plays an important role in the realization of systems consolidation, which requires the establishment of connections across many modules [25]. Indeed, the presence of slow oscillations has been positively correlated with the strengthening of memory traces [61, 62]. In addition to intra-cortex coherence, slow oscillations also affect the activity of subcortical areas. Most remarkably, both the hippocampus and thalamus can be modulated in this manner through the slow oscillation [63]. The fast oscillatory events in these areas that relate to slow oscillations are sharp wave ripples (SWRs) in the hippocampus and spindles in the thalamus.

The influence of SWRs is not limited to the hippocampus. Ripples also appear to influence the specific spatiotemporal firing patterns of neocortical cells [41, 44, 64, 65]. Sharp wave ripples propagate to the cortex, resulting in the depolarization of prefrontal cells within 100 ms of hippocampal activity [41, 63–65]. In turn, although sharp waves are thought to be generated within the hippocampus, the timing of their occurrence appears to be modulated by the state of the neocortex, that is, whether it is in an UP or DOWN state. More specifically, even the exact timing of hippocampal activity and ripple events appears to be influenced by neocortical activity. Sirota [65] showed that cortical firing consistently preceded ripple events by 50–100 ms. Hippocampal activity [58] and membrane potentials of hippocampal cells [66, 67] are phase locked to hippocampal sharp waves, though interestingly with varying phase relationship depending the subfield: Isomura et al. [58] found that while activity in the dentate gyrus and CA1 peaks during cortical UP states, the activity in CA3 is larger during DOWN states. Inhibitory interneurons in CA1 have a more depolarized membrane potential during cortical UP states [66], most likely due to the influence of direct inputs from entorhinal cortex. For principal cells, the situation is more diverse: DG granule cells are depolarized during UP states, whereas CA1 pyramidal cells are hyperpolarized; meanwhile, cells in CA3 show diverse hyper- or depolarizing responses to UP states.

These results highlight the complexities of the hippocampal circuitry and of the interactions between cortex and hippocampus, which are yet to be completely clarified. Still there are some enticing speculations: as hypothesized by [33], hippocampal activity during the cortical UP states appears to facilitate consolidation. Select pyramidal neurons in CA3 fire while other cells in this area are suppressed through the activation of inhibitory cells, facilitating cell specificity in consolidation. These patterns of select activation and suppression appear to be mediated by the dentate gyrus. Granule cells transmit neocortical information at a gamma frequency to CA3 pyramidal cells through the potentiation of mossy fiber terminals [68]. Simultaneously these dentate cells also activate interneurons to achieve inhibition of the cells not directly activated [69]. The widespread inhibition observed in CA3 during the UP states is much less prominent in the DOWN state. As a result, activity is not limited to the rigid, predetermined patterns but instead allows for transient self-organized activity to emerge. The ripples that occur during DOWN states may serve to reorganize hippocampal information through the modification of intrahippocampal, subicular, and entorhinal connectivity without affecting neocortical targets. The nature of hippocampal activity in this case appears to depend on the state of the neocortex, which may regulate which patterns are replayed by the hippocampus depending on the current needs of cortical processing [37, 58, 70].

The other side of the loop, the flow of information from the hippocampus to the neocortex, is perhaps best supported by the coincidence of SWR events with transitions between DOWN and UP states in the cortex. [64] and [65] have shown that SWRs tend to coincide with the transition between DOWN and UP states, looking at diverse cortical areas. This is most likely due to the powerful excitatory input from hippocampal bursts helping to reenact activity in the recovering cortex during DOWN states. Vice versa, when looking at the medial prefrontal cortex, a neocortical area receiving direct afferents from CA1 and the subiculum [71], Peyrache et al. [63, 72] found that SWRs were also generated at an enhanced rate at the UP to DOWN transition. This may be understood by reminding the “bistable” nature of the slow oscillation process [49]. The same input can destabilize both the UP and the DOWN state, causing temporary settling in the other state. Indeed it has been observed [73] that other subcortical events, like stimulation of the ventral tegmental area, can trigger UP states or that K-complexes (a fast UP-DOWN-UP state alternation) may be generated by a brief sensory stimulus (e.g., auditory; [74]).

Closely associated to slow oscillations are sleep spindles, cortical LFP oscillations of roughly 7–17 Hz that tend to occur at the beginning of an UP state. These oscillations of thalamic origin appear to arise from the interaction between GABAergic neurons in the nucleus reticularis and glutamatergic thalamocortical projections. The former cells act as pacemakers, while the latter propagate the activity to cortical regions [29, 47, 75, 76]. The inhibitory and excitatory thalamic cell ensembles involved in the generation of spindles differ per spindle, as shown by Sirota et al. who observed remarkable variability in spindle power over spindle episodes [64].

In many studies, spindles have been associated with memory consolidation processes, as their occurrence is related to successive memory retention performance (see, e.g., [77–79]). Moreover, spindles tend to be phase locked to hippocampal [80] and parahippocampal [81] ripples. For those reasons, spindles have been speculated to be conducive to replay (or in any case, to active memory consolidation processes) in the cerebral cortex [29]. This point deserves further scrutiny: in the only study where this was specifically analyzed, replay in the prefrontal cortex peaked on average before spindle episodes, rather than simultaneously [72]. Moreover, while membrane potential of cortical neurons is strongly entrained by spindles (see, e.g., [82]), spiking activity in the somatosensory cortex [82] and prefrontal cortex [63] does not seem to be increased during spindle episodes. In contrast, a strong recruitment of inhibitory interneurons, in particular in superficial layers, was observed. Possibly for this reason, prefrontal responses to hippocampal SWRs were found to be damped [63] during spindles. Taken together, these observations seem to speak against a direct role of spindles in the hippocampo-cortical-directed replay phenomena. Rather, the data suggest a role of spindles in “deafferenting” the cortex not only from sensory stimuli as traditionally hypothesized (see, e.g., [83]), but also from subcortically generated inputs, for example, from the hippocampus, by means of strongly recruited feed-forward inhibition. Based on these considerations, an alternative view of the function of spindles is the tagging of networks to be consolidated. Inhibition-mediated deafferentation effectively isolates the cortex from other major inputs that could cause interference. This ensures that the cells activated during the most recent sharp wave ripple remain the most predisposed to synaptic plasticity. Moreover, the selective enhancement of Ca2+ levels in cells during spindles [84] decreases the probability that extraneous information is consolidated.

Dynamics of Cortico-Hippocampal Replay

Memory replay has been thought to be generated by attractor-like dynamics [35]. In other words, replay occurs due to the effect of recurrent excitatory connections driving the activity state of a neural network to a “fixed point” dictated by the values encoded in the synaptic matrix and reflecting previous experience. Recurrent connectivity is especially strong in hippocampal subfield CA3, commonly taken as the first generator of replay processes [25]. However, it is also widespread in the cerebral cortex, lending credence to the role of attractor dynamics in spontaneous activity. Yet, in most simple models, local attractors are viewed as stable states of reverberating activity, the final states reached by a relaxation type of dynamics. That is, they coincide with the distribution of firing rates across neurons that the local network would tend to reach in the absence of new perturbing inputs. This is however not necessarily the case: other views of neural dynamics stress more the convergence to the attractor as the key phase in the process. Persistence of the attractor state may then be hampered by several dynamical factors, such as neural adaptation, synaptic depression [85], or by the changing inputs. Furthermore, interneurons may play a decisive role in the selection of which cell assembly is allowed to activate and in its successive disbanding. This would result in the transient emergence of attractor state, the sudden activation of a cell assembly, a tight integrated group of cells, which since the days of Donald Hebb [86] has been postulated to be the basic currency for information encoding. Cell assemblies were detected in the hippocampus with a targeted statistical analysis [87] as group of cells that were simultaneously activated within the time window of a gamma cycle (~30 ms). A similar dynamics was discerned in the medial prefrontal cortex, both during wakefulness and during sleep. In the studies of [88] and [72], rats performed a decision-making task on a Y-maze, while neural activities in the hippocampus and the prefrontal cortex were recorded. During task execution, these two structures oscillate coherently at theta frequency (6–10 Hz) in a behavior-dependent way: coherence is maximal at the “choice point” on the maze, and only after the task has been learned. Interestingly, the synchronization is accompanied by an increase in inhibitory efficacy of prefrontal interneurons. Because prefrontal interneurons, which receive direct afferents from the hippocampus [89], are fairly stably entrained by hippocampal theta [90], this increase in inhibitory synaptic efficacy may have the result of enforcing oscillatory coherence of pyramidal cells. Ultimately, interneurons may play a role in selecting which cells are “allowed” to fire at each gamma cycle and cause strictly timed co-firing of prefrontal cortex cells. Similar patterns of interneuronal activity resulted from prefrontal infusions of dopamine in anesthetized animals [88], which raises the possibility that neuromodulation has a handle on cell assembly formation through interneurons. Neural ensemble synchronization and the emergence of assemblies were analyzed in these studies by relatively simple statistical techniques, based on principal component analysis (PCA) of the population vectors formed by the activity of tens of PFC neurons recorded simultaneously. Each principal component denotes a group of cells that is more likely than chance to coactivate within the window selected for binning. Specially tailored statistical methodologies were used to find out which of these coactivate groups were indeed significant [91].

With these methods, a time series could be produced measuring the instantaneous degree of coactivation of these ensembles, enabling correlation with behavioral state on one hand and electrophysiological events on the other. In fact, during behavior, assembly synchronization was most likely to occur at the choice point and after the task rule was learned [88], that is, with the same behavioral correlates as hippocampal-prefrontal coherence in theta oscillations. One may speculate that both long-range coherence (hippocampal-prefrontal) and local synchronization (across prefrontal neurons) are generated during behavior as the dopaminergic tone increases, for example, at the time of the last action selection before obtaining reward (i.e., at the Y-maze fork) as the animal becomes more confident of the upcoming reward. Also, the enhanced synchronization may have the effect of favoring spike-dependent synaptic plasticity. In fact, in the sleep following task performance [72], cell assemblies that were active at the choice point were the most likely to replay. It is therefore possible that cell assembly synchronization helps select the activity configurations based on their behavioral relevance and their usefulness towards reaching a reward (as signaled by dopamine). These selected configurations would be the ones that would engender the most plasticity and remain more strongly stored in memory. The statistical methods used in this series of studies were designed to describe the exact time course of replay during sleep. Several conclusions could be reached from this analysis. First, replay only occurs during electrophysiologically characterized slow-wave sleep, which was also shown in [92].

Replay is assessed by measuring the similarity between neural activity patterns recorded during active experience and in the ensuing sleep. Even in the absence of any memory-related phenomena, however, this similarity will not be zero, because of activity correlations that are induced by existing, stable connections. Because of this, it is necessary to compare the obtained similarity value with a baseline level, computed by comparing activity during active behavior with what observed in a sleep period prior to the task.

In the experiment of Peyrache et al., a replay value higher than baseline was only measured in periods when slow waves were present in the local field potential.

Second, replay in the prefrontal cortex peaks during hippocampal SWRs, demonstrating the importance of hippocampal input for initiating replay processes. Third, replay was correlated to cortical slow oscillations in a peculiar way: replay peaked before delta waves, the local field potential correlate of a DOWN state. This somewhat surprising result may be explained in two ways. Either the final part of an UP state, when the intrinsic cortical recurrent drive is subsiding, makes the cortex more amenable to follow the hippocampal drive and replay information under hippocampal control, or the combination of a SWR and a replay event is strong enough to destabilize the UP state.

In fact, synchronizations both during behavior and during sleep [72, 88] are strong, rare events, characterized by power-law, fat-tail distributions of amplitudes and inter-event interval durations. With these probability distribution, there is a sizable chance to observe very large events, much more likely than with more customary distributions (e.g., Gaussian), where probability decays exponentially (or faster) with event size. These large events are the signatures of avalanche-like dynamics [93]: activity fluctuates randomly, until it finds a configuration that is uncommonly amplified by recurred feedback, up to the point where it can extend to large swaths of tissue. Thus, this peculiar behavior is another manifestation of attractor dynamics and of the convergence of dynamics towards one of a discrete set of states. A key question that will need to be addressed by further studies is the spread of these replay events. For system consolidation to occur, it is not enough that replay takes place in one cortical area. The prefrontal cortex is a critical structure for long-term memory and is thought to take on the “hub” role that is normally assigned to the hippocampus for recent memories. That is, the prefrontal cortex would contain the “index code” that connects representations spread out over the cortex, joining them into a unitary memory [21, 94]. However, for this to happen, it would be necessary for synchronization to spread from prefrontal cortex (and the hippocampus) to other cortical areas, in the sensory systems, as well as in the medial temporal lobe, which has to be tested in targeted experiments.

The study by Peyrache et al. [72] had a drawback in that it disregarded the temporal ordering aspect of neuronal activity. In fact, it has been known [95, 96] that hippocampal replay preserves not only the composition of synchronized cell groups but also the sequences in which these take place. This is important, as it may be an important drive for simulating and selecting among multiple possible sequences of action. In fact, this type of replay takes place also in wakefulness during small pauses in behavior [97–100], where it may play an important role in goal-directed behavior.

Replay of sequences has also been shown in prefrontal cortex [101] and the visual cortex [102]. A striking feature of these replay events is that they take place at a much compressed temporal rate (~7 times faster) with respect to the rate at which the same sequences are expressed during active experience.

Outlook: Cortico-Hippocampal Interactions as an “Engine” for Memory Reprocessing

The existence of a “dual memory system” with a fast learner for the initial encoding (the hippocampus) and a final, high-capacity store (the prefrontal cortex) is supported by a huge body of literature in the neurosciences of memory, which is not possible to review here in any detail. In the standard view, this dualism is needed to overcome the difficulties of combining flexible storage of new information with the maintenance of a large body of memorized information [19].

Yet, increasingly in recent years, emphasis has been placed on the qualitative changes that memories undergo as they are placed into a more general framework of previous memories [103, 104]. This involves not simply passive repetition of memory traces but an active process that reorganizes the newly encoded memory traces in order to strategically position them into a relevant context. The reorganization of the memory trace is said to facilitate the extraction of features from the trace, allowing the forming of new associations that can lead to novel inferences and insights [105]. The framework for characterizing such processes has existed for a long time and has been recently revived by a series of studies in rats, showing that even lower mammals are capable of organizing knowledge into “schemas” [106, 107], which facilitate the learning of congruent information. In these experiments, schemas appeared as a result of memory consolidation. The instantiation of these schemas requires the presence of the hippocampus. Their final locus of storage is most likely neocortical and is likely to involve the interaction between multiple cortical areas, with an important role reserved for the medial prefrontal cortex [106]. Little is known about the computational processes behind the formation of schemas, but it seems plausible that replay plays an important role in this. If that is the case, then it is most likely that the concept of replay will have to be extended from a verbatim repetition of previous experience to a more complex process, which may support the search for the common underlying structure of several episodic memories [108]. The search for such processes has just started, in particular in experiments on human subjects [109]. But finding traces of these processes in neural ensemble activity will likely require new, more precisely targeted experiments, as well as analytical techniques beyond the current state of the art. This search will not only be relevant for the traditional field of memory studies, but also will also be applicable in the broader context of the neuroscience field. For instance, the organization of perceptual and decision-making processes poses similar requirements for storing and analyzing complex information, which may very efficiently be performed off-line, in a manner similar to memory consolidation processes [110].