Keywords

Introduction

While watching a car pass by, you notice that the car is a two-door mini cooper colored in orange. Meanwhile, a labile transient trace about having seen the car is generated in your brain. After the car has disappeared from your field of view, this transient trace is either consolidated, and eventually becomes a stable representation of the experience, or fades out of your brain (Fig. 1). We experience a large number of events in daily life, yet only some of them are remembered later on. Why do we remember some events but not others? Forgetting can be due to shallow processing during encoding or inappropriate cuing during retrieval. Another possible reason is failed consolidation during the time interval between encoding and retrieval. Consolidation is a theoretical construct which has attracted more and more research interest in recent years (see chapter by Genzel & Wixted).

Fig. 1
figure 1

When you experience an episode, like seeing an orange mini cooper on the street, a specific neural engram pattern is established. When this perception is transformed into an episodic memory trace, the hippocampus binds together the different elements of that episode (here, for example, street, mini cooper and the color orange) by establishing links to already existing neocortical representations. Thus, the hippocampus functions like an index card in a library catalogue, listing which elements belong to an episode and where they can be found. After memory formation, the hippocampus may replay this engram pattern. The replay process may be triggered by sharp-wave ripple (SWR) complexes, thereby also activating the associated neocortical links, which in turn strengthens connections between the neocortical representations. This process has been described as memory consolidation. In the end, the original hippocampal memory trace may be weakened or even be lost (according to standard consolidation theory (Alvarez and Squire 1994); multiple trace theory (Nadel and Moscovitch 1997) suggests that hippocampal traces remain; see chapters by Genzel and Wixted, by Sekeres, Moscovitch and Winocur and by Cheng), but the memory is represented by strong connections between the neocortical representations of the elements. So, when the memory is cued by activating one element (e.g. the cue “street”, when someone asks you: “What did you see on the street that day?”), the strong connections to the other elements enable reactivation of engrams of other elements, thereby recollecting the entire episode. Without replay, however, hippocampal as well as neocortical representations become weaker and fade over time. Cueing one element will then not lead to successful retrieval of the associated elements, and the memory for that episode is lost

Full size image

Here, we assume that the replay of previously established neuronal activity patterns is critical for memory consolidation. Many researchers have studied replay in animals after it was reported for the first time (Pavlides and Winson 1989; Skaggs et al. 2007; Dave and Margoliash 2000; Nokia et al. 2010; Buhry et al. 2011). In these experiments, rodents are typically trained to navigate in a corridor or an open field while place cells firing to a set of temporally-sequenced place fields along the animal’s path are recorded. During subsequent non-locomotion periods of waking state and sleep, these place cells fire in the same order in a temporally compressed manner while maintaining the temporal sequence they exhibited during navigation (for review, Carr et al. 2011; Axmacher et al. 2009). Even before the first experimental study reported replay in rodents, David Marr had already proposed the idea that neurons that are active during encoding are reactivated afterwards during a consolidation period (Marr 1971). However, few empirical studies so far have directly investigated replay in humans, and crucial questions remain. In this chapter, we will first review the existing evidence for replay in humans and then specify which issues still need to be addressed.

What Is Replay and What Is Being Replayed?

According to the engram theory of memory proposed by Richard Semon around 100 years ago (Semon 1921, 1923), memories are formed as biophysical and biochemical changes of the brain, e.g., strengthened synaptic connectivity or synchronized firing pattern of neurons. Semon described four characteristics of the engram (Schacter 2001), which have recently been thoroughly reviewed by Josselyn et al. (2015). They are persistence, ecphory, content, and dormancy. We would like to add one more feature, which is uniqueness. If we can distinguish an SUV and a minivan at a behavioral level, the corresponding engram patterns differ at the neuronal level as well. From the reverse perspective, if engrams of two encounters of a car are the same, we would recognize the car as the same. For instance, Xue et al. (2010) reported an fMRI study in which participants viewed pictures of faces, with each face repeated multiple times. They found that brain patterns were more similar when viewing the same face compared with different ones, indicating stimulus-specific or unique representations. Furthermore, the level of similarity across repetitions was positively related to later memory performance for that face. An intracranial EEG (iEEG) study from our lab (Zhang et al. 2015) reported similar results while epilepsy patients were watching videos of navigating through virtual houses during encoding. During the subsequent testing period, patients were asked to find their way out of the same house. We found that the topographical patterns of brain activity in the gamma frequency band (around 80 Hz) were more similar when viewing the same, rather than different, video sequences. Interestingly, this was only true for when patients were able to remember the route in the virtual house. Together, these and other findings show that stimulus-specific representations of events (unique engrams) can be identified in humans on the level of distributed neural activity, i.e., via “engram patterns” (Watrous et al. 2015).

Given that memories are formed as engrams, we define replay as reactivation of memory engrams without the experience of the actual stimuli. Figure 1 illustrates this replay theory in a simplified way. As shown in the figure, a person first sees a ‘mini cooper’, colored in ‘orange’, on the ‘street’. During memory formation, in both hippocampus and neocortex, engrams consisting of representations of different elements (mini cooper, orange color, street) are formed. After memory formation, the engram pattern of the stimuli might be replayed when triggered by sharp-wave ripple (SWR) complexes (Kudrimoti et al. 1999; Nadasdy et al. 1999; Lee and Wilson 2002; Karlsson and Frank 2009) that are temporally linked to sleep spindles (Clemens et al. 2007; see chapters by Bergmann and Staresina on sleep-related oscillations and by Maier and Kempter on sharp waves and ripples). Thus, during replay engrams of elements are stabilized and connections between different elements are enhanced. In the meantime, the neocortex becomes more and more involved in engram representation of each item, and the engram in the hippocampus gradually fades out. As a result, when cued by ‘street’, people can remember the ‘orange’ ‘mini cooper’. Without replay of engram patterns, the engram—and the connection between different elements—fades out.

In general, replay can be divided into two formats—intentional and spontaneous replay. Intentional replay occurs when people voluntarily intend to repeat previously learned items, e.g., when a telephone number from a phone book is rehearsed in order to dial it. Intentional replay occurs during waking state and involves conscious effort. On the other hand, spontaneous replay occurs when previously acquired memory engrams reappear without any voluntary efforts. This may happen during both waking state and sleep. In this chapter, we will focus on the literature on spontaneous replay in humans.

Methods to Study Replay in Humans

As mentioned above, both the engram theory of memory and the idea of replay were proposed quite some time ago. However, empirical studies on the two phenomena were scarce until recently due to several reasons, including constraints of tools for data acquisition and limitations of analysis methods. Technological advancements have therefore had a significant role in enhancing the development of neuroscience in general, and the study of replay mechanisms for memory consolidation in particular.

An ideal tool to study replay, and one which may link human cognitive neuroscience with animal electrophysiology, consists of invasive recordings in epilepsy patients (Engel et al. 2005). These recordings provide either intracranial EEG (iEEG) data from clinical depth electrodes (“macroelectrodes” with a diameter of around 1–1.5 mm), electrocorticography (ECoG) data from subdural strip or grid electrodes, or local field potential and multi- or single unit data from “microelectrodes” (with a diameter of around 40 µm). Due to obvious ethical issues, these invasive methods can only be applied to a limited number of patients in a small number of hospitals around the world. In addition to being rare, patients implanted with macro- or microelectrodes by definition have a severe and often longstanding neurological disorder (e.g., a hippocampal sclerosis) and often take antiepileptic medications, both of which likely affect the results of these studies. Therefore, non-invasive neuroimaging tools, such as fMRI, MEG, and scalp EEG, offer an alternative to study replay. The difficulty with these methods is to identify the neuronal signature of an engram, so that its later reoccurrence can be detected.

It is known that our brain processes information via networks, or assemblies, comprising large numbers of neurons (Abeles 1982; Cohen et al. 1993; Palm 1990; Waydo et al. 2006). However, conventional mass-univariate approaches to data analysis neglect information that is represented by patterns across multiple voxels. In recent years, new multivariate approaches have been developed, such as pattern classification (Rissman and Wagner 2012; Haynes 2015) and representational similarity analysis (RSA; Kriegeskorte et al. 2008; Haxby et al. 2001). Both methods allow one to estimate item-specific activity, which is a prerequisite for studying replay of unique engrams.

First Evidence of Replay in Humans

Only a small number of studies have investigated replay in humans. At present, there are three fundamental lines of research to evaluate replay during memory consolidation. The first is searching for replay of item-specific engram patterns during memory consolidation; the second is using cuing experiments to study whether enhancing replay increases memory consolidation; and the third is applying electrical or magnetic stimulation to test how interrupting replay harms memory consolidation. In Sections “Replay During Sleep” and “Replay During the Waking Period”, we describe studies showing replay of engrams during a resting period, related to the first line of research mentioned above. In Sections “Cue Triggered Replay Enhances Memory and Interrupting Replay Disrupts Memory”, we address research lines 2 and 3 and review studies attempting to increase or decrease replay via cues or stimulation, respectively.

Replay During Sleep

Peigneux, Maquet and colleagues are among the pioneers to study replay during sleep (Maquet et al. 2000; Peigneux et al. 2004, 2006; see also chapter by Schönauer and Gais on the role of sleep for memory consolidation). In one of their studies (Peigneux et al. 2004), participants performed either a spatial navigation task or a serial reaction time task (SRTT) before sleep in the PET scanner. The spatial navigation task involved the hippocampus, while the SRTT did not. They found that, during the following sleep period, activity of the hippocampus was higher when participants had performed the spatial navigation task before sleep than when they had performed the SRTT. Furthermore, activation of the hippocampus during slow wave sleep correlated positively with performance on a spatial retrieval task the next day. A similar fMRI study, conducted by Bergmann and colleagues (Bergmann et al. 2012), had participants perform either a landmark-face association task or a visuomotor control task, before sleep (see also chapter by Bergmann and Staresina). Following the association task, the authors observed increased activation of category-specific face and landmark areas (the fusiform face area and the parahippocampal cortex, respectively), and of the hippocampus, compared with a control task. This replay was also temporally coupled with sleep spindles.

Although the studies mentioned above showed a replay-like effect, they did not directly assess whether the pattern replayed during the sleep period matched the engram pattern during memory formation. An fMRI study from our lab (Deuker et al. 2013) provides direct evidence for replay of item-specific engram patterns during the resting state and sleep. In this study, participants learned two sets of picture-location associations before and after an afternoon nap in the MRI scanner. A pattern classifier was trained to dissociate item-specific engram patterns, after which the re-occurrence of these engram patterns was tested on data acquired during a resting period including both waking state and sleep. We found that item-specific engram patterns learned before sleep were replayed more often than would be expected by chance. Most importantly, individual engrams that were replayed more often were afterwards remembered more accurately, indicating that replay is indeed behaviorally relevant.

Replay During the Waking Period

There is also evidence of replay during awake resting states after learning. Tambini and Davachi (2013) reported an fMRI study in which they measured multivoxel hippocampal patterns during the separate encoding of either object-face or scene-face associations. They found that task-specific patterns persisted into a post-encoding rest period. Again, the extent of pattern replay was positively related to later memory performance. In another fMRI study by Staresina et al. (2013), participants were shown a list of unique object-scene pairs during encoding. The authors found that the encoding engram pattern was replayed at a higher level if participants could successfully remember the object-scene pair in a cued recall task. One interesting aspect of this study was that during the interval between encoding and retrieval, participants performed a distractor task by judging odd/even numbers. Thus, the replay of encoding patterns is spontaneous rather than intentionally rehearsed. Together, these studies show that (1) unique engram patterns can be identified in human fMRI data, (2) replay of these engram patterns occurs spontaneously after encoding during both awake resting state and sleep, and (3) replay facilitates later memory, as would be expected for a neural correlate of memory consolidation.

Cue Triggered Replay Enhances Memory

Given that replay of previous engram patterns correlates with behavioral measures of memory consolidation, one may assume a causal role of replay for memory consolidation. In line with this idea, researchers have used cued recall tasks that selectively improve memory by presenting cues during sleep (Rasch et al. 2007; Rudoy et al. 2009; Diekelmann et al. 2011; van Dongen et al. 2012; Schreiner and Rasch 2015; see also chapters by Talamini and by Schreiner, Lehmann and Rasch). In a study by Rudoy and colleagues, participants were asked to learn fifty object-location pairs. Each pair was coupled with a specific sound. During the following non-rapid eye movement (non-REM) sleep period, half of the sound cues were presented to the participants through headphones. After waking up, participants viewed all previously learned objects and positioned each of them at their original location. The researchers found that objects for which sound cues had been presented during sleep were positioned more accurately than objects without sound cues during sleep. Van Dongen and colleagues, using a similar paradigm in an fMRI study, found that the right parahippocampal cortex was more active during periods of non-REM sleep when sound cues were presented than periods where sound cues were not presented (van Dongen et al. 2012) (see also chapter by Fernandez). During a subsequent retrieval session, they observed an inter-individual correlation between parahippocampal and medial prefrontal cortex connectivity and object-location memory. These studies show that cueing increases memory consolidation, and suggest that this is via an effect on replay. Indeed, presentation during sleep of odors that were associated with visual items presented in one hemifield specifically increased sleep spindle amplitudes over the contralateral hemisphere (Cox et al. 2014; see chapter by Talamini). However, none of these studies directly examined engram pattern replay. Future studies need to test the assumption that cueing during sleep indeed triggers replay of unique engram patterns.

Interrupting Replay Disrupts Memory

In rodents, replay is triggered by SWR events, which are oscillatory patterns in the mammalian hippocampus during immobility and slow wave sleep (O’Keefe 1976). Interrupting these SWR events impairs memory performance in rodents (Girardeau and Zugaro 2011; Girardeau et al. 2009; Ego-Stengel and Wilson 2010). In human studies, there is no direct evidence for this so far. However, studies have shown that interference with post-encoding processes, in task-related regions, impairs later performance (Muellbacher et al. 2002; Robertson et al. 2005). In a study conducted by Muellbacher and colleagues, participants were trained on a motor task, specifically involving the primary motor cortex (M1). Repetitive transcranial magnetic stimulation (rTMS) was applied to M1 and control regions between training sessions. The study found that the M1-rTMS condition resulted in reduced performance compared with control-rTMS. Another study by Robertson and colleagues showed that TMS on the primary motor cortex, specifically during waking and following a motor learning task, impairs learning performance (Robertson et al. 2005). However, none of the stimulation studies reported above directly examined whether replay of engram patterns was interrupted. This should be tested in future studies. Furthermore, in addition to these interference studies, artificially improving memory consolidation via electric or magnetic stimulation would be important as well, especially for clinical purposes (Lee et al. 2013). Ideally, new methods that can flexibly turn target neuron activities on and off without inducing any harm to human subjects are needed in this line of research.

Open Questions

As discussed, many aspects of replay still require scientific investigation. For the remainder of this chapter, we would like to discuss some open questions regarding replay, which are testable and may offer more insight in future studies.

The Difference Between Spontaneous and Intentional Replay

No study to date has addressed the difference between spontaneous and intentional replay. From the reviewed literature, however, it is likely that some form of replay supports different memory-related functions. During short-term memory, an intentional form of replay may support the rehearsal of previously presented information (LaRocque et al. 2013; Lepsien and Nobre 2007; Polania et al. 2012). With regards to cued or free long-term memory recall, the reinstatement of the engram of a stimulus may be necessary (Staudigl et al. 2015; Staresina et al. 2012; Polyn et al. 2005), and this could occur both intentionally and spontaneously (e.g., via ecphory; Waldhauser et al. 2016). Finally—and in the context of this chapter, most importantly—replay may support long-term memory consolidation. Notably, while all these processes involve “replay” of some previously established engram pattern, they may differ with respect to the format of this representation. Intentional replay involves vivid rehearsal of previously seen stimuli, which may rely on engram patterns within both early and associative sensory areas (Farah 1989; Mellet et al. 1998). By contrast, spontaneous replay during sleep is probably related to the integration of novel information into existing networks (e.g., Takashima et al. 2009). Thus, it may rely not only on replay of detailed perceptual representations within early sensory areas (Deuker et al. 2013), but also on replay of more conceptual and abstract representations within higher sensory areas.

Cued Replay and Spontaneous Replay During Wakefulness and Sleep

As mentioned above, the relationship between cued recall (see Section “Cue triggered replay enhances memory”) and spontaneous replay (reviewed in Sections “Replay during sleep” and “Replay during the waking period”) is still unclear. In particular, studies using cued recall have shown that the presentation of a cue during sleep, but not during waking, improves later memory performance (Diekelmann et al. 2011; Schreiner and Rasch 2015). By contrast, experiments investigating the spontaneous replay of engram patterns have shown beneficial effects of replay during waking periods as well (Deuker et al. 2013; Tambini and Davachi 2013; Staresina et al. 2013). There are several possible explanations for this discrepancy. First, during waking the replayed engram pattern is susceptible to external input, which may modify or even destabilize the original engram (Rodriguez-Ortiz and Bermudez-Rattoni 2007) in cueing experiments, as reactivation during waking has been shown to trigger reconsolidation processes under certain circumstances (see chapter by Kessler, Blackwell & Kehyayan). By contrast, spontaneous waking state replay may occur, and may support consolidation, when the relevant brain areas are not involved in an ongoing task, which protects brain activities from being interrupted (Tambini and Davachi 2013; Staresina et al. 2013). Second, cue triggered replay of the engram is initiated from the primary sensory cortex by processing the cue, which propagates from bottom-up to higher brain regions. By contrast, spontaneous replay may be initiated from higher level brain regions (e.g., triggered by hippocampal SWRs; Axmacher et al.2008). Third, the mechanisms underlying memory consolidation may differ between the waking state and sleep state—for example, memory consolidation during waking may depend on strengthening of individual memories, while consolidation during sleep may also occur if larger networks are activated that represent categorically related information (Oudiette et al. 2013). It may be that cueing during waking actually triggers such larger networks and thus does not improve memory consolidation, whereas spontaneous replay is more specific and therefore plays a beneficial role (Deuker et al. 2013; Staresina et al. 2013). This speculative idea needs to be further tested, though.

Network Replay in Humans Versus Sparse Replay in Rodents

In the rodent literature, replay was first described as the temporally compressed sequential firing of a set of place cells, which are mainly located in the hippocampus. In human studies, by contrast, researchers usually investigate “engram patterns” across large brain regions. The obvious reason for this difference is that recording methods differ between rodents and humans. In rodent studies, a limited number of electrodes are implanted in each animal. Thus, the reported results are relatively sparse compared to the total number of neurons in the brain. By contrast, human studies typically use methods such as fMRI or intracranial EEG, which cover large brain areas. Furthermore, replay in rodents occurs as temporally sparse events linked to sleep spindles and ripples, whereas it is considered a more sustained process in most human studies. No study to date has linked replay of engram patterns to hippocampal ripples in humans. In addition, it would be tremendously important to perform simultaneous recordings of single neurons, intracranial EEG and fMRI—both in humans and animals—to bridge the gap between the two research approaches (see Logothetis et al. 2012; Kaplan et al. 2016). This would allow one to test the relationship between sparsely replayed neurons in the hippocampus and replay of engram patterns in the neocortex.

Conclusion

In conclusion, there is now first evidence that engram patterns are spontaneously replayed during both sleep and resting state in humans, and that this replay supports memory consolidation. Other studies have demonstrated that cueing during sleep facilitates, and interrupting activity impairs, memory consolidation; but the relationships to engram replay still need to be tested directly. Other open questions concern the relationships between intentional and spontaneous replay, between sparse replay of single cell sequences and of engram patterns, between replay during waking and sleep, and the role of SWRs for replay.