Introduction

It is well established that the medial temporal lobe (MTL) and the hippocampus play a crucial role in declarative long-term memory [1,2,3]. Declarative memory has been posited to rely on processes such as pattern completion and separation [4], sequence encoding [5], and mismatch detection [6]. These processes enable memory to be flexibly encoded and retrieved so that information can emerge from experience. These hippocampal-dependent faculties have resulted in many theories to explain the ultimate goal and function of memory [1, 7]. It is still unclear if and how the hippocampus is involved in working memory. Although this region has been traditionally thought to subserve long-term declarative memory only [8], numerous lesion, electrophysiology, and human imaging studies have provided evidence that the hippocampus plays a key role in both the encoding and retrieval of working memory [9,10,11,12]. Nevertheless, examination of the different forms of working memory in detail revealed that working memory is not affected by temporal lobectomy [7]. These results indicate that the role and different components of working memory may show different effects in the hippocampus or in its different sub-regions. This is also consistent with the results of Piekema et al. [13] who showed that working memory in patients with medial temporal lobe epilepsy decreases when the integration of different features is required. However, if evaluated by the traditional digital memory span test, their working memory seems to be intact. The discrepancy between memory batteries and different working memory paradigms provides a window to study the different components involved in working memory and to extend the general knowledge of hippocampal functions.

In daily life, we encounter things and compare them with our stored representations. The ability to discriminate between previously encountered and novel stimuli is an essential mnemonic function that relies on multiple neural mechanisms [14,15,16]. Whether the incoming events match or mismatch what we expect can influence the ability of the hippocampus to switch between memory encoding and retrieval modes. Emerging evidence has revealed that, within the hippocampus, neurons discriminate between novel and familiar stimuli through a firing rate increase to previously encountered stimuli (match enhancement), or through a firing rate increase to novel stimuli (mismatch enhancement) [17,18,19].

Miller and Desimone’s research on monkeys confirmed the parallel neuronal mechanisms for visual short-term memory with a delayed-match-to-sample task. This revealed that in the perirhinal cortex some neurons are suppressed by any picture repetition, regardless of relevance, whereas others are enhanced, but only when a picture matches the sample [17]. An event-related functional magnetic resonance imaging study has revealed enhanced hippocampal activation for images that match a maintained goal. However, due to the task, mixing perceptual-mismatches with goal-matches is inevitable [20].

Mismatch enhancement effects—in electrophysiological, regional cerebral blood flow, and BOLD fMRI data—have garnered considerable attention, being linked to familiarity-based recognition decisions [21, 22], novelty detection [6, 23], and prediction error [24].

Ducan et al. have adopted a delayed match-to-sample task, with multiple intervening items between the sample and matching test stimulus, fully crossed whether stimuli match or mismatch a goal state or a previous perception. This indicated that relational match enhancement bilaterally in the posterior hippocampus was selective for matches between the probe stimulus and the goal state (consistent with the study of Hannula and Ranganath in 2008 [20]), but was not modulated by whether that goal was perceptually novel, whereas mismatch enhancements were influenced by perceptual novelty. However, no regions within the medial temporal lobe were found to show mismatch enhancements in response to goal-mismatches or subtle perceptual-mismatches.

In the endeavor to understand hippocampus’s role in memory encoding and retrieval, much evidence has been established. A hippocampal involvement in active maintenance was suggested by an fMRI study. With a delayed-match-to-sample task, right lateralized hippocampal activity was evident when participants had to maintain object-location associations, but not when they had to maintain object-color associations or single items [25]. In macaques, it has been found that enhanced gamma-band synchronization during encoding predicts greater subsequent recognition memory performance [26]. An fMRI study has shown a significant interaction effect between spatial processing and working memory in the right posterior hippocampus [27]. Ducan et al. proposed that the CA1 area is a match/mismatch detector [28].

In this study, we were interested in the working memory performance modulated by top-down influences and how the hippocampus plays its fluid role in relational memory encoding and retrieval. We explored the dynamics of goal match enhancement and perceptual mismatch enhancement with electrocorticographic (ECoG) recording, which provided an advantage to uncover subtle perceptual mismatch enhancement.

Methods

Participants

Intracranial field potentials were recorded from 120 depth electrode contacts implanted in 8 human epilepsy patients who suffered from pharmaco-resistant epilepsy in Xuanwu Hospital (5 male, 3 female). The history of epilepsy in these patients ranged from 5 years to 25 years. They were all right-handed with normal or corrected to normal vision and their IQs were in the average range. As part of a pre-surgical evaluation, all patients were stereotaxically implanted with depth electrodes from the occipital pole to the anterior temporal area along the hippocampal longitudinal axis. The electrode locations were based exclusively on clinical criteria. This electrode configuration was used for every patient and allowed us to collect neuronal response data from the same (or at least similar) brain region, with the same electrode order across patients, making grand average of event-related potentials (ERPs) possible [30]. At the time of recordings, patients received antiepileptic therapy. All patients underwent epilepsy surgery after recording (summarized in Table 1). None of the eight patients showed visible pathology, especially hippocampal sclerosis. No seizure occurred in any patient during the 24 h preceding the experiment. We included only patients with a depth electrode in a morphologically intact hippocampus (as defined by MRI). All patients gave informed consent before the experiment. For all eight subjects, although the standard anterior temporal lobectomy required a 3 cm resection of the hippocampus from the head to the body (here, the area of the first three electrodes at the hippocampus was removed), their seizure foci were not in the hippocampus and during the interictal period these electrodes did not show spike-like discharges. Moreover, their behavioral performance was above average (Table 2). None of the electrode data were excluded. Experimental procedures were conducted in accordance with the ethical principles of the 1964 Declaration of Helsinki. The Institutional Review Boards of Xuanwu Hospital approved all procedures.

Table 1 Surgical techniques and electrodes located within the resection area.
Full size table
Table 2 Behavioral performance measured by mean reaction time and accuracy.
Full size table

Stimuli

Stimuli were identical to those used by Duncan [29]. They were generated from 299 color drawings of objects. In each block, the scenes were different from one to another. In every scene, the stimulus was composed of a room and two objects. Each object was presented in one of the 9 possible locations defined by an invisible 3 × 3 grid. The room had black-and-white square patterned carpet/wallpaper to provide a feeling of depth and distance. The two objects (e.g., a sofa and a jacket) were presented in alignment with the carpet/wallpaper so that each object occupied one compartment of a grid defined by the carpet/wallpaper (Fig. 1). All the stimuli were generated in real time during the experiment using the Psychophysics Toolbox PTB-3 [31, 32] for MatLab (Mathworks, Natick, MA) on a MacBook Pro laptop computer positioned on a table over the patient’s bed.

Fig. 1
figure 1

Reconstructed electrode positions. A Coronal view of electrode contacts #1 to #8 implanted along the longitudinal axis of the left hippocampus. B Coronal view of electrode contacts #9 to #16 implanted along the longitudinal axis of the right hippocampus. C Electrode contacts 3 (red) and 11 (green) located in the bilateral posterior hippocampus.

Full size image

Recording

Depth ECoGs were referenced to linked forehead electrodes. Intracranial EEG was recorded with a clinical recording device (an SD32, sampling rate of 1024 Hz, for the first 4 patients or an SD128, sampling rate of 512 Hz, for the last 4 patients) from Micromed Inc (Italy). An Ag/AgCl scalp plate electrode was placed in the middle of the forehead as common reference. TTL (transistor-transistor logic) signals generated by the laptop computer running the experimental program were sent to the trigger-in port of the recording device through a USB AD/DA board (USB-1208FS, USA). This trigger information was recorded on a separate channel and helped to define the zero time point for data processing.

Procedure

MRI anatomical images were obtained with a SPRAGE sequence (1 × 1 × 1 mm3) before implanting the depth electrodes. Before the scan, a CRW (MN, USA) stereotaxic frame was fixed on the head of the patient, allowing calculation of the position of the hippocampus relative to the frame. This calculation was used to guide electrode placement during surgery. After implantation, high-resolution CT images (0.45 × 0.45 × 0.75 mm3) were collected. The structural MRI and CT images were co-registered and precise electrode positions were reconstructed from the CT images using AFNI software (http://afni.nimh.nih.gov/afni). Next, the electrode information was overlaid on the structural MRI images and visualized with MRIViewerRGL (http://zxliu.mit.edu), as illustrated in Fig. 1.

A modified version of the delayed-match-to-sample task was used. This paradigm dissociates goal match/mismatch from perceptual match/mismatch. The task began with the presentation of a room with two objects (Sample). After 500 ms the scene was replaced by a fixed cross. A verbal instruction from a loudspeaker lasting 700 ms informed the participant whether the trial was a match or switch task. Regardless of the task requirement, the participant was required to remember both the identity and location of the sample objects. After an interval of 300 ms to 1500 ms, the scene was presented again (Sample repetition) for 500 ms followed by a 1000 ms to 3000 ms delay. Then, a room containing the two probe objects (Probe) was presented for 2000 ms. The participant was required to respond within this 2000 ms time window. If the trial was a match task, the participant was required to discriminate whether the identity and location of the two probe objects was exactly the same as the sample objects. If the trial was a switch task, the participant was required to judge whether the two sample objects had switched locations. Thus, there were 4 possible object configurations in the Probe display: 1, the same objects at the same locations, called BOTH in the match task (correct response for this trial is ‘yes’) or PERCEPT in the switch task (correct response is ‘no’); 2, the same objects but switched locations, called NEITHER in the match task (correct response is ‘no’) or GOAL in the switch task (correct response is ‘yes’); 3, the same objects, with one in a new location, called LOC in both tasks (correct response is ‘no’); 4, the same locations, but one object is replaced by a new one, called OBJ in both tasks (correct response is also ‘no’). The last two conditions were designed for control purposes. The major difference between this modified version of the match or switch task and the original task was the repeated presentation of the sample after the auditory instruction (sample repetition) (Fig. 2). This modification was implemented to aid patient performance because, in general, their memory performance was impaired compared to normal participants, and they had difficulty performing the original version of the task. The durations of the displays and intervals used in the current version were also much shorter than those used in the original task because the temporal restrictions imposed by the use of fMRI were not present in our intracranial EEG experiment.

Fig. 2
figure 2

Sample of experimental conditions and predicted patterns of activation. A Trial structure and corresponding response types. The same Target and Probe may be categorized differently according to the verbal instruction of the current trial, as illustrated. B Predicted activation patterns for the four conditions (BOTH, NEITHER, PERCEPT, and GOAL) for a brain area showing the goal-match enhancement effect (upper panel) or the perceptual mismatch enhancement effect (lower panel).

Full size image

The experiment was arranged in separate runs composed of 126 trials. Each individual participated in 3 to 5 runs, depending on performance, in order to obtain no less than 80 [33] correct trials.

Data Processing

Intracranial EEG data were bandpass filtered (0.01 Hz to 40 Hz) and exported from the clinical recording device in ASCII format. The following data processing was performed using MatLab. The data were first notch-filtered at 50 Hz to reduce any artifact from the power source. The data from the first four patients were down-sampled from 1024 Hz to 512 Hz for consistency with the other datasets. Next, the continuous data were separately aligned to probe (the third trigger, referred to as the TTL 3 dataset) and sample repetition (the second trigger, referred to as the TTL 2 dataset) onset creating an epoch ranging from −200 ms to 1000 ms. After baseline (−200 ms to 0 ms) correction, trials with potential artifacts were rejected based on the amplitude of the data from 1 ms to 700 ms for each participant. The artifact rejection threshold for each participant varied individually and was defined manually by observation of all epochs. We managed to retain >80% of all trials after artifact rejection. The absolute values of ERPs for correct trials were averaged for each stimulus condition.

Results

Behavioral Results

Mean reaction time and accuracy are summarized in Table 2. Paired sample t tests revealed that the responses in the match task were faster than those in the switch task (t (7) = −3.608, P < 0.01). There was no significant reaction-time difference between ‘yes’ responses and ‘no’ responses (t (7) = −1.306, P > 0.1). The response time in the LOC condition was slower than in the BOTH and NEITHER conditions (BOTH vs LOC: t (7) = −2.844, P < 0.05; NEITHER vs LOC: t (7) = −2.395, P < 0.05). Other possible comparisons did not show any significant differences. Only correct trials were used in the following ERP analyses.

Goal Match Enhancement and Perceptual Mismatch Enhancement

In our behavioral results, the reaction time in the switch task was significantly longer than in the match task. In order to adequately measure a gradual accumulation of information about the enhancement patterns of the ERPs, to the TTL3 dataset (events occurring after the probe display), we first applied a t-test between the conditions GOAL and PERCEPT. The sample-by-sample t-test gave a time-resolved significance level for whether the local field potential (LFP) values differed across conditions. If at least 10 ms consecutive time points of the t-test trace crossed the significance threshold of 0.05, and GOAL > PERCEPT, that time window was defined as a potential significant interval. During the interval, we investigated the relationship between the conditions of BOTH and NEITHER with another t-test. If at least 10 ms consecutive time points of the t-test trace crossed the significance threshold of 0.05, and BOTH > NEITHER, the beginning time point of this interval was defined as the latency of the goal match enhancement. Otherwise, if at least 10 ms consecutive time points of the t-test trace crossed the significance threshold of 0.05, and BOTH < NEITHER, the beginning time point of this interval was defined as the latency of the perceptual mismatch enhancement. To further confirm the existence of enhancement patterns, a sample-by-sample two-way analysis of variance (ANOVA) was used to the four ERPs (BOTH, GOAL, NEITHER and PERCEPT) during the defined goal match enhancement interval or perceptual mismatch enhancement interval. The same analysis was performed at the level of individual participants. Data were pooled using a moving window that created 5-time-point bins and moved at a step size of one time point. In this single participant level test, we slightly loosened the constraint defining a significant interval from continuous 10 ms to 5 ms. The latency of the enhancement effect for the individual participant was defined in an interval when the GOAL condition was significantly larger than the PERCEPT condition, and then as the time point when the BOTH and NEITHER conditions that fitted the two enhancement patterns started to be significant.

According to the goal match enhancement prediction, if areas in the hippocampus were driven by the match of goal instead of perceptual novelty, the neural responses in these areas would be greater for BOTH compared with NEITHER and for GOAL compared with PERCEPT. The reconstruction confirmed that the electrode configuration was very stable and grand averaging across patients based on this configuration was reasonable [30]. Two similar time intervals were found (535 ms to 735 ms on electrode contact #3, and 520 ms to 590 ms on electrode contact #4) to be significant (Fig. 3). Reconstructed electrode contacts in anatomical MRI structure images showed that the two 8-contact depth electrodes reached from the occipital white matter to the temporal pole, going through part of the visual cortex and along the hippocampus longitudinal axis. The most anterior 4 contacts of each electrode were within the hippocampus. Contacts #3 and #4 were located in the left posterior hippocampus. The exact contacts in each participant varied a little but all were within the left posterior hippocampus.

Fig. 3
figure 3

Significant goal match enhancement confirmed by four conditions on electrode contacts #3 (A) and #4 (B). BOTH (blue), GOAL (red), NEITHER (green), PERCEPT (cyan); gray bar, interval with a significant interaction and conjunction of BOTH > NEITHER and GOAL > PERCEPT.

Full size image

A perceptually driven mismatch enhancement was revealed bilaterally in the hippocampus including both the anterior and posterior areas (electrode contacts #1, #2, #3, #4 and #9, #10, #11, #12). This enhancement effect occurred at different intervals on different electrode contacts, averaging from 376 ms to 555 ms (Fig. 4).

Fig. 4
figure 4

Significant perceptual mismatch enhancement confirmed by four conditions on electrode contacts #1, 2, 3, 4 and #9, 10, 11, 12. A–H Significant perceptual mismatch enhancement was confirmed at electrode contacts #1 (A), #2 (B), #3 (C), #4 (D) and #9 (E), #10 (F), #11 (G), #12 (H) by the two-step t-test and an interaction of the four condition. BOTH (blue), GOAL (red), NEITHER (green), PERCEPT (cyan); gray bar, interval with a significant interaction and conjunction of BOTH < NEITHER and GOAL > PERCEPT.

Full size image

Considering the variability among electrodes across participants, we divided the electrodes within the area of the hippocampus into four parts: left anterior (electrodes #1 and #2), left posterior (electrodes #3 and #4), right anterior (electrodes #9 and #10), and right posterior (electrodes #11 and #12). The goal match enhancement was found in the left posterior hippocampus with a latency of 524 ms. The latency of the perceptual mismatch enhancement varied across areas: left anterior 411 ms, left posterior 405 ms, right anterior 430 ms, and right posterior 409 ms. This finding is consistent with the above results based on single electrodes .

The perceptual mismatch enhancement began earlier than the goal match enhancement for all participants (latency of match enhancement vs latency of mismatch enhancement in left posterior hippocampus: t (7) = 2.43, P < 0.05).

A late perceptual match enhancement was found on electrodes #2 and #3 with a latency of 717 ms and 900 ms, which fitted a pattern of BOTH > NEITHER and GOAL < PERCEPT.

We checked the statistical significance of these effects through Monte Carlo permutations. For each electrode showing a significant effect, we computed 1000 random permutations of the observed effects in the four conditions with data pooled from all electrodes; for each permutation, we then conducted the same two-step t-test analysis to get a significant latency. We then sorted the latencies from the 1000 permutations and calculated the probability of the significant latency of the match and mismatch enhancement effects. For each electrode and each significant effect, the probability of the observed latency was <0.05. For the goal match enhancement, perceptual mismatch enhancement, and perceptual match enhancement, we conducted a multiple comparison of three-way ANOVA analysis with Bonferroni confidence interval adjustment [electrodes (16 electrodes) * task (match task or switch task) * response (yes or no)] during the significant intervals on each electrode separately. All of the comparisons showed a significant main effect of the electrodes factor. Moreover, the interaction of the three factors and the interaction between the task and response factors were also significant.

We also investigated the relationship between the effect of the enhancements and behavioral performance of individual participants. The latency of the perceptual mismatch enhancement effect found in the right posterior hippocampus was significantly correlated with the accuracy of memory performance (Fig. 5) (ρ = 0.74, P < 0.05).

Fig. 5
figure 5

Positive correlation between perceptual mismatch enhancement latency and correct percentage. Each square represents a single participant.

Full size image

Single-Trial Decoding

To explore the mechanism of the memory encoding, we defined the intracranial field potential (IFP) response as the range of the IFP signal, that is, max(IFP)-min(IFP), in a time window from 150 ms to 850 ms after sample repetition. The time window from 150 ms to 850 ms was chosen to account for the approximate latency of the previously found enhancement effect. The IFPs were divided into two groups, according to the task demand: match or switch task. We used a statistical learning approach [34] to quantify whether the information from IFPs available in single trials was sufficient to distinguish whether it is memory maintenance (match task) or simulation of a specific future event (switch task). A linear discriminant analysis was performed with the principal component analysis approach for dimensionality reduction in the classification. The output of the classifier was when the statistical significance P < 0.05 indicated the possibility that the test trials being correctly distinguished from match or switch task was above chance level. With the TTL 2 datasets, 6 out of 8 patients consistently showed a significantly reliable response in the hippocampus representing the task demand. More interestingly, we found that for all 6 patients, the IFP signal in the right hippocampus efficiently discriminated the task demand (Table 3). This finding is consistent with the hypothesis that the process of constructing a detailed representation of a novel and specific future event differentially engages the right anterior hippocampus compared with other forms of event simulation and recall [25, 35].

Table 3 Individual participant’s output of the classifier, P value, and electrode on which a significant response was found.
Full size table

Discussion

The design of the current paradigm allowed us to distinguish the different memory signatures by the pattern of LFP activation. In the object switch task, if the match enhancement was driven by the match between the probe and the goal, the activation evoked by the GOAL condition would be larger than that by the PERCEPT condition; otherwise, if the PERCEPT activation was larger, it would be more likely to indicate a match to the previous perception. However, for the goal match enhancement, the BOTH condition would evoke stronger activation than the NEITHER condition in the match task, while the reverse would be true for the perceptual mismatch enhancement. Consistent with our proposal and a previous study [29], we found goal match enhancement at electrode contacts #3 and #4, which were located on the left posterior hippocampus, and perceptual mismatch enhancement at electrode contacts #1, #2, #3, #4, and #9, #10, #11, #12, which included the anterior and posterior hippocampi bilaterally. The dissociation of different components of working memory at hippocampus revealed more information beyond previous electrophysiological research [18, 19], demonstrating that match enhancement is more related with goal match at the hippocampus and mismatch enhancement is more related with perceptual mismatch. Moreover, the current result, for the first time, showed that the goal match enhancement effect happened at ~520 ms after probe onset and perceptual mismatch enhancement occurred as early as 376 ms after probe onset. This differed from other studies in which goal match and perceptual match are not dissociated. It has been reported that immediate repetition of old items enhances neural activity at the left MTL between 250 ms and 400 ms, compared to new items and repetition of delayed old items, revealed by depth electrodes [36] or scalp EEG [37]. On the other hand, an MTL-P300 component has been found at a latency of ~450 ms in which a target (novel) visual stimulus evokes larger responses than standard by intracranial [38] or scalp recordings [39]. This is consistent with our study, in that the perceptual mismatch happens earlier than the goal match enhancement.

The accuracy was found to be positively correlated with the perceptual mismatch enhancement latency on the right posterior hippocampus. Unlike the other neuropsychological studies [40], none of our patients showed hippocampal sclerosis. Thus, their hippocampus was still functioning properly. This suggested that the latency of the perceptual mismatch enhancement at the right posterior hippocampus could indicate the performance of memory retrieval. This result showed the advantage of our approach in exploring the perceptual mismatch enhancement compared with the fact that fMRI research only found a subtle effect. In the single-trial decoding analysis, we also found the right hippocampus to be crucial for memory encoding. Six out of 8 participants showed that the task demand could be decoded from the IFP response on the right hippocampus. This supported the finding that the right hippocampus participates in memory maintenance (match task) and in constructing a representation of a specific future event (switch task) [34]. The result of the decoding might be compromised, because the dissociation between the switch and match task relies on a different mental manipulation. Rather than maintaining the sample when it was the match task and simulating a new scenario when it was the switch task, the participant tended to maintain the sample no matter what the task was, which may lead to a poor performance in decoding.

It should also be noted that in the current paradigm (derived from Duncan [29]), a combination of two streams of information, object identity and position, are involved in the task. Even without the trial-by-trial changes in task requirement, increasing numbers of studies have revealed that multiple features and combinations of features in working memory activate the hippocampus [13]. Finke et al. found that MTL lesions selectively reduce color-location associations, while color-only or location-only working memory is retained [41]. Using fMRI in normal participants, Piekema et al. demonstrated that the hippocampus is involved in maintaining object-location associations but is not activated in object-color association or single items [13]. Our results agree with these studies, in that object and location information are combined to finish the task requirements, no matter whether it is in a match task trial or a switch task trial. The convergence to binding with locations in the current data and previous studies [13] consists of the well understood role of hippocampus in spatial memory and navigation (see [42] for a review), though animal models were used in most of the studies. It will be of interest to test the results from the animal models on epilepsy patients before and after resection and to extend our understanding of the role of the hippocampus in different components of working memory and their interactions.

In summary, the left posterior hippocampus might be crucial for goal match enhancement, and both the left and right hippocampus may play an important role in perceptual mismatch enhancement, distinct from the repetition-induced perceptual match effect and novelty-induced mismatch enhancement. The goal match enhanced a potential at 520 ms, while the perceptual mismatch enhanced a potential as early as 376 ms, about 150 ms earlier than the goal match enhancement. The right hippocampus is of great importance for memory encoding and retrieval.