Abstract
Content maintained in visual working memory changes concurrent visual processing, suggesting that visual working memory may recruit an overlapping neural representation with visual perception. However, it remains unclear whether visual working memory representations persist as a sensory code through time, or are recoded later into an abstract code. Here, we directly contrasted a temporal decay + visual code account and a temporal decay + abstract code account within the temporal dynamics of the interaction between working memory and perception. By manipulating the ISI (inter-stimulus interval) between working memory encoding and a perceptual discrimination task, we found that task-relevant and therefore actively maintained perceptual information parametrically altered participants’ ability to discriminate perceptual stimuli even 4 s after encoding, whereas task-irrelevant information caused only an acutely transient effect. While continuously present, the size of this shift in discrimination thresholds gradually decreased over time. Concomitantly, the size of the bias in working memory reports increased over time. The opposing directions of threshold and bias effects are consistent with the local maintenance of information in perceptual areas, explained by a temporal decay + visual code account. As the maintained representation decays over time, its ability to alter incoming perceptual signals decreases (reduced threshold effects) while its likelihood of being impacted by those same signals increases (increased bias effects). Altogether, these results suggest that the readout of working memory relies on a sensory representation at a cost of increased interference by ongoing perception.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
Introduction
Working memory, the temporary maintenance and/or manipulation of information, is fundamental to various high-level cognitive processes and critical to producing goal-directed behavior. Simple daily tasks often require holding task-related information for seconds or minutes in working memory (WM) while concurrently processing new sensory input, and these two processes are known to interact with each other. Evidence for this interaction includes overlapping neural substrates for the processing and maintenance of visual information (e.g., Emrich et al., 2013; Harrison & Tong, 2009; Rademaker et al., 2019), and a bidirectional influence between WM content and a variety of perceptual tasks (e.g., Gayet et al., 2013; Olivers et al., 2006; Soto et al., 2005; Teng & Kravitz, 2019; Teng & Postle, 2021). Yet the temporal dynamics of this WM-perception interaction remain mostly unknown, despite the potential to facilitate an understanding of the interplay between these two fundamental cognitive functions.
When information is maintained in visual WM, it influences visual processing through enhancing bottom-up salience of a WM-matching stimulus (Gayet et al., 2013; Kiyonaga & Egner, 2016; Olivers et al., 2006), and distorts subjective perception in the form of an attractive or repulsive bias (Teng & Kravitz, 2019; Kang et al., 2011). Meanwhile, visual WM can also be disrupted by passive viewing of irrelevant distractors (Magnussen & Greenlee, 1992, 1999; Rademaker et al., 2015; Sun et al., 2017) or active visual processing in a secondary task (Fukuda et al., 2022; Teng & Kravitz, 2019; Teng & Postle, 2021; Teng et al., 2022; Saito et al., 2023). Consequences of distraction in WM are reduced precision (Rademaker et al., 2015; Sun et al., 2017; Bae & Luck, 2019) and/or presence of attractive (Rademaker et al., 2015; Teng & Kravitz, 2019) or repulsive (Bae & Luck, 2019) bias in memory recall. Further, the degree of WM-perception interaction depends on the similarity between WM content and visual stimuli in the feature space (Kiyonaga & Egner, 2016; Teng & Kravitz, 2019).
As WM is a dynamic process, information is maintained for only a brief duration of time, and then seemingly forgotten when it is no longer of use. One classic finding of forgetting in WM is the increase in error over time (Brown, 1958; Conrad, 1967; Cowan, 1988). Performance for color- and luminance-contrast WM drops rapidly with increasing interval size between encoding and probe (Cornelissen & Greenlee, 2000). Recognition accuracy for unfamiliar/unmeaningful shaped items held in WM also declined over longer intervals and eventually decreased completely, even when no distraction information was presented during the delay interval (Ricker & Cowan, 2010; Vergauwe et al., 2009). Types of forgetting errors include increased guess rate (Pertzov et al., 2017; Zhang & Luck, 2009), reduced precision (Rademaker et al., 2018; Shin et al., 2017), increased responses to non-target items (Pertzov et al., 2017), or target confusability (Schurgin et al., 2020). More recent insights come from network models that instead of a decline in representational strength, the increased variability in WM recall may be accounted for by diffusion (random drifts) in the encoded feature value over time (e.g., Panichello et al., 2019; Schneegans & Bays, 2018).
Beyond the behavioral, quantitative changes of WM representations over time (i.e., loss of information), there is also evidence to suggest that time may cause a qualitative change in the neural representation. On one hand, neuroimaging studies showed that information held in visual WM can be kept in visual cortex throughout the delay period up to 10 s in a code similar to that of perception (Albers et al., 2013; Harrison & Tong, 2009), even when facing passively presented distractors (Rademaker et al., 2019) or active perceptual discrimination (Hallenbeck et al., 2021). The recruitment of visual cortex for WM maintenance comes with a cost. Commensurate attractive biases in WM representations have been found in visual cortex (e.g., Hallenbeck et al., 2021; Lorenc et al., 2018). Yet on the other hand, there is evidence that under certain scenarios, WM representation may be transferred from a visual-like code into a more abstract code (e.g., verbal) to avoid interference. For example, the presence of visual distractors during a memory delay causes a decline in precision and an increase in categorical bias in orientation WM (Bae & Luck, 2019). In a different study, with increased stimulus onset asynchronicity (SOA) between memory encoding and the onset of visual search target, WM’s influence on visual search was found to be diminished (Han & Kim, 2009), which may be a result of increased cognitive control and/or a strategic usage of verbal code under the long SOA condition. Similar to these behavioral findings, physiological analyses proved that distractors caused a shift in the neural substrate recruited for maintaining orientation information from visual to parietal areas (Bettencourt & Xu, 2016), initiating a potential change in neural code for the representation in the parietal cortex (Rademaker et al., 2019). When a WM representation exists in different codes, it leads to unique consequences as result of the WM-perception interaction: when maintaining WM information in a verbal code was sufficient to complete the task, WM did not affect visual search performance; however, when the WM content must be maintained visually for successful future recognition, distractors matched with WM captured attention, causing interference (Olivers et al., 2006).
In the current study, we examined the time course of the interaction between visual WM and active ongoing perception, testing specific predictions of temporal decay and changes of representational code. The paradigm (adopted from Teng & Kravitz, 2019) consisted of a perceptual discrimination task embedded between WM encoding and report (Fig. 1). Participants memorized a low-level feature (color or orientation) and then made a same/different judgment on a set of two new features in the discrimination task. We previously found that the actively maintained information systematically altered the thresholds to reliably differentiate the two discrimination stimuli, and even task-irrelevant features of the discrimination stimuli created an attractive bias in WM report, providing support for the co-localization of perceptual processing and maintenance that lies at the heart of the sensorimotor recruitment framework. Here, we directly manipulated the ISI (inter-stimulus interval) between the memory cue and the discrimination to test the interaction between WM and perceptual processing at different stages of maintenance.
It is predicted that based on the temporal decay + visual code account, WM representation would be maintained in a visual format and persistently interact with perception through time. Thus, over short delays, the visual representation of the maintained information should be at its most stable and precise, exerting a strong impact on ongoing visual processing while being resistant to perceptual interference from concurrent sensory input (Fig. 2). With longer delays and decreased fidelity, the influence of the maintained information on perception would decrease and, critically, the memory representation also becomes more susceptible to perceptual interference. In contrast, based on the temporal decay + abstract code account, the visual representation degrades over time and then is shifted into an abstract/categorical code. Thus, the WM-perception interaction would only be observed at short ISIs but not for longer ISIs when the code changes, since WM maintained through verbal code does not influence visual processing (Olivers et al., 2006). The two accounts have two key distinctions in their predictions:
-
1)
Previous research has shown that the interference between WM and external visual processing is domain-specific (Bae & Luck, 2019; Lorenc et al., 2021). Thus, the interference should be more severe when the two share the same code (both visual) compared to different codes (one visual, one abstract/verbal). Under the visual code account, both the perceptual threshold shift and the WM recall bias would persist even at longer delays because of the continued use of shared resources, whereas the abstract code account would predict a drastic drop of both effects at longer delay when WM and visual perception no longer share the same representational code.
-
2)
While the visual code becomes more easily interfered with over long delays, the abstract code protects WM representations from interference. With the visual code deteriorating over time, visual input (the discrimination stimuli) entering the same system would be more dominating and cast greater interference on WM. A network model of WM (Bouchacourt & Buschman, 2019) showed that overlap of connections causes mutual interference among multiple WM items stored in a sensory network with recurrent connections to a random layer, and importantly, as memory degrades over time, WM representations drift towards each other to a greater extent. The increased attractive bias may serve as a mechanism to counteract the increasing noise towards a stable representation (Bouchacourt & Buschman, 2019; Chunharas et al., 2022). Empirical evidence also showed that serial dependence bias increases with longer delay (Bliss et al., 2017) and that higher noise leads to increasing attractive bias towards previously seen stimuli (e.g., Gallagher & Benton, 2022; van Bergen & Jehee, 2019). In comparison, the transfer of WM representation to a different coding scheme is hypothesized to increase distractor resistance in WM (e.g., Lorenc et al., 2021; Xu, 2017, 2020). Support for this prediction comes from evidence that WM representation in the parietal cortex was found to be unbiased by task-irrelevant distractors (Bettencourt & Xu, 2016; Lorenc et al., 2018) together with unimpaired behavior (Bettencourt & Xu, 2016). Thus, we predict a decrease of recall bias, rather than an increase of bias under the abstract code account.
To foreshadow, we verified the predicted temporal changes of the temporal decay + visual code account in the context of color perception and memory, showing that (1) task-relevant content in WM persistently changed early color perception; (2) as the ISI increased, the memory representation decayed and was more prone to perceptual interference; and (3) task-irrelevant content in WM had a brief impact on perception but then decayed quickly after encoding. These results constitute further behavioral evidence for a temporal decay + visual code account, while additionally providing critical insight into the temporal dynamics of WM maintenance.
Experiment 1
Experiment 1a deployed a dual-task paradigm to capture the influence of WM on perception as well as the interference upon WM from concurrent perceptual processing. Participants maintained the color of a colored Gabor patch in mind while performing a color discrimination task on a set of two new colored Gabor patches (Fig. 1). The relationship between the maintained item and the discrimination stimuli was manipulated. In one condition (“Middle”), the maintained item was between the two discrimination stimuli in color space (Fig. 3b). We predicted that the perception of both discrimination stimuli will be equally shifted toward the maintained item, making it more difficult to tell the two apart and resulting in a higher discrimination threshold. In the other condition (“Side”), the maintained item was to one side of the two discrimination stimuli in color space. We previously found that distractors more similar to the WM representation caused more attentional capture and biased the WM recall to a greater extent than less similar ones (Teng & Kravitz, 2019; also see Kiyonaga & Egner, 2016). Therefore, we predict that WM should shift the perception of the closer discrimination stimuli more strongly, widening the gap between them and leading to a lower discrimination threshold. The magnitude of this threshold shift between Middle and Side conditions therefore reflects the influence of WM on perception. Concurrently, the memory representation should be interfered with by the discrimination stimuli. In the Side condition, both discrimination stimuli attract memory in the same direction in color space and cause a bias in the WM report, whereas in the Middle condition, the two discrimination stimuli would have opposite effects on the memory representation and cancel out the memory bias (see also Teng & Kravitz, 2019, for further explanation).
To examine the temporal dynamics of this interaction, the ISI between the WM sample and the discrimination task was manipulated. The temporal decay + visual code account predicts that the maintained information will remain in its visual format throughout the delay period, decaying over time. Thus, the threshold shift between Middle and Side should be robust over short delays and reduced at longer delays. Further, the weakened WM representation would become more susceptible to incoming interference from the discrimination stimuli of the same code, and as a result, bias in the memory report should increase with time.
Experiments 1b and 1c served as control conditions to rule out alternative accounts. Experiment 1b tested whether the threshold shift could result from passive priming, as despite the irrelevance of these features, they are likely to cause short-lived perceptual effects, disappearing immediately without maintenance. Here, participants maintained the orientation rather than color of the Gabor, while performing a color discrimination task (Fig. 4a). Based on findings in Teng and Kravitz (2019), we predicted that the now task-irrelevant color information from the encoding stimulus would have only a brief transient effect on color discrimination, verifying the need for a maintained visual code in order to elicit the discrimination effect, as a transfer to an abstract code would create no interference whatsoever.
In Experiment 1c we examined whether the memory distortion bias requires active processing of the discrimination stimuli. Participants maintained color while performing an orientation discrimination (Fig. 4c). The visual code account predicts that the memory bias will occur even for the now task-irrelevant color of the discrimination stimuli because even passive perception of the stimuli engages the same perceptual modality in which the WM representation is maintained. Alternatively, a shift of code for maintained information would predict no such interference with low-level perception. Therefore, although the color of the discrimination stimuli is task-irrelevant, we still predict a significant bias in the Side condition.
Method
Participants
An a priori power analysis was conducted based on the smallest effect size reported in Teng and Kravitz (2019) using G*Power (Faul et al., 2009). To achieve 95% power and a two-tailed alpha of 0.05, the projected sample size was 45 participants to detect a Cohen’s d of 0.55 reported in the study. Thus, we decided to recruit 50 participants for each of the three experiments to be capable of detecting the predicted effects. Three separate sets of 50 Amazon Mechanical Turk workers (Exp. 1a: average age of 40.0 ± 9.5 years; 23 females; Exp. 1b: average age of 38.17 ± 12.28 years; 26 females; Exp. 1c: average age of 35.8 ± 7.1 years; 26 females) were recruited with compensation of $8 per hour for each of the three experiments. Data from participants who did not perform the discrimination task adequately (reached fewer than six reversals for any of the ISI conditions) were removed from further analysis. Further, we excluded data from participants whose absolute mean error in WM recall exceeded 90°. Participants gave informed consent approved by the Institutional Review Board of George Washington University and had normal color vision and normal or corrected-to-normal visual acuity.
Stimuli and procedure
Experiment 1a
A dual-task paradigm was used to test how information maintained in WM interferes with and is interfered with by an ongoing visual-processing task. While participants were memorizing the color of a Gabor patch, they performed a secondary color discrimination task. Each trial began with a colored Gabor patch (diameter of 125 pixels with five cycles) presented for 500 ms (Fig. 1a; memory sample). Subsequently, two new Gabors (diameter of 125 pixels with five cycles) were presented for 500 ms and participants made a judgment whether they were the same or a different in color by making a button press. The interval between the memory sample and the discrimination was set to be 100, 550, or 1,000 ms, leading to the three ISI conditions. A color wheel showed up 1,000 ms after the participants responded to the discrimination task, and they reported the memorized color by clicking on the wheel. The wheel was rotated randomly to avoid spatial coding of the memory color.
There were two key manipulations. First, we manipulated the relationship between the memory sample color and the discrimination colors with the Middle and Side conditions (Fig. 1b). The memory sample was set to be the center of the two discrimination colors in the Middle condition or on one side of the discrimination colors in the Side condition in the CIEL*a*b* color space (centered at L* of 70, a* of 20 and b* of 38, with a radius of 60). The distance between the two discrimination colors was determined by a staircase procedure (described in detail below). In the Side condition, the gap between the memory color and the close-by discrimination color was fixed to 9°. Second, we manipulated the ISI (100, 550, or 1,000 ms) between the memory cue and the discrimination stimuli to map out the time course of the interference effects between the memory cue and the discrimination task. This manipulation led to a two (Middle or Side) by three (ISI: 100, 550, or 1,000 ms) design. Trials for these six conditions were presented intermixed and participants completed 216 test trials in total (36 of each condition; the number of trials per condition was determined based on pilot testing and the data from the pilot was not included in our final dataset). To make the mechanics of the step function less apparent, 30 additional filler trials were added where the two discrimination stimuli had completely the same color and those trials were randomly placed in the experiment. These 30 trials served as the foil and were not part of the threshold calculation.
We used a staircase procedure separately for each of the six conditions to derive the discrimination threshold at which participants could reliably tell the two colors apart. The difference in color between the two stimuli was determined with a one-down, one-up stair-case method before the first reversal and then switched to a two-down, one-up to increase the efficiency in deriving the threshold. The color difference started at 18° in the color space. The size of the step (either decrease or increase) was determined as such: 3° for the first two reversals, 2° for the third to six reversals, and 1° for all the following trials. The upper limit for the color difference was 30° and the lower limit was 1°. It should be noted that we opted for a staircase procedure rather than the psychometric curve method in Teng and Kravitz (2019) because staircase is more efficient in estimating the threshold considering the additional conditions in the current study. Given the differences between the two measures, it is thus crucial to compare them side by side. For this reason, we included the same 1,000-ms ISI condition as a replication for the original study.
Experiments 1b and 1c
The displays for these two experiments were exactly the same as Experiment 1a except for the task instructions. In Experiment 1b, participants were told to memorize the orientation of the first Gabor and then perform a discrimination task on the colors of the following two Gabor patches (Fig. 3a). The color of the memory Gabor was task-irrelevant and was set to be in the center or to one side of the discrimination stimuli. The parameters of the color discrimination task were exactly the same as in Experiment 1a. The relationships between the memory sample and the discrimination stimuli in orientation and color space were manipulated simultaneously and in opposite directions: when the relationship in color was Middle, the relationship in orientation would be Side, and vice versa.
In Experiment 1c, the task dimensions were flipped so that participants memorized the color of the Gabor and then performed the discrimination task on the orientation of the two Gabor patches (Fig. 1a). The orientation of the memory Gabor could be in the center or to one side to the orientations of the discrimination stimuli. The color relationship was also manipulated and in the opposite direction to the orientation relationship. The staircase procedure described above was applied to the orientation difference between the two discrimination stimuli, which started at an 18° difference, with an upper bound of 30° and a lower bound of 1°.
Data analysis
Discrimination threshold
We followed the procedures in Leek (2001) to calculate the threshold independently for each condition. We removed trials before the second reversal and the threshold was taken as the average of color difference of all the rest trials. A reversal was defined as the point at which the staircase changes direction (either an incorrect response following more than two correct trials, or the second correct trial in a string of “wrong-correct-correct” trials).
Recall bias
We used a model-free approach to calculate bias in WM recall. For the Middle condition, bias was calculated as the mean error (signed) across trials. A positive bias means a general tendency of clockwise bias and a negative bias means a tendency of counterclockwise bias. For the side condition, we first sort trials by whether the discrimination stimuli were clockwise or counterclockwise to the memory color and then flip the counterclockwise trials and collapse them with the clockwise ones. Then we calculate the mean signed bias and a clockwise bias is deemed to be attractive towards the discrimination stimuli whereas a counterclockwise bias is deemed to be repulsive away from them.
Statistical analyses were performed using computer software JASP (JASP Team, 2023. Version 0.17.2). All multiple comparisons are Bonferroni–Holm corrected.
Results
Experiment 1a
As predicted, the discrimination threshold was affected by the memorized color (Fig. 3a). A two-way repeated-measures ANOVA with relationship (Middle, Side) and ISI (100, 550, and 1,000 ms) as factors revealed a significant main effect of relationship, with lower thresholds for the Side than Middle condition (F(1, 49) = 38.47, p < 0.001, ηp2 = 0.44). The main effect of ISI was not significant (F(2, 98) = 0.43, p = 0.65, ηp2 = 0.01), suggesting roughly equivalent discrimination performance across ISIs. Critically, the interaction between relationship and ISI was significant (F(2, 98) = 3.62, p = 0.03, ηp2 = 0.07), indicating that the difference in threshold was modulated by ISI. Pairwise comparisons showed a greater difference in threshold in the 100-ms compared with the 550-ms condition (t(49) = 2.82, p = 0.021, Cohen’s d = 0.40). The threshold shift in all three ISI conditions was greater than zero (all ts >2.29, ps < 0.049, Cohen’s ds < 0.32). These results are consistent with decreasing interference from the maintained content on ongoing perception as predicted by the temporal decay + visual code account.
In contrast to the threshold effect, bias in the memory report increased with time (Fig. 3b), consistent with the temporal decay + visual code account. A two-way repeated-measures ANOVA revealed greater bias in the Side condition than the Middle condition (F(1, 49) = 108.36, p < 0.001, ηp2 = 0.69). The main effect of ISI was also significant (F(2, 98) = 4.39, p = 0.02, ηp2 = 0.08), with larger biases at the longer than shorter ISIs. Further, there was a significant interaction between relationship and ISI (F(2, 98) = 3.27, p = 0.04, ηp2 = 0.06). Pairwise comparisons showed that the difference in bias between Middle and Side conditions was greater in the 1,000-ms condition than the 100-ms condition (t(49) = 3.28, p = 0.006, Cohen’s d = 0.46; all other ps > 0.43, Cohen’s ds > 0.15). The difference in memory bias in all three ISI conditions was greater than zero (all ts > 6.98, ps < 0.001, Cohen’s ds > 0.99). These results show that with a longer duration the memory representation weakened and was more likely to be affected by additional matching perceptual input. This result (summarized in Fig. 3c) suggests a long-lasting interaction between WM and ongoing perception and supports the predictions of a maintained visual format.
Experiment 1b
When color was task-irrelevant for WM, we still observed a significant main effect of color relationship (F(1, 49) = 9.25, p = 0.004, ηp2 = 0.16) in the discrimination threshold, as the threshold in the Side condition was lower than the Middle condition. This difference in threshold was further modulated by time (F(2, 98) = 4.02, p = 0.02, ηp2 = 0.08). Pairwise comparisons showed that the difference between Middle and Side conditions was greater in the 100-ms condition than in the 550-ms and 1,000-ms conditions (t(49) = 2.12, p = 0.04, Cohen’s d = 0.30; t(49) = 2.56, p = 0.01, Cohen’s d = 0.36, respectively). Importantly, the difference between Middle and Side conditions was only significant- for the 100-ms ISI (t(49) = 4.08, p < 0.001, Cohen’s d = 0.58), but not for 500 ms and 1,000 ms, (both p > 0.13, Cohen’s d < 0.22), demonstrating the transience of the effect (Fig. 4a).
To directly compare the effect of task-relevant and -irrelevant information on visual discrimination, a three-way mixed ANOVA was conducted for the combined data with task relevancy (relevant in Exp. 1a and irrelevant Exp. 1b) as a between-subject factor, and relationship and time as the within-subject factors. We found a significant main effect of relationship, as the threshold was lower in the Side condition than in the Middle condition (F(1, 98) = 44.48, p < 0.001, ηp2 = 0.31). Further, this difference between Middle and Side was modulated by task relevancy (F(2, 196) = 7.07, p = 0.01, ηp2 = 0.07), with a greater effect in the task-relevant condition (Exp. 1a) than the task-irrelevant condition (Exp. 1b). Time interacted with Middle versus Side (F(2, 196) = 7.29, p < 0.001, ηp2 = 0.07), such that the threshold difference was greater in the 100-ms condition than in the 550-ms and 1,000ms conditions (t(99) = 3.19, p = 0.002, Cohen’s d = 0.33; t(99) = 3.10, p = 0.003, Cohen’s d = 0.35). None of the other main effects or interactions were significant, (all ps > 0.35, ηp2 < 0.01). Therefore, consistent with our previous experiment, task relevancy modulated the effect on the difference in threshold (Fig. 4b).
For the memory report in Experiment 1b, as the memory dimension was orientation, the analysis was based on the orientation relationship between the memory sample and the discrimination stimuli. A two-way repeated ANOVA revealed a main effect of relationship that there was a significantly greater bias in the Side condition than the Middle condition, F(1, 49) = 10.84, p = 0.002, ηp2 = 0.18. The duration of ISI did not modulate the magnitude of bias and did not interact with the relationship. Thus, despite the general difference in bias between Side and Middle condition, its magnitude did not vary with time.
Experiment 1c
To examine how task-irrelevant color information during the orientation discrimination task influences color WM, we focused the analysis on the color memory report. There was a main effect of relationship that the bias in the Side condition was significantly greater than the Middle condition (F(1, 49) = 71.87, p < 0.001, ηp2 = 0.59). The main effect of time and the interaction were not significant (ps > 0.41, both ηp2 < 0.02). Therefore, regardless of task relevance, perceiving additional color input caused bias in the memory representation (Fig. 4c).
We then conducted a three-way mixed ANOVA for the combined data with task relevancy (relevant in Exp. 1a vs. irrelevant Exp. 1c) as a between-subject factor, and relationship and time as the within-subject factors. We found a significant main effect of relationship that the attraction bias was greater in the Side condition than the Middle condition (F(1, 98) = 176.29, p < 0.001, ηp2 = 0.64). Interestingly, the difference between Middle and Side was not modulated by task relevance (F(1, 98) = 0.55, p = 0.47, ηp2 = 0.02). None of the other main effects or interacts were significant (ps > 0.07, all ηp2 < 0.03; Fig. 4d).
For the threshold performance in the orientation discrimination task, there was a significant main effect of time (F(1, 49) = 6.97, p = 0.001, ηp2 = 0.13) that the threshold in the 100-ms condition was higher than the 550-ms and 1,000-ms conditions (t(49) = 3.03, p = 0.004, Cohen’s d = 0.43; t(49) = 2.95, p = 0.01, Cohen’s d = 0.42). The main effect of side was not significant (F(1, 49) = 3.05, p = 0.09, ηp2 = 0.06). The duration of ISI modulated the difference between the Middle and Side conditions (F(2, 98) = 4.22, p = 0.02, ηp2 = 0.08). Pairwise comparisons showed that the difference between the Middle and Side conditions were greater in the 100-ms condition than in the 1,000-ms condition (t(49) = 2.878, p = 0.01, Cohen’s d = 0.41). Further, the difference between the Middle and Side conditions was only significant for 100 ms ISI (t(49) = 2.71, p = 0.01, Cohen’s d = 0.38), but not for 500 ms and 1,000 ms (ps > 0.19, Cohen’s ds < 0.23), indicating that perceiving an orientation briefly altered the orientation discrimination threshold, consistent with the results in Experiment 1b (OSM Fig. 1b).
Discussion
Experiment 1a tested the temporal dynamics of the interaction between WM and perception. We found that the effect of WM upon perception decreased in its strength but remained robust across time, and as for the effect of perception upon WM, attraction bias in the memory report increased with time. Additionally, we observed a significant bidirectional interference confirming validity of the paradigm (Teng & Kravitz, 2019). These results indicate a decrease in stability in the memory representation over time so that its effect on perception becomes weaker and it is more susceptible to interference from additional perceptual input. They further suggest that information is maintained in a sensory code that persistently interacts with perception. Alternatively, if WM maintenance relies on an abstract code distinct from visual processing, this interaction should decrease in both directions over time, contradicted by the increase in memory bias we observed here.
The two control experiments, Experiments 1b and 1c, provided further insights by utilizing task-irrelevant discrimination stimuli with the goal of ruling out alternative interpretations for results in Experiment 1a that the observed effect could simply be attributed to factors such as priming or confusability. Experiment 1b demonstrated that irrelevant information did impact ongoing perception, but to a smaller extent. The influence was only robust at 100 ms but disappeared at 550 ms, suggesting that it is a transient effect. In addition, this finding speaks to previous studies that did not find WM’s influence on visual process (e.g., Bloem et al., 2018): if actively maintained memory dimension did not match that of the visual task, the irrelevant information in WM might decay more quickly than the actively maintained memory dimension and thus only had a brief effect on perception that disappeared with longer delay.
Experiment 1c, similarly to Experiment 1b, investigated the effect of task-irrelevant stimuli; however, Experiment 1c investigated whether or not passive perception could impact WM representation. It was found that attraction bias still existed throughout the time course, despite remaining unchanged, potentially due to the difficulty of the task. Since the interaction between task relevance and time did not reach statistical significance, we do not further speculate on the numerically different time courses between Experiments 1a and 1c (Fig. 4d). This result was congruent with previous findings (Teng & Kravitz, 2019) and provides additional support for the account that incoming sensory distraction alters the memory representation regardless of the level of processing and the amount of attention. These results provide further evidence against a separation in code between WM and visual perception because a separation would predict that irrelevant sensory input is less likely to influence WM.
Experiment 2
Method
Experiment 1 found evidence for the visual code account in the time course of the interaction between ongoing perception and maintained information shortly after encoding. Experiment 2 aimed to expand the quantification of the time course by including additional brief delays and delays closer to those found in neuroimaging studies (e.g., Harrison & Tong, 2009; Riggall & Postle, 2012; Serences et al., 2009). In Experiment 1a, the threshold shift decreased from 100 ms to 550 ms and remained stable for 1,000 ms. We thus added brief delays to quantify the changes between 100 ms and 550 ms. Moreover, as the time scales usually differ between different methods (e.g., neuroimaging studies typically rely on seconds of long delays), it is important to test that predictions of the visual code account hold at longer delays.
The paradigm was exactly the same as Experiment 1a, except for the ISIs between the memory cue and the discrimination task (Experiment 2a: 200, 316, 433, 550 ms; Experiment 2b: 2, 3, 4 s). Importantly, the purpose of these experiments is to provide (1) a quantification of the time course and (2) evidence that the interference effects are consistently present throughout delays from 100 ms to 4 s. We recruited two separate groups of Mechanical Turk workers (50 each) to complete the experiments. In Experiment 2a, the average age was 40.1 ± 12.8 years and there were 26 females. The average age in Experiment 2b was 37.3 ± 10.5 years and there were 31 females. Participants gave informed consent and reported normal color vision and normal or corrected-to-normal visual acuity.
Results
In Experiment 2a, using a repeated-measures ANOVA with relationship and ISI (200, 316, 433, 550 ms) as factors, we replicated the impact of maintained content on discrimination thresholds with a main effect of relationship (F(1, 49) = 30.86, p < 0.001, ηp2 = 0.39). No other main effects or interactions reached significance (all ps > 0.49, ηp2 < 0.02). An analogous ANOVA on bias revealed a significant main effect of relationship (F(1, 49) = 118.63, p < 0.001, ηp2 = 0.71). There was also a significant interaction between time and relationship (F(3, 147) = 2.90, p = 0.04, ηp2 = 0.06). Pairwise comparisons showed a trend of greater bias in the 316-ms condition than the 200-ms condition (t(49) = 2.52, p = 0.09, Cohen’s d = 0.43). For all these ISIs, we compared the threshold shift and difference in bias against zero and confirmed that they were all greater than zero (all ts > 2.42, ps < 0.03, Cohen’s ds > 0.34).
In Experiment 2b, a repeated-measures ANOVA on the discrimination thresholds with relationship and time (2, 3, 4s) revealed a significant main effect of relationship (F(1, 49) = 19.85, p < 0.001, ηp2 = 0.29). No other main effects or interactions reached significance (all ps > 0.27, ηp2 < 0.03). An analogous ANOVA on bias also showed main effects of relationship (F(1, 49) = 118.25, p < 0.001, ηp2 = 0.71) and time (F(2, 98) = 3.66, p = 0.03, ηp2 = 0.07). The interaction did not reach significance (F(2, 98)=2.49, p = 0.09, ηp2 = 0.04). For all these ISIs, the threshold shift and difference in bias were all greater than zero (all ts > 2.14, ps < 0.04, Cohen’s ds > 0.30).
Figure 5a and b provide a further quantification of the time course of the bidirectional interaction between working memory and visual perception, showing that both threshold and bias effects persisted at extended ISIs. In total, these results confirmed that WM content still maintained a bidirectional interaction with ongoing perceptual processing up to 4 s after encoding, inconsistent with a framework that suggests a shift of code. Moreover, the bias in the memory report actually increased over time, suggesting that the strength of the maintained information reduced and was more likely to be interfered with by incoming sensory input.
General discussion
In this set of experiments, we tested the temporal dynamics of visual WM representations over a wide range of times (from 0.1 to 4 s), by examining its interaction with ongoing visual perception. If the bidirectional interaction ceased to exist beyond the shorter intervals, evidence to suggest the temporal decay + abstract code account would hold weight. However, we found that the bidirectional interactions between the two processes persisted even with long intervals between memory encoding and the onset of the perceptual task, suggesting the temporal decay + visual code account. Additionally, for shorter intervals, changes in threshold shifts and bias went opposite directions: the influence of WM on perception (threshold shift) reduced with time, while the interference from perception on WM representation (bias) increased over time. This overall pattern is strikingly consistent with the temporal decay + visual code account that (1) the strength of the WM representation reduces over time and as a result, its influence on perceptual processing is weaker and becomes more susceptible to perceptual interference, and (2) we did not observe a sudden drop in the bidirectional interactions, suggesting that, at least in the context of this task, visual WM representation remains in a visual code and continually interacts with perception.
Previous studies have reported mixed results regarding the temporal decay in WM (Magnussen et al., 1990; Magnussen & Greenlee, 1992) and whether WM representations are robust against perceptual interference (Hallenbeck et al., 2021). One explanation for such discrepancy could be the flexible and strategic utilization of different codes of WM to optimize performance under variable task demands. The choice of code may be task-dependent, influenced by factors such as complexity and sensitivity of memory report method (e.g., change detection or continuous recall; whether the report requires fine perceptual report), salience of perceptual distractors, and memory load. Different neural codes of WM representations (e.g., sensory, categorical) have been demonstrated in a number of studies that there could be coexisting sensory representations and abstract, categorical representations along the visual hierarchy (Christophel et al., 2017; Kwak & Curtis, 2022; Rademaker et al., 2019). With the same memorandum, subjects relied on the abstract code in frontoparietal regions or the sensory code in visual processing regions depending on whether the WM report tested categorical information or fine visual detail, respectively (Lee et al., 2013). Thus, our current findings do not deny the possibility of recruiting an abstract/verbal code under certain task demands. However, our results do suggest that recoding of the visual WM representation into an abstract code is not obligatory: even when participants knew there was going to be a secondary task during the delay, making it optimal to proactively recruit the abstract code for the WM content, we still observed a persistent low-level interaction. It would be of interest for future studies to explore the factors that determine the format that WM representations are maintained in, and the cost and benefit of flexibly switching among different codes. In addition, continuous stimulus-specific information of features and objects in WM has been reported in frontal, parietal, and occipital regions in human (e.g., Cai et al., 2019; Christophel et al., 2017; Yu & Shim, 2017; and animal research (e.g., Buschman et al., 2011; Meyer et al., 2011) and one important future direction would be to determine the functional roles of these co-existing representations.
Regarding the mutual bias between WM and perception, the sources of attractive and repulsive biases and their functional relevance remains unclear. Using the dual-task paradigm in our current study and that of Teng and Kravitz (2019), we have observed attractive bias in all the experiments. However, both directions of bias have been observed in a wide literature of visual WM, visual perception, and WM-perception interaction across a variety of stimulus types and task structures (e.g., Bae & Luck, 2017; Brady & Alvarez, 2011; Chunharas et al., 2022; Fischer & Whitney, 2014; Kiyonaga et al., 2017). Functionally, attractive bias is proposed to stabilize visual perception/WM by integrating successive or simultaneously presented stimuli to counteract uncertainty arising from largest set sizes or noise accumulated across time, while repulsive bias is thought to differentiate similar visual or WM representations to increase their discriminability against each other (Chunharas et al., 2022; Czoschke et al., 2019; Kiyonaga et al., 2017). The two types of bias may reflect different computations within the visual system (Fritsche et al., 2020; Sheehan & Serences, 2022) and are sensitive to task demands (Bae & Luck, 2020; Teng et al., 2022). Attractive bias has been observed in both early sensory (Hallenbeck et al., 2021; Lorenc et al., 2018; Goettker & Stewart, 2022; John-Saaltink et al., 2016) and post-perceptual processing stages (Sheehan & Serences, 2022), whereas repulsive bias is mostly linked to low-level visual adaptation (Sheehan & Serences, 2022). One outstanding question is whether these two types of bias originate from perceptual or post-perceptual levels and how task demands may shape the direction of bias. Future studies may also investigate whether similar neural mechanisms subserve bias observed across different domains (i.e., visual perception and visual WM).
When encoding a specific feature of an item into WM, do task-irrelevant features of the same item exert an influence on sensory processing? This question touches upon an ongoing inquiry of whether task-irrelevant features are automatically encoded into WM alongside the task-relevant ones, as evidenced by some studies (Marshall & Bays, 2013). However, others suggest that task-irrelevant features may be encoded with lower fidelity and the process may not be fully automatic (e.g., Fougnie & Alvarez, 2011; Shin & Ma, 2016; Tam & Wyble, 2023). In the present study, results from Experiment 1b demonstrated that task-irrelevant features did influence perception shortly after encoding (100 ms after the offset of memory sample) but this effect waned over extended periods (550 ms and beyond). It is likely that the task-irrelevant feature is initially encoded into WM automatically and is subsequently discarded – either through a selectively removal process or due to withdrawn of attention. Nevertheless, because our study did not include a backward mask after the memory sample, the effect at 100 ms might equally be a reflection of perceptual priming, with no involvement of WM. Broadly speaking, these results are consistent with previous findings that did not find an influence of task-irrelevant feature on visual processing over longer delays (e.g., Olivers et al., 2006; Teng & Kravitz, 2019), suggesting that merely perceiving a feature may not be sufficient for its sustained maintenance within WM, and its subsequent influence on perception.
Another open question relates to prior TMS (transcranial magnetic stimulation) studies in which stimulating sensory regions failed to induce a substantial effect on WM performance (Rademaker et al., 2017; van Lamsweerde & Johnson, 2017), apparently counter to the sensory recruitment framework. However, if we consider the effect of TMS as adding random noise to the local circuit, the center of the populational response of WM would likely be unchanged. During retrieval/readout, it is still possible to reproduce the memorized orientation based on the peak of the activation. Nevertheless, this does not preclude the usage of TMS to examine the neural mechanisms of WM with a more sensitive approach, for the controversy surrounding the role of sensory regions in memory processes is often suggested to arise from methodological inconsistencies between relevant studies (Phylactou et al., 2022). With the current paradigm, adding noise to the sensory cortex would broaden the tuning of the population response and attenuate the difference in thresholds between the conditions. TMS could also be used to examine the interaction between the fronto-parietal cortex and visual cortex. Applying stimulation at different time points would allow for causal studies of when top-down signals are needed to stimulate WM maintenance.
To conclude, our previous work demonstrated that WM and perception are necessarily and intimately related through their shared representations. Here, we showed that the strength of WM’s influence on perception gradually decreases over time while it is simultaneously increasingly biased by perception. Our results support a temporal decay + visual code account, and that although multiple codes of visual WM representations may coexist, behavioral readout predominantly utilizes the visual code, even if it results in an increase in interference. Further, these results have potential implications for future EEG (Electroencephalogram) and TMS studies that require precise timing manipulation. Targeting different temporal windows might allow us to further tease apart the relative contributions of visual, parietal, and frontal cortices and the interactions among these circuits. As the timing of WM tasks is usually quite different across fMRI, EEG, and behavioral studies, understanding the changes of the WM representation over time is crucial to understand its mechanisms.
References
Albers, A. M., Kok, P., Toni, I., Dijkerman, H. C., & De Lange, F. P. (2013). Shared representations for working memory and mental imagery in early visual cortex. Current Biology, 23(15), 1427–1431.
Bae, G. Y., & Luck, S. J. (2017). Interactions between visual working memory representations. Attention, Perception, & Psychophysics, 79, 2376–2395.
Bae, G. Y., & Luck, S. J. (2019). Reactivation of previous experiences in a working memory task. Psychological science, 30(4), 587–595.
Bae, G. Y., & Luck, S. J. (2020). Serial dependence in vision: Merely encoding the previous-trial target is not enough. Psychonomic Bulletin & Review, 27, 293–300.
Bettencourt, K. C., & Xu, Y. (2016). Decoding the content of visual short-term memory under distraction in occipital and parietal areas. Nature Neuroscience., 19, 150–157. https://doi.org/10.1038/nn.4174
Bliss, D. P., Sun, J. J., & D’Esposito, M. (2017). Serial dependence is absent at the time of perception but increases in visual working memory. Scientific Reports, 7(1), 14739.
Bloem, I. M., Watanabe, Y. L., Kibbe, M. M., & Ling, S. (2018). Visual memories bypass normalization. Psychological Science, 29, 845–856. https://doi.org/10.1177/0956797617747091
Bouchacourt, F., & Buschman, T. J. (2019). A flexible model of working memory. Neuron, 103(1), 147–160.
Brady, T. F., & Alvarez, G. A. (2011). Hierarchical encoding in visual working memory: Ensemble statistics bias memory for individual items. Psychological Science, 22(3), 384–392.
Brown, J. (1958). Some tests of the decay theory of immediate memory. Quarterly Journal of Experimental Psychology, 10, 12–21.
Buschman, T. J., Siegel, M., Roy, J. E., & Miller, E. K. (2011). Neural substrates of cognitive capacity limitations. Proceedings of the National Academy of Sciences, 108(27), 11252–11255.
Cai, Y., Sheldon, A. D., Yu, Q., & Postle, B. R. (2019). Overlapping and distinct contributions of stimulus location and of spatial context to nonspatial visual short-term memory. Journal of Neurophysiology, 121(4), 1222–1231.
Christophel, T. B., Klink, P. C., Spitzer, B., Roelfsema, P. R., & Haynes, J. D. (2017). The distributed nature of working memory. Trends in Cognitive Sciences, 21, 111–124. https://doi.org/10.1016/j.tics.2016.12.007
Chunharas, C., Rademaker, R. L., Brady, T. F., & Serences, J. T. (2022). An adaptive perspective on visual working memory distortions. Journal of Experimental Psychology: General, 151, 2300–2323.
Conrad, R. (1967). Interference or decay over short retention intervals. Journal of Verbal Learning and Verbal Behavior, 6, 49–54.
Cornelissen, F. W., & Greenlee, M. W. (2000). Visual memory for random block patterns defined by luminance and color contrast. Vision Research, 40(3), 287–299.
Cowan, N. (1988). Evolving conceptions of memory storage, selective attention, and their mutual constraints within the human information-processing system. Psychological Bulletin, 104(2), 163.
Czoschke, S., Fischer, C., Beitner, J., Kaiser, J., & Bledowski, C. (2019). Two types of serial dependence in visual working memory. British Journal of Psychology, 110(2), 256–267.
Emrich, S. M., Riggall, A. C., Larocque, J. J., & Postle, B. R. (2013). Distributed patterns of activity in sensory cortex reflect the memory. Journal of Neuroscience, 33, 6516–6523. https://doi.org/10.1523/JNEUROSCI.5732-12.2013
Faul, F., Erdfelder, E., Buchner, A., & Lang, A. G. (2009). Statistical power analyses using G* Power 3.1: Tests for correlation and regression analyses. Behavior Research Methods, 41(4), 1149–1160. https://doi.org/10.3758/BRM.41.4.1149
Fischer, J., & Whitney, D. (2014). Serial dependence in visual perception. Nature Neuroscience, 17(5), 738–743.
Fougnie, D., & Alvarez, G. A. (2011). Object features fail independently in visual working memory: Evidence for a probabilistic feature-store model. Journal of Vision, 11(12), 3–3.
Fritsche, M., Spaak, E., & De Lange, F. P. (2020). A Bayesian and efficient observer model explains concurrent attractive and repulsive history biases in visual perception. Elife, 9, e55389.
Fukuda, K., Pereira, A. E., Saito, J. M., Tang, T. Y., Tsubomi, H., & Bae, G. Y. (2022). Working memory content is distorted by its use in perceptual comparisons. Psychological Science, 33(5), 816–829.
Gallagher, G. K., & Benton, C. P. (2022). Stimulus uncertainty predicts serial dependence in orientation judgements. Journal of Vision, 22(1), 6–6.
Gayet, S., Paffen, C. L., & Van der Stigchel, S. (2013). Information matching the content of visual working memory is prioritized for conscious access. Psychological Science, 24, 2472–2480. https://doi.org/10.1177/0956797613495882
Goettker, A., & Stewart, E. E. (2022). Serial dependence for oculomotor control depends on early sensory signals. Current Biology, 32(13), 2956–2961.
Hallenbeck, G. E., Sprague, T. C., Rahmati, M., Sreenivasan, K. K., & Curtis, C. E. (2021). Working memory representations in visual cortex mediate distraction effects. Nature Communications, 12(1), 4714.
Han, S. W., & Kim, M. S. (2009). Do the contents of working memory capture attention? Yes, but cognitive control matters. Journal of Experimental Psychology: Human Perception and Performance, 35(5), 1292.
Harrison, S. A., & Tong, F. (2009). Decoding reveals the contents of visual working memory in early visual areas. Nature., 458, 632–635. https://doi.org/10.1038/nature07832
JASP Team (2023). JASP (Version 0.17.3)[Computer software].
John-Saaltink, E. S., Kok, P., Lau, H. C., & De Lange, F. P. (2016). Serial dependence in perceptual decisions is reflected in activity patterns in primary visual cortex. Journal of Neuroscience, 36(23), 6186–6192.
Kang, M. S., Hong, S. W., Blake, R., & Woodman, G. F. (2011). Visual working memory contaminates perception. Psychonomic Bulletin & Review, 18, 860–869.
Kiyonaga, A., & Egner, T. (2016). Center-surround inhibition in working memory report center-surround inhibition in working memory. Current Biology, 26, 1–5. https://doi.org/10.1016/j.cub.2015.11.013
Kiyonaga, A., Scimeca, J. M., Bliss, D. P., & Whitney, D. (2017). Serial dependence across perception, attention, and memory. Trends in Cognitive Sciences, 21(7), 493–497.
Kwak, Y., & Curtis, C. E. (2022). Unveiling the abstract format of mnemonic representations. Neuron, 110(11), 1822–1828.
Lee, S. H., Kravitz, D. J., & Baker, C. I. (2013). Goal-dependent dissociation of visual and prefrontal cortices during working memory. Nature Neuroscience., 16, 997–999. https://doi.org/10.1038/nn.3452
Leek, M. R. (2001). Adaptive procedures in psychophysical research. Perception & Psychophysics, 63(8), 1279–1292.
Lorenc, E. S., Mallett, R., & Lewis-Peacock, J. A. (2021). Distraction in visual working memory: Resistance is not futile. Trends in Cognitive Sciences, 25(3), 228–239.
Lorenc, E. S., Sreenivasan, K. K., Nee, D. E., Vandenbroucke, A. R., & D'Esposito, M. (2018). Flexible coding of visual working memory representations during distraction. Journal of Neuroscience, 38(23), 5267–5276.
Magnussen, S., & Greenlee, M. W. (1992). Retention and disruption of motion information in visual short-term memory. Journal of Experimental Psychology: Learning, Memory, and Cognition, 18(1), 151.
Magnussen, S., & Greenlee, M. W. (1999). The psychophysics of perceptual memory. Psychological research, 62(2-3), 81–92.
Magnussen, S., Greenlee, M. W., Asplund, R., & Dyrnes, S. (1990). Perfect visual short-term memory for periodic patterns. European Journal of Cognitive Psychology, 2(4), 345–362.
Marshall, L., & Bays, P. M. (2013). Obligatory encoding of task-irrelevant features depletes working memory resources. Journal of Vision, 13(2), 21–21.
Meyer, T., Qi, X. L., Stanford, T. R., & Constantinidis, C. (2011). Stimulus selectivity in dorsal and ventral prefrontal cortex after training in working memory tasks. Journal of Neuroscience, 31(17), 6266–6276.
Olivers, C. N., Meijer, F., & Theeuwes, J. (2006). Feature-based memory-driven attentional capture: Visual working memory content affects visual attention. Journal of Experimental Psychology: Human Perception and Performance, 32, 1243–1265. https://doi.org/10.1037/0096-1523.32.5.1243
Panichello, M. F., DePasquale, B., Pillow, J. W., & Buschman, T. J. (2019). Error-correcting dynamics in visual working memory. Nature Communications, 10(1), 3366.
Pertzov, Y., Manohar, S., & Husain, M. (2017). Rapid forgetting results from competition over time between items in visual working memory. Journal of Experimental Psychology: Learning, Memory, and Cognition, 43(4), 528.
Phylactou, P., Traikapi, A., Papadatou-Pastou, M., & Konstantinou, N. (2022). Sensory recruitment in visual short-term memory: A systematic review and meta-analysis of sensory visual cortex interference using transcranial magnetic stimulation. Psychonomic Bulletin & Review, 29(5), 1594–1624.
Rademaker, R. L., Bloem, I. M., De Weerd, P., & Sack, A. T. (2015). The impact of interference on short-term memory for visual orientation. Journal of Experimental Psychology: Human Perception and Performance, 41, 1650–1665. https://doi.org/10.1037/xhp0000110
Rademaker, R. L., Park, Y. E., Sack, A. T., & Tong, F. (2018). Evidence of gradual loss of precision for simple features and complex objects in visual working memory. Journal of Experimental Psychology: Human Perception and Performance, 44(6), 925.
Rademaker, R. L., Van De Ven, V. G., Tong, F., & Sack, A. T. (2017). The impact of early visual cortex transcranial magnetic stimulation on visual working memory precision and guess rate. PLoS ONE, 12, e0175230. https://doi.org/10.1371/journal.pone.0175230
Rademaker, R. L., Chunharas, C., & Serences, J. T. (2019). Coexisting representations of sensory and mnemonic information in human visual cortex. Nature Neuroscience, 22, 1336–1344. https://doi.org/10.1038/s41593-019-0428-x
Ricker, T. J., & Cowan, N. (2010). Loss of visual working memory within seconds: The combined use of refreshable and non-refreshable features. Journal of Experimental Psychology: Learning, Memory, and Cognition, 36(6), 1355.
Riggall, A. C., & Postle, B. R. (2012). The relationship between working memory storage and elevated activity as measured with functional magnetic resonance imaging. The Journal of Neuroscience, 32, 12990–12998. https://doi.org/10.1523/JNEUROSCI.1892-12.2012
Saito, J. M., Kolisnyk, M., & Fukuda, K. (2023). Perceptual comparisons modulate memory biases induced by new visual inputs. Psychonomic Bulletin & Review, 30(1), 291–302.
Schneegans, S., & Bays, P. M. (2018). Drift in neural population activity causes working memory to deteriorate over time. Journal of Neuroscience, 38(21), 4859–4869.
Schurgin, M. W., Wixted, J. T., & Brady, T. F. (2020). Psychophysical scaling reveals a unified theory of visual memory strength. Nature Human Behaviour, 4(11), 1156–1172.
Serences, J. T., Ester, E. F., Vogel, E. K., & Awh, E. (2009). Stimulus-specific delay activity in human primary visual cortex. Psychological Science, 20(2), 207–214.
Sheehan, T. C., & Serences, J. T. (2022). Attractive serial dependence overcomes repulsive neuronal adaptation. PLoS Biology, 20(9), e3001711.
Shin, H., & Ma, W. J. (2016). Crowdsourced single-trial probes of visual working memory for irrelevant features. Journal of Vision, 16(5), 10–10.
Shin, H., Zou, Q., & Ma, W. J. (2017). The effects of delay duration on visual working memory for orientation. Journal of Vision, 17(14), 10–10.
Soto, D., Heinke, D., Humphreys, G. W., & Blanco, M. J. (2005). Early, involuntary top-down guidance of attention from working memory. Journal of Experimental Psychology: Human Perception and Performance, 31, 248–261. https://doi.org/10.1037/0096-1523.31.2.248
Sun, S. Z., Fidalgo, C., Barense, M. D., Lee, A. C., Cant, J. S., & Ferber, S. (2017). Erasing and blurring memories: The differential impact of interference on separate aspects of forgetting. Journal of Experimental Psychology: General, 146(11), 1606.
Tam, J., & Wyble, B. (2023). Location has a privilege, but it is limited: Evidence from probing task-irrelevant location. Journal of Experimental Psychology: Learning, Memory, and Cognition, 49(7), 1051.
Teng, C., & Kravitz, D. J. (2019). Visual working memory directly alters perception. Nature Human Behaviour, 3, 827–836. https://doi.org/10.1038/s41562-019-0640-4
Teng, C., & Postle, B. R. (2021). Understanding occipital and parietal contributions to visual working memory: Commentary on Xu (2020). Visual cognition, 29(7), 401–408.
Teng, C., Fulvio, J. M., Jiang, J., & Postle, B. R. (2022). Flexible top-down control in the interaction between working memory and perception. Journal of Vision, 22(11), 3–3.
Van Bergen, R. S., & Jehee, J. F. (2019). Probabilistic representation in human visual cortex reflects uncertainty in serial decisions. Journal of Neuroscience, 39(41), 8164–8176.
van Lamsweerde, A. E., & Johnson, J. S. (2017). Assessing the effect of early visual cortex transcranial magnetic stimulation on working memory consolidation. Journal of Cognitive Neuroscience, 29, 1226–1238. https://doi.org/10.1162/jocn_a_01113
Vergauwe, E., Barrouillet, P., & Camos, V. (2009). Visual and spatial working memory are not that dissociated after all: A time-based resource-sharing account. Journal of Experimental Psychology: Learning, Memory, and Cognition, 35(4), 1012.
Xu, Y. (2017). Reevaluating the sensory account of visual working memory storage. Trends in Cognitive Sciences., 21, 794–815. https://doi.org/10.1016/j.tics.2017.06.013
Xu, Y. (2020). Revisit once more the sensory storage account of visual working memory. Visual Cognition, 28(5–8), 433–446.
Yu, Q., & Shim, W. M. (2017). Occipital, parietal, and frontal cortices selectively maintain task-relevant features of multi-feature objects in visual working memory. Neuroimage, 157, 97–107.
Zhang, W., & Luck, S. J. (2009). Sudden death and gradual decay in visual working memory. Psychological science, 20(4), 423–428.
Acknowledgements
We would like to thank our colleagues in the Psychological and Brain Sciences Department of George Washington University for their helpful discussion. In particular, we thank Stephen Mitroff for his insightful comments on the manuscript. This research project was funded by the National Science Foundation (Grant/Award Numbers: BCS-2022572 to D.J.K and S.S).
Author note
De-identified data for all experiments posted on the Open Science Framework at https://osf.io/paqs8.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
There are no declarations of interest or known conflicts of interest to disclose.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Significance statement
This study demonstrated the long-lasting interaction between working memory and visual perception and highlighted the sensory nature of working memory representation.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Teng, C., Kaplan, S.M., Shomstein, S. et al. Assessing the interaction between working memory and perception through time. Atten Percept Psychophys 85, 2196–2209 (2023). https://doi.org/10.3758/s13414-023-02785-3
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.3758/s13414-023-02785-3