Steven Yantis has been a major contributor to our present understanding that selective visual attention to objects and locations involves interactions between deliberate, goal-directed strategies and autonomous neural responses to sensory input (Yantis, 2000; see also Chun, Golomb, & Turk-Browne, 2011; Lavie, 2010; Posner, 1980; Treisman, 1988; Yantis & Jonides, 1990). In other work, Yantis noted that models of working memory do not address how visual attention is deployed, or indeed whether shifts of attention within working memory and perception are mediated by the same mechanisms (Tamber-Rosenau, Esterman, Chiu, & Yantis, 2011). This is despite a body of work that has examined these issues (see, e.g., Awh, Vogel & Oh, 2006; D’Esposito & Postle, 2015; Gazzaley & Nobre, 2012), and it is certainly true of our own multicomponent model of working memory (Baddeley & Hitch, 1974), which only went so far as assuming that deliberate strategies are controlled by a limited capacity central executive and did not address perceptual attention. However, a subsequent revision of the model (Baddeley, 2000) led us to investigate visual working memory in greater depth, resulting in its elaboration to include interactions between goal-directed strategies and visual selective attention (Allen, Baddeley & Hitch, 2014; Baddeley, Allen & Hitch, 2011; Hu, Hitch, Baddeley, Zhang, & Allen, 2014). We briefly summarize some of this research before going on to report new findings. Although our present focus is upon visual working memory, we should perhaps note that we regard responsibility for the interplay between external and internal attention as a general characteristic of working memory in all modalities.

We use a visual working memory task in which participants view a short series of briefly presented colored shapes and are immediately probed on their memory for any one of them, using either recognition or cued recall. Sequential presentation differs from the more usual method of simultaneous presentation and has the advantage of yielding serial position (SP) curves. These curves allow fine-grained analysis of retention that help distinguish between effects of external stimulus-driven selection and those attributable to internally motivated, goal-directed control. SP curves in this task typically show a marked recency effect, sometimes combined with a modest primacy effect restricted to the first item (Allen, Baddeley, & Hitch, 2006, 2014; Brown & Brockmole, 2010; Hu et al., 2014). We interpret the recency effect in terms of rapid forgetting whereby representations of more recently presented objects interfere retroactively with the representations of earlier objects in working memory, possibly through overwriting (Allen et al., 2006).

Using this paradigm, we have explored the impacts of executive control on perceptual and goal-directed attention in visual working memory using three broad manipulations. In one series of experiments, we studied the contribution of executive processes by varying the cognitive load of a concurrent verbal task performed while encoding the visual memory items (Allen et al., 2014). Performance of a demanding concurrent task had a clear disruptive effect, impairing memory for all items apart from the very last, which was recalled at the same high level regardless of concurrent load. The absence of an effect on the final item suggests that encoding information in visual working memory is relatively automatic, while the impairment in memory for earlier items suggests that executive resources are used to offset retroactive interference (RI) and attempt to ensure these items remain active and accessible. We assume there is little to be gained by devoting limited executive resources to maintaining the most recent item because this is free from RI.

In a second strand, we examined the impact of perceptual selective attention by presenting a colored shape distractor soon after (e.g., 250-ms) the presentation of to-be-remembered items. Despite explicit instructions to ignore this “stimulus suffix,” we found it disrupted memory for the study items (Allen, Castellà, Ueno, Hitch, & Baddeley, 2015; Ueno, Allen, Baddeley, Hitch, & Saito, 2011; Ueno, Mate, Allen, Hitch, & Baddeley, 2011). The amount of interference depended critically on whether the suffix was drawn from the same set as study items or a noticeably different set. Thus, a “plausible” suffix with color and shape features from the same pool as study items caused more disruption than a suffix with distinctive color and shape features that never appeared in study items. Furthermore, when a plausible suffix was presented, intrusion errors in cued recall tended to consist of a feature of the suffix itself (Hu et al., 2014; Ueno, Mate et al., 2011). These intrusion errors suggest that a plausible suffix tends to draw perceptual attention and become encoded in visual working memory. This is further supported by the observation that a suffix with only one plausible feature produced the same amount of interference as a suffix with two plausible features (Ueno, Mate et al., 2011). We assume that in order to perform the memory task, participants form an attentional set for the pool of potential study items. Given that selective attention involves feature detection (Treisman, 1988), a distractor with one or more features that match the attentional set is likely to be selected and encoded in error. Building on this, Hu et al. (2014, Experiment 1) found that presentation of a suffix disrupted memory for the most recent items in a series while having no effect on earlier items, thus directly contrasting with effects of concurrent cognitive load, which emerged on memory for all items except the last (Allen et al., 2014). This, in turn, suggests a separation between the effects of perceptual selective attention and executive control, with selective attention acting as the gateway to visual working memory and executive control concerned with actively maintaining information once in visual working memory.

In subsequent work (Hu et al., 2014, Experiments 2, 3 & 4), we examined how the disruption caused by the physically salient input of a suffix distractor interacted with internally driven, goal-directed selective attention. Specifically, we examined whether participants would be able to strategically direct their attention toward a particular item within a sequence by informing them that correct recall of either the first or final item would be rewarded with more “points” in a notional reward scheme. We found a strong recency effect in all conditions, regardless of instructions. Over and above this, instructions to prioritize either the first or last item enhanced that item’s recall, giving a substantial boost when it was the first item and a much smaller boost when it was the last item. Presentation of a post-list suffix distractor disrupted recall of recent items regardless of whether the final item was prioritized and, in addition, removed the boost to the first item when that particular item was prioritized.

We took these findings as suggesting that recent and prioritized items have a common status in working memory that renders them both more accessible for recall and yet more vulnerable to suffix interference. We interpreted this in terms of a limited subset of items occupying a transient “privileged state” within working memory. This position is broadly in line with previous claims that items in visual working memory can occupy fundamentally different states, with one or more items being retained in a focus of attention (e.g., Cowan, 1999, 2011; Oberauer & Hein, 2012), or in an active state capable of biasing attention selection (e.g., Olivers, Peters, Houtkamp, & Roelfsema, 2011). Our research suggests that this limited capacity state holds the most recently attended perceptual input together with optional, goal-relevant information, and that information in this state is vulnerable to overwriting when perceptual attention is drawn to a new stimulus (Hu et al., 2014).

However, our work so far has carried several assumptions concerning the role of executive control in driving perceptual and goal-directed selection that are as yet untested, and the present study aims to address some of these. First, we have assumed that executive control is critical for the goal-directed prioritization of items in visual working memory and, furthermore, that this is particularly the case for maintaining early items in a sequence, relative to the final item. We test these assumptions in Experiment 1 by examining whether the ability to prioritize the first versus final item in a sequence is reduced by concurrent performance of an executive-demanding verbal task. Second, we have assumed that the requirement to ignore a suffix involves some degree of executive control, even though the encoding of a suffix distractor into visual working memory by its drawing of perceptual attention is largely automatic (Ueno, Allen, et al., 2011; Baddeley et al., 2011). Experiments 2 and 3 test this possibility by examining whether a concurrent executive load influences the extent to which presentation of a suffix distractor disrupts recall from visual working memory.

Experiment 1

In our first experiment, we examined whether the increase in recall accuracy for a prioritized study item is reduced or even abolished by concurrent performance of a demanding secondary task, and whether this interacts with its sequence position. More specifically, we examined the effect of a concurrent load when prioritization is directed toward either the first or last item in the sequence. If the most recent item occupies the privileged state or focus of attention in a relatively automatic and cost-free manner, while earlier items require executive support (Allen et al., 2014), we would predict a substantial recency effect regardless of whether or not participants are prioritizing the final item (replicating Hu et al., 2014), and regardless of whether or not they are performing a demanding secondary task. We also expected to replicate the finding that prioritizing the first item results in a substantial boost to its recall, whereas prioritizing the most recent item would boost its recall only slightly. As in previous work, we expected a concurrent task to impair recall of earlier items the most. More critically, if prioritizing information in visual working memory depends on resources for executive control, the performance boosts deriving from prioritization should be particularly vulnerable to disruption from a demanding concurrent task.

We used counting aloud in twos from a randomly chosen two-digit number as the concurrent task involving high load and compared this with a low-load, articulatory suppression condition in which participants simply repeated the two-digit number over and over. These two tasks are well-suited for our present purpose because they differ in their demands on executive processes while being balanced for speech output, thus disrupting any tendency to utilize verbal recoding in the visual memory task to an equal extent.

Method

Participants

Twenty students (ages 18–27 years, mean age 23; 12 female, 8 male) from the Northeast Normal University were tested individually and were paid for participation. All reported having normal color vision.

Materials

The experiment was programmed in E-Prime (Version 2.0). Stimuli were colored shapes (approximately 3° × 3°) viewed against a white background on a 43.2-cm PC screen from a distance of 50-cm. Sets of four study items were selected from a pool of 64 items formed by crossing eight colors (red, blue, yellow, green, sky blue, purple, gray, and black) with eight shapes (circle, diamond, triangle, cross, arrow, star, flag, and arch). Selection was random, subject to the constraint that no shape or color could appear more than once among each set of four study items. The recall cue on each trial was either a color blob or a shape outline matching the color or shape of one of the four study items. These features were chosen randomly, subject to the constraint that each of the four SPs was cued equally often by color or shape in each block of trials.

Design and procedure

A repeated-measures design was implemented, manipulating concurrent load (low vs. high), strategy (primacy vs. recency), and cued SP (1–4). Concurrent load conditions were implemented in separate blocks of trials, counterbalanced between participants. Within these blocks, primacy and recency strategy instruction trials were run in separate blocks of 16 practice trials followed by 40 test trials, with half the participants taking the primacy instruction trials first for each load condition and the other half taking the recency trials first. Within each block the four SPs were each cued 10 times, distributed randomly over trials, and a short rest was given after 20 trials.

In an initial phase, participants were familiarized with the stimuli by being shown all the potential study items together with their proper names. Next, they were instructed that the memory task was to recall the name of the color (or shape) of the study item that had the same shape (or color) as the recall cue. They were also told that different numbers of reward points were assigned to each study item, depending on the experimental condition. In the primacy strategy condition, participants were informed they would receive four points for correctly recalling the first item and one point for each of the other three items. In the recency strategy condition, they were told they would get one point for correctly recalling any of the first three items and four points for the final item. Participants were informed that points were awarded with purely notional rewards.

The procedure for each trial is illustrated in Fig. 1. It began with a 500-ms warning cross followed by a 500-ms blank screen and a 1,000-ms number chosen randomly from the range 20 to 99. This was followed by a 250-ms blank screen and the four study items, each shown for 250-ms and separated by a blank interval of 250-ms. Each study item appeared at a different vertex of a 6° × 6° invisible square centered 3° above the middle of the screen, in an unpredictable spatial order. The final study item was followed by a 1,000-ms blank screen and then presentation of the recall cue, which was always shown 3° below the middle of the screen.

Fig. 1
figure 1

Time course on each trial in Experiment 1

Full size image

A verbal concurrent task was performed from the onset of the double digit until the onset of the test cue. In the low-load condition, participants were required to repeat aloud the two-digit number. In the high-load condition, they were required to count up from it in steps of two. The experimenter monitored concurrent task behavior to ensure compliance with instructions.

Results

Data are collapsed across cue type because there were no significant differences associated with type of cue (ps > .10). As expected, the SP curves for correct responses showed a pronounced recency effect (see Fig. 2a). Prioritizing the first item led to a large boost to its probability of recall, whereas prioritizing the last item gave only a modest boost to its recall. These observations were principally due to performance in the low-load condition (see Fig. 2b). In the high-load condition, recall was generally poorer (illustrated in Fig. 3a) and, importantly, strategy effects were almost absent (see Fig. 2c).

Fig. 2
figure 2

a Mean proportion of correct responses for each strategy condition in Experiment 1. b Effect of strategy in the low-load condition. c Effect of strategy in the high-load condition. Error bars denote standard error

Full size image
Fig. 3
figure 3

a Mean proportion of correct responses for each load condition in Experiment 1. b Effect of load with primacy instructions. c Effect of load with recency instructions. Error bars denote standard error

Full size image

A 2 (strategy) × 2 (concurrent load) × 4 (SP) ANOVA on correct responses revealed significant effects of strategy, F(1, 19) = 5.39, MSE = 0.08, p < .05, ŋ2 = 0.22, load, F(1, 19) = 40.74, MSE = 0.84, p < .001, ŋ2 = 0.68, and SP, F(3, 57) = 61.20, MSE = 3.09, p < .001, ŋ2 = 0.76. There were two significant interactions. One was Strategy × Position, F(3, 57) = 12.87, MSE = 0.36, p < .001, ŋ2 = 0.40, whereby the first and last items were recalled significantly more accurately when they were prioritized, t(19) = 5.01, p < .001, and t(19) = 2.09, p = .05, respectively, whereas other items were unaffected by strategy (see Fig. 2a). The three-way interaction was also significant, F(3, 57) = 3.50, MSE = 0.08, p < .05, ŋ2 = 0.16, reflecting a reduction in the strategy boost at the first SP in the primacy condition and the last SP in the recency condition, under high concurrent load (compare Fig. 3b and c).

The three-way interaction was broken down using separate 2 (load) × 2 (strategy) ANOVAs at each SP. These revealed a significant interaction at SP1, F(1, 19) = 8.45, MSE = 0.19, p < .01, ŋ2 = 0.31, reflecting a significant effect of concurrent load on recall of the first item with the primacy strategy, t(19) = 7.48, p < .001, d = 1.24, but not with the recency strategy, t(19) = 1.69, p = .11, d = .45 (see Fig. 3b and c). The first item was recalled significantly better with the primacy strategy in both load conditions, t(19) = 5.36, p < .001, d = 1.44, for low load and t(19) = 2.51, p < .05, d = .70, for high load, though the difference was much reduced in the latter case (see Fig. 2b and c). There were no significant effects in the 2 × 2 ANOVAs for SP2 and SP3, except a main effect of load for SP2, F(1, 19) = 7.49, MSE = 0.19, p < .05, ŋ2 = 0.29. In contrast, the interaction was significant at SP4, F(1, 19) = 5.76, MSE = 0.06, p < .05, ŋ2 = 0.23, reflecting enhanced recall of the last item with the recency strategy in the low-load condition, t(19) = 3.90, p = .001, d = .88, but not in the high-load condition, t(19) = .40, p = .69, d = .09 (see Fig. 2b and c). Examining load effects revealed significant concurrent task interference in the recency condition, t(19) = 2.61, p < .05, d = .81, but not in the primacy condition, t(19) = .42, p = .68, d = .09 (see Fig. 3b and c).

Errors were categorized as within-sequence confusions (i.e., recalling the shape or color of one of the other items presented on the trial in question) or intrusions (i.e., recalling a shape or color that was not presented on that trial). Omissions were rare in all the experiments reported here (<0.01 %). Overall, within-sequence confusions were about twice as frequent as intrusion errors (M = 40 % vs. M = 21 %).

Discussion

The results replicate the pattern of increased cognitive load effects at early rather than later sequence positions (Allen et al., 2014), and the increase in the accuracy of recalling primacy or recency items when they were prioritized through instructions (Hu et al., 2014). Beyond this, the way these effects interact casts new light on visual working memory and attentional control. As predicted, prioritization effects at both the first and final sequence positions were substantially reduced when participants engaged in a demanding concurrent verbal task, suggesting a key role for executive control. Furthermore, the form of the interaction between concurrent load and strategic priority across SPs was consistent with our earlier evidence that executive resources are more important for actively maintaining earlier items than the most recent item. Thus, primacy was only observed when instructions to prioritize the first item were combined with a relatively easy concurrent task, whereas recency was present regardless of strategy and the difficulty of the concurrent task. The further observation that recency instructions resulted in a modest boost to recall of the final item that disappeared when performing a demanding concurrent task is consistent with recency as a predominantly automatic effect that can nevertheless be supplemented by goal-directed executive processes. Finally, the interfering effect of concurrent load at the final position implies participants complied with the instruction to maintain verbal task performance throughout presentation of the study items.

Having provided firm evidence that resources for executive control are required for goal-directed attention in visual working memory, we turned next to examine whether these same resources are required for controlling shifts of stimulus-driven attention in visual working memory. This would bear on the answer to Steven Yantis and colleagues’ question regarding whether common mechanisms underpin shifts of attention in working memory and perception (Tamber-Rosenau et al., 2011).

Experiment 2

Our next experiment was designed to explore the role of executive control resources in mediating the impacts of stimulus-driven perceptual selective attention on visual working memory. First, we sought to confirm within a single cued recall experiment that verbal concurrent task difficulty and the presentation of a “to-be-ignored” visual stimulus suffix impact on earlier and later sequence positions, respectively (Allen et al., 2014; Hu et al., 2014). Second, and crucially, we examined whether these factors would interact, in line with the suggestion that ignoring a suffix depends on executive control (Baddeley et al., 2011; Ueno, Allen, et al., 2011), and evidence from other visual tasks that the ability to resist distracting stimuli is impaired under conditions of high cognitive load (e.g., Lavie, 2010). If this were the case, we would expect to find an increase in the extent to which a suffix interferes with recall when performing a more demanding concurrent task. We used a stronger manipulation of load than in Experiment 1 in that the demanding concurrent task involved counting backwards rather than forwards by twos.

Method

Participants

Twenty students (ages 19–32 years, mean age 23, 11 female, 9 male) from the University of York were tested individually. All participants reported having normal color vision and were paid or given course credit.

Materials

Stimuli were those used in Experiment 1 with the exception that on half the trials the study items were followed by a suffix. The suffix was selected randomly from the pool of 64 experimental items, subject to the constraint that neither its color nor shape matched any of the study items for that trial.

Design and procedure

A factorial repeated-measures design was used, testing all combinations of concurrent load (low vs. high), suffix (present vs. absent), and cued SP (1–4). The trial procedure was closely based on Experiment 1—the only change being what happened after presentation of the final target item. On half the trials a visual suffix was presented. These trials consisted of a 250-ms blank screen followed by a 250-ms presentation of the suffix at the center of the invisible square, followed by a 500-ms blank screen. This was followed by the recall cue, presented 3° below the middle of the screen, as before. To help participants differentiate the suffix from the preceding study items, it was accompanied by a 250-ms auditory beep. On no-suffix trials, the final study item was followed by a 1,000-ms blank screen followed by presentation of the recall cue. An auditory beep was played during the blank screen with the same timing as suffix trials. Participants were instructed to respond to the recall cue as before and to ignore the suffix. Unlike the previous experiment, there were no instructions to prioritize particular study items.

As in Experiment 1, a verbal concurrent task was performed from the onset of the double-digit number until the onset of the test cue. The only difference was that participants were required to count backwards in twos from this number (rather than forwards) in the high-load condition.

The load conditions were performed in separate blocks of trials with order counterbalanced across participants. Each load condition consisted of a block of 16 practice trials and four blocks of 20 experimental trials with short rests between blocks. The various permutations of suffix condition (2), cue type (2) and SP of the cued item (4) were randomly ordered within each block.

Results

As before, data are collapsed over the shape- and color-cue conditions, there being no significant differences associated with type of cue (ps > .10).

As expected, the SP curves showed recency over all items except the first, for which there was a small primacy effect. Figure 4 illustrates this and shows also that current backward counting impaired memory for early items but not for the last. In contrast, presentation of a suffix disrupted memory for the most recent items but had no effect on memory for earlier items, as shown in Fig. 5. A 2 (concurrent load) × 2 (suffix condition) × 4 (SP) repeated-measures ANOVA revealed significant main effects of load, F(1, 19) = 49.82, MSE = 1.00, p < .001, ŋ2 = .72, suffix, F(1, 19) = 46.38, MSE = 0.89, p < .001, ŋ2 = .71, and SP, F(3, 19) = 43.70, MSE = 1.90, p < .001, ŋ2 = .70. There were just two significant interactions. One was Load × SP, F(3, 57) = 4.23, MSE = 0.11, p < .01, ŋ2 = .18, reflecting significant effects of load at SPs 1, 2, and 3, t(19) = 4.46, 4.47 and 3.92, ps < .001 and p = .001, respectively, but no reliable difference at SP 4, t(19) < 1 (see Fig. 4). The second significant interaction was Suffix × SP, F(3, 57) = 10.45, MSE = 0.23, p < .001, ŋ2 = .36, reflecting significant suffix interference at SPs 3 and 4, t(19) = 4.53, p < .001 and t(19) = 7.46, p < .001, respectively, but no effect at SPs 1 and 2, t < 1 in each case (see Fig. 5). Importantly, neither the load by suffix interaction nor the three-way interaction were significant, ps > .10.

Fig. 4
figure 4

Mean proportion of correct responses as a function of concurrent load and serial position in Experiment 2. Error bars denote standard error

Full size image
Fig. 5
figure 5

Mean proportion of correct responses in each suffix condition as a function of serial position in Experiment 2. Error bars denote standard error

Full size image

Table 1 summarizes the means for the load by suffix interaction in the final column and shows that there was slightly more suffix interference in the high-load condition. We reexamined the interaction using data from SPs 3 and 4 combined because these are the items sensitive to suffix interference. The interaction remained nonsignificant in this more focused analysis, F(1, 19) = 0.35, MSE = 0.01, p = 0.56, ŋ2 = 0.02. We assessed the strength of support for the null hypothesis by means of a Bayesian analysis of the interaction comparison, using JASP software (Love et al., 2015). This gave a Bayes factor of 0.38, indicating a likelihood ratio of 2.6:1 in favor of the null hypothesis relative to the experimental hypothesis.

Table 1 Mean proportion of correct responses (and SE) as a function of concurrent task, suffix, and serial position (SP) in Experiment 2
Full size table

As in Experiment 1, errors were categorized as within-sequence confusions (i.e., recalling the shape or color of another study item) or intrusions (i.e., recalling a shape or color from outside the study set). Overall, within-sequence confusions were about twice as frequent as intrusions (M = 41 % vs. M = 21 %), as in Experiment 1. In addition, errors of recalling a feature of the suffix formed 45 % and 47 % of intrusions in the low- and high-load conditions, respectively (i.e., approaching twice the chance rate of 25 %).

Discussion

The results confirm the robustness of our previous observations of contrasting SP effects of a concurrent load and a poststimulus suffix on visual working memory. Thus, an irrelevant executive load disrupted memory for all items except the most recent, consistent with Allen et al. (2014), whereas an irrelevant suffix disrupted memory for only the most recent items, consistent with Hu et al., 2014 (Experiment 1). We also confirmed the observation that the presentation of a suffix tended to induce errors of recalling one of its features, consistent with our view that a suffix interferes by drawing perceptual attention.

We note that the suffix interference we find here is unlikely to reflect overwriting in a sensory store (Crowder & Morton, 1969) or perceptual grouping (Khaneman & Henik, 1977) because it extends to the first study item when prioritization instructions emphasize that item (Hu et al., 2014, Experiments 3 & 4).

The important new finding is that the amount of suffix interference was independent of concurrent load. Given previous evidence that the ability to ignore perceptual distractors is impaired under conditions of high cognitive load (Lavie, 2005, 2010), we had expected an increase in the amount of suffix interference when performing a demanding concurrent task. This argues against our earlier suggestion that attending selectively to a series of target items and rejecting a subsequent distractor places some demands on executive resources (Baddeley et al., 2011; Ueno, Allen, et al., 2011). However, the present results alone are not decisive, because Bayesian analysis showed they provide only borderline support for concluding that suffix interference is independent of concurrent cognitive load.

A further concern is that the experimental procedure may have allowed participants to form a strong temporal expectation of when the suffix and recall cue would occur, making it a useful strategy to stop repeating numbers aloud before this critical time window. If so, this could potentially explain the lack of increase in suffix interference with the more demanding concurrent task.Footnote 1 We think it unlikely participants did stop counting at the end of the list, because they were monitored by the experimenter to ensure compliance. In addition, Experiment 1 provides indirect evidence that participants continued counting because counting impaired memory for the last item when they were instructed to prioritize this item. However, it would clearly be more convincing if we could show objectively that participants maintained counting throughout presentation of the study items and thereby rule out the possibility that the absence of an interaction between suffix and load might reflect an artifact associated with being able to anticipate when the presentation sequence would end.

Experiment 3

Our final experiment reexamined the question of whether ignoring a visual suffix depends on resources for executive control using dual-task methodology but with two substantial changes from Experiment 2. First, we monitored the counting task to ensure that participants did not stop repeating numbers as the study list came to an end. Second, we made it difficult for participants to form a strong temporal expectation about the end of the study list by varying the number of study items. Thus, participants were given lists of three, four, or five study items, the number varying unpredictably from trial to trial. To avoid ceiling and floor effects and facilitate comparison with Experiment 2, we collected recall data for list-length four only.

We measured counting performance in terms of number of steps completed and accuracy. This gave us the opportunity to examine whether the number of steps increased linearly with list length, as would be expected if participants maintained a constant counting rate.

Method

Participants

Twenty students (ages 18–30 years, mean age 23, 14 female, 6 male) from the Northeast Normal University of China were tested individually. All participants reported having normal color vision and were paid for their assistance.

Materials

Stimuli were the same as in Experiment 2 except that lists contained three, four, or five study items.

Design and procedure

A factorial repeated measures design was used testing all combinations of concurrent load (low vs. high), suffix (present vs. absent), and cued SP (1–4).

The procedure was the same as Experiment 2 except that sequence length varied randomly from three to five across trials, and study items appeared at the vertices of an invisible regular pentagon (appropriate 6 ° × 6 °) in a predictable clockwise order. The bottom edge of the invisible pentagon was parallel to the x-axis and the first study item appeared at its peak.

There were four blocks of 25 experimental trials (10 for list-length three, 80 for list-length four, and 10 for list-length five) for each load condition. The 10 trials per load condition at lengths three and five were randomly selected (without replacement) from the 12/20 combinations of probe Type (2) × Suffix (2) × SP (3/5). Memory data for these trials were not analyzed.

The load conditions were the same as in Experiment 2 and were performed in successive blocks of trials in counterbalanced order. Participants were required to continue speaking until the onset of the probe for each trial and the experimenter noted their performance.

The experimental trials for each load condition were preceded by 12 practice trials. These comprised an equal number of trials at each of the three list lengths in an unpredictable order to generate a strong expectation of variable list length.

Results

Accuracy in the backward counting task was high, with a mean error rate of less than 5 %.

The mean numbers of steps completed per trial were 3.66 ± 0.20 for list-length three, 3.94 ± 0.21 for length four, and 4.22 ± 0.24 for length five. A 2 (suffix) × 3 (list length) ANOVA on these data showed a significant effect of list length, F(2, 38) = 43.58, MSE = 3.08, p < .001, ŋ2 = .70, no effect of the suffix and no interaction, ps > .10 in each case. It is noteworthy that the increase in number of steps completed from list-lengths three to four and from four to five were identical, suggesting participants maintained an even rate of counting throughout lists.

The recall data are once again collapsed over the shape- and color-cue conditions, there being no significant differences associated with type of cue (ps > .09). In all four conditions SP curves were characterized by recency with no primacy (see Fig. 6). Figure 6 shows that current backward counting impaired memory for all items in a sequence whereas, in contrast, presentation of a suffix disrupted memory for the most recent items but had no effect on memory for earlier items.

Fig. 6
figure 6

Mean proportion of correct responses as a function of concurrent task, suffix, and serial position in Experiment 3. Error bars denote standard error

Full size image

A 2 (concurrent load) × 2 (suffix condition) × 4 (SP) repeated-measures ANOVA revealed significant main effects of load, F(1, 19) = 44.35, MSE = 0.90, p < .001, ŋ2 = .70, suffix, F(1, 19) = 58.36, MSE = 0.99, p < .001, ŋ2 = .75, and SP, F(3, 19) = 39.59, MSE = 1.42, p < .001, ŋ2 = .68. There was only one significant interaction. That was Suffix × SP, F(3, 57) = 18.06, MSE = 0.37, p < .001, ŋ2 = .49, reflecting significant suffix interference at SPs 3 and 4, t(19) = 3.37 and 8.37, p < .01 and p < .001, respectively, but no effect at SPs 1 and 2, ps ≥ .09. The Load × Suffix interaction and the Load × SP interaction were both nonsignificant, F(1, 19) = 0.63, MSE = 0.01, p = .44, ŋ2 = .03, and F(3, 57) = 0.93, MSE = 0.03, p = .43, ŋ2 = .05, respectively, as was the three-way interaction, F(3, 57) = 0.19, MSE = 0.01, p = .90, ŋ2 = .01.

Table 2 shows the means for the nonsignificant load by suffix interaction in the final column. In order to run a more powerful test for the interaction, data from the first two items were discarded and data for the last two items were combined, as in the analysis of Experiment 2. A 2 (load) × 2 (suffix condition) ANOVA on these data showed significant main effects of load, F(1, 19) = 19.68, MSE = 0.36, p < .001, ŋ2 =.51, and suffix, F(1, 19) = 70.79, MSE = 0.83, p < .001, ŋ2 =.79. However, once again the load by suffix interaction was nonsignificant, F(1, 19) = 1.00, MSE = 0.01, p = .33, ŋ2 =.05, with the small difference in the size of the suffix effect in the opposite direction to that predicted. The Bayes factor for the interaction was 0.13, indicating “moderate” support for the null hypothesis with a likelihood ratio of 7.5:1.

Table 2 Mean proportion of correct responses (and SE) as a function of concurrent task, suffix, and serial position (SP) in Experiment 3
Full size table

As in Experiments 1 and 2, errors in recall were classified as either within-sequence confusions or extra-list intrusions. Once again, extra-list intrusions were much more frequent than within-list confusions (M = 43 % vs. M = 26 %), and a common type of intrusion consisted of recalling a feature of the suffix, accounting for 50 % and 58 % of intrusions in the low- and high-load conditions, respectively, t(19) = 1.27, p = 0.22.

Discussion

The results confirmed Experiment 2 in showing no increase in suffix interference when performing a demanding concurrent task. Replication of this outcome, with list length made unpredictable and counting performance measured, suggests that the absence of the interaction in Experiment 2 was not an artifact of participants anticipating when the study sequence would end and tending to stop counting. Moreover, when Experiments 2 and 3 are considered together by combining the independent Bayes factors, we see that support for the null hypothesis is substantial (with a likelihood ratio of 20:1). This suggests separate effects of cognitive load and suffix interference on visual working memory, whereby executive resources are not involved in ignoring a visual distractor presented immediately after encoding study items.

There were, however, some interesting differences in outcome compared with Experiment 2. Thus, while the recency and suffix effects had the same form, backwards counting disrupted memory for all study items rather than earlier items only, and the modest primacy effect observed previously was no longer present. The reduction in primacy was not unexpected because Crowder (1969) found this with unpredictable list length in verbal short-term memory, attributing it to a change in rehearsal strategies. We suggest that some analogous change in visual memorization strategies underpins the difference in outcomes when list length was predictable (Experiment 2) or unpredictable (Experiment 3). For example, the variation of dual-task interference with SP reported by Allen et al. (2014) suggests that participants typically remain passive for the last item, relying on automatic encoding. We assume this was the case when list length is predictable, as in Experiment 2 here. However, when list length is unpredictable, it is plausible to assume that participants will be less confident of remaining passive for the fourth item because it may not be the last. We already know from Experiment 1 that they can deploy executive resources to boost recall of the last item when instructed to prioritize it. If participants were to have done this spontaneously in Experiment 3, it would account for the extension of dual-task interference to the most recent item. Furthermore, this strategy would take executive resources away from actively maintaining earlier items and could therefore account for the reduction in primacy, too.

General discussion

The present experiments examined how executive control might contribute to two different aspects of attention in the operation of visual working memory, namely, stimulus-driven perceptual selective attention and internally driven, goal-directed control. We used a task in which participants were probed for recall of a single item from a short series. This task provides information about SP effects that have already proved useful in identifying separate and contrasting roles of perceptual selective attention, goal-directed prioritization, and executive control (Allen et al., 2014; Hu et al., 2014). The present experiments replicated and extended outcomes from these earlier studies and will be discussed in turn.

Experiment 1 demonstrated that particular items within a sequence can be prioritized to enhance their recall but that this process is dependent on the availability of executive support. Earlier items in a sequence are reliant on executive support to remain accessible, while the most recently encountered item tends to be automatically accessible (though this can be supplemented via executive support). How might we characterize the nature of the executive control mechanism that drives this active prioritization in working memory? One possibility is that executive control serves a general role in overseeing effective item rehearsal, ensuring that more specialized resources (e.g., space-based and object-oriented attention; see Shen, Huang, & Gao, 2015) are appropriately allocated. Along similar lines, executive control may support a process of active and sustained visualization (e.g., Phillips, 1983) that serves to maintain older items in the face of RI from incoming stimuli. Alternatively, the contribution of executive control might reflect a form of attentional refreshing (e.g., Barrouillet, Bernardin, Portrat, Vergauwe, & Camos, 2007; Camos & Barrouillet, 2014; Hollingworth & Maxcey-Richard, 2013; Souza, Rerko, & Oberauer, 2015) or recirculation through the focus of attention (Cowan, 1995) that is distinct from verbal rehearsal (Camos, Lagner, & Barrouillet, 2009; Raye, Johnson, Mitchell, Greene, & Johnson, 2007). Regardless of the specific nature of this process, it is typically more important for earlier sequence items, particularly when they are identified as a goal-relevant priority. The increased attentional demands imposed by our concurrent task manipulation would then mean that participants were less able to keep these items active.

Experiments 2 and 3 indicated that an external stimulus suffix and a demanding concurrent task have separate effects on visual working memory. Thus, having to ignore a suffix distractor interferes with memory for recent items only, whereas a concurrent cognitive load disrupts memory for earlier items (Experiment 2) or all items (Experiment 3), depending on task context. In both experiments the size of the suffix effect was independent of cognitive load, suggesting that ignoring the suffix distractor involves automatic, stimulus-driven perceptual selection. Although previous work has suggested that attentional capture is subject to a degree of top-down modulation (e.g., Yantis & Jonides, 1990), such modulation does not appear to be dependent on availability of resources for executive control in the present context. This apparent independence of perceptual distractor interference and executive control runs counter to cognitive load theory (Lavie, 2005, 2010) and may reflect the serial presentation of a single distractor after target items in our visual working memory task. Future work could usefully examine the extent to which distractors encountered simultaneously alongside targets in a visual working memory task require executive support for their exclusion.

In order to begin to explain the dynamics of storage, we, like others, have found it useful to assume that information in visual working memory can occupy different states (Cowan, 2011; Hollingworth & Hwang, 2013; Lewis-Peacock, Drysdale, Oberauer, & Postle, 2012; Oberauer & Hein, 2012; Olivers et al., 2011). We assume that a limited amount of information can be held in a privileged state whereby it is readily available and yet unstable and highly vulnerable to interference (Hu et al., 2014). As each item in a sequence is encountered, it is automatically encoded and temporarily held within this state. A to-be-ignored suffix distractor may also sometimes gain access by drawing perceptual selective attention. In each case, the environmental input is likely to interfere with and displace earlier items already being held. The current study indicates that participants can actively prioritize goal-relevant items for retention, in a process that is executive dependent. However, this does not prevent the automatic consolidation of subsequently attended items, even if these are less goal-relevant (see Maxcey-Richard & Hollingworth, 2013, for a similar conclusion in the context of scene memory). Thus, some information is retained in the privileged state through the deployment of executive control, while the most recent item is privileged relatively automatically. We identify the privileged state with a modality-general episodic buffer that can be accessed either actively, using executive resources, or passively, via the visual short-term store (Allen et al., 2014; Baddeley et al., 2011; Hu et al., 2014). While recognizing that this tentative account leaves many questions open, it may nevertheless have some value in encouraging more integration between research on selective attention and on working memory, and in suggesting a dynamic structure for working memory that may extend to nonvisual modalities.

We began with the observation that models of working memory do not say whether shifts of attention within working memory are mediated by the same mechanisms as in perception (Tamber-Rosenau et al., 2011). We assumed that the success with which items are initially perceived and encoded into working memory is likely to be determined by a combination of stimulus-driven attentional selection and top-down biases (Desimone & Duncan, 1995; Serences & Yantis 2006). Our results suggest that to-be-ignored external stimuli (i.e., the suffix in Experiments 2 and 3) take advantage of this, often being automatically encoded into working memory and disrupting target recall. It appears that when external attention is inadvertently drawn to a stimulus, this is independent of executive control, at least under the serial presentation conditions of our experiments. In contrast, deliberately maintaining items in an active and accessible state through visualization or attentional refreshing does appear to require substantial executive support (Experiment 1). Thus, what is accessible in working memory reflects both top-down, goal-driven priorities under executive control and the results of automatic perceptual selection from the external environment. The present experiments demonstrate how the impacts on visual working memory of these different forms of attentional selection and control can be distinguished and manipulated, and thus move toward addressing the theoretical gap identified by Steven Yantis between models of working memory and visual attention.