Spatial coding for the Simon effect in visual search

Abstract

Two experiments were conducted to examine the Simon effect (i.e., faster responding when irrelevant stimulus location corresponds with response location than when it does not) in visual search tasks. The search items were arranged in a 4 × 4 grid, and grid locations were coded into sets of four, two involving inner columns and two involving outer columns. In experiment 1, three different types of inefficient search tasks were used. The Simon effects were shown to be larger when the target appeared in one of the outer columns than in one of the inner columns (“laterality effect”). This pattern of results was not observed when distractors were absent, suggesting that the laterality effect depends on the operation of selective attention. In experiment 2, a pop-out search task was used, and no significant effect of target location on the Simon effect was found. Interpretations of the results based on the attention-shift account and referential-coding account were discussed.

Introduction

In many situations, stimulus location in the visual field is automatically coded and it affects task performance even though it is irrelevant to the task at hand. This effect of automatic spatial coding can be clearly observed in the Simon task (Simon and Rudell 1967) in which left–right manual choice responses are required for a stimulus dimension other than location (e.g., color or identity). A faster response to the target is obtained when the location of the target is congruent with the location of the instruction-assigned response than when the location of the target is incongruent with the side of the response. It is commonly assumed that this Simon effect occurs at the response-selection stage (Lu and Proctor 1995; Rubichi et al. 1997). When a stimulus set and a response set are perceptually, conceptually, or structurally similar (i.e., having dimensional overlap), presentation of a target automatically activates its most strongly associated response in the set, regardless of the stimulus-response assignment in the task (Kornblum 1994; Kornblum et al. 1990). This response activation may interfere with the execution of the instruction-assigned responses if they are not congruent, resulting in the spatial congruency effect.

For decades, numerous studies have been conducted to investigate why and how the irrelevant spatial information is processed and what mechanisms are responsible for the impact of irrelevant spatial information upon task performance. Almost all of these studies, however, present a target in isolation, although its relation to fixation, precue, or simple context could be manipulated (e.g., Hommel 1993b; Wascher and Wolber 2004; Zimba and Brito 1995). Even in a few studies with multiple-item arrays (e.g., van der Lubbe et al. 1999, 2004), the target is indicated by a cue and no active search is required to find the target (but see Ward et al. 2005). In daily life, however, we often face more complex visual scenes and need to search for a target among a number of distractors. This target can appear randomly at a number of different locations and its specific location can vary dynamically from scene to scene. It is not clear whether and how the Simon effect would occur in this context.

The main purpose of this study was therefore to examine whether the Simon effect could be observed in visual search tasks, and to what extent this finding can inform us about the mechanism underlying the activation of the spatial code of the target. As indicated above, it is widely accepted that a spatial code is automatically generated for a stimulus, even though this code is completely task-irrelevant. However, the origin of this spatial code remains controversial. Two major alternatives have been proposed: the referential-coding account and the attention-shift account. The referential-coding account (Hommel, 1993a) assumes that a spatial code is formed by relating the imperative stimulus (i.e., the stimulus that delivers the task-relevant information) to a reference frame or object. The attention-shift account (Proctor and Lu 1994; Rubichi et al. 1997; Stoffer 1991; Stoffer and Umilta 1997; Stoffer and Yankin 1994), on the other hand, postulates that a spatial code is generated when there is a shift in spatial attention towards the location occupied by the imperative stimulus. Moreover, when multiple attention shifts take place over time and location, only the most recent shift is responsible for the generation of the spatial code for the target.

The attention-shift account is consistent with some previous evidences. For example, Nicoletti and Umilta (1989) instructed participants to respond, with a left or right keypress, to a rectangle or square that appeared inside one of six boxes arranged in a row. In their experiment 3, participants had to maintain fixation on a plus sign located at one end of the row and to orient attention onto a small solid square that was shown for 500 ms in one of the five gaps between the boxes. At the offset of the square the imperative target appeared in one of the immediately adjacent boxes. A Simon effect was observed with respect to the location at which attention was initially oriented (i.e., the square), regardless of where the orienting square was placed. Thus a further attention shift from the square (the precue) to the target determined the spatial code of the target (see also Rubichi et al. 1997). Nicoletti and Umilta (1994) demonstrated that the Simon effect was not obtained when attentional focus must remain at fixation. The display was similar to their former study; except that participants had to keep attention at fixation to detect a letter presented there, and could not voluntarily direct attention to the target. In addition to these studies with attentional focus shifting from fixation to the periphery, recent studies suggest that attention shift from the periphery to a centrally presented target can also generate the Simon effect. For example, Notebaert and Soetens (2003) asked participants to respond to the color of a centrally presented visual stimulus while presenting a sound to one of their ears. A Simon effect in relation to the peripheral sound was observed.

Hommel (1993a), on the other hand, proposed a referential-coding account in which spatial codes are generated in relation to a referential frame. Hommel and Lippa (1995) demonstrated that the face of a famous movie star established a context for the spatial coding of left and right. In their experiment, the target was presented on the left or right eye of the face, and the Simon effect occurred constantly relative to the context of face, even though the face was rotated up to 90°. There are essentially two versions of the referential account: static reference system and attentional focus reference. The static reference account suggests that a spatial position is defined relative to a set of coordinates, and that the origin of the set of coordinates is determined by a static reference object, which usually is the fixation point (as discussed in Proctor and Lu 1994). Alternatively, Nicoletti and Umilta (1989) proposed that the origin of the set of coordinates is determined by the position of the attentional focus, which can be moved in space. This attention version of the referential hypothesis is clearly not very different from the attention-shift account.

Hence there are two main opposite models concerning the origin of spatial code: a dynamic model and a static model. The difference between them lies in the definition of the origin of the referential system. The dynamic coding model postulates that the origin of the reference system is the focus of attention just before it being shifted to the target, whereas the static coding model assumes that the referential frame remains the same in spite of the attention-shifts. We suggest that the dynamic and static models can be distinguished in contexts that naturally requires attention shift. One such context is visual search.

In a typical visual search experiment, participants are presented with a display containing a number of items. On each trial, participants must determine whether or not a specific target has appeared in the display. The number of items (set size) in a search array varies from trial to trial. The response times (RTs) and correct response rates are measured. The change in RTs with set size is termed RT × set size search function, and the slope of search function designates the efficiency of performance, with the steeper slope being observed in the more difficult search task.

Recently, a large body of evidence supports an integrated model of parallel and serial processing in visual search (e.g., Bricolo et al. 2002; Horowitz and Wolfe 1998; Maioli et al. 2001; Wolfe et al. 1989; Woodman and Luck 1999, 2003; Zelinsky and Sheinberg 1997; see Chelazzi 1999, for an overview). According to the integrated model, the shift of attention is guided by a parallel feature analysis, by which several items are simultaneously compared with a target template held in working memory. Once a candidate target is detected, a shift of attention occurs. If the target is distinct from the distractors, the parallel process will guide attention shift to the target directly. If the target cannot “pop out” from its distractors, and the attended item is not the target, another parallel processing of multiple items is performed, followed by a new shift of attention. In this manner, attention shifts from item to item randomly until the target is detected. The amount of attention shifts depends on the level of signal-to-noise ratio between the target and its environment.

Consider the display illustrated in Fig. 1. Search items are presented in 4 × 4 grids, and there are two possible targets requiring left or right response. Congruency effects would be calculated according to the correspondence between response location and target location (relative to the central fixation). If the spatial code of the target is generated with reference to a static referential frame, the Simon effect will be observed, but this effect should not be influenced by attention shift during search. If, however, the spatial code originates from attention shift or attention focusing, then the serial attention mechanisms in visual search could cause different patterns of the Simon effect in different regions of the search display. Specifically, because attention shifts randomly from item to item, the direction of attention shift will be either right-to-left or left-to-right before the target is located in an inner column. If the Simon effect for the target is determined by the spatial coding of this attention shift, then this effect might be diminished or cancelled out by averaging across trials. Conversely, if a target appears in a, say, right outer column, then its accompanying distractors are more likely to appear on the left side of this target, and attention shifts from distractors to the target are mostly likely to occur from left to right, rather than from other directions. If the Simon effect for the target is determined by the spatial coding of attention shift, then a sizeable Simon effect should be observed for targets appearing in outer columns. We may call the neutralization of the Simon effect in inner columns and the appearance of the Simon effect in outer columns the “laterality effect”.

Fig. 1

The 4 × 4 matrix and the definition of target locations in the search array. Lines and numbers were invisible to participants. The numbers indicate the target location categories in relation to the central fixation. The fixation cross was visible constantly during a trial

Nevertheless, the attention coding account is not the only one which predicts the laterality of Simon effect. The multiple references coding account is also compatible with the laterality effect. As postulated by this account, the reference frame can be both egocentric such as the body midline, the point of gaze and head position, and environmental such as spatial relations between and within objects (Danziger et al. 2001; Lamberts et al. 1992; Roswarski and Proctor 1996; Umilta and Liotti 1987). On this account, when the target is in one of the inner columns, either the right outer column or the left outer column may serve as reference frame. As a consequence, an ambiguous spatial code will be generated for targets in the inner columns. Whereas for a target in, say, the right outer column, a right spatial code will be always generated, regardless of which of the other three columns may serve as reference frame. Therefore, the Simon effect that will be observed on the inner columns will be smaller than that on the outer columns.

This effect of multiple reference frames is often difficult to be separated completely from the attentional shift effect. To this end, search tasks with different degree of difficulty were introduced in experiment 1 in order to examine whether the laterality effect is influenced by search efficiency. If a statistically reliable interaction between search type and laterality effect is found, this result would support the attention shift account, because it would provide a straighter and simpler interpretation of the results than the multiple reference account. Indeed, to explain this interaction with the multiple reference account, we need to assume that the role of the outer columns in serving as extra reference frames might be quantitatively changed by search efficiency. Additionally, according to the attentional coding account, we can predict that the laterality of the Simon effect should not be reliably observed in pop-out search task, because attention automatically shifts from fixation to the pop-out target, regardless of its location in inner or outer columns. However, the multiple reference account makes a different prediction, namely that the laterality of the Simon effect should be observed in pop-out search because outer columns serve as frame of reference. To test these two opposite predictions, a pop-out search task will be used in experiment 2. Note that, if the laterality of the Simon effect is not observed in the pop-out search task, then the multiple reference account can either be refuted or modified by suggesting that a spatial location serves as reference frame only when attended. With this modification, the two accounts become functionally indistinguishable, at least with the current experimental conditions.

Because the search mechanism is determined by the difficulty of search task rather than by whether the task is a feature or conjunction search (e.g., Chelazzi 1999), we predict that the laterality effect will be observed in various inefficient search tasks (i.e., when the slopes of search function are greater than zero). Therefore, we used three search tasks in experiment 1 to test the laterality effect. The first type of search was a feature search with uniform distractors, in which participants were required to detect figure-of-two or figure-of-five among figure-of-eight (type 1, see Fig. 2a). The second type of search was similar to the first one, except that the distractors were variable letters or digits (type 2, see Fig. 2b). The third type of search was a conjunction search in which the target is different from non-targets along two clearly defined dimensions (color and orientation, type 3, see Fig. 2c). The type of visual search was treated as a between-participant factor in experiment 1.

Fig. 2

Stimuli in all the search tasks. a shows the search items in type 1 search. The figure-of-five and figure-of-two were the imperative targets, whereas the figure-of-eight was the distractor. b shows examples of distractors in type 2 search. The target here was the same as the one in type 1 search. c shows the colored stimuli in type 3 search (horizontal fill = red; vertical fill = green). The thin ellipses in the upper row were the imperative targets, and the ellipses in the lower row were distractors. d shows the colored stimuli in type 4 (pop-out) search (horizontal fill = red; vertical fill = green; grid fill = blue). The red and green tilted ellipses in the upper row were the imperative targets, and the blue vertical thin ellipse in the lower row was the distractor

Experiment 1

Methods

Participants

A total of 48 undergraduate students from Peking University participated in the experiment, with 16 for each of the three types of visual search. They were all right handed, with normal or corrected-to-normal vision. They gave their informed consent and were paid for their participation.

Design and stimuli

This experiment consisted of four factors. The between-participant factor was search type, which included type 1, 2 and 3, as described in the Introduction and the following. The other three within-participant factors were target location, search set size, and Simon congruency. As illustrated in Fig. 1, the 4 × 4 grids were divided into 4 regions, according to the eccentricity of the grid to the central fixation. A target could appear in any of the four regions in a particular trial. The factor of set size had three levels, which had 6, 11 or 16 displayed items. Target location could be either congruent or incongruent with the side of the responding hand. There were two potential targets, one requiring a left hand response, and the other a right hand response. The correspondence between target and responding hand was counterbalanced across participants in all the search tasks.

In addition, in types 1 and 2 search, there were trials in which a target was presented without distractors (target-only trials). The target-only conditions were not included in the type 3 search because we wanted to balance the total number of trials across the three search types. The main purpose of including target-only trials was to examine whether the potential laterality effect was caused by the distance between the target and center fixation i.e., the stimulus eccentricity. If the Simon effect was observed in all regions of the search grid, we could conclude that the absence of the Simon effect in the inner columns for the target with distractors could not be simply attributed to the target eccentricity per se.

In types 1 and 2 search, targets and distractors were created by removing segments of a figure-of-eight, which was composed of short bars, like those used in digital clock displays. A number or letter was 0.65°in width and 1.15°in height. In both search types, the target was either “2” or “5”. Distractors were uniformly “8” in type 1 search (Fig. 2a) or were randomly selected from various numbers or letters in type 2 search (Fig. 2b). In type 3 search, the target was either tilted (30° from vertical) red thin ellipse or vertical green thin ellipse. Distractors were both tilted green thin ellipses and vertical red thin ellipses (Fig. 2c).

In type 1 search and type 2 search, there were no target-absent trials. In type 3 search, however, the search items could be grouped according to their colors and participants might adopt a strategy by which they segment the items into two color groups and searched only in one group of items (e.g., all the red items). If there were no target-absent trials, then a response could be issued based on whether a target was present or not in one color sub-group, without the need of searching in the other color sub-group. Having target-absent trials would prevent participants from using this sub-set search strategy. Thus in type 3 search the target was absent in 1/3 of the total trials.

White or colored stimuli were presented against a black background on a CRT screen. The search items were randomly displayed in an imaginary 4 × 4 matrix (Fig. 1), with a visual angle of 4° between the outer imaginary grid and the central fixation. As labeled in Fig. 1, the 16 grid cells were grouped into four regions according to their distances to fixation. Lines and numbers in Fig. 1 were invisible to participants.

All trials for different experimental conditions were randomly mixed and were divided into six test blocks. Each condition had 24 trials, giving a total of 768 trials (including target-only trials) in type 1 or 2 search, and a total of 864 trials (including target-absent trials) in type 3 search. Participants received 64 practice trials before the formal test.

Procedures

Presentation software (http://www.nbs.neuro-bs.com/) was used to present stimuli and record responses. The participants held with both hands a double handle joystick that was positioned in front of them, along the body midline. Manual responses were made by pressing either a left or right button located on the front of the joystick, with the right or left index finger, respectively. A third central button on the joystick was pressed with the right thumb to respond when the target was absent.

In each trial, a white cross (0.65° of visual angle) was presented at the center of the screen for 1,000 ms. Participants were instructed to fixate the central cross and avoid eye movement throughout the trial. Then a search display was presented until participants’ response. Speed and accuracy were equally stressed.

Results

Participants made incorrect responses on a total of 2.66, 4.73, and 5.91% of trials in search types 1, 2, and 3, respectively. After incorrect responses were excluded, outliers in response times (RTs) in each experimental condition were removed, using a recursive procedure as suggested by van Selst and Jolicoeur (1994). In this procedure, the slowest and fastest RTs were removed, one at a time, and the mean and the standard deviation (SD) of the resulting distribution were calculated. If the extreme RT was more than 3 SDs away from the mean, it was considered an outlier and was removed. This procedure was repeated until no outliers remained. A total of 4.56, 3.6, and 3.34% of trials were removed as outliers in search types 1, 2, and 3, respectively.

The congruency effect for targets with distractors

For trials with distractors, mean RTs and percentages of error responses were then calculated for experimental conditions and are reported in Table 1.

Table 1 Mean RTs (ms) and standard errors (mean ± SD), and error percentages (in parentheses) for targets with distractors in type 1, 2 and 3 search in experiment 1, and type 4 (pop-out) search in experiment 2

RTs from the 3 search types were entered into a 3 (search type) × 3 (set size) × 4 (target location) × 2 (congruency) analysis of variance (ANOVA), with search type as a between-participant factor, and set size, target location and congruency as three within-participant factors. The main effect of search type was significant, F(2, 43) = 278.94, P < 0.001, with RTs fastest in type 1 search (545 ms), slowest in type 3 search (1,471 ms), and in the middle in type 2 search (758 ms). Not surprisingly, the main effect of set size was also significant, F(2, 86) = 269.17, P < 0.001, with RTs fastest at size 6 (814 ms), slowest at size 16 (1039 ms), and in the middle at size 11 (921 ms). The overall slopes of search functions were 3.6 ms/item for type 1 search, 24.6 ms/item for type 2 search, and 39.0 ms/item for type 3 search. The slope of search function for target-absent trials in type 3 search was 96.5 ms/item. The interaction between set size and search type was significant, F(4, 86) = 55.69, P < 0.001, indicating that the increases of RTs over set size were of different magnitudes over the different search types.

The main effect of target location was significant, F(3, 129) = 133.06, P < 0.001, with RTs fastest at location 1 (821 ms), slowest at location 4 (1,004 ms), and in the middle at location 2 and 3 (930 and 944 ms). All the differences between locations were significant in Bonferroni corrected pairwise comparisons (P < 0.001), except the difference between location 2 and 3. The interaction between target location and search type was significant, F(6, 129) = 14.67, P < 0.001, so the interaction between location and set size, F(6, 258) = 13.66, P < 0.001, and the three-way interaction between location, set size and search type, F(12, 258) = 6.66, P < 0.001. These results replicated previous studies of eccentricity effect of visual search (cf. Carrasco et al. 1995; Carrasco and Frieder 1997).

More importantly, the main effect of congruency was significant, F(1, 43) = 13.93, P < 0.005, with RTs faster for the congruent trials (913 ms) than for the incongruent trials (936 ms). This factor did not interact with set size, F(2, 86) < 1, or with search type, F(2, 43) = 1.15, P > 0.1. The three-way interaction between congruency, set size and search type was not significant either, F(4, 86) = 1.23, P > 0.1. However, it interacted significantly with target location, F(3,129) = 4.83, P < 0.005, indicating that across search types and search set size, the magnitude of Simon effect varied over different target locations. Figure 3 illustrates the mean RTs over set size at different locations, collapsed over search types. Moreover, the interaction of congruency, location and search type was also significant, F(6, 129) = 2.38, P < 0.05, indicating that the laterality effect was influenced by search efficiency. Simon effects were averaged over outer and inner columns for each search type, and the values of laterality effect were indexed by the differences of Simon effects between outer and inner columns. In this manner, the laterality effects for search type 1, 2 and 3 were 9.5, 23.5, and 87 ms, respectively, increasing as the slopes of search function increasing.

Fig. 3

The RT × set size search functions for the congruent and incongruent conditions in the four target locations, collapsed over the three search types in experiment 1

Separate analyses were then conducted for the congruency effect at different locations, with the set size and congruency as two within-participant factors, and search type as a between-participant factor. At location 1, the main effect of congruency was not significant, F(1, 45) < 1, and it did not interact with search type, F(2, 45) = 1.35, P < 0.1, or with set size, F(2, 90) < 1. Similarly, at location 3, there was no main effect of congruency, F(1, 45) < 1, and no interaction of congruency with search type, F(2, 45) < 1, or with set size, F(2, 90) = 1.22, P > 0.1. These results indicated that the Simon effect was absent at location 1 or 3 (see Fig. 3). At location 2, the main effect of congruency was significant, F(1, 45) = 9.46, P < 0.005, but this effect did not interact with search type, F(2,45) < 1, nor with set size, F(2, 90) < 1. At location 4, both the main effect of congruency, F(1,45) = 23.37, P < 0.001, and the interaction between congruency and search type, F(2,45) = 6.85, P < 0.005, were significant, although the interaction between congruency and set size was not, F(2, 90) < 1. Further analyses showed that the congruency effect was significant at location 4 for type 1 search, F(1,15) = 12.13, P < 0.005, type 2 search, F(1,15) = 5.87, P < 0.05, and type 3 search, F(1,15) = 13.52, P < 0.005, although the effect was apparently larger in Type 3 search (105 ms) than in types 1 and 2 search (17 and 30 ms, respectively).

Error rates for trials with distractors were also entered into an ANOVA, with search type as a between-participant factor, and set size, target location and congruency as three within-participant factors. The main effect of search type was not significant, F(2, 45) = 1.26, P > 0.1, indicating that the error rates were not different between search types. The main effect of set size was significant, F(2, 90) = 14.37, P < 0.001, with the error rate being the highest at size 16 (7.0%), lowest at size 6 (4.9%), and in the middle at size 11 (5.4%). The main effect of target location was also significant, F(3, 135) = 16.42, P < 0.001, with the rate being the highest at location 4 (7.5%), the lowest at location 1 (4.2%), and in the middle at locations 2 and 3 (5.6 and 5.7%, respectively).

The main effect of congruency was significant, F(1, 45) = 15.71, P < 0.001, with more errors in the incongruent conditions (6.7%) than in the congruent conditions (4.8%). Importantly, this congruency effect interacted with target location, F(3, 135) = 2.93, P < 0.05, although the three-way interaction between congruency, location and search type was not significant, F(6,135) = 1.20, P > 0.1. Separate analyses were then conducted for the congruency effects at different locations, with set size and congruency as two within-participant factors and search type as a between-participant factor. Results were similar to the RT analysis, with the congruency effect being significant at Location 2, F(1, 45) = 14.65, P < 0.001, and location 4, F(1, 45) = 14.12, P < 0.001. This effect was not significant at Location 3, F(1, 45) = 2.01, P > 0.1, although it did reach significance at Location 1, F(1, 45) = 5.62, P < 0.05.

The congruency effect for targets without distractors

RTs for trials without distractors in types 1 and 2 search, collapsed over search type and set size, were reported in Table 2.

Table 2 Mean RTs (ms) and standard errors (mean ± SD), and error percentages (in parentheses) for targets without distractors in types 1 and 2 search in experiment 1, collapsed over search types and set sizes

RT data were entered into a 2 (search type) × 4 (target location) × 2 (congruency) analysis of variance (ANOVA), with search type as a between-participant factor, and target location and congruency as two within-participant factors. The main effect of congruency was significant, F(1, 30) = 21.87, P < 0.001, with RTs faster to congruent stimuli (489 ms) than to incongruent stimuli (510 ms). The interaction between congruency and location was not significant, F(3, 90) < 1, nor the three-way interaction between congruency, location, and search type, F(3, 90) = 1.32, P > 0.1, indicating that the congruency effect did not vary according to the target location, in contrast with the laterality effect for targets with distractors. Another significant effect was the main effect of location, F(3, 90) = 26.97, P < 0.001, with RTs being slowed increasingly over locations 1–4 (482, 498, 501 and 517 ms, respectively).

Error rates for trials without distractors were also entered into an ANOVA, with search type as a between-participant factor, and target location and congruency as two within-participant factors. The main effect of search type was not significant, F(1, 30) < 1. The main effect of location was significant, F(3, 90) = 4.32, P < 0.01, with the error rate being the highest at location 4 (7.1%), lowest at location 1 (3.9%), and in the middle at locations 2 and 3 (5.7 and 5.9%, respectively). The main effect of congruency was also significant, F(1, 30) = 14.85, P < 0.01, with more errors in the incongruent conditions (7.7%) than in the congruent conditions (3.7%). However, the interaction between congruency and location, F(3, 90) = 1.41, P > 0.1, and the three-way interaction between congruency, location and search type, F(3, 90) < 1, were not significant. Thus, the results of error rate analysis mirrored the RT analysis.

In summary, the results for distractor-absent conditions replicated the Simon effect in previous works.

RT distributions

To rule out the possibility that the differences between Simon effects at different locations were due to different speeds of response times, the RT distributions with distractors were analyzed, with each participant’s RTs in each experimental condition sorted in ascending order, and divided into quintiles (see Ratcliff 1979; de Jong et al. 1994; Zhang and Kornblum 1997). Because the interaction between congruency and set size was generally not significant, as revealed by the previous analyses, we collapsed the data over set size in quintiles, and constructed RT functions of quintile for each location. These quintile data were entered into a 3 (search type) × 4 (location) × 2 (congruency) × 5 (quintile) ANOVA.

Not surprisingly, the main effect of congruency was significant, F(1, 45) = 15.51, P < 0.001, so the main effect of quintile, F(4, 180) = 1106.12, P < 0.001. Importantly, the interaction between congruency and quintile was not significant, F(4, 180) < 1, nor the three-way interaction between congruency, quintile, and search type, F(8, 180) < 1. These results suggested that the Simon effect did not change over quintiles, i.e., the lengths of RTs. The main effect of location was significant, F(3, 135) = 127.66, P < 0.001, so the interaction between congruency and location, F(3, 135) = 5.40, P < 0.05. But the three-way interaction between congruency, location and quintile was only marginally significant, F(12, 540) = 1.67, 0.05 < P < 0.1, suggesting that the presence of Simon effects at locations 2 and 4 and the absence of Simon effects at locations 1 and 3 were generally not affected by response speed.

The RT distributions for distractor-absent conditions were entered into a 2 (search type) × 4 (target location) × 2 (congruency) × 5 (quintile) ANOVA. The main effect of search type was not significant, F(1,30) = 1.18, P > 0.1. Both the main effect of congruency and the main effect of quintile were significant, F(1,30) = 26.25, P < 0.001, and F(4,120) = 315.06, P < 0.001, respectively. But the interaction between these two factors was not significant, F(4,120) < 1. This result was different from previous works (e.g. de Jong et al. 1994; Vallesi et al. 2005; Wiegand and Wascher 2005), which revealed that the Simon effect decreases as RT increases. Figure 4a plotted the RT distributions in the incongruent and congruent conditions for distractor-absent trials.

Fig. 4

a The distribution of RT data from the distractor-absent trials in experiment 1. The RT data were collapsed across search type, set size, and target location for each quintile. b The distribution of RT data from experiment 2. The RT data were collapsed across set size and location for each quintile

Discussion

The crucial findings from this experiment can be summarized as follows. There was an overall Simon congruency effect in various visual search tasks when the target was accompanied by distractors. This effect was neither affected by search type, nor by search set size. However, it was affected by the location of the target in the search array, with the effect being absent when the target was in the inner columns near fixation, and being present when the target was in the outer columns far from fixation. This pattern of laterality of the Simon effect was not affected by the speed of responses, but the laterality effect could be influence by the difficulty of search tasks. When the target was presented alone in the search array, the Simon effect appeared regardless of target location and was not affected by response speed either.

The laterality of the Simon effect in a search array is consistent with the attention-shift account of spatial coding but inconsistent with the static referential-coding account. As we reviewed earlier, the static referential coding account assumes that the spatial code for the target is generated according to its location in relation to a static frame (i.e., the fixation). This account would predict that there should be equal Simon effects for the inner columns (at least location 1) and outer columns, and for targets with or without accompanying distractors. This prediction was clearly refuted by the findings of this study.

On the other hand, the laterality of the Simon effect is consistent with the attention-shift account. As we argued earlier, because attention shifts randomly over items during visual search, the probability of a target receiving attention shift from other directions may depend crucially on its location in the search array. When the target is in an outer (say, right at location 2 in Fig. 1) column, it is more likely for it to receive attention shift from a distractor, which may have first summoned attention, located at the opposite direction. Averaging over trials, the left-to-right attention shift is the strongest way for the target to receive attention, and hence a “right” code would be consistently generated for the target. By contrast, when the target in an inner column (right at Location 1 in Fig. 1), its accompanying distractors could be scattered over the target’s left or right sides. Assuming that in most cases attention jumps to a distractor before it lands on the target, averaging over trials, the target could receive attention shifts from the left as well as from the right. Thus, the overall Simon effect would be around zero for targets in inner columns.

The absence of laterality of Simon effects in the no-distractor condition is consistent with the results from previous Simon effect studies. In this condition, attention shifts directly from fixation to the location of the target, generating the spatial code for the target. The equivalent Simon effects over different locations demonstrate, in the opposite way, that the laterality of the Simon effect for the target with accompanying distractors is due to dynamic attention shift and spatial coding, rather than to target location per se.

The fact that the size of the Simon effect in the distractor-absent condition was not affected by the target’s eccentricity is consistent with Stins and Michaels (2000) and Logan (2003) (see also Proctor et al. 1993) despite the general slow down of responses over eccentricity. It is possible that the spatial coding is categorical (i.e., left or right), not metric. Our results suggest further that, in visual search in which the location of the single target is not predictable, the Simon effect does not differ with distances on either vertical or horizontal meridians.

As we discussed earlier, one basic assumption concerning the Simon effect is that it occurs at the response-selection stage (Lu and Proctor 1995; Rubichi et al. 1997). There is a widely accepted notion that the slope of search function is determined by the search process, whereas the intercept of search function is influenced by response selection (e.g., Horowitz and Wolfe 1998; Chelazzi 1999). In the present experiment, it is clear from Fig. 3 that the slopes of search function for congruent and incongruent trials were parallel. The intercepts for the congruent and incongruent conditions were different at locations 2 and 4 but were equal at locations 1 and 3. Thus the congruency between the target location and response hand does not affect the process of detecting the target in a search array, but may affect the process at the response selection stage, consistent with the above general assumption.

The pattern of RT distributions in the three types of visual search was different from traditional pattern in which Simon effect decreases as RT increases (e.g. de Jong et al. 1994; Vallesi et al. 2005; Wiegand and Wascher 2005). According to Zhang and Kornblum (1997), the decreasing or increasing Simon effect function is determined by the difference in variance for the pair of RT distributions. When the variances for RTs of the congruent and incongruent conditions are not significantly different, the Simon effect does not change with RT increases. In fact, the RTs in the present search tasks include two components: one is search time before the target is detected, and the other is response time after the target is detected. In the inefficient search task, the search time is longer and has larger variance than response time, and the variance of response time is tiny compared to the large scale of response time. This may be a reasonable explanation for the RT distribution in search trials. However, it is difficult to explain why the RT distributions were parallel in distractor-absent conditions in which no search process was needed. It is possible that this pattern reflects the adjustment of cognitive strategy in a search context, an issue that deserves further exploration.

As discussed in Introduction, there is a competitive explanation of the laterality of Simon effect, that is, the multiple references coding account. Given the relationship between search difficulty and the laterality of Simon effect, the attention coding account might be a more straightforward explanation for the laterality effect than the multiple references coding account. On the basis of the relationship between search efficiency and laterality effect, we tentatively predicted that the laterality effect should not be reliably observed in a pop out search task carried out in experiment 2.

Experiment 2

If the above argument concerning the origin of laterality of the Simon effect in visual search can take its stand, an alternative should be ruled out, for this account assumes that the absence of Simon effects in inner columns is not due to the averaging of attention shifts over different directions, but due to the inefficiency of the fixation as a reference frame when there are multiple items presented in a search array. Compared with the display in which there are only a fixation and a target, the fixation in a crowded display with multiple items is perceptually less salient and its function as a spatial reference point may be reduced. This reduction has less influence on localizing the target appearing at the leftmost or rightmost periphery than on localizing the target near fixation, because a leftmost target can use other items (distractors) as references to compensate for the reduction of the function of the fixation. An inner target, on the other hand, cannot use this compensation mechanism because other items can be both on the left and on the right of the target, and no reliable references can be established.

In experiment 2, we still presented the fixation in a crowed display but increased the possibility of the target capturing attention directly, i.e., the search efficiency in finding the target. According to the attention shift account of the Simon effect, this pop-out target will cause the attentional focus to shift directly from fixation to the target and a spatial code is generated automatically. Thus the Simon effect should not be affected by where the target appears. According to the referential coding account, however, the crowded display reduces the function of the fixation as reference and this reduction should be invariant with respect to the perceptual saliency of the target. Thus the absence of the Simon effect in inner columns should also be observed for the pop-out target.

Methods

Participants

Sixteen undergraduate students from Peking University participated in this experiment. None of them had been tested for experiment 1. As is in experiment 1, all participants were right handed, with normal or corrected-to-normal vision. All gave their informed consent and were paid for their participation.

Stimuli and procedure

The design and experimental procedures were the same as experiment 1. The stimuli were similar to those used in type 3 search in experiment 1, with the two potential targets being tilted red or green thin ellipse (0.65° in width and 1.15° in height) and the distractor being a vertical blue thin ellipse which is a little smaller than the target (0.5° in width and 1° in height, see Fig. 2d). Thus, the distractor and the targets differed in three dimensions: color, direction, and size. These distinctions guaranteed that the target would pop out from distractors when the search items were presented. The two targets were assigned respectively to the left or right response and this assignment was counterbalanced between participants.

The target was absent in 1/3 of the total trials, and there were a total of 864 trials in the experiment. As in experiment 1, participants received 64 practice trials before the formal test.

Results and discussion

After the incorrect responses (2.81% of the total trials) were excluded, outliers in each condition cell were removed (4.48% on average), using the recursive procedure advocated by van Selst and Jolicoeur (1994). Mean RTs and standard errors for conditions are listed in Table 1.

RTs for experimental conditions were entered into a 3 (set size) × 4 (target location) × 2 (congruency) ANOVA. The main effect of set size was significant, F(2, 30) = 5.47, P < 0.01, with RTs to the target fastest at size 6 (508 ms), slowest at size 16 (518 ms), and in the middle at size 11 (513 ms). However, the slope of the search function was only 1 ms/item, suggesting that the slope was near flat and the visual search task here could be considered as “pop-out” search in which the target was detected automatically (see Chelazzi 1999).

The main effect of location was significant, F(3, 45) = 4.17, P < 0.01, with RTs being the fastest at location 1 (507 ms), slowest at location 4 (518 ms), and in the middle at locations 2 and 3 (512 and 515 ms, respectively). More importantly, the main effect of congruency was significant, F(1, 15) = 28.91, P < 0.001, with faster responses in the congruent conditions (505 ms) than in the incongruent conditions (521 ms), and this effect did not interact with location, F(3, 45) = 1.95, P > 0.1. This indicated that there were equal Simon effects at the four locations and there was no laterality of the Simon effect in pop-out search. The interaction between set size and congruency was also non-significant, F(2, 30) < 1, replicating the previous finding that the Simon effect was not affected by set size.

In the analyses of error rates, the main effect of set size was not significant, F(2, 30) < 1, confirming that the efficiency of detecting target did not, in general, increase with set size in this search task. The main effect of location was significant, F(3, 45) = 2.83, P < 0.05, indicating a variation of error rates over target locations: 3.3% at location 1, 2.1% at location 2, 2.8% at location 3 and 3.0% at location 4. More importantly, the main effect of congruency was significant, F(1, 15) = 4.72, P < 0.05, with more errors in the incongruent conditions (3.3%) than in the congruent conditions (2.3%). But the interaction between congruency and location was not significant, F(3, 45) = 2.16, P > 0.1, indicating that the congruency effect did not vary across target locations.

RT distribution analysis was also applied to experiment 2. Since the main effect of set size was not significant, RT data were collapsed over set size and were quintiled according to the method used in experiment 1. These quintile data were entered into a 4 (target location) × 2 (congruency) × 5 (quintile) ANOVA. Main effects of both congruency and quintile were significant, F(1, 15) = 15.92, P = 0.001, and F(4,60) = 126.90, P < 0.0001, respectively. The interaction between congruency and quintile was significant too, F(4,60) = 7.51, P < 0.0001, indicating that the Simon effect was greater at shorter RTs than at longer RTs (see Fig. 4b). This pattern was consistent with previous works (e.g. De Jong et al. 1994; Vallesi et al. 2005; Wiegand and Wascher 2005), and the explanation of difference in RT variance inferred by Zhang and Kornblum (1997). In this pop-out search task, the target is detected immediately without any effortful search process, and the length of RT is dominantly determined by the response selection.

There was no laterality of the Simon effect in the pop-out search, suggesting that the crowding of the fixation with search items was not the reason behind the absence of the Simon effect in inner columns in experiment 1. Attention was captured directly by the singleton target and the automatic attention shift from fixation to the target provided a spatial code for the target, wherever the target appeared. There was no jumping of attention between distractors and the target and hence no averaging of attentional shifts over trials.

General discussion

From what has been discussed above, it is quite evident that not only Simon effects in the visual search tasks, but also the laterality of the effect with respect to the location of target in the search array were found in this study. In inefficient or serial searches, the Simon effect appeared only when the target was in the outer columns far from fixation, not when the target was in the inner columns near fixation. This finding was, in general, affected neither by the search set size, nor by response speed. In contrast, this laterality effect was influenced by the search efficiency. On the other hand, when the target was presented alone without distractors, or when the search was efficient or parallel, equivalent Simon effects were observed at different locations with no sign of laterality.

Our findings present a different picture from what is envisaged by the notion that the spatial code of the target depends on its position relative to a static reference frame, which is usually the central fixation. However, in this study, it is difficult to refute the multiple reference frames coding account, which postulates that, when targets are located in inner columns, either outer column might serve as a frame of reference and thus cause a reduction of the Simon effect. But the absence of the laterality effect in the pop out search task suggests that whether the outer columns can serve the function of reference frame is contingent on whether they receive attention or not. When the items in the outer columns were not attended, they did not operate as reference frames. Based on this finding, we believe that the attention shift account offers a simpler explanation for the laterality of the Simon effect than the multiple reference account.

In contrast, the laterality of the Simon effect can be accommodated easily by the dynamic attention shift hypothesis of spatial coding. During the serial search process, attention shifts randomly from item to item, and the last attention shift before target localization determines the direction of the spatial code. When the target is presented in an outer column (e.g., the right location 2 in Fig. 1), the last attentional focus just before target detection would be most likely on a distractor left of the target. Thus an attention shift from this distractor to the target (i.e., from left to right) would generate a “right” code for the target, and this code causes the Simon effect observed. However, when the target is presented in an inner column (e.g., the right location 1 in Fig. 1), although it is possible an attention shift from a distractor on the left would generate a “right” code for this target, it is equally possible that the last attention shift before the detection of the target is from a distractor on the right (e.g., the right location 2 in Fig. 1). In the latter case, a “left” code would be generated for the target. Therefore, averaging over trials, the target at the right Location 1 in Fig. 1 would probably receive attention shift equally often from the left or right, leading to a reduced or null Simon effect.

Thus, the laterality of the Simon effect reflects the dynamic spatial coding for the target at different locations in the search array. This coding is controlled by attention shifts over items in a display. If, however, there is no need for attention to shift between items, such as when target is presented alone, or when it pops out and captures attention directly, the laterality of the Simon effect should not emerge. Indeed we observed equal Simon effects at different locations for the target-alone conditions in Experiment 1 and for the pop-out search in experiment 2.

Our results and arguments are consistent not only with many previous data concerning the Simon effect when an isolated target is presented (see Lu and Proctor 1995 for a review), but also with a recent study which also used a visual search task (Ward et al. 2005) and showed the role of attention in generating and suppressing task-irrelevant spatial codes. In Ward et al’s study, the search array consisted of two columns, one on the left and one on the right side of fixation, and with an equal number of items in each column. Participants were asked to make speeded left or right key-presses to the color of a target letter O that appeared among varying numbers of distractor Qs. In their experiment 1, the time of target onset was separated from the time of target selection by using a difficult search task with a variable number of distractors. Although reaction times increased as a function of the number of distractors, the Simon effects were similar for both small and large set sizes, as in the present study. Thus, regardless of how long a target was on the screen, there was no suppression of involuntary response codes before the target was found and selected. Suppression of involuntary spatial response activation is not tied to object onset, but to the time of target selection. In Ward et al.’s experiment 2, the color information needed to determine response only appeared after a variable delay; however, participants could still select the target object based on its form. It was found that with delays long enough, target selection could occur before a response could be made and the Simon effect was reduced as the delay between target selection and the availability of relevant response information increased. Suppression of irrelevant spatial response activation begins only after the target is attended.

Hommel (1994) proposed that the magnitude of the Simon effect depends on stimulus complexity. In his experiment, participants were presented with two frames, either to the left or to the right of a central fixation. One of them contained a stimulus, which could be a rectangle (or square) or a red (or green) square. Participants were asked either to identify the shape or the color. Because the frame itself may interfere with the identification of the visual shape, shape discrimination was more difficult than color discrimination. It was found that the Simon effect was apparent in color discrimination but was absent in shape discrimination. Hommel suggested that the size of the Simon effect depends critically on the temporal overlap between spatial coding for the target location and coding of the task-relevant stimulus information. The spatial code can be generated once the left or right frames are presented, but processing of the task-relevant stimulus information may take shorter or longer time, depending on the difficulty of the task. Because the spatial coding is supposed to decay over time, the Simon effect, which depends on the interaction between spatial coding for target location and response selection, is hence affected by the difficulty in processing task-relevant stimulus information. However, the present study did not find obvious variation of the Simon effect over search types, even though the difficulty of target discrimination differed significant between types. This apparent contradiction with Hommel’s argument can be explained by the fact that the spatial code for the target in the present study could not be generated until the target was identified and located in the search array. As discussed previously, the reaction time for a trial in this study was affected by processing in two stages, the search process and response selection. The interaction between the spatial code and response code took place only at a time after the search process was finished. The differences in difficulty between search types could affect the search process, but not the later interaction in the response selection stage. Hence, the Simon effect in the present study was not simply affected by search difficulty or target eccentricity.

In conclusion, this study demonstrated that the Simon effect appears in visual search tasks, but the appearance of this effect depends on the target location in the visual display when the search is difficult or serial. When attention has to shift randomly between items in the search array, only the target in the outer columns shows strong Simon effect. The target in the inner columns near fixation shows reduced or null effect. This laterality of the Simon effect provides evidence in favor of the hypothesis that the spatial code for the Simon effect emerges from the dynamic attention-shift or attention-focus rather than from a static referential frame. From an ecological point of view, the dynamic spatial code is more flexible than the static code as we often need to search for a target among distractors. This dynamic spatial coding probably makes it easier for us to cope with in the complex and changeable environment in which we live.