Keywords

Research into the visual control of movement over the past 30–40 years has focused predominantly on cortical networks. These cortical networks are fed by information travelling along pathways from retina to occipital cortex and then to motor cortical areas via parietal cortex (dorsal “where” stream) or temporal cortex (ventral “what” stream) (Ungerleider and Mishkin 1982). Although these cortical networks are very powerful, they are not the only routes by which visual information can influence action. This chapter reviews some behavioral, neurophysiological, and neuroimaging data which converge on the idea that subcortical structures have a significant role to play in the visual control of reaching, even in primates.

5.1 Fast Arm Reactions with Limited Modifiability by Intention

It was thought for a long time that visual information influences movement through a process that is relatively sluggish compared to other sensory channels. For example, proprioceptive signals can drive muscle responses in the human hand at latencies of ~ 25 ms through spinal circuitry, but even when the motor cortex is involved proprioceptive input can evoke hand-muscle responses at a remarkably short latency of 50 ms (Marsden et al. 1976). In contrast, the latency of visually driven reactions was traditionally put at 200–300 ms (Keele and Posner 1968; Beggs and Howarth 1970). From the 1980s onwards, however, experiments investigating the speed at which new visual information can modify ongoing limb movements revised this value down dramatically to 100–150 ms (Carlton 1981; Soechting and Lacquaniti 1983; Zelaznik et al. 1983; Paulignan et al. 1990, 1991; Prablanc and Martin 1992).

The explanation for this halving of visuomotor latency was not immediately apparent. It could have been due to methodological differences or possibly due to technological advances offering more sensitive means of recording limb trajectories. Another and more physiological possibility is that these discrepant latencies actually reflect the operations of distinct visuomotor processes. This was indeed the conclusion that was reached based on the results of an experiment that investigated the ability of human subjects to modify voluntarily their visually driven reach adjustments (Day and Lyon 2000).

In this experiment, subjects were seated in a darkened room and were required to reach out and touch an illuminated disk with the index finger. In random and sparse trials (one in three), at the instant the finger was lifted, the illuminated disk slid 10 cm either to the right or the left with equal probability. In one block of trials, subjects were instructed to keep trying and touch the disk even if it moved. In a separate block, they were instructed to touch the disk if it remained stationary, but if they saw it move, then to move their finger in the opposite direction. In effect, this was an anti-reach task not dissimilar to that of an oculomotor anti-saccade task (Hallett 1978; Hallett and Adams 1980). The idea here was to see whether the fast visual-evoked reach adjustments are truly voluntary. If so, subjects should have no difficulty in replacing the fast response in the direction of target movement with an equally fast response in the opposite direction during the anti-reach task. The results of the experiment showed clearly that subjects were unable to do this.

When the target moved during the pro-reach task, subjects adjusted the trajectory of the reaching finger at latencies of between 120–160 ms, ending up accurately on the shifted target. In the anti-reach task, more often than not the finger’s trajectory was also adjusted at short latency, but as in the pro-reach task this acted to take the finger in the direction of the target rather than in the opposite, instructed direction (Fig. 5.1). Despite this inappropriate early adjustment, the finger underwent a subsequent reversal at around 220 ms, this time to drive the finger in the opposite direction to comply with the instructions.

Fig. 5.1
figure 1

Spatial and temporal properties of arm reach adjustments evoked by a moving target. a 3-D superimposed trajectories of the finger from multiple trials of one representative subject when the target remained stationary (green), moved right (red), or moved left (blue). The subject executed either a normal reach (left panel) or an anti-reach (right panel) task (see text). Note the initial deviation of the finger towards the target in both tasks. b Latency histograms of the initial deviation of the finger in all trials in which the target moved. Responses grouped in 10 ms bins and shaded according to whether in the correct (black bars) or incorrect direction (white bars). Note for the anti-reach task most responses were initially in the incorrect direction, with the same latency as correct response in the normal reach task. The small number of initially correct responses in the anti-reach task tended to occur with longer and more variable latency. (Modified from Day and Lyon 2000)

Full size image

The two sequential responses observed in the anti-reach task are highly suggestive of two processes being active. One appears to be a fast-acting process that spatially links the arm movement with the target movement in a compatible and relatively inflexible way. The other process takes longer to act but is flexible allowing an arbitrary spatial association between the visual stimulus and the action. It is tempting to speculate that this slower, flexible process involves cortical processing (Wise et al. 1996) whereas the fast inflexible one is subcortically organized.

5.2 Does the Leg Behave like the Arm?

Most people spend a good proportion of their waking hours reaching for objects with their hand. Perhaps with the exception of professional soccer players, the foot is not routinely used in this way. However, ordinary walking often requires accurate foot placement that relies upon on-line visual control (Reynolds and Day 2005b). It is reasonable to enquire, therefore, whether the lower limb has access to the same fast-acting visuomotor process that controls the upper limb. One might expect this to be the case if upper-limb reaching movements have evolved from stepping movements, as has been suggested (Georgopoulos and Grillner 1989). On the other hand, mid-flight adjustments of a stepping foot carry potentially serious consequences that upper limb adjustments do not. This arises from the need to maintain balance when stepping. It has been shown that the trajectory of the body during a step and the position of the stepping foot at the end of the step are closely coordinated and determined before the foot leaves the ground (Lyon and Day 1997, 2005). Any mid-flight alteration of stepping-foot trajectory will violate this relationship and potentially leave the body in an unstable state should the foot return to the ground at an unplanned location.

To examine this issue, experiments analogous to the arm-reaching experiments were performed using a jumping floor-mounted target which subjects stepped onto (Reynolds and Day 2005a). In one condition, subjects stepped in a natural fashion onto the rectangular illuminated target, which in some trials could jump laterally or medially at the instant the foot left the ground. In a second condition, the same stimuli were employed with the subject upright, but now fully supported by an external framework. In this latter condition, therefore, the usual balance constraint was not present and the leg movement task more closely resembled an arm-reaching task. The key result of this study was that the jumping target evoked short-latency (< 150 ms) mid-flight adjustments of the leg indistinguishable in latency from those of the arm (Fig. 5.2b). The amplitude of the leg adjustment, particularly for medial target jumps (Fig. 5.2a), was less during free steps compared to frame-supported steps, reflecting the limitations imposed by the need to maintain balance. However, the latency was identical irrespective of whether or not the body was supported by the frame. Thus, it seems that the leg, like the arm, has access to a fast-acting visuomotor process, but with additional integration with balance processes. Given the correspondence in latency, a parsimonious explanation is that the upper and lower limbs share a common fast-acting visuomotor mechanism.

Fig. 5.2
figure 2

Spatial and temporal properties of foot adjustments when stepping onto a floor-mounted jumping target. a Group mean final foot position at the end of the step when the target remained stationary (black), jumped laterally (red), or jumped medially (blue). Note the smaller deviation of the foot for medial target jumps. b Group mean mediolateral acceleration of the foot with time (lateral positive), colour coding as in a. Note the short latency of foot adjustment irrespective of target-jump direction. (Modified from Reynolds and Day 2005)

Full size image

5.3 Testing the Subcortical Hypothesis

The opportunity arose to test the hypothesis that the fast (< 150 ms) visual-evoked reach adjustment stems from a subcortical visuomotor process (Day and Brown 2001). This was provided by a patient who was discovered to have agenesis of the corpus callosum and an absent or vestigial anterior commissure, and therefore no direct connection between the left and right cerebral cortices. She was unaware of her condition and was otherwise healthy (apart from a headache that initiated the brain magnetic resonance imaging (MRI) scan to reveal the abnormality) with normal functioning corticospinal tracts. The idea was to measure the crossed–uncrossed latency difference in this patient, a method that was conceptualized 100 years ago (Poffenberger 1912) and which had been used previously in the study of split-brain patients (Jeeves 1969; Kinsbourne and Fisher 1971; Milner 1982; Milner et al. 1985; Clark and Zaidel 1989). The method relies upon cortical lateralization whereby visual information in each hemifield is transmitted to contralateral occipital cortex while each upper limb is controlled by contralateral motor cortex. If the visual hemifield and responding limb are on the same side (e.g., right arm reaching for right-sided target while looking straight ahead), then all cortical processing can be done within one cerebral hemisphere resulting in an “uncrossed” response. However, if the arm and visual hemifield are on opposite sides (e.g., right arm reaching for left-sided target), then information has to pass from one hemisphere to the other resulting in a “crossed” response. In the absence of a corpus callosum, which is the most direct route for communication between the left and right cortices, a more circuitous route is required resulting in a crossed response with a longer latency than normal.

To establish the soundness of this theory in the acallosal subject, a task was employed that required an arbitrary movement in response to a visual stimulus. It was thought that the arbitrary association between visual stimulus and action would engage a cortical network (Wise et al. 1996) and reveal a significant crossed–uncrossed latency difference. The tasks were to lift vertically either the index finger (distal movement) or the forearm (proximal movement) in response to illumination of a disk randomly presented laterally in one or the other hemifields while fixating a central light-emitting diode (LED). As predicted, the crossed reaction time in the acallosal subject was 36 ms slower on average than the uncrossed reaction time (Fig. 5.3a), a difference that was not present in healthy subjects.

Fig. 5.3
figure 3

Manual responses to visual stimuli appearing in the right or left hemifield in a subject with agenesis of the corpus callosum. a Reaction time to initiate arbitrary motor tasks (finger extension, elbow flexion) in response to target appearance in right (red) or left (blue) hemifield during central fixation. Note the significant interaction between arm side and target side for both arbitrary tasks. b Mediolateral motion of the finger and eyes when reaching to a central target which unpredictably jumped into right (red) or left (blue) hemifield during the reach. Top traces show mean mediolateral velocity of finger in jump trials after subtraction of mean no-jump trials. Bottom traces show mediolateral motion of the eyes in jump trials recorded electro-oculographically (high-pass filtered). Note the same short latency of finger deviation irrespective of limb side or target side, and that eyes deviated towards target after limb movement was initiated. P-values above targets (left panels) denote probability of target appearance at respective locations. (Modified from Day and Brown 2001)

Full size image

Having established a clear crossed–uncrossed latency difference in the acallosal subject for arbitrary visuomotor associations, the next step was to see whether the same difference was present for the nonarbitrary task of reaching for a jumping target. Importantly, the target and its locations for this task were identical to that used in the arbitrary association task. The critical observations were that for this reaching task, the reaction times were considerably faster than for arbitrary visuomotor associations (120 vs. 320 ms) and were the same irrespective of which hand was used and which hemifield the target jumped into (Fig. 5.3b). In other words, for the acallosal subject, there was no apparent crossed–uncrossed latency difference when executing a mid-flight reach adjustment to acquire a displaced target. This result could not be explained by very early eye movements bringing the displaced target back into central vision since for this subject, lateral saccades always occurred after the arm adjustment was initiated (Fig. 5.3b). The compelling conclusion was therefore that the visual-evoked reach adjustment did not use the cortical networks employed for arbitrary visuomotor associations. This suggested strongly that subcortical centers were being employed to drive the hand towards visually presented targets.

5.4 Evidence for Involvement of the Superior Colliculus in Visually Guided Reaching

The superior colliculus and underlying mesencephalic reticular formation are good candidates for a subcortical visuomotor processing centre controlling the limbs. The superior colliculus receives visual input directly from the retina (Kaas and Huerta 1988), and, although classically involved in gaze control through control of eye (Sparks 1978) and head (Cowie and Robinson 1994), was discovered also to contain arm-movement-related neurons (Werner et al. 1997a). These neurons occur throughout the depth of the monkey’s superior colliculus, apart from the most superficial layers, as well as in the underlying mesencephalic reticular formation. The “reach” neurons have a different spatial distribution to visually responsive cells or saccadic units in that they occur most frequently in the deeper layers (Werner et al. 1997b). For retinal information to drive arm movements, visual signals need to undergo transformation from an oculocentric coordinate frame to an arm-centered frame. There is some evidence of this sort of transformation in the superior colliculus as shown by two kinds of reach neurons (Stuphorn et al. 2000). One type, located in intermediate layers of superior colliculus, has links to an oculocentric frame in that its firing is gaze-related. The other type, located in the deep layers of superior colliculus and underlying mesencephalic reticular formation, is gaze-independent and therefore associated with an arm-centered frame. Their firing patterns correlate well with electromyography (EMG) patterns of arm muscles during reaching movements, particularly for the proximal muscles of the shoulder girdle (Stuphorn et al. 1999).

Evidence in man for the involvement of these structures in reaching control comes from a functional MRI (fMRI) study (Linzenbold and Himmelbach 2012). In this study, human subjects simply reached with either arm to visual targets presented in the left or right hemifields, whereupon increases in the BOLD signal were observed in the region of the superior colliculus. Unsurprisingly, some of this signal change could be attributed to saccadic activity, but there was another dissociable activity present. This activity, in contrast to saccade-related activity, tended to be lateralized to the reaching arm and occurred in deeper layer locations. Thus, the spatial distribution of these human signals corresponds to the distribution of reach neurons in the monkey superior colliculus and plausibly reflects the operation of an analogous process.

If activity within the superior colliculus is rapidly able to influence reaching movements, there must be a fast-acting route that transmits the motor signal from colliculus to spinal motoneurons. Courjon and colleagues showed that such a motor pathway is functional in the cat during a visually guided reach (Courjon et al. 2004). They stimulated in the deep layers of the superior colliculus using a 300 Hz train of stimuli for 70–200 ms during a reach for a piece of food. This stimulus consistently caused the cat’s reaching limb to deviate from its unstimulated trajectory with an average latency of just 56 ms. In about half the stimulated trials, this deviation occurred without a concomitant gaze or head perturbation thereby ruling out the possibility of the limb deviation being indirectly evoked by a gaze shift. The output pathway could be via direct tectospinal fibers, which project to C3–C4 propriospinal neurons (Illert et al. 1978) thereby engaging spinal reach machinery. They are dense in the cat (Nudo and Masterton 1989; Olivier et al. 1991), but much less so in other mammals including primates (Nudo and Masterton 1989). An alternative and not mutually exclusive possibility is that motor signals travel via the tectoreticulospinal tract (Werner et al. 1997b). Alstermark and colleagues provided some support for the involvement of this pathway by demonstrating a prolongation of the cat’s reach adjustment latency to a jumping target after a presumed lesion of the tectoreticulospinal tract (Alstermark et al. 1987).

5.5 Indirect Evidence for Reticulospinal Involvement in Man

Reticulospinal tracts are thought to transmit the auditory startle response from the caudal reticular formation of the human brainstem to spinal cord both in healthy subjects (Brown et al. 1991a) and in patients with pathological exaggerated startle or hyperekplexia (Brown et al. 1991b). This raises the possibility that the putative subcortical mechanism controlling visually guided reaching may share some circuitry with the auditory startle mechanism. This was investigated during stepping to visual targets by interacting target jumps and loud sounds (Reynolds and Day 2007).

In this experiment, subjects stepped naturally onto an illuminated rectangle, which in a third of trials jumped to the left or right with equal probability at the moment the foot lifted from the ground. In just 18 of the total 240 trials, a startling loud sound (120 dB SPL) was given through headphones at the point of foot lift, but with the startle trials occurring with equal probability during no-jump, jump-right, and jump-left trials. Therefore the loud sound conveyed no information about the presence of a target jump or its direction. As in a previous study (Reynolds and Day 2005), the jumping target evoked a fast adjustment of the foot trajectory, at 134 ms on average when measured from acceleration traces, with medial displacement magnitudes being less than lateral displacements. The loud sound had no effect on control steps or on the magnitude of foot displacement following a target jump. However, the loud sound did affect the latency of the foot correction, shortening it by some 20–30 ms (Fig. 5.4).

Fig. 5.4
figure 4

Effect of loud sound on latency of foot adjustments to a jumping target during a step. a Group mean mediolateral acceleration of foot during trials with no target jump (black), lateral target jump (red) or medial target jump (blue), and with (dashed lines) or without (continuous lines) a loud sound at moment of foot lift from floor. b Same as a but with foot acceleration during no-jump conditions subtracted from all traces. Note the hastening of response to target jump when combined with a loud sound. (Modified from Reynolds and Day 2007)

Full size image

This hastening of the visual-evoked foot trajectory adjustment was not a nonspecific effect of a startle because the foot always moved in the direction of the target jump when present and did not deviate during no-jump trials. Nor was it due to release of a subcortically stored motor program (Valls-Sole et al. 1999; Carlsen et al. 2004) since the required adjustment was not known in advance of the sound stimulus. Instead, it seemed that the sound stimulus interacted with the visual stimulus provided by the target jump to cause a reduction in visuomotor processing time. By taking account of likely afferent and efferent conduction delays, Reynolds and Day (2007) estimated that even a modest hastening of 18 ms would represent a substantial reduction of 30 % in central processing time. It is plausible that this interaction occurs in the brainstem, possibly in the superior colliculus, but perhaps more likely in the reticular formation.

5.6 Indirect Evidence for Superior Colliculus Involvement in Man

When there is choice in selecting one action over another, and if the selected action is determined by an external signal such as a visual stimulus, it is often found that the time taken to react increases with the number of choices available. This relationship between number of response choices and reaction time was formalized by Hick using information processing theory (Hick 1952) and is often referred to as Hick’s law. Although Hick’s law holds for many manual tasks, it does not hold for visually guided saccades. Kveraga et al. (2002) showed that the latency of a saccade to a visual target is the same irrespective of whether there are eight possible targets or just one. They explained this violation of Hick’s law by referring to the anatomical arrangement of the superior colliculus, where visual afferents form a retinotopic map that lies in spatial register with an oculomotor map determining saccade direction and amplitude. With this arrangement, a visual stimulus on the retina automatically evokes a saccade that brings the visual target onto the fovea, making the central processing time immune to the number of possible target locations. When this highly compatible spatial relationship between stimulus and response was destroyed by asking subjects to make eye movements in the opposite direction to a peripheral visual stimulus (anti-saccade), Hick’s law was reinstated (Kveraga et al. 2002).

It could be argued that fast visually guided limb movements also should violate Hick’s law if they utilize the neural machinery of the superior colliculus. This was tested using a centre-out pointing task (Reynolds and Day 2012). For this, subjects held a finger stationary in front of a central target on a vertical board and were required to make a rapid discrete movement in response to the appearance of a peripheral target (Fig. 5.5a). Four visual targets were located 15 cm above, below, left and right of the central target. To eliminate temporal uncertainty about when an event might happen, the subject initiated a trial by raising the opposite index finger from a touch switch. In 67 % of trials this did nothing (null trials), but in the remaining 33 % it caused the central target light to be extinguished and one of the four peripheral targets to be illuminated. Subjects were given advance information which of the peripheral targets might be illuminated in a trial (between one and four targets) thereby manipulating the number of action choices. In one condition, the stimulus–response compatibility was high in that subjects were required to move their finger in the direction of the illuminated target (normal). In a second condition, the stimulus–response(S–R) compatibility was low as they were required to move their finger in a direction 90° clockwise to the illuminated target (orthogonal).

Fig. 5.5
figure 5

Effect of stimulus–response compatibility and number of choices on reaction time to visual stimulus. a Experimental set-up (see text for details). b Reaction time against number of choices for high S–R compatibility (normal pointing) and low S–R compatibility (orthogonal pointing) conditions. Reaction times for erroneous responses during orthogonal pointing are shown when the finger moved in the direction of the target (T-error) or in some other incorrect direction (O-error). Note the constancy and brevity of normal pointing reaction time regardless of number of choices. (Modified from Reynolds and Day 2012)

Full size image

As predicted, for normal pointing with high S–R compatibility, the reaction time was fast at around 140 ms and remained the same regardless of the number of choices. In contrast, for the orthogonal pointing task the reaction time was slower, being 173 ms for the 1-choice condition (i.e., simple reaction time), and it increased with number of choices, thus obeying Hick’s law (Fig. 5.5b). Therefore, in many ways the pointing behavior was very similar to the saccade behavior described above (Kveraga et al. 2002). As with the eyes, the violation of Hick’s law when there was a direct spatial correspondence between the visual stimulus and the limb movement suggests a mechanism with relatively hard-wired direct mapping between the two. A parsimonious interpretation is that the superior colliculus lies at the heart of the mechanism for visually guided pointing as it does for visually guided saccades. The radically different behavior during orthogonal pointing suggests the possibility of a slower mechanism being brought into play. Of course, for a slower mechanism to dominate the faster mechanism, the latter would have to be suppressed in some way. Interestingly, subjects tended to make errors by sometimes pointing incorrectly at the illuminated target in the orthogonal condition (T-error in Fig. 5.5b), with these erroneous movements usually being initiated faster than the correct movements. This is compatible with two competing mechanisms where there is an incomplete suppression of the faster process. As suggested in an earlier section to explain anti-reach behavior (Day and Lyon 2000), orthogonal pointing could engage a cortical mechanism, which would give the advantage of flexibly associating any action with a visual signal thereby avoiding the necessity of a direct spatial correspondence between stimulus and response. The cost is a longer response latency that grows with choice (Hick’s law) because of the additional information processing required for stimulus identification and action selection.

5.7 Communication Between Cortical and Subcortical Visuomotor Processes

The picture painted so far in this chapter is of two processes for visual guidance of limbs, one subcortical and the other cortical. The subcortical process is fast, but rigid, and is well suited for direct interactions between the limb and an object when both occupy a common space. In contrast, the slower cortical process conceivably is infinitely flexible and may be better suited for visuomotor interactions during tool use. This flexibility effectively detaches the limb from the object and allows any arbitrary spatial relationship between the two, for instance, tracking vertical movements of a target on a computer screen with a cursor controlled by forward–backward movements of a mouse. However, it is unlikely that the two processes operate independently of each other. As a minimum, the subcortical process would need to be suppressible under certain circumstances. As we have seen, when a person is engaged in interacting with an object directly with the limb, suppression is difficult to achieve and often incomplete. Thus, during an anti-reach task, invariably the limb is initially drawn towards the target’s new position, although with less vigor compared to a standard reach (Day and Lyon 2000). Similarly, orthogonal pointing is possible but at the expense of occasional errors towards the visual stimulus, indicating an intrusion of the fast process (Reynolds and Day 2012). Presumably, there are stronger suppression signals available when the limb operates remotely from the near space of an object or via an interposed tool. Communication between the two processes would also be necessary if, as is likely, they act in concert during direct interactions between a limb and an object. There is a wealth of connections between action-related areas of cortex and relevant subcortical structures such as the superior colliculus (e.g., Kuypers and Lawrence 1967; Goldman and Nauta 1976; Catsman-Berrevoets et al. 1979; Fries 1984, 1985) and the reticular formation (e.g. Catsman-Berrevoets and Kuypers 1976; Keizer and Kuypers 1984). These fibres could provide the communication required for cooperation between cortical and subcortical visuomotor processes.