New insights into action–perception coupling

Abstract

According to a view that has dominated the field for over a century, the brain programs muscle commands and uses a copy of these commands [efference copy (EC)] to adjust not only resulting motor action but also ongoing perception. This view was helpful in formulating several classical problems of action and perception: (1) the posture-movement problem of how movements away from a stable posture can be made without evoking resistance of posture-stabilizing mechanisms resulting from intrinsic muscle and reflex properties; (2) the problem of kinesthesia or why our sense of limb position is good despite ambiguous positional information delivered by proprioceptive and cutaneous signals; (3) the problem of visual space constancy or why the world is perceived as stable while its retinal image shifts following changes in gaze. On closer inspection, the EC theory actually does not solve these problems in a physiologically feasible way. Here solutions to these problems are proposed based on the advanced formulation of the equilibrium-point hypothesis that suggests that action and perception are accomplished in a common spatial frame of reference selected by the brain from a set of available frames. Experimental data suggest that the brain is also able to translate or/and rotate the selected frame of reference by modifying its major attributes—the origin, metrics and orientation—and thus substantially influence action and perception. Because of this ability, such frames are called physical to distinguish them from symbolic or mathematical frames that are used to describe system behavior without influencing this behavior. Experimental data also imply that once a frame of reference is chosen, its attributes are modified in a feedforward way, thus enabling the brain to act in an anticipatory and predictive manner. This approach is extended to sense of effort, kinesthetic illusions, phantom limb and phantom body phenomena. It also addresses the question of why retinal images of objects are sensed as objects located in the external, physical world, rather than in internal representations of the brain.

Introduction

Although action and perception may involve different central pathways and brain structures (Goodale and Westwood 2004; Milner and Goodale 1988), they are usually interrelated (Gibson 1968; Goodale and Westwood 2004; Warren 2006; Merriam et al. 2007; Turvey 2007). It is often assumed that action–perception coupling is based on neural signals that represent the anticipated patterns of motor commands, i.e., electromyographic (EMG) activity required for action. After a seminal paper by Von Holst and Mittelstaedt (1950/1973), these anticipatory signals are called efference copy (EC) but the term sensation of innervation (Helmholtz 1866/1963) or corollary discharge (Sperry 1950) have a similar connotation (Bridgeman 2007). According to a more recent, internal model theory, the brain pre-computes and then specifies ECs for actions based on imitations of the properties of neuromuscular elements interacting between themselves and with the environment (Wolpert et al. 1995; Kawato 1999; for review see Ostry and Feldman 2003).

The EC concept has been used to explain several aspects of action and perception. Helmholtz (1866/1963) tackled the problem of visual space constancy related to the fact that motion of retinal images caused by changes in the gaze direction during saccades is not sensed as world motion. He suggested that, in order to perceive that the world is stable, a copy of motor commands to eye muscles is used during visual processing at some brain level to compensate for the motion of the retinal image.

Von Holst and Mittelstaedt (1950/1973) applied the EC concept not only to visual constancy but also to motor actions. They emphasized that actively specified postures of the body or its segments are stabilized by proprioceptive, optokinetic, and vestibular reactions such that deviations from an initial posture elicited by external forces are met with position- and velocity-dependent resistance. Deviations from an initial posture can result not only from involuntary movements elicited by external, mechanical perturbations but also from intentional, self-initiated movements to another posture. It appears that, in contrast to involuntary movements, intentional movements away from a stable posture do not evoke resistance of posture-stabilizing mechanisms. Von Holst and Mittelstaedt used the EC concept to explain why these mechanisms work differently in the two types of movements (the posture-movement problem).

Kinesthesia (sense of motion and position of limb segments) is associated with yet another problem. Experimental data described below show that proprioceptive and cutaneous afferents typically transmit ambiguous information about the position of body segments to the brain. Specifically, position sense is predominantly based on signals related to changes in muscle lengths and joint angles transmitted to the brain by primary (Ia) and secondary (II) afferents of muscle spindle receptors (Matthews 1972; Windhorst 2007). Primary afferents and, to a less degree, secondary afferents are also sensitive to velocity of muscle stretch. Complied with the principle of action–perception coupling, spindle afferent signals are involved not only in kinesthesia but also in position- and velocity-dependent regulation of activity of α-motoneurons. Terms stretch reflex or proprioceptive feedback are usually used to denote this type of regulation. The output of spindle receptors can be modified by central (efferent) influences from γ-motoneurons that innervate the fibers of muscle spindle receptors. These γ-influences are controlled by descending brain systems. Depending on the activity of γ-motoneurons, spindle afferent output may or may not be related to the position of body segments. In other words, because of γ-influences, spindle afferent signals often transmit ambiguous positional information. For example, groups Ia and II spindle afferent discharges in humans hardly changed during a slow isotonic finger movement that tracked a ramp-and-hold trajectory on the oscilloscope screen and caused length changes of the muscle containing the spindles (Hulliger et al. 1982). Despite the position-independent spindle discharges, position sense remains fairly adequate in this situation: the finger is perceived as moving, even if the movement is made with eyes closed.

Another example of positional ambiguity of spindle afferent activity is that, in isometric condition, discharges of spindle afferents increase with the increasing isometric force (Vallbo 1974) whereas the arm is correctly perceived as motionless.

Many muscular, articular, and cutaneous receptors do not have efferent innervation, and one can think that unlike muscle spindles, they can correctly inform the brain about limb position in isotonic and isometric tasks. However, the spinal neurons that transmit to the brain such information also receive either independent signals from the brain or/and signals from spindle afferents, as is the case for neurons of ventral and dorsal spino-cerebellar tracts (Arshavsky et al. 1978). Therefore, ambiguity of positional information is characteristic of all afferent signals involved in kinesthesia.

How is it possible that our position sense is good despite ambiguous positional signals from receptors? This question reflects the essence of the kinesthesia problem. McCloskey (1981) and Gandevia (1996) assumed that the nervous system uses an EC to get an adequate sense of limb position despite the positional ambiguity of afferent signals.

Practically all sensory systems are centrally controlled, either directly, by efferent inputs to receptors, or indirectly, by central inputs to neurons at which sensory afferents terminate. This is the case for vestibular and retinal receptors (Warr 2004; Honrubia and Elliott 1970). Therefore, the kinesthesia problem is a part of a general problem of how the nervous system deals with ambiguous sensory signals in order to adequately perceive body parts and objects in the environment.

Efference copy theory has been dominant for over a century and is continued to be appreciated by many researchers (e.g., Wolpert and Flanagan 2001). This is understandable: the theory and its extension in the form of the internal model formulation carry an important idea that the organism may anticipate and, if necessary, prevent negative consequences of self-initiated actions, the idea supported by many studies (e.g., Belen’kiĭ et al. 1967; Wing et al. 1997; Levin and Dimov 1997; Pilon et al. 2007). However, since the time when the EC theory emerged (Helmholtz 1866/1963) and shaped in a more or less final form by Von Holst and Mittelstaedt (1950/1973), many physiological data that could challenge the theory were unknown. Bridgeman (2007) noted that, by 1989 evidence was converging that should have eliminated compensation theories for visual space constancy, but it did not. This paradoxical situation might be related to the fact that a major and hardly questionable idea about predictive power of the brain promoted by the theory, shielded its faulty aspects. Bridgeman (2007) thinks that the lack of an alternative keeps the theory floating.

Here it is shown that the solutions to the three classical problems of action and perception (the posture-movement problem, problems of kinesthesia, and visual space constancy) offered by the EC theory in particular or by the internal model theory in general are physiologically unfeasible. A major purpose of this paper is to present alternative solutions to these problems. This analysis is extended to sense of effort, kinesthetic illusions, phantom limb and phantom body phenomena.

Posture-movement problem

It is necessary to explain why intentional movements away from a stable posture can be produced without evoking resistance of the posture-stabilizing mechanisms, although these mechanisms regularly come into play when movements are elicited involuntary, by external forces. One might think that the principle of reciprocal inhibition (Sherrington 1910) had solved this problem since the onset of intentional movement is associated not only with activation of agonists but also with inhibition of antagonist muscles (Hufschmidt and Hufschmidt 1954), thus preventing resistance of antagonists to movement. However, afferent feedback returns to antagonists when they are re-activated later in the movement (Latash and Gottlieb 1991; Adamovich et al. 1997). The principle of reciprocal inhibition leaves unanswered the question why the restored afferent feedback does not drive the limb to the previously stabilized position and instead decelerates it to a new position.

Resistance of posture-stabilizing mechanisms to intentional movement could be prevented if the afferent feedback to muscles (proprioceptive reflexes) were fully suppressed when movement is made. Von Holst and Mittelstaedt (1950) correctly rejected this idea by noticing that the initial and final postures are usually stabilized with afferent feedback in a similar way. In addition, recent studies showed that proprioceptive feedback remains fully functional not only before and after but also during movement, even if movement is made as fast as possible (Adamovich et al. 1997; Pilon and Feldman 2006).

In a seminal paper, Von Holst and Mittelstaedt (1950/1973) offered an EC-based solution to the posture-movement problem (Fig. 1). They considered position- and velocity-dependent afferent feedback to motoneurons (reafference) resulting from intentional, self-initiated motion. They distinguished this feedback from that elicited by external forces (exafference). They further assumed that, to prevent resistance of posture-stabilizing mechanisms to intentional movement, control centers not only send pre-programmed motor commands (EMG activity) to muscles but also use a copy of these commands (i.e., an EC) in order to neutralize the influence of the reafference on motoneurons. If the compensation is incomplete, the difference (residue) between the EC and reafference signals is sent to high control levels where it is algebraically summed with the previous EC (Fig. 1). Following this continuous process, the EC eventually compensates for the reafference. It was also assumed that, unlike the reafference elicited by intentional motion, the component of afferent feedback that existed before the movement onset (exafference) remains responsible for involuntary responses to perturbations elicited by external forces and stabilizes the new posture like the previous one. Von Holst and Mittelstaedt (1950/1973) called this solution of the posture-movement problem the reafference principle. A similar EC-based compensatory scheme (Fig. 1) is used in the more recent, internal model theory and other theories based on the idea of pre-programming of motor commands for actions (e.g., Wolpert and Ghahramani 2000).

Fig. 1

A hypothetical solution of the posture-movement problem based on the notion of efference copy (EC) (redrawn from Von Holst and Mittelstaedt 1950/1973). When control centers send motor commands (efference) to muscles to intentionally move an effector, they simultaneously generate a copy of these commands (EC) to neutralize the changes in the afferent signals elicited by motion (reafference). If the compensation is incomplete then the difference (residue) is sent to control centers. They change the EC until the reafference is fully compensated. Afferent signals (exafference, not shown) resulting from involuntary deviations of the effector from the initial position by external forces, remain uncompensated and stabilize the new position as the initial one

The detailed analysis below shows that the reafference principle is inconsistent with basic properties of proprioceptive feedback to muscles as well as with typical EMG characteristics of human movements. Most importantly, it conflicts with the empirically established solution of the posture-movement problem (Asatryan and Feldman 1965; Feldman and Levin 1995; Archambault et al. 2005).

One limitation of the EC-based solution of the posture-movement problem is that it assumes that it is sufficient to compensate for the reafference in order to prevent its resistance to movement. It is known that position- and velocity-dependent intrinsic (“damping–elastic”) forces of active muscle fibers (“preflexes”) are not less essential for stability of posture and movement (Nichols and Houk 1976; Loeb et al. 1999). These forces increase following the enhancement of the background EMG activity when an external load counteracting the muscles is added. Moreover, unlike afferent feedback, preflexes are generated practically without delay, which helps stabilize posture and movement whether it is made intentionally or involuntary (e.g., Gribble et al. 1998; Pilon and Feldman 2006). Indeed, the velocity-dependent (damping) component of preflexes is reduced to zero after the movement offset and therefore may only intermittently resist intentional movement. In contrast, even if the reafference is suppressed during intentional movement, the position-dependent (elastic) component of preflexes would be accumulated in proportion to the deviation of the limb from the initial position and eventually returned the limb to this position after the end of movement. This essential aspect of the posture-movement problem was not addressed in the EC theory.

The EC theory conflicts with basic empirical data characterizing isotonic movements (i.e., movements made against a constant, in particular, zero external resistance). Specifically, after transitional EMG bursts in such movements, the limb arrives in a new position in which the EMG activity eventually returns to its pre-movement tonic level (Fig. 2b; Ostry and Feldman 2003; Foisy and Feldman 2006). Thereby, muscles not only hold the new position without tending to return the limb to the initial position but also actively resist deviations of the arm from the new position if perturbed (Fig. 2a, c).

Fig. 2

An experimental observation that the efference copy theory fails to explain: after isotonic movements, muscle activity usually returns to its pre-movement level. There are shown: a rapid elbow flexion movement (b) and reactions of muscles to passive oscillations at the initial (a) and final (c) positions. The activity of elbow flexor (BB, BR) and extensor (TRl, TRm) muscles (four lower traces in b) at the initial elbow position is practically zero (background noise level) and, after transient EMG bursts, it returns to zero at the final position. Muscles are activated in response to passive oscillations of the arm at the initial (a) and final (c) positions, implying that, at both positions, motoneurons of these muscles were near their activation thresholds, rather than in the state of relaxation. An elastic connector was used to compensate for the small passive torque of non-active flexor muscles at the initial position of about 140°. The compensation was unnecessary for the final position (about 90°) since it is known that at this position the torque of passive elbow muscles is zero (Ostry and Feldman 2003). BB, BR, TRl, TRm: biceps brachii, brachioradialis, triceps lateralis, triceps medialis, respectively)

The fact that the muscle activity may be the same at pre- and post-movement positions (see also Foisy and Feldman 2006) can be explained in simple physiological terms: EMG activity is basically related to muscle force (Bigland and Lippold 1954) and if no external force counteracts limb muscles, their activity is eventually reduced to its pre-movement, near zero level. In terms of the EC theory, however, the observation of the same EMG levels in the pre- and post-movement positions implies that, following changes during movement, the EC would also return to its initial pre-movement value. This means that the position-related component of the reafference (not to mention the intrinsic elastic resistance of active muscles) accumulated following limb deviation from the initial position would not be compensated by the EC and would produce muscle forces driving the limb back to its initial position. The EC theory (Fig. 1) is therefore unable to explain why the limb can be stabilized in different positions by muscles generating similar EMG activity at these positions.

In the EC theory, the situation when the pre- and post-movement EC are the same cannot be tolerated: the reafference accumulated during the movement would be not compensated and transmitted as a residue to command centers and added to the previous EC (Fig. 1). As a result, the EMG activity and forces of appropriate muscles would be amplified and eventually neutralize the reafference and its mechanical effects. Thus, the final posture would be held due to additional tonic EMG activity that compensates for the position-related muscle resistance to the deviation from the initial posture. This tonic activity must be produced in addition to the activity needed to accelerate and decelerate the movement and to overcome external forces (loads), if those are present. Results of isotonic experiments (Fig. 2) do not confirm the existence of such an additional, distance-related EMG component predicted by the EC theory (Ostry and Feldman 2003; Foisy and Feldman 2006).

The finding by Hulliger et al. (1982) that in isotonic finger motion spindle dischargers are position-independent (see “Introduction”) can be considered as an empirical example of reafference compensation. However, instead of supporting the EC theory, it also illustrates its failure. Hulliger and his colleagues explained the spindle behavior in a simple physiological way, not relying on the EC concept: γ-efferent signals to agonist muscle spindles continuously increase during the isotonic finger motion but the increase in the spindle discharges is neutralized by reaction of spindles to muscle shortening, making their discharges position-independent.

One can try to explain the spindle behavior in isotonic condition by using the EC concept. In this case, we should assume that the nervous system predicts that spindle discharges of agonist muscles (reafference) would decrease due to muscle shortening during isotonic motion. By increasing an EC for agonist muscles in proportion to muscle shortening, the system prevents the change in the reafference. In this explanation, the EC is identified with the γ-efferent influences on muscle spindles since these influences increase with muscle shortening. The prediction that the EC increases with muscle shortening conflicts with the finding that EMG activity returns to its pre-movement value in isotonic movement (Fig. 2). Therefore, even if reafference compensation is accomplished in a predictive manner, no EC is responsible for that. In addition, this analysis shows that γ-efferent influences do not represent a copy of motor commands to muscles: the former steadily increase, whereas the latter return to their pre-movement values.

There is a more fundamental reason of why the EC theory fails to solve the posture-movement problem: it disregards some basic properties of natural laws that constrain the possible ways the brain can control motor actions. These properties are formulated below to identify some “dead ends” that should be avoided in theoretical approaches to motor control.

Natural laws constrain the ways the brain can control motor actions

Natural laws link up certain properties of the neuromuscular system interacting with the environment. These properties are characterized by variables called here natural variables. Natural laws also include system parameters, i.e., constants or variables that can be changed independently of natural variables but influence the system behavior under the action of natural laws. The movement acceleration, external and muscle forces or torques are natural variables since the relationships between them are governed by laws of mechanics. Mechanical laws also make velocity, position, and trajectory of body segments related to acceleration. Other natural laws relate muscle forces to muscle, reflex and neural properties described by appropriate natural variables. Some of these variables characterize the motor output—the magnitude of EMG activity, muscle force, length and velocity of muscle contraction, stiffness, damping. Note that stiffness of the muscle-reflex system is related to muscle force (Bigland and Lippold 1954). Therefore stiffness, like force, is a natural variable. In contrast, in some non-living systems, stiffness can be changed independently of force (e.g., in a mass-spring device, by heating the spring): in these cases stiffness is considered as a system parameter, not as a natural variable.

Our motor actions obey natural laws, implying that the nervous system can influence but cannot directly pre-determine natural variables. For example, grip force emerges only when the finger pads begin to touch an object and even after that the system cannot fully pre-determine the grip force since it depends on how finger pads interact with the object. This interaction depends on the properties of the object (e.g., soft, solid or fragile) that are beyond the ability of the brain to control. The cases when the object slips of the fingers or collapses illustrate this point.

Theories that did not comply with the rule that the nervous system cannot directly pre-determine the values of natural variables have been unsuccessful. Such is the theory that assumed that a natural variable—the position of body segments—is directly determined by γ-innervation of muscle spindles (Merton 1953). In other words, the theory assumed that the muscle-reflex system acts as a servo-controller that generates forces that are strong enough to nullify deviations elicited by external forces from the position pre-determined by γ-influences. This hypothesis was rejected since it required unrealistically high stiffness of the muscle-reflex system.

The internal model formulation (Wolpert et al. 1995; Kawato 1999) is another theory that also disregards the rule that the nervous system cannot directly specify natural variables. It suggests that actions are produced based on some preliminary neural computation or pre-programming of such natural variables as the muscle force, movement trajectory, EMG activity or its prototype, EC (for review see Ostry and Feldman 2003). To identify a major problem of such a theory, suppose that there is a desire, say, to accelerate elbow flexion. According to the internal model theory, the system pre-computes the muscle torque, T, required for angular acceleration ε (for simplicity, external forces are zero). Newton law ε = T/I (where I is inertia of the forearm and hand together) implies that elbow muscle torque causes acceleration, not the other way around. In contrast, in order to compute T, the system should imitate the inversion of the natural law, T = Iε. Similar inverse-dynamic computations of torques or forces are commonly used to describe mechanical motions of living and non-living bodies, a quite useful method. In contrast, based on the assumption that the brain produces actions based on inverse representations of natural laws, the internal model theory implies that the causality of the natural law (ε = T/I) coexists with the inverse causality (T = Iε) imposed by the specification of torques based on inverse-dynamic computations. Apparently, this theory is intrinsically inconsistent and conflict with empirical observations (Ostry and Feldman 2003) some of which have been described in the previous section.

Indeed, natural laws do not prevent the brain from influencing natural variables indirectly and these laws actually imply how this can be done: the brain can modify system parameters to influence natural variables and guide motor actions in a task-specific way.

It is usually easy to identify parameters that can influence behavior of simple non-living systems such as a pendulum (a mass on the rope). Most known are such pendulum’s parameters as the mass and length of the rope. It is less known that the coordinates of the suspension point are also pendulum’s parameters: they are not conditioned by laws of mechanics but are chosen by the person who suspended the pendulum where he wanted. By changing the coordinates of the suspension point and the rope length, one can prescribe where, in space, and how, in terms of frequency, the pendulum oscillates. Parametric control can thus be effective in modifying essential characteristics of system behavior.

System parameters that the brain modifies to control body actions are hidden. However, at least some of them can be uncovered in simple experiments such as that represented in Fig. 2. Panel B shows a fast elbow movement resulting from EMG bursts in agonist and antagonist muscles. Indeed, these bursts are responsible for muscle forces that are required to accelerate and then decelerate the forearm to the final position. To identify a parameter used by the nervous system to control such a movement and elicit these bursts, we need to focus not on these bursts but on the ability of the system to establish the same near-zero activity of elbow muscles at the pre- and post-movement positions. Mechanical perturbations applied to the forearm (Fig. 2a, c) show that although the background EMG activity of four elbow muscles was near zero at the initial position, these muscles were easily activated in response to mechanical perturbations. This behavior is different from that during full muscle relaxation when no muscle activation occurs in response to mechanical perturbation, unless it is very rapid (as is the case during a knee reflex elicited by hitting the quadriceps tendon with a medical hammer). The ability to fully relax muscles at any elbow angle was demonstrated by Wachholder Altenburger (1927/2002). In contrast, Fig. 2a, c shows that muscles were ready to be activated, rather than fully relaxed.

Reactions to perturbation in Fig. 2a show that α-motoneurons of elbow muscles at the initial position were near their thresholds of activation. This implies that the nervous system was able to set the threshold position of the elbow joint (i.e., the position at which muscles are silent but ready to be activated in response to deviations from this position). After the end of motion, their near-threshold state was restored but at the new elbow position (Fig. 2c), showing that the system was able to shift the threshold position when intentional motion to another position was made. The threshold position is thus one of system parameters controlled by the nervous system.

Somewhat more complicated experiments have shown that the nervous system may set and reset not only the threshold (referent) position for muscles of a single joint (Fig. 2) but also for muscles of the whole arm—referent arm configuration—or even for muscles of the whole body—referent body configuration (Archambault et al. 2005; Foisy and Feldman 2006; St-Onge and Feldman 2004). Moreover, the system can set a referent position of the whole body in the environment and elicit a step or walking by shifting this referent position (Feldman et al. 2007).

The notion that the brain controls actions by changing system parameters also comes out when different steady postures are considered. Specifically, steady postures are associated with an equilibrium state of the neuromuscular system interacting with the environment. This is the state in which the system may remain motionless, muscle and external forces (torques) are balanced, and the relevant neural structures directly or indirectly influencing the motor output maintain a stable (tonic) activity. The notion of equilibrium state also implies that it virtually exists at any time during action and can be reached if stability conditions are met.

Knowing the limb posture and forces that are balanced at this posture does not help in answering the question of what variables pre-determined the choice of this but not another equilibrium state by the nervous system. A physical rule rarely mentioned in textbooks gives a principal answer to this question: the choice between different equilibrium states is pre-determined by system parameters, not by natural variables (Glansdorf and Prigogine 1971). Therefore, the major reason why internal models in general and the EC theory in particular failed to solve the posture-movement problem is that they were unable to go beyond the set of natural variables and indicate system parameters—threshold positions—that are responsible for the choice between different equilibrium states (see also the next section).

This analysis brings us to two additional conclusions. First, our motor skills reflect the ability of the brain to organize, exercise, memorize, select in task-specific way, and modify during learning parametric control of the system. Second, by exercising such a control, the brain takes advantage of natural laws not only in producing desired actions but also in anticipating and preventing undesirable consequences of these actions.

It is important to note that by exercising parametric control of natural laws in action production, the brain does not need to know these laws. For comparison, a person can be an experienced car driver without any knowledge of the mechanical, chemical and thermodynamical laws he/she deals with during driving.

The ability of the brain to parametrically influence the patterns of actions emerging from natural laws makes the impression, albeit mistaken, that the brain somehow knows these laws. The fine difference between taking advantage of and knowing natural laws is obscured in the internal model theory (Wolpert et al. 1995; Kawato 1999) that postulates that the brain is able to imitate natural laws and their inverses in order to produce actions.

Empirical solution of the posture-movement problem

Several experimental studies in humans (Asatryan and Feldman 1965; Feldman and Levin 1995; Archambault et al. 2005; St-Onge and Feldman 2004; Foisy and Feldman 2006; Feldman et al. 2007) have shown that the nervous system has the capacity to reset (“re-address”) all posture-stabilizing mechanisms (resulting from afferent feedback and intrinsic muscle properties) when the limb moves to different positions. Specifically, it was found that the nervous system is able to set and if necessary to reset the threshold (referent) limb position, R (Fig. 2).

Consider the elbow joint when the forearm is placed on a horizontal manipulandum to exclude external forces acting on the forearm. In the absence of external forces, the initial forearm position, Q _i , coincides with the initial threshold position, R _i at which elbow muscles are silent but generate activity and torque in proportion to the deviation and rate of change of the actual limb position (Q) from R _i (Fig. 3a). The EC theory offers two ways of producing a movement, for example, an extension to a final position (Q _f ). First, the system may generate muscle activity and torques that not only accelerate and decelerate the movement but also compensate for the resistive posture-stabilizing torques of antagonist muscles (flexors in the given case). No such compensatory activity is observed when a posture to posture movement is made (Fig. 2). Second, the system may do the same but suppress the posture-stabilizing activity and torques to prevent the limb from moving back to the initial position. This would conflict with the empirical observations that the initial and final postures are stabilized in similar ways.

Fig. 3

Major aspects of empirically based solution of the posture-movement problem. a A body segment (the forearm in this example) can be stabilized at position Q _i that, in the absence of external forces, coincides with the centrally set threshold position, R _i . Muscles are silent at this position but generate EMG activity and resistive torques in proportion to deviation from this position as shown by the shaded areas. b To produce an intentional extension movement, the nervous system shifts, in a feedforward way (see Fig. 4), the threshold position from R _i to R _f . The initial position, Q _i  = R _i , becomes deviated from the new threshold position, resulting in an increase in the activity and torque (vertical arrow) of agonist muscles (extensors in this example). The body segment begins to move toward position R _f and eventually arrives at this position at which the muscle activity returns to its pre-movement, zero level, as observed experimentally (Fig. 2)

Empirical evidence of the threshold position control (Asatryan and Feldman 1965) helps solve the posture-movement problem in a simple way. To produce an intentional elbow extension, the system shifts the referent position in the appropriate direction (R _i  → R _f , Fig. 3b). As explained in the next section, such shifts are accomplished in a feedforward way, such that the forearm remains motionless for some time whereas the threshold elbow position appears to be shifted. In this way, the initial arm posture, Q _i becomes the posture that deviates from the newly specified threshold posture, R _f . Following the threshold shift, elbow extensors will be activated and generate torques (vertical arrow in Fig. 3b). In other words, the same stabilizing mechanisms that otherwise resisted the movement, give rise to the muscle activation and forces that drive the arm away from the initial posture. The movement continues until the gap between the Q and R _f is either fully eliminated (in the absence of opposing or assisting external forces, as in Fig. 3) or diminished to a degree such that the residual muscle activity gives rise to forces just sufficient to counterbalance the external forces (Fig. 5b, c; for more detail, see St-Onge et al. 1997).

By resetting the threshold position, the nervous system not only entirely eliminates resistance of posture-stabilizing mechanisms to movement from the initial posture but also takes advantage of these mechanisms to move the arm and stabilize both the final posture and motion to it. Thus, posture-stabilizing mechanisms in this control scheme do not resist but rather assist the movement. In addition, the nervous system does not need to pre-program EMG bursts or forces required for motion. Instead, they emerge following threshold position resetting (Fig. 3b, vertical arrow). Thus, the threshold position control solves the posture-movement problem in a very efficient way.

Since the threshold position control is a consistent part of the equilibrium-point theory (Feldman et al. 2007), some readers may notice that this theory has been criticized and have the impression that it has been rejected. The claims of rejection have been addressed in detail with the conclusion that they were unfounded (Feldman et al. 1998; Ostry and Feldman 2003; Feldman and Latash 2005). Specifically, two fundamentals underlie the theory. First, intentional actions are produced by changing the equilibrium state of the neuromuscular system interacting with the environment. Second, to make such changes, the nervous system shifts the threshold position of the body or its segments in a task-specific way. Taken together, the two fundamentals identify a global physical factor underlying motor actions and a neural tool that the nervous system uses to influence this factor and thus guide motor actions in the desired manner.

The first fundamental underlying the EP hypothesis is a physical principle applied to systems with position-dependent forces such as those generated by muscle-reflex structures. A known attempt to cast doubt on the applicability of this principle to the neuromuscular system (Gomi and Kawato 1996) was based on misrepresentation of basic muscle-reflex properties and appeared to be misleading (see Feldman et al. 1998; Gribble et al. 1998). The second fundamental is an experimental fact initially demonstrated by Matthews (1959) in decerebrated cats, then by Asatryan and Feldman (1965) in humans. Further studies in decerebrated cats have shown that cortico-, reticulo-, rubro- and vestibulo-spinal pathways influencing α-motoneurons have the capacity to shift the threshold position of the body segments (Feldman and Orlovsky 1972; Nichols and Steevs 1986). Such shifts may result from direct (mono-synaptic) inputs from these pathways to α-motoneurons or indirect (polysynaptic) inputs mediated by spinal interneurons or/and γ-motoneurons. Pre-synaptic inhibition of α-motoneurons also effectively influences the threshold position (Capaday 1995), thus complementing post-synaptic sources of threshold position control.

Criticisms touched upon secondary issues of the EP theory, such as stiffness control and movement equifinality in the cases of velocity-dependent force perturbations (e.g., Lackner and Dizio 1994; Hinder and Milner 2003). These criticisms required clarifications of respective aspects of the theory but in no way shook its two fundamentals (Feldman and Latash 2005). Moreover, the notion of threshold position control has been reinforced in several recent studies showing that this notion is applicable to multiple muscles of the body (e.g., Lestienne et al. 2000; St-Onge and Feldman 2004; Archambault et al. 2005; Lepelley et al. 2006). The functional significance of threshold position control was also shown by Levin et al. (2000) and Musampa et al. (2007). They demonstrated that muscle weakness, spasticity, as well as impaired inter-joint coordination in stroke survivors and individuals with cerebral palsy are primarily caused by deficits in threshold position control.

Indeed, the solid theoretical and experimental basis of the EP theory does not protect it from further testing. In what follows, we investigate the capacity of this theory to solve some problems of perception that other theories seem failed to solve. Before that, it might be helpful to explain how, physiologically, a threshold position is formed and controlled.

Physiological origin and different forms of threshold position control

The observations that the nervous system can modify the threshold position at which muscles become active imply that electrochemical signals descending from the brain to motoneurons are somehow transformed into positional variables, thus placing our actions in a spatial frame of reference associated with the body and its interactions with the environment. It appears that such a transformation results from the well-organized integration of proprioceptive and other sensory inputs with independent control inputs at the level of the motoneuronal membrane that has an electrical threshold (Fig. 4a; Pilon et al. 2007). Specifically, due to predominantly facilitatory proprioceptive feedback from muscle spindles receptors, the membrane potential of the motoneuron increases with the increasing muscle length during a passive quasi-static stretch (Fig. 4b, lower diagonal line). The motoneuron begins to generate spikes when the current membrane potential starts to exceed the threshold potential (V ₊). In the presence of length-dependent feedback, the same event becomes associated with spatial variables: motoneuronal recruitment occurs when the muscle length (x) reaches a specific, threshold length (λ₊).

Fig. 4

Physiological origin of threshold position control. a Basic components of sensory-control integration underlying threshold position control at the level of motoneurons (MN). Each MN receives afferent influences that depend on the muscle length (x) as well as on central control influences that are independent of muscle length. It is recruited when the membrane potential exceeds the electrical threshold (V ₊). b When the muscle innervated by the MN is stretched quasi-statically from an initial length (x _i) the motoneuronal membrane potential increases from its initial value (V _i) following length-dependent afferent feedback from the muscle (solid diagonal line). The electrical threshold (V ₊) is eventually reached at length λ ₊ , at which the motoneuron begins to be recruited. When independent control inputs are added (↑), the same stretch elicits motoneuronal recruitment at a shorter threshold length (λ). c Shifts in the spatial threshold (horizontal arrow) can also result from changes in the electrical threshold (vertical arrow). In both cases (b or c), shifts in the membrane potentials and respective changes in the threshold position are initiated prior to the onset of EMG activity and force generation (a feed-forward process). Thereby, motoneuronal activity and muscle force emerge depending on the difference between the actual (x) and the threshold (λ) muscle length (reproduced from Pilon et al. 2007; see also this reference for further details regarding the dynamic dependency of the threshold position on the stretch velocity and intermuscular interaction)

Now consider the case when a constant facilitatory control input from descending systems is added thus enhancing the membrane potential of the motoneuron (Fig. 4b, vertical arrow). This control input can be delivered either directly (mono-synaptically) to the motoneuron and/or indirectly (poly synaptically), via spinal interneurons or/and γ-motoneurons influencing muscle spindles. When the membrane potential is enhanced, the same muscle stretch elicits motoneuronal recruitment at a shorter threshold length, λ (Fig. 4b, upper diagonal line). The electrical effect of the control input is thus transformed into a spatial variable—a change in the threshold muscle length.

It has been shown that descending influences from the brain stem can shift the electrical threshold of motoneurons (Fedirchuk and Dai 2004; Krawitz et al. 2001).These neuromodulatory influences affect excitability of motoneurons. In the presence of the length-dependent feedback, they might be an additional source of shifts in the spatial threshold (Fig. 4c).

Figure 4 illustrates an important aspect of shifts in the spatial threshold: it is accomplished by changes in the excitability of motoneurons, rather than by changes in their output activity. This implies feedforward nature of threshold shifts: these shifts symbolize substantial modifications in the state of the neuromuscular system before the onset of motor action. The capacity of the nervous system to shift the threshold position ahead of actual motion plays a key role in the solution of the posture-movement problem (Fig. 3). Adaptive feedforward control is probably best-suited for resolving other difficult control problems (Houk 1988; see also below).

Position sense, kinesthetic illusions, phantom limb and other phantoms

It was suggested in the “Introduction” that the brain treats afferent signals in some specific way in order to overcome the ambiguity of positional information from receptors and to form adequate position sense. McCloskey (1981) and Gandevia (1996) proposed that ECs are used with this purpose. This proposal, however, is inconsistent with the data showing that after isotonic movements, not only spindle discharges (Hulliger et al. 1982) but also motor commands to the host muscles return to their pre-movement values (Fig. 2). In this case, ECs in isolation or in combination with constant spindle discharges would mistakenly report to the brain that the limb returned to or remained at the initial position.

If not ECs then what central signals complement proprioceptive feedback to produce position sense? The finding that discharges of agonist muscle spindle in isotonic finger motion are position-independent (Hulliger et al. 1982) implies that γ-efferent signals to these spindles monotonically increase during motion (otherwise, spindle afferent discharges would decrease due to muscle shortening). However, it has been shown that rather than representing EC, these cumulative signals represent a component of threshold position resetting (Matthews 1959; Feldman and Orlovsky 1972). Based on these threshold shifts, the finger can be perceived as moving even though spindle afferent discharges do not signal changes in the finger position.

A more general rule underlying position sense is suggested in Fig. 5. It shows that muscle activation of, say, elbow flexors, starts when the actual angle (Q) exceeds its centrally set referent angle (R). The flexor activation and torque increases with the increasing deviation (P) of Q from R. This activation represents a position-dependent component of postural stabilization. It is assumed that afferent feedback provides signals that are related not to the actual position but to its deviation, P, from the referent position, R, set by the brain. Therefore, adequate position sense, i.e., correct evaluation of the actual angle, Q, is derived at the level of the brain by adding the referent joint angle to positional measure P of proprioceptive signals (Fig. 5):

$$ Q = R + P $$

(1)

Fig. 5

Central and afferent components of position sense. a When muscles compensate an external load, the segment arrives at position Q deviated from the threshold position R by P. Position Q can be sensed adequately by combining the P delivered by proprioceptive signals with central control signals responsible for setting the threshold position, R, resulting in formula Q = R + P. b An isotonic movement is produced by changing the threshold position (ΔR). The resulting change in the joint angle (ΔQ) can be perceived based on the central component (ΔR) of position sense even if its afferent component remains constant (ΔP = 0). c When movement is prevented (isometric condition), changes in the threshold position (ΔR) results in an increase in proprioceptive feedback (by ΔP) and isometric torque. In this case, the central and afferent components of position sense are equal but opposite (ΔP = −ΔR) and, taken together, produce no sensation of motion (ΔQ = 0)

An isotonic elbow flexion is produced by a monotonic decrease in the R that simultaneously appears to be a component of position sense. The joint angle is perceived as changing following this decrease in the R, even though the proprioceptive component, P, of position sense remains unchanged and motor commands to muscles return to their pre-movement levels (Fig. 5b).

Rule 1 is applicable to position sense not only during isotonic movements (when P hardly changes) but also during isometric and other motor actions when both R and P, or only P changes (Feldman and Latash 1982). In particular, in isometric condition, when the joint angle cannot be changed (i.e., Q is constant), a monotonic decrease in the R results in a continuous increase in the difference between Q and R (Fig. 5c) such that the flexor EMG activity, torque, and afferent component of position sense (P) increase. The changes in the two components of position sense are equal but opposite (ΔP = −ΔR; Fig. 5c). Added together, these components cancel each other such that the actual position is perceived as motionless.

Rule 1 helps explain the phantom limb phenomenon. A phantom limb is the sensation that an amputated limb is still attached to the body; the missing limb often feels in a distorted and painful position (Mitchell 1971; Melzack 1990; Ramachandran and Hirstein 1998). In normal situation, when the limb and afferent feedback are intact, the value of R may go beyond the biomechanical limits of joint angles, for example, to relax muscles at the biomechanically maximal position (for details see Feldman et al. 2007). Even in these cases, the above rule results in adequate perception of the joint angle within the biomechanical limits. In the absence of the limb, the proprioceptive component (P) of position sense is missing but the absent limb may still be sensed as present due to the central, referent component (R) of position sense. In absence of proprioceptive feedback, the central component may produce sensation of a distorted limb position (e.g., twisted elbow or knee joint). Such distorted positions can be sensed as painful.

One can also explain the success of mirror therapy in treating phantom pain when the phantom limb is associated with a mirror image of the intact limb (Chan et al. 2007). Due to neural coupling between the limb centers, proprioceptive signals from the latter are used for position sense of both the limbs, bringing the phantom limb position into the biomechanical range and thus ceasing pain.

The same rule helps explain kinesthetic illusions elicited by vibration of muscle tendons in intact organisms. For example, tonic vibration of muscle tendons may change the proprioceptive and/or central component of position sense, such that the limb is sensed as moving although it actually remains motionless (Feldman and Latash 1982). Moreover, like perception of body segments, perception of the whole body (“self”) may be formed based on a combination of the centrally specified referent body configuration and deviation from it defined by sensory inputs. By electrical stimulation of certain brain regions one can artificially change the combination of R + P underlying perception of the whole body and elicit the awkward sense of “split personality”—an illusion of the presence of a person who mirrors motions of the stimulated person (Arzy et al. 2006).

Tonic vibration may distort the perceived size of the stimulated body parts such as in a phantom illusion of an elongated nose (“Pinocchio illusion”) in combination with the illusion of elbow extension in subjects who had their biceps tendon vibrated while the fingers of the vibrated arm touched the tip of their nose (Lackner 1988). These data support the notion that body parts are sensed in relation to other parts of the body or within a “body scheme” (Paillard 1991) such that the sensation of one body part may be distorted by the illusion that an adjacent body part is moving, which is consistent with the notion of referent body configuration.

Based on kinesthetic rule 1, one can derive a testable prediction. Suppose the transmission of signals from axons of α- and γ-motoneurons to elbow muscles is blocked by local injections of a mio-relaxant into muscle nerves. When the muscles are immobilized, the subject may try, in vain, to change the elbow angle. According to rule 1, this resembles the situation when the R but not P changes. This will result in the illusion of a change in the joint angle in the direction of intended movement, which can be tested in appropriate experiments. This illusion may be compared with that of motion of the visual world when the eyes are immobilized and the subject tries to make a saccade (see below).

Sense of effort

An essential feature of sense of effort (some feeling that allows us to say whether objects we deal with are light or heavy) was discovered by Weber (1834/1978). He gradually increased the load that a blindfolded man was holding and asked him to respond when he first felt the increase. Weber found that the minimal changes in the load (ΔF) that the subject felt (force sensation threshold) increased in proportion to the load (F). In other words, the ratio of the force sensation threshold to the force itself remained approximately invariant (Weber’s law or Weber–Fehner’s law after Fechner (1860) who gave its mathematical formulation). Interestingly, kinesthetic rule Q = R + P can be used to derive this law and determine some factors that influence the sense of effort.

It has been shown that the static active muscle force, F (or the external load compensated by this force) exponentially increases with the deviation, P, of the joint angle from its referent value:

$$ F = a\left( {{ \exp }\left( {sP} \right) - 1} \right) $$

(2)

where a and s are constants (Feldman 1966; Gribble et al. 1998). One can rearrange (2) to get

$$ P = s\ln \left( {1 + F/a} \right) $$

(3)

The active force is thus a measure of the deviation (P) of the joint from its referent position. Therefore, force-sensitive afferent signals after logarithmic scaling can be combined with spindle position-sensitive signals as well as with central referent signals to form position sense.

To derive Weber’s law based on Eqs. 1 and 3, consider two cases: (1) R remains constant when loads are added or removed thus eliciting changes in the limb position, as observed in unloading reflexes (Asatryan and Feldman 1965; Forget and Lamarre 1990); (2) isometric conditions, when Q is constant and the R changes and influences the isometric force. Let ΔQ, ΔR, ΔF be the thresholds for discrimination of changes in the respective variables. Since these thresholds are relatively small quantities, one can identify them with the differentials of respective variables. By finding partial derivatives of Eqs. 1 and 2, one can get two expressions for ΔF/F ratio in the two conditions:

$$ \Updelta F/F = \Updelta Q/s\quad {\text{when }}R{\text{ is constant}} $$

(4)

$$ \Updelta F/F = - \Updelta R/s\quad {\text{when }}Q{\text{ is constant}} $$

(5)

These equations imply that sense of effort and position sense are interrelated. Moreover, they also show that if the force sensitivity threshold (ΔF) is scaled with the weight (i.e., if ΔF/F is constant according to Weber–Fechner’s law), then the sensitivity threshold to changes in the limb position, either actual (ΔQ) or referent (ΔR), remains the same regardless of weight or muscle force. This implication can be tested in future studies.

The suggestion that position sense and sense of effort are interrelated may help explain the size-weight illusion when the smaller of two objects is perceived as heavier even if the two objects have the same weight (Müller and Schumann 1889). To show this, one can rearrange Eqs. 4 and 5 to see that sense of effort is determined by the ratio of the force sensitivity threshold to the positional or referent threshold sensitivity, respectively. For example, as follows from Eq. 4,

$$ F = - s\Updelta F/\Updelta R\quad {\text{if }}Q{\text{ is constant}} $$

(6)

The sensitivity threshold (ΔR) can likely vary somewhat depending on the previous neuromuscular activity, fatigue, or inter-modal sensory interactions. For example, when seeing two objects of different sizes, subjects may tend to scale the perceptual sensitivity of all spatial-dimensional variables, including ΔR, in proportion to the size of the object. Then the absolute value of ΔR will be smaller for the small object and, as follows from Eq. 6, the smaller object will be perceived as heavier even if the two objects have the same weight.

Further implications of threshold position control: physical or action-producing frames of reference

Figure 4 showed that the threshold position control is a system property resulting from the fact that motoneurons have a membrane with an electrical threshold, gradual sensory inputs, and independent central inputs. However, this type of integration is not unique for motoneurons: many if not all neurons have a similar integrative capacity, although the type of sensory and central inputs they integrate may broadly vary. Therefore, the membrane of any such neuron is the site where electro-chemical central inputs acquire the dimensionality of the sensory input. Decoded in this way, the control signal defines the threshold (referent) value that the sensory signals should exceed to activate the neuron and execute the action characteristic of this neuron. Depending on the type of sensory inputs, neurons can stipulate different forms of threshold (referent) position of the body or its segments some of which has been named in the previous section (see also Feldman et al. 2007; Pilon et al. 2007). This generalization of the threshold position control has led to an efficient solution of the redundancy problem in guiding multiple muscles and body segments (Feldman et al. 2007).

Threshold position control is related to the notion of spatial frames of reference or system of coordinates in which the nervous system operates. By definition, each spatial frame is a geometric structure (Cartesian, spherical, polar, etc.) comprised of a set of points each of which represents the location of a symbolic or physical element in relation to a specific element represented by the origin point of the frame. Some rule (the metrics) defines how far each point in a frame is located from the origin and other points in this frame. The origin and direction of the axes of a frame characterizes its location and orientation in another frame.

The threshold angle, R, of a joint can be considered as the origin of a spatial frame of reference, Q, comprised of all biomechanically possible angles of this joint. Another form of threshold position, the referent body configuration can be considered as the origin of a spatial frame of reference comprised of all-possible body configurations represented by points in this frame. Yet another frame of reference represents all-possible positions of the whole body in the environment. By specifying a threshold position of the body in the environment, the nervous system ranks other body postures in terms of metrically defined distance and direction separating it from the threshold posture (Feldman et al. 2007). A body of literature provides strong evidence that spatial tasks are solved utilizing combinations of different frames as summarized in 24 chapters of the book Brain and Space assembled by Paillard (1991). Threshold position control exposes additional properties of neural frames of reference that are worth emphasizing.

First, the integration of afferent feedback with independent central inputs (Fig. 4) implies that any motoneuron and probably most of neurons function in appropriate frames of reference the nature of which is defined by the sensory inputs to these neural elements.

Second, to describe motor actions on paper or in computers, we use frames of reference or systems of coordinates that can be called abstract, symbolic or mathematical in the sense that any change in the selected frame (e.g., shift in the origin or rotation) only influences our description of system behavior without any consequences for the actual system behavior. In contrast, neuromuscular elements in any selected frame of reference generate activity depending on the difference between its current state and the threshold state defined by the frame origin. Therefore, by shifting the origin of the chosen frame, the nervous system modifies the ongoing or initiates a new action. Frames of reference used by the nervous system are thus not only action-specific (Colby 1998) but physical or action-producing, rather than symbolic or mathematical. A first step to the understanding that neural frames of reference are physical was likely made by Paillard (1991) who hypothesized that the metrics of the neural frame dealing with body configurations (“body schemes”) reflects a “path structure” that relate different body postures and transitions between them. In the present context, the metrics of a physical frame is related to the emergent activity and forces leading to involuntary posture-to-posture transitions resulting from changes in external forces whereas intentional actions result from global shifts in the frame following a change in its origin determined by the respective threshold position. The origin is likely one of several attributes (parameters) of frames of reference. Changes in the metrics of frames and their orientation in other frames is an additional way of action control. In particular, somersaults may be initiated and guided by an appropriate rotation of the frame comprised of body configurations in a frame of reference associated with the environment.

Third, our analysis showed that shifts in the origin of a frame may influence both action and perception (see Eq. 1). This implies that although action and perception may involve different neural mechanisms and brain structures (Goodale and Westwood 2004), they are constrained by functioning in common, action–perception frames. This suggestion might be fundamental for the understanding of how the brain works.

The notion that neural referent frames are physical implies that they can be considered as intrinsic neural tools from which the organism may choose to execute a desired action. Like tools used in everyday life (a screwdriver, a saw, a violin, its bow, or a computer), neural tools are selected to meet the task demand(s) but they are not entirely task-specific (like mechanical tools: a screwdriver can be used not only to drive a screw but also to make a hole in a soft material or as a weapon). Rather, each neural tool is associated with a group of possible actions. For example, the system may change the threshold joint angle to establish, depending on external condition, a desired isometric torque, position or both. This example also shows that different forms of the threshold position should not be considered as goals of action. This is especially obvious in isometric conditions when changes in the R give rise to an increase in the muscle torque while the limb position remains unchanged (Fig. 5c). Therefore, the threshold position may not represent the desired goal of action.

Efference copy-based and internal model theories consider a problem of a mapping between desired movements and associated motor commands. It is assumed that this problem is solved by pre-programming of the requisite commands with the help of inverse and forward internal models. In contrast, by utilizing frames of reference as action-producing tools, the system does not need to program these commands. Nor it needs to compute or pre-program changes in the origin (threshold position) or/and orientation of these frames: the system may directly shift these attributes of the frames to get the desired result. This strategy is reminiscent of driving a car: the steering wheel is a tool that the driver rotates to move the car in the desired direction without any concern or pre-programming of the position of the steering wheel associated with the driving direction. Similarly, to reach a certain point in space, the system may simply shift the referent position of the hand such that its actual position would shift toward the desired point (Feldman et al. 2007).

Visual space constancy

Visual space constancy is the sensation that the world remains motionless despite shifts in its retinal image following saccadic or other eye motion. The dominant theory suggests that at some level of visual processing, retinal shifts are compensated by a copy of motor command to eye muscles in order to perceive that the world remains stable (Helmholtz 1866/1963; Sperry 1950; Von Holst and Mittelstaedt 1950/1973).

Bridgeman (2007) identified several limitations of the EC theory applied to visual space constancy. In particular, he pointed out that, because of its predictive nature, EC cannot be exact, which conflicts with the typical perception that the external world is stable after re-fixation of gaze. Nevertheless, space constancy could be maintained despite small mismatches of EC and retinal displacement if the imprecision of EC was less than the perception of displacement threshold during saccades (Matin 1974). Bridgeman et al. (1975) found that the perceptual threshold is not less than one-third of the saccade itself. Bridgeman (2007) concluded that any visual mechanism that tolerates such an error could not support perceptual space constancy.

The compensation theory can be questioned at a more fundamental level: it is doubtful that eye movements are produced by direct specification of motor commands and ECs. Like limb positions, eye positions are stabilized for extended periods. A saccadic eye movement would evoke substantial muscle resistance to the rapid deviation from the previously stabilized position, unless the stabilizing mechanisms are reset toward the position to which the eyes move (see “posture-movement problem”). Rather than by specifying motor commands, this can be done independently of them, by changing some system parameter that pre-determines the eye position at which all eye muscle forces become balanced (see “Natural laws constrain the ways the brain can control motor actions”).

McConkie and Currie (1996), Currie et al. (2000) and Bridgeman (2007) advanced a new theory of visual space constancy, not relying on the EC concept. In the Bridgeman’s formulation this theory is described in the following way: “According to the visual search interpretation, attention shifts to the saccade target before the saccade is executed (Deubel et al. 2004). Due to the attention shift, location and visual attributes of the target and of surrounding objects are stored in trans-saccadic memory. After the saccade, the visual system searches for the previous object of attention within a restricted spatiotemporal ‘constancy window’ which is about 50 ms in duration, and is confined to a few degrees around the saccade target. If the object is found, the world is assumed to be stable. Spatial information from the previous fixation is discarded or ignored, and localization proceeds using the currently available information”.

This theory uses essential experimental findings in the explanation of visual space constancy but leaves unanswered some questions. The saccade target may appear within the critical window not only when the remaining world is stable but also when it is moving (broken space constancy). How does this theory distinguish between these cases? Also, the saccade target may appear outside the critical window if the target is either moving within the motionless world or if the target and world are moving together. It seems that the visual search theory is successful in identifying whether or not the target is moving but it has difficulties in identifying whether or not the remaining visual world remains stable.

Interestingly, the problem of visual space constancy can be reformulated and solved without intermingling it with the properties of saccades. Before doing this, note that Milner and Goodale (1988) distinguished between vision for action and vision for perception and that perception is for identifying objects instead of for locating them. A more detailed discussion of this essential point goes beyond of the scope of this paper. In the present section, the word perception is used in reference to both object identification and localization.

Consider retinal projections of the visual world for different directions of gaze (Fig. 6a). The same object may project to different retinal spots. Also, for different gaze directions, different objects can be projected to the same retinal spot. These are signs of ambiguity in the representation of objects on the retina, implying that information on retinal projections of objects is not sufficient to determine where in the physical world these objects are localized. To overcome this ambiguity, it is necessary to take into account that each retinal projection depends on the eye position defined by the direction of the eye optical axis. The latter is the line that goes through the center of the lens (eye optical center or EOC), the eye center of rotation (ECR) and the fovea center (Fig. 6c). In skeleto-muscular system, we distinguished between the actual rotation of body segments and their referent rotation controlled by the brain. Similarly, for the visual system, one can distinguish between the actual position (Q) of the eye and its centrally specified, referent position (R). These positions can be defined as the deviations of the eye from its neutral position, when gaze is directed straight ahead (Fig. 6c). The suggestion that the R is a control variable implies that the oculomotor system is driven by the tendency to nullify the discrepancy between the actual and the referent eye position. Given the referent position, the retinal projections of objects are defined by the known rules of geometric optics (Fig. 6a). Without going into details, one can denote such optic projections as determined by a matrix, O(R), components of which are dependent on R. A physical object in the world (p _w) is transformed into a retinal image (i _r):

$$ O(R) \cdot p_{\text{w}} = i_{\text{r}} $$

(7)

Fig. 6

A solution to the problem of visual space constancy (a schematic diagram). a Retinal projections (dashed lines) of two objects (a, b) from the external world for two eye positions (blue and red). These optico-geometrical projections can be described by a matrix, O(R) that depends on the referent eye position, R, shown in c. b It is assumed that a neural network transforms retinal images such that the person associates these transformed images with the objects of the external world, rather that with representations of these objects on the retina or inside the brain. Such inverse neural transformation is described by a matrix N(R) that mathematically resembles the inverse of the optico-geometrical transformation O(R). Symbolically, the N(R) is shown by dashed lines directed from the retinal images of objects a and b via the eye rotation center (ERC) and its optical center (EOC) back (ascending arrows) to the physical world. Taken together, O(R) and N(R) produce an identity transformation. Therefore, any object or a part of the visual scene that remained motionless during re-fixation will be recognized by the visual system as such, whether or not their retinal images are shifted. The identity transformation will be broken for any object or a part of visual scene that moved in the physical worlds during re-fixation and, based on this criterion, these motions or no-motions will be captured by the system

Apparently, while processing visual signals, the brain focuses on determining the localization and motion of objects in the external, physical world from which these signals arrived, rather than on the localization of the images of these objects on the retina. Mathematically, the question of how the brain can localize objects in external space is not as difficult as it seems. In order to do this, one can invert all straight lines (“light rays”) that produced a given retinal pattern O(R) and thus to determine geometrically where all objects that produced a given retinal pattern are localized in the external world. Figuratively, this procedure reverses the optical flow coming to the eye back to the external world via the eye lens (Fig. 6b). This transformation can be described by a matrix, N(R). The major property of this matrix is that it converts all retinal images back to their sources in the physical world. Therefore, applied sequentially, transformations G(R) and N(R) produce an identity matrix, E (a matrix with unit values on the diagonal and zero values outside of it):

$$ O(R) \cdot N(R) = E $$

(8)

In Figs. 6, 7, and 8, all transformations described by matrix O(R) are shown by dashed lines with arrows pointing down and those resembling N(R) are shown by dashed lines with arrows pointing upward.

Fig. 7

Visual space constancy for eye re-fixation elicited by saccade. a Alleged central pattern of shift in the referent eye position (R) underlying saccade. Like in the skeleto-motor system (Fig. 3), the EMG activity of eye muscles is not programmed but emerges depending on the magnitude and rate of change in the difference between the actual (Q) and the referent eye position; α is the final value of R and Q that bring the projection of the saccade target to the fovea; dashed curve is the eye velocity. b, c Re-fixation from object a to object b. Gist is a general (“abstract”) structure of the physical world perceived by the visual system and retained in trans-saccadic memory. The retinal images of objects and gist are shifted following re-fixation, but due to the inverse neural transformation symbolically shown in c by dashed lines with ascending arrows (see also Fig. 6), the system perceives the objects and gist as remaining stable in the physical world if they actually remained motionless. The identity transformation resulting from the optico-geometrical, O(R), and inverse neural transformation, N(R), will be broken for any object or a part of visual scene that moved in the physical world during re-fixation. Based on this criterion, these motions will be captured by the system while the remaining part of the visual scene will be perceived as stable

Fig. 8

Illusion of motion of the external world when subjects try to make a saccade with the immobilized eye. a Initially, object a is projected to the fovea. b The referent position of the eye is shifted when the subject tries to make a saccade to object b. Although the eye and retinal images of the objects and gist remain motionless, the inverse neural transformation (see Figs. 6, 7) projects them into the physical world (dashed lines with ascending arrows) according to the new referent position of the eye (red). As a result, the whole world is perceived as shifted in the direction of saccade

It is assumed that a transformation mathematically equivalent to the N(R) is made by a neuronal network—the inverse neural transformation (INT). In other words, given the appropriate frame of reference, R, the N(R) transforms all retinal images into their sources in the external world. In particular, if i _r is a retinal image of an object then the image i _w of the same object projected to the external world is defined as

$$ N(R) \cdot i_{\text{r}} = i_{\text{w}} $$

(9)

By comparing the sequential positions of the i _w, the visual system can decide whether or not the actual object is motionless.

These notions can be used to explain visual space constancy. It is known that visual memory retains only general features or an overall structure (gist) of the total visual scene in combination with a small amount of objects of attention (Sampanes et al. 2008). If one or several objects are moving in motionless physical world, the visual system will recognize them as moving in motionless gist since the inverse transformation (9) will reveal that only these objects are moving. The INT (9) also helps recognize when objects are motionless whereas the world (gist) or its part or the whole visual world together with objects of attention are moving, reflecting the respective broken and preserved components of inverse transformation. Note that the notion that visual processing is accomplished in an appropriate spatial frame of reference with help of INT not only resolves the problem of visual space constancy but also makes a step in answering the classical question of why retinal images of objects are sensed as real objects located in the external, physical world, rather than in internal representations of the brain.

Traditionally, the problem of visual constancy was mostly considered in relation to saccades but the above explanation of visual space constancy remains valid whether or not the gaze is fixed or re-fixed following slow eye motion or saccades. One can assume that, like perception of visual world, saccadic motion is accomplished by shifting the referent eye position, R. This assumption can be justified by the necessity to resolve the problem that is similar to the posture-movement problem for limb skeleto-muscular system (see above). The notion that action and perception are accomplished in a common frame of reference is thus applicable to the visual system.

In normal conditions, saccade is presumably results from a rapid monotonic shift in the referent position, R, of the eye until its final value (α) is reached. Such a shift presumably results from propagation of excitation in a neural ensemble (e.g. Munoz et al. 1991). Eventually bringing the projection of the saccade target to the fovea or more precisely, to the critical window defined in the Bridgeman’s theory (Fig. 7a). The activity of eye muscles emerges depending on the difference between the actual eye position, Q, and its referent position, R, as well as on the rate of change in this difference. Eye rotation continues until Q matches R and the saccade target (if it did not move during re-fixation) becomes projected to the fovea (Fig. 7b, c). Vision is suppressed during saccade (Matin 1974). In the present theory, this makes sense: the system should wait until the actual and referent eye positions match each other to avoid perceptual confusion. Like in non-saccadic eye movements, the system can recognizes different behavioral situations occurring in the physical world during re-fixation: the world or its gist was stable together with the objects of attention; one or several objects were moved whereas the gist remained stable; the gist or its part was moved in combination or in isolation with the objects of attention. These situations are recognized depending on whether or not the respective components of the inverse transformation (9) were shifted after saccade.

By using the INT, the system does not need to compensate for the retinal shifts of visual images. Instead, it transforms them into images that can be directly associated with the actual objects in the external, physical world. Thereby, the system can produce saccades to better focus on the selected target without any concern that the retinal images of the world are shifted following re-fixation. This interpretation of the system behavior resembles that of Gibson who challenged the EC theory of visual constancy by arguing that no compensation of shifts of retinal images is necessary if saccades are considered as sequential fixations in scanning the visual scene (Gibson 1968, p. 39 and 252).

The identity transformation (8) presumably characterizing the visual system can be artificially broken to elicit illusions of world displacement. This can be achieved, for example, by changing the natural optical properties of the oculomotor plant represented by matrix O(R). This can be done by pressing on the eye with a finger. This method actually elicits an illusion of the world shift (Bridgeman 2007). Another way to elicit a similar illusion is to artificially change the neural part, N(R) of the INT, e.g., by immobilizing the eye. Consider this illusion in detail.

Experimentally, attempts to produce a saccade when the eyes are immobilized elicit sense of motion of the visual scene in the intended direction of the saccade (Helmholtz 1866/1963; Bridgeman 2007). One can assume that an attempt to make a saccade with the immobilized eyes also involves central shifts in the referent eye position, except that they do not result in eye motion. In contrast, the motionless retinal images of the world are sensed in relation to the new referent position, R, of the eye. At this position, the referent fovea and the lens appear as rotated by angle R whereas the actual fovea, the lens and the retinal images of objects remain where they were before the saccade (Fig. 8a, b). If the intended eye rotation is counterclockwise, then the post-saccadic image of the object initially projecting to the actual fovea will appears to the left of the referent fovea. Following the INT, this object will be perceived as shifted in the direction of the intended saccade. This will occur with the perception of other objects and gist (Fig. 8b), as it actually occurs in experiments with immobilized eyes (Bridgeman 2007).

Consider possible objections to the theory presented here. It suggests that shifts in the referent eye position underlie both oculomotor action and visual perception. This control strategy critically relies on the existence of position-dependent proprioceptive feedback to oculomotor motoneurons (Fig. 4a). In the skeleto-muscular system, muscle spindles play a major role in such feedback. These or other position-sensitive receptors may play a similar role in other motor systems. In particular, in squirrel monkeys, position-related feedback is presumably delivered by receptors called palisades localized in the extrinsic eye muscles; Dancause et al. 2007). Such properties as delay, the number of synapses involved in the transmission of this feedback to motoneurons or its strength play a secondary role in the mechanism of shifting the referent eye position. The existence of position-dependent feedback to motoneurons of extrinsic eye muscles was questioned in the past (e.g., Robinson 1970), but recent findings point to its existence (Dancause et al. 2007). Resetting of posture-stabilizing mechanisms, control of eye movements, as well as visual space constancy can thus rely on central shifts in the threshold (referent) eye position rather than on pre-programming of EMG activity and EC-based compensation of shifts of visual images during saccades.

Another major assumption of the theory is that the optical transformation of real objects into retinal images is inverted by a specialized neural network (INT) such that the person associates these transformed images with the objects of the external world, rather that with representations of these objects on the retina or inside the brain. This part of the theory needs experimental verification. The place where the INT may occur is uncertain but the first candidate for this is the retina. It is known that the retina receives central, efferent innervation (Honrubia and Elliott 1970). This centrifugal innervation presumably reflects perceptual shifts in the referent eye position that are required for correct functioning of the INT. A testable consequence of this hypothesis is that, blocking this innervation by appropriate anesthetic, on can complicate visual perception. Indeed, a priori, one cannot rule out that the INT is produced by the visual cortex or/and other brain levels.

There is considerable evidence showing that a mismatch of the entire retinal image before and after a saccade is not detected—only a mismatch of features near saccade target (McConkie and Currie 1996; Currie et al. 2000; see also Deubel et al. 2004; Bridgeman 2007). On the other hand, there are data showing that a general (“abstract”) structure of the entire retinal image (“gist”) in combination with a small amount of objects of attention is retained in trans-saccadic memory (Sampanes et al. 2008). Further experiments are needed to decide whether visual space constancy relies on the entire gist or only on its part that represents surrounding features of the target. The theory of visual space constancy presented here can accommodate either of these possibilities and only empirically one can decide which of them is valid.

Challenging a major doctrine in approaches to action and perception

Threshold position resetting results from changes in the membrane potentials of motoneurons (Fig. 4) and therefore it represents a forward process that starts before any changes in variables characterizing the motor output, i.e., kinematic and kinetic variables and variables directly related to them, such as stiffness and motor commands (EMG signals). By setting threshold positions, the brain specifies where, in spatial coordinates, neuromuscular elements can work but it does not instruct them how they should work: motor commands emerge or not following interactions of these elements between themselves and with the environment in the centrally specified spatial boundaries.

The threshold control theory does not conflict with data showing that the activity at different brain levels, including the primary motor cortex, correlates with output kinematic and kinetic variables such as forces, movement direction, or combinations of EMG activity of multiple muscles (e.g., Holdefer and Miller 2002). These correlations can be considered as emergent properties of threshold position control. For example, let us assume that some neurons of the motor cortex produce shifts in the activation thresholds of appropriate arm muscles, which has actually been shown in decerebrated cats (Feldman and Orlovsky 1972). In isotonic conditions, these shifts will cause a change in the arm position whereas, in isometric conditions, the same shifts will result in an increase in isometric torques. Thus, although the neural activity responsible for shifts in the thresholds may remain independent of the output variables, it may correlate with them, and the type of correlation depends on external conditions. Indeed, these examples just confirm the known statistical rule that correlations between variables do not necessarily imply causality between them.

Failures of EC-based theories and internal model theories in general are rooted in the conventional view that the brain computes (“programs”) and specifies motor commands with an assisting role of reflexes and ECs. To justify this view, researchers usually refer to the observed correlations between the activity of different descending systems and EMG patterns. As explained above, correlations are neutral to any theory: they do not imply causality postulated by computational or programming theories, nor they conflict with alternative theories such as the threshold position control theory.

The idea of programming of motor commands based on ECs is often justified by the findings that movements can be produced after proprioceptive deafferentation and that patterns of motor commands associated with these movements approximately resemble those in intact organisms (e.g., Gallistel 1980). No doubt, deafferentation experiments were essential in demonstrating that the nervous system plays much more active role in action and perception than previously thought. They also revealed the existence of well-organized neural structures of actions (central pattern generators or CPGs) and uncovered substantial plasticity of the nervous system in adapting to the loss of a sensory function (e.g., Pearson et al. 2003).

In contrast, deafferentation experiments were not particularly helpful in the understanding of how motor actions are controlled in intact organisms. Moreover, results of such experiments were misinterpreted, which is especially clear from a vantage point of the threshold position control theory that implies that, in intact organisms, facilitations and inhibitions issued by CPGs or descending systems only establish spatial boundaries within which motoneurons can generate motor commands to muscles. In other words, in intact organisms, activation of motoneurons by CPGs and descending systems is always conditional. This modus of action is only possible in the presence of positional information delivered by proprioceptive afferents (Fig. 4). By depriving the system of positional information, the conditional activation of motoneurons by CPGs is artificially converted into unconditional, especially if deafferentation is combined with injections of drugs enhancing overall excitability of motoneurons and the entire nervous system (for review see Shik and Orlovsky 1976).

The threshold control theory does not conflict with the observation that the EMG patterns after deafferentation may qualitatively resemble those before deafferentation and the resemblance might be improved to a certain degree after long-lasting training, adaptation and neural plasticity. The threshold control theory suggests, however, that deafferentation may permanently damage spatial aspects and stability of movements and that these deficits can only partly be compensated by vision. For example, deafferented cats generate scratching hind limb movements, but instead of the ear, the limb often robustly “scratches” the air near it (Deliagnina et al. 1975), making the movements non-functional. After long-lasting practice, deafferented patients can make rapid arm movements by producing a try-phasic EMG pattern (Forget and Lamarre 1990) but, unlike healthy subjects (Fig. 2), they cannot reduce agonist–antagonist coactivation after the end of movement without driving the arm away from the targeted position, not to mention that they cannot maintain the same arm position in the absence of vision. Moreover, deafferented patients cannot stand or walk without substantial assistance, even with vision (Levin et al. 1995). A web site created by Jack Paillard (http://deafferented.apinc.org) list numerous studies showing that conversions of CPGs into unconditional generators of motor commands by deafferentation have a heavy price: they destroy the fine spatio-temporal organization characteristics of action–perception coupling in intact organisms.

Conclusions

The conventional notion of programming of motor commands with assistance of sensory feedback and EC computed with the help of internal models is unable to solve several classical problems of action and perception in a physiologically feasible way. One is reminded of a known statement of Einstein that problems cannot be solved by thinking about them within the framework in which they were created. The classical problems considered in this paper were created in a predominantly mechanistic framework complemented by ideas of computational programming based on internal models. Formulated outside this framework is the notion that the brain uses common spatial frames of reference to control action and perception. These action–perception frames are selected and shifted by the brain in feedforward and task-specific ways. This approach leads to a solution of several problems and offers a physiologically feasible and testable alternative to conventional views on action–perception coupling.