Error generalization as a function of velocity and duration: human reaching movements

Abstract

Our sensory-motor control system has a remarkable ability to adapt to novel dynamics during reaching movements and generalizes this adaptation to movements made in different directions, positions and even speeds. The degree and pattern of this generalization are of great importance in deducing the underlying mechanisms that govern our motor control. In this report we expand our knowledge on the generalization between movements made at different speeds. We wished to determine the pattern of generalization between different speed and duration movements on a trial-by-trial basis. In addition, we tested three hypotheses for the pattern of generalization. The first hypothesis was that the generalization was maximum for the speed of the movement just made with a linear decrease in generalization as one moves away from that preferred speed. The second was that the generalization is always highest for the fastest speed movements and linearly decreases with speed. The last hypothesis came from our preliminary results, which suggested that the generalization plateaus. Human subjects made targeted reaching movements at four different maximum speeds (15, 35, 55 and 75 cm/s) presented in pseudorandom order to one spatial target (15 cm extent) while holding onto a robotic manipulandum that produced a viscous curl field. Catch trials (trial where the curl field was unexpectedly removed) were used to probe the generalization between the four speed/durations on a movement-by-movement basis. We found that the pattern of generalization was linear between the first three speed categories (15–55 cm/s), but plateaued after the 55 cm/s category. We compared the subjects’ results with a simulated adaptive controller that used a population code by combining the output of basis elements. These basis elements encoded limb velocity and associated this with a force expectation at that velocity. We found that using a basis set of Gaussians the adaptive controller produced movements that generalized in virtually the exact manner as the subjects, as we have previously demonstrated for movements made to different spatial targets. Thus, the human internal model may employ such a population code.

Introduction

While making reaching movements we must plan the direction (Georgopoulos et al. 1982, 1988), distance (Fu et al. 1993; Churchland et al. 2006), speed (Moran and Schwartz 1999), and magnitude of forces (Evarts 1968) necessary to achieve a goal. It has been suggested that internal models (Lackner and Dizio 1994; Shadmehr and Mussa-Ivaldi 1994) transform desired movements, or goals into motor patterns that accomplish the goal by reaching the target position while perhaps minimizing some kinematic (Flash and Hogan 1985), dynamic (Uno et al. 1989; Nakano et al. 1999) or variance constraints (Harris and Wolpert 1998).

We hypothesize that the internal model forms a mapping between desired movements (kinematics) and the muscle forces necessary for producing these movements by combining the output of motor primitives. These primitives form a basis set that is summed to produce a given force at the hand at a particular point in state space, such as at a particular velocity of the hand. Previous work has demonstrated that this view holds for movements made to different target positions (Thoroughman and Shadmehr 2000; Donchin et al. 2003). In the work presented here we wished to determine if this view of the internal model holds true when subjects make movements with different maximum velocities and durations. Important to this work is the idea that the shape of the basis elements comprising the internal model will determine how it generalizes error made in one portion of state space to others, such as from one speed movement to other speed movements.

One of our goals in the present work was to determine if the simple basis element model previously reported (Thoroughman and Shadmehr 2000) could also account for the generalization pattern seen in human movements made at different velocities, expanding the movement state space that such a model could account for. Thus, we have chosen to use the curl field paradigm that was previously used (Thoroughman and Shadmehr 2000; Donchin et al. 2003). Subjects made visually guided reaching movements to a single target at four randomly interleaved speeds while holding onto a robotic manipulandum that produces a velocity dependant curl field, which on occasion is secretly turned-off, called a catch trial. This paradigm allowed us to track changes in the subject’s performance on a trial-by-trial basis. Changes in this trial-by-trial performance should reflect changes in the internal model’s mapping between kinematics and dynamics.

Below we outline the theory used in this work, which is similar to that we have previously presented for movements made in different directions (Donchin et al. 2003). We first introduce a simulated adaptive controller that learned to make reaching movements at different speeds against a viscous curl field. We then introduce a simple time series model that can be fit to both the simulations data as well as the human subjects’ data, allowing us to compare the generalization function between these two systems, where the generalization function determines the pattern of generalization for error made during reaches at one speed to all other speed movements.

Theory

We start by presenting a simulation that encompasses the dynamics of the experiment performed by the subjects that is a simulation of a humanlike arm holding a robotic manipulandum. The task was to make 15 cm reaching movements to one spatial target at one of four pseudorandomly chosen maximum speeds. The robotic manipulandum produced a velocity dependant curl force field. Thus, it produced forces tangential, and proportional to the velocity of the subjects hand. The subjects learned to compensate for this curl field. Likewise the simulation will comprise an adaptive controller allowing it to adapt to the force field. The form of the internal model, or adaptive controller, of the simulation we will consider is ${\hat{F}}(\dot{x}) = Wg(\dot{x}),$ where ${\hat{F}}(\dot{x})$ is the internal model’s vector approximation of forces acting on the hand at velocity $\dot{x}.$ This force is generated by summing over a vector of scalar primitives $g(\dot{x}) = {\left[ {g_{1} (\dot{x}),\ldots,g_{m} (\dot{x})} \right]}^{T}$ that encode limb velocity $\dot{x},$ where each primitive is associated with a force vector W _i . We chose a basis set comprised of Gaussians and tessellated these such that they covered the possible velocities visited by the subject’s hand. One can think of these primitives as neurons with receptive fields that are Gaussian in velocity space. The output of the internal model is the sum of this population of basis elements, or a population code (Pouget and Snyder 2000) as follows:

$$ \begin{aligned} {\hat{F}}(\dot{x}) &= Wg(\dot{x}) \\ g(\dot{x}) &={\left[ {g_{1} (\dot{x}),\ldots,g_{m} (\dot{x})} \right]}^{T}\\ W&={\left[{\begin{array}{lll} w_{{x1}}&\ldots&w_{{xm}}\\ w_{{y1}}&\ldots&w_{{ym}}\end{array}} \right]} \end{aligned}. $$

(1)

If the internal model is in error and does not compensate for the task dynamics it must be able to update it self, that is learn. Our simulation’s adaptive controller has the ability to change the force vectors associated with each basis element and did so using gradient decent. Specifically, the simulation produced movements in discrete time (sampling rate of 10 ms). We assumed that the desired hand trajectory be a straight line from the start position to the end and these movements have bell shaped velocity profiles (Flash and Hogan 1985). At each time point in a movement we could determine an error, which was the differences between the forces expected and the actual forces encountered. The updating of the force vectors took on the form: $w^{{n + 1}}_{{ij}} = w^{{(n)}}_{{ij}} - \eta \frac{{\partial (F - {\hat{F}})}}{{\partial w_{{ij}}}},$ where F were the forces encountered and again ${\hat{F}}$ the forces expected by the internal model. This updating occurred at the end of each movement, and thus our simulation had the ability to change its performance from trial-to-trial. The overall learning rate of this model could be controlled by the parameter η.

We now describe a time series model that will allow us to compare the simulation’s patter of generalization between speeds to the human subjects’ pattern of generalization. For a given movement (n) made with a target maximum speed (ts) the subject or simulation could make an error Y ⁽ⁿ⁾. This error is simply the perpendicular distance between a straight line connecting the start position to the target and the position of the nth reaching movement taken at the maximum speed point during that movement. We can relate this error or perpendicular displacement with an error in the internal model’s force expectation through a compliance term that determines how much a given force would move, or displace the hand such that:

$Y^{{(n)}} = D_{{ts(n)}} {\left({F^{{(n)}} - {\hat{F}}^{{(n)}}_{{ts(n)}}} \right)}.$ The compliance as well as the expected force depends on the target maximum speed (ts) of the nth movement, as the force field was linearly proportional to the velocity. The error experienced will have an influence on the future force expectation of the internal model for all speeds. The main goal of the work presented here was to determine just how much an error experienced at one speed influences the internal model’s force expectation at all four speeds that we used in our experiments. As we could not easily measure the expected force of the internal model from the subjects we will represent it in terms of a displacement via a change in variables such that: $Z^{{(n)}}_{{ts(n)}} \equiv D_{{ts(n)}} {\hat{F}}^{{(n)}}_{{ts(n)}}.$ As we are interested in how error made during one movement generalizes to others we need to give our time series model the ability to update its internal expectation of force, that is update the Z. We assume that this occurs such that a portion of the error made on the last trial at speed (ts) is used to update the internal force expectation for all possible movement speeds (l), leading to:

Z ⁽ⁿ⁺¹⁾ _l  = Z ⁽ⁿ⁾ _l + B _l,ts(n) × Y ⁽ⁿ⁾ l = 1,2,3,4. Here the generalization of the error made on movement n is dictated by B _l,ts(n). We can now fit this time series model to the sequence of perpendicular displacements made by the simulation as well as the subjects and compare the generalization functions (B) between the two.

Previous work on generalization between speeds using velocity dependant force fields (Goodbody and Wolpert 1998) has suggested that the internal model generalizes linearly between different amplitude and duration movements, however this previous work was limited to only two different velocities, or amplitudes. Here we establish that a simple computational model of an adaptive controller, comprised of a set of Gaussian basis functions (Pouget and Snyder 2000) tessellated in velocity space generalizes in a manner that is statistically indistinguishable from the subjects. The pattern of generalization is close to linear between the slower speeds, but apparently reaches a maximum at approximately 55 cm/s where it plateaus.

Methods

Thirteen healthy adult subjects (age: 20–32 years) consented to the experiment (all experimental protocols were deemed acceptable by the IRB at Johns Hopkins University where the experiments were conducted). Participants held the handle of a robotic manipulandum (Shadmehr and Mussa-Ivaldi 1994) while performing a reaching task to a single visual target 15 cm directly in front of them, approximately at their midline and in the horizontal plane. The subject’s arm was supported by a sling. Movement of the manipulandum caused movement of a cursor on a computer monitor mounted vertically in front of the subject. Before each movement a cursor on the computer monitor traced a minimum jerk movement profile in a straight line from the start position to the end target with a maximum speed of 15, 35, 55 or 75 cm/s. These four speed categories were presented in a randomly interleaved manner. Subjects were instructed to make a movement similar to the one they had just seen. If the subject attempted to track the example movement, they were warned with a tone and the movement was disregarded. After each movement, text was presented on the monitor indicating the maximum speed they were instructed to go and their actual maximum speed. The accuracy of the duration of their movement was specified as too short (red) or too long (green) via a change in the target color. If the movement was within 5 cm/s of the goal speed and within 200 ms of the goal duration (approximately 550, 650, 950 and 1,800 ms) an animation of the target exploding indicated a successful movement. The durations used were chosen after preliminary testing such that the subjects would be able to complete the task successfully. For example a true minimum jerk trajectory with a maximum speed of 75 cm/s should take 375 ms, but this proved to be too difficult for the subjects, especially with the force field and catch trials, even our simulation needed to be given more time to complete the movements using durations of (460, 600, 950 and 1,900 ms). Upon termination of a movement and presentation of visual feedback, the subject’s arm was brought back to the center actively by the manipulandum; subjects naturally assisted their return to the starting position. A computer recorded the position and velocity of the handle at 100 Hz.

Subjects first performed one to three target sets, which consisted of 192 movements (48 pseudorandomly interleaved at each of the four speeds), with the manipulandum motors turned off (null sets). If the subject felt comfortable with their ability to make the movements consistently at the appropriate speed they moved on to the next phase of the experiment, otherwise they were allowed to take a couple minute break before starting a second, or third null set. During the fielded sets, the manipulandum produced a viscous curl field, applying forces that were linearly proportional to the instantaneous velocity of the handle and perpendicular to its velocity (Francis 2005). Subjects performed the null sets and the first fielded set on day 1 of the experiment and completed four additional fielded sets on day 2. Table 1 describes the target set structure indicating the number of target sets with the number of null movements made in each set, as well as the number of subjects that participated in each target set.

Table 1 Outline of the target set (top row) structure and the number of subjects (bottom row) that performed each target set

Quantifying generalization from trial-by-trial behavior

In recent years, a simple time series model (Thoroughman and Shadmehr 2000; Donchin et al. 2003) has provided insight into the motor control system and how its sensory motor representation generalizes between different portions of state space. Here we apply the same type of model to the present data using the perpendicular displacement (PD) at the maximum speed point of the movement as our error proxy. The general form of the model is

$$ \begin{aligned} Y^{{(n)}}&= {\left({D_{{ts(n)}} \times F^{{(n)}}} \right)} - Z^{{(n)}}_{{ts(n)}} \\ Z_{l}^{{(n + 1)}}&= Z_{l}^{{(n)}} + B_{{l,ts^{{(n)}}}} *Y^{{(n)}} l = 1,2,3,4 \end{aligned} $$

(2)

where Y ⁽ⁿ⁾ is the perpendicular displacement of the nth movement taken at the maximum speed, D _ts(n) is the compliance that translates the force F ⁽ⁿ⁾ experienced on a movement made with the target speed ts(n) into a displacement, Z ⁽ⁿ⁾ _l represents the internal states of the motor control system with one state representing each of the four different speed movements, B, called the generalization function, determines how the system generalizes errors from one movement speed to all other speeds. As there were four target speeds B is a 4 × 4 matrix allowing for asymmetries in the generalization. For instance, if our internal model gathers more information about the external forces during longer duration movements then we might expect errors made during such movements to update the internal model more than after making a short duration movement that is the effective learning rate might be greater after long duration movements. We found no evidence for this in the subjects data and therefore propose that only 4B’s are necessary, one for each speed category, which specify generalization at each speed regardless of the speed of the current movement. We found that subtracting the mean of the time series of PDs for each target set allowed us fit all of the data (960 movements) with one set of parameters (4Ds, 4Bs and 4Z ₀s, where Z ₀s are the initial values for the Zs), if we did not demean each target sets data then the time series model fit to the full 960 movements diverged drastically from the actual data over time apparently due to changes in the desired trajectory (see results).

We used the maximum speed in this model rather than the velocity as all of the movements were to one target with a majority of the velocity being in the direction toward the target, and we found the sum of squared error between the fit and the data to be slightly smaller, however not significantly different when using the speed.

Simulation of an adaptive internal model

The simulation that we used has been presented previously (Donchin et al. 2003). We modeled the subjects arm in the horizontal plane attached to the robotic manipulandum at the hand/handle junction. The physics can be expressed with the following set of equations:

$$ \begin{aligned} I_{r} (p)\ddot{p} + G_{r} (p,\dot{p})\dot{p} &= E(p,\dot{p}) + J^{T}_{r} (p)F_{{\text handle}} \\ I_{s} (q)\ddot{q} + G_{s} (q,\dot{q})\dot{q} &= C(q,\dot{q},q^{*} (t)) - J^{T}_{s} (q)F_{{\text handle}} \\ \end{aligned} $$

(3)

where r denotes robot, s the simulated subject, I the inertial matrix, p the robot joint angles, q the subject joint angles, q ^*(t) the minimum-jerk desired joint trajectory (Flash and Hogan 1985), G the Coriolis-centripetal matrix and E and C the forces produced actively by the robotic system and the simulated subject. J is the Jacobian for joint to Cartesian coordinate transformation. We used a simple feedforward internal model with a reflexive like feedback mechanism:

$$ C = {\hat{I}}_{s} (q^{*})\ddot{q}^{*} + {\hat{G}}_{S} (q^{*}, \dot{q}^{*})\dot{q}^{*} + {\hat{J}}^{T}_{s} (q^{*}){\hat{F}}_{{\text handle}} - K(q - q^{*}) - V(\dot{q} - \dot{q}^{*}). $$

(4)

In this equation ^∧ indicates the controllers estimate, K and V are the joint stiffness and viscosity matrices, respectively (Shadmehr and Mussa-Ivaldi 1994), and act as the reflexive feedback component of the controller in a proportional and differential manner respectively. The term ${\hat{F}}_{{\text handle}}$ combines the controller’s estimate of the robot’s passive and active dynamics that is

$$ {\hat{F}}_{{\text handle}} = {\hat{F}}_{{\text robot}} + {\hat{F}} $$

(5)

where ${\hat{F}}$ is the internal models force expectation of the force field produced by the robot. For simplicity sake, we assume that the subjects have an accurate estimate of their arm’s dynamics $({\hat{I}}_{s} + {\hat{G}}_{S})$ and after the null sets they have gained an accurate estimate of the robots passive dynamics $({\hat{F}}_{{\text robot}})$ as well. Thus, we simplified the above equation by setting

$$ {\hat{F}}_{{\text handle}} = {\hat{F}} $$

(6)

and did not simulate the passive dynamics of the robot.

In order for the simulation to learn, we assumed that the internal model is composed of basis elements g _j that map velocity into force via the relation ${\hat{F}}(\dot{x}) = Wg(\dot{x})$ where

$$ \begin{aligned} {\hat{F}}(\dot{x}) &= Wg(\dot{x}) \\ g(\dot{x}) &={\left[ {g_{1} (\dot{x}),\ldots,g_{m} (\dot{x})} \right]}^{T}\\ W&={\left[{\begin{array}{lll} w_{{x1}}&\ldots&w_{{xm}}\\ w_{{y1}}&\ldots&w_{{ym}}\end{array}} \right]} \end{aligned} $$

(1)

x is the position of the hand/handle with velocity $\dot{x}.$ Each basis element g _j has a Gaussian receptive field $\hbox{exp}(-{\left\|{\dot{\hbox{x}}}-{\dot{\hbox{x}}}_{\rm j} \right\|}^{2}/2\sigma^{2})$ in velocity space with ${\dot{\hbox{x}}}_{\rm j}$ being the center of the receptive field for basis element g _j . The centers of these basis functions were spaced one σ from their neighbors, and thus the number of basis elements changed as a function of σ. As in our previous work, we found that as long as the tiling was reasonably dense, the spacing between the basis only changed the learning rate (Donchin et al. 2003). This basis network learns via gradient decent with the error function e as follows:

$$ \begin{aligned} {\tilde{F}}(t) &= F(t) - {\hat{F}}(\dot{x}^{*} (t)) \\ e &= \frac{1}{{2T}}{\int\limits_0^T {{\tilde{F}}(t)^{T} {\tilde{F}}(t)\hbox{d}t}} \\ \end{aligned} $$

(7)

where ${\tilde{F}}(t)$ is the difference between the actual forces experienced during the movement F(t) and the controllers expectation of force ${\hat{F}}(\dot{x}^{*} (t))$ given the desired velocity trajectory $\dot{x}^{*} (t).$ The update rule for the weights w ⁽ⁿ⁾ _ij is

$$ w^{{n + 1}}_{{ij}} = w^{{(n)}}_{{ij}} - \eta \frac{{\partial e}}{{\partial w_{{ij}}}}, $$

(8)

with the result in matrix notation

$$ W^{{(n + 1)}} = W^{{(n)}} + \frac{1}{T}\eta {\int\limits_0^T {{\tilde{F}}^{{(n)}} (t)g(\dot{x}^{*}_{{ts(n)}} (t))\hbox{d}t}} $$

(9)

where $\dot{x}^{*}_{{ts(n)}}$ is the desired velocity trajectory at the specified target speed on movement n.

Statistics and analysis

In order to determine differences between the target sets and the velocity categories seen in Fig.3 we used a standard ANOVA followed by a multiple comparisons test (MCT). These were carried out using Matlab’s Anova function followed by the multcompare function using Tukey’s honestly significant difference criterion. In order to determine specific trends we used the cftool from Matlab’s curve fitting toolbox to determine the significance of polynomial fits with a linear (LT, linear trend), or a quadratic (QT, quadratic trend). In order to characterize the parameters in the time series model $\begin{aligned}& {\hat{Y}}^{{(n)}} = {\left({D_{{ts(n)}} *F^{{(n)}}} \right)} - Z^{{(n)}}_{{ts(n)}} \\& Z^{{(n + 1)}} = Z^{{(n)}} + {\left({B*{\hat{Y}}^{{(n)}}} \right)} \\ \end{aligned}$ we used a non-linear optimization algorithm (Matlab’s fsolve) to minimize the summed squared error between the time series model’s prediction and the mean of the subject’s data ${\sum_{n = 1}^N {{\left\| {{\hat{Y}}^{{(n)}} - Y^{{(n)}}} \right\|}}}^{2},$ where Y ⁽ⁿ⁾ was the time series of PDs from the subjects and ${\hat{Y}}^{{(n)}}$ the models predictions. After fitting the data and obtaining estimates on the time series model parameters (Ds, Bs and Z ₀ s, where Z ₀ s are the initial values for the state variables (Zs) and are fit as are the model parameters), we ran the time series model with these parameter estimates using the same series of target speeds and CT used for the subjects. We then computed the cross-correlation between this data (the time series model’s prediction) and the subject’s data. In order to determine the significance of the model, we used standard bootstrap techniques by constructing 200 resampled data sets (called bootstrap samples) from the human subjects. These bootstrap samples were generated by including a random set of the subjects data sets, precisely the same number as the number of subjects, and then taking the average of this data. The subject’s data sets were chosen at random and with replacement, thus a single subject’s data could be represented multiple times in one boot strap sample or not at all. Therefore, after this procedure we had 200 cross-correlations between the 200 bootstrapped samples and the 200 time series model predictions to each of these. To determine if the model significantly fit the data, we mandated that no more than 5 of the 200 cross-correlation coefficients be less than zero. If the model was in fact producing random data, we would expect half of the cross-correlations to be negative and half to be positive.

In order to establish the significance for each of the individual parameters in the time series model, we fit reduced versions of the model to the subjects’ data as described above.

The most reduced model is one with just the initial conditions on the internal states (Z ₀ s) without any additional parameters $({\hat{Y}}^{{(n)}}=Z^{{(0)}}_{{ts(n)}}).$ As before, we formed 200 boot strapped samples and fit this reduced model to those data sets and declared the parameters significant if 195 of the cross-correlations were above zero. This acted as our base model. The next reduced model was ${\hat{Y}}^{{(n)}}={\left({D_{{ts(n)}} \times F^{{(n)}}} \right)} - Z^{{(0)}}_{{ts(n)}},$ which included the force information along with the D parameter that transforms the force into a displacement. Again we carried out 200 boot strapped samples for this model. In order to determine the amount of additional cross-correlation that is due to the new parameters we simply subtracted the rank ordered cross-correlation values for the base model from the new more complete model. The mean difference is the amount of cross-correlation due to the new parameters. In order to determine if this additional correlation was significant we mandated that 195 of the 200 differences be greater than zero. A similar procedure was used to determine the cross-correlation due to the generalization parameters (B) and their significance. This same procedure was used for the adaptive controller simulation’s data.

Results

Raw data and general observations

Raw movement trajectories from a typical subject are plotted in Fig. 1. The target set is labeled on the left hand side of the figure; refer to Table 1 for details on the individual target sets. The first row of this figure is taken from the first null set. The speed category is labeled at the top of each column. The second row is taken from the first fielded (F) target set with the largest perpendicular deviations (PD) occurring in the fastest speed category, as one would expect due to the fact that the magnitude of the force field is linearly proportional to speed, the catch trials (CT) deviate to the left and are plotted with dotted lines. The very first F movement was made at the 55 cm/s speed and is the obvious trajectory deviated to the right in Set 4. The last row is taken from the eighth target set in which all of the 55 cm/s movements were CT. The speed profiles for each of the movements shown in Fig. 1. are plotted in Fig. 2.

Fig. 1

Plotted are all movements from a typical subject for several target sets (left column labels) for each of the four speeds (top row labels). CT-trials are plotted with a dotted line and deviate to the left of the page. The movements were 15 cm in extent and the x-axis spans from −5 to 5 cm

Fig. 2

Here we have plotted the speed profiles for each of the trajectories seen in Fig. 1. Once again CT-trials are plotted with dotted lines

The mean for the PD during F movements as well as CT–PD (PDs on CT), the maximum speed, and the parallel displacement (ParrD) taken at the maximum speed point of the movements are plotted in Fig. 3. In panels A and B, we have plotted the mean PDs during the null and F sets, respectively for each of the eight target sets averaged across subjects. There was no significant difference between the mean PDs (Fig. 3a) for the different speed categories during the null sets; however there was a significant difference between each of the null target sets (MCT p < 0.05). During the F sets, the mean PD for the slowest speed category was significantly closer to zero than the other three categories (Fig. 3b, ANOVA p < 0.001, MCT p < 0.01). In general each speed category had a significantly increasing linear trend over the five fielded sets, except for the 15 cm/s category (LT p < 0.05). The mean PD for the 75 cm/s category became significantly more deviated over the first four F sets with a significant difference between the first and fourth target sets (ANOVA p < 0.02, MCT p < 0.05). The 55 cm/s category also showed a significant increase in the deviation of the PD over the first four F sets with a significant difference between the second and fourth target sets (ANOVA p < 0.01, MCT p < 0.01). Each category demonstrated an obvious difference between the first four F sets and the last F set as expected due to the large number of CT in the last target set (ANOVA p < 0.001 for each speed category), as all of the 55 cm/s movements were CT in the fifth F set. This obvious increase in the PD for the last target set is clear evidence that there is significant generalization between the velocity categories. In panel C, we have plotted the mean CT–PDs during the F sets. Each of the speed categories had a significant trend in the CT–PDs such that they became less deviated with target set (LT, p < 0.05) with this trend being more robust for the faster categories (ANOVA p < 0.01 for each speed category). As with the PDs each speed category had a significant difference between the first and last target set (MCT p < 0.01) with the 75 cm/s also having significant differences between the first two and second two sets (MCT p < 0.01).

Fig. 3

Plotted are the mean PD for F movements as well as for CT, parallel displacement (ParrD) and maximum speed all taken at the maximum speed point of the movement averaged over all subjects for each target set. Panel A. Data from the three null sets with the following F data for the PD in panel B. There were no significant differences for the PDs between the speed categories during the null sets with the 15 cm/s category being the closest to zero during the F trials. Both the 55 and 75 cm/s categories showed a significant increase in the PD over the first four F sets with each category demonstrating a significant difference between the first four sets and the last in which all of the 55 cm/s movements were CT-trials. In panel C, we have plotted the mean CT–PDs (PD for catch trials) during the F sets. Each of the speed categories had a significant trend in the CT–PDs such that they became less and less deviated with target set. As with the F–PDs each speed category had a significant difference between the first and last target set with the 75 cm/s also having significant differences between the first two and second two sets. In panel D we have plotted the mean for the ParrD variables. The ParrD for the categories were significantly different for the three faster categories with a significant increasing trend for the 15 cm/s category. In panel E we have plotted the mean maximum speed for each speed category and each target set averaged over all subjects along with a line representing the target maximum speeds with each category being significantly different from the others. (see text for ANOVA values)

As the movements were 150 mm long we would expect the subjects to reach their maximum speed at approximately 75 mm if they were in fact making bell shaped speed profiles (Flash and Hogan 1985). We can see in Fig. 3d that the 75 and 55 cm/s categories were very close to 75 mm when they reach their maximum speed, with the 75, 55 and 35 cm/s categories being significantly different from each other (ANOVA p < 0.001, MCT p < 0.05). The 35 cm/s category tends to reach the max speed slightly earlier in the movement. Interesting is the fact that as the subjects make more movements at the 15 cm/s speed they continually shift the position at which they reach the maximum speed, that is in the early sets they reach the max speed spatially earlier in the movement (ANOVA p < 0.05), but by the last set they appear to have converged on reaching the max speed at the half way point (LT, p < 0.05). The inset in Fig. 3d gives some perspective on just how close all four categories are to reaching the max speed at the midpoint of the movement. In this inset we re-plotted the data on axis that show the full extent of the movement, which was 150 mm.

As the maximum speed was one of the target criteria for the subjects we wished to determine how capable they were at reaching the correct speeds. In Fig. 3e we have plotted the mean maximum speed for each target set and for each speed category averaged over all subjects. Clearly the subjects were able to make the movements at the appropriate speed, and the speed categories differed significantly from each other (ANOVA p ≪ 0.001, MCT p ≪ 0.001). In general, the 35 cm/s category was slightly below the target speed while the faster categories were more obviously below the target speed, but still within 5 cm/s allowed, perhaps as a strategy to conserve energy. Only the first target set for the 75 cm/s category had a mean maximum speed more than the allowed 5 cm/s outside the target speed (mean 68 cm/s). Each speed category had a significant linear trend over all eight target sets with the faster three categories speeding up over time, and the 15 cm/s category slowing down (LT, p < 0.05). Only the 75 and 15 cm/s categories demonstrated such trends when looking only at the fielded movements (LT p < 0.05). We also found an obvious decreasing linear trend for the standard deviation (STD) of the speed over time that was significant (LT p < 0.05) for all, but the 15 cm/s category with the adjusted R-square values being 0.26, 0.13 and 0.05 for the 75, 55 and 35 cm/s categories respectively. The mean duration for the different speed movements was 661, 725, 938 and 1771 ms.

Error generalization and motor learning, human subjects

In order to determine the pattern of generalization that the subjects made between different speed movements we fit the dynamic system of equations (Eq. 2), that is the time series model, to the subjects’ time series of PDs as described in the methods. Initially we used the full 16 B version of the model, but found that the non-linear solver did not converge on a single solution. Therefore, we tested three hypotheses by seeding the initial values for the Bs and then allowing the non-linear solver to determine the best values for the Bs. In doing this the non-linear solver did converge and these results are presented in Fig.4. In the left column of Fig.4 we have the initial values for the Bs corresponding to the three hypotheses we tested. The first hypothesis (Fig.4a) we tested was one in which the generalization was maximum for the speed of the movement just made with a linear decrease in B as one moves away from that preferred speed. The second hypothesis (Fig. 4c) was that the generalization is always highest for the fastest speed movement and it linearly decreases with speed. The last hypothesis came from the convergent solution from the non-linear solver for the 4B version of the time series model, and corresponds to a plateau hypothesis were the generalization hits a maximum and plateaus (Fig. 4e). The right column of Fig. 4 show the mean Bs that the non-linear solver found by minimizing ${\sum\limits_{n = 1}^N {{\left\| {{\hat{Y}}^{{(n)}}- Y^{{(n)}}} \right\|}}}^{2}$ for all 960 movements (all five fielded target sets) for the 200 bootstrap data sets (see "Methods"). We have included the mean cross-correlation (r) between the 200 outputs of the time series model using these Bs and the 200 subjects’ bootstrapped data sets first for all of the movements and secondly for just the F movements. The later two hypothesis were significantly better at fitting the subjects data than was the first, however there was no significant difference between the later two hypothesis and in general the Bs for the different speed groups were not significantly different within each model.

Fig. 4

Plotted in the left column are the initial B values we used to seed the non-linear solver corresponding to the three hypothesis we tested; the first being that of a linear decrease in generalization as one moves away from the speed that the movement was made at (a). The second hypothesis was that the generalization is linear and decreases as a function of the speed as well as being the same no mater what movement speed was just made (c) and the third was similar to the second, but plateaus (e). Plotted in the right hand column are the subsequent Bs found by the non-linear solver. We have included the cross correlation (r) between the output from the time series model (Eq. 2) and the subjects’ data for all of the movements and then for just the fielded movements. The later two hypotheses had significantly higher r values than the first, but not significantly different from each other and all of these models had significantly lower rs than the 4B version. The B values are unitless

It seems clear that 16Bs are not necessary and these models are over specified, thus we now present the results from the 4B version of the model. In fitting the 4B version we did not specify any starting point for the Bs, but rather started with random values as the non-linear solvers solution for the 4B version was convergent. We have plotted the mean generalization function (Bs) ± the standard deviation from the fit to the 200 surrogate data sets (see "Methods") for all five F sets in Fig. 5. Also shown are the mean cross correlation coefficient (r) between the 200 surrogate data sets and the 200 time series model fits to that data, first using the full model with all of the data points, and secondly we have given the partial r for B using only the F data. We did this due to the fact that much of the r for the full data set is accounted for by the D variable along with the CT and as our interest is in the generalization of the internal model we wished to focus on its contribution to the fits, which is captured by the B’s. In the set 1 panel of Fig. 5 we have plotted the B’s from set one in bold while plotting all of the B’s from the other target sets with thin lines for ease of comparison, with each of the latter sets being plotted on their own axes in the following panels. The D variables for each target set have been plotted in the lower right panel, showing the expected result, that is a higher level of arm stiffness at higher speeds/forces (Gribble et al. 2003; Osu et al. 2004). There was no significant difference between the target sets with respect to the Ds. All Ds and Bs were significantly different from zero (bootstrap p < 0.01).

Fig. 5

Plotted in each panel is the mean generalization function (B) taken from the time series model fit to 200 bootstrapped data sets (see "Methods") ±std. An asterisk indicates that a B is significantly different from the other Bs (one asterisk for p < 0.05 and two asterisks for p < 0.01). In the set 1 panel we have plotted the Bs from the first target set in bold and the Bs from all of the other target sets with thin lines for comparison. In each panel the r value first for the full time series model followed by the partial r for the Bs for F movements only are shown (see "Methods" and text). Plotted in the last Panel are the D’s for each of the F sets with no significant differences between the target sets. Both the Ds and Bs are unitless

In Figs. 6 and 7 we have plotted the mean time series of the PDs for the subject’s (black lines) as well as the mean time series model fits to that data (gray line). In Fig. 6a we have plotted the first 80 PDs from F set 1 with CT (large circles) deviating toward the bottom of the page. We have coded the different speed movements (see figure key). As one would expect, the first F trial is the most deviated and was made at the 55 cm/s speed. Panel B is the first 80 PDs from the last F set in which all of the 55 cm/s movements were CT. Note that even with the large number of CT, the model fits the data very well for the F trials.

Fig. 6

Plotted are the mean time series of PDs from the subjects (gray) and the time series model fit (black) shown for the first 80 movements from the first fielded target set in A. We have coded the different speed movements (see Key) for comparison. Plotted in B are 80 consecutive movements from the last target set where all of the 55 cm/s movements were CT

Fig. 7

Here we have plotted the partial r for each of the individual variables that comprise the time series model for each fielded target set. These values are for the fielded trials only that is we did not calculate these rs using the CT-trials (see test)

In order to illustrate the extent to which each of the time series model parameters contributes to the fits, we have plotted the partial r for them and for each target set in Fig. 7 only for the F movements, due to the fact that the CT are so large they tend to account for much of the r if not looked at separately. There was a general trend for the r to increase for the full model with practice as can be seen in Panel A. The trend for the generalization parameter (B) was the opposite with a decreased partial r with target set over the first four sets. The Z ₀’s r generally increased with each subsequent target set as can be seen in Fig. 7c. There was no clear trend for the D variable with the second F target set differing significantly from all the other sets.

Comparison of subjects and simulation results

We used a simple simulation comprised of an adaptive controller that learned to move a model of a human arm attached to the robotic manipulandum that produced a viscous curl field. This simulation utilized an internal model, comprised of a tessellation of Gaussian basis functions that spanned velocity space allowing it to learn to account for the perturbing forces of the curl field (see "Methods"). We fit the time series model (Eq. 2) to the data generated by the adaptive controller in the same manner used for the human subjects. In this way, we were able to compare the results from the subjects and the adaptive controller in order to determine if our assumption on the shape of the basis functions matched the human subject’s data as the shape of the basis functions will influence the shape of the generalization function B. We found a surprising similarity between the adaptive controller and the human subjects’ data as is depicted in Fig. 8.

Fig. 8

A comparison between the subjects’ and the adaptive controller. Plotted in panel A are the mean generalization functions (Bs) ±std from the time series model fit to the mean of the subject’s data (black) and the simulation’s data (gray). Included are the rs for the subjects (rH) and the adaptive controller simulation (rS). First, we give the r for the time series model fit with the full data sets followed by the r for the Bs using only the fielded data (see text). In panel B we have plotted the mean (±std) of the Ds. Here we have included the partial r for the Ds using all of the data followed by the partial r using only the fielded data. The large difference is an indication of correlation, or r that is due to the CT. In panels C and D we have plotted the time series of PDs for the human subjects (black) as well as for the simulation (gray) for two sequences of movements. In panel C, we plotted the early stages in the fielded trials while in D we plotted data from the last fielded target set in which all of the 55 cm/s movements were CT-trails. In panel D we have included the r between the two data sets for all movements (labeled full) followed by the r between the data sets during the fielded trials alone. These values are for the full data sets not only the data shown

In Fig. 8 panel A we have plotted the mean B’s ± the std for the subjects (black) and the adaptive controller (gray). All 960 movements from the 5 F data sets were used in fitting these Bs. As in previous figures, these are the means for the parameters taken from 200 bootstrap realizations of the data. In the case of the simulation, it is simply the mean from running the non-linear solver 200 times starting with the single realization of the simulation and starting each non-linear solver with random initial conditions for the parameters to be fit. There was no significant difference between any of the B parameters between the subjects and the simulation using Gaussians with a sigma of 14 cm/s. We have included the mean correlation coefficient between the mean subject’s data and the time series model fits to that data (rH) as well as the time series model fits to the simulation data (rS). The first of the two numbers are for the full model and the second is for the B’s looking only at the F movements. We fit both a linear and a quadratic polynomial to the subjects’ Bs (means shown in Fig. 8a) in order to determine the significance of the apparent trends, with the result that both the linear and quadratic fits were highly significant as were all of the parameters in each polynomial fit. The adjusted r-square value for the quadratic fit was higher at 0.83 vs. 0.7 for the linear fit indicating that the plateau seen in Fig. 8a is significantly different from linear.

In panel B we have plotted the D’s with similar results between the simulation and the subjects. Included in the panel are the r’s for the D’s using all of the data and secondly for the D’s using only the F data. In panels C and D we have plotted the output from the adaptive controller (gray) as well as the subjects (black) with good agreement between the two. The movement number is given on the x-axis. Thus, panel C shows the first 50 data points in the force field, while panel D is taken from the last target set with all of the 55 cm/s movements being CT. The correlation coefficient between the two data sets was 0.94 using all 960 data points, and 0.68 if we only used the 753 F data points.

Discussion

In the work presented here we have tested the hypothesis that humans use a sensory motor control system for reaching movements that is consistent with an adaptive controller, which uses a population code comprised of Gaussian basis functions, representing force as a function of velocity. We had subjects make reaching movements to a single visual target at four different speeds that were linearly spaced with respect to the maximum speed and visited in a pseudorandom order. Subjects encountered a viscous curl field that they learned to compensate for. Using catch trials to induce error, we probed the pattern of the subjects’ learning using a simple linear dynamical system of equations, the time series model (Eq. 2). In turn, we were able to test the velocity generalization of the sensory motor control system on a trail-by-trial basis, in substitution for the more traditional paradigm of first training the subjects in one portion of state space with subsequent testing for generalization at other portions of state space (Goodbody and Wolpert 1998). We were able to capture the subjects’ behavior on a trial-by-trial basis using this simple model as we and others have done in the past for movements to different spatial targets (Thoroughman and Shadmehr 2000; Donchin et al. 2003). We then used this same time series model as a means to compare a simulated adaptive controller to the human subjects with the result that an adaptive controller made up of Gaussian (sigma 14 cm/s) basis functions tessellated in velocity space had a generalization function (B) statistically indistinguishable from the human subjects (Fig. 8a). This width of the Gaussian basis functions is in good agreement with previous work (Thoroughman and Shadmehr 2000).

The update rule and generalization between speeds

In general, one might expect that when making a reaching movement in a given portion of state space that errors experienced during that movement will most strongly influence the internal model’s representation at that state (Shadmehr and Mussa-Ivaldi 1994; Gandolfo et al. 1996), what we and others have seen for subjects in the past (Thoroughman and Shadmehr 2000; Scheidt et al. 2001; Donchin et al. 2003). However, this is not exactly what we see in the subjects data presented here. In Fig. 8a where we used all 960 movements giving us the best statistical confidence on the Bs, it is clear that errors experienced while making slow movements influence the internal states at the faster speeds more than at the speed the movement was made at, and this was not simply due to under-specifying the regression model, as when we used the full 16 B version we saw a similar pattern (Fig. 4).

How does time play a role in motor learning? Recently, we have demonstrated that the amount of time between movements can play a role in the amount of learning that takes place (Francis 2005), where a very short delay (0.5 s) between movements appeared to decrease the amount of learning as compared to longer delays (≥5 s), perhaps indicating an interaction, or sharing of resources between the neural substrates involved in motor production and motor learning. These results have recently been corroborated (Bock et al. 2005; Huang and Shadmehr 2007). How can time influence the amount of learning that takes place during a given movement when the movements are of different duration? We hypothesized that during a longer duration movement the internal model may be able to “learn more” than for short duration movements as it would be able to sample the force field for a longer period of time and thus might have a higher learning rate, that is B value, after such movements. However, the pattern of Bs seen in Figs. 4 and 8 indicate that the internal model did not update it self more after movements made with longer durations. Perhaps error experienced during a movement is scaled by the duration of the movement.

In 1998, Goodbody and Wolpert (1998) suggested that the generalization between movements made at different speeds and durations was linear. In that work the authors used a velocity dependant curl field as we have here. Looking at Fig. 8a one would have to agree that for the 15, 35 and 55 cm/s reaches there seems to be evidence for a linear generalization as indicated by the linear increase of the Bs for those speed categories. To quantify this generalization we determined the slope for the Bs seen in Fig. 8a. The Bs are a gain that gives us the percentage of the error made, which is used to update the internal model. Thus, to determine the generalization between the different speeds we took the slope from the Bs (Fig. 8a) curve plotted as percentages of the maximum gain, which was for the 55 cm/s movement along the y-axis and as percentages of that speed on the x-axis. In doing this we find that the slope between the 15 and 35 cm/s categories’ Bs is 1.2, 0.83 between the 35 and 55 cm/s categories and −0.12 between the 55 and 75 cm/s categories. Clearly the amount of generalization is very close to having a slope of one for the first three speed categories, even if it does seem to be steadily decreasing, in agreement with Goodbody and Wolpert (1998). However, the generalization clearly deviates from this linearity between the 55 cm/s category and the 75 cm/s category were it plateaus (Fig. 8a). We also found that a quadratic polynomial fit this trend significantly better (adjusted R-square 0.83) than a linear fit (adjusted R-square 0.7).

The desired trajectory

We found evidence that the desired trajectory was changing with time during the null sets, and at least for the faster two speed categories during the F sets as is indicated in Fig. 3a–c, where the 55 and 75 cm/s categories’ PDs became slightly, but significantly more deviated with F target set number and their corresponding CT became less deviated (Fig. 3c). These types of changes have been reported previously (Donchin and Shadmehr 2004), and may be part of an energy conservation strategy used by the subjects. As seen in Fig. 3e, the two faster speed categories had mean speeds that were consistently slower than the target speed while still adhering to the goal criterion of the task that allowed the subjects to be 5 cm/s slower than the target speed. As the force field was linearly proportional to the speed, the subjects were decreasing the maximum amount of force they would need to compensate for the perturbation by keeping their speed lower. This strategy did not appear to be adopted for the 15 cm/s category where the subjects performed almost perfectly at the target speed. Perhaps at the lowest velocity, the amount of maximum force production saved by decreasing ones velocity is very small rendering the savings insignificant or undetectable. We do not have a good reason as to why the 55 cm/s PDs where consistently larger than the 75 cm/s category at the maximum velocity position, given that the force field was linearly proportional to the velocity we would have expected that the 75 cm/s category would be the most deviated. However, as the 55 cm/s categories CT were smaller than the 75 cm/s categorie’s it seems reasonable that the desired trajectory for the 55 cm/s movements was simply set to be more deviated from zero than was the 75 cm/s category . Recent research has suggested that the duration/force profiles that subjects prefer are not a simple relation and this may in part be why we have observed the above counterintuitive results (Kording et al. 2004).

As we mentioned in the "Methods" section, we found that by simply demeaning the PD data on a set by set basis, we did not need to include terms in the time series model for a time dependant baseline for each speed category. That is, we did not need to explicitly include the assumed changing desired trajectory information for the model to produce good fits. Along with evidence for a changing and or different desired trajectory for the different speed categories, we found that the size of the generalization parameters (Bs) decreased with target set as shown in Fig. 5. Not only did the absolute value of the Bs decrease over time, but also the correlation coefficients attributable to them as seen in Fig. 7b. It would appear that by increasing the number of CT in the last target set we increased the size of the Bs comparing sets 4 and 5, where in set five all of the 55 cm/s movements were catch trials, however it did not recover to the level seen in the first two sets, which were the first F sets conducted on consecutive days (see "Methods"). Thus, if this decreasing trend is real it must not be due simply to time, as there were 24 h between the first and second F sets, but perhaps to the number of trials, or the amount of time actually engaged in the reaching task within a session.

Gain modulation

It has been proposed that the motor control system has the ability to change its gain on different control strategies, such as having one gain associated with an inverse dynamics model and a second gain associated with a compliance controller (Franklin et al. 2003). We did not detect any significant difference in the D parameters between the target sets, which are an indirect measure of the subjects’ compliance. However, when we calculated the difference between the PDs and the CT–PDs on a set by set basis, we did detect a significant decreasing trend that would indicate the subjects were increasing the stiffness of their arms (ANOVA p < 0.01). Only the first and the last target sets were significantly different (MCT p < 0.05). We also detected changes in the amplitude of the B parameters over time, which indicates a changing gain on the inverse dynamics model. Thus, it appears the subjects were changing the gain on their compliance controller slightly while also changing the gain on the inverse dynamics model over the course of time, such that they were still adhering to the task goals while perhaps minimizing their energy output. This could explain the results seen in Fig. 7 as one would expect the partial r for the Z ₀s to increase as the partial r for the Bs decreases. For instance, if the B values continued decreasing in value to zero then there would be no trail to trial learning, and thus the internal states would not change and would remain at Z ₀, therefore the amount of cross correlation due to the Z ₀s would increase.

Separation of internal models without contextual cues

In the fifth F set, all of the 55 cm/s movements were CT, as we wished to determine if the subjects were capable of allowing their internal model of this speed to return to the null state, while continuing to maintain the other speed categories at a force field appropriate state. Not a single naive subject was able to separate the 55 cm/s category such that they no longer had deviated trajectories as can be seen by the representative subject in Fig. 1, as well as in the average data shown in Fig. 3 where the F movements for the other three speed categories become significantly more deviated, indicating substantial generalization between the 55 cm/s null movements and these other speeds. We even tested several experienced subjects that did not have explicit knowledge of the task; they too were unable to dissociate their internal models. However, the author having full knowledge of the situation and experience on this task was fully capable of making straight movements on all of the 55 cm/s reaches in the last target set (data not shown). This is at least a proof that it is possible for humans to use separate internal models or strategies for different velocity movements and perhaps a simple cue is all that would be needed for the naive subjects to do the same, however further experimentation is clearly needed. Cues have been successfully used in the past to allow subjects to deal with conflicting force fields (Krouchev and Kalaska 2003; Shadmehr et al. 2005) supporting this preliminary finding. We have not presented results on the parallel error in this paper as we did not find any significant generalization in that dimension, a result that has been seen for prism adaptation as well (Kitazawa et al. 1997).