Human Control of Interactions with Objects – Variability, Stability and Predictability

Abstract

How do humans control their actions and interactions with the physical world? How do we learn to throw a ball or drink a glass of wine without spilling? Compared to robots human dexterity remains astonishing, especially as slow neural transmission and high levels of noise seem to plague the biological system. What are human control strategies that skillfully navigate, overcome, and even exploit these disadvantages? To gain insight we propose an approach that centers on how task dynamics constrain and enable (inter-)actions. Agnostic about details of the controller, we start with a physical model of the task that permits full understanding of the solution space. Rendering the task in a virtual environment, we examine how humans learn solutions that meet complex task demands. Central to numerous skills is redundancy that allows exploration and exploitation of subsets of solutions. We hypothesize that humans seek solutions that are stable to perturbations to make their intrinsic noise matter less. With fewer corrections necessary, the system is less afflicted by long delays in the feedback loop. Three experimental paradigms exemplify our approach: throwing a ball to a target, rhythmic bouncing of a ball, and carrying a complex object. For the throwing task, results show that actors are sensitive to the error-tolerance afforded by the task. In rhythmic ball bouncing, subjects exploit the dynamic stability of the paddle-ball system. When manipulating a “glass of wine”, subjects learn strategies that make the hand-object interactions more predictable. These findings set the stage for developing propositions about the controller: We posit that complex actions are generated with dynamic primitives, modules with attractor stability that are less sensitive to delays and noise in the neuro-mechanical system.

Submitted to: Laumond, J.-P. Geometric and Numerical Foundations of Movement.

1 Introduction

Imagine a dancer, perhaps Rudolf Nureyev or Margaret Fonteyn, both legends in classical ballet: we can only marvel at how they are in complete control of their body, combining extraordinary flexibility and strength with technical difficulty and elegance. And yet, I submit that Evgenia Kanaeva, two-times all-around Olympic champion in rhythmic gymnastics, equals, if not surpasses their level of skill: Not only does she move her lithe body with perfection and grace, she also plays with numerous objects: she throws, catches and bounces a ball, she rolls and swivels a hoop, and sets a 6 m-long ribbon into smooth spirals with the most exquisite movements of her hands and fingers – and yes, sometimes also using her arms, shoulders, or her legs and feet. Her magical actions and interactions with objects arguably represent the pinnacle of human motor control.

How do humans act and interact with objects and tools? After all, tool use is what gave humans their evolutionary advantage over other animals. In robotics, manipulation of tools has clearly been one of the primary motivations to develop robots, going back to the first industrial robots designed to automate repetitive tasks such as placing parts or tightening screws. However, these actions lack the dexterity that not only elite performers, but all healthy humans display. Opening a bottle of wine with a corkscrew or eating escargot with a fork and tongs are skills that require subtle interactions with complex tools and objects. How do humans control these actions and interactions?

Research in motor neuroscience has only arrived at limited answers. To assure experimental control and rigor, computational research has confined itself to simple laboratory tasks, most commonly reaching to a point target, while restricting arm movements to two joints moving in the horizontal plane [57, 58]. Research on sequence learning has typically been limited to finger presses evaluated with simple discrete metrics of timing and serial errors [43, 81]. Grasping has been reduced to isometric finger presses with predetermined contact points to analyze contact forces [37, 82]. The obvious benefit of such simplifications is that the data are accessible and tractable for testing theory-derived hypotheses. Over the past 20 years, numerous studies in computational neuroscience have embraced control-theoretical concepts, such as Kalman filters [39], Bayesian multi-sensory integration [2, 80], and optimal feedback control [74] to account for such experimentally controlled human data. While advances have been made, nobody would deny that this approach encounters challenges when the actions become more complex and realistic. This is particularly problematic when actions are no longer free, as in reaching, but involve contact with objects, ranging from pouring a glass of wine to moving the ribbon in gymnastics. Needless to say, current state-of-the-art movements of robots are still a far cry from those of Elena Kanaeva. Why do humans perform so much better, at least to date? What can robotic control learn from human neuromotor control?

1.1 The Paradox: Delays and Noise in the Human Neuromotor System

A first look into the biological neuromotor control system reveals some puzzling facts: information transmission in the human central nervous system is extremely slow and also very noisy. Action potentials, the basic unit of information coding, travel at a speed of approximately 100 m/s [32]; the shortest feedback loop is around 50 ms and reserved for startle reactions [35, 47]. When feedback is integral to more meaningful responses, loop times of 200 ms and longer are a more realistic estimate. In addition to such long delays, the biological neuromotor system displays noise and fluctuations at all levels [13]. The biological system is an extremely complex nonlinear system with multiple levels of spatiotemporal scales, ranging from molecular and cellular processes to motor units and muscle contractions, and to overt behavior. Noise and fluctuations from all these levels manifest themselves at the behavioral level as ubiquitous variability. For example, in simple rhythmic finger tapping even trained musicians exhibit at least 5% variance of the period [19, 72]. In a discrete throwing action, humans display a limit in timing resolution of 9 ms [8]. Such large delays and high levels of noise pose extreme challenges for any control model. And yet, humans are amazingly agile and dexterous.

While the human controller appears clearly inferior to robotic systems, the biological “hardware” with its compliant muscles and soft tissues defy any comparison with the heavy actuators of robots. It seems highly likely that the dexterous human controller exploits these features. More recent developments in robotics have produced actuators with variable compliance, such as hands or grippers made of soft material [12] or actuators with mechanically adjustable series compliance [78]. However, the flexibility that comes with variable stiffness may also incur costs, such as loss in precision or higher energy demands. How do humans combine their software limitations and use their compliant and high-dimensional actuators to solve complex task demands?

1.2 Intrinsic and Extrinsic Redundancy

The biological sensorimotor system has a large number of hierarchical levels with high dimensionality on each level. One important consequence of this high dimensionality is that it affords redundancy and thereby an infinite variety of ways a given action can be performed. At the behavioral level, hammering a nail into a wooden block can be achieved with multiple different arm trajectories and muscle activation patterns. The adage “repetitions without repetition” conveys that the ubiquitous and ever-present fluctuations prevent any action to be the same as another one. Importantly, this intrinsic redundancy faces an additional extrinsic redundancy that is inherent to the task. Imagine dart throwing: the bull’s eye or the rings on the dartboard allow a set of hits that achieve a given score. Further, orientation angle of the dart stuck on the board does not change the score. Hence, the task has extrinsic redundancy that permits a manifold of solutions [68]. However, not all solutions are equally suitable: some may not be biomechanically optimal, others may be risky, yet others may have a lot of tolerance to error and noise. Examining human performance may reveal how humans navigate the task’s redundancy and preferences may give insight into the controller. Hence, a suitably constructed extrinsic redundancy presents an important entry point into examining human control, strategies, or objective functions.

1.3 An Agnostic Approach to Human Motor Control

Recognizing these challenges, our research has adopted an approach with minimal assumptions about human neuromotor control. Instead of starting with a hypothesized controller and the plant, i.e., the brain and the musculo-skeletal system, connected by forward and feedback loops transmitting motor and sensory signals, we take an agnostic stance. We begin with what is known and can be analyzed: the physical task that the actor performs. Under simplified conditions, very few assumptions need to be made about the human controller.

This chapter will review this task-dynamic approach as it was developed in three experimental paradigms that examine human interactive skills. These three skills progress from the simple action of throwing a ball, to rhythmic intermittent bouncing of a ball, to the continuous manipulation of a complex object, a cup with a rolling ball inside, mimicking a cup of coffee – or a glass of wine. Mathematical analyses and exemplary results will show that variability, stability and predictability matters in human motor control. I will close with a still largely speculative hypothesis on how the human control system generates such actions, a perspective that may be less hampered by long delays and noise: control via dynamic primitives.

2 A Task-Dynamic Approach to Understanding Control of Interactions

Using mathematical modeling and virtual technology we developed a task-dynamic approach to study the acquisition and control of simple and more complex interactive skills. Following a brief outline of the methodological steps, three exemplary lines of research will be reviewed with some selected results.

2.1 Identifying a Motor Task

The important initial step is choosing a motor task that satisfies several desiderata: First, it should represent a core aspect germane to many other tasks that is “interesting” from a control perspective. Second, the motor task should have redundancy: the well-defined goal should allow for a variety of solutions to achieve the task goal. Third, the task should be novel and sufficiently challenging to require practice to achieve success. The changes over practice provide an important lens to reveal how humans navigate through the space of solutions. (Note this differs from studying everyday behaviors, such as reaching or grasping, where only adaptations to novel scenarios produce change.) Fourth, improvement should happen within one or few experimental session(s), but should also allow for fine-tuning over a longer time scale. These stages are likely to reveal processes underlying motor learning.

We selected and designed three tasks: The arguably simplest (inter-)active task is to throw a ball to a target. While the ball only needs to be released, the size and location of the target imposes constraints on the release that fully determine the projectile’s trajectory and thereby the hitting accuracy. A next step in interaction is to repeatedly contact the ball – such as in bouncing a ball rhythmically in the air. This intermittent interaction extends the control demands, as the propelled object needs to be intercepted again. Any error at one contact influences the subsequent contact – these repeated interactions render the task a dynamic system. The third task takes interactions one significant step further: motivated by the seemingly mundane action of carrying a cup of coffee or glass of wine, we designed a simplified task that exemplifies the continuous interaction with a complex object.

2.2 Mathematical Model of the Task

Once the core control challenge is identified, the task is modeled mathematically to formalize and prune away irrelevant aspects of the real-life task. A simple physical model also facilitates subsequent analyses of both model and human data. What system captures the essential demands of ball release and permits a full analysis of the solution space? What is the simplest intermittent dynamical system that lends itself to mathematical analysis? What is the simplest physical system that captures the continuous interaction between the human and a dynamically complex object? One core element in our mathematical modeling and analysis is the distinction between the execution variables x and the result variables r: The result variable(s) are defined by the task goal and the instruction to the subject and quantify the quality of performance. This is typically an error measure, although this error measure can take many forms. Execution variables are under control of the performer and determine the task result. For the analysis it is important to identify all execution variables that fully determine the result, in order to have an analytic or numerical understanding of the space of solutions. The functional relation between execution and result is the essence of the model and analysis: $\mathbf{r} = f(\mathbf{x})$.

2.3 Mathematical Analysis and Derivation of Hypotheses

Based on the physical model, the space of all possible solutions to the task can be derived. As the model system is typically nonlinear, the space of solutions may be complex and subsets of solutions may have additional properties, such as dynamic stability, risk, or predictability, as elaborated below. The model structure determines the mathematical tools that can be used to derive predictions. Core to our task-dynamic approach are analyses of stability, error sensitivity, or robustness to perturbations and noise. Importantly, exact quantitative hypotheses can be formulated that define those solutions with the greatest probability of success.

2.4 Implementation in a Virtual Environment

Based on the explicit mathematical model, the task is rendered in a virtual environment that permits precise measurement of human execution and errors, i.e., the execution and result variables. The execution variables are those that the subject controls via interfacing with the virtual system. For example, while the subject performs a throwing task, the real arm trajectory controls the ball release, but the ball and the target are virtual. The virtual rendering has the advantage that it confines the task to exactly the model variables and its known parameters. There are no uncontrolled aspects as would occur in a real experiment. Further, the parameters and result variables can be freely manipulated to test hypotheses about human control strategies.

2.5 Measurement, Analysis, and Hypothesis Testing of Human Performance

Subjects interact with the virtual physics of the task via manipulanda that simultaneously render the task dynamics and measure human performance strategies. The measured execution variables and the task result are then evaluated against the mathematical analysis of the solution space. The virtual environment affords easy manipulation of the model, its parameters, and specific task goals. Hypotheses about preferred solutions are derived from model analysis and can be evaluated based on the human data. As shown below, the task can be parameterized to create interesting task variations to contrast alternative control hypotheses.

2.6 Interventions

Based on the findings, the controlled virtual environment can also be used to create interventions that guide or shape behavior. This is significant for clinical applications, where scientifically-grounded quantitative assessments and interventions are still rare. While this review will focus on the basic science issues, some applications to questions on motor control in children with dystonia or on interventions for the elderly can be found in Sternad [60], Chu et al. [5], Hasson and Sternad [24].

Fig. 1

The virtual throwing task. a Schematic of the real task. b The 2D model from a top-down view. c The experimental set-up with force and position sensors for recording of human movement. Measured movements are shown in real time on the screen (Reproduced from [68])

3 Throwing a Ball to Hit a Skittle – Variability, Noise, and Error-Tolerance

3.1 The Motor Task

This experimental paradigm was motivated by a ball game found in many pubs and playgrounds around the world: The actor throws a ball that is tethered to a virtual post by a string like a pendulum; the goal is to hit a target skittle (or skittles) on the opposite side of the pole (Fig. 1a). Accurate throwing requires a controlled hand/ball trajectory that prepares the ball release at exactly the right position with the right velocity to send the ball onto a trajectory that knocks over the target skittle. The practical advantage of this game is that the tethered ball cannot be lost and the game can be played in a small space; the theoretical advantage is that the pendular motions of the ball introduce “interesting” dynamics with a nonlinear solution space including discontinuities that present challenges to trivial learning strategies such as gradient descent. Importantly, the task has redundancy and thereby offers a manifold of solutions with different properties.

3.2 The Model and Its Virtual Implementation

To simplify the three-dimensional task, the ball was confined to the horizontal plane, eliminating the pendular elevation during excursion (Fig. 1b). In the model, the ball is attached to two orthogonal, massless springs with its rest position at the center post. In the virtual implementation, the actor views the workspace from above on a backprojection screen (Fig. 1c). S/he throws the virtual ball by moving his/her real arm in a manipulandum that measures the forearm rotations with an optical encoder; these measured movements are shown online by a virtual lever arm (Fig. 1b). Deflecting the ball from the rest position and throwing the ball with a given release angle and velocity, the ball traverses an elliptic path generated by the restoring forces of the two springs. The following equations describe the ball motion in the $x-y$ coordinates of the workspace:

$$\begin{aligned} \left( \begin{array}{c}{x(t)} \\ y(t) \end{array} \right) ~ = ~\left( \begin{array}{c} {x_{p} } \\ {y_{p} } \end{array} \right) \, cos\, \omega t + ~\left( \begin{array}{ll}{\, cos\, \phi _{r} } &{} { - \, sin\, \phi _{r} } \\ { - \, sin\, \phi _{r} } &{} {\, cos\, \phi _{r} } \end{array} \right) \left( \begin{array}{c} {l\, cos\, \omega t~} \\ {v_{r} /\omega t} \\ \end{array} \right) \end{aligned}$$

(1)

$\omega$ denotes the natural frequency of the springs, $(x_p, y_p)$ denotes the lever’s pivot point, and l the length of the arm (Fig. 1b). Damping of the springs can be added; asymmetric damping and also stiffness may be used to introduce a more complex force field in the workspace. For a given throw, the two execution variables angle $\phi _r$ and velocity $v_r$ of the virtual hand at ball release fully determine the ball trajectory in the workspace x(t), y(t) (for more details see [7]).

The actor’s goal is to throw the ball to hit the target skittle, without hitting the center post. The latter restriction eliminates simple ball releases with zero velocity. Post hits are therefore penalized with a large fixed error. Otherwise, error is defined as the minimum distance between the ball trajectory and the target center (Fig. 1b). Thus, the result variable is the scalar error that is fully determined by $\phi _{r}$ and $v_{r}$. Importantly, there is more than one combination of $\phi _r$ and $v_r$ that leads to zero error, i.e. the task has the simplest kind of redundancy: two variables map onto one. While this low dimensionality permits easy visualization in 3D to develop intuitions, the manifold of zero-error solutions can also be analytically derived and expressed in implicit form:

$$\begin{aligned} \frac{{v_{r} }}{\omega } = \frac{{\left| {\left( { - l\, sin\, \phi _{r} - y_{p} } \right) x_{t} + \left( {l\, cos\, \phi _{r} + x_{p} } \right) y_{t} } \right| }}{{\sqrt{\left( {l + \, cos\, \phi _{r} x_{p} + \, sin\, \phi _{r} y_{p} } \right) ^{2} - \left( {\, cos\, \phi _{r} x_{t} + \, sin\, \phi _{r} y_{t} } \right) ^{2} } }} \end{aligned}$$

(2)

Fig. 2

Two target constellations (a, b) and their corresponding result spaces (c, d). For each task, three exemplary ball trajectories are shown which correspond to the three release points plotted in the result spaces (green dots). White denotes zero-error solutions, increasing error is shown by increasingly darker grey shades, black denotes a post hit. In both constellations, two ball trajectories exemplify how different release variables can lead to the same result with zero error (1, 2, dashed lines). Trajectory 3 shows a trajectory that does not intersect the target (Modified from [61])

3.3 Geometry of the Solution Space

Figure 2 illustrates two different target constellations that generate two different topologies of the result space [61]. Figure 2a, b show the top-down view of the workspace with the red circle representing the center post and the yellow circle the target. The manipulandum is shown at the bottom with its angular coordinates. The three elliptic trajectories are three exemplary ball trajectories with different release angles and velocities. In both work spaces two ball trajectories (1, 2) go through the target and have zero error, while one (3) has a non-zero error. Figure 2c, d show the respective result spaces, spanned by release angle and velocity; error is depicted by shades of gray, with lighter shades indicating smaller errors. White denotes the zero-error solutions, or the solution manifold. Black signifies those releases that hit the center post, which incur a penalty in the experiment. The three points are the ball releases pertaining to the three ball trajectories above.

The two result spaces present several interesting features: In target constellation (a) the solution manifold has a nonlinear J-shape that represents solutions over a wide range of release velocities and angles. As indicated by the grey shades, the regions adjacent to the solution manifold have different gradients and the sensitivity of the zero-error solution changes along the solution manifold. Further, the region on the J-shaped manifold with the lowest sensitivity is directly adjacent to the black penalty region. Hence, strategies with the lowest velocity were adjacent to penalized post hits; this poses risk and a simple gradient descent may run into problems. In target constellation (b) the zero-error solutions are independent of velocity and fully specified by the release angle, as the solution manifold runs parallel to velocity. As visible from color shading, low-velocity solutions have slightly less error tolerance compared to high-velocity solutions and again transition directly into the penalty region. Note that other target locations have yet different geometries of the solution manifold creating different challenges to the performer [68].

3.4 Generating Hypotheses from Task Analysis

One study created two result spaces with different topologies to generate specific predictions [61]. Given that humans have limited control accuracy due to the pervasive noise in their neuromotor system, we hypothesized that in such redundant tasks humans seek solutions that are tolerant to their intrinsic noise and also to extrinsic perturbations (Hypothesis 1). Such error-tolerant solutions have higher likelihood to be accurate and would therefore also obviate some error corrections. This is advantageous as error corrections incur computational cost and, importantly, the sensorimotor feedback loop suffers from the long delays in the human system. Note that our definition of error tolerance differs from standard sensitivity analyses that assess local sensitivity in a linearized neighborhood. As humans make relatively large errors and the topology is highly nonlinear, we calculated error tolerance as the average error over an extended neighborhood around a chosen solution; this neighborhood is defined by the individual’s variability. An alternative hypothesis was that humans select strategies that minimize velocity at release to avoid costs associated with higher effort or signal-dependent noise (Hypothesis 2). There is much evidence that movements at slow velocities are preferred, as higher speed tends to decrease accuracy (speed-accuracy trade-off) [16, 17, 42]. This observation concurs with the information-theoretical expectation that noise increases with signal strength. In motor control, signal strength is typically equated with firing rate of action potentials, i.e. force magnitude in the isometric case or, in the dynamic case, movement velocity or acceleration. A third hypothesis discussed in the human motor control literature is that risk is avoided, and participants stay at a distance from the penalty area (Hypothesis 3) [6, 40, 48].

Fig. 3

Hypotheses and experimental results for two task a (left column) and task b (right column). The top row shows the expected results, E(R) for Hypothesis 1: Maximizing error tolerance; the second row shows simulated predictions for Hypothesis 2: Minimizing velocity and signal-dependent noise. The expected result E(R) was computed as average error over a neighborhood scaled by a softmax function (for details see [61]). The peaks highlighted by the red circles denote the expected solutions. The third row shows the data as histograms plotted over the result spaces to compare against the predicted solutions (Modified from [61])

3.5 Error Tolerance Over Minimizing Velocity and Risk

Nine participants practiced 540 and 900 throws with Task (a) and (b), respectively. Figure 3 illustrates the predictions as computed for Hypothesis 1 and 2 in the top two rows. Error tolerance was quantified as the expected error or result E(R) over a neighborhood around each strategy, simulating that human strategies are noisy: it was then by a softmax function. For Hypothesis 2, expected velocity was computed over the same neighborhood, again scaled by a softmax function. The solutions that are most error-tolerant and those with lowest velocity are indicated by red circles in the middle panels. Examining all throws after removing the initial transients, the bottom panels show the histograms of all subjects’ releases in both result spaces (from Fig. 2c, d).In Task (a) the data distribution clustered along the solution manifold at low velocities and close to the discontinuity. The mode at angle 236 deg and velocity 136 deg/s was close to the maximally error-tolerant point as predicted by Hypothesis 1. However, the solutions also had relatively low velocity, which was consistent with Hypothesis 2. These two benefits seemed to outweigh that these solutions were close to the high-penalty area, i.e. risky strategies were not avoided, counter to Hypothesis 3. Task (b) was designed to dissociate Hypotheses 1 and 2. The histograms on the right panel illustrate that the data were distributed across a large range of velocities between 140 and 880 deg/s, with the mode of the data distribution at 544 deg/s, although individual preferences were more clustered on the velocity axes. The fact that individuals chose solutions over a wide range of velocities, without a specific preference for low-velocity or the high-tolerance point was at first sight inconsistent with both Hypotheses 1 and 2. However, in further analyses the observed variability of each individual was regressed against release velocity,  which revealed that variability did not increase at higher velocities, as would be expected from Hypothesis 2. Instead, these analyses showed that strategies were better explained by error-tolerance, consistent with Hypothesis 1 (for details see [61]).

Taken together, this first study showed how a task analysis can generate predictions that permit direct tests based on human data. The conclusion from this study is that humans seek out error-tolerant strategies, i.e., those where variability at the execution level has minimal detrimental effect on the result. As these strategies attenuate noise effects on the result, fewer errors occur that in turn require fewer corrections to stay on target. This not only reduces computations but also diminishes the negative effect that delays may cause.

Fig. 4

Data from an exemplary subject who practiced the throwing task for 15 days. The initially broad scatter translated to a more error-tolerant strategy, rotated to covary with the solution manifold (white) and scaled of reduced the amplitude of dispersion over the course of practice (Modified from Cohen and Sternad [7])

3.6 Tolerance, Covariation, and Noise

Increasing error-tolerance is only one of three avenues to deal with unavoidable variability in execution. Two more, conceptually different avenues exist for how variability can be transformed to lessen its effect on the task result. Figure 4 illustrates this notion with data from a representative subject who practiced the same throwing task for 15 days, 240 throws per day [7]. The geometry of the result space shows a U-shaped solution manifold due to a different target constellation. The broad scatter of the data on Day 1 reflects initial exploratory attempts with inferior results compared to those after some practice. Most visibly, on Day 6 the data not only translated to a location on the solution manifold with more error-tolerance (shown as a wider band of white), but the observed variability also started to covary with the direction of the solution manifold, while overall variability was only moderately reduced. The distribution on Day 15 clearly reveals a third transformation: the overall dispersion was significantly reduced or scaled, over and above the further enhanced covariation. These three data transformations, corresponding to the matrix transformations of translation, rotation, and scaling, were numerically quantified from individual data distributions as costs: The average result of a given data set could be improved by 1.2 cm on Day 1, if it were translated to the optimal location. The difference in average result from actual to optimal renders Tolerance-cost. If the actual data were rotated or permuted optimally, the difference in result with the real data would quantify Covariation-cost. If the real data distribution was scaled or its noise was reduced optimally, the difference between initial and optimal data quantifies Noise-cost. The parallel, but differential evolution of the three costs was shown in Cohen and Sternad [7].

3.7 Covariation, Sensitivity to Geometry of Result Space in Trial-by-Trial Learning

A separate study specifically focused on covariation and examined not only the distributions of the data, but also their temporal evolution to assess whether subjects’ trial-by-trial updates were sensitive to the direction of the solution manifold [1]. Three detailed hypotheses guided our experimental evaluation: Hypothesis 1: Humans are sensitive to the direction of the solution manifold, which is reflected in preferred directions of their trial-to-trial updates. Hypothesis 2: This direction-sensitivity becomes more pronounced with practice. Hypothesis 3: The distributional and temporal structure is oriented in directions orthogonal and parallel to the solution manifold. Note that sensitivity to the directions of the null space is also core to several other approaches, which employ covariance-based analyses that linearize around the point of interest using standard null space analysis [10, 55]. In contrast to our approach, those analyses do not exploit the entire nonlinear geometry of the result space.

Thirteen subjects practiced for 6 days throwing to the same target as above, with 240 throws per day (4 blocks of 60 trials). To assess the distribution and also trial-to-trial evolution, each block of 60 throws was examined as illustrated in Fig. 5a. To assess whether the trial-to-trial changes had a directional preference, the 60 data points were projected onto lines through the center of the data set (red lines in Fig. 5a). The center was typically on or was close to the solution manifold. The direction parallel to the solution manifold was defined as $\theta _{par}$, the direction orthogonal to the solution manifold was defined as $\theta _{ort}$. The black horizontal line in Fig. 5a defines the direction of $\theta = 0$ deg. The time series of the projected data was then analyzed using autocorrelation and Detrended Fluctuation Analysis (DFA).

Fig. 5

a Result space with solution manifold (green), with angle and velocity normalized to variability of each individual. Red lines denote directions parallel and orthogonal to the solution manifold. The black line denotes $=$ 0 rad. Data are projected onto lines between $0<\theta <\pi$ rad and autocorrelations are computed for each projection. b Time series of projected data where autocorrelation was at a minimum and a maximum. Note that these directions do not necessarily correspond to parallel and orthogonal directions (Reproduced from [1])

Fig. 6

Autocorrelation of time series of projected data in all directions in result space. The modulation across directions becomes more pronounced with practice, expressing increased sensitivity to the geometry of the result space. Note that while the extrema are close to the directions of the solution manifold (SM$_\text {par}$ and SM$_\text {ort}$) they are not coincident (Modified from [1])

This line was then rotated through $0<\theta <\uppi$ rad, in 100 steps, with its pivot at the center of the data. At each rotation angle $\theta$, the data were projected onto the line and time series analyses conducted. We expected that in directions orthogonal to the solution manifold $\theta _{ort}$ successive trials show negative lag-1 autocorrelation, reflecting error corrections; in the parallel direction $\theta _{par}$ correction was not necessary, as deviations have no effect on the task result. Note that the result space is spanned by angle and velocity, i.e. with different units; hence, both axes had to be normalized to each individual’s variance to ensure orthogonality and a metric.

Figure 5b shows two time series of projected data from those directions that rendered maximum and minimum autocorrelation. Note the visible difference in temporal structure, reflecting that direction in the result space does matter. Plotting the results of the lag-1 autocorrelations across angle of the projection in Fig. 6 reveals a marked modulation: The red lines (with variance across subjects shown by shaded bands) show autocorrelation values for each rotation angle. The modulation supports Hypothesis 1 that trial-by-trial updates are sensitive to the angle, and implicitly, the direction of the solution manifold. The green vertical lines denote the orthogonal and parallel directions of the solution manifold. The minima and maxima of the autocorrelation values are indicated by triangles. Consistent with Hypothesis 2, the modulation gets more pronounced across the three practice blocks, expressing that after the initial stage, trial-to-trial dynamics became more directionally sensitive. The structure in the orthogonal direction changed from initially positive autocorrelations to white noise and eventually very small negative values [1].

3.8 Orthogonality and Sensitivity to Coordinates

This analysis also revealed important discrepancies to Hypothesis 3. The directions of minimum and maximum autocorrelation were near, but not coincident with the orthogonal and parallel directions, as hypothesized. This finding alerts to an important issue: orthogonality is sensitively dependent on the chosen variables. In the present case, the original physical variables, angle and velocity, had different units and required normalization. While technically correct, it raises the question whether these units accurately reflect the units of the central nervous system. One important caveat for this and related approaches is that the structure of variability is fundamentally sensitive to the chosen coordinates.

This fact was highlighted in a separate study, which showed that this sensitivity is particularly pertinent for covariance-based analyses [69]. Even simple linear transformations can critically alter the results, as demonstrated by a simulation that examined 2-joint pointing movements to a target line in the horizontal plane. Given the univariate error measure, distance to the line, the mapping between error and bivariate joint angles was redundant. Analysis of variability of error as a function of joint angles, revealed that the anisotropy of the data distribution depends on the definition of joint angles: relative angles or absolute angles with reference to the shoulder. While covariance-based analysis of anisotropy of data is dependent on the coordinates, we also demonstrated that our analysis of error tolerance, covariation and noise is significantly less sensitive, as it projects the execution variables into the result space. Nevertheless, these critical questions open an interesting avenue for conceptually deeper questions: What are the coordinates of the nervous system? What is the appropriate metric? What is the best or most suitable representation of the problem? While data may be dependent on the coordinates, can data be used to reversely shed light on the coordinates that the nervous system uses?

To pursue these questions, the study by Abe and Sternad further examined how a rescaling of the execution variables in a simple model of task performance with similar redundancy may reproduce these deviations [1]. While this revealed possible sources for these observations, much more work is needed. For example, scaling noise in different execution variables or sensory signals might also give rise to such “deviations”. These are clearly important issues for understanding biological movement control, and possibly also worth reflection when designing control in robotic systems.

3.9 Interim Summary

The throwing skill illustrated our model-based approach and its opportunities to shed light on human control. The findings showed that humans choose strategies that obviated the potentially detrimental effects of intrinsic noise. With less noise and variability, less error corrections are needed. Error corrections are not only computationally costly, they are also hampered by the slow transmission speed in biological systems. Are similar strategies also possible in different tasks, especially when interacting with an object?

4 Rhythmic Bouncing of a Ball – Dynamic Stability in Intermittent Interactions

4.1 The Motor Task

Rhythmically bouncing a ball on a racket is a playful and seemingly simple task. Yet, it requires a high degree of visually-guided coordination to intercept the ball at the right position and with the right velocity to reach a target amplitude and perform in a rhythmic fashion (Fig. 7a–c). As in the throwing task, success is determined at one critical moment when the racket intercepts the ball, as this impact fully determines its amplitude. Hence, the core challenge of this task is the control of collisions, a feature germane to numerous other behaviors, ranging from controlling foot-ground impact in running to playing the drums. One key difference to throwing is that these impacts are performed in a repeated fashion, and errors from one contact propagate to the next. Hence, the actor becomes part of a hybrid dynamical system combining discrete and continuous dynamics [11, 44, 46, 53].

Fig. 7

Bouncing a ball with a racket. a The real task. b The physical and mathematical model. c Simulated time series assuming invariant sine waves of the racket. d Redundancy of the result space: Racket position and velocity and ball velocity determine ball amplitude. Blue data points are from early practice, yellow data points are from late practice (Reproduced from [68])

4.2 The Model

The physical model for this task is again an extremely simple dynamical system, originally developed for a particle bouncing on a vibrating surface [21, 75]. The model consists of a planar surface moving sinusoidally in the vertical direction; a point mass moving in the gravitational field impacts the surface with instantaneous contact (Fig. 7b). The vertical position of the ball $x_{b}$ between the kth and the $k+1$th racket-ball impact follows ballistic flight:

$$x_{b} (t)~ = ~x_{r} (t_{k} )~ + v_{b}^{ + } (t - t_{k} ) - g/2(t - t_{k} )^{2}$$

where $x_{r}$ is racket position, $v_{b}^{ + }$ is the ball velocity just after impact, $t_{k}$ is the time of the kth ball-racket impact, and g is the acceleration due to gravity. With the assumption of instantaneous impact, the ball velocity just after impact $v_{b}^{ + }$ is determined by:

$$v_{b}^{ + } ~ = ((1 + \alpha )v_{r}^{ - } - \alpha v_{b}^{ - } )$$

where $v_{b}^{ - }$ and $v_{r}^{ - }$ are the ball and racket velocities just before impact, and the energy loss at the collision is expressed in the coefficient of restitution $\alpha$. The maximum height of the ball between $t_{k}$ and $t_{k+1}$ depends on $v_{b}^{ - }$ and $v_{r}^{ - }$ and the position at impact $x_{r}$:

$$\begin{aligned} {max}_{{t_{k} \le t \le t_{{k + 1}} }} x_{b} (t) = ~x_{r} (t_{k} ) + (((1 + \alpha )v_{r}^{ - } - \alpha v_{b}^{ - } )(t - t_{k} ))^{2} /2g \end{aligned}$$

(3)

4.3 Redundancy

The task goal is to bounce the ball to a target height, and the error is defined as the deviation from the target height (Fig. 7c). Even in this simplified form, the task has redundancy, as the result variable error is determined by three execution variables: $v_{b}^{ - },~v_{r}^{ - }$ and $x_{r}$. Figure 7d shows the execution space with the solution manifold, i.e. the planar surface that represents all solutions leading to zero error. The blue and yellow data points are two exemplary data sets from early and late practice, respectively; each data point corresponds to one ball-racket contact. As to be expected, the early (blue) data show a lot of scatter, while the late practice data (yellow) cluster around the solution manifold.

4.4 Dynamic Stability

While the redundancy analysis is performed on separate collisions, the racket and ball model also lends itself to dynamic stability analysis. To facilitate analysis, the racket movements are assumed to be sinusoidal, such that racket position and velocity at impact collapse into a single state variable, racket phase $\theta _{k}$. Applying a Poincare section at the ball-racket contact, where $x_{r}$ and $x_{b}$ are identical, a discrete map can be derived with $v_{k}^{ + }$ and $\theta _{k}$ as state variables:

$$\begin{aligned} \begin{array}{c} \ \ \, v_{{k + 1}}^{ + } = (1 + \alpha )A\omega \, cos\, \theta _{{k + 1}} - \alpha v_{k}^{ + } + g\alpha (\theta _{{k + 1}} - \theta _{k} )/\omega \\ 0 = A\omega ^{2} (\, sin\, \theta _{k} - \, sin\, \theta _{{k + 1}} ) + v_{k}^{ + } \omega (\theta _{{k + 1}} - \theta _{k} ) - g/2(\theta _{{k + 1}} - \theta _{k} )^{2}\end{array} \end{aligned}$$

(4)

A and $\omega$ are the amplitude and frequency of the sinusoidal racket movements [11, 53, 65]. This nonlinear system displays dynamic stability and, despite its simplicity, shows the complex dynamics of a period-doubling route to chaos [21, 75]. For present purposes, only stable fixed-point solutions are considered as they correspond to rhythmic bouncing. Local linear stability analysis of this discrete map identifies a stable fixed point, if racket acceleration at impact $a_{r}$ satisfies the inequality:

$$\begin{aligned} - 2g\frac{{(1 + \alpha ^{2} )}}{{(1 + \alpha )^{2} }}< a_{r} < 0 \end{aligned}$$

(5)

4.5 Hypotheses

In this dynamically stable state, small perturbations of the racket or ball die out without requiring corrections. Hence, if subjects establish such dynamically stable regime, they need not correct for small perturbations that may arise from the persistent neuromotor noise. Thus, we hypothesized that subjects learn these “smart” solution and exploit dynamic stability by hitting the ball with negative racket acceleration (Hypothesis 1). Further, due to the system’s redundancy infinitely many stable solutions can be adopted. Hence, we administered perturbations to test if subjects established and re-established such stable states (Hypothesis 2).

4.6 Virtual Implementation

In the experiments, the participant stood in front of a projection screen and rhythmically bounced the virtual ball to a target line using a real table tennis racket. Similar to the throwing task, the projected racket movements were shown on the screen in real time impacting the ball. The display was minimal and only showed the modeled and measured elements, a horizontal racket and a ball, both moving vertically to a target height (Fig. 7b). A light rigid rod was attached to the racket and ran through a wheel, whose rotations were registered by an optical encoder, which measured the vertical displacement of the racket, in analogy with the model, and shown on the screen. Racket velocity was continuously calculated. The vertical position of the virtual ball was calculated using the ballistic flight equation initialized with values at contact. To simulate the haptic sensation of a real ball-racket contact, a mechanical brake, attached to the rod, was activated at each bounce and decelerated the upward motions. Racket acceleration at or just before the impact was analyzed after the experiment and served as the primary measure of dynamic stability to test Hypothesis 1 [79]. Ball position and velocity and racket velocity at contact were measured and analyzed to evaluate the data with respect to the solution manifold (Hypothesis 2).

Fig. 8

Ball amplitude errors and racket accelerations over 48 trials. All data points are averages over 8 subjects. a, b Stage 1 of the experiment. c, d Stage 2 of the experiment. The shading denotes the perturbed trials

4.7 Learning and Adaptation to Perturbations

Did human subjects seek and exploit dynamic stability of the racket-ball system? How robust is this system if the actor has to change and adapt to new situations? An experiment tested these questions in two stages: On Day 1, 8 subjects performed a sequence of 48 trials of rhythmic bouncing to a target height, each trial lasting 60 s. With the target height at 0.8 m from the lowest racket position, and $\alpha = 0.6$, the average period between repeated contacts was 0.6 s, leading to approximately 100 contacts per trial. On Day 2, subjects performed 10 trials under the same conditions as on Day 1, but then performed another 48 trials after a perturbation was implemented.

Stage 1: Figure 8a shows the ball amplitude errors averaged of all subjects across 48 trials. As expected, the error decreased with practice with a close-to exponential decline. Concomitantly, the acceleration of the racket at contact decreased from an initially positive to a negative value, indicative of performance attaining dynamic stability (Fig. 8b). Importantly, it took approximately 11 trials for subjects to “discover” this strategy, showing that it was not trivial and required practice to learn it. The parallel evolution of both error and racket acceleration with practice provide strong support for Hypothesis 1 that subjects seek dynamic stability.

Stage 2: The second experimental session presented an even stronger test. Starting with 10 regular trials as on Day 1, subjects were exposed to a perturbation over the subsequent 48 trials (yellow shading in Fig. 8c, d). This perturbation was calculated using the redundancy of the execution: three execution variables, $v_{b}^{ - } , v_{r}^{ -}$ and $x_{r}$, determined the one result variable, absolute error of ball peak amplitude to the target height. Following Day 1, the average and standard deviations of $v_{b}^{ - }$ and $v_{r}^{ - }$ and $x_{r}$ of the first 10 baseline trials were calculated for each individual to render an ellipsoid in result space representing the individually preferred solution (9). In the subsequent perturbed trials this preferred strategy was penalized with an error in ball amplitude. This error was delivered by replacing the veridical ball release velocity with one calculated based on the execution ellipsoid. This new ball velocity over- or undershot the target height as calculated. By simply replacing the ball velocity at the contact discontinuity, subjects did not explicitly perceive the perturbation. Within the ellipsoid, the penalty was maximal at its centroid and it linearly decreased to zero towards the boundaries (defined by one standard deviation around its centroid). Hence, assuming sensitivity to the error gradient in result space and the redundancy of the task, subjects were expected to search for a new un-penalized solution. This perturbation was calculated and delivered only in the virtual display such that subjects saw their drop in performance, but did not notice its cause explicitly.

Figure 9 illustrates the performance of one representative subject. Starting with the (larger) execution ellipsoid from the initial 10 trials (Fig. 9a), upon onset of the perturbation the subject gradually translated her execution along the planar solution manifold to a new location. The smaller and darker ellipsoid on the right depicts the average execution of the last trial: The strategy shifted and the variability decreased even further; importantly, there was no overlap with the initial ellipsoid (Hypothesis 2). This illustrates that the subject not only found a new successful solution without penalty, but the non-overlap also suggested that the subject was aware of her variability.

Fig. 9

Presentation of performance in execution space; the planar surface is the solution manifold. a The large execution ellipsoid represents the initially preferred strategy that is subsequently penalized during the perturbation phase. The smaller ellipsoid represents the final strategy that is established during the perturbation phase to avoid the penalty. b The right panel shows the same data and execution ellipsoid. The points are the sequence of trial means following the perturbation onset. It shows that subjects stay on the manifold but migrate outside the penalty ellipsoid

Returning to the measures or error and racket acceleration at impact for these same data, shown in Fig. 8c, d, reveals that upon perturbation onset, both errors and racket acceleration changed significantly as expected. However, over the course of the 48 perturbed trials, subjects incrementally decreased their errors and reestablished the previously preferred racket acceleration of −3 m/s$^{2}$. In fact, this acceleration value was determined to be optimal for the given parameters in additional Lyapunov analyses of the model system [53]. This result shows that subjects successfully established dynamic stability in multiple different ways.

Experimental evidence that subjects learn to hit the ball with a decelerating racket has been replicated in several different scenarios. The different experimental set-ups included a pantograph linkage with precise control of the haptic contact, a real tennis racket to bounce a real ball attached to a boom, and freely bouncing a real ball in 3D [65, 66]. The findings were robust: with experiences, performers learn to hit the ball with negative racket acceleration; based on stability analyses of the model we concluded that they learn to tune into the dynamic stability of the racket-ball system. Based on these findings, we also designed an intervention to guide subjects towards this dynamically stable solution. Manipulating the contact parameters via a state-based shift indeed successfully accelerated subjects’ learning the dynamically stable solution, which correlated with faster performance improvement [30].

4.8 Interim Summary

These studies provided strong evidence that humans seek dynamic stability in a task, a solution that is computationally efficient as small errors and noise converge without necessitating explicit error correction. In the face of perturbations, subjects successfully navigated the result space and established new solutions available due to the redundancy. There was also evidence that they were aware of their own variability. As in skittles, subjects seek solutions where noise matters less.

5 Chaos in a Coffee Cup – Predictability in Continuous Object Control

5.1 The Motor Task

Leading a cup of coffee to one’s mouth to drink is a seemingly straightforward action. However, transporting a cup filled with sloshing fluid to safely contact the mouth without spilling remains a challenge not to be underestimated for both humans and robots. Carrying a cup of coffee (or a glass of wine) exemplifies a class of tasks that require continuous control of an object that has internal degrees of freedom. How do humans control interactions with such an object, where the sloshing fluid creates time-varying, state-dependent forces that have to be preempted and compensated to avoid spill? Can humans or robots really have a sufficiently accurate internal model of the complex fluid dynamics to online predict and react to the complex interaction forces? In search of human strategies that apparently deal with this problem easily, we started again with the analysis of the task dynamics, following the steps outlined above.

Fig. 10

Carrying a cup of coffee. a The model task. b The conceptual model: a 2D arc with a ball rolling inside. c Control model of the cart-and-pendulum. d Virtual implementation with the HapticMaster robot to control the cup in the horizontal direction. e The interactive screen display; the green rectangles specify the amplitude of the cup movement. The lower panel shows a sequence of moving cups with the arrows depicting the respective forces of cup and ball (Reproduced from [60])

5.2 The Model

In principle, the task presents a problem in fluid dynamics [38, 49]. To make this complex infinitely-dimensional system more tractable, several simplifications were made [23]: (1) the 3D cup was reduced to 2D, (2) the sloshing coffee was reduced to a ball with point mass rolling in a cup, (3) the hand contact with the cup was reduced to a single point of interaction, (4) the cup transport was limited to a horizontal line (Fig. 10a–c). More precisely, the moving liquid is represented by a pendulum suspended to a cart that is translated in the horizontal x-direction. The pendulum is a point mass m (the ball) with a mass-less rod of length l with one angular degree of freedom $\theta$. Subjects control the ball indirectly by applying forces to the cup, and the ball can escape if its angle exceeds the rim of the cup. The cup is a point mass M that moves horizontally. The hand moving the cup is represented by a horizontal force F(t). Despite these simplifications, the model system retained essential elements of complexity: it is nonlinear and creates complex interaction forces between hand and object. The equations of the system dynamics are:

$$\begin{aligned} (m+M)&\ddot{x}=ml(-\ddot{\theta }cos \phi +\dot{\theta }^2sin\phi )+F(t)\nonumber \\&\,\,l\ddot{\theta } = - \ddot{x}cos\theta - gsin\theta \end{aligned}$$

(6)

where $\theta , \dot{\theta }$, and $\ddot{\theta }$ are angular position, velocity, and acceleration of the ball/pendulum; $x ,\dot{x}$, and $\ddot{x}$ and are the cart/cup position, velocity, and acceleration, respectively; F is the force applied to the cup by the subject; g is gravitational acceleration. The model has four state variables $x , \dot{x} , \theta , \dot{\theta }$ and the externally applied force F(t) that determines the behavior of the ball and cup system. Hence, only one variable F(t) is under direct control of the subject, but this is co-determined by the ball/pendulum interacting with the cart. These instantaneous interaction forces make the distinction into execution and result variables significantly more complicated than in the previous two examples.

5.3 Virtual Implementation

The ball-and-cup system was implemented in a virtual environment. The cart and the pendulum rod was hidden, leaving only the ball visible. In addition, a semicircular arc with radius equal to pendulum length l was drawn on the screen so that the ball appeared to roll in a cup (Fig. 10d, e). Subjects manipulate the virtual cup-and-ball system via a robotic arm, which measures hand forces $F_{{External}}$ applied to the cup but also exerts forces from the virtual object onto the hand (HapticMaster, Motek [76]). $\theta$ and $\dot{\theta }$ were computed online and the ball force $F_{{Ball}}$ was computed based on system equations such that the force that accelerated the virtual mass $( m + M )$ was $F_{{applied}} = M\ddot{x} = F_{{External}} + F_{{Ball}}$. Two rectangular target boxes set the required movement distance and spatial accuracy (for more details see [23]).

5.4 Model Analysis and Hypothesis

The cup of coffee can be moved as a relatively short discrete placement to a target, or in a more continuous fashion, as for example carrying the cup while walking. A previous study examined a single placement onto a target focusing on the discontinuous aspect of the task: the coffee can be spilled [23, 24]. Given the noise intrinsic to the neuromotor system and the fluctuations created by the extrinsic cart-and-pendulum system, avoiding spilling coffee, or losing the ball, became the core challenge when the task was to move as fast as possible. The “distance” from losing the ball was quantified by an energy margin, defined as the difference between the current energy state and the one where the ball angle would exceed the rim angle. Results showed that this continuous metric sensitively captured performance quality and learning in healthy and also older subjects.

Here, we review another study that examined more prolonged interaction, where the nonlinear dynamics manifests its full complexity and, technically, displays chaos [41, 67]. To this end, the task instruction was to move the cup rhythmically between two very large targets leaving amplitude under-specified; the task-specified frequency defined the result variable. Movement strategies were fully described by the execution variables cup amplitude, frequency, and initial angle and velocity of the ball, $A,f,\theta _{0},\dot{\theta }_{0}$. To derive hypotheses about the space of solutions, inverse dynamics analysis was conducted to calculate the force F(t) required to satisfy the task. Numerical simulations were run for combinations of the scalar execution variables $A,f,\theta _{0} ,\dot{\theta }_{0}$. To keep the number of simulations manageable, frequency f was fixed to the task-required frequency, and $\dot{\theta }_{0}$ was set to zero.

Figure 11 shows two example profiles generated by inverse dynamics calculations with two different initial ball states $\theta _{0} ( \dot{\theta }_{0}=0)$ that both result in a sinusoidal cup trajectory x(t). The left profile F(t) shows irregular unpredictable fluctuations for $\theta _{0} = 0.4$ rad, while the right profile initialized at $\theta _{0} = 1.0$ rad shows a periodic waveform with high regularity. To characterize the pattern of force profiles with respect to the cup dynamics, F(t) was strobed at every peak of cup position x(t). The marginal distributions of the strobed force values are plotted as a function of initial ball phase $\theta _{0}$ in the bottom panel. This input-output relation reveals a bifurcation diagram with a pattern similar to the period-doubling behavior of chaotic systems, indicating chaos in the cup-and-ball system.

Fig. 11

Inverse dynamics simulations of the cart-and-pendulum model. Top panels show two different simulation runs with different initial ball angles $\theta _{0}$, requiring a complex and a relatively simple input force (top row). Strobing force values at maxima of the cup profile x and plotting the marginal distributions against all initial ball angles renders the bifurcation-like diagram (Reproduced from [41])

5.5 Hypotheses for Human Control Strategies

It seems uncontested that controlling physical interaction requires “knowledge” and prediction of object dynamics. On the other hand, it is reasonable to doubt that the complex details of object dynamics are known or faithfully represented in an internal model. In chaotic dynamics, small changes in initial states can dramatically change the long-term behavior and, technically, lead to unpredictable solutions. Can or should internal models be able to represent this complex dynamics? To make this challenge more tractable for the neural control system we hypothesized that subjects seek solutions that render the object behavior more predictable to reduce computational effort and facilitate at least some prediction.

To quantify the concept of predictability of the object dynamics based on the human’s applied force, we computed mutual information MI between the applied force and the kinematics of the cup, i.e. long-term predictability of the object’s dynamics [9]. MI is a nonlinear correlation measure defined between two probability density distributions that quantifies the information shared by two random variables, F(t) and the kinematics of the cup x(t):

$$\begin{aligned} MI\left( {x,F} \right) = \iint p\left( {x,F} \right) log_{e} \frac{{p(x,F)}}{{p\left( x \right) p(F)}}dxdF \end{aligned}$$

(7)

Fig. 12

Result space computed for three different hypothesized control priorities. The space is computed for different initial ball angles and cup amplitudes; frequency is set to 1 Hz, and ball velocity is set to zero. a Mutual information. b Effort defined as mean squared force over a given trial. c Smoothness or mean squared jerk defined over a given trial. The optimal strategy for each hypothesis is noted by the large dot (Reproduced from [41])

MI presents a scalar measure of the performer’s strategy calculated at each point of the 4D execution space spanned by $A,f,\theta _{0} ,\dot{\theta }_{0}$. The higher MI, the more predictable the relation between force and object dynamics. Hence, we expected that subjects would seek strategies with high MI (Hypothesis 1, Fig. 12a). Predictability as a control priority had to be tested against alternative hypothesis. The experiments permitted testing two alternative control priorities: minimizing effort (Hypothesis 2, Fig. 12b) and maximizing smoothness (Hypothesis 3, Fig. 12c); both are commonly accepted and widely supported criteria in free unconstrained movements. To calculate the effort required for each strategy, the Mean Squared Force of the force profile was calculated: $MSF = \frac{1}{{nT}}\mathop \smallint \limits _{0}^{{nT}} F(t)^{2} dt$, where n denoted the number of cycles and $T = 1/f$ the period of each cycle. Mean Square Jerk was calculated as $MSJ=\frac{1}{T(\displaystyle \mathop {\theta }^{\ldots }{\!}_{max}-\displaystyle \mathop {\theta }^{\ldots }{\!}_{min})}\int \limits _{0}^{T}{\,|\theta |^{\displaystyle \mathop {2}^{\ldots }}dt}$, where the value was normalized with respect to ball jerk amplitude to make it dimensionless [27]. Similar to MI, MSF-values were calculated for all strategies in 4D execution space. To constrain the calculations, the initial value of the angular velocity $\dot{\theta }_{0}$ was set to zero, consistent with the experimental data. Figure 12 compares the corresponding predictions for MI, MSF, and MSJ. Color shades express the degree as explained in the legend. The large dots denote the points of maximum MI, minimum MSF and MSJ. Importantly, these predicted strategies are at very different locations in result space.

To test these hypotheses, equivalent measures had to be calculated from the experimental data to evaluate observed human strategies against the simulated result space. In contrast to the simulations, the experimental trajectories were not fully determined by initial values as online corrections were likely. Therefore, to attain better estimates of the execution variables from the experimental trajectories, estimates were extracted at each cycle k of the cup displacement x during each 40 sec trial (see Fig. 11); trial averages $\bar{A},\bar{f}, {\bar{\theta }_0},{\bar{\dot{\theta }}_0}$ served as correlates for the variables in the simulations. MI, MSF, and MSJ were calculated for each measured strategy ${\bar{A}_k},{\bar{f}_{k}}, {\bar{\theta }_k},{\bar{\dot{\theta }}_k}$.

Fig. 13

Result space with Mutual Information as the result variable, shown by shades of purple. The left panel plots trial data from all 9 subjects showing that they converge to the area with highest MI. Each data point is one trial; darker color shades denote later in practice. The arrows in the right panel show each subject with initial trial values the start of the arrow and the final practice trial the tip of the arrow (Reproduced from [41])

5.6 Predictable Interactions

An experimental study provided first evidence that subjects indeed favored predictable solutions over those that minimized the expended force and smoothness [41]. Subjects performed rhythmic cup movements paced at the natural frequency of the pendulum, which corresponded to the anti-resonance of the coupled system. This facilitated the emergence of the system’s nonlinear characteristics with chaotic solutions that maximized the challenge. Amplitude was free to choose and relative phase between ball and cup was also unspecified. Each subject performed 50 trials (40 s each). By choosing the cup amplitude and phase, subjects could manipulate interaction forces of different complexity and predictability.

The main experimental results are summarized in Fig. 13; the plot shows MI in shades of purple (lighter shades denote higher MI) and contours of selected values of MSF (green) from the simulations overlaid with the results from human subjects; each data point represents one trial (red). The data clearly show how subjects gravitated towards areas with higher MI, i.e. strategies with more predictable interactions, consistent with Hypothesis 1. The left panel shows individual trials pooled over all subjects; darker red indicates early practice and lighter red indicates late practice. The right panel shows the same data separated by subject: the red arrows mark how each subject’s average strategy changed from early practice (mean of first 5 trials) to late practice (mean of last 5 trials). The majority of subjects switched from low- to high-predictability regions in the result space. Both figures also show that all subjects increased their movement amplitude, associated with an increase in overall exerted force. None of the subjects moved toward the minimum force strategy, nor towards a strategy with maximum smoothness (counter to Hypotheses 2 and 3). In fact, overall force exerted, or MSF, rather increased with practice.

5.7 Interim Summary

These results highlight that humans are sensitive to object dynamics and favor strategies that make interactions predictable. In the case shown, these predictable solutions were even favored over those with less effort. This is plausible because unpredictable interaction forces are experienced as disturbances that continuously require reactions and corrections. Knowing that in real life we carry a glass of wine without paying much attention to the carrying, more predictable strategies appear plausible. Analogous to the dynamically stable solutions in ball bouncing, predictable solutions may require fewer computations as they obviate error corrections. Given that in chaotic solutions small changes due to external or internal perturbations lead to unpredictable behavior, noise matters less in predictable solutions.

6 From Analysis to Synthesis: Dynamic Primitives for Movement Generation

This brief overview of our research revealed potential control priorities or cost functions that humans may use to coordinate simple and complex interactions. Humans favor strategies that are sensitive to dynamics and stability, that exploit redundancy of the solution space to channel their intrinsic noise into task-irrelevant dimensions, and that exploit predictable solutions of potentially very complex task dynamics. The review also demonstrated what can be learnt from analysis of human data in conjunction with mathematical understanding of the task and its solution space. The only assumption is that the dynamics and stability properties of the task are fundamental and determine “opportunities” and “costs”. The known solution space serves as reference to evaluate human movement.

The task-dynamic approach as outlined is analytic and largely agnostic about details of the controller. This contrasts with other research in computational motor neuroscience that starts with a hypothesized controller and then compares the predicted with the experimentally observed behavior. One recent prominent example for this direction is work that has sought evidence that the brain operates like an optimal feedback controller [56, 73, 74]. Other control models include internal models with Kalman-filters or tapped-delay lines, to mention just a few [39]. Our approach refrains from such assumptions directly borrowed from control theory; rather, we aim to extract principles from human data with as few assumptions as possible. Nevertheless, the question of synthesis remains: what controller or control policy would generate these strategies? While still largely speculative, our task-dynamic perspective presents a sound foundation for a generative hypothesis.

To begin, let’s return to the initial pointer to the seemingly inferior features of the human neuromotor system - the high degree of noise and the slow information transmission. These features seem puzzling given the extraordinary dexterity of humans that by far surpasses that of robots, at least to date. Therefore, the direct translation of control policies that heavily rely on central control and feedback loops may remain inadequate to achieve human dexterity. As mentioned earlier, the human wetware with its compliant actuators and high dimensionality appears to provide an advantage. Hence, lower levels of the hierarchical neuromotor system should be given more responsibility. Consistent with our task-dynamic perspective, we have therefore suggested that the biological system generates movements via dynamic primitives, defined over the high-dimensional nonlinear neuromotor system [26, 28, 45, 50, 51, 59, 64]. We propose that the human neuromotor system exploits attractors states, defined over both the neural and mechanical nonlinear system. If the neuromotor system is parameterized to settle into such stable states, central control may only need to occasionally intervene. In principle, nonlinear autonomous systems have three possible stable attractor states: fixed point, limit cycle, and chaotic attractors. Putting chaotic attractors aside for now, we proposed fixed-point and limit cycle attractors for primitives.

The two main stable attractors fixed points and limit cycles directly map onto discrete and rhythmic movements. To understand discrete movements such as reaching to a target as convergence to a stable end state is not completely new. Equilibrium-point control was first posited by Feldman for simple position control [14, 15]. Numerous subsequent studies, both behavioral and neurophysiological, have given evidence for attractive properties in reaching behavior [4, 20, 25, 36]. This work has widened to include a virtual trajectory, even though details are still much contested. For rhythmic behavior a similar host of experimental and modeling studies have presented support for stable limit cycle dynamics. For example, bimanual rhythmic finger movements showed transitions from anti-phase to in-phase coordination that bear the hallmarks of nonlinear phase transitions in coupled nonlinear oscillators [22, 33]. Our own work has shown how extremely simple oscillator models can account for synchronization in bimanual rhythmic coordination, including subtle phase differences between oscillators with different natural frequencies [62, 70, 71]. Several different oscillator models have been developed that produce autonomous oscillations to represent central pattern generators in the spinal cord of invertebrates [31, 45]. Support for the distinction between rhythmic and discrete movements also came from a neuroimaging study [54]. Brain activation revealed that in rhythmic movements only primary motor areas were activated, while significantly more areas were needed to control discrete movements.

In an attempt to synthesize this evidence from largely disparate research groups, our own research made first forays into combining the two types of building blocks. Playing piano is after all a combination of complex rhythmic finger movements combined with reaches across the keyboard. Note that in principle, optimal feedback control could also achieve such movements, including those with dynamic stability. In fact, there is no inherent limit to what optimal feedback control may achieve. It is this omnipotence that contrasts with the well-known coordinative limitations that may reveal features of the human controller. Beyond “patting your head while rubbing your stomach”, research has revealed that rhythmic bimanual actions tends to settle into in-phase and anti-phase coordination [34, 71], humans avoid moving very slowly [3, 77], and the 2/3 power law in handwriting and drawing may reveal intrinsic geometry or other limitations [18, 52]. Several modeling and experimental studies showed the possibilities and limitations of combining two dynamic primitives. Wiping a table rhythmically, while translating the hand across the table revealed that rhythmic and discrete elements cannot be combined arbitrarily [63, 64].

However, research is still far from having generated conclusive evidence that dynamic motion primitives underlie observed behavior. More specifically, interactions with objects cannot be addressed with the two primitives alone. Therefore, recently Hogan and myself argued that impedance is needed as a third dynamic primitive to enable the system to interact with objects and the environment [28, 29]. Combining discrete and rhythmic primitives with impedance in an equivalent network is a first proposal on how humans may interact with objects in the environment. More details and first theoretical developments can be found in the chapter of Hogan in the same volume. With these theoretical efforts under way, also further complementary empirical work is needed. The challenge for the future is to combine analysis and synthesis. How can dynamic primitives be employed to pour—and enjoy—a glass of wine?