A systems-theoretic analysis of low-level human motor control: application to a single-joint arm model

Abstract

Continuous control using internal models appears to be quite straightforward explaining human motor control. However, it demands both, a high computational effort and a high model preciseness as the whole trajectory needs to be converted. Intermittent control shows great promise for avoiding these drawbacks of continuous control, at least to a certain extent. In this contribution, we study intermittency at the motoneuron level. We ask: how many different, but constant muscle stimulation sets are necessary to generate a stable movement for a specific motor task? Intermittent control, in our perspective, can be assumed only if the number of transitions is relatively small. As application case, a single-joint arm movement is considered. The muscle contraction dynamics is described by a Hill-type muscle model, for the muscle activation dynamics both Hatze’s and Zajac’s approach are considered. To actuate the lower arm, up to four muscle groups are implemented. A systems-theoretic approach is used to find the smallest number of transitions between constant stimulation sets. A method for a stability analysis of human motion is presented. A Lyapunov function candidate is specified. Thanks to sum-of-squares methods, the presented procedure is generally applicable and computationally feasible. The region-of-attraction of a transition point, and the number of transitions necessary to perform stable arm movements are estimated. The results support the intermittent control theory on this level of motor control, because only very few transitions are necessary.

1 Introduction

It has been shown, that intermittent commanding of multiple muscles, grouped together to synergies, could be one strategy of a higher level motor control scheme to reduce the overall control effort (Gawthrop et al. 2011; Karniel 2013). Intermittent control, in that respect, has the benefit that the input control signal is only calculated at certain points, the transition points, and not for the whole desired trajectory (Gawthrop et al. 2011; Karniel 2013).

In this work, we will be looking at the lowest level of motor control, the monosynaptic reflex loop. There, sensor signals coming from the muscle spindle are processed to motor commands. In the sense of control, motor commanding in the monosynaptic reflex loop can be seen as setting a target muscle fibre length and comparing it with the current muscle fibre length to produce a stimulation for the muscle. There exists already a control model for this kind of signal processing, the $\lambda$-model (Feldman 1986). Although the current muscle fibre length is continuously available as Ia feedback from the intrafusal fibres (the muscle spindles), the input target length can be set on the intrafusal fibres via $\gamma$-activation, intermittently. Depending on the weighted difference of both lengths, $\alpha$-activation of the extrafusal fibres (the main contracting skeletal muscle fibres) is triggered on the motoneuron. This way, a motion trajectory is the result of a stepwise switching between intermittent control inputs, a continuous feedback, a weighted comparison of input and feedback, and the underlying dynamics of the control system. The $\lambda$-model is a pure feedback model in which the whole set of target fibre lengths and the controller gains of the feedback control form an equilibrium point (EP). Therein, an EP represents a stable static equilibrium position of the motion system (Bayer et al. 2017).

Another motor control model on the monosynaptic reflex loop level is the $\alpha$-model (Bizzi et al. 1984). The $\alpha$-model is a pure feed forward model. Similar to the $\lambda$-model, motor commanding is done through the setting of EPs. However in this model, an EP is formed by a suitable constant (open-loop) stimulation for every muscle. Physiologically, the $\alpha$-activation of the extrafusal fibres has to be accompanied by simultaneous $\gamma$-activation, the $\alpha -\gamma$ co-activation, such that the weighted comparison of current and target muscle fibre length is actually switched off. In the perspective of the $\alpha$-model, a motion trajectory is the result of a stepwise switching between intermittent control input and the underlying dynamics of the control system.

As stated above, similar to higher levels of motor control, intermittency in the control can be identified on lower levels of motor control, too. Therefore, it seems worth to have a closer look on the control models on this level of motor control. Thus, the $\alpha$-model was chosen and combined with a muscle-driven, single-joint arm model. A finite set of EPs was generated. EPs were chosen to command a certain but simple movement. The smallest number of EPs to generate this movement was searched for. We hypothesize, that intermittent control on the lowest level of motor control can be assumed, if the number of EPs and therefore the number of transitions is small. In particular, we ask how many intermittent transitions of the input muscle stimulations, the $\alpha$-activation, are necessary to ensure a stable arm movement?

Indeed, it turns out that typical arm movements can be performed using only very few intermittent transitions, thus supporting the intermittent control hypothesis. The remainder of this paper is structured as follows. First, Sect. 2.1 addresses the description of human arm movements actuated by the intermittent control theory. The fundamental concepts of stability are presented in Sect. 2.2. A computationally tractable procedure based on sum-of-squares (SOS) methods to determine and enlarge the provable region-of-attraction (ROA) of an equilibrium point (EP) for nonlinear systems is introduced. Afterwards, the two fundamental concepts are combined to examine the stability of a human arm movement. The results of this combination are presented in Sect. 3. The overall conclusions and the future work are given in Sect. 4.

2 Methods

To test the hypothesis, a systems-theoretic approach is used to find the smallest number of transitions to generate a stable movement. Therefore, a single-joint arm movement is taken as test case. The arm dynamics is based on Kistemaker’s model (Kistemaker et al. 2006). The arm is actuated using four Hill-type muscle models consisting of four elements (Häufle et al. 2014; Bayer et al. 2017). The stability of arm movements has been already studied, elsewhere, e.g. (Giesl and Wagner 2007). In comparison to the mentioned study, we used a more detailed biophysical model of the human arm, namely (i) four lumped muscles to represent the monoarticular flexors (m. brachioradialis, m. brachialis, m. pronator teres, m. extensor carpi radialis), the biarticular flexors (m. biceps brachii caput longum and m. caput breve), the monoarticular extensors (m. triceps lateralis, m. triceps medialis, m. anconeus, m. extensor carpi ulnaris), and the biarticular extensors (m. triceps brachii caput longum) (Bayer et al. 2017). (ii) We used a different Hill-type muscle model including a visco-elastic tendon (Häufle et al. 2014), and (iii) we included two different representations of the activation dynamics (Zajac and Hatze). As described below, the complexity of the biophysical arm model had to be reduced to apply the presented approach. To analyze the stability of the arm movement, the ROA of the point, which defines the desired end position of the arm movement, is estimated. If the start position of the arm movement is already inside this ROA estimate, the whole movement can be performed without any additional intermittent transition. If this is not the case, an additional point in between the start and end position and inside the ROA estimate is added. Further points are added until the whole movement is contained in overlapping ROA estimates of the used intermittent points. To estimate a ROA, a computationally tractable procedure based on SOS methods is applied.

2.1 Fundamentals of human movement

A single-joint arm movement is chosen as the application case. In this section, the corresponding system model is derived. As explained above, a Lyapunov function candidate needs to be found to estimate the ROA of a given position. This will be done by SOS techniques, i.e., a polynomial Lyapunov function is found (Topcu et al. 2010). Hence, the system is expanded as a Taylor series at the considered position, which is an EP of the system model. Therefore, the developed model needs to be sufficiently smooth.

2.1.1 Musculoskeletal system

Skeletal System The used arm model is taken from Kistemaker et al. (2006) and illustrated in the “Appendix A”. The whole arm is abducted to $90^\circ$ from the upper torso. The shoulder angle $\psi$ is fixed at $\psi = 45^\circ$. The movement is described by the elbow angle $\varphi$, which is the only degree-of-freedom. The dynamics of the arm is described by the acceleration of the elbow angle $\varphi$ as

$$\begin{aligned} \ddot{\varphi }=\frac{M^{\text {net}}}{J}, \end{aligned}$$

(1)

where J is the inertia of the lower arm shifted into the elbow joint by the parallel axis theorem. The net moment was calculated assuming linear muscle moment arms (see Table 4 in the appendix and Bayer et al. (2017), Section 2.2). Further details such as the calculation of the net elbow joint moment $M^{\text {net}}$ and used parameters can be found in Kistemaker et al. (2006) and Bayer et al. (2017).

Muscle Model The lower arm is actuated by up to four lumped muscle groups. Details on the muscle groups can be found in Bayer et al. (2017). The muscle contraction dynamics of each muscle group is described by a Hill-type muscle model. The model is composed of an active contractile element (CE), a passive parallel elastic element (PEE), a serial elastic element (SEE) and a serial damping element (SDE) (Häufle et al. 2014). The force of the CE depends on the activity a, the muscle fibres’ length $l^{\text {CE}}$ and the velocity ${\dot{l}}^{\text {CE}}$. The dependencies are represented as

$$\begin{aligned} F^{\text {CE}}(a,l^{\text {CE}},{\dot{l}}^{\text {CE}})=a \, F^{\text {isom}}(l^{\text {CE}}) F^{\text {v}}({\dot{l}}^{\text {CE}})F^{\text {max}}, \end{aligned}$$

(2)

see (Häufle et al. 2010). The muscle-specific parameter $F^{\text {max}}$ is extracted from Bayer et al. (2017). For the length-dependent force $F^{\text {isom}}$ the parabolic function proposed by Kistemaker et al. (2006) is implemented. The corresponding parameters are taken from the work of Bayer et al. (2017).

The force–velocity relation $F^{\text {v}}$ is modeled by a hyperbolic function in the paper of Häufle et al. (2014). At the EP ${\dot{l}}^{\text {CE}}=0$, the function is non-differentiable and reaches the value 1. However, the applied ROA-estimating process is based on a Taylor series expansion of the model in the EP as mentioned in the beginning of this section, and hence the dynamics needs to be differentiable. In the current work, the stability of a movement in the normal operation range of the muscle fibres shall be analyzed. Hence, the movement from $\varphi =1$ rad ($\approx 57^\circ$) to $\varphi =1.8$ rad ($\approx 103^\circ$) is suitable for an analysis. A linear representation

$$\begin{aligned} F^{\text {v}}({\dot{l}}^{\text {CE}})=1+\mu ^{\text {v}} {\dot{l}}^{\text {CE}} \end{aligned}$$

(3)

promises to be a good approximation in this movement range. The muscle-specific parameter $\mu ^{\text {v}}$ is calculated with the help of the maximum contraction velocity

$$\begin{aligned} \begin{aligned} {\dot{l}}^{\text {CE,max}} =&\arg \max _{{\dot{l}}^{\text {CE}}} |{\dot{l}}^{\text {CE}}| \\&\text {s.t. } 0 \le \varphi \le 1.8 \,\text { (rad)}. \end{aligned} \end{aligned}$$

(4)

Note that the parameter ${\dot{l}}^{\text {CE,max}}$ is negative in the case of flexion. Finally, the parameter $\mu ^{\text {v}}$ is specified with the relation

$$\begin{aligned} F^{\text {v}}({\dot{l}}^{\text {CE,max}}) \overset{!}{=} 1+\mu ^{\text {v}} {\dot{l}}^{\text {CE,max}}. \end{aligned}$$

(5)

The values of the muscle-specific parameter $\mu ^{\text {v}}$ are listed in Table 1. With this approach, the damping effect of the CE is underestimated. Hence, if stability with the linear force–velocity relation (3) can be proven, it seems reasonable that the system with the more complex force–velocity relation of Häufle et al. (2014) is also stable. This is due to the fact, that by introducing a small damping into the muscle model in parallel to the serial elastic element, oscillation where significantly reduced (Günther et al. 2007).

Table 1 Muscle-specific parameter $\mu ^{\text {v}}$

In the case of the PEE, the force $F^{\text {PEE}}=0$ lies below the slack length $l^{\text {PEE,}0}$. Above the length $l^{\text {PEE,}0}$ a nonlinear behavior is assumed (Rockenfeller 2016). Here, the operating range of the muscles is yet below the slack length $l^{\text {PEE,}0}$ during the whole considered movement. Hence, the force of the PEE can be neglected.

The elastic properties of the tendon are modeled by a differentiable piecewise polynomial of Kistemaker et al. (2006). The relation and the corresponding values of the SEE can be found in Bayer et al. (2017).

The SDE is modeled as a viscous damper-like force. Mörl et al. (2012) models the damping coefficient d force-depending. During the considered movement, the operating length of the muscle fibres $l^{\text {CE}}$ is next to the muscle-specific value $l^{\text {CE,opt}}$. With this knowledge, the force is around its optimal value $F^{\text {max}}$ and the damping coefficient d can be assumed to take the constant value

$$\begin{aligned} d = d_{\text {SDE,max}}. \end{aligned}$$

(6)

The value of the maximum damping coefficient $d_{\text {SDE,max}}$ can be found in Mörl et al. (2012).

The contraction dynamics models the velocity of the CE ${\dot{l}}^{CE}$. The dynamics is derived from the force equilibrium (Bayer et al. 2017) as

$$\begin{aligned} {\dot{l}}^{\text {CE}} = \frac{-a\, F^{\text {isom}}(l^{\text {CE}}) F^{\text {max}}+F^{\text {SEE}}(l^{\text {CE}},\varphi )+d \, {\dot{l}}^{\text {MTU}}({\dot{\varphi }}, \varphi )}{a\, F^{\text {isom}}(l^{\text {CE}})\mu ^{\text {v}} \, F^{\text {max}}+d}, \end{aligned}$$

(7)

see Table 4 for more details. In conclusion, the derived model (7) has a smooth right-hand-side in the EP. Hence, a Taylor series expansion around the EP is possible and SOS methods can be applied.

2.1.2 Activation dynamics

The activation a of the muscles is not instantaneous. The activation dynamics describes the change in the activity a as a response of the neuronal excitation $0\le u \le 1$ by the neural system. In the current work, two different formulations are analyzed, namely the activation dynamics of Zajac (1989) and Hatze (1977). The corresponding equations and parameters can be found in Bayer et al. (2017) in compact form. Herein, the neural stimulation signal u is related to the muscle activity a in two steps in the case of Hatze’s formulation. First, the free calcium ion concentration $\gamma$ is related to the neuronal stimulation u. Next, the activity a is calculated depending on the concentration $\gamma$. Bayer et al. (2017) shows that for high muscle stimulation, the time of increase in activity a is shorter in Hatze’s activation dynamics than in Zajac’s. The time of decrease in activity is longer in the case of Hatze than in Zajac’s. At low stimulation levels around $u=0.1$, the inverse effect can be observed.

The whole system dynamics is summarized in “Appendix A”.

2.1.3 Neural control: equilibrium point theory

The neural controller transforms a movement goal $\varphi = \varphi _{\text {d}}$ into electromechanical output signals u. In the current work, the muscle stimulation u is introduced to the system dynamics by an open-loop $\alpha$-controller (Polit and Bizzi 1979; Bizzi et al. 1984; Kistemaker et al. 2006). For a given desired angle $\varphi$, infinitely many open-loop stimulation combinations exist, which satisfy the EP constraint

$$\begin{aligned} \begin{aligned}&\qquad \left( \begin{array}{c} {\dot{a}}\\ {\dot{l}}^{\text {CE}}\\ \ddot{\varphi }\\ {\dot{\varphi }} \end{array}\right) = \left( \begin{array}{c} 0\\ 0\\ 0\\ 0 \end{array}\right) . \end{aligned} \end{aligned}$$

(8)

The controlled variable is an EP of the arm model having one degree-of-freedom resulting from the interaction of agonist and antagonist muscles. Besides, the conditions $\varphi = \varphi _d$ and $a_0 \le a \le 1$, where $a_0$ is the minimum activity, must be fulfilled. To resolve the indeterminacy of the possible inputs u which all satisfy these constraints, an appropriate cost function is applied. The optimal stimulation combination minimizes this cost function, here the maximum low-frequency stiffness $k_{\text {lf}}=\partial M^{\text {net}}/\partial \varphi$ (Kistemaker et al. 2006).

To solve the optimization problem, the MATLAB function “fmincon” with the interior-point method is used.

2.2 Stability analysis

We now describe the methods that were employed to obtain an estimate of the ROA of a given EP of the previously derived model. The standard approach for stability analysis of a general nonlinear system is to find a Lyapunov function that satisfies a list of conditions, called the direct method of Lyapunov (Khalil 1996). The class of systems to be examined is typically described by a set of general nonlinear ordinary differential equations (ODEs)

$$\begin{aligned} \dot{\mathbf {x}}(t)=f({\mathbf {x}}(t)), \quad {\mathbf {x}}(0) = {\mathbf {x}}_0, \end{aligned}$$

(9)

where ${\mathbf {x}}(t)\in {\mathbb {R}}^n$ denotes the vector of state variables at time t, and the system dynamics $f:{\mathbb {R}}^n \rightarrow {\mathbb {R}}^n$ are assumed to be locally Lipschitz continuous. Without loss of generality, in the following we consider the equilibrium ${\mathbf {x}}_s=0$ of the ODEs (9). Lyapunov’s direct method uses a continuously differentiable function $V:{\mathbb {R}}^n \rightarrow {\mathbb {R}}$ with

$$\begin{aligned} V({\mathbf {0}})=0 \quad \text {and} \quad V({\mathbf {x}}) > 0 \quad \text {for all }{\mathbf {x}} \ne {\mathbf {0}} \end{aligned}$$

(10)

in order to characterize local asymptotic stability of the equilibrium ${\mathbf {x}}_s=0$. Namely, assume that

$$\begin{aligned} {\dot{V}}({\mathbf {x}}):=\frac{\partial V}{\partial {\mathbf {x}}} f({\mathbf {x}}) < 0 \quad \text {for all } {\mathbf {x}} \in {\mathcal {N}}\backslash \{ {\mathbf {0}}\} \end{aligned}$$

(11)

is satisfied in a neighborhood ${\mathcal {N}}$ of the origin. Then, the EP ${\mathbf {x}}_s={\mathbf {0}}$ is locally asymptotically stable and any bounded sublevel set of V,

$$\begin{aligned} \varOmega _{V,r} := \{ {\mathbf {x}} \in {\mathbb {R}}^n |V({\mathbf {x}}) \le r \}, \quad r > 0, \end{aligned}$$

(12)

which is contained in ${\mathcal {N}}$, is an invariant subset of the ROA. In the following, we apply two methods to maximize the volume of the set $\varOmega _{V,r}$ in order to obtain a large ROA estimate. Determining a Lyapunov function V can be a difficult task, and no systematic procedure exists to this end for general nonlinear systems. On the other hand, such methods exist in case of linear or polynomial systems, the latter based on SOS techniques (Topcu et al. 2010; Papachristodoulou and Prajna 2002; Hachicho and Tibken 2002). The two procedures described in the following are based on (low-order) Taylor series approximations of the nonlinear system (9) in order to be able to exploit these methods; to this end, in the following we assume that f is sufficiently smooth.

2.2.1 Region-of-attraction estimation based on linearization: method A

Step 1

Let the matrix ${\mathbf {A}}:=\partial f/\partial {\mathbf {x}}|_{{\mathbf {x}}={\mathbf {x}}_{\text {s}}}$ be the linearization of system (9) at the EP ${\mathbf {x}}_{\text {s}}$. If the matrix ${\mathbf {A}}$ is Hurwitz (i.e., all eigenvalues have negative real part), then for each symmetric and positive definite matrix ${\mathbf {Q}}$ there exists a (unique) symmetric and positive definite matrix ${\mathbf {P}}$ that satisfies the Lyapunov equation

$$\begin{aligned} {\mathbf {A}}^T{\mathbf {P}}+{\mathbf {P}}{\mathbf {A}}=-{\mathbf {Q}}, \end{aligned}$$

(13)

see, e.g., (Khalil 1996). Choosing $V({\mathbf {x}})={\mathbf {x}}^T{\mathbf {P}}{\mathbf {x}}$ results in the fact that for the linearized system, conditions (10) and (11) are satisfied globally, i.e., (11) is satisfied with $f({\mathbf {x}})$ replaced by ${\mathbf {A}}{\mathbf {x}}$ and ${\mathcal {N}}={\mathbb {R}}^n$.

In order to obtain a large ROA for the (original) nonlinear system (9), instead of solving (13) for some fixed ${\mathbf {Q}}$ the following optimization problem has been solved:

$$\begin{aligned} \begin{aligned} \min _{\epsilon _2, {\mathbf {P}}} \epsilon _2\\ \text {s.t.} \quad&\epsilon _2 {\mathbf {I}}-{\mathbf {P}} \ge 0, \\&{\mathbf {P}} \ge {\mathbf {I}},\\&{\mathbf {P}}{\mathbf {A}}+{\mathbf {A}}^T {\mathbf {P}} \le - \epsilon _1 {\mathbf {P}}. \end{aligned} \end{aligned}$$

(14)

For $0<\epsilon _1<-2\max _{i}{Re(\lambda _i({\mathbf {A}}))}$ with $\lambda _i({\mathbf {A}})$ denoting the eigenvalues of ${\mathbf {A}}$, (14) has a feasible solution. For the computations in the following chapter, we set $\epsilon _1 = 10^{-2}$, and problem (14) has been solved with the MATLAB toolbox YALMIP (Löfberg 2004) using the solver MOSEK (MOSEK 2015). The solution ${\mathbf {P}}$ to problem (14) is such that the eigenvalues of the matrix ${\mathbf {P}}$ are in the range $[1, \, \epsilon _2]$. By minimizing over $\epsilon _2$, one ensures that the eigenvalues of ${\mathbf {P}}$ are as close together as possible, i.e., the condition number of ${\mathbf {P}}$ is as small as possible. This results in the fact that the semiaxes of the ellipsoidal ROA estimate $\varOmega _{V,r}$ defined by (12) have similar magnitude.¹ Similar as discussed above, choosing $V({\mathbf {x}})={\mathbf {x}}^T{\mathbf {P}}{\mathbf {x}}$ results in the fact that for the linearized system, conditions (10) and (11) are satisfied globally.

Step 2

In Step 1, a Lyapunov function V satisfying conditions (10) and (11) for the linearized system dynamics has been determined. Since $f({\mathbf {x}})-A{\mathbf {x}} = {\mathcal {O}}(\Vert {\mathbf {x}}\Vert ^2)$, (11) is also satisfied in a (small enough) neighborhood ${\mathcal {N}}$ of the origin for the original nonlinear system (9), and hence one can determine $r>0$ such that $\varOmega _{V,r}\subseteq {\mathcal {N}}$ is a ROA, which can be done as follows. First, the parameter r for the ROA estimate of the original system is initialized with some $r_{\text {init}}>0$. Second, the optimization problem

$$\begin{aligned} \begin{aligned} \max _{{\mathbf {x}}} \frac{\partial V}{\partial {\mathbf {x}}} f({\mathbf {x}}) \\ \text {s.t.} \quad&V({\mathbf {x}}) \le r \end{aligned} \end{aligned}$$

(15)

is solved. If the optimal value of this optimization problem is positive, the parameter r is reduced until the (unique) maximizer is at the origin and hence $\frac{\partial V}{\partial {\mathbf {x}}} f({\mathbf {x}})<0$ for all ${\mathbf {x}} \in \varOmega _{V,r} \backslash \{ {\mathbf {0}}\}$. Note that a similar procedure can be found in Chen and Allgöwer (1998) in the context of model predictive control. For the computations in Sect. 3, Problem (15) is solved using the MATLAB function “fmincon”. Due to the nonconvexity of problem (15), in general only a local maximum is attained. Hence different initial conditions ${\mathbf {x}}_0$ have to be supplied to the solver in order to ensure that all local maxima are found and hence indeed $\frac{\partial V}{\partial {\mathbf {x}}} f({\mathbf {x}})<0$ for all ${\mathbf {x}} \in \varOmega _{V,r} \backslash \{ {\mathbf {0}}\}$.

The presented procedure to maximize the ROA estimate is based on the linearization at the EP ${\mathbf {x}}_s$, see Step 1. Hence, the size of the ROA estimate in general highly depends on the nonlinearity of the system. If system (9) is highly nonlinear, applying a procedure on the basis of linearization as the presented one can result in a rather small ROA estimate. In this case, the following approach based on higher-order Taylor series approximations can be beneficial.

2.2.2 Region-of-attraction estimation based on Taylor series: method B

Step 1.

As a first step, a Taylor series expansion of order $N\in {\mathbb {N}}$ of the system dynamics f at the equilibrium point ${\mathbf {x}}_s$, denoted by $T_{N}f({\mathbf {x}})$, is calculated. Clearly, using higher order terms allows for a better approximation of the nonlinear system; on the other hand, the optimization problem below might become intractable when using approximations of too high order. In Sect. 3, we use Taylor series expansions of order two and three.

Step 2.

In order to determine a Lyapunov function candidate V for the Taylor series expansion $T_{N}f({\mathbf {x}})$, we first reformulate the decrease condition (11) using the so-called Generalized S-Procedure (Boyd et al. 1994; Topcu et al. 2010). Namely, one can show that $\frac{\partial V}{\partial {\mathbf {x}}} T_{N}f({\mathbf {x}}) < 0$ for all ${\mathbf {x}} \in {\mathcal {N}}\backslash \{ {\mathbf {0}}\}$ with ${\mathcal {N}}$ given by ${\mathcal {N}}=\{{\mathbf {x}}|{\mathbf {x}}^T{\mathbf {P}}{\mathbf {x}}\le c\}$ for some positive definite matrix ${\mathbf {P}}>0$ and some constant $c>0$, if there exists a positive definite function $l: {\mathbb {R}}^n \rightarrow {\mathbb {R}}$ and a positive semidefinite function $s:{\mathbb {R}}^n \rightarrow {\mathbb {R}}$ such that the following inequality is satisfied for all ${\mathbf {x}} \in {\mathbb {R}}^n$:

$$\begin{aligned} -\left( l({\mathbf {x}}) + \frac{\partial V}{\partial {\mathbf {x}}} T_{N}f({\mathbf {x}}) \right) + s({\mathbf {x}})\left( {\mathbf {x}}^T{\mathbf {P}}{\mathbf {x}} - c \right) \ge 0. \end{aligned}$$

(16)

In Sect. 3, we use the matrix ${\mathbf {P}}$ computed via Method A (see Sect. 2.2.1), and l is fixed to $l({\mathbf {x}})=10^{-6}{\mathbf {x}}^T{\mathbf {x}}$.

Using the above idea, one can formulate the following optimization problem in order to find a Lyapunov function candidate V for the Taylor series expansion $T_{N}f({\mathbf {x}})$:

$$\begin{aligned}&\displaystyle \min _{\epsilon _2,V \in {\mathbb {P}}_2,\, s \in {\mathbb {P}}_m} \epsilon _2 \end{aligned}$$

(17a)

$$\begin{aligned}&\displaystyle \text {s.t.} \quad \epsilon _2 \Vert {\mathbf {x}} \Vert ^2 \ge V({\mathbf {x}}) \ge \epsilon _1 \Vert {\mathbf {x}} \Vert ^2 \end{aligned}$$

(17b)

$$\begin{aligned}&\displaystyle s({\mathbf {x}}) \in \varSigma [{\mathbf {x}}], \end{aligned}$$

(17c)

$$\begin{aligned}&\displaystyle -\left( l({\mathbf {x}}) + \frac{\partial V}{\partial {\mathbf {x}}} T_{N}f({\mathbf {x}}) \right) + s({\mathbf {x}})\left( {\mathbf {x}}^T {\mathbf {P}} {\mathbf {x}} -c \right) \in \varSigma [{\mathbf {x}}]. \end{aligned}$$

(17d)

Here, the set ${\mathbb {P}}_n$ denotes the set of polynomials in ${\mathbf {x}}$ of a fixed degree n. In the above formulation, for computational tractability we restrict V to be a quadratic function (i.e., $V \in {\mathbb {P}}_2$); alternatively, one can also search for higher order Lyapunov functions. Furthermore, $\varSigma [{\mathbf {x}}]$ denotes the class of SOS ploynomials (Topcu et al. 2010; Papachristodoulou and Prajna 2002; Hachicho and Tibken 2002). Requiring s in (17c) and the left hand side of (17d) to be SOS is a sufficient condition for nonnegativity of these polynomials. In particular, (17d) implies that (16) is satisfied. The advantage of SOS problem formulations such as (17) is that they can be solved by reformulating them as semidefinite program (SDP), which in turn are solved efficiently, e.g., using interior-point methods. Afterwards, the SDP solution is converted back to the solution of the original SOS problem. To this end, for the computations in Sect. 3, the MATLAB toolbox YALMIP (Löfberg 2004) and the interior-point solver in MOSEK (MOSEK 2015) have been used.

Constraint (17b) (where $\epsilon _1>0$ is some fixed parameter) ensures that V is positive definite, i.e., (10) is satisfied. As in Sect. 2.2.1, minimizing $\epsilon _2$ again ensures that the semiaxes of the ellipsoidal ROA estimate $\varOmega _{V,r}$ defined by (12) (see Step 3 below) have similar magnitude. In order to obtain a large ROA, the set ${\mathcal {N}}=\{{\mathbf {x}}|{\mathbf {x}}^TP{\mathbf {x}}\le c\}$ should be as large as possible, i.e., c should be as large as possible. If c were an additional optimization variable, constraint (17d) would contain a bilinearity in the optimization variables and hence the optimization problem could not be reformulated as an efficiently solvable SDP. Hence, we (iteratively) solve problem (17) for fixed c, possibly increasing c (for a larger region ${\mathcal {N}}$) or decreasing c (in order to obtain a feasible solution to problem (17)). For the computations in Sect. 3, we use $c=1$.

In summary, if Problem (17) admits a feasible solution, conditions (10) and (11) are satisfied for the Taylor series expansion of order N, i.e., with $f({\mathbf {x}})$ replaced by $T_{N}f({\mathbf {x}})$ in (11).

Step 3.

In Step 1, a Lyapunov function V satisfying conditions (10) and (11) for the Taylor series expansion of order N, $T_{N}f({\mathbf {x}})$, has been determined. Since $f({\mathbf {x}})-T_{N}f({\mathbf {x}}) = {\mathcal {O}}(\Vert {\mathbf {x}}\Vert ^{N+1})$, one can again proceed as in Step 2 of “Method A” to determine a ROA estimate $\varOmega _{V,r}$ for the original nonlinear system (9).

3 Results

First, the “Method A” is applied to analyze an arm trajectory. In doing so, it is tested how many EPs are necessary for the trajectory from $\varphi _{\text {start}}=0.52$ rad to $\varphi _{\text {d}}=1.8$ rad. During this movement, the muscle fibres are in between the normal operating range, which is the focus of the current paper. As the biarticular muscles cross multiple joints, they are able to contribute to flexion and extension simultaneously. Consequently, the biarticular muscles are supposed to have a comparatively smaller influence on the movement than the monoarticular muscles. Hence, as a starting point, we examine a model actuated only by the two monoarticular muscle groups. In this case, the system is described by six (instead of ten) equations. The muscles are activated by Hatze’s activation dynamics. Figure 1 illustrates the EPs and its corresponding ROA estimates for the whole movement projected into the $\varphi \,{\dot{\varphi }}$-plane. To cover the whole trajectory by overlapping ROA estimates, 4 EPs are necessary in between the start position $\varphi _{\text {start}}$ and end position $\varphi _{\text {d}}$. In doing so, each EP is inside the ROA estimate of the following EP.

The distance between the EPs is larger in the beginning of the movement at the elbow angle $\varphi =0.52$ rad. The smaller the elbow angle $\varphi$, the larger is the guaranteed ROA estimate. This result is in agreement with the work of Giesl and Wagner (2007).

Fig. 1

EPs and the projection of the corresponding ROA estimates to generate a stable movement from $\varphi =0.52$ rad to $\varphi =1.8$ rad with two muscle groups. The ROA estimates are shown as a projection on the $\varphi -{\dot{\varphi }}$ plane, i.e., with the other four states fixed to their respective equilibrium values. The projection of the ROA estimate corresponds to: $\varphi =0.88$ rad and $\varphi =1.8$ rad (both solid lines); $\varphi =1.2$ rad (dotted line); $\varphi =1.46$ rad (dash-dotted line); $\varphi =1.663$ rad (dashed line). The corresponding activation levels of the respective muscle group per each elbow angle $\varphi$ (rad) are: $\varphi =0.52$: $\gamma _1=0.132$, $\gamma _2=0.558$; $\varphi =0.88$: $\gamma _1=0.1667$, $\gamma _2=0.5401$; $\varphi =1.2$: $\gamma _1=0.2073$, $\gamma _2=0.5401$; $\varphi =1.46$: $\gamma _1=0.2524$, $\gamma _2=0.5401$; $\varphi =1.663$: $\gamma _1=0.3039$, $\gamma _2=0.5401$; $\varphi =1.8$: $\gamma _1=0.3532$, $\gamma _2=0.5316$

The “Method A” is based on the linearization at the EPs, resulting in possibly conservative ROA estimates. This is even more the case if we apply this method to the full model including all four muscle groups. Thus, in the following we use higher-order Taylor series approximations within “Method B” to obtain less conservative ROA estimates. We also investigate the influence of the different activation dynamics on the stability and estimate the ROAs for an arm model activated by both Hatze’s and Zajac’s activation dynamics. In particular, we examine how many EPs are at most necessary to generate a stable arm movement from $\varphi _{\text {start}}=1$ rad to $\varphi _{\text {d}}=1.8$ rad with both Hatze’s and Zajac’s activation dynamics.

First, the arm model actuated by four lumped muscle groups, which are activated by Hatze’s activation dynamics, is analyzed. The third order Taylor series $T_{3}f({\mathbf {x}})$ is chosen for “Step 1”. Figure 2b illustrates the projection of the ROA estimate of the EP ${\mathbf {x}}_{\text {d}}$. The start position ${\mathbf {x}}_{\text {start}}$ is not inside the ROA estimate. Hence, a further EP in between the EP ${\mathbf {x}}_{\text {start}}$ and the EP ${\mathbf {x}}_{\text {d}}$ is inserted. The EP ${\mathbf {x}}_{\text {mid}}=[0.2254, 0.7475, 0.7119, 0.6704, 0.0749, 0.0713, 0.0976, 0.0846, 0, 1.45]^T$ is inside the ROA estimate of the EP ${\mathbf {x}}_{\text {d}}$. The ROA estimate of the EP ${\mathbf {x}}_{\text {mid}}$ is illustrated in Fig. 2a. The EP ${\mathbf {x}}_{\text {start}}$ is inside this ROA estimate. Hence, a stable arm movement is possible from $\varphi =1.0$ rad to $\varphi =1.8$ rad with only one EP in between. This result supports the intermittent control theory, showing that a rather large arm movement is possible with only one intermittend control value. The result also suggests that smaller elbow angles have larger ROAs as the distance between the EP ${\mathbf {x}}_{\text {start}}$ and the EP ${\mathbf {x}}_{\text {mid}}$ is larger than between the EP ${\mathbf {x}}_{\text {mid}}$ and the EP ${\mathbf {x}}_{\text {d}}$.

Fig. 2

Projection of the ROA estimate of the EP: a${\mathbf {x}}_{\text {mid}}$; b${\mathbf {x}}_{\text {d}}$. The ROA estimates are shown as a projection on the $\varphi -{\dot{\varphi }}$ plane, i.e., with the other eight states fixed to their respective equilibrium values. The muscles are activated by Hatze’s activation dynamics. The corresponding activation levels of the respective muscle group per each elbow angle $\varphi$ (rad) are: $\varphi =1.0$: $\gamma _1=0.1149$, $\gamma _2=0.8433$, $\gamma _3=0.9226$, $\gamma _4=0.0048$; $\varphi =1.45$: $\gamma _1=0.2254$, $\gamma _2=0.7475$, $\gamma _3=0.7119$, $\gamma _4=0.6704$; $\varphi =1.8$: $\gamma _1=0.3364$, $\gamma _2=0.5220$, $\gamma _3=0.4698$, $\gamma _4=0.5132$

Second, the arm model with four muscle groups is activated by Zajac’s activation dynamics. In contrast to the system with Hatze’s activation dynamics, the second order Taylor series $T_{2}f({\mathbf {x}})$ is chosen for “Step 1”. Zajac’s activation dynamics is a linear function in contrast to the strongly nonlinear dynamics of Hatze’s activation dynamics. This suggests a good approximation already with a second order Taylor series. Figure 3 illustrates the ROA estimate of the EP ${\mathbf {x}}_{\text {d}}$. The whole trajectory from $\varphi _{\text {start}}=1$ rad to $\varphi _{\text {d}}=1.8$ rad can be shown to be inside this ROA estimate. This result supports the reasonability of the intermittent control theory. In combination with the low-frequency stiffness, the arm movement modeled with Zajac’s activation dynamics is slower than in the case of Hatze’s activation dynamics. Hence, the bigger ROA estimate suggests that slower arm movements are more stable. In “Appendix B”, a sensitivity analysis for this scenario is provided, where we vary different model parameters. The results show that the size of the ROA only slightly changes (i.e., shows a rather low sensitivity to parameter variations), thus supporting the above discussed findings.

Fig. 3

The projection of the ROA estimate for the EP ${\mathbf {x}}_{\text {d}}$. The ROA estimate is shown as a projection on the $\varphi -{\dot{\varphi }}$ plane, i.e., with the other eight states fixed to their respective equilibrium values. The muscles are activated by Zajac’s activation dynamics. The corresponding activation levels of the respective muscle group per each elbow angle $\varphi$ (rad) are: $\varphi =1.0$: $a_1=0.1151$, $a_2=0.8183$, $a_3=0.8879$, $a_4=0.0450$; $\varphi =1.8$: $a_1=0.3435$, $a_2=0.4164$, $a_3=0.3860$, $a_4=0.6882$

4 Discussion and conclusions

In recent years, intermittent control has gained growing attention in human motor control (Gawthrop et al. 2011; Karniel 2013). Especially, the minimum transition hypothesis has been shown to reduce the control effort of higher levels of motor control by using muscle synergies (Karniel 2013). Complementary, the present study tries to shed light on the lowest level of motor control in order to find hints to intermittent control, too.

Indeed, we have shown, that a well established model of motor control, the $\alpha$-model (Bizzi et al. 1984), was able to generate a simple, one dimensional, stable arm movement using only a very small number of EPs. To analyze the stability of the arm movement, the region-of-attraction (ROA) of the EP, which defines the desired end position of the arm movement, was estimated. If the start position of the arm movement is already inside this ROA estimate, the whole movement is asymptotically stable without an intermittent EP. If this was not the case, an additional EP in between the start and end position and inside the ROA estimate was added. Further points were added until the whole movement consists of overlapping ROA estimates of the used positions. To estimate a ROA, a computationally tractable procedure based on sum-of-squares (SOS) methods was applied. As the number of necessary EPs was always very small, we infer from our findings, that the systems-theoretic analysis of simple arm movements revealed hints to intermittent control on low-level human motor control.

The limitation of the study is the very reduced biophysical model of the human arm. Although we had already chosen a macroscopic model of a biological skeletal muscle, see (Häufle et al. 2014), we had to reduce the complexity even further (see Sect. 2.1). The modifications made in the formulation of the contraction dynamics would affect especially high velocity and cyclic movements. In contrast, the pointing task, which was studied in this work, does not pose a specific demand in high velocity. Therefore, for the case of this study, the assumptions are valid. Likewise, the pointing task was limited to a certain operation range for which the validity of the reduced model can be assumed, too. In comparison with the Haeufle et al. model (2014), our model was in good agreement with the forces and the muscle fibres lengths in the studied operating range of the arm. More importantly, the number of EPs to generate a movement using our reduced model did not differ from the full model used in Bayer et al. (2017). Only one EP in between the starting and the end position was necessary for the stable movement from $\varphi =1.0$ rad to $\varphi =1.8$ rad with Hatze activation. Using Zajac activation, no EP in between was necessary. The motion range was arbitrarily chosen, in order to show stable movement in a considerable range of elbow angle and with the least amount of intermediate EPs. For us, this proves that, although we could not apply our systems-theoretic analysis to the full model used in Bayer et al. (2017) because of its internal complexity, the quantity of motor commands did not differ. That is, the criteria of intermittency holds up.

The EP hypothesis as formulated by Feldman and co-workers, see for example (Feldman 1986), is still under debate. It remains unclear, whether the biological system uses this approach to produce, control and store movement. In this contribution, we will not take on this ongoing discussion. Rather, we re-interpret the idea of EPs as characteristic states in the biological motion, which could potentially be used for movement generation, control and storage. The benefits seem obvious: (i) The storage effort is very low, since only a few points have to be stored to perform a movement. (ii) The control effort is very low, since the input stimulation does not change very often and therefore the information flow is low (Haeufle et al. 2014). (iii) External disturbances do not necessarily lead to a different input command and might therefore be better tolerated. In all, with this contribution we could show, that an EP, as defined and used in literature, can be found by a systems-theoretic approach. By using only such EPs a stable movement was produced. Additionally, we found that smaller elbow angles and slower movements turned out to be more stable. It remains open, whether and how the biological system performs such stability analysis to generate EPs. At least for the case of numerical simulations, this approach proved to be practical to generate simple movements using many muscles (Rupp et al. 2015) and fast movements (Bayer et al. 2017).

4.1 Future developments

In this work, for simplicity we have considered the $\alpha$-model, which is a pure feed forward control model. Future work will consist of examining stability properties of the (feedback) $\lambda$-model, as well as a combination of feedforward and feedback control. Furthermore, it would be interesting to apply the presented approach to more sophisticated human motion patterns, including, e.g., multiple joints. While the proposed procedure is—at least in theory—readily applicable to such settings, the available methods based on SOS techniques are typically only computationally tractable in cases where the system dimension is not too high. Hence the development of novel concepts and solvers for SOS problems and semidefinite programs for large scale systems, maybe based on distributed optimization, would be worthwhile.

Appendices

Appendix A

Fig. 4

Arm model with one degree-of-freedom at the elbow joint

In the case of Hatze’s activation dynamics and four muscle groups, the complete system dynamics is given by

$$\begin{aligned} \left( \begin{array}{c} {\dot{\gamma }}_1\\ {\dot{\gamma }}_2\\ {\dot{\gamma }}_3\\ {\dot{\gamma }}_4\\ {\dot{l}}^{\text {CE}}_1\\ {\dot{l}}^{\text {CE}}_2\\ {\dot{l}}^{\text {CE}}_3\\ {\dot{l}}^{\text {CE}}_4\\ {\dot{\varphi }}_{\text {h}}\\ {\dot{\varphi }} \end{array} \right) = \left( \begin{array}{c} m_{\text {a}}(u_1-\gamma _1)\\ m_{\text {a}}(u_2-\gamma _2)\\ m_{\text {a}}(u_3-\gamma _3)\\ m_{\text {a}}(u_4-\gamma _4)\\ \frac{-a_1\, F^{\text {isom}}(l^{\text {CE}}_1) F^{\text {max}}_1+F^{\text {SEE}}(l^{\text {CE}}_1,\varphi )+d \, {\dot{l}}^{MTU}_1({\dot{\varphi }}, \varphi )}{a_1\, F^{\text {isom}}(l^{\text {CE}}_1)\mu _1 \, F^{\text {max}}_1+d}\\ \frac{-a_2\, F^{\text {isom}}(l^{\text {CE}}_2) F^{\text {max}}_2+F^{\text {SEE}}(l^{\text {CE}}_2,\varphi )+d \, {\dot{l}}^{MTU}_2({\dot{\varphi }}, \varphi )}{a_2\, F^{\text {isom}}(l^{\text {CE}}_2)\mu _2 \, F^{\text {max}}_2+d}\\ \frac{-a_3\, F^{\text {isom}}(l^{\text {CE}}_3) F^{\text {max}}_3+F^{\text {SEE}}(l^{\text {CE}}_3,\varphi )+d \, {\dot{l}}^{MTU}_3({\dot{\varphi }}, \varphi )}{a_3\, F^{\text {isom}}(l^{\text {CE}}_3)\mu _3 \, F^{\text {max}}_3+d}\\ \frac{-a_4\, F^{\text {isom}}(l^{\text {CE}}_4) F^{\text {max}}_4+F^{\text {SEE}}(l^{\text {CE}}_4,\varphi )+d \, {\dot{l}}^{MTU}_4({\dot{\varphi }}, \varphi )}{a_4\, F^{\text {isom}}(l^{\text {CE}}_4)\mu _4 \, F^{\text {max}}_4+d}\\ \frac{\sum _{i=1}^{4} r_i(\varphi )\,F^{\text {MTU}}(a_i,l^{\text {CE}}_i,{\dot{l}}^{\text {CE}}_i) }{J}\\ {\varphi _{\text {h}}} \end{array} \right) . \end{aligned}$$

(18)

The formulation of muscle activation was introduced by Hatze (1977) and slightly restated by Kistemaker et al. (2006) (Fig. 4). Herein, the activity $a_i$ is calculated depending on the free calcium ion concentration $\gamma _i$ as well as on the length of the contractile element $l^{\text {CE}}_i$

$$\begin{aligned} a_i(\gamma _i, l^{\text {CE}}_i)= \frac{a_0 + \left( \rho (l^{\text {CE}}_i) \gamma \right) ^3}{1 + \left( \rho (l^{\text {CE}}_i) \gamma \right) ^3}, \end{aligned}$$

(19)

where the monotonically increasing function $\rho$ is calculated as

$$\begin{aligned} \rho (l^{\text {CE}}_i)=c \eta \frac{k-1}{k-\frac{l^{\text {CE}}_i}{l^{\text {CE,opt}}}}\frac{l^{\text {CE}}_i}{l^{\text {CE,opt}}}. \end{aligned}$$

(20)

The values of the constants $m_{\text {a}},\, c,\, \eta , \,a_0$ and k are based on the work of Kistemaker et al. (2006) and are given in Table 2.

Table 2 Activation dynamics parameters for the formulation of Hatze

According to Zajac (1989), the activity and the external neural stimulation $u_i$ are directly related via the ODE

$$\begin{aligned} \dot{a_i}=&\frac{1}{\tau _{\text {act}}(1-a_0)} \nonumber \\&\cdot \left( u_i(1-a_0)-u_i\left( 1-\frac{\tau _{\text {act}}}{\tau _{\text {deact}}}\right) (a_i-a_0)-\frac{\tau _{\text {act}}}{\tau _{\text {deact}}}(a_i-a_0) \right) , \end{aligned}$$

(21)

where the time constants $\tau _{\text {act,deact}}$ specify the time for activation growth and decay after a neural impulse, respectively. The parameter $a_0$ represents the minimum activity a that is assumed when no stimulation signal exists. The values of the parameters $\tau _{\text {act}}, \tau _{\text {deact}}$ and $a_0$ are extracted from Bayer et al. (2017) and documented in Tables 3 and 4.

Table 3 Activation dynamics parameters for the formulation of Zajac

Table 4 Parameters used in this study

Appendix B

To explore the sensitivity of the presented simulation results to changes in model parameters, additional simulations were carried out. We focused on the model parameters determining the dynamic properties of the model. The following parameters were varied one by one: the damping coefficient d (set to $d=R_{SDE}=0.01$), the inertia J of the lower arm (increased by $10\%$), the muscle-specific parameter $F^{max}$ (for every muscle increased by $10\%$). As basis for comparison, the last result of the arm model using four muscle groups, activated by Zajac’s activation dynamics, was taken, see Fig. 3.

It was found that the decreased damping coefficient led to an increased arm velocity (Fig. 5a). The corresponding smaller ROA estimate, shown in Fig. 5b, supports our finding that slower arm movements are more stable. The increased inertia J influences the acceleration of the elbow angle, see Eq. 1. Slower acceleration of the elbow angle led to a bigger ROA estimate in the ${\dot{\varphi }}$-direction, see Fig. 6. The increased muscle-specific parameter $F^{max}$ results in a smaller ROA estimate, see Fig. 7.

In all, the sensitivity of the results to a variation of certain model parameters can be shown. However, the results using modified model parameters do not significantly differ from the main results. Thus, the the general findings of this study are supported.

Fig. 5

a Time course with $d=R_{SDE}$ (solid line) and with $d=_{SDE,max}$ (dashed line). b The corresponding projections of the ROA estimates are illustrated with ${\mathbf {x}}(1:8)={\mathbf {x}}_{\text {s}}(1:8)$

Fig. 6

The projection of the ROA estimates with increased intertia J (solid line) in contrast to the basis (dashed line)

Fig. 7

The projection of the ROA estimates with increased parameter $F^{max}$ (solid line) in contrast to the basis (dashed line)