1 Introduction

Wearable exoskeleton robots, which can apply torques directly to the wearer’s joints, are capable of either assisting the motion of the wearer, by applying torques which aid those of their muscles, or resisting the motion of the wearer, by applying torques that oppose those of their muscles. This makes them an ideal platform for designing systems for the purposes of exercise and rehabilitation of human muscles or diagnosis of muscle related disorders due to their ability to apply torques directly to human joints [8, 12, 15, 17, 20, 22]. In these tasks, it is often desired to induce certain target muscles to generate a desired force level. This is useful in exercise or rehabilitation to strengthen the muscle, and for diagnostic cases to induce some expected muscle activation pattern and observe any deviation from it. However, targeting a specific muscle or group of muscles can be difficult, due to the large number of muscles compared to the relatively small number of joints where torque may be applied.

Several studies have presented methods for predicting the activation pattern of human muscles under a known applied loading [2, 7, 14]. These are usually based on the Optimality Principle in the neuromotor system, which was developed from empirical data on how the human neuromuscular system optimizes the activation pattern of muscles. This muscle force prediction allows a mathematical formulation that has been reliably demonstrated in several studies [5, 7, 9, 16].

With a mathematical formulation for a muscle activation pattern for a given applied loading, it should be possible to perform the inverse calculation and find the required applied loading to achieve a certain activation pattern for some target muscles. To accomplish this objective, the mechanisms of redundant muscle coordination during human–robot physical interaction need to be rigorously formulated and understood. Such a method has been developed by [6, 18–20], wherein it is possible to provide target forces for a certain subset of arm muscles.

The previous method could exactly achieve the desired target muscle forces without inducing activity in otherwise inactive muscles. Thus, the algorithm was able to calculate the necessary joint torques to be applied by an exoskeleton. However, this method had several key limitations. It included several feasibility criteria that had to be met in order for the muscle activation pattern to be physically feasible [20]. These criteria required that target muscle forces be achieved exactly, which may not always be possible or necessary. For many exercise or rehabilitation situations, approximate forces may be acceptable. In addition, it required that inactive muscles be maintained as inactive, another requirement that may not be strictly necessary. Finally, the formulation did not address the upper limit of muscle forces, requiring only that the calculated forces be nonnegative. In reality, using such a system on a person could result in excess applied force, causing muscle strain. These limitations were accepted to simplify the resultant formulation. This study will relax these requirements and reframe the feasibility criteria as optimization problems to achieve the target muscle pattern with as little error as possible, as well as account for the upper limit of muscle forces.

Furthermore, an ideal exoskeleton should be able to apply force to all joints of the wearer, but practical difficulties in the design of exoskeletons leads to nonideal devices that may be able assist/resist movement of only certain joints. For example, the shoulder joint is more complicated than the elbow joint in terms of the number of degrees-of-freedom (DOF). The application of torques to the torso and the neck joints is almost impossible, due to the complex structure of the vertebral column and limitations of wearable designs. Inadequate application of joint torques of such a nonideal exoskeleton may result in errors in individual muscle forces. Similar problems can widely be observed in existing exoskeletons, nearly all of which are nonideal in this fashion: the wearer’s joints have more DOF than the device can control. The hip joint has three DOF. However, some leg exoskeletons only control movements in the frontal plane. Therefore, robustness to the lack of actuators needs to be examined and incorporated with the computational algorithm for practical use. Instead of finding a solution that perfectly realizes a muscle activation pattern in targeted muscles, a quasi-optimized solution that minimizes errors in individual muscles is obtained.

2 Problem formulation

2.1 System equation

The methods explained in this article will consider the individual muscle force control for static tasks, where it is assumed that a subject does not change his or her posture during a task and all muscle contractions are isometric. The dynamics of the body and robot are neglected. Figure 1(a) shows a schematic diagram where a human operator is performing a motor task, which requires the human to exert their muscles to accomplish some target force or motion, while interacting with a robotic manipulator and wearing an exoskeleton device. The robot is capable of applying force to the human’s hand in three dimensions, while the exoskeleton is assumed to be ideal and designed such that it is capable of applying torques to all nine of the human’s arm joints. The goal, as shown by Fig. 1(b), is to modulate the muscle forces applied by the human through these forces and torques applied by the manipulator and exoskeleton. For a human musculoskeletal model that has M joints and N muscles, the net torques in the human joints is given by (1).

$$ \boldsymbol{\tau}_h = \boldsymbol{g}(\boldsymbol{\theta})+\boldsymbol{J(\boldsymbol{\theta})}^T {\boldsymbol{F}} - \boldsymbol{\tau}_a $$
(1)

Here, τ h ∈ℜM is a vector of human joint torques, θ=[θ 1,…,θ M ]T∈ℜM is a vector of joint angles, F=[F x ,F y ,F z ]T is the translational force at the tip (i.e., the force the human applies to the environment or the force applied by the manipulator to the human), J ( θ ) is the Jacobian between the joints and end-point, g(θ) is the gravity force, and τ a ∈ℜM is the joint torque applied by the exoskeleton. The static equation of the musculoskeletal system is then given by (2). In order to achieve the desired muscle control, the muscle forces should be induced such that the joint torques are those applied externally by (1).

(2)

In this relation, A∈ℜM×N is the moment-arm matrix of the muscles and f=[f 1,…,f N ]T∈ℜN is the human muscle force vector. The element a ij of A represents the moment arm of muscle j for joint i, so a ij =0 is given if f j does not affect joint i. Note f j ≥0 (j=1,…,N) because muscle contraction cannot generate negative forces. g(θ), J ( θ ), and A ( θ ) for a given posture θ can be calculated by the musculoskeletal model developed in [18–20]. This model of the upper-right limb with 51 muscles has been developed to analyze the physical interaction between the exoskeleton and the wearer at the level of individual muscles. Its model consists of 5 rigid links and 9 joints. For more detail, see [20]. To simplify the problem, no upper-bound is given for τ a .

Fig. 1
figure 1

Concept of individual muscle control via physical human–robot interaction

Full size image

2.2 Optimization principle in the neuromuscular system and muscle force prediction

The human body has a much larger number of muscles than the number of joints, i.e., N≫M, leading to muscle redundancy. This fact makes the prediction of muscle forces f by knowing joint torques τ h an ill-posed problem. Various optimization approaches have been proposed to model the Optimization Principle in the neuromuscular system [2, 13, 14, 21] and to solve this problem by minimizing a cost function. The main difference among the approaches is the structure of cost functions that represent performance criteria on which the neuromuscular system optimizes the activation of muscle forces. In the literature that deals with isometric or relatively slow motions, the cost functions have a general form comprised of the sum of muscular stress or force raised to a power. The static optimization method can be formulated as follows:

(3)
(4)

The cost function u(f) is determined by the weighting factors c j ’s, which are determined based on the physical characteristics of each muscle, and the integer power r. It should be noted that arguments still exist on the choice of the weighting factors c j and the integer r of the power [2, 13, 14, 21]. There are still arguments and criticism of the neurological background of this muscle force prediction; however, this approach has been implemented in publicly or commercially available software [3, 4]. The effectiveness of this approach for predicting stereotyped motor performances has been reported in the literature [5, 7, 9].

2.3 Individual muscle control

Individual muscle control requires solving the inverse problem of the aforementioned muscle force prediction in (3) to control N muscles by adjusting M joint torque inputs. Note that this inverse solution is not straightforward since any induced muscle activation pattern must satisfy the physiology-based optimality criterion. The first condition in (4) represents the balance of joint torques. The second condition means that each muscle can only “pull,” exerting positive contracting force limited by its maximum voluntary force. Mathematically, this can be treated as finding an adequate equality constraint for the joint torques such that desired muscle forces are obtained as a result of the cost function minimization. As described earlier, the exoskeleton merely modifies human joint torques, which is equivalent to the modification of the first equality condition in (4).

Let f 0 be the nominal muscle forces when a subject is performing a nominal motor task. The human force vector f 0 may be permutated according to three groups of muscles: target muscles, which will be controlled to achieve a desired force level; non-target muscles, which are active but not directly controlled; and inactive muscles, which provide no force. Let \(\tilde{N}\leq N\) be the number of the active muscles, and \(N-\tilde{N}\) be the number of the inactive muscles. The active muscles have elements with nonzero values in f 0, and the inactive muscles have zeros. The active muscle group consists of target muscles \(\boldsymbol{f}_{t}\in \Re^{N_{t}}\) and non-target muscles \(\boldsymbol{f}_{n}\in \Re^{N_{n}}\) where \(N_{t}+N_{n}=\tilde{N}\). Without the loss of generality, the order of the N muscles may be permutated according to these three groups for the simplicity of description.

$$ \boldsymbol{f} \buildrel \triangle \over = \left[ {\begin{array}{c} \underline{\begin{array}{c} \boldsymbol{f}_{t} \\ \boldsymbol{f}_{n} \\ \end{array}} \\ {\boldsymbol{0}} \end{array}} \right] {\begin{array}{l} \cdots \mathrm{target muscles}\\ \cdots \mathrm{non\hbox{-}target muscles}\\ \cdots \mathrm{inactive muscles}\\ \end{array}} $$
(5)

The above permutation for f is also applied to the moment-arm matrix A, accordingly

$$ \boldsymbol{A}^T = \left[ {\begin{array}{c} \underline{\begin{array}{c} \boldsymbol{A}_{t} \\ \boldsymbol{A}_{n} \\ \end{array}} \\ {\boldsymbol{A}_v} \end{array}} \right] {\begin{array}{l} \cdots \mathrm{target muscles}\\ \cdots \mathrm{non\hbox{-}target muscles}\\ \cdots \mathrm{inactive muscles}\\ \end{array}} $$
(6)

Hereafter, these permutated vectors and matrices will be used.

The desired target muscle forces f td are given as follows by explicitly specifying the ratio of change for each of the target muscles:

$$ \boldsymbol{f}_{td} = \mathrm{diag}[\gamma_1,\gamma_2,\ldots, \gamma_{N_t}] \boldsymbol{f}_{t0} $$
(7)

where γ j (>0) is the ratio of change of the jth target muscle. The subscript d denotes the desired muscle forces, and 0 denotes the nominal muscle forces. Let the total external torque τ ex ∈ℜM be

$$ \boldsymbol{\tau}_{ex}=\boldsymbol{J}^T \boldsymbol{F} -\boldsymbol{\tau}_a $$
(8)

Equation (8) shows how the joint toques can be adjusted by applying τ a from the exoskeleton and by exerting F at the tip either by performing a motor task or through a robot manipulator. Note that there is a certain freedom in choosing F and τ a to achieve a given τ ex . This will be discussed in a later section. The problem to realize f td can be formulated as follows.

Perfect individual muscle control

Find F and τ a that will achieve a τ ex such that the solution of (3) includes a desired muscle activation pattern for the target muscles, f td , i.e.,

$$ \bigl[\boldsymbol{f}_{td}^T, \boldsymbol{f}_{nd}^T,\boldsymbol{0}^T \bigr]^T = \mathop{\mathrm{argmin}}\limits_{\boldsymbol{f}(\boldsymbol{\tau}_{ex})} u(\boldsymbol{f}) $$
(9)

with minimum changes in nontarget muscle forces, i.e., |f nd −f n0|→min.

2.4 Solution for perfect muscle force control

The previous work by Ueda, Ming, et al. [6, 18–20] provided a solution to exactly realize the desired target muscles forces, which will be referred to as perfect muscle control. Performing the minimization in (9) to find a solution to (2) results in the external torque τ ex for the perfect control given by

$$ \boldsymbol{\tau}_{ex}= \bigl[ \boldsymbol{A}_{t}^T\quad \boldsymbol{A}_{n}^T\bigr] w^{-1} \left( \left[ \begin{array}{c} \boldsymbol{A}_{t} \\ \boldsymbol{A}_{n} \\ \end{array}\right] \boldsymbol{\alpha} \right) $$
(10)

where α is the control input vector and represents in a moment arm independent manner the change in torques that must be applied to achieve the desired muscle activation pattern. The transformation w(∗) is a function that converts the muscle force vector f to a new vector q as q=w(f) where the jth elements of f and q are given as

$$ q_j \buildrel \triangle \over = \frac{\partial u(\boldsymbol{f})}{\partial f_j} = r c_j {f_j}^{r-1},\quad (j=1, \ldots, N) $$
(11)

and w −1(q)=f is the inverse function of w(∗) and the c j ’s are as in (3). q can be permutated in a similar manner to f:

$$ \boldsymbol{q} = \left[ {\begin{array}{c} \underline{\begin{array}{c} \boldsymbol{q}_{t} \\ \boldsymbol{q}_{n} \\ \end{array}} \\ {\boldsymbol{q}_v} \end{array}} \right] {\begin{array}{l} \cdots \mathrm{target muscles}\\ \cdots \mathrm{non\hbox{-}target muscles}\\ \cdots \mathrm{inactive muscles}\\ \end{array}} $$
(12)

The control input vector α is the main element in the solution and is given as

$$ \boldsymbol{\alpha} = \boldsymbol{A}_t^+ \bigl[ w(\boldsymbol{f}_{td})-w(\boldsymbol{f}_{t0})\bigr] + \bigl(\boldsymbol{I} - \boldsymbol{A}_{t}^+ \boldsymbol{A}_{t} \bigr) \boldsymbol{\beta} $$
(13)

where I is the identity matrix, and β is a free parameter that represents the remaining redundancy for controlling the nontarget muscles as the second priority. This solution holds regardless of the choice of the parameter r in (3) and (11), which was verified in [14] (see the Appendix for further details). In [20], the existence of this solution relied on three criteria:

  1. 1.

    The target muscles forces must all be linearly independent of one another:

    $$ \mathrm{rank}({\boldsymbol{A}_{t}}) = \mathrm{rank} \bigl(\bigl[\boldsymbol{A}_{t}\quad w(\boldsymbol{f}_{td})-w(\boldsymbol{f}_{t0})\bigr]\bigr) $$
    (14)
  2. 2.

    The inactive muscles must be maintained as exactly inactive:

    $$ -\boldsymbol{A}_{v} \left[\begin{array}{c} \boldsymbol{A}_{t} \\ \boldsymbol{A}_{n} \\ \end{array} \right]^+ \left[\begin{array}{c} \boldsymbol{q}_{t0} \\ \boldsymbol{q}_{n0} \\ \end{array} \right] - \boldsymbol{A}_{v} \boldsymbol{\alpha} >0 $$
    (15)
  3. 3.

    The nontarget active muscles must be maintained as active:

    $$ \boldsymbol{A}_{n} \boldsymbol{\alpha} + w(\boldsymbol{f}_{n0})>0 $$
    (16)

Due to space limitation, only the final criteria are given here. Please see [20] for detailed derivation. The method attempted to find a solution by minimizing the change in nontarget active muscles while satisfying the three criteria. If this was not possible, then no solution existed that could exactly realize the target muscles forces within physiological constraints. This resulted in a very narrow range of feasibility that could be practically expanded by relaxing the required assumptions.

In addition, it was assumed that the exoskeleton is ideal and, therefore, has means to adjust all of the joints, i.e., τ h is fully modifiable by τ ex , and a combination of F and τ a always exists that realizes a given τ ex via the physical human–robot interaction.

  1. 4.

    The exoskeleton is ideal, and therefore τ a exists (as well as F in the case where a manipulator arm is also used) and can realize τ ex in (8).

This paper shows theoretical results that relax these requirements and reframe the feasibility criteria to achieve the target muscle pattern with as little error as possible.

3 Relaxing the feasibility conditions

3.1 Problem reformulation

The above limitations on the use of the perfect muscle control solution motivated generalizing the solution to eliminate assumptions and make the control scheme more widely applicable. To do so, it is necessary to examine the three key assumptions, criteria 1–3 discussed in Sect. 2.4.

First, the solution ignores the upper bound on an individual muscle’s force. While mathematically sound, this assumption has serious physiological consequences. Should an exoskeleton attempt to apply the resultant torques, and one or more muscles are required to supply more force than physically possible, the wearer of the device could sustain serious injury. Accounting for this limit requires adding the right side of the inequality condition, f j ≤f max j (j=1,…,N), into the Karush–Kuhn–Tucker (KKT) equation, which will result in a second set of λ’s. For clarity, the λ’s in the initial solution, which correspond to the lower bound 0≤f j , will be denoted as λ l , while the λ’s corresponding to the upper bound will be denoted as λ u .

The second assumption is that it is necessary and possible to match the desired target muscle forces exactly. In reality, it is never possible to exactly match all muscle forces, and it is likely not absolutely necessary, either. Therefore, instead of requiring f j =f jd , a small error tolerance e j is allowed such that f j =f jd +e j . When performing the transformation q=w(f), this becomes q t =q td +ϵ t , where ϵ t is the allowable error tolerance in q t and takes the same units as q.

Finally, the third assumption states that inactive muscles must stay inactive, and non-target active muscles must stay active. Physically, this may be difficult to realize. Therefore, in the interest of finding a control scheme that is as widely application as possible, it is desired to allow initially active non-target muscles to become inactive, while allowing initially inactive muscles to generate a small amount of force. The former requires including the Lagrange multipliers in the KKT condition as with the generalization applied for the first assumption. The latter requires the introduction of an error tolerance term, ϵ v , as with the generalization for the second assumption.

3.1.1 New KKT condition

Combining the generalizations for all three assumptions, a new feasibility equation is obtained from the KKT condition [1, 11]. For convenience, the inequality constraint 0≤f j ≤f maxj (j=1,…,N) was split into two conditions, with the lower and upper bounds corresponding to g jl and g ju , respectively. The updated KKT condition is shown in (17) and is subject to the conditions in (18), (19), and (20).

(17)
(18)
(19)
(20)

The constraints h i , g jl , and g ju are given by (21), (22), and (23).

(21)
(22)
(23)

The derivatives of the three constraints are then given by (24), (25), and (26).

(24)
(25)
(26)

These, combined with the definition of q in (11), which remains unchanged, give the updated equation given by (27).

(27)

Permutating this equation as done for (12), but without making any simplifications or assumptions, results in (28).

(28)

It is possible to make a few simplifications regarding the inequality constraint multipliers. Target muscles will necessarily always be active and, therefore, will supply a nonzero force, so the lower constraint is always satisfied (λ tl =0). In addition, inactive muscles by definition supply zero force, so the upper constraint is satisfied (q v =0 and λ vu =0). This yields the result in (29).

(29)

The result can be evaluated for both the nominal muscle forces, f 0, and the desired muscle forces, f d . For the nominal case, by definition, the active nontarget muscles will have a positive force, so the lower bound is automatically satisfied (λ 0nl =0), giving (30). For the desired case, it can be assumed that the desired values of the target muscles will never be the maximum force for safety reasons, while there is no assumption that the nontarget active muscles will stay active. Also, the allowable error tolerance terms, ε, are incorporated, which gives (31).

(30)
(31)

3.1.2 Muscle control

Using the generalized KKT conditions given in (30) and (31), it is possible to derive a control law that can be used to find the value of τ h that will induce the desired muscle activation pattern. Since μ corresponds to the vector of human joint torques, it is the ideal quantity to control, as finding an appropriate value for μ d will lead to finding τ ex , and subsequently F and τ a , to realize the desired muscle forces. Therefore, let μ d =μ 0+α, where α is the control input vector, similar to that presented in (13) for the case of perfect muscle control. Through the derivation below, this leads to (32).

(32)

The unknown to be solved for in (32) is the control input for the joint torques, α. The q 0’s are known from calculating the nominal case and q dt is known from the desired muscle activation pattern, while A T is known from the geometry and configuration of the muscles. This leaves q dn , both ϵ’s, and all λ’s as parameters that can be chosen to make the solution feasible. There are so many parameters due to the large difference between the number of muscles, N, and the number of joints, M (N≫M).

Examining (32), it can be seen that each term has a physical meaning. The left-hand side, from (11), gives the desired change in muscle forces. The first term on the right-hand side provides the effect of the control input on the muscles, since A T gives the mapping from joint torques to muscles forces, while the remaining terms account for any muscles, which are exactly equal to zero or the muscle’s maximum force. It is important to note that the inequality constraint multiplier vectors, λ’s, will tend to be sparse. The elements of these are only nonzero if that particular muscle is exactly on the constraint multiplied by that λ. For example, if nontarget muscle j is providing maximum force in the nominal case, but is between zero and maximum in the desired case, then λ j0nu >0, λ jdnu =0, and λ jdnl =0. In general, many of the elements in these vectors will be zero.

3.1.3 Solution

The goal of this algorithm is to match the desired target muscle forces as exactly as possible. Since the number of outputs (muscle forces), N, is significantly larger than the number of inputs (joint torques), M, a priority based approach is used to find a solution. Therefore, the solution for α can be found from the first row of (32): q dt +ϵ t −q 0t =A t α+λ 0tu . Rearranging slightly, this becomes (33).

$$ \boldsymbol{q}_{dt} - \boldsymbol{q}_{0t} + \boldsymbol{\epsilon}_t - \boldsymbol{\lambda}_{0tu} = \boldsymbol{A}_t \boldsymbol{\alpha} $$
(33)

Assuming the solution exists (the existence of a solution is discussed in Sect. 3.1.4), it is given by (34), which is analogous to (13) for the perfect control case.

$$ \boldsymbol{\alpha} = \boldsymbol{A}_t^+ (\boldsymbol{q}_{dt} - \boldsymbol{q}_{0t} + \boldsymbol{\epsilon}_t - \boldsymbol{\lambda}_{0tu}) + (\boldsymbol{I} - \boldsymbol{A}_t^+ \boldsymbol{A}_t) \boldsymbol{\beta} $$
(34)

This new solution is applied to (10) to obtain external torques. The free parameter β indicates the redundancy of the solution, and has an effect on the values of the remaining non-target (active and inactive) muscles. The choice of β is generally arbitrary, and will be discussed further with the choice of the other parameters in Sect. 3.1.4.

3.1.4 Feasibility and choice of parameters

Due to the way in which the matrices were permutated, each row of (32) deals with a different set of muscles. The first row is concerned with the target muscles, the second row with the active (in the nominal case) nontarget muscles, and the last row with the inactive (again, in the nominal case) nontarget muscles. Therefore, each row can be extracted from the matrix equation to determine a set of feasibility criteria for the existence of the solution discussed in Sect. 3.1.3.

The first row, previously extracted to get (33), is the most important, since it deals with the target muscles. In order to solve (33) using linear algebra, a matrix composed of A t augmented with the left-hand side of the equation must have the same number of linearly independent rows as A t itself. This is concisely expressed in (35).

$$ \mathrm{rank}(\boldsymbol{A}_{t}) = \mathrm{rank} \bigl( \bigl[ \boldsymbol{A}_{t}(\boldsymbol{q}_{dt} - \boldsymbol{q}_{0t} + \boldsymbol{\epsilon}_t - \boldsymbol{\lambda}_{0tu})\bigr]\bigr) $$
(35)

When this condition is satisfied, it is possible to obtain the desired target muscle forces while also satisfying the bounds on muscle force. The error tolerance ϵ t is a parameter that can be chosen to make this condition hold when it is not possible to exactly realize the target forces. In this case, a small value can be given to some or all elements of ϵ t to make an approximation of the desired q dt realizable. The multiplier λ 0tu is known from the nominal muscle distribution, and is only nonzero for muscles that are initially at their maximum force.

The next priority in choosing the solution is to maintain inactive muscles as inactive (or keep them minimally active if this is not possible). The third row of (32), given by (36), provides the necessary condition for this.

$$ \boldsymbol{\lambda}_{dvl} - \boldsymbol{\epsilon}_v = \boldsymbol{\lambda}_{0vl} - \boldsymbol{A}_v \boldsymbol{\alpha} $$
(36)

For inactive muscles to remain inactive, λ dvl must be positive and ϵ v will be zero, requiring the right-hand side of the equation to be greater than zero. If, however, it is not possible (or not necessary) to meet this exactly, then elements of ϵ v can be chosen to be nonzero, leading to the condition given by (37).

$$ \boldsymbol{A}_v \boldsymbol{\alpha} + \boldsymbol{\lambda}_{0vl} > -\boldsymbol{\epsilon}_v $$
(37)

The final priority is to minimize the change in the forces of the nontarget muscles. The second row of (32) can be written as in (38).

$$ \boldsymbol{q}_{dn} - \boldsymbol{q}_{0n} = \boldsymbol{A}_n \boldsymbol{\alpha} + \boldsymbol{\lambda}_{dnl} - \boldsymbol{\lambda}_{dnu} + \boldsymbol{\lambda}_{0nu} $$
(38)

In the ideal case, none of the nontarget muscles are at maximum force, and it is desired to keep them from becoming inactive, so all the λ n ’s are zero. For this to be the case, (39) must be true, where q max=w(f max).

$$ 0 < \boldsymbol{A}_n \boldsymbol{\alpha} + \boldsymbol{q}_{0n} < \boldsymbol{q}_{\max} $$
(39)

Substituting the solution for α from (34) into (38), then making the same decisions that lead to (39), the only remaining free parameter is β. Therefore, a reasonable way to choose β is to avoid a change in the non-target muscles, or to minimize ∥q dn −q 0n ∥, where q dn −q 0n is given by (40).

(40)

In general, this is consistent with the condition given by (39). However, due to the need to use the pseudoinverse of A (since A is not square), it is possible for numerical inaccuracies to arise. In this case, if the condition is not satisfied, then one or more nontarget muscles is at the upper or lower limit. To maintain feasibility, the appropriate elements of λ dnu and λ dnl must be chosen such that the condition is again satisfied (note that λ 0nu is known from the nominal muscle distribution).

3.2 Validation

The relaxed formulation was validated and compared to the perfect muscle control solution using a computer model. The model, developed in prior work by Ueda, Ming et al., [18–20], simulated the muscles of the human upper arm using a total of 9 joints (M=9) from the torso to wrist joint with 51 muscles (N=51). The Crowninshield’s cost function (see the Appendix) is applied for the optimization criterion in the model. The joint torques an exoskeleton should apply for a variety of target muscle activation patterns, given in Table 1, were calculated using both the perfect control method and the relaxed control method, for comparison. The feasibility of each pattern for both methods is shown in Table 2. For the relaxed method solution, the norm of the error tolerances, ϵ t and ϵ v , are presented normalized by the nominal muscle forces, q 0. For all three patterns, the arm is in held straight down at the side of the body with the elbow bent 90∘ and the forearm held out, as in Fig. 2, with an external applied force [F x ,F y ,F z ]=[10,10,0] for the nominal task.

Fig. 2
figure 2

Nominal task, F=[10,10,10]

Full size image
Table 1 Desired muscle activation patterns for testing
Full size table
Table 2 Desired muscle activation pattern feasibility results
Full size table

To demonstrate that the relaxed method includes the solutions the perfect method could find, Fig. 3(a) shows the results of both methods for pattern 1. Both solutions are identical, showing that a problem solvable by the perfect control method would yield the same solution using the relaxed. For patterns 2 and 3, however, the perfect method could not find a physiologically feasible solution, but the relaxed method could. Figure 3(b) shows the resulting muscle forces for some of the muscles returned by the relaxed method for target pattern 2. The relaxed method was able to find a feasible solution and match the target muscles by allowing inactive muscles to become active. For pattern 3, the results are shown in Fig. 3(c). A feasible solution was found by not matching the target muscles exactly. The muscle forces are presented normalized by q 0.

Fig. 3
figure 3

Simulation results of relaxed method

Full size image

Figure 4 explores the feasibility of combinations between the muscles used in the three patterns for both the perfect and relaxed methods. The hatched areas in both plots represent the feasibility region of the perfect control method. Outside this region, only the relaxed control method was able to find a solution. The shading of the plot indicates the required magnitude of the error tolerance terms, ϵ t and ϵ v , normalized by the nominal muscle force, q 0. The relaxed method demonstrated a much larger region in which a feasible solution could be found than the perfect method, with large regions requiring only small allowable errors. In both cases, however, as the limits of the muscle forces were approached, large allowable errors were required to find a feasible solution.

Fig. 4
figure 4

Comparison of feasibility region of both methods: Hatched region shows perfect method feasibility, shading indicates required error tolerance to obtain feasible result using relaxed method

Full size image

4 Solutions for a nonideal exoskeleton

4.1 Limitation in adjusting joint torques

The problem in realizing desired joint torques with incomplete torque application to the joints (i.e., with lack of means to apply torques to certain human joints) is considered. Criterion 4 discussed in Sect. 2.4 is relaxed. Instead of finding a solution that perfectly realizes a muscle activation pattern in targeted muscles, a quasi-optimized solution that minimizes errors in individual muscles is obtained.

Consider the individual muscle control shown in Fig. 1(a) where the exoskeleton is not capable of controlling all of the joints torques involved in a task due to a lack of means to apply torques. Even if f td is realizable and a solution of τ ex exists mathematically, a combination of F and τ a that satisfies (8) may not exist. In other words, τ ex may not be realizable due to the inability to control all of the joint torques. Such use of a nonideal exoskeleton may result in errors in target muscle forces. Recall that F is projected onto the torque space through J T. The number of elements of F is three (i.e., F x ,F y ,F z ), but that of τ ex is much larger than three in general. Therefore, the control of the tip-force via a manipulator arm F could mitigate the issue but would not completely resolve the problem. Note that other controllability issues, such as misalignment of or insufficient torque from exoskeleton actuators are not considered here and may be addressed in future work.

To overcome such a nonideal exoskeleton problem, the realizability of (8) is further analyzed. Equation (8) is rewritten as follows by considering the controllable degrees-of- freedom of the exoskeleton:

(41)
(42)

where J e =[J T −E]T and \(\boldsymbol{F}_{e}=[\boldsymbol{F}^{T}\ \boldsymbol{\tau}_{ac}^{T} ]^{T}\). τ ac is a vector that only contains the torque components controllable by the exoskeleton’s actuators. Matrix E maps these components onto the joint torque space. For example, if τ ex ∈ℜ9×1 (i.e., a total of 9 joints are involved in a movement) and the exoskeleton robot can control three torques, τ 3,τ 5,τ 9, then τ ac =[τ 3,τ 5,τ 9]T and

Substituting (41) for (10) gives

$$ \left[ \boldsymbol{A}_{t}^T\quad \boldsymbol{A}_{n}^T\right] w^{-1} \left( \left[ \begin{array}{c} \boldsymbol{A}_{t} \\ \boldsymbol{A}_{n} \\ \end{array}\right] \boldsymbol{\alpha} \right) = \boldsymbol{J}_e^T \boldsymbol{F}_e. $$
(43)

4.2 Perfect individual muscle control with a nonideal exoskeleton

While previous sections have left r unspecified, this section explicitly utilizes Crowninshield’s cost function with r=2 (see the Appendix) to obtain closed-form solutions. Generalization with respect to r will be considered in future work. The Crowninshield’s cost function is quadratic:

(44)
(45)
(46)

Although the range of feasible f td is limited, perfect control of target muscles is still feasible if the following condition holds.

Theorem

(Perfect individual muscle control with a nonideal exoskeleton)

Assume that either the feasibility conditions presented in [20] or the relaxed conditions presented in the previous sections hold; therefore, α exists. If the following condition also holds, a solution for F e exists for a given α that perfectly realizes the target muscle forces:

$$ \mathrm{rank}(\boldsymbol{G}) = \mathrm{rank}\bigl([\boldsymbol{G}\quad \boldsymbol{\alpha}]\bigr) $$
(47)

where

$$ \boldsymbol{G} = 2\biggl(\bigl[ \boldsymbol{A}_{t}^T\quad \boldsymbol{A}_{n}^T\bigr] \boldsymbol{S}^2 \left[ \begin{array}{c} \boldsymbol{A}_{t} \\ \boldsymbol{A}_{n} \\ \end{array}\right]\biggr)^{-1} \boldsymbol{J}_e^T $$
(48)

The solution of F e is obtained by

$$ \boldsymbol{F}_e =\boldsymbol{G}^+ \boldsymbol{\alpha} + (\boldsymbol{I}-\boldsymbol{G}^+ \boldsymbol{G}) \boldsymbol{\psi}, $$
(49)

where ψ is a free vector parameter mapped onto the null space of G, defined by (I−G + G). The parameter ψ can be chosen to tune the solution given by α.

Proof

From (43) and (46),

$$ \frac{1}{2} \bigl[ \boldsymbol{A}_{t}^T\quad \boldsymbol{A}_{n}^T\bigr] \boldsymbol{S}^{2} \left[ \begin{array}{c} \boldsymbol{A}_{t} \\ \boldsymbol{A}_{n} \\ \end{array}\right] \boldsymbol{\alpha} = \boldsymbol{J}_e^T \boldsymbol{F}_e. $$
(50)

If a motor task induces activities in a number of muscles greater than or equal to M, the matrix \([\boldsymbol{A}_{t}^{T}\ \boldsymbol{A}_{n}^{T}]\) is full row rank. This assumption most likely holds for a whole-arm task involving multiple joints. Therefore, the square matrix is invertible, which yields

$$ \boldsymbol{\alpha} = \boldsymbol{G} \boldsymbol{F}_e $$
(51)

Hence, the existence of F e can be checked by (47). Under this condition, F e is given by (49). Note that this condition guarantees perfect realization of target muscle forces although condition (47) is more restrictive than (35). □

Remark 1

(Reducing changes in non-target muscles)

If (I−G + G) is a nonzero square matrix, ψ can be chosen to reduce the changes in non-target muscles, e.g., ∥q tn −q 0n ∥→min. Roughly speaking, using an exoskeleton able to apply more joint torques is expected to lead to a wider choice of ψ, resulting in smaller changes in non-target muscles. This additional controllable DOF provided by such an exoskeleton can also be used to modify the distribution between F and τ a . In some cases, where the rank of (I−G + G) is large enough, the application of τ a using an exoskeleton may not be necessary; performing a task by only exerting an end point force F using a manipulator arm would appropriately adjust target muscles.

4.3 Relaxed individual muscle control with a nonideal exoskeleton

The rank condition (47) may not always hold. In other words, F e may not exist that perfectly realizes a given α. It still is possible to obtain desired target muscle forces with a certain range of error.

Remark 2

(Relaxing (47))

For simplicity, consider the unrelaxed criterion (14), a special case of (35) where ϵ t =0 and λ 0tu =0. Assume that (14) holds but (47) does not hold. This means that the individual muscle control problem has a solution with α. However, there is no combination of F and τ a to realize α due to a limitation of the hardware. It is still possible to induce changes in target muscles by using

$$ \boldsymbol{F}_e = \boldsymbol{G}^+ \boldsymbol{\alpha} $$
(52)

This solution does not guarantee perfect realization of f td , but minimizes errors in the target muscles in the sense of least mean square. Note that this solution does not have any redundancy, i.e., there is no free parameter than can be tuned to adjust nontarget muscles.

4.4 Simulation

First, the association between the controllability of target muscle forces and the number of joints the exoskeleton can apply torques to is investigated by simulation.

Consider a static posture shown in Fig. 2 where the angle of the elbow joint angle is 90∘. Similar to Sect. 3.2, the musculoskeletal system has a total of 9 joints (M=9) from the torso to wrist joint with 51 muscles (N=51). The Crowninshield’s cost function (see the Appendix) is applied for the optimization criterion. In the nominal task, a subject exerts a tip-force of F=[F x ,F y ,0]T=[10,10,0]T [N] to a handle in the horizontal plane. Among 51 muscles, 26 muscles are active and the remaining 25 muscles are inactive. The Biceps Longus muscle (BIC), Flexor Carpi Ulnaris muscle (FCU), and Extensor Carpi Ulnaris muscle (ECU) are selected as target muscles and controlled as shown in Table 3.

Table 3 Target muscles and desired ratios of change for simulation
Full size table

Five types of force/torque application shown in Fig. 5 are considered. In Type 1, a wearable device can apply torques to all 9 joints of the skeletal model. The mathematical formulation presented in [20] assumed this ideal mechanism. In Type 2, the wearable robot applies torques to a total of 7 joints from the shoulder to the wrist, excluding the torso and neck joints. This type uses a manipulator that can apply 3-dimensional forces to the tip. In Type 3, the wearable robot applies torques to a total of 4 joints from the elbow to the wrist joint. The device used in the authors’ past research [6, 18–20] has the same actuator configuration. Type 4 applies torque to the elbow joint. In Type 5, wearable actuators are not used but only a manipulator is used.

Fig. 5
figure 5

Exoskeleton configurations

Full size image

Table 4 shows computed tip-forces and assist-torques for types 1 to 5. Only types 1 and 2 satisfied (47), realizing perfect individual muscle control. Equation (52) for relaxed muscle control was used for types 3 to 5. As can be observed in the results, Types 1 and 2 had no error in desired target muscle forces. Type 3 could not accomplish perfect control, but it still achieved the target muscle forces at a maximum error of 25 %, recorded form the Biceps muscle. The forces for the Flexor Carpi Ulnaris muscle and Extensor Carpi Ulnaris muscle were still accurate. The table also shows average errors in 23 nontarget muscles. Errors in the target muscles and changes in nontarget muscles tend to increase as the number of controllable joints decreases toward Type 5. Type 5 particularly exhibited large errors. This implies that the sole exertion of a tip-force is insufficient to induce a desired muscle activation pattern, justifying the use of a wearable exoskeleton that can apply joint torques. This result also indicates that an ideal exoskeleton with a capability of applying all the joints is not always necessary. An exoskeleton with a reduced number of controllable joints would suffice for muscle control tasks with a certain allowable error range.

Table 4 Simulation results
Full size table

4.5 Experiments

Two experiments were conducted to confirm the efficacy of an exoskeleton with a reduced set of controllable joints. These experiments made use of a manipulator arm equipped with a force sensor to apply an end point force to the human’s arm.

4.5.1 Exoskeleton joints and changes in non-target muscles

A nominal task applies an external downward force of 30 [N] (i.e., [F x ,F y ,F z ]=[0,0,−30]). The Biceps Longus muscle (BIC) was selected as a target muscle and the desired change was given as ×0.7. Both cases satisfied (47); therefore, ψ in (49) can reduce changes in nontarget muscles.

Two configurations shown in Fig. 6 were considered. The system shown in Fig. 6(a), named Type 4h, uses an exoskeleton that applies torque to the elbow joint and a manipulator that applies 3-dimensional forces to the tip, totaling 4 controllable DOF. The device shown in Fig. 6(b), named Type 5h, only uses a manipulator, totaling 3 controllable DOF. The device types 4h and 5h correspond to types 4 and 5 in Fig. 5, respectively. It is expected that a configuration with more controllable DOF has a higher capability of controlling nontarget muscles than a mechanism with a fewer controllable DOF. Table 5 shows the computed assist torques and tip forces for types 4h and 5h.

Fig. 6
figure 6

Experimental setup for individual muscle control

Full size image
Table 5 End-effector force and applied joint torque for experiment 1: Biceps (BIC) × 0.7
Full size table

Electromygraphic signals (EMG) during tasks were processed in the same manner as [20]. The results are shown in Fig. 7 for 8 healthy subjects. Each subject performed five trials of the assigned task. White bars show desired changes in the two target muscles and gray bars show means of collected EMG signals with error bars indicating standard deviations (SD). As shown in Fig. 7(b), the expected/simulated average change in the non-target muscle activations increased from 21 % to 24 % as the number of controllable DOF of the system decreased from 4 to 3. In the experiment, the average change increased from 12 to 15 %.

Fig. 7
figure 7

Experimental result of individual muscle force control using a manipulator and exoskeleton: Biceps (BIC) ×0.7

Full size image

4.5.2 Individual muscle control using solely a manipulator

If the objective of a motor task is primarily to adjust target muscles, with less emphasis on changes of nontarget muscles, one might argue that the sole use of a manipulator would suffice. Such a system would, in practice, be advantageous in terms of reduced complexity of operation. It is likely that the condition (47) holds if the number of target muscles is not very large, for example, 1 or 2. Hence, motor tasks can be planned by using (49).

In the second experiment, the tasks shown in Table 6 with the manipulator in Fig. 6(b) are considered. The nominal task applied an external downward force of 30 N (i.e., [F x ,F y ,F z ]=[0,0,−30]). The Anterior Deltoid and Biceps Longus muscles were chosen as the target muscles. The tasks satisfied (47). The results are shown in Fig. 8. Each subject performed five trials. White bars show desired changes in the two target muscles and gray bars show means of collected EMG signals with error bars indicating standard deviations (SD). The observed tendencies to increase or decrease EMG signals, with respect to desired changes, were as expected overall. For example, a larger change in the Anterior Deltoid muscle was observed than that in the Biceps muscle where the Anterior Deltoid muscle was controlled to be ×2.0 while maintaining (i.e., ×1.0) the nominal force of the Biceps muscle. Similarly, both of the two target muscle forces increased in task D where desired changes greater than 1.0 were given to the both muscles. Note that we cannot rule out the possibility that the magnitude of EMG signals may not be strictly proportional to corresponding muscle force. Needle EMG signals will be used for further evaluation in the future.

Fig. 8
figure 8

Experimental result of individual muscle force control using only a manipulator

Full size image
Table 6 Motor tasks using a robot manipulator
Full size table

5 Conclusion

Prior work has developed a method by which the forces of human muscles can be controlled via an exoskeleton. However, the previous method relied on several limiting assumptions and requirements which simplified the mathematical formulation, but restricted its applicability. A relaxed method which removed several of these was presented. It reframed the problem from one attempting to exactly achieve a target muscle pattern to one in which the solution was optimized to minimize the error from a target pattern. This allowed patterns that were previously infeasible to be realized. The enhanced method was demonstrated using computer models and the improvement in feasibility region was shown. The efficacy was confirmed by simulation and experiments.

There are many situations in which such a method could be applied. Exoskeleton robot development is increasing, and this method would allow such a robot to apply torques to human joints to achieve some desired muscle pattern. This could be useful for exercise devices attempting to build muscle strength or for rehabilitation devices attempting to recover lost muscle abilities. This is due to the pinpointed nature of the method, targeting only the muscles that are desired and minimizing the changes in all others. In addition, it can be used for diagnosing muscle related disorders. Since the expected muscle pattern due to the applied load is known, any deviation from the expected pattern would indicate a problem in a person’s muscle control abilities. The relaxed method proposed here allows a much wider range of the applications than the previous method.