Robot adaptation to human physical fatigue in human–robot co-manipulation

Abstract

In this paper, we propose a novel method for human–robot collaboration, where the robot physical behaviour is adapted online to the human motor fatigue. The robot starts as a follower and imitates the human. As the collaborative task is performed under the human lead, the robot gradually learns the parameters and trajectories related to the task execution. In the meantime, the robot monitors the human fatigue during the task production. When a predefined level of fatigue is indicated, the robot uses the learnt skill to take over physically demanding aspects of the task and lets the human recover some of the strength. The human remains present to perform aspects of collaborative task that the robot cannot fully take over and maintains the overall supervision. The robot adaptation system is based on the Dynamical Movement Primitives, Locally Weighted Regression and Adaptive Frequency Oscillators. The estimation of the human motor fatigue is carried out using a proposed online model, which is based on the human muscle activity measured by the electromyography. We demonstrate the proposed approach with experiments on real-world co-manipulation tasks: material sawing and surface polishing.

1 Introduction

Human–robot collaboration is one of the key enablers for a successful integration of robots in human daily lives. Its field of application ranges from household and industrial activities using well-performing robotic platform such as humanoids (Tsagarakis et al. 2017; Kaneko et al. 2008; Albu-Schäffer et al. 2007), to augmentation of human body capabilities through exoskeleton systems (Fleischer and Hommel 2008; Peternel et al. 2016).

One of the main challenges to achieve a seamless and intuitive human–robot collaboration is to make the robot capable of reading the intention and objective of the human partner in on-line manner to appropriately assist him/her in given tasks. This signifies that the robot must communicate with the human counterpart through the detection and processing of the sensory data.

A common approach to establish such a communication channel is built on the use of force sensors mounted at the robot’s side to measure the interaction forces between the two agents (Ikeura and Inooka 1995; Kosuge and Kazamura 1997; Tsumugiwa et al. 2002; Evrard et al. 2009; Gribovskaya et al. 2011; Ikemoto et al. 2012; Peternel and Babič 2013; Donner and Buss 2016), providing a simple tool for the monitoring of the energy flow between the human and the robot. Alternatively, human-induced position changes can be directly read by the robots encoders (Peternel et al. 2014) to warn the robot control framework about the kinematic variations of the system in contact. Other common sensory systems are based on visual feedback (Agravante et al. 2014) and reading human auditory commands (Medina et al. 2012), creating an intuitive communication channel from the human standpoint.

While the above-mentioned approaches can be very efficient to control the collaborative behaviour of the robot, they have several drawbacks. For example, if the human and the robot simultaneously interact with an unpredictable environment, irregular and noisy interaction forces will be induced into force sensor readings (Peternel et al. 2014). In this case, it is difficult to distinguish which force components are coming from the human and which are coming from the interaction with the environment. In addition, some important properties of the human behaviour cannot be read by either of the above-mentioned methods; one of such property is the impedance of the human limb which plays a key role in achieving a stable contact with the uncertain environment (Burdet et al. 2001).

To overcome the disadvantages of the classical human–robot interfaces, we recently proposed an interface for human–robot co-manipulation control (Peternel et al. 2017) [based on tele-impedance principle (Ajoudani 2016)] where the intention of the cooperating human partner is read through the acquired muscle activity signals and his/her limb dynamic manipulability measure. This interface enables the robot to directly read the motor function of the human arm, as well as its mechanical properties such as stiffness.

As an alternative to the direct intention recognition, the robot can use human demonstrations (Evrard et al. 2009; Lee and Ott 2011; Peternel et al. 2014; Ben Amor et al. 2014; Rozo et al. 2015; Maeda et al. 2017) or gradual adaptation to human behaviour (Peternel et al. 2016) to learn elementary skills of the collaborative task. However, compared to the methods that are based on real-time intention recognition interfaces, off-line learning approaches fail to demonstrate the adaptability to state variations, while on-line adaptation can be relatively slow.

The common aspect of most previously proposed human–robot cooperation methods is the assumption that the human partner has a constant level of physical endurance. Such assumption is justified in simplistic tasks that require small physical effort or short execution time. However, other rough and more dynamic interaction scenarios can affect the human performance. While in (Peternel et al. 2016) the human effort is minimised due to the nature of the method, the fatigue was not estimated or monitored. Unlike a sturdy humanoid robot, the human partner is prone to the physical fatigue; a subject-dependent phenomenon that can affect his/her physical interaction capabilities rapidly and unpredictably (De Luca 1984). In such a case, the robot should be able to recognise the human fatigue and adapt its behaviour to offer additional physical support in the given task. As a result, the human partner should then only supervise the task on a cognitive level and produce less physical effort.

Extensive studies of human physical fatigue has been conducted in fields of neuroscience and physiology (De Luca 1984; Giat et al. 1993; Ding et al. 2000; Liu et al. 2002; Enoka and Duchateau 2008; Ma et al. 2009). However, human fatigue has not been well-studied in human–robot collaboration scenario. In a recent study, Sadrfaridpour et al. (Sadrfaridpour et al. 2016) used a model-based human muscle fatigue estimation (Ma et al. 2009) to adjust the working speed of collaboration in a component assembly task. The task of the robot was to pick the necessary parts and place them near the human, who then sequentially performed the assembly alone. Nevertheless, this method did not address direct physical human–robot interaction or simultaneous co-manipulation (i.e. human and robot were not physically coupled with a tool/object while performing the task). In addition, robot behaviour did not change beyond adaptation of the execution speed.

To go beyond the state-of-the-art and overcome the above-mentioned limitations, we propose a novel human–robot co-manipulation method that enables the robot to adapt and modulate the delivered physical assistance as a function of the human fatigue. The robot governs its behaviour by hybrid force/impedance controller (Peternel et al. 2017), while a machine-learning based shared control between the human and the robot is established. The robot begins as a follower and imitates the human leader to facilitate the cooperative task execution. At the same time, the robot gradually learns the task parameters from the human lead. When the robot detects the human fatigue in an online manner, it uses the learnt skill to take over the control of the physically demanding task aspects. On the other hand, the human counterpart remains in the control of cognitively complex aspects and the ones which cannot be completely produced by the robot on its own due to the collaborative nature of the task.

We propose a new model to estimate the human fatigue inspired by the dynamical response of an RC circuit. This model has a similar dynamics to the previously proposed models based on the human muscle force (Ma et al. 2009). The main difference is that our model estimates the fatigue based on the observed muscle activity obtained from the electromyography (EMG). One advantage of muscle activity based model is that it can also provide the information of effort that comes from muscle co-contraction, which cannot be observed through the measured force at the endpoint or measured torque in the joint. Another advantage is that force measurement is not required on the human side.

We demonstrate the proposed approach with experiments on KUKA Lightweight Robot equipped with a Pisa/IIT Softhand (see Fig. 1 for the setup). We validated our methods on two cooperative tasks: human–robot wood sawing and robot-assisted surface polishing.

Fig. 1

The proposed human–robot co-manipulation framework for robot adaptation to human fatigue. The robot is controlled by a hybrid force/impedance controller. The muscle activity interface provides the robot controller with information about human motor behaviour to achieve an appropriate reciprocal behaviour in different phases of the task. The human fatigue estimation system provides the robot with the state of the human physical endurance. The learning system allows the robot to acquire skill of the task execution during the cooperation with the human partner and then replicate it when the human fatigue is detected

The preliminary study was presented at 2016 IEEE-RAS International Conference on Humanoid Robots (Peternel et al. 2016). In this paper, we extended the human fatigue model to include recovery when the effort is reduced. We have further extended the previous work with novel experimental results, including multi-subject analysis of the method and validation on a new human–robot collaborative task. In addition, the article provides a more detailed description of the underlying concepts and techniques.

2 Methods

The proposed method is described by block scheme in Fig. 1. We controlled the motion of the robot by a hybrid force/impedance controller that uses information about the human intention provided by a muscle activity based human–robot interface (Peternel et al. 2017). We estimated fatigue of the collaborating human partner by a model that was based on the human muscular effort measured by the EMG. The robot learned the physical behaviour (reference trajectories) during the initial stages of collaborative task execution. We used Dynamical Movement Primitives (DMPs) (Ijspeert et al. 2003) to encode the robot motion trajectories, which were learnt online by Locally Weighted Regression (Schaal and Atkeson 1998). The robot used Adaptive Frequency Oscillators (Petrič et al. 2011) to estimate the desired task execution speed and to control the phase and frequency of the learnt DMPs.

2.1 Robot control through human intention interface

The task frame force was controlled at the robot joint-torque level

$$\begin{aligned} {\varvec{M}}({\varvec{q}})\ddot{{\varvec{q}}} + {\varvec{C}}({\varvec{q}},\dot{{\varvec{q}}})\dot{{\varvec{q}}} + \varvec{g}({\varvec{q}}) + {\varvec{J}}^T {\varvec{F}}_{int} = \varvec{\tau }, \end{aligned}$$

(1)

where ${\varvec{F}}_{int}$ is the Cartesian space interaction force/torque acting from the robot on the environment, ${\varvec{q}}$ are joint angles, $\varvec{\tau }$ are joint torques, $\varvec{J}$ is robot Jacobian matrix, ${\varvec{M}}$ is mass matrix, ${\varvec{C}}$ is centrifugal and Coriolis matrix and $\varvec{g}$ is gravity vector.

A hybrid force/impedance controller was implemented to govern the robot motion and force behaviour and was defined as

$$\begin{aligned} \varvec{F}_{int} = \varvec{F}_{for} + \varvec{F}_{imp}, \end{aligned}$$

(2)

where the term ${\varvec{F}}_{for}$ is related to the force task and performs force control in predefined axes and term ${\varvec{F}}_{imp}$ is related to impedance controlled axes.

The force tasks were controlled by a PI controller

$$\begin{aligned} \varvec{F}_{for}= & {} \varvec{K}_{P}^{F} \varvec{e}_{F} + \varvec{K}_{I}^{F} \int \varvec{e}_{F} \text {d}t, \end{aligned}$$

(3)

$$\begin{aligned} \varvec{e}_{F}= & {} \varvec{S}_F \big ( \varvec{F}_{d} - \varvec{F}_{a}\big ), \end{aligned}$$

(4)

where $\varvec{K}_{P}^{F}$ and $\varvec{K}_{I}^{F}$ are the gains of the PI controller,¹ ${\varvec{e}}_{F}$ is the error signal and ${\varvec{F}}_{a}$ and ${\varvec{F}}_{d}$ are actual and desired end-effector force values. Diagonal matrix ${\varvec{S}}_F$ is used to select the axes in which force controller is applied.

The impedance term was defined as

$$\begin{aligned} {\varvec{F}}_{imp}={\varvec{K}} \big ({\varvec{x}}_d-{\varvec{x}}_a\big ) + {\varvec{D}} \big (\dot{\varvec{x}}_d-\dot{\varvec{x}}_a\big ), \end{aligned}$$

(5)

where $\varvec{x}_a$ is actual and $\varvec{x}_d$ is reference pose of the robot end-effector, and $\varvec{K}$ and $\varvec{D}$ are robot virtual stiffness and damping matrices in Cartesian space. Stiffness matrix was controlled by recently proposed interface (Peternel et al. 2017), while damping matrix was obtained by double diagonalisation design (Albu-Schäffer et al. 2013).

In the sawing experimental task, the robot controlled the stiffness in different phases of the collaborative task by the estimated human intention. The two cooperating agents had to produce reciprocal/inverse behaviour in each phase. For example, if the human pulled the saw, the robot had to command a low stiffness to comply with the motion. In the opposite case, when the saw reached the human’s end, the robot had to became stiff to pull the saw back to the reference position at the robot’s end. The Cartesian stiffness matrix was defined as (Peternel et al. 2017)

$$\begin{aligned} \varvec{K} = \varvec{K}_{const} + \varvec{S}_{K} \big ( 1 - k_{r} \big ), \end{aligned}$$

(6)

where $\varvec{S}_{K}$ is a diagonal matrix that is used to select the axes in which the stiffness should be modulated (in our case sawing motion axis), $\varvec{K}_{const}$ is matrix that is used to set constant stiffness for the other axes and $k_{r}$ is robot stiffness modulation parameter defined as (Peternel et al. 2017)

$$\begin{aligned} k_{r} = c_{h}' ( k_{max} - k_{min} ) + k_{min}, \end{aligned}$$

(7)

where $c_{h}'= a \cdot c_{h}$ is scaled human-estimated stiffness index $c_{h}$ from the intention interface ($c_{h}' \in [0,1]$ and $c_{h} \in [0,1]$) and $k_{max}$ and $k_{min}$ are maximum and minimum controllable robot stiffness range.

Estimation of human stiffness trend was defined as (Ajoudani et al. 2014)

$$\begin{aligned} c_{h} = b_1 \frac{1- e^{-b_2\left( A_1 + A_2\right) }}{1 + e^{-b_2\left( A_1 + A_2\right) }}, \end{aligned}$$

(8)

where $A_1$ and $A_2$ are muscle activation levels of dominant muscles acting on the shoulder joint. In our experiments we used posterior deltoid (PD) and anterior deltoid (AD) muscles. The measurement of antagonistic muscles enables detection of both pulling and pushing actions. Parameter $b_1$ defines the maximum amplitude of mapping and $b_2$ defines the shape of mapping. Parameter a determines the task-related operational range of mapping. Parameters $b_1=20,b_2=0.05$ and $a=12$ determine the mapping between muscle activations and human stiffness trend, and were determined experimentally as in (Peternel et al. 2017).

Muscle activity for each muscle was measured by EMG using Delsys Trigno Wireless system. We processed (filtered and rectified) and normalised the EMG signals using maximal voluntary contraction (MVC). The mapping between the processed EMG and muscle activation level was defined as

$$\begin{aligned} 0 \le A_i(t)&= \frac{EMG_i(t)}{MVC_{i}} \le 1, \end{aligned}$$

(9)

where $A_i$ is muscle activation level for each muscle i, EMG(t) is processed EMG signal and MVC is EMG signal under maximal voluntary contraction (MVC).

2.2 Human fatigue estimation

Several models for estimation of human muscle fatigue can be found in literature (Giat et al. 1993; Ding et al. 2000; Liu et al. 2002; Ma et al. 2009). These models mostly rely on complex and time-consuming biomechanical systems, offline calibration and muscle force measurements. Although such models can provide a very accurate estimate of human muscle capacity in terms of fatigue, the underlying complexity can severely limit their application in practical daily-life robotics applications. To simplify the subject-dependent calibration procedures, we propose a novel fatigue model by observing muscle activity of the dominant involved muscles. In comparison to the use of expensive force sensors to indicate muscle fatigue, we explore the use of lightweight and wearable EMG electrodes that do not interfere with the task.

Several well-known EMG-based methods for estimating human muscle fatigue already exist in the literature. These methods are based on observing changes of EMG signal in frequency domain; such as mean amplitude and frequency of the spectrum (De Luca 1984). While this approach is straightforward for sustained constant muscular effort, the estimation can be difficult in dynamical tasks, such as collaborative sawing, where the effort is rapidly changing. To avoid this issue, we directly integrate the muscle activity value obtained from the EMG signal to estimate the fatigue.

Our human fatigue model was inspired by RC circuit dynamics. The fatigue of a certain muscle increases according to the first-order dynamics based on the current effort of the muscle, which is estimated through EMG, and the capacity of the muscle in terms of endurance, which is specific to an individual muscle. We defined the muscle fatigue estimation model with the following first-order system of differential equations

$$\begin{aligned} \frac{dV_i(t)}{dt}&= {\left\{ \begin{array}{ll} \big (1-V_i(t)\big )\frac{G_i(t)}{C_{Fi}} &{}\quad \text {if } A_i(t) \ge A_{th}\\ -V_i(t)\frac{R}{C_{Fi}} &{}\quad \text {if } A_i(t) < A_{th} \end{array}\right. }, \end{aligned}$$

(10)

$$\begin{aligned} G_i(t)&= A_i(t), \end{aligned}$$

(11)

where $V_i \in [0 1]$ represents the fatigue index, $G_i$ is parameter that represents the current effort dynamics, $C_{Fi}$ is fatigue-related capacity of the given muscle i and R is recovery rate. The recovery rate defines how fast the muscle can recover the strength after the fatigue. We selected a conservative value $R=0.5$ for our experiments. Alternatively, the recovery rate can be obtained from the literature (Ma et al. 2010). Muscle activity threshold $A_{th}$ determines the use of either fatigue or recovery mode. Parameters $C_{Fi}$ and R encode human muscle specifics in terms of fatigue and recovery and should be tuned based on the individual subject and muscles used in the given task. The higher the parameter $C_F$ is, the more effort G over time it takes for the fatigue to take effect.

Model in (10) incorporates fatigue and recovery, which correspond to charging and discharging of fatigue index $V_i$, respectively. The fatigue is induced when muscles activity $A_i$ is detected beyond some threshold $A_{th}$. When muscle activity is reduced, below the threshold $A_{th}$, the recovery is modelled by gradually reducing the value of fatigue index $V_i$.

Our model behaviour corresponds well to fatigue dynamics observed/modelled in human studies (Ma et al. 2009), where they used force measurement as the effort estimation. Their model exhibits exponential response over time subject to the level of muscle effort

$$\begin{aligned} F_c(t) = F_{MVC} e^{-\int k \frac{F_{load}(t)}{F_{MVC}}dt}, \end{aligned}$$

(12)

where $F_c(t)$ is the force production capacity of the muscle, which is related to fatigue, $F_{MVC}$ is muscle force at MVC, $F_{load}$ is load force on the muscle related to the effort and k is a time constant. By deriving from (10), a similar exponential response can be observed in our model

$$\begin{aligned} V(t) = 1 - e^{-\int \frac{G(t)}{C_{F}}dt}. \end{aligned}$$

(13)

Model in (Ma et al. 2009) deals with the force production capacity with respect to force at MVC, which exponentially decays under muscle effort. Force production capacity has essentially a reversed relation to fatigue, i.e. the fatigue lowers the force capacity. On the other hand, our model deals with fatigue directly and therefore the fatigue exhibits growth subject to the level of muscle effort. While the two models exhibit similar response in terms of fatigue, they operate with different muscle effort estimation variables, i.e. force and muscle activity. Nevertheless, the muscle activity is related to the muscle force and the relation is proportional within a certain operation range (Hogan 1984).

We estimated the parameter $C_F$ for each muscle (PD and AD) by a preliminary experiment. In this experiment, we instructed the human to produce a reference muscle activity $A_{ref}$ by exerting muscle force until he/she could not endure it any more.² We measured the time $T_{end}$ that the human could endure each reference effort $G_{ref}$ (in our experiment we used two reference efforts: 0.2 and 0.5). We obtained the fatigue capacity $C_F$ for each reference by using the subject-dependant endurance time $T_{end}$ and an assumption that full capacity is reached at $V=0.993$ (corresponding to five time constants)

$$\begin{aligned} C_F = - \frac{G_{ref} \cdot T_{end}}{\log (1-0.993)}. \end{aligned}$$

(14)

The final parameter $C_F$ for a specific muscle was obtained by an average value of $C_F$ for each reference.³

In the proposed approach, the robot takes over the production of the specified task aspect when either of $V_i$ reaches some predetermined threshold $V_{th}$. At that point, the trajectory learning stopped and the robot repeated the learnt behaviour according to the estimated execution speed. The human partner could then partially relax and recover some of the physical strength, while he/she continued to perform the collaborative aspects that the robot cannot take over by itself. Examples include: stabilisation of the saw movements at the human end in the cooperative sawing, positioning of polishing spot in cooperative polishing task, high-level decisions such as pausing and resuming the task execution, and changing the task coordinates as described in (Peternel et al. 2017).

2.3 Robot adaptation

We used periodic DMPs (Ijspeert et al. 2003) to encode the robot behaviour that was learnt from the collaboration with the human. The DMP is based on the dynamics of second-order system

$$\begin{aligned} \dot{z}= & {} \varOmega \left( {\alpha \left( {\beta \left( - y\right) - z} \right) +f} \right) , \end{aligned}$$

(15)

$$\begin{aligned} \dot{y}= & {} \varOmega z, \end{aligned}$$

(16)

where y is the encoded trajectory, f is the term that modulates the trajectory shape, $\alpha =8$ and $\beta =2$ are positive constants and $\varOmega$ is the execution frequency.

The desired motion trajectory shape $f_d$ was approximated by

$$\begin{aligned} f_d = \frac{\ddot{x_d}}{\varOmega ^2} - \alpha \left( {\beta \left( - x_d\right) - \frac{\dot{x}_d}{\varOmega } } \right) . \end{aligned}$$

(17)

where $x_d$, $\dot{x}_d$ and $\ddot{x}_d$ are the robot motion that we wish to learn and its derivatives. The desired trajectory shape $f_d$ was used to build a nonlinear shape function f in (15) defined as (Ijspeert et al. 2003)

$$\begin{aligned} f(\phi )=\frac{{\sum \nolimits _{i = 1}^N {\psi _i(\phi ) w_i} }}{{\sum \nolimits _{i = 1}^N {\psi _i(\phi ) } }}, \end{aligned}$$

(18)

where w are weights used to determine shape of trajectory and $\psi _i(\phi )$ are Gaussian kernels uniformly distributed across the phase space $\phi$. Kernels were defined as

$$\begin{aligned} \psi _i(\phi ) = e^{{h\left( {\cos \left( {\phi - c_i } \right) - 1} \right) }}. \end{aligned}$$

(19)

where h determines the width, $c_i$ centres and N number of Gaussian kernels. We selected $N=25$ in our experiments.

Each weight $w_{i}$ of kernel $\psi _i$ was updated using recursive least squares method (Schaal and Atkeson 1998)

$$\begin{aligned} w_{i}(t+1)= & {} w_{i}(t)+\psi _{i}P_{i}(t+1)re_r(t), \end{aligned}$$

(20)

$$\begin{aligned} e_r(t)= & {} f_{d}(t)-w_{i}(t)r, \end{aligned}$$

(21)

$$\begin{aligned} P_{i}(t+1)= & {} \frac{1}{\lambda }\left( P_{i}(t)-\frac{P_{i}(t)^{2}r^{2}}{\frac{\lambda }{\psi _{i}}+P_{i}(t)r^{2}}\right) , \end{aligned}$$

(22)

where $\lambda$ is forgetting factor that controls the forgetting rate of older data and was set to 0.995 for our experiments. Initial setting of parameters was $r=1$, $w_{i}(0)=0$ and $P_{i}(0)=1$ for $i=1,2,\ldots ,N$.

Phase and frequency of the learnt DMP were controlled by an adaptive oscillator (see (Petrič et al. 2011) for details on adaptive oscillators). The adaptive oscillator outputs the phase and frequency estimation of some input signal. Parameters of adaptive oscillator were set the same as in (Peternel et al. 2016). The estimation was done based on the actual robot motion $x_a$ (input signal) that enabled the human to alter the execution speed during the compliant phase of the robot motion (Peternel et al. 2014).

Fig. 2

Experimental setup for collaborative surface polishing. The arrows show the robot base frame

3 Experiments

To validate the proposed method we conducted experiments on two collaborative tasks: wood sawing and surface polishing. Experimental setup for wood sawing is shown in Fig. 1. Experimental setup for surface polishing is shown is Fig. 2. Four male subjects participated in the experiments (29.0 ± 3.6 years old). Each subject performed the preliminary calibration of the fatigue estimation model individually as described in Sect. 2.2.

In the experiments we observed the PD and AD muscles that actuate the human shoulder joint. Since these muscle contribute to control of arm endpoint through synergetic activation of other muscles (Turvey 2007), they give a simplified estimation of overall arm behaviour. The estimated fatigue threshold, in which the robot takes over some aspects of the task performance to reduce the human effort, was set to 30% for both muscles and for both experimental tasks. Considering that the selected tasks are dynamically quite demanding, we used relatively low threshold with the intention to keep the human at a good force production capacity. Muscle activity threshold $A_{th}$ for triggering the recovery mode was set to 0.1. These thresholds can be adapted if different tasks, conditions or preference require so.

Fig. 3

Results of cooperative sawing experiment for one subject. The first plot shows the motion of the robot end-effector in the sawing axis (x-axis), where blue line is reference and red line actual motion. The second graph shows the human deltoid muscle activity as measured by EMG sensors on anterior (AD) and posterior (PD) fibres. Threshold $A_{th}$ is marked by dotted black line. The third graph shows the estimated human fatigue for each muscle (blue and red lines) and corresponding thresholds $V_{th}$ (dotted black line). The fourth graph shows robot stiffness in the sawing axis. The fifth graph shows the estimated frequency of task execution (Color figure online)

3.1 Sawing

To produce the two-person sawing in a smooth manner, the robot had to remain compliant while the human was pulling the saw. In this manner, the compliant behaviour of the robot ensured the effort of the human was not obstructed. When the saw reached the human end of the motion, the human stopped the effort and became compliant. At that instance, the robot detected this human motor function through the intention interface and started pulling the saw towards its own end. This behaviour then periodically repeated to cooperatively execute the given task.

The sawing motion was aligned with the robot base frame x-axis (see Fig. 1). The robot controlled the sawing motion indirectly by impedance controller through stiffness modulation based on the information from the human muscle activity interface (6). The controllable stiffness range of the robot in x-axis was set to $k_{max}=1500$ N/m and $k_{min}=75$ N/m. The contact force between the saw blade with the beam of wood was maintained in z-axis by the force controller. The reference contact force was set to −5 N. The beam of wood was aligned with y-axis in which the robot was compliant ($k=0$ N/m) in order to let the human guide it to the desired incision location along the beam.

For the video of the experiment please refer to the accompanying Online Resource 1. The results of the experiment for one subject are shown in Fig. 3. The first graph shows the motion of the saw blade in the sawing axis (x-axis). Initially, the robot had a fixed position reference at its own end of the saw motion and simply followed the human lead. When the human was stiff to pull the saw (see muscle activity in the second graph), the robot reciprocally remained compliant (see robot end-effector stiffness in the fourth graph), and vice-versa.

Fig. 4

Results of cooperative sawing experiments for other subjects. The first row graphs show the robot motion, the second row graphs show muscle activity and the third row graphs show estimated fatigue

The robot used the proposed model to estimate the collaborating human’s fatigue (see the third graph). When the estimated fatigue reached the predefined level, the robot increased the stiffness and used the learnt sawing motion trajectory to take over this physically demanding aspect of the task and offloaded the human partner. The task was executed with the frequency estimated from the human lead (see the fifth graph). The human could then relax to recover some of the strength, while supervising cognitive aspects and producing physical aspects of the task that the robot cannot fully take over (e.g., stabilisation of the saw at the human’s end).

By observing the third graph we can see the evident decrease of human muscle activity after the robot adapts its physical behaviour to reduce the human effort. When the human exerted the muscular effort, the estimated fatigue gradually increased according to the fatigue model (10). After the robot took the lead and the human could relax, the estimated fatigue started to gradually reduce according to the recovery model (10). The same can be observed in Fig. 4, where results of experiments on the rest of subjects are presented. On average, the muscle activity in the initial stage was 19.2 ± 11.2%.⁴ After the robot took over the performance of the physically demanding aspect of the sawing task, the human muscle activity was on average for 6.0 ± 2.7%.

When the fatigue reaches the lower threshold, the robot can either give the control partially back to the human, or continue to execute the task with the learnt skill. In the former case, the method can revert to the initial collaboration stage [i.e. behaviour described by (6)] where the robot can repeat the learning to adapt to any potentially changed human behaviour.

3.2 Polishing

The cooperative polishing was performed by the human holding the polishing machine together with the robot. The robot compensated the weight of the machine the entire time, while the human was controlling the position of the polishing on the surface. Initially, the human guided the machine toward the desired surface and established the contact. Then he/she started to produce the force perpendicularly to the surface in order to establish an appropriate friction force vector for the polishing action. Unlike the sawing task, this is non-periodic task and frequency estimation system was not required.

The force production direction was aligned with the robot base frame x-axis. The robot used force controller in this axis and the reference force was learnt from the human online. The surface was aligned with y–z plane of the robot base frame. The robot was compliant in this plane ($k=0$ N/m) in order to allow the human to select and change the location of the polishing.⁵

For the video of the experiment please refer to the accompanying Online Resource 1. The results of the experiment for one subject are shown in Fig. 5. By observing the first graph we can see that the measured force increased when the human guided the tool to the surface and started to perform the task. As a result of this, human muscle activity increased (see the second graph). While the human exerted the force necessary to produce the friction for the surface polishing, the estimated fatigue gradually increased (see the third graph). During this time, the robot learnt the desired force from the cooperating human. In the learning process the robot used an average of measured force in 5 s time window. When the estimated fatigue reached the predefined threshold, the robot used the learnt force as a reference and took over the control of the force production task.

Fig. 5

Results of cooperative polishing experiment for one subject. The first graph shows the reference (blue line) and measured (red line) force perpendicular to the surface (x-axis). The second graph shows the human muscle activity. The third graph shows the estimated human fatigue for each muscle. The fourth graph shows the measured robot end-effector position in the plane of the surface (y–z plane) (Color figure online)

While the robot performed the force production task, the human could partially relax to recover the strength. In the meantime, the human remained in the control of the cognitive aspects, such as overall task supervision and choosing the location of polishing action in y–z plane. We can see how the human was changing the position of the task production in the surface plane by observing the fourth graph.

The results for the other subjects are shown in Fig. 6. The average human muscle activity in the initial stage was 17.2 ± 1.7%. After the robot took over the execution of the physically demanding force production task aspect, the human muscle activity was on average 4.6 ± 1.1%.

Fig. 6

Results of cooperative polishing experiments for other subjects. The first row graphs show the robot motion, the second row graphs show muscle activity and the third row graphs show estimated fatigue

4 Discussion

We proposed a new method for adaptation of robot physical behaviour based on human muscle fatigue estimation for human–robot co-manipulation tasks. We validated the proposed method experimentally with challenging co-manipulation tasks. The main aim of robot adaptation was to reduce some effort of the human partner after the robot estimated the human fatigue level beyond the predefined threshold. The robot started as a follower with a minimal knowledge of the task and gradually obtained the skill of the task while performing it collaboratively with the human partner. When fatigue threshold was reached, the robot used the obtained skill to take over the aspect of the task that it could perform by itself. On the other hand, the human continued to perform the aspects of the task that are of collaborative nature and cannot be performed by a single agent (i.e. stabilisation of human’s end of the saw). Additionally, the human controlled the cognitive aspects of the task during the entire experiment.

The method helps to considerably reduce the human muscular effort in collaborative task performance. When the robot takes over the execution of physically demanding aspects of the task, the human can partially relax and recover the strength. While a considerable amount of effort can be reduced due to the robot’s assistance, the muscle activity may not be zero, since the human may still require to make some effort to ensure the collaborative task is performed. The exact human physical requirement is task-dependent.

In the current setup the robot continued to learnt the behaviour until the fatigue reaches the threshold $V_{th} = 30\%$. However, the human performance can degrade with fatigue and the robot can potentially learn suboptimal behaviour. If higher thresholds $V_{th}$ are used, it is reasonable to consider to stop learning before the fatigue reaches such a threshold in order to sample the human at its peak performance. Alternatively, the learning process can be weighted so that data sampled at lower human fatigue has more impact on the learnt robot behaviour. Furthermore, the method could be upgraded to incorporate more complex shared control systems (Dragan and Srinivasa 2013; Nikolaidis et al. 2016; Peternel et al. 2016) to facilitate a continuous co-adaptation based on fatigue and current state of robot skill.

To give the robot an estimation of the human muscle fatigue level, we proposed a new fatigue model based on the measured human EMG signals. More complex models with more complicated calibration procedures and muscle force measurements might give more precise estimation. However, this kind of complexity can hinder the application of such system in real-world scenarios, therefore the trade-off between complexity and applicability should be considered.

Initially, we tried using the median frequency of power density spectrum of measured EMG signal to estimate the human muscle fatigue. It has been shown that the median frequency of power density spectrum decreases and its amplitude increases as a result of muscle fatigue (De Luca 1984). While this kind of estimation was good in case of constant muscle contraction as observed in (De Luca 1984), in case of sawing task the observed estimation was unpredictable and inconsistent. This can probably be attributed to rapid changes in muscle activation amplitude that are required due to the dynamic nature of the selected co-manipulation task. Such changes probably affected the dynamics of observed median frequency. In addition, in such case it is difficult to determine the appropriate sampling window to track the median frequency changes throughout the task execution.

The main advantage of the proposed fatigue model is its easy application and low complexity. Compared to other low-complexity models that are based on force measurement (Ma et al. 2009), our model does not require force sensors. While measuring arm force at the endpoint or joint torque can be straightforward, it can only measure the combined contribution of the entire arm or joint. On the other hand, EMG can be placed to individual muscles and effort can be estimated separately. However, if muscle force can be directly measured, the effort estimation might be better since EMG measurement is subject to a certain amount of noise. The main limitation of the proposed fatigue model compared to more complex models (Giat et al. 1993; Ding et al. 2000; Liu et al. 2002) is limited precision. For example, more precise models include representation of muscle activity on a fibre level. In such a model the muscle fibres are split into there groups (active, fatigued and resting units) and their function is considered separately (Liu et al. 2002). However, the precision and complexity may limit the model application compared to less complex models.

The proposed method could be applied in industry to prevent productivity and health issues related to overwork and physical fatigue. The estimated fatigue could be used to pull any particular worker from the production line when a certain level of fatigue is estimated. Such procedures could potentially be effective in reducing negative socioeconomic impacts associated with excessive workload and injuries at workplace and maintain good productivity of workers.

The scope of this paper was to introduce and experimentally validate the proposed human–robot collaboration method. Our current goal was to devise easy-to-use and easy-to-calibrate human fatigue model that can be applicable in practical human–robot collaboration tasks. In future, we will consider a more detailed study of the fatigue model from biomechanical perspective.