Teleimpedance Control: Overview and Application

Abstract

In previous chapters, human hand and arm kinematics have been analyzed through a synergstic approach and the underlying concepts were used to design robotic systems and devise simplified control algorithms. On the other hand, it is well-known that synergies can be studied also at a muscular level as a coordinated activation of multiple muscles acting as a single unit to generate different movements. As a result, muscular activations, quantified through Electromyography (EMG) signals can be then processed and used as direct inputs to external devices with a large number of DOFs. In this chapter, we present a minimalistic approach based on tele-impedance control, where EMGs from only one pair of antagonistic muscle pair are used to map the users postural and stiffness references to the synergy-driven anthropomorphic robotic hand, described in Chap. 7. In this direction, we first provide an overview of the teleimpedance control concept which forms the basis for the development of the hand controller. Eventually, experimental results evaluate the effectiveness of the teleimpedance control concept in execution of the tasks which require significant dynamics variation or are executed in remote environments with dynamic uncertainties.

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Teleimpedance Control

The need to execute tasks in unstructured or hostile environments has lead to the development of several Master-Slave teleoperation interfaces, commonly recognized by two classes: unilateral, position-based and bilateral, force-reflecting teleoperation systems. The most basic teleoperation interface receives position commands from the master and replicates them on the slave side in an open-loop fashion. Despite the stability and simplicity of such systems, generation of high interaction forces between the rigid manipulator’s end-effector and the uncertain environment has severely limited their application in real-world scenarios.

Later generations of teleoperation interfaces thus investigated new techniques to provide the master with information about the interaction forces between the manipulator and the remote environment. Although it has been demonstrated that force-reflecting teleoperation interfaces outperform the unilateral ones [26, 27], several drawbacks such as imposed additional costs (due to the need for force measurement devices), transparency, or even stability issues (due to the latencies in the communication channel between the master and slave robot) reduce the efficiency of such systems [18, 31, 41]. In fact, despite the continuous advancement in hardware design and software architecture of the bilateral teleoperation interfaces, still a large class of tasks (e.g. drilling and chipping) which are intuitively executed by humans cannot be effectively performed.

Indeed, humans are able to establish a reliable contact between the limb endpoint and the object/environment by generating task-efficient restoring forces in response to the environmental displacements [23, 24]. This behavior arises from effective modulation of the task impedance which appears to be carried out using different techniques. One way to achieve this is by the co-contraction of muscle groups acting on the limb. Alternatively, it is performed through adaptations in the sensitivity of reflex feedback [5] or selective control of the limb configuration [43].

To realize a similar interaction performance in a teleoperation setup, firstly, rigid slave robots must be replaced by compliant ones to enable task impedance modulations. In this direction, drawing inspiration from the compliant structure of the human limb, the soft robotics design (either by torque control techniques [7, 9] or using passively adaptive joints [10, 30, 42] has provided the possibility of teleoperating compliant slave robots to accomplish a task in an uncertain environment. However, the second, and probably bigger, issue relates to the planning of the variable impedance slave robot to accomplish the task.

To further address this issue, the concept of teleimpedance control [2], as an alternative approach to the force-reflecting teleoperation has been proposed. In teleimpedance, a compound reference command which includes the desired motion and impedance profiles of the operator, is obtained using a suitable Human-Machine Interface (HMI) and realized by a compliant slave robot in real-time. Therefore, from one side, the need for an appropriate and real-time modeling/measurement of the operator’s stiffness and position trajectories is highlighted. On the other hand, robust and effective impedance control techniques must be implemented to replicate the operator’s reference commands on the slave robot in real-time.

Concerning the modeling of the master’s impedance trajectories, in our studies, electromyography (EMG) signals, which are formed by superimposed patterns of activations of involved motor units, are considered as the process input. This is due to the high correlations between the muscle activations, generated muscular forces and the consequent joint torques. Furthermore, easy accessibility, fast adaptivity and stability of EMG signals are other advantages which motivated our choice of adopting EMGs in the real-time control of our teleimpedance system. An alternative approach to teleimpedance for processing multiple EMG signals through machine learning as a direct input to external devices with a large numebr of DOFs is presented in the next chapter (Chap. 11).

EMGs can also be used to provide information on the limb posture, which has been used e.g. for classification of hand gestures [13, 20] or arm movements [44]. However, since EMGs directly relate to muscle forces and not limb configurations, their application to extract position references has to be indirect. A classical way to achieve this is to relate muscle forces to the limb postures using inverse dynamics methods [17]. The problem becomes even more intricate once the external forces (e.g. object mass) act on the master’s limb which would affect the position estimation accuracy. Therefore, we tend to maximize the use of the external tracking systems to extract the position profile of the master, and use EMGs to estimate the task-appropriate stiffness profiles (the static component of an impedance profile) in real-time.

While on the slave side, relying on the task requirements and the slave robot’s compliant structure, robust Cartesian or joint impedance control techniques can be implemented to realize the operator’s reference commands. For instance, in [6], some techniques for the Cartesian impedance control of torque controlled robots are provided. Additionally, in [3, 8], the role of robot configuration in Cartesian stiffness control is discussed, particularly for robots with passive elastic joints or the ones in which not enough degrees of freedom are available to realize a full desired Cartesian stiffness matrix.

In this chapter, we review some of the work done within The Hand Embodied (THE) project regarding teleimpedance control of a robotic arm and an anthropomorphic robotic hand. In particular, in Sect. 10.2.1, a 3D model of the human arm endpoint force/stiffness will be introduced. Consequently, a Cartesian stiffness controller is developed to replicate the estimated stiffness profile and tracked wrist trajectories of the master by a 7-DOF torque controlled robot in real-time. Experimental results are provided to evaluate the efficiency of the proposed algorithm in rendering a desired interaction performance while performing dynamic tasks or the ones executed by the slave robot in a remote uncertain environment.

Meanwhile, in Sect. 10.2.2, teleimpedance control of the anthropomorphic and synergy-driven robotic hand, described in Sect. 6, is studied. In this setup, the hand postural and synergy reference commands (as defined in Chaps. 2, 3 and 5) are estimated using an antagonistic pair of muscles on the forearm. Two tactile interfaces, namely mechano- and vibrotactile, are developed to provide the user with some information about the grasping forces and the environment/object’s texture. Grasp robustness and improved interaction performance using teleimpedance control are evaluated through grasping experiments.¹

2 Application

2.1 Teleimpedance Control of a Robotic Arm

This section reviews some of the work done in [1, 2] aimed at remote impedance control of a 7-DOF torque controlled robot arm in real-time. Here, the compound reference command consists of master’s wrist position and stiffness profiles. Corresponding to the high priority given in teleoperation interfaces to position accuracy, our teleimpedance interface uses accurate measurement of arm position references through an optical tracking system. Meanwhile, we acquire and process eight EMG channels to estimate the 3D arm endpoint stiffness² profile in real-time, as we will elaborate in the following section.

2.1.1 Arm Endpoint Impedance Modeling in 3D

It has been demonstrated that variations in viscoelastic components of the human arm endpoint strongly correlate with the patterns of activations of the involved muscles in task execution [19, 36, 40]. While this dependency appears to be highly nonlinear in general (due to the nonlinear nature of the EMG-to-Force mapping [32] and the joint-angle dependency of the moment arms), it can be safely and almost accurately implemented by a linear mapping in a constant configuration of the arm [19, 40]. To that end, the overall mapping between EMG measurements and consequent arm endpoint force and stiffness variations in Cartesian coordination in a constant joint configuration of the arm can be described by

$$\begin{aligned} \left[ \begin{array}{c} F \\ \sigma \end{array}\right] = \left[ \begin{array}{c} T_F \\ T_\sigma \end{array}\right] P + \left[ \begin{array}{c} 0 \\ \sigma _0 \end{array}\right] , \end{aligned}$$

(10.1)

where ${F, {\sigma } \in \mathbb {R}^{3}}$ are the endpoint force and stiffness vectors, respectively, $\sigma _0$ is the intrinsic stiffness in relaxed conditions. ${P}\in \mathbb {R}^{n}$ is the vector of muscular activities of the n considered muscles, as obtained from preprocessing EMG signals which includes high-pass filtering, full-rectification, low-pass filtering and normalization stages.

In an ideal condition, force (${T}_{{F}}$) and stiffness (${T}_{{\sigma }}$) mappings can be experimentally identified through a rich and varied set of data samples both for force and stiffness measurements. While the first measurement can be easily and accurately carried out using 6 axis force/torque sensors, accurate identification of the EMG-to-stiffness map $T_\sigma$ is more difficult [36]. The reason for that lies in the difficulty of the endpoint stiffness measurements which is commonly and traditionally carried out by perturbing the wrist and probing the restoring forces [35].

To address this problem, we identify a basis for ${T}_{{\sigma }}$ using straightforward and accurate force measurements, and acquire a smaller set of the endpoint stiffness data to calibrate this mapping. This is achieved by taking into account that, in general, end-point impedance has three components, depending on posture, force, and co-contraction, respectively. While the first two components may be large and even dominating [33] in a large enough range of variations, an ample literature reports the existence and independence of co-contraction contribution to stiffness: e.g. [5, 16, 34]. In addition, in our experiments, the master will perform the tasks in a fixed arm configuration with no significant generation of the endpoint force. Therefore, we consider a decomposition of the space of muscular activations $\mathscr {P} \ni P$ as the direct sum of a force-generating subspace $\mathscr {P}_F$ and the force-map null space $\mathscr {P}_k = \text {ker} \{T_F\}$, i.e.

$$ \mathscr {P} = \mathscr {P}_F \oplus \mathscr {P}_k. $$

By choosing a right-inverse $T_F^R$ of $T_F$, i.e. any $n \times 3$ matrix³ such that $T_F T_F^R = I$, we also affix a system of coordinates to these subspaces. In these coordinates, we can decompose the vector of muscular activations P in a force-generating component $P_F$ and a null-space component $P_k$ as

$$ P = T_F^R T_F \, P \, + \, \left( I-T_F^R T_F \right) \, P \mathop {=}\limits ^{def} P_F + P_k . $$

The null space component $P_k$ contains information on the co-contraction component of stiffness generation. It is convenient to give an alternative description of $P_k$ as follows. Let $N_F$ denote a basis matrix for the kernel of $T_F$, and let $\lambda = N_F^+ P_k = Q P$ be the coordinates in that basis of $P_k$, where $Q \mathop {=}\limits ^{def} N_F^+ \left( I-T_F^R T_F \right)$. Hence the model of cartesian stiffness regulation through co-contraction is written as

$$\begin{aligned} \sigma - \sigma _0 = M_\sigma \, Q\,P \end{aligned}$$

(10.2)

where ${M}_\sigma \in \mathbb {R}^{3 \times 5}$ is a mapping from the kernel of $T_F$ (the set of muscle activations that do not change endpoint force, in the selected coordinate frame) to stiffness variations. The map $M_\sigma$ can then be identified and calibrated once, based on direct measurements of human arm end point stiffness, at different coactivation levels as described in the following section.

2.1.2 Stiffness Model Calibration-Identification

Identification of $M_\sigma$ is carried out through two sets of experiments. The first set concerns the identification of the EMG-to-Force mapping and is performed by the measurement of endpoint forces and eight channel EMG electrodes (see muscle names in Table 10.1). In this set, a KUKA LWR was serving only as a support structure for a 6 axis F/T sensor (ATI Inc.) mounted at endpoint of the arm (see Fig. 10.1). The subject was asked to apply constant forces of $\pm 5$N, $\pm 10$N, $\pm 15$N and $\pm 20$N, respectively, along 6 directions ($\pm x$, $\pm y$ and $\pm z$) while holding the handle (isometric conditions). A graph with three colored bars on the screen was used to provide the user with the information about the measured force components. Each trial was 60 s long. Data from the first 10 s were discarded to eliminate transient force fluctuations. For each direction and force level, 4 trials were executed and recorded (for an overall number of $4 \times 3 \times 2 \times 3$ trials) in EMG-to-force map identification experiments. Consequently, the mapping $({T}_{{F}})$ was identified by means of a least-squared-error algorithm, and a basis of its nullspace and the projector matrix Q were computed.

Table 10.1 Muscles used for EMG measurements

Fig. 10.1

Experimental setup used for the first calibration experiments. Subject applies constant forces in 6 directions while holding the handle attached to an idle spherical joint

In the second set, with the purpose of off-line calibration of the EMG-to-Stiffness mapping, the subject’s arm endpoint impedance profile was measured in different levels of muscular co-contraction. Following Perreault et al. [38], continuous stochastic perturbations with the maximum peak-to-peak value of 20 mm were applied to the subject’s wrist through the handle in x, y and z directions (see [2] for details). A KUKA robot with fast research interface [39] was programmed and controlled in position to apply the desired perturbation profile. Subject’s wrist position and restoring force profiles were synchronously measured using an optical tracking system (NaturalPoint, Inc.) and a FT sensor, respectively. A rough co-contraction indicator was graphically shown consisting of a bar of length proportional to the norm |P| of the vector of muscle activations. Four levels of stiffness reference were provided in different trials, where the first level (minimum muscle activity), was aimed at the identification of the intrinsic stiffness profile $\sigma _0$.

Identification of the endpoint impedance profiles in different levels |P| consisted of two non-parametric and parametric identification procedures. Firstly, multiple-output (MIMO) dynamics of the endpoint impedance was decomposed into the linear subsystems associating each input to each output. Based on this assumption, and indicating with ${F}_{x}(f)$, ${F}_{y}(f)$ and ${F}_{z}(f)$ the Fourier transforms of the endpoint force along the axes of the Cartesian reference frame, with x(f), y(f) and z(f) the transforms of the human endpoint displacements, the dynamic relation between the displacements and force variations can be described by

$$\begin{aligned} \begin{bmatrix} {F}_{x}(f) \\ {F}_{y}(f) \\ {F}_{z}(f) \\ \end{bmatrix} = \begin{bmatrix} {G}_{{xx}}(f)&{G}_{{xy}}(f)&{G}_{{xz}}(f) \\ {G}_{{yx}}(f)&{G}_{{yy}}(f)&{G}_{{yz}}(f) \\ {G}_{{zx}}(f)&{G}_{{zy}}(f)&{G}_{{zz}}(f) \\ \end{bmatrix} \begin{bmatrix} x(f) \\ y(f) \\ z(f) \\ \end{bmatrix} \end{aligned}$$

(10.3)

Fig. 10.2

Non-parametric (solid lines) and parametric second order (dotted lines) transfer functions of SISO impedance subsystems obtained from stochastic perturbations

A non-parametric algorithm was adopted to identify the empirical transfer function of each of the SISO subsystems described above in frequency domain (MATLAB, The MathWorks Inc.). Consequently, we adopted a parametric, second order, linear model of each impedance transfer function of the type

$$\begin{aligned} {G}_{{ij}}(s)={I}_{{ij}}s ^{2}+{B}_{{ij}}s +{K}_{{ij}}, s =2\pi \textit{f}\sqrt{-1} \end{aligned}$$

(10.4)

where I, B and K denote the endpoint inertia, viscosity and stiffness matrices, respectively. The parameters of the second order linear model were identified based on least squares algorithm in frequency range from 0 to 10 Hz.

Eventually, experimental EMG vectors P were mapped in the EMG-to-force map nullspace through the previously computed projector matrix Q. The elements of the stiffness matrix K were used as estimates for the components of $\sigma$, and the map ${M}_{{{\sigma }}}$ was estimated by applying a least-squared-error method to (10.2).

The strength of linear dependency between measured force signals and estimates via the least-squared-error identification of the components of ${T}_{{F}}$ was evaluated by Pearson’s product-moment correlation coefficient. The coefficient is defined as

$$\begin{aligned} {R_{k}}=\frac{\sum \hat{f}_{k}{f}_{k} -\frac{\sum \hat{f}_{k} \sum {f}_{k}}{N}}{\sqrt{( \sum {{f}_{k}}^{2}-\frac{({\sum {f}_{k}})^{2}}{N})( \sum {{\hat{f}_{k}}}^{2}-\frac{({\sum \hat{f}_{k}})^{2}}{N})}}, \ \ k=x, y, z \end{aligned}$$

(10.5)

where ${f}_{k}$ and $\hat{f}_{k}$ are measured and estimated values of force in the Cartesian directions, and N is the number of pairs of data. The fit was consistently good in the three directions, resulting in average ${{\textit{R}}}^{{2}} = 81\,\%$.

Figure 10.2 demonstrates the results of non-parametric and second order model identification of the hand impedance transfer functions in the frequency range from 0 to 10 Hz, according to methods described above. The second order parametric impedance models presented $69.7\,\%$ of the data variance across all directions in minimum muscular activity trials in the frequency range of 0–10 Hz.

Once the EMG-to-stiffness mapping was calibrated, it was used for the real-time estimation of the 3D endpoint stiffness matrix of the operator using EMG measurements of the muscles as illustrated in Table 10.1.

2.1.3 Experiments

The efficiency of the teleimpedance approach in rendering a desired interaction performance while executing tasks with dynamic requirements was evaluated in a ball reception task. In this experiment, two identical rigid balls ($m=0.92\,{\text {kg}}$, radius 52.5 mm) were suspended at the same distance above the human and robotic arm endpoints. The subject was prepared to receive the ball and instructed to hold his arm in a posture very close to that used during calibration experiments. The slave arm position, under gravity compensation, was corresponding.

The subject was instructed to receive the ball and stabilize its position in a natural way. The position of the slave endpoint was controlled along the master’s wrist trajectory while executing the task, whereas the Cartesian stiffness values were commanded and controlled in three different approaches: In the first approach, the Cartesian stiffness of the slave endpoint was set to a relatively high, constant level ($K= [1200, 1200, 1200]$ N/m) throughout the task. The second one was analogous, with low constant stiffness values ($K= [120, 120, 120]$ N/m). In the third approach, variable impedance was implemented in three directions, as estimated from the stiffness model. Damping coefficient in all experiments was set to a constant value of 0.7 N.s/m.

The experimental setup and information flow are shown in Fig. 10.3. A body marker was attached to the wrist aiming at reference trajectory calculation for robot motion. The robot base frame was considered as the overall reference frame for other frames (tracking system and FT sensor). The position path of the human wrist was measured, low-pass filtered (cutoff 15 Hz) and used for trajectory planning. At the same time, EMG signals were acquired from the master arm and used to evaluate its endpoint stiffness based on the model and calibration described in the previous section. All processing and control algorithms were performed in real-time in C++ environment. KUKA interface was similar to the ones explained in identification trials.

Fig. 10.3

Experimental setup of the ball-catching experiments. The slave KUKA LWR arm, EMG electrodes, position tracking markers and F/T sensor are shown

2.1.4 Experimental Results

The measured forces at the endpoint of the slave robotic arm while executing the task in the three stiffness control modes (constantly high, constantly low, and teleimpedance) are reported in Fig. 10.4, while the corresponding deviation errors from the reference equilibrium position are in Fig. 10.5. The regulation of the human arm muscle activations and resulting endpoint stiffness modifications during the catching experiment are shown in Fig. 10.6. Increased stiffness at the time of impact and its progressive decrease afterward are the results of explicit muscular activity regulation by the subject.

Fig. 10.4

Measured force values in z direction during the task with the slave robotic arm under constantly high, constantly low, and teleimpedance stiffness control

Fig. 10.5

Absolute tracking position error in z direction during the task with the slave robotic arm under constantly high, constantly low, and teleimpedance stiffness control

As expected, the stiffer the arm, the smaller the deviation, as seen in the experimental results under constant high stiffness (Fig. 10.5, left). The tradeoff for the accuracy and reduced deviation from equilibrium position with high values of endpoint stiffness is the occurrence of bouncing: indeed, the second force peak (at $t \approx 5.26\,\mathrm{s}$) in the stiff case (Fig. 10.4, left) shows a second impact of the ball (see also Extension 1).

To obtain a more stable contact is to reduce the endpoint stiffness values; however, using constantly low stiffness directly affects the position deviation, which may grow to very large, possibly unacceptable values (Fig. 10.5, middle). Another drawback of such compliant control is the insufficiency of generated torques for repositioning the ball to its equilibrium even after transient end.

The transient behavior of the system under teleimpedance appears to benefit from the active control of stiffness, increasing at the very first moment of impact (from $t\approx 2.3\,\mathrm{s}$ to $t\approx 2.4\,\mathrm{s}$), leading to a reduced deviation from reference equilibrium position. Also, the bouncing phenomenon appears to be avoided due to the subsequent stiffness reduction phase (between $t\approx 2.4\,\mathrm{s}$ to $t\approx 2.7\,\mathrm{s}$, see Fig. 10.6). This behavior is in accordance with previous studies which have shown the capabilities of the human body to minimize soft-tissue vibrations and impact transitions by means of increased damping or decreased stiffness (modified resonance frequency) within involved tissues (see e.g. [45]). In addition, other behavioral studies demonstrated an increase of cocontraction levels in human arm while performing tasks which need quick torque generations and/or to cancel components of torques orthogonal to the desired direction [25].

Fig. 10.6

Fully rectified eight channel raw EMGs (upper plot) and estimated and mapped endpoint stiffness (lower plot) in real-time for ball-reception task

Table 10.2 Performance indexes

A comparative performance analysis of the three control methods was done by defining different indexes, which are summarized in Table 10.2. LOI is computed as the integral of the difference between the vertical component of wrist force $F_z$ and its steady-state value (i.e., the hand plus ball weight $F_w$), where $\varDelta t$ is the time interval duration between the first impact and steady stabilization. A high value of the “lift off index” LOI indicates a reception with multiple bouncing and/or long underdamped ball trajectories.

The second index is the maximum deviation from the equilibrium position in z direction at steady state. As a third index, we consider an estimate of the damping ratio of the bouncing phenomenon, experimentally estimated (see e.g. [21]) using the logarithmic decrement between the first and second force peaks. Fourth and last, the “bouncing time index” BTI was introduced as the duration of the interval during which contact between the ball and robot’s end effector is completely lost. The value is calculated by summing the intervals $\varDelta t_B$ along which $f_{{z}}$ is zero (complete disconnection) or positive (as result of acceleration of bowl) after the first impact.

Figure 10.7 shows the values obtained in experiments for the four indices in the three different stiffness regulation modes. Teleimpedance control appears to strike a good compromise among the two extremes, consistently scoring close to the best performance obtained by either of the two constant settings, thus enabling the human ability to to be effectively transferred to the slave arm.

Fig. 10.7

Performance index plots over different elastic endpoint profiles (S Stiff, C Compliant and T Teleimpedance)

2.2 Teleimpedance Control of a Robotic Hand

Following the implementation of teleimpedance control for teleoperation of a robotic arm described above, we began to explore translating this approach to the control of a prosthetic hand. Although using a prosthesis is not typically thought of as a teleoperation scenario, the user is driving a terminal device in real-time often with only visual feedback as guidance. The control of these devices is suboptimal and research strategies including incorporating feedback, machine learning, and peripheral technology are being investigated. This section reviews the initial steps towards implementing a teleimpedance prosthetic controller [22] and further refinement [4] of this technique.

2.2.1 Pisa/IIT SoftHand

The Pisa/IIT SoftHand was developed through a collaboration between the University of Pisa and the Istituto Italiano di Tecnologia. The SoftHand was used in the experiments presented below and will be described here in brief. For a more detailed description, please see [14] and Chap. 8.

The goal of the SoftHand was to design and build a robotic hand that is highly functional yet simple and robust. This was achieved by combining the soft synergies approach [11] with underactuation [12]. The former uses human hand grasping synergies as a reference position for a virtual hand. The virtual hand position or stiffness profile connecting the virtual and real hands can thus be varied to control the interaction forces between the hand and the environment. The latter employs fewer actuators than available degrees of freedom, thus lowering cost, weight, and complexity of the device. Underactuation also imparts a degree of adaptability to the hand, thus the combination of these techniques was termed “adaptive synergies.” Additionally, to make the hand more robust and safer in human-robot interaction scenarios, the hand was designed with soft robotics principles in mind: the fingers can be bent, twisted, struck, etc., and will deform out of the way and then return to their original conformation, protecting both the hand and the environment from damage in the event of a collision. The SoftHand is anthropomorphic and contains a single motor. This motor pulls a tendon that winds through the fingers and thumb to simultaneously flex and abduct the fingers. To enable testing of the hand with non-amputee subjects, a forearm adapter was employed, see Fig. 10.8.

Fig. 10.8

SoftHand equipped with the able-bodied adapter (forearm support) and fingertip accelerometers

2.2.2 Initial Evaluation

A pilot experiment is presented in [22]. In this study, we implemented a standard proportional and a novel impedance controller with and without vibrotactile feedback using MATLAB Simulink and the Real-Time Windows Target (Mathworks, Inc.). The main finger flexor and extensor muscles (the flexor digitorum superficialis (FDS) and extensor digitorum communis (EDC), respectively) were sampled using surface electromyography (EMG) electrodes. With both controllers, the reference position of the hand was proportional to EMG amplitude. With impedance, the stiffness with which the proportional control was applied was based on the average of the flexor and extensor EMG signals. Both control modes were also tested with vibrotactile feedback applied using a small (7 $\times$ 2 mm) eccentric mass motor (Precision Microdrives Ltd.). The amplitude and frequency of the feedback was proportional to the grasping force. When an object is grasped, interaction forces occur as the reference position moves inside the object. In this way, the error between the reference and measured position can be used to estimate the grasping force.

In testing, each subject attempted to grasp four everyday objects of varied size and weight (water bottle, screwdriver, spray bottle, and ball; see Table 10.3) with each type of control mode. In total, four controls modes were tested: standard (proportional), impedance, vibrotactile (standard with feedback), and vibrotactile-impedance (VI). Each grasp was attempted three times for each of the objects and control modes, resulting in 48 grasps per subject. Mode and grasp order was fixed, but subjects were allowed a brief familiarization period in each condition to minimize learning effects. After each condition, subjects were also asked to evaluate the amount of physical and mental exertion required using a 5-point Likert scale. After all conditions were completed, subjects were also asked whether each feature (impedance and feedback) made the hand easier to use and whether the combination made the hand easier to use.

Table 10.3 Dimensions and weights of test objects

While only a pilot experiment, results suggested using teleimpedance in prosthetic control could provide an improvement in control of the prosthetic hand and subsequently the user’s experience. Grasp success rate was above 90 % in all conditions, implying that the SoftHand was generally easy to control with minimal training and its conformal grasp was effective. Figure 10.9 shows the quantitative EMG results including duration of EMG activity (left) and cumulative and average EMG amplitudes (right). Subjects spent the longest time above minimum EMG thresholds in standard mode, less time with impedance and vibrotactile modes, and finally the shortest time in VI mode. Cumulative EMG was used as a proxy for physical exertion. This was highest again in standard mode, lower in vibrotactile, and lowest in impedance and VI modes. Average EMG amplitudes were similar across the conditions; the variations showed that subjects had a tendency to contract more with feedback in either control modality, and less in impedance mode. Because the motor used to provide feedback had low resolution, it is possible subjects produced larger contractions to increase their benefit from this feedback. Finally, the qualitative results from the Likert surveys mirrored the quantitative results: subjects reported lower mental and physical effort with impedance and vibrotactile modes compared to standard and lowest with the VI mode. These results suggest that both impedance control and vibrotactile feedback provide improved prosthetic control and user satisfaction. It is worth noting, however, that grasp success rates were still high without these features and that order effects had a potential influence on the results.

Fig. 10.9

Time spent above threshold averaged across subjects (top). Average FDS EMG amplitude (bottom, bars) and cumulative EMG (bottom, line). a Duration of EMG activity, b EMG activity

2.2.3 Extension of the Hand Controller

Following the results of the pilot experiment, more advanced versions of the above proportional and teleimpedance controllers were developed. The goal was to map the FDS and EDC EMG signals to position and stiffness models to a achieve more accurate control for each subject. Subjects then attempted to grasp everyday objects with three types of controllers: stiff, using the position model and a fixed, high impedance value; compliant, using the position model and a fixed, low impedance value; and teleimpedance, using a varying control gain based on the users’ stiffness profile. A block diagram of the control scheme employed is presented in Fig. 10.10. Further, two haptic interfaces were included. In the first, a mechanical version of the force feedback described above was developed. A mechanical cuff tightened around the upper arm as grasp force increased so as to provide modality-matched feedback to the user. In the second, surface texture was measured by placing accelerometers on the SoftHand fingertips and then replicating the measured vibrations with a bracelet of eccentric mass motors on the forearm. This setup was used in combination with the teleimpedance controller for a blind surface discrimination and grasping experiment. For further information on these haptic experiments, please see [4]. Below we describe the modeling, parameter identification, and evaluation.

Fig. 10.10

Block diagram of the synergy-driven hand teleimpedance

Muscle force increases with muscular activity, and as individual muscle forces increase, they affect the torque at the joints they cross. Cocontraction of antagonist pairs, however, affects the impedance of the joint [37]. To begin, we consider the forward dynamics of the first grasp synergy and write

$$\begin{aligned} \begin{aligned} \tau&=a_\tau \delta ,\\ a_{\tau } \delta&=I \ddot{q_s} + c \dot{q_s} + K_s(q_s-q_0) + \tau _E, \end{aligned} \end{aligned}$$

(10.6)

where $\tau$, $a_\tau$ and $\tau _E$ denote the torque synergy, its gain, and external torques, respectively; $q_s$ and $q_0$ are the position of the hand and the object along the first synergy; $\delta$ is a function of the difference in activation of the antagonistic muscles ($FDS-EDC$), and I, c and $K_s$ are the inertia, damping and stiffness of the hand along the first synergy, respectively. The effects of inertia and external torques can be neglected, leaving us with

$$\begin{aligned} \dot{q_s}=\frac{-K_s}{c}(q_s-q_0) + \frac{a_{\tau }}{c}\delta . \end{aligned}$$

(10.7)

Next, we can use T and k to represent the time step and iteration number and estimate the dynamics in discrete time as follows

$$\begin{aligned} q_{s_{k+1}}=(1-\frac{K_s T}{c})q_{s_{k}} + \frac{T a_{\tau }}{c}\delta + \frac{K_s T}{c}q_0. \end{aligned}$$

(10.8)

Finally, we use two modified hyperbolic tangents [15] to map the position and stiffness synergy references:

$$\begin{aligned} \delta =\frac{a_q[1-\mathrm {e}^{-b_q(\mathrm {FDS}-\mathrm {EDC})}]}{[1+\mathrm {e}^{-b_q(\mathrm {FDS}-\mathrm {EDC})}]}, \end{aligned}$$

(10.9)

$$\begin{aligned} K_{s}=\frac{a_k[1-\mathrm {e}^{-b_k(\mathrm {FDS}+\mathrm {EDC})}]}{[1+\mathrm {e}^{-b_k(\mathrm {FDS}+\mathrm {EDC})}]}, \end{aligned}$$

(10.10)

where FDS and EDC are the processed EMG signals of the corresponding muscles, and the gains $a_q$, $b_q$, $a_k$, and $b_k$ are identified experimentally. To do so, the subject was asked to open and close his or her hand while FDS and EDC activity were recorded; meanwhile, the SoftHand opened and closed as a visual reference. Twenty natural, self-paced open/close movements were recorded to determine the parameters of the position synergy model. A further 20 movements were recorded while asking the subject to maintain various levels of cocontraction. Subjects were given visual feedback of their cocontraction levels and asked to perform 4 movement cycles at 5 different levels.

Fig. 10.11

Sample results of the SoftHand grasping a hard object (mug), with the controller under high, fixed stiffness gain (left top pair, $K=40\,{\text {Nm}}/{\text {rad}}$), low, fixed stiffness gain (left mid pair, $K=10 {\text {Nm}}/{\text {rad}}$), and teleimpedance (right four, $a_{q_{norm}}=1$, $b_{q}= 5.03$, $a_{k}= 1.87$, and $b_{k}= 0.579$)

Half of each set of trials was used to identify the parameters of the models and the other half to evaluate the modeling accuracy. Averaged across subjects, we found normalized root-mean-squared error (NRMSE) values of 17.6 and $13.4\,\%$ for the postural and stiffness test trials, respectively. Ultimately, mental imagery, bilateral action using a mirror box, or similar techniques could be used to identify these parameters in persons with amputations.

After parameter identification, subjects attempted to grasp everyday objects with each type of controller: stiff, compliant, and teleimpedance. A sample grasp of a rigid object (a mug) with each of the controllers is presented in Fig. 10.11. Subjects were highly successful with all controllers. However, grasp quality and stability was highest with teleimpedance. With the stiff controller, subjects would occasionally apply excessive force and damage deformable objects. In contrast, with the compliant controller, subjects would occasionally lose the grip on and drop heavier objects. The teleimpedance controller seemed to mitigate both of these problems. While these results are preliminary, they suggest teleimpedance control of a prosthetic hand is both functional and intuitive. Future work will apply this novel controller in a clinical setting.