Keywords

Electrically powered hand protheses are typically controlled with electromyographic (EMG) signals, acquired from muscles of the residual limb. In this chapter we will give an overview on classical EMG control as well as recent developments based on machine learning. Classical approaches utilize two EMG electrodes and allow to control only a single prosthetic function at a time. Machine learning-based approaches utilize more electrodes and can be divided into classification and regression. Classification-based approaches have become recently commercially available and allow a direct access to many prosthetic functions, while classification-based approaches allow for an independent simultaneous control of two degrees of freedom (DOF). Targeted muscle reinnervation is a surgical procedure to acquire additional control sites in the amputees and enables to directly control up to three DOF simultaneously.

Introduction

Electrically powered hand protheses are assistive devices that can help to compensate for the impact an amputation has to a person’s life. First prototypes for electrically powered hand prostheses were developed after the 2nd World War in Germany [41] and the first device that became commercially available was released 1964 in the UDSSR [50]. From the very beginning, electromyographic (EMG) signals acquired form residual muscles have been the most important way to control the prostheses. Over many decades myoelectric-prostheses had only one actuated degree for freedom (DOF) for opening and closing the hand. Then a second DOF rotation of the wrist was introduced. In the last years, great advances were achieved in the development of highly functional electrically powered hand prostheses with a high number of actuated joints. Currently at least four manufacturers offer multifunctional hand prostheses with 6 to 11 actuated joints. However, the bottleneck for introducing advanced functionality is not the protheses hardware, but the techniques to read the intention from the user and control the prosthesis. In this chapter we will give an overview on the commercially available control techniques as well as recent developments in research.

EMG Signal Acquisition

Physiological Background

Electromyographic signals are electric potentials in the range of 50 μV to 10 mV that are generated by skeletal muscles during their contraction [31]. Muscle fibers, which constitute the muscle, are innervated by terminal axonal branches of motor neurons. The motor neuron and its innervated muscle fibers constitute a motor unit (MU) [18]. Each action potential of the motor axon triggers a motor unit action potential (MUAP) that propagates toward both ends of the fibers and causes their contraction. The MUAPs of all MUs superimpose and form the electromyogram that can be measured on the surface of the skin. Since an increase in muscle force is mediated by increases in both the number of active MUs and their firing rates, the amplitude of the stochastic interference EMG signal proportionally increases with force [30].

Noninvasive EMG Acquisition

In clinical diagnostics and in electrophysiological research, disposable pre-gelled electrodes are often used in combination with monopolar EMG signal derivation. On the other hand, for the control of active prostheses, bipolar derivation with active electrodes is the commonly applied configuration. Here, the electric potential difference between two electrode contacts, typically located at a distance of 20 mm in the direction of the muscle fibers, is picked up and amplified by a biosignal amplifier [28]. Due to its high common mode rejection ratio and high input impedance, noise that is present on both bipolar electrode contacts are suppressed, as the EMG signal gets amplified [29]. In order to suppress motion artifacts and high-frequency contamination that tends to corrupt the EMG signals, commonly a band-pass filter with cut-off frequencies ranging from 5 to 30 Hz at the low end and 300–500 Hz at the top end is applied [31]. A notch filter is often used to eliminate the power line interference at 50 Hz or 60 Hz depending on the region.

For practical reasons dry metal electrodes are typically used in prosthetics. Due to their relatively large electrode-skin impedance, it is essential to keep the leads to the first amplification stage very short in order to prevent artifacts from saturating the signals. Therefore, active electrode modules are commonly applied, which integrate the electrodes with the amplifiers and the filters in a compact space. Many advanced control approaches use the raw EMG to extract multiple features per channel. However, electrode modules for conventional myoelectric control more often include a rectification and additional low-pass filtering of the signal and thus provide access to the EMG envelope.

Classical Control Approaches

Two-Channel Approaches

The most common myoelectric hand prosthesis control methods are based on two bipolar EMG signals picked up from a pair of antagonistic muscles or muscle groups available at the stump (Fig. 13.1). In case of an amputation, in most cases a phantom representation of the lost limb remains, and the residual muscles contract during phantom limb motions [47]. For individuals with transradial amputation, the wrist flexors and extensors are typically employed while the biceps and triceps brachii are commonly used in cases of transhumeral amputation. The pectoralis major or minor or infraspinatus or teres minor might be selected as source muscles for deriving control signals for amputations at the shoulder level. The electrode modules are integrated into the inner socket of the prosthesis and pressed against the skin with flexible polymer suspension that can partly compensate for stump volume variations [42].

Fig. 13.1
figure 1

Conventional two-channel proportional myoelectric control with antagonistic muscles. Only one DOF is controlled at a time and the active function is altered by performing a co-contraction. (© Ivan Vujaklija)

Full size image

Commonly, EMG activity of the flexor muscles is mapped to a closing function of the prosthetic hand, while activity on the extensors related to its opening. Most devices provide a proportional control, i.e., the stronger the muscles contract, the faster the prosthesis moves, or a higher grip force is delivered. This control scheme, often referred to as direct control, was already applied in the first commercially available myoelectric prosthesis in the 1960s [50].

With the exception of targeted muscle reinnervation (TMR) patients, there are typically not enough separately addressable muscles to generate independent EMG signals, which could directly extend the prosthesis control to more degrees of freedom (DOFs) simply by increasing the number of channels. To control more than one DOF with only two EMG channels, several heuristics have been developed and are in commercial use. For a good overview on clinically available techniques, please refer to [34].

The most common approach for switching between the DOFs is the co-contraction-based triggering. Once the user contracts both muscle groups at the same time, the active DOF is switched, and the two EMG signals now control another function of the prosthesis, i.e., changing from grasping to wrist rotation. For above-elbow prostheses, even alternating between three functions (grasping, wrist rotation, and elbow flexion) is common. However, as only one function can be controlled at a time, cycling through DOFs is a rather cumbersome control method and thus limits the benefit of additional functions.

Another commercially popular approach distinguishes between grasping and rotation based on the slope with which the EMG signal increases. Slowly increasing EMG activity causes the opening or closing of the prosthetic hand, while a quickly increasing activity causes rotation of the wrist. Once a high or a low slope is detected, the prosthesis stays within the corresponding DOF until the user relaxes the muscles completely and therefore allows for a proportional control of the speed or the force by adapting the contraction force. Here, no explicit mode switching is required, but the DOFs still have to be activated sequentially.

Single-Channel Approaches

In cases when only one EMG channel can reliably be controlled, both directions of a single DOF have to be addressed using a single channel. The direction is selected either by the initial slope of the signal or by its absolute level (e.g., high level for opening, low level for closing). Once the controller detects the direction, it locks into that state until a rest phase is detected and thus allows for a proportional control via modulation of the EMG activity. Alternatively, in those cases where even the proportional modulation of the EMG amplitude is not an option (commonly in kids), a hand can be open using a single channel, and the closing is automatically done by the prosthesis when the user relaxes. This approach is commonly referred to as “cookie crusher” [27].

Non-EMG Approaches

In a clinical setting, control inputs other than EMG are also used as an alternatively or in addition to it. In prostheses with individually actuated fingers, which are usually controlled with the two-channel approach, grip patterns can be preconfigured and selected by pressing a button on the prosthesis with the other hand, or by moving the prosthesis into a certain direction after an EMG-trigger-signal. Also RFID tags placed on certain objects can be used to automatically select grip patterns, when the hand approaches the tag [52].

Control Approaches Following TMR

In TMR, additional EMG sites are obtained by surgically reconnecting the still intact nerves of the lost limb to other muscles in the vicinity [26]. In this way, intuitively controlled muscles that react to phantom limb motions of the lost limb are obtained and used for prosthesis control. Since the muscles are spatially well separated and can be actuated independently by the user, the two-channel direct control approach can be extended to multiple DOFs (Fig. 13.2). In a typical TMR prosthesis following a shoulder disarticulation, up to six EMG signals can be detected and grouped in three pairs of commands. Thus, independent, proportional, and simultaneous control of three DOFs (elbow flexion/extension, wrist rotation, and hand opening/closing) is possible [32].

Fig. 13.2
figure 2

A schematic of a shoulder disarticulation TMR fitting (ventral side). (© Ivan Vujaklija)

Full size image

In those TMR cases where fewer control signals are available, a combination of other conventional techniques can be adopted. For instance, when only five independent EMG channels are present, a pair of electrodes could be mapped to hand open/close and another one to elbow flexion and extension. The signal of the remaining electrode could then be split so that the initial slope maps either pronation or supination and vice versa. With four available channels, two electrodes can be used for elbow flexion/extension exclusively and the other two channels to either control grasping or wrist rotation, which can be switched through co-contraction.

Machine Learning-Based Approaches

To overcome the limitations of the classical control approaches, significant research has been conducted in the past decades with the goal to employ machine learning techniques for extracting more control information from a larger number of EMG signals. Most of these advanced approaches follow an established control chain containing the processing blocks shown in Fig. 13.3. Instead of just two, typically six to ten EMG channels are used, either specifically placed on certain muscles or equally distributed over the area of interest. In order to extract as much information from the signals, the raw EMG is typically used. The signals are digitized and processed in blocks (windows) of 100–200 ms in duration as to satisfy the optimal controller delay [10]. After applying similar noise removal filters as in the conventional control, certain features are extracted from the filtered EMG to condense and describe the information that can be used by the control algorithm. A large number of different features and their combinations (feature sets) can be extracted in time and/or frequency domain. The most common ones include root mean square (RMS), mean absolute value (MAV), slope sign changes (SSC), zero crossings (ZC), wavelength (WL), auto recursive coefficients, short-time Fourier transform features, wavelets, and more [19]. After feature selection, a machine learning algorithm, commonly a classifier or a regressor, interprets the signal features and transforms them into control signals to be used by the prosthesis. Most approaches employ supervised algorithms, i.e., they have to be trained with samples of labeled training data prior to their application. To obtain these training samples, data is recorded for which the type of contraction is known, i.e., by relying on visual cues [1] or by performing bilateral symmetric motions and measuring kinematic or forces of the contralateral, unaffected side [35]. In both cases (visual cues and bilateral motions), a certain error in the labels may be introduced due to variability in task execution.

Fig. 13.3
figure 3

Block diagram describing the signal processing chain in most machine learning approaches for prosthesis control. (© Ivan Vujaklija)

Full size image

Classification

In classification-based approaches, an algorithm is trained with a pre-recorded feature samples of all motions the controller should be able to detect. It then compares the current feature vector with the model generated from the training data and decides for a certain motion (class). To achieve a high classification accuracy, several types of classifiers for myoelectric prosthetic control have been examined, such as linear discriminant analysis (LDA) [8], artificial neural networks (ANN) [24], support vector machines (SVMs) [49], k-nearest neighbors [24], and many more.

A classifier only estimates which motion is active, but it does not provide any information on the strength of the activation. It can therefore be combined with a parallel signal path to estimate the activation level (e.g., from the mean amplitude of all channels) and enable the clinically required proportional control of the velocity [4]. Compared to the co-contraction control, the mode switching is omitted, but different DOFs still have to be executed one by one, which requires complex motions to be separated into sequentially executed sub-motions. In the last years, however, EMG pattern recognition has been extended to concurrent classification of motion intent by introducing additional classes for all motion combinations that should be activated simultaneously [36]. In this way, the pattern recognition approach enables simultaneous control and thus promotes a more natural interaction with the environment. However, with increasing number of classes, the classification accuracy decreases, which increases the risk of false motions.

Although classification-based control approaches for myoelectric control have been proposed already several decades ago [13], their clinical impact has been so far limited. Most commercial prostheses still use the established two-channel control approaches. As of recent Coapt LLC [5] offers an FDA approved controller with a classification-based control as an extension to most common hand and wrist devices. In 2018, Otto Bock has introduced their own classification-based controller to the market [46]. The reasons for a limited transfer into clinical applications are related to reliability problems under real-world conditions. Factors such as a change in arm position [11], electrode shifts [54], and time between training and application [51] or changing skin conditions, e.g., due to sweat [21], alter the signal patterns and cause significantly decreasing classification accuracies.

Regression

To overcome some of the limitations that classification-based approaches present, regression-based techniques have been investigated to achieve an independent simultaneous and proportional control of multiple DOFs [22]. Similar to classification, regression is also a machine learning technique and needs to be trained with some calibration data. The essential difference to classification is that a regressor does not estimate a specific class (movement) but instead a continuous physical value (force, speed, position, etc.) for each DOF individually. In this way the activation ratio can be controlled independently in all DOF, which allows, e.g., to open the hand quickly while performing a slow rotation at the same time. Various nonlinear and linear techniques have been investigated for regression-based myoelectric control. Nielsen et al. demonstrated successful control of two DOF with artificial neuronal networks [35] trained on bilateral mirrored motions, and Muceli et al. extended this approach to three DOF [33]. Ameri et al. employed nonlinear kernel-based support vector regression [2] and investigated different strategies for obtaining the labels for supervised training, including visual cues [1]. Gijsberts et al. demonstrated an effective approximation of the kernel approach by Random Fourier Features [12]. Matrix factorization techniques have been able to successfully estimate anywhere from two DoF [23] up to seven DoF [20] control. Finally, regression control established using autoencoders has shown to outperform classic control when driving multiple wrist DOFs [53].

An extensive comparison of linear and nonlinear regression techniques has been made [14] showing that simplest of the approaches still seem to be sufficiently effective in delivering satisfactory user performance. This well may be the consequence of continuous feedback that regression-based approaches offer. Unlike classification where due to discrete nature of the estimation the users are not fully aware in which way the misestimation occurred, regression approaches, with their unbounded solution space, allow implicit adaptation to the user. This way, they can actively compensate for false estimations of the algorithm despite the commonly occurring disturbances [15, 48]. As recently shown in five prosthesis users, this leads to relative high robustness against potential sources of non-stationarities such as changing arm position or donning and doffing the prosthesis [16]. However, the additional capabilities of simultaneous and proportional control are followed by the risk of unintended co-activations in other DOFs. Thresholds to suppress small activations can be introduced, but this problem increases with increasing number of DOFs.

Modeling

It has been shown that simultaneous and proportional control can also be achieved through musculoskeletal modeling that estimates joint moments and joint torques from the muscle activations [7, 43, 44]. This approach has been successfully demonstrated in both upper and lower limb prostheses [6, 45]. Instead of focusing on the explicit properties of the data or the correlations, forward musculoskeletal models recreate the biological process of motion generation. They incorporate the physiological and biomechanical constraints in order to estimate natural limb motions.

In particular, since TMR allows the detection of the neural activity of all nerves involved into the task, including missing muscles, musculoskeletal models allow rather detailed reconstruction of the internal biomechanical representation of missing limbs [9].

Transfer Learning

Various sources of disturbance, such as posture change and resulting electrode shifts can impede the user in everyday prosthesis control [17, 54]. Several approaches have been proposed to improve robustness, such as implanted EMG electrodes instead of surface electrodes [14, 38], high-density EMG surface electrode grids [33, 51] more sophisticated feature extraction [25], and post hoc error detection within the algorithm [3]. However, for the most commonly used surface EMG control, a quick and easy approach to counteract electrode shift is the concept of transfer learning, an approach which adapts the machine learning model to the disturbed data such that the original model is applicable again [37, 39, 40]. Thereby, a pattern recognition model is trained on recorded data (original) (see Fig. 13.4). The colored circles indicate a different class each. Then, incoming data is disturbed through electrode shift, and the learnt model is not applicably anymore (disturbed). The disturbance is estimated by recording only few new instances from only few selected classes in the disturbed condition (record new data). Grey circles indicate possible future positions of transformed data. Finally, the algorithms learns an updated model based on the newly acquired data to virtually transform the data to its original domain so that the model can be employed again (transfer learning) [40].

Fig. 13.4
figure 4

Overview of the transfer learning process [40]: a pattern recognition model (original) is trained on recorded data, through a shift in electrodes, the incoming data is thus disturbed and cannot be accurately recognized by the original model. This occurrence is counteracted by re-recording only a few new data points with this shifted position to estimate new data, so that the original model can be applied again. Thereby virtually shifting the electrodes back into place, when in reality they are not. This transformation of data is called transfer learning. (Used with permission from IEEE)

Full size image