Keywords

6.1 Introduction

Human upper limb represent an integral part of the body that provides incredible array of functionalities during activities of daily living (ADL). In addition to possessing dexterous and fine motor grasping/sensing capabilities, the upper limb enables individuals to freely and effectively communicate with their natural environments through artistic expressions, sign languages, and greetings (handshake/hand gesture), among others. Therefore, the loss of one or both arms severely limits individuals’ ability especially when they need to perform tasks requiring their arm functions during ADL, thus leading to decreased quality of life [1]. Limb loss also has profound negative impact on amputees including stump pain or phantom limb pain, or perhaps even both, depression, changes in body image, and psychological burden [1,2,3,4,5]. Under certain circumstance, upper limb loss may also affect amputees’ stability (dis-balance) during mobility, thus making them prone to falls or collisions with other individuals/objects. To this end, limb loss causes a devastating experience necessitating the need for proper psychological and physical rehabilitation strategy for arm amputees.

A large number of the commercially available upper limb prostheses are either mechanically (body-powered) or electrically (motorized) driven. Meanwhile, body-powered upper limb prosthetic devices were primarily developed as alternative to aid amputees in carrying out ADL requiring their arm functions [6]. Fundamentally, these prostheses operate by using cables to link the movement of the body to the device and thus actuating simple hand tasks such as opening or closing of the hook/gripper. To achieve simple hand grasp tasks, the amputee would need to possess a significant amount control over his/her shoulder, chest, and even the residual limb which should be long enough to support the motion. Although body-powered prostheses have fine cosmetic appearances that are comparable to the human natural arm [7], they are nonintuitive and require significant amount of efforts (burdensome) to perform simple hand tasks and as well support only single degree of freedom (DOF) movement per time [7]. With these limitations, motorized upper limb prostheses are considered as viable alternative for intuitive restoration of multiple DOF arm functions in amputees. At the forefront of this technology are the prostheses that utilize electromyography (EMG) recordings to characterize upper limb movement intentions of the amputees [8,9,10,11,12]. Thus, the properties of the EMG signals are encoded in form of control commands to the prosthetic controller which then actuates the device accordingly. It is noteworthy that in the last two to three decades, the motorized upper limb prosthetic technology has progressively advanced leading to the development of better control methods as well as designs. Hence, this article focuses on the EMG-based prosthetic control with emphases on methods/technologies that could help improve the overall performance and acceptability of the prostheses system.

The remaining part of the chapter is organized as follows: Section 6.2 discusses different EMG-based prosthetic control methods; Section 6.3 introduces some potential techniques for improving the performance of EMG-based prosthetic control; Section 6.4 presents the concluding remarks.

6.2 EMG-Based Prosthetic Control Methods

6.2.1 Background

Fundamentally, EMG signals represent electrical manifestation of neuromuscular activation associated with a contracting muscle. Such signals have been proven to contain rich set of neural information from which different types of upper limb movements could be accurately decoded, thus making them useful control input to prosthetic devices as well as other forms of rehabilitation robots [8, 13]. Additionally, EMG signals carry significant information for diagnosis of neuromuscular disorders and other muscle-related diseases. It has been established that EMG signal patterns for a specific limb movement are repeatable over time and distinct from the patterns of other limb movements. By exploring this phenomenon, researchers have been able to decode a range of arm movements from EMG recordings using intelligent computational methods. To this end, dozens of previous studies have reiterated the potential of EMG-based method in the control of multifunctional prostheses [14,15,16]. Thus, the following section will focus on discussing the currently applied EMG-based prosthetic control schemes including: pattern recognition (PR) control method and the simultaneous and proportional control (SPC) method.

6.2.2 Pattern Recognition-Based Prosthetic Control

Although the concept of EMG pattern recognition (EMG-PR) is by no means new, it has been proven over time to be promising especially for improving the dexterity of control in upper limb prosthetic devices in comparison with the conventional amplitude-based methods. It is noteworthy that the first generation of pattern recognition-driven prosthetic controllers was built in the late 1960s/early 1970s [17,18,19]. Since then, advancement in signal processing, multichannel recording, and microprocessor technology has expedited implementation of EMG-PR method in embedded control systems. Also, with a lot of synergistic efforts from the academia and industries, EMG-PR-based prosthetic arms have been recently made available for commercial use.

EMG-PR prosthetic control method is primarily based on the assumption that EMG signal patterns corresponding to a particular class of limb movement are consistently repeatable across trials and as well distinct from EMG signal patterns of the other classes. Due to the promising nature of this control method, it has been widely explored in the last two to three decades with significant progress reported in terms of its robustness. The control schemes as a whole consist of a sequence of steps beginning with: EMG signal acquisition, preprocessing of the recorded signals, extraction of feature set, classification of the extracted features, and issuing of control commands to drive prosthesis. Figure 6.1 shows the schematic diagram of a typical EMG-PR-based prosthetic control system.

Fig. 6.1
figure 1

Schematic diagram of EMG-PR-based prosthetic control system

Full size image

In the above represented control scheme, myoelectric signals are collected via electrodes placed on the skin surface underlying the arm muscles. Often times, the electrodes are integrated with miniaturized amplifier to improve the quality of the recorded signals through standard amplification and filtering procedures. Afterwards, the acquired EMG recordings are subjected to preprocessing procedure, so as to attenuate inherent interferences. In this regard, the 50/60 Hz notch filter is usually applied to the raw acquired signals to minimize interferences resulting from power line noise. In addition to the notch filter, a band-pass filter with cut of frequency between 10/20 Hz and 500 Hz is often applied to the EMG signals to obtain the most useful spectrum with high motor information content.

EMG signals recorded during targeted limb movements are continuous in nature, thus for a specific length of recording, a decision corresponding to a subset of the observed limb movements may be required to actuate the prosthesis. Therefore, the use of data segmentation technique in handling the preprocessed data prior to feature extraction is necessary. This step helps increase the probability of the prosthetic control system achieving high accuracy as well as quick response time. Ideally in real life applications, a response time of no more than 300 ms is required [20]. Note that the response time (which is a function of the segment length and the processing time) is the time required to produce the control commands that drive the prostheses. In a study conducted by Englehart and Hudgins [21], it was shown that by adopting continuous segmentation on a steady-state EMG signal, the segment length can be reduced to about 128 ms without a significant reduction in classification accuracy. Also, it has been shown previously that segmented steady-state EMG signals achieved better accuracy compared to segmented transient EMG signals, and less degradation were recorded with shorter segments of the steady-state EMG data. After considering the segment length and the state of the data, the third important data segmentation factor is the windowing technique adopted. Data segmentation is often achieved via either of the following windowing techniques: sliding analyses windowing and adjacent windowing method [20]. The adjacent windowing method uses disjoint segments of predefined length for its feature extraction; and the intended limb movement is classified after a certain processing delay. Because the processing time often denotes a small fraction of segment length, the processor is found to be mostly idle during the remaining time of the segment length. Meanwhile, in the sliding windowing method, the new window segment slides over the current window segment, with an increment time less than the segment length. In comparison, this sliding windowing method takes advantage of the processor’s idle time in adjacent windowing method to produce more classified outputs. Thus, the sliding windowing technique has been widely applied to steady-state EMG signals for enhanced feature extraction process in most recent studies on EMG-PR-based prosthetic control.

Aside from the feature extraction stage, the choice of classification algorithm has been reported to be another important factor in EMG-PR-based prosthetic control systems. Thus, choosing an efficient classifier becomes necessary to ensure accurate and consistent classification performance. In this regard, a number of classification schemes have been examined with respect to their performance in identifying different upper limb movements. For instance, classification schemes based on Bayesian statistics [22], adaptive neural networks (ANN) [23], fuzzy logic [24], support vector machine [25], and linear discriminant analysis (LDA) [11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26] were previously studied. The ANN-based classifier and the LDA classifier were found to be more accurate for the prediction of limb movement intention. However, the LDA has been widely applied because it is relatively less complex in terms of implementation and performs well in comparison with the ANN and other complex classification Schemes [26].

After reaching a decision on the classification scheme to adopt, a classifier is developed by utilizing part of extracted EMG feature vector. Afterwards, the performance of the built classifier in identifying multiple classes of limb movement intent is evaluated using the testing set as obtained from the remaining part of the feature vector. The performance of the EMG-PR-driven prostheses in terms of its controllability is mostly examined in both off-line and real-time metrics. In general, metrics including recognition rate (classification accuracy) of limb movements/subjects, the time taken to complete a motion (motion completion time), number of successfully completed motion (motion completion rate), and the time taken to select the target motion (motion selection time) are often considered. Also, the performance of the device could be assessed based on the number success recording reaching and grasping tasks.

6.2.3 Simultaneous and Proportional EMG-Based Prosthetic Control

Multiple DOF pattern recognition-based prostheses control schemes have recently been implemented to enable upper limb amputees perform the usual arm function tasks during ADL. Even with this remarkable achievement, the available EMG-PR-based controlled prostheses do not support simultaneous control of multiple DOFs [27, 28]. This limitation prevents upper limb prostheses users from having the natural feel of the coordinated joint control often associated with an intact limb. Although advanced prosthetic arms [29, 30], including multiple DOF wrists, offer the mechanical means to restore arm movements, there is still a significant need for systems that enable simultaneous control of such devices.

To develop a simultaneous prosthetic control scheme which offers multiple DOF movements, different approaches have been proposed in the recent years. These approaches include the use of artificial neural networks for joint kinematics predictions during upper limb movements [31], analysis of the muscle synergies underlying a range of upper limb movements [32], and the utilization of joint kinetics [33]. While these methods have been reported to be promising toward clinical realization of upper limb prostheses, they however mostly focus on providing control for limited classes of limb movements such as wrist without hand or have imposed identical velocities on all active degrees of freedom movements and thus do not provide independent proportional control of individual degrees of freedom.

Additionally, toward achieving simultaneous and proportional control scheme, some investigators developed a model of muscle synergy driven by nonnegative matrix factorization (NMF) method to resolve myoelectric signal factorization issue that is capable of hindering continuous control [32, 34]. The NMF technique often utilizes a DOF-wise training approach, based on which simultaneous control of multi-DOF is achieved via linearly combined single degree of freedom. This kind of strategy supports simultaneous and proportional myoelectric-based control [32, 34]. Despite the promising results of this scheme, there are still some shortcomings that have hindered its real life implementation. The most critical disadvantage of the NMF-driven EMG-based prostheses is that it needs a DOF-wise calibration during which requires a user to activate precisely single DOF in a specific manner, e.g., firstly, trigger DOF1 for several trials followed by DOF2. Besides the inconvenience and the time needed for the execution of this phase, more notably, if the prostheses users are unable to trigger an individual DOF (i.e., the activation patterns during the training session are not exactly of single DOF), the DOF-wise-driven NMF approach will produce inadequate muscle synergy matrix that may result in poor control outcome. This behavior could be attributed to the fact that the classical NMF-based scheme lacks the ability to produce a factorization that is sparse enough for efficient control of multiple DOF [35, 36]. Hence, we proposed an alternative approach to the classical NMF that identifies the factorization which separates the generated basis functions robustly and concurrently in a quasi-unsupervised way to simultaneously and proportionally control the prostheses.

In the proposed approach, the subject does not need to follow a predefined sequence to activate single DOF in the calibration phase, and can even activate more than one DOF simultaneously in the training stage, which is not allowed in the classical NMF method. Nevertheless, the idea of the synergy matrix could be realized in one step, which is contrary to the conventional NMF approach that attempts to sequentially extract the bases for each DOF. For this reason, constraints were included in the factorization method and the method is at maximizing the sparseness of the resulting control outcome. The constraints associated with the sparseness limits the space of possible NMF solutions. Precisely, the solution with bases functions associated to single DOF, which is the target solution, represents the sparsest of the other solutions. In this regard, the factorization with constraints usually does not need prior calibrations/activations of single DOF and could be utilized to any set of target tasks carried out by the user with no any form of strict labeling. The proposed sparse nonnegative matrix factorization (SNMF)-based simultaneous and proportional myo-control scheme is presented mathematically as follows.

In myoelectric control, multiple DOF limb movements could be re-represented as a linearly combined individual DOFs, and the synergy basis and control input are nonnegative [34]. Suppose we represent the root mean square (RMS) values of an N-channel with T-length surface myoelectric signal as Z, in which the t-th column is the myoelectric signal at a given time t and m represents the number of DOFs. Thus, the multichannel observation can be approximated by the product of a N × 2m latent nonnegative synergy matrix W and a 2m × T latent nonnegative control signal matrix.

F is expressed in the following equation:

$$ {\mathbf{Z}}_{N\times T}\approx {\mathbf{W}}_{N\times 2m}{\mathbf{F}}_{2m\times T} $$
(6.1)

While the sparseness of the solution is obtained by imposing a specific single DOF activation sequence for calibration, the above factorization problem can be solved in a semi-supervised way with the summation property of DOFs in the muscle signal domain.

A mathematical way to generate a sparse solution without the need for the specific set of calibration data for individual DOF activation is utilized. Meanwhile, this is achieved by imposing a sparseness constraint on the control signals, which leads to the proposed SNMF Scheme [37].

Mathematically, the extent (degree) of sparseness could be controlled by utilizing either 11-norm or l0-norm. To build a computationally efficient system, we chose the l1-norm-based sparse NMF scheme, SNMF [37]. Meanwhile, the objective function of the SNMF method is expressed mathematically as follows:

$$ {\displaystyle \begin{array}{l}\underset{\mathbf{W},\mathbf{F}}{\min}\kern0.5em \frac{1}{2}\kern0em {\left\Vert \mathbf{Z}-\mathbf{WF}\right\Vert}_{Fro}^2+\lambda \sum \limits_{t=1}^T{\left\Vert \mathbf{F}\left(:,t\right)\right\Vert}_1^2\kern1em s.t.\kern0.5em \mathbf{W},\mathbf{F}\ge \mathbf{0},\kern0.5em \\ {}\iff \underset{\mathbf{W},\mathbf{F}}{\min}\kern0.5em \frac{1}{2}\kern0em {\left\Vert \mathbf{Z}-\left[{W}_1^{+},{W}_1^{-},\cdots {W}_m^{+},{W}_m^{-}\right]\left[\begin{array}{l}{F}_1^{+}\\ {}{F}_1^{-}\\ {}\vdots \\ {}{F}_m^{+}\\ {}{F}_m^{-}\end{array}\right]\right\Vert}_{Fro}^2+\lambda \sum \limits_{t=1}^T{\left\Vert \left[\begin{array}{l}F{\left(:,t\right)}_1^{+}\\ {}F{\left(:,t\right)}_1^{-}\\ {}\vdots \\ {}F{\left(:,t\right)}_m^{+}\\ {}F{\left(:,t\right)}_m^{-}\end{array}\right]\right\Vert}_1^2\kern1em \\ {}\kern1em s.t.\kern0.5em {W}_{ij},{F}_{ij}\ge \mathbf{0}\end{array}} $$
(6.2)

where F (:, t) is the t-th column vector of F [Eq. (6.1)], Fro is the Frobenius norm, and λ > 0 is a regularization parameter to balance the accuracy of the factorization and the degree of sparseness of F. In the experiment, we choose the optimalλ through cross-validation with the value of m = 2. The superscripts “+” and “–” denote the positive and negative direction of each DOF, respectively. The above formulation can be rewritten as:

$$ \underset{\mathbf{W},\mathbf{F}}{\min }{\left\Vert \left(\begin{array}{l}\mathbf{W}\\ {}\sqrt{\lambda }{\mathbf{e}}_{1\times 2m}\end{array}\right)\mathbf{F}-\left(\begin{array}{l}\mathbf{Z}\\ {}{\mathbf{0}}_{1\times T}\end{array}\right)\right\Vert}_{Fro}^2\kern1em s.t.\kern0.5em \mathbf{W},\mathbf{F}\ge \mathbf{0},\kern0.5em $$
(6.3)

where e1 × 2mrepresent a row vector that contains values equal to 1 and 01 × Trepresent entries with values equal to 0. Equation (6.3) could be adequately solved via the alternating nonnegative least squares (ANLS) approach. Meanwhile, for the ANLS method, F and W are iteratively updated by adjusting the other one:

$$ {\mathbf{F}}^{\left(k+1\right)}=\underset{\mathbf{F}}{\arg \min }{\left\Vert \left(\begin{array}{l}{\mathbf{W}}^{(k)}\\ {}\sqrt{\lambda }{\mathbf{e}}_{1\times 2m}\end{array}\right)\mathbf{F}-\left(\begin{array}{l}\mathbf{Z}\\ {}{\mathbf{0}}_{1\times T}\end{array}\right)\right\Vert}_{Fro}^2 $$
(6.4)
$$ {\mathbf{W}}^{\left(k+1\right)}=\underset{\mathbf{F}}{\arg \min }{\left\Vert {\mathbf{W}\mathbf{F}}^{\left(k+1\right)}-\mathbf{Z}\right\Vert}_{Fro}^2 $$
(6.5)

It could be seen that Eqs. (6.4) and (6.5) represent the classical least square problem, and either of the equation has a closed-form solution. Meanwhile, the derivation in [37] reveals that the SNMF method could converge to a stationary point. The SNMF method can equally extract all the basis functions in a single step from the EMG recordings produced via arbitrary combinations of DOFs per user. Only the ordering of the control input signals (basis functions) would be undetermined using this method, but this issue could be simply addressed. In practice, the labels of the first two single DOFs need to be recorded in order to decide the sequence of the basis in the matrix (F) from which control signal is obtained, thus making the method quasi-unsupervised instead of completely unsupervised.

Having computed the synergy basis matrix W from the recorded myoelectric signals, it is assumed to be a constant (at least over the period of the test experiments). To estimate the control input signal associated with the intended DOF activation level, a pseudo inverse of W (represented as W+) is applied, and it was multiplied with the newly obtained surface myoelectric signals \( \left(\overset{\sim }{\ \boldsymbol{Z}}\ \right) \). Thus, the estimated control input signal (\( \overset{\sim }{\mathbf{F}}\Big) \) is defined as follows:

$$ \overset{\sim }{\mathbf{F}}={\mathbf{W}}^{+}\overset{\sim }{\mathbf{Z}} $$
(6.6)

where

$$ \overset{\sim }{\mathbf{F}}=\left[{\overset{\sim }{F}}_1^{+};{\overset{\sim }{F}}_1^{-};{\overset{\sim }{F}}_2^{+};{\overset{\sim }{F}}_2^{-};\right] $$
(6.7)

Toward ensuring that no components are surpassed by the others, the individual component of \( \tilde{\mathbf{F}} \) in (6.7) is subject to a normalization process based on its maximum value. An estimate of the control input signal in (6.7) is thus scaled by the scalar correction factorsτij that are utilized to account for the indetermination of the signal power (range of control signals) during the factorization process:

$$ \left\{\begin{array}{c}{\tilde{F}}_1={\tau}_{11}{\tilde{F}}_1^{+}-{\tau}_{12}{\tilde{F}}_1^{-}\\ {}{\tilde{F}}_2={\tau}_{21}{\tilde{F}}_2^{+}-{\tau}_{22}{\tilde{F}}_2^{-}\end{array}\right. $$
(6.8)

where we name \( {\tilde{F}}_1 \) and \( {\tilde{F}}_2 \)the control signals for DOF1 and DOF2, respectively. The multiplicative factors τijare determined for each subject so that the final control signals,\( \tilde{\mathbf{F}}=\left[{\tilde{F}}_1;{\tilde{F}}_2\right] \) match the range of joint angles in the respective DOFs. The tau-s parameters are manually set before operating online tasks according to different subjects. The obtained control signals, low-pass filtered at 6 Hz (kinematic bandwidth), are then used by the subjects for controlling the DOFs.

It is noteworthy that the proposed SNMF scheme is aimed at robust and concurrent extraction of basis functions for myo-control in upper limb prostheses with a quasi-unsupervised scheme. This factorization scheme shown in our study has merits compared to the classical NMF because it selects the sparsest solution among infinite feasible solutions. To that end, there is no need for pre-calibration of the constraint tasks by users but rather the constraint on the solution. The method has been compared in similar conditions to the NMF and linear regression with results proving it to be superior over these methods even when a single DOF activation was used for the factorization which is an ideal condition for the NMF. Furthermore, the approach could be used by factorizing the myoelectric signals from arbitrary tasks by combining individual DOFs. The proposed factorization algorithm allows robust simultaneous and proportional control and is superior to previous supervised algorithms, and its minimal supervision characteristic paves the way to online adaptation in myoelectric control.

6.3 Neural Interfaces for Improvement of EMG-Based Prosthetic Control

6.3.1 Background

The previously described sections show that EMG-based prostheses control method could potentially allow amputees to intuitively manipulate the prosthetic arm with high level of dexterity and as well achieve multiple DOF controls. Developing EMG-based prostheses with these characteristics mainly depends on whether the users have enough residual muscles which act as source of EMG control signals. For instance, individuals with below the elbow amputation (transradial amputees) generally have enough residual muscles for the control of wrist functions such as wrist flexion/extension, wrist pronation/supination, and so on. Also, these category of amputees could provide enough EMG signals from their residual forearm muscles to control hand and even finger movements as reported in a number of previous studies [7]. On the other hand, individuals with high-level amputations (transhumeral or shoulder disarticulation) usually do not have enough residual muscles to provide myoelectric signals for the successful control of the wrist and hand movements although they can produce enough EMG signals for the control of elbow functions (elbow flexion/extension). Thus, intuitively controlling multiple DOF hand/wrist movements becomes a major challenge for these categories of amputees. One possible means of addressing the issue of insufficient generation of myoelectric control signals for these users would be to consider neural machine interfaces approaches such as targeted muscle reinnervation (TMR) and targeted nerve function replacement (TNFR). The theoretical background is that muscles can be considered as the bio-amplifier of the neural signals.

6.3.2 Targeted Muscle Reinnervation Technology

Toward enabling high-level upper limb amputees producing sufficient myoelectric control signals again, a neural machine interface technology known as TMR was proposed by Todd Kuiken et al. [28]. TMR is a surgical procedure that involves the transfer of residual nerves from an amputated limb unto alternative muscle group usually located around the chest region. During the surgery, brachial plexus nerves that provide motor control and sensory feedback prior to amputation are transferred to the arm or chest muscles that are biomechanically nonfunctional. Afterward, a functional link is established between the implanted nerves and the targeted muscles, thereby inducing the contractility of targeted muscles and enabling the generation of additional myoelectric signals. Importantly, this surgical technique is aimed at restoring the peripheral nerve functions in amputees thus enabling the provision of sufficient myoelectric control signals again. A schematic diagram of the TMR surgical procedure is conceptualized in Fig. 6.2a. For the subject shown in Fig. 6.2b, the surgery was performed on his right chest region, and it involves transferring the median, ulnar, radial, and musculocutaneous nerves to the subject’s pectoralis major muscle segmented into four parts, respectively.

Fig. 6.2
figure 2

The schematic representation of the TMR technology. (a) TMR surgical procedure for shoulder disarticulation amputee. (b) A bilateral amputee with successful TMR surgery controlling a prosthetic arm [28]

Full size image

The above described neural machine interface technology for upper limb prostheses control has been implemented and tested on a number of human subjects. The test sessions conducted with a bilateral amputee (Fig. 6.2b) shows that the high-level amputee could efficiently perform additional hand function tasks such as drinking and eating which could not be achieved prior to the TMR surgery. Importantly, the success of performing additional tasks was objectively linked to the realization of more EMG control signals after the TMR surgery.

6.3.3 Targeted Nerve Function Reconstruction Technology

Despite the success recorded by the TMR technology, it still has some limitations which have slowed down its widespread adoption. For instance, TMR surgery involves establishing functional links between the nerves and targeted muscles leading to two major challenges: First, the targeted muscle would atrophy with time because it has been denied of the required nutrients provided by the original nerve which has been replaced with the implanted nerve. Second, intramuscular nerve distribution (a key indicator of the motor function recovery) will also degenerate after a while due to the lack of nutrients from the surrounding cell body.

Peripheral nerves which play an important role after TMR surgery have been reported to have a slow growth rate. That is, it takes about 3~6 months or even more after TMR for proper redistribution of the implanted nerve into the targeted muscle, thus leading to atrophy of the targeted muscle. Aside the slow growth rate, the absence of nerve supplements is another factor responsible for atrophy of the targeted muscle [38,39,40]. In such a situation, high-level upper limb amputees would need training and treatment over an extended period for new functional connections to be established between the targeted muscle and the implanted nerves. It is worth noting that end-to-end anastomosis has been proven to be the best approach for restoring the functions of injured nerves in patients [41]. Thus based on this principle, targeted nerve function reconstruction (TNFR) technology is considered to address the abovementioned problems associated with TMR. This is because TNFR has the potential to better restore the motor functions associated with multiple degrees of freedom upper limb movements. Aiming at improving intramuscular nerve distribution as well as preventing muscle atrophy, TNFR establishes functional links between the nerves of the targeted muscles and nerves from the amputated region/arm of the individual. A schematic representation of the TNFR technology is shown in Fig. 6.3.

Fig. 6.3
figure 3

A representation of the TMR/TNFR technology. (a) TMR/TNFR surgical procedure for a transhumeral amputee, (b) EMG recordings after undergoing a TNFR surgery when the subject was doing different arm movements

Full size image

Shown in Fig. 6.3 is a male transhumeral amputee that underwent a TNFR surgery. He actually got his left limb amputated in 2012 as a result of injury sustained from a rolling machine. At the time of the surgery (2015), the amputee was approximately 48 years old. Prior to the TNFR surgery, his residual limb was subjected to a number of assessments to ensure that he meets up with the requirement. From the assessments, we found that the residual limb has a length of 22 cm and has not been infected. In addition, the results of an MRI scan show that the musculus biceps brachii and musculus triceps brachii of the residual limb were intact with neuroma tissue formed at median nerve, ulnar nerve, and musculospiral nerve terminals. Afterward, the TNFR surgery was performed on the subject by transecting the MCN, MN, RN, and UN from the residual limb and implanting them into the corresponding MCN, MN, RN, and UN on the amputee’s chest region.

About 7 days after the surgery, the operation regions were observed twice a week for 8 consecutive weeks. Indexes, such as status of the incision (healing rate), skin color and temperature, and limb peripheral circumference, were closely monitored. The pre-surgery and post-surgery experiences of the subject were also utilized to assess the limb status. These experiences include the subject’s feelings of missing fingers, hand, and arm as well as his experience during observing different movements (finger, wrist, hand, and elbow). Subsequently, a TNFR testing environment was setup to demonstrate the feasibility of our proposed technology. In the testing environment shown in Fig. 6.3b, the amputee was able to control the virtual prostheses reliably across a range of upper limb movements compared to his pre-TNFR experience. It is noteworthy that the TNFR technology is proposed by our research group and we hope it will revolutionize prostheses control technology and other related EMG-based rehabilitation device technologies in the future.

We further built a number of animal models recently to compare the performance of proposed TNFR technology with that of TMR and found the TNFR model obviously demonstrates better intramuscular nerve distribution compared to the TMR model. Additionally, we found out that bicep brachii muscles on the operational sides of the TNFR model achieved better wet mass compared to that of the TMR model.

6.3.4 Restoration of Sensory Feedback

Tactile sensation is an essential element for a dexterous hand manipulation. In an ideal bidirectional hand prostheses, the user’s intentions are first decoded through the efferent pathway, and then a nearly “natural” sensory feedback is delivered through the remnant afferent pathways, simultaneously in real time, as illustrated in Fig. 6.4.

Fig. 6.4
figure 4

Closed loop architecture for restoring sensory feedback

Full size image

The lack of proper sensory feedback mechanism limits prostheses users in grasping and manipulating objects, which makes the use of prostheses cumbersome and uncomfortable, leading to prosthetic abandonment. In fact, lack of sensory feedback schemes in the currently available multiple-DOF upper-limb prostheses have slowed down its clinical and commercial success [42]. Improper feedback of tactile perception may also lead to “over-grasp” causing damage to an object or “under-grasp” causing slip of object. Report from previous survey indicated that about 95% of male amputees desire to have force feedback in their prostheses and 88% placed more emphasis on grip force and proprioceptive information of prosthesis position and movement. With the rejection rate of upper limb prostheses reaching 39% and the increasing rate of people living with the loss of a limb projected to be doubled by the year 2050 [43], the need for integrating efficient sensory feedback mechanism into the prostheses becomes more necessary.

To restore the lost sensory feedback, Raspopovic et al. [44] were able to stimulate the median and ulnar nerve fascicles using transversal multi-channel intrafascicular electrodes (TIMEs) connected to the artificial hand sensors. The subject was able to control the prosthesis through information provided by the artificial sensors. This feedback enabled users to effectively control the grasping force of the prosthesis with no visual or auditory feedback. To restore a sense of touch that is close to the natural perception, the artificial receptors known as sensors need to first register the characteristics of the sensed object/environment [45], which works in a similar way to the receptor organs in humans. These sensory receptors are broadly classified into pain receptors, cold receptors, warm receptors, and four mechanoreceptors. Then, the receptors encode information into trains of action potentials and also possess characteristic innocuous temperature variation evoked by different afferents between ~5 and 48 °C [46]. The mechanoreceptors are grouped into slow-adapting receptors (SA-I and SA-II) and fast-adapting receptors (FA-I and FA-II) having different receptive fields to which they respond to mechanical stimulus, as well as different rates of adaptation. Also, tactile stimuli are transferred from the artificial receptors to the intact skin through the actuators. At this stage, signals are transported from receptors through nerve fibers to the brain where complex information are interpreted. The time taken for the signal to get to the brain is less than tens of milliseconds [47]. And lastly, a learning process is initiated by the central nervous system to adapt to the new type of afferent signals. A closed loop architecture for restoring a sensory feedback is described in Fig. 6.4. This figure shows the transduction and interfacing system which transmit the electronic signals and the control system which performs signal processing and interpretation.

To provide a sensory feedback mechanism, data are firstly acquired through the use of tactile sensors that come in contact with the target object (Fig. 6.4). Secondly, the analog-to-digital converter digitizes the signal at the signal transduction stage prior to further analysis. Before further signal processing, unwanted signal disturbances called artifact are removed by applying a wavelet transform and empirical mode decomposition strategies to reduce the noise in the signal [48]. At the last stage, the transmitted signal which mimics the real signal (action potential) is then sent to the brain for interpretation.

With the invasive mechanism, action potentials traveling in the form of electrical signals through the nervous systems could possibly regain all somatic sensory information with modality matching by implanting neural electrodes in the peripheral nerves [49]. This approach is able to restore the sense of touch by means of implantable nerve interfaces but always associated with risk of surgical procedure, and their long-term usability needs to be fully investigated [50]. There are three major approaches for invasive sensory feedback which are peripheral nervous system (PNS) stimulation, targeted sensory reinnervation (TSR), and central nervous system (CNS) stimulation. On the other hand, the promising noninvasive mechanism could be realized through techniques such as mechanotactile, vibrotactile, or electrotactile stimulations. To achieve any of these stimulation techniques, the characteristics of the actuator (weight, size, and power consumption) would need to be considered. Importantly, more research focus has been shifted toward the noninvasive sensory feedback mechanism in the recent years with a lot of work currently ongoing in this area.

In summary, integrating an efficient noninvasive sensory feedback mechanism into the currently available multifunctional upper limb prostheses will not only improve the control performance of the prostheses but also give users a feel of their true natural environment during ADL.

6.4 Conclusion

The human upper limb is an important part of the body that enables the accomplishment of a range of tasks in workplace, at home, and in social gathering during ADL. The loss of this part of the body severely limits the capability of amputees while undergoing ADL. In this regard, prosthetic device driven by different control methods have been built to restore the lost limb functions in amputees. At the forefront of these prostheses, control method is the EMG-based control which has attracted considerable attention in both the academia and industry in the recent years. Hence, this chapter focuses on EMG-based prostheses control methods with emphases on pattern recognition-based control as well as simultaneous and proportional control techniques, with details on the fundamental procedure of decoding limb movement intents. Also, a number of techniques for improving the overall performance of EMG-based prosthetic system were discussed with emphases on the advantages and limitations.