Continuous Time Normalized Signal Trains for a Better Classification of Myoelectric Signals

Abstract

State-of-the-art hand prostheses differ from those of previous generations, in that more hand positions and programmable gestures are available [4]. As an example, consider multi-finger prostheses like the i-limb^TM ultra from Touch Bionics or the Bebionic^TM Hand from RSL Steeper. Both prosthetic effectors are primarily controlled by myoelectric signals, derived with two or more cutaneously applied sensors, placed atop residual muscles. After preprocessing and classification of these signals, three to five different movement states or hand positions can be accurately distinguished. Zardoshti-Kermani et al. [5] show that the classification becomes increasingly difficult as the number of gestures grows, because decision spaces and feature clusters overlap [3]. Since static separation becomes increasingly difficult, Hudgins et al. [3] and Attenberger [1] used time-dependencies inherent to electromyographic (EMG) signals to improve classification.

1 Introduction

State-of-the-art hand prostheses differ from those of previous generations, in that more hand positions and programmable gestures are available [4]. As an example, consider multi-finger prostheses like the i-limb^TM ultra from Touch Bionics or the Bebionic^TM Hand from RSL Steeper. Both prosthetic effectors are primarely controlled by myoelectric signals, derived with two or more cutaneously applied sensors, placed atop residual muscles. After preprocessing and classification of these signals, three to five different movement states or hand positions can be accurately distinguished. Zardoshti-Kermani et al. [5] show that the classification becomes increasingly difficult as the number of gestures grows, because decision spaces and feature clusters overlap [3]. Since static separation becomes increasingly difficult, Hudgins et al. [3] and Attenberger [1] used time-dependencies inherent to electromyographic (EMG) signals to improve classification.

In this contribution, we bring forward a method to classify, for example, EMG signals before a motion or gesture is finished, by using continuously normalized EMG feature trains. Building on previous work [2], in which we classified EMG signal traces after the completion of a movement sequence, we here depart from established approaches.

2 Method

With previous solutions, recordings of an EMG signal are divided into individual independent sections, then preprocessed and finally classified. This leads to the loss of temporal dependence within a signal path. This procedure is called the standard method in the following. In previous work we showed that by time normalization for each gesture, signal-trains are formed, which are similar for one gesture, but can be clearly distinguished from other gestures [2]. It was also explained that this improves the classification quality for a variety of classifiers. Figure 1(a), depicts 15 normalized RMS signal-trains, derived from two sensors for a hand-flexion, which obviously look very similar. If one combines several such EMG-feature-trains of a motion sequence into an enveloped body, the similarity of the individual signal courses become even more obvious. When building enveloped bodys, outlier points are detected by calculating a probability density function or by the Mahalanobis distance followed by select point removal. For the gestures: Extension, Flexation and Fist the enveloped bodys are pictured in Fig. 1(b). The three envelopes are clearly separated from each other for a large part. The strongest overlap occurs towards the end of the signal when it becomes weaker and the different envelopes start to merge. When a new signal-train is generated during the use-phase of the prosthesis, it is then very easy to determine whether it belongs to a certain body or not. The body represents hereby the training data and a single signal-train within relates to a single motion.

Fig. 1.

Signal paths become enveloping bodies.

In previous work when only normalization was employed, the signal trains as well as the bodies could only be calculated on completion of a movement. This posed the main drawback. To take advantage of normalization before a gesture is finalized, we switched to building a body in discrete time-intervals for the classification. To achieve this, recorded samples are added at each time step until the gesture is complete. For each time step, the envelope can be formed analogously to the normalization. Therefore, classification can also be done after each time step. In the following examination, a time step is 100 ms long, thus the first envelope is formed during this time and a corresponding signal train can also be classified for the first time in this period. After further 100 ms the now newly recorded data are added. By this a data basis for the envelope of 200 ms and a signal train of the same length is created. These steps are repeated until a threshold value indicates the end of the movement of a gesture. This leads to the fact that the last envelope formed is identical with the envelope generated by the normalization as shown in Fig. 1(b) for, e.g., flexion-, extension- and fist-gestures. We call this second method continuous normalization. It is expected that the classification quality decreases with continuous normalization when compared to the initially used normalization method. Figure 2 presents four resulting envelopes of the gesture flexion at different times-steps. The first frame in the upper left corner shows the enveloped body after 100 ms. No clear structure has been established. The second frame in the upper right corner depicts the envelope after a just over 20% of the average movements. The first characteristic of a particular body can be identified. Like the clear bending and deflection in the direction of RMS1. The third frame shows the envelope after about 60%. It is loosing intensity and approaches low values. A characteristic body has been formed already at this point. The last frame shows the body after completion of the movement which is identical to the body created by the normalization.

Fig. 2.

Continuously forming body of a gesture. Shown in the frames is a body at the beginning of the gesture, after about 20% and 60% of the time, and at the end of the movement.

Which shows that at the beginning of a movement, individual envelopes of gestures are very similar and differ only with increasing data volume, so that these are better distinguishable. An exception are envelopes of gestures, which are basically easy to discern, such as flexion and extension. These should be clearly separable at an early stage even with continuous normalization. During the use-phase of the prosthetic device, a continuous signal-train is generated from sensor-data, normalized and matched to the body envelopes of the training phase. In this research, all experiments were performed with a DELSYS^® Bagnoli-4 EMG system and a National Instruments Ni USB-6229 16-bit data acquisition system^TM with two Sensors.

For the study five probands performed nine different gestures. Each gesture was repeated 15 times in one day. Data was collected on six days spread over three weeks. Thus, each gesture was repeated 90 times. Hence, 810 repetitions per person were performed. The evaluation thus covers a total of 4050 recorded movements. The following nine hand gestures were performed:

For each execution of a gesture, four features were calculated for the EMG signal to enable classification:

In order to investigate whether continuous normalization leads to an overall improvement in classification quality, 22 different classifiers were considered. The classifiers hereby belong to the four main groups: discriminant analytics, decision trees, support vector machines (SVM) and nearest neighbor algorithms (KNN). The F1 score is used as the classification metric and to assess of the methods, because the armonic mean of the precision and recall is more informative than the accuracy (ACC) measure, especially when there are many classes to distinguish. A comparison is made between the results of the standard method and the two normalization methods.

3 Results

The average F1 score is shown in Fig. 3 achieved by the different classifiers across all days, subjects and gestures, as a box plot. Clearly, the Normalized Method leads to an improvement in the F1 score, compared to the Standard Method for each classifier. This is because, the quantile 25 as well as the quantile 75 and the median are higher than the corresponding counterpart. When comparing the continuous normalization and the standard method, the F1 score is improved for 18 out of 22 classifiers compared to normalization. Note, that not every classifier improves. Three out of 22 classifiers deteriorate significantly due to continuous normalization. These are LinearDiscriminant, QuadraticDiscriminant and Supscpace Discriminant. This shows that the methods which use a discriminant to distinguish the gestures are not suitable for continuous normalization. In contrast, BaggedTree, SubSpaceKNN and SVMFineGaussian achieve significantly better results. The latter is particularly noteworthy, as it manages to increase the median F1 score of the standard method from 40% with normalization to 54% and to 70% with continuous normalization. The three classifiers which achieved the best F1 score are the BaggedTree, the SVMCubic and the SVMQuadratic, see Fig. 1. Further evaluations are performed with these three classifiers. The achieved quantiles of the classifiers are shown in Table 1. Here, we can see once again how all three quantiles improve when normalization methods are used. It is interesting to note, that this also applies to continuous normalization. Which improves the median by 13 percentage points (p.p.). The quantile 75 even reaches an average value of 94%. Therefore, it can be seen that these three classifiers not only succeed in achieving a higher F1 score than the standard method, but also perform better than normalization.

Table 1. The 25, 50, and 75 quantiles of the F1 score of the best three classifiers BaggedTree, SVMCubic and SVMQuadratic across all subjects, gestures, and days.

Fig. 3.

Box plot of the average F1 score achieved across all subjects, days, and gestures, for the three methods standard, normalized, and continuous normalization.

When considering individual gestures, F1 scores, as depicted in Fig. 4, result in an improved classification for eight out of nine gestures across all subjects. The sole exception is Extension, as the median remains almost unchanged, between 94% for normalization and 92% for continuous normalization. The $Q_{75}$ is 96% for continuous normalization and reaches its maximum with normalization at 98%. The part that deteriorates the most for this gesture is $Q_{25}$. It falls from the standard of 85% to the minimum of 82% for continuous normalization and 85% for normalization only.

Fig. 4.

Box plot of the F1 score of the three best classifiers for nine different gestures averaged over all subjects and recording days.

In conclusion, for an Extension the standard method is marginally better than continuous normalization method. The achieved values of the F1 score are on a quite high level with a median of 92%, Table 2. The gestures that benefited most from the new normalization method are Cocontraction, Index and Thumb-Up. Cocontraction’s $Q_{50}$ was improved by 27 percentage points. The improvement of the gesture is so significant that the $Q_{25}$ of the continuous normalization at 82% is 10 p.p. higher than the $Q_{75}$ of the standard method. Likewise, the Index gesture which improves by 22 p.p. at its median. The $Q_{25}$ of the continuous normalization is also 74% and therefore 6 p.p. higher than the $Q_{75}$ of the standard method, which reaches a value of 68%. The third strongest improvement is found in the Thumb-Up gesture. Which increases by 17 p.p. on median compared to the standard method. The $Q_{25}$ of the continuous normalization is equal to the $Q_{75}$ of the standard method at 76%, while the $Q_{75}$ of the continuous normalization reaches a value of 91% and thereby is 15 p.p. higher than the counterpart of the standard method. Also worth mentioning are the improvements for the OK, Fist and Pronation gestures, as listed in Table 2. These range from 15 to 12 p.p. while, the median F1 score improves by more than 10 p.p. for six out of nine gestures. Which are in the range of 82% and 93%. This is a significant increase compared to the standard method.

Table 2. Median F1 score of the best three classifiers comparing the standard and continuous Normalization method. Averaged across all subjects and days.

4 Conclusions

In this work, it is demonstrated that the method for normalization of EMG signals can be improved by modification such that a classification is already possible before the movement is completed. With the new method of continuous normalization it is possible to greatly-improve the classification quality similar to the method of normalization and to separate numerous gestures in distinct classes. By the continuous normalization it comes to an increase of the dimensionality within the data. This leads to the situation that not all classification methods yield better results. Especially classifiers which use a discriminant are not suitable for this method. Decision trees, KNNs and SVMs have proven to be useful and achieve a considerably better classification quality with continuous normalization compared to the standard method. This study shows that the three classifiers which yield the best results for continuous normalization are BaggedTree, SVMCubic and SVMQuadratic. For each of the nine gestures examined, on average, a higher F1 score resulted when continuous normalization is employed. This is particularly noteworthy since this method results in a median above 80% for each gesture. Three gestures even manage a median above 90%. The standard method manages this with only one gesture. The lowest median amounts to only 59% showing that continuous normalization can classify significantly more gestures with a higher accuracy in comparison to the standard method. Since this is now also possible during the movement of a gesture, the results clearly show that this method is suitable for the use in modern prostheses to appropriately control the large number of possible gestures a modern prosthesis can perform.

This research is funded by dtec.bw - Digitalization and Technology Research Center of the Bundeswehr - project VITAL-SENSE.