Keywords

1 Introduction

Hand is an important part of the human body, many things in daily life have to complete relying on the hand. Hand function is delicate and complex [1, 2], and it is difficult to recover after injury or hemiplegia, which seriously affect the normal life of patients [3, 4]. The traditional rehabilitation therapy is mainly based on one-to-one rehabilitation training between therapist and patient. In this process, patients’ subjective participation is low but the cost is high, so many patients cannot get effective treatment. However, clinical data shows that the repeated active motor intention of hemiplegic patients can accelerate the recovery of injured motor nerves and help shorten the recovery time. Decoding human motion intentions from surface EMG signals is an important method to realize prosthesis control [5]. Therefore, in order to better apply sEMG signals to human-computer interface control, decoding and recognizing hand motion patterns from sEMG signals has received extensive attention in recent years [6, 7].

In the early stage, the domestic research on EMG signal was mainly through the use of multiple electrodes to detect the EMG signal of specific muscle tissue including radial wrist flexor, ulnar wrist extensor, superficial flexor digitorum, palmaris longus and so on [8, 9]. But in this study, we designed a novel sEMG armband using 5 equidistant placed Delsys sEMG sensors to collect sEMG signals from subjects’ forearm. The armband simplifies the acquisition process of sEMG signals and makes it more convenient for users to wear. Meanwhile, two machine learning methods, convolutional neural network and multi-layer perceptron were selected as classifiers in this experiment in order to achieve a high classification rate and lay the foundation for prosthetic hand control.

2 Signal Acquisition and Processing

2.1 Experimental Preparation

9 healthy right-handed subjects (5 males and 4 females) with an average age of 23.3 years were recruited for the experiment. Each subject was given an informed consent prior to the experiment. The experimental procedures were approved by the Institutional Review Board of Shandong University and were in accordance with the Declaration of Helsinki (Fig. 1).

Nine common gestures were selected in the experiment, involving single finger movements, multiple fingers movements and grasping movements. Single finger movements include thumb opposite palm (TOP), index finger bending (IFB), middle finger bending (MFB) and ring finger bending (RFB); Multi-finger movements include pinching the index finger and thumb (TIP), keeping the five fingers together (KFT) and bending the three fingers (TFB) which is index finger, middle finger and ring finger; Grasping movements include clenching fist (CF) and cylinder grasping (CG), as shown in Fig. 2.

Fig. 1.
figure 1

Experimental process.

Full size image
Fig. 2.
figure 2

The nine predefined gestures.

Full size image

2.2 sEMG Acquisition

In this study, we designed a novel sEMG armband using 5 equidistant placed Delsys sEMG sensors to collect surface EMG signals from subjects’ forearms (Fig. 4), and the sampling frequency was 1000 Hz. To ensure sEMG signal quality, the corresponding arm skin of subjects were cleansed with a scrub and disinfected with 75% alcohol before wearing the armband, and then, which was placed at the position from about 1/3 of the left forearm to the elbow joint (Fig. 3).

Fig. 3.
figure 3

Delsys Trigno surface EMG tester

Full size image
Fig. 4.
figure 4

Position of the EMG armband

Full size image

The sEMG signal acquisition program was designed by the software LabView. The acquisition process mainly include the following steps. First, we used a test procedure to check whether the collected sEMG signals were normal or not. Next, Subjects were asked to sit using a prescribed seated position with their left forearm placing on a mat about 15 cm on the table (Fig. 4). And the sEMG signals of 9 hand gestures were collected in random order. There were 243 trials (9 hand gestures, 9 subjects, 3 times for one trial) in the experiment. One trial included 3 s of gesture forming time, 77 s of gesture holding time, and 3 s of gesture recovery time, as shown in Fig. 5. In order to facilitate the experiment of subjects, every trial has corresponding prompt tone, and the subjects can be also familiar with the experimental process before the formal experiment. 30-s rest was set between trials for subjects to prevent muscle fatigue.

Fig. 5.
figure 5

Single experiment process

Full size image
Fig. 6.
figure 6

Raw sEMG signals

Full size image

2.3 Feature Extraction

The data in the gesture holding time was used to the further research. The raw sEMG signals of 9 hand motions during 1-s gesture holding time from one representative subject are shown in Fig. 6. The sEMG signals should be preprocessed before feature extraction. In this study, a Butterworth bandpass filter (10-Hz–450-Hz) was used to filter the sEMG signals, and a 50 Hz odd notch filter was used to eliminate the specific power frequency interference. Meanwhile, in order to expand the amount of data and facilitate the training of neural network, the data in the gesture holding time was extracted via sliding window method with 200-ms window width and 200-ms sliding distance. The data after sliding window processing was directly input into the CNN network for classification, and the features of the data are automatically extracted by the convolution layer.

Similarly, 9 time domain features were extracted via sliding window method in the phase of feature extraction, including mean value (MEAN), root mean square value (RMS), variance (VAR), standard deviation (SD), mean absolute value (MAV), number of zeros crossing (ZC), waveform length (WL), number of slope sign changes (SSC) and Willison amplitude (WAMP), as shown in Table 1. The window width and sliding distance are the same as above. Then the 9 features were sequentially arranged as the MLP network’s input for training.

Table 1. 9 time domain features.
Full size table

3 Classifier Design

3.1 CNN Design

The model of the classification method based on CNN is shown in Fig. 7. The model consists of an input layer, an output layer and two convolution blocks. Every convolution block has a convolution layer (3 × 3 convolution kernel) and a down-sampling layer (2 × 2 convolution kernel), which had 32 filters in the first convolution block, but 64 filters in the second convolution block. Besides, we added the dropout layer after two convolutional layers to prevent overfitting. The softmax function were used in output layer to achieve 9 classification problems. Adam optimizer and cross entropy loss function were selected in this CNN design.

Fig. 7.
figure 7

The model of CNN

Full size image

3.2 MLP Design

The model of the classification method based on MLP is shown in Fig. 8. The model consists of one input layer, three hidden layers and one output layer, and the number of three hidden layer nodes are 64, 32 and 16 respectively. In addition, the layers are fully connected with each other in this model, and data labels adopts a form of One-Hot Encoding. Similar to the classification model design based on CNN, Adam optimizer and cross entropy loss function were used in the MLP classification model design.

Fig. 8.
figure 8

The model of MLP

Full size image

The parameters of the two networks including the kernel size, the number of layers and hidden nodes were determined by experiment.

4 Experimental Results and Analysis

The data from 9 gestures of 9 subjects who met the experimental requirements were analyzed and classified by two kinds of networks which were built with Python’s keras library. We used about 80% of the data as the train set and the remaining 20% as the test set. The results showed that both methods have high classification accuracy but there is a big difference in time consumption. We calculate the average accuracy of the network by taking 10 accuracies on the test set. The average accuracy of CNN network in the test set is 99.47%. The average accuracy of MLP network in the test set is 98.42%. This lays the foundation for the online recognition of hand movements and is of great significance for the rehabilitation of hand motor function in patients with hemiplegia in the future. In the case of obtaining similar accuracy, the time consumed by CNN network is longer than that consumed by MLP network. One round of training on CNN network takes about 1 min, but one round of training on MLP network takes about 1−s. A training process of CNN is shown in Fig. 9 and a training process of MLP is shown in Fig. 10.

Fig. 9.
figure 9

A training process of CNN

Full size image
Fig. 10.
figure 10

A training process of MLP

Full size image

5 Conclusion

This paper realizes the high accuracy classification of array sEMG signals. In this paper, a convenient EMG armband is designed to collect the sEMG signal of forearm. The method of machine learning is used to realize the off-line gesture recognition based on sEMG, and the high accuracy is achieved, which lays the foundation for the precise control of prosthesis control.