Keywords

1 Introduction

There are many kinds of control methods for electric wheelchairs. Traditional methods are usually dependent on the hand to control the joystick or keypads, which are unavailable to the people who have disability in hand. Researchers have explored a variety of new control methods based on biological signals, such as tongue movement signals [1], eye movement signals [2] and EEG-SSVEP signals [3]. In addition to the human machine interface (HMI) mentioned above, sEMG also plays an important role in the field of HMI because of its non-invasive characteristics. State-of-the-art researchers have explored the characteristics and applications of sEMG. Because sEMG can capture information of hand motion by sensors with stable robustness and accuracy rate of it, a kind of hand motions recognition systems based on sEMG has been used in controlling the prosthetic hand [4]. While sEMG is usually accompanied by noises, identifying action intent accurately is not easy [5]. It is significant to choose suitable methods of feature extraction and classification algorithms for sEMG signals.

This paper proposed a control method in which the hand motions recognized from the sEMG signals were selected to control the electric wheelchairs. In the hand motion recognition system, RMSR and AR model [6] were selected to extract time-domain features of the sEMG, and SVM [7] was chosen as a classifier to identify the hand motions.

The rest of this paper is arranged as follows. Section 2 introduces three key components of the control system which includes data segmentation, feature extraction and classification. Experiment and discussion are shown in Sect. 3. The conclusion and future work are presented at the end of this paper.

2 Methodology

The architecture of the proposed myoelectric control method was shown in Fig. 1. The system can regard as two stages. The first part recorded the features of muscle activity characteristic that were using to train SVM model. In other word, extent of muscle contraction mapped related hand motion label. Stable EMG data were difficult to obtain during switch of hand motions. Thus, we need to segment active data. After extracting features based on previous work [8, 9], we trained SVM classifier with given labels of default hand motions. Once that model was been built, the second part which is online control system for a wheelchair. According to the defined mapping between hand motions and move orders, control commands corresponding to the identified action will be sent to the control system of the electric wheelchair through WIFI. Finally the wheelchair moves as the user desired. The main method used in whole control strategy are described herein.

Fig. 1
figure 1

The architecture of the proposed control system

Full size image

2.1 Data Segmentation

Continuous stream of inputs signals is one characterize of on-line recognition system. Information of hand motions contained in continuous signals. To distinguish between contracting state and relaxed state, it is important to choose a suitable segmentation way. Comparing with on-line system, off-line system always extract useful signals by manually as previous work, because researcher can ensure quality of signals of segmentation. Nonetheless, an intelligent recognition processing requires automatically determine the start and the end of active hand motions. Main challenge of segmentation is movement epenthesis [10, 11]. To solve this problem, this paper proposed a method to segment signals automatically using inertial measurement unit (IMU). IMU consists of accelerometer (ACC) and gyroscope, it had widely applied to dynamic detection [12]. The segmentation consists of three steps: calculated muscle energy, estimated motion amplitude, and segment using the threshold by experiments.

During movement, value of several channels of sEMG signals will amplify due to muscle contraction. The energy of sEMG reflect movement intension in time-domain. Thus, we calculated energy of sEMG at the beginning. The formula as shown in Eqs. (1) and (2). In formula (1), \( n \) is the number of channels. To smooth out short-term fluctuations, a moving average algorithm was applied. Here, \( w \) is size of window.

$$ EMG_{ave} (t) = \frac{{\sum\nolimits_{c = 1}^{n} {emg_{c} } }}{n} $$
(1)
$$ EMG_{energy} (t) = \frac{1}{w}\sum\nolimits_{i = t - w + 1}^{t} {EMG_{ave} (i)^{2} } $$
(2)

After obtaining energy of muscle contraction, all EMG data that are linear array would been segment by this energy of EMG. There are two threshold, onset threshold and offset threshold, to avoid erroneous judgments because of data noise. The onset threshold was greater than the offset threshold. Then, we estimated motion amplitude from IMU signals. In laboratory environment experiment, most experiment defaulted subjects performed hand motion in a quiet status. Once they performed hand motion, forearm would have a shake that is easy to detect. We amplified this active shake to judge the subject start or stop to perform hand motions.

$$ M{\text{o}}tion_{gyro} = \sum {(gyro_{x}^{2} \text{ + }} gyro_{y}^{2} + gyro_{z}^{2} ) $$
(3)

For sEMG signals and IMU signals generated at the same time index, the energy of muscle contraction could be segmented between two active shake signals, which were from IMU signals.

2.2 Feature Extraction

Feature extraction is a crucial component in pattern recognition which influents accuracy of classification to a great extent. Most literature extracted various feature based on their own acknowledge and previous research [13]. Generally, time domain (TD), frequency domain (FD), and time-frequency domain (TFD) is the traditional partition method of feature of sEMG signals. Time domain feature extraction is relatively simple, so the time domain analysis methods in the sEMG signal application have been more extensive. For example, many mature EMG products are using EMG signal characteristics of the time domain as the control signal of the prosthetic hand. In this paper, we used time domain features as an input to SVM classifier.

In time-domain category, there are quite a lot methods, such as mean absolute value (MAV), root mean square (RMS), waveform length (WL), and AR model. And RMS can indicate the effective value of sEMG signals, reflect the average power of the signal and represent the energy information. However, RMS is easy to be influenced by the power of muscle contraction. To solve this problem, we replaced RMS by RMSR. Because RMSR has the advantage over RMS and it is unaffected by different grip strength. The RMSR value reflected ratio of two channels. And the formula of it is shown as:

$$ RMSR\text{ = }\frac{{RMS_{1} }}{{RMS_{2} }} $$
(4)

Auto regressive model, as a linear predicted model, widespread used in analysis of time series data such as electromyography. Based on \( n \) continuous data of the time series data, we can find a suitable model to express its rule and calculate required value of next data \( (n + 1) \). Therefore, AR model able to reflect attributes of EMG signals applied in recognizing corresponding hand motion. Four-order AR model coefficient was proved as best choice in this circumstance. A standard AR model is described in Eq. (5).

$$ y(t)\text{ = }\sum\nolimits_{i = 1}^{4} {a_{i} \,^{\text{ * }} \,y(t\text{ - }1)\text{ + }e(t)} $$
(5)

The \( a_{i} (i{ \in }[1,4]) \) is AR model coefficients that we used as features. And \( e(t) \) is white Gaussian noise. In this paper, we used Burg algorithm to calculate AR model coefficients.

So in this paper, RMSR and coefficients of AR model are selected to extract features of sEMG signals. A nine-dimensional feature vector which includes four AR model coefficients per channel and a RMSR values is formed.

2.3 Classification

For pattern-recognition-based method myoelectric control, an appropriate classifier could improve the accuracy rate. Several types classifier were developed within this area in the past decades such as artificial neural network (ANN) [14], linear discriminant analysis (LDA), fuzzy systems, Bayesian techniques, and some hybrid algorithms. Recently, SVM as one of the most outstanding supervised classifiers attracted more and more attention and that method has showed improved results applied for myoelectric signals classification [15]. SVM is a sorting algorithm which can improve the generalization ability of machine learning by minimizing structured risk. Moreover, it can realize the minimization of empirical risk and confidence range, to achieve a good statistical law when the sample size is small.

Originally, SVM was designed for classifying binary proble. However, many researcher wanted to use SVM to deal with multiple problem depend on great performance of SVM. Thus, there are two traditional construct method to form adapted multiple problem classifier. The directed method is considering parameter computation of multiple problem into one optimization problem by modifying target function. It looks sample, but computation complexity is high. The indirected method is constructing and combining several binary classifier. There are two ordinary ways to combine binary classifier, one-against-all (OAA) and one-against-one (OAO). In OAO method, binary classifiers are trained as combined way. The OAO method has advantage of that it conducts binary classifications on all pairs of classes, and decided classification results based on the probability for each class. For example, we wanted to classified new sample into three classes. In Fig. 2, each binary classifier would output a result. A class that gains most votes is seen as the final output that here is label one. In particular condition that each class gets equal votes, we default the class which label is smallest as the final output. In this paper, we employed library LibSVM [16] as the classifier that using C-SVM and OAO method above.

Fig. 2
figure 2

The schematic diagram of SVM OAO method

Full size image

3 Experiment Procedure and Result

3.1 Experiment Preparation

The hand motions recognition method of this paper uses one off-the-shelf sensors that named MYO armband (Thalmic, Inc., Firmware version 1.5.1970). The sensor integrated eight medical grade electrode and one inertial measurement units that contains three-axis gyroscope, three-axis accelerometer and three-axis magnetometer. To verified high efficiency of proposed method, we only used two channels of sEMG signals to classifying five hand status. The MYO armband was putted on the forearm of the subject, and we used the two bipolar electrode on forearm to recognize hand motions from the extensor the flexor carpi radialis muscle, and the extensor digitorum muscle approximately.

To test the proposed control method in real-world environment, this paper designed an on-line experiment and a control experiment.

In the on-line experiment, five healthy subjects who had no history of disease of neuromuscular took part in. The ratio of female is 2–3. This experiment aimed to recognize four daily hand motions (wrist extension, wrist flexion, stretch, and fist) and relax state showed in Fig. 3.

Fig. 3
figure 3

The predefined hand motions and MYO armband. In bottom line, there were F (fist), S (stretch), WE (wrist extension), WF (wrist flexion), and R (relax) from left to right

Full size image

The procedural experiment contains two sections where each section consists of five hand states. Interval between two hand motions was relax state. Between each section, the subject has break time and decides when next section start. The first section was trained part. The subject was required to perform WE (wrist extension), WF (wrist flexion), S (stretch), and F (fist) in order. Each hand motion was collected for ten times as a training set. Therefore, training set of one subject includes 40 samples. During training phase, a features-to-actions map (a SVM model in this paper) was built in supervised machine learning condition and saved. Later section is test parts. The sequence of hand motions was {Relax, WE, WF F, S}. Thus, the subject needed to adjust order to comply with requested hand motions. In testing phase, the on-line experiment required each subject to perform each hand motion for a certain time, during which, there contained 20 samples. In total, there were 2000 samples rely on five subjects. These samples were used to verify the performance of the saved model in training phase by analyzing accuracy rate of hand states classification.

In the control experiment, the five subjects in on-line experiment also took part in. And each subject sat on the wheelchair and controlled it to move in according with predetermined route showed in Fig. 4. Firstly, the wheelchair stopped at point A was controlled to go forwards to point B. After stopping at point B, the wheelchair turned left 90° and stopped, then, the wheelchair turned right 90° and stopped. Finally, the user controlled the wheelchair go back to point A and stop. Each subject was required to finish the ten times which was regarded as ten samples. The label of each sample was defined as success or failure depended on whether the subject completed the whole task without any control error or not.

Fig. 4
figure 4

Moving path of the wheelchair in experiment

Full size image

In the whole experiment, the wheelchair was controlled based on a mapping between four kinds of hand motions as well as relaxed state and different control commands for control experiment which was shown in Table 1. The SVM models trained and saved in on-line experiment were applied in control experiment to recognize the hand motions and relax state.

Table 1 Mapping between hand motions and control commands
Full size table

3.2 Results and Discussion

Thanks to features-to-actions map had been built, this method can make system adapted to anyone quickly only based few training data. In previous off-line classify problem, we used cross validation (CV) to verification proposed method. However, real-time performance allowed us test more samples in circumstance that is close to practice environment. An improvement comparing with off-line system, we do not need to increase samples to divide sample into several pieces usually used in previous literature. Results of on-line experiment and control experiment were respectively listed in Table 2 and Table 3.

Table 2 The result of the classification
Full size table
Table 3 The success rate of the control experiment
Full size table

It can be found that the average accuracy rate of the hand motions recognition in real-time has reached 95%, which showed that the proposed method above performed well. In the control experiment, the average success rate of control reached 88%, which was a little bit lower compared with result of on-line result. Because the subject was required to finished a task which contained many classification samples. Only if all these samples were identified correctly, can this control sample be regarded as a positive one. So the result of 88% also showed that the proposed control method in this paper was feasible and practical.

4 Conclusion and Future Work

In this paper, we proposed a control method that controls electric wheelchairs based on the recognition of sEMG signals. Fusion time feature of AR coefficient and RMSR value are adopted for increasing classification accuracy rate. We employed SVM classifier to seek out correspondence from EMG signals to predefined hand motion. Quantitative and complete assessment have verified that proposed method could control wheelchair for a high accuracy in practice environment. In the designed on-line experiment, the accuracy rate of hand motion recognition reached 95%. The control experiment also acquired an average success rate of 88% in control.

Our ongoing work is to use multiply types of sensor for analysis hand motions as we got IMU and EMG information already. In detail, we found EMG signals would drifting after long time experience. Many reasons cause this problem including muscle fatigue, skin sweat, and sensors displacement and so on. But the IMU signals would not been influenced by muscle fatigue and skin sweat. This provided a new way that using IMU information to calibrate EMG information. In other word, we should retrained system parameter once we detected EMG signals drifting by comparing with EMG signals and IMU signals.