1 Introduction

During the contraction process, the muscle generates a motion unit action potential on the surface of the skin, which is measured and recorded to form sEMG [1, 2]. Because of the advantages of non-invasive and convenient, sEMG has been widely studied in the control of prosthetic hands [3, 4]. With the development of virtual reality technology, simulated rehabilitation training can provide more abundant training methods for patients. Therefore, the use of sEMG can not only provide a more diverse means of interaction, but also real-time detection of patients’ muscle information, such as fatigue, which can be applied to conduct patients’ rehabilitation training. With the maturity of wireless sensor technology, there will be more and more human-computer interaction methods based on multi-sensor networks [5,6,7]. Based on sEMG, patients with upper limb disability can independently control the grasping mode and gripping power of the myoelectric artificial hand. Since sEMG cannot only be applied to recognize patterns of human hand operations such as gestures and wrist angles [8, 9], but also detect gripping strength during human hand operation and speed control during gesture operation, hence, many studies are looking for the relationship between the output of hand force and surface electromyogram (SEMG) signals [10]. At the same time, tactile feedback can also be realized by combining the electromyographic stimulation, which enables people to have better on-the-spot perception and interactivity in the process of operating and controlling the manipulator [11]. Li et al. [12, 13] explored the importance of tactile feedback in human perception by studying the way of electrical stimulation, which can enhance the interaction between human and machine. Therefore, the current research on surface EMG signals is not limited to the control of myoelectric artificial hands, but also has more research on human–computer interaction. Thence, the research on sEMG signals is not only limited to the control of myoelectric artificial hands. Fang et al. [14] designed and developed a portable acquisition device to achieve identification of several typical gestures, which promoted greatly the application of sEMG in human–computer interaction. At present, vision-based gesture recognition already has good results [15,16,17,18,19]. He et al. [20, 21] optimized the sparse representation method and improved the real-time and accuracy of the gesture recognition process; however, they ignored impact of background environment. As the application of deep learning and intelligent algorithm is more and more extensive and mature, such as CNN [22]and evidential reasoning [23, 24], multi-sensor data information processing system has also been widely concerned [25]. Li et al. [26,27,28,29] further optimized and improved the effect of gesture recognition by designing a corresponding deep learning model. Jiang et al. [30,31,32] achieved gesture recognition and extraction in complex backgrounds by combining depth information and skeleton extraction algorithm. Meanwhile, Sun et al. [33,34,35] combined surface EMG signals and image information to identify gestures, which further improved the visual-based human–computer interaction performance under dynamic environment for further improving the effect of human–computer interaction in dynamic environment. Although the visual-based human–computer interaction method has achieved a lot of research results [36,37,38], it can only obtain the operation mode of the human hand and cannot better obtain the information such as the magnitude of the force during the crawling process, requiring additional sensors such as data gloves and so on to do it. Additional equipment may increase the complexity of the system, affecting the interactive experience [39,40,41,42,43], while single-modal interactions are no longer sufficient to meet existing market needs, so the performance of interactive feedback need to be further improved included haptic feedback. Gesture recognition based on surface EMG signals has a lot of research results, which can basically meet the effects of human–computer interaction. Furthermore, some researchers hope the recognition of multimodal information based on sEMG. In order to establish the relationship between the surface EMG signal and the force in the human hand operation, a mathematical model and a machine learning method are generally adopted [44,45,46,47]. And some researchers hope to improve the gesture recognition rate with gaining some new features [48]. In order to reduce the influence of date difference, Qi [49] utilized intelligence algorithm to make it. Since the surface EMG signal is a nonlinear dynamic signal, this makes it difficult to establish a suitable mathematical model. The black box model is established, with the surface EMG signal as the input, and the force signal as the output to establish the corresponding nonlinear relationship is a hot spot for researchers to focus on [50,51,52,53]. Commonly used are support vector machines, convolutional neural networks, LSTM, BP neural networks, and fuzzy neural networks [54,55,56]. In order to accurately recognize the force of the human hand during operation, a force gauge is often applied to measure the magnitude of the force. However, it is difficult to guarantee the accurate output of force in the process of control, which is generally controlled within a certain range, such as large force, general force, and small force. In order to deal with this situation, the method of fuzzy control is generally adopted. For better realizing the control of the manipulator by the human hand, it is necessary not only to establish the relationship between the force and the surface EMG signal, but also to construct a corresponding mechanical control scheme. Due to the presence of muscle fatigue, the control effect of the force based on the surface EMG signal is not ideal. Therefore, some researchers avoid muscle fatigue by setting corresponding conditions. On the other hand, by extracting new eigenvalues to judge the degree of muscle fatigue, thereby reducing the effect of muscle fatigue on model accuracy, such as fuzzy entropy [57,58,59]. In this paper, the traditional features of surface muscle electrical signals are analyzed and selected, and the fuzzy entropy value is extracted as a new eigenvalue, which reduces the impact of muscle fatigue on the system. Then, the fuzzy neural network is constructed to establish the relationship between the surface EMG signal and the force, and the force evaluation during the manual operation is well realized.

2 Feature selection and extraction

2.1 Collection and segmentation of sEMG

The sEMG signal acquisition instrument used in this paper has 10-bit A/D conversion accuracy and sampling frequency of 1000 Hz. It communicates with the host computer through the USB interface and can simultaneously collect 16 channels of sEMG signals at the same time. The use of an electronic grip gauge enables the acquisition of forces during manual operation. The acquisition of surface muscle electrical signals is shown in Fig. 1. Although 16 channels are shown in the figure, channel 1st is chosen because only grasped gesture in the following experiment and whose signal changes obviously (Fig. 2).

Fig. 1
figure 1

Surface electromyography collection equipment

Full size image
Fig. 2
figure 2

Signal acquisition system interface

Full size image

Since the paper only explores the relationship between surface EMG signals and grip strength, it is necessary to extract the surface muscle electrical signals during the grasping process. The methods of segmentation of myoelectric signals are generally manual segmentation, time windows, and energy based values. Manual splitting has a large workload, and it takes a long time to process large data samples and is susceptible to human factors. Extraction methods based on fixed-time panes lead to the loss of data, which easily leads to the loss of data. As shown in Figs. 3 and 4, the sliding window is difficult to ensure accurate extraction of the full grip. Therefore, this paper uses the energy envelope method (Fig. 5) to extract the surface EMG signal that realizes a gripping action of the surface EMG signal, as shown in Fig. 6. Firstly, the envelope of EMG signal is extracted by calculating and extracting, and then the minimum points are obtained. Finally, signal segmentation is realized by locating the adjacent minimum points.

Fig. 3
figure 3

Original signal of sEMG

Full size image
Fig. 4
figure 4

Sliding pane signal acquisition error diagram

Full size image
Fig. 5
figure 5

Signal envelope

Full size image
Fig. 6
figure 6

Schematic diagram of signal segmentation

Full size image

2.2 Extraction of electrical characteristics of traditional surface muscles

Feature extraction of myoelectric signals is usually a time domain method, a frequency domain method, and a time frequency domain method. Commonly used time threshold feature extraction methods include absolute mean, variance, zero crossings, and Willis amplitude.

Absolute value mean

sEMG exhibits strong randomness in amplitude, and the positive and negative amplitudes are usually symmetrical. Absolute value operation converts the amplitude of the signal all A positive value can intuitively reflect the contractile strength of the muscle. The larger the absolute value, the greater the contraction strength of the muscle. The mathematical expression for the absolute value of the mean value under the sliding window is as follows:

$$ {\mathrm{MAV}}_i=\frac{1}{N}\sum \limits_{j=i-N+1}^i\mid {x}_i\mid $$
(1)

In which, xj is current sampled data, N is sliding window length.

Variance

Variance is a measure of the degree of dispersion of a random variable or a set of data. The larger the variance value, the larger the difference between most data and the mean. The mathematical expression is:

$$ \sigma =\frac{1}{N}\sum \limits_{i=1}^i\left({x}_i-\overline{x}\right) $$
(2)

In which, \( \overline{x} \)is the mean of data, N is the length of data.

Zero crossings

The zero-crossing (ZC) point describes the number of times the signal passes through the 0-axis during a period of time. This feature estimates the frequency domain characteristics of the signal from the perspective of the time domain. The mathematical expression of the zero-crossing point under the sliding window can be:

$$ {\mathrm{ZC}}_i=\sum \limits_{j=i-N+1}^{\mathrm{i}}\operatorname{sgn}\left({x}_j{x}_{j-1}\right) $$
(3)

In which, \( \operatorname{sgn}(x)=\Big\{{\displaystyle \begin{array}{c}1,\kern0.84em x>0;\\ {}0,\kern0.84em x\le 0;\end{array}} \).

Willison magnitude

Willison’s magnitude is a measure proposed by Willison in 1963 to calculate the number of changes in signal amplitude over a period of time. The Willison amplitude under the sliding window can be expressed as:

$$ {\mathrm{WA}}_i=\sum \limits_{j=i-N+2}^if\left({x}_j-{x}_{i-1}\right) $$
(4)

In which, \( f(x)=\Big\{{\displaystyle \begin{array}{c}1,\kern0.36em \mid x\mid >\mathrm{threshold}\\ {}0,\kern1.68em \mathrm{others}\end{array}} \).

Wavelet energy value

sEMG is essentially a nonstationary bioelectrical signal. Therefore, in addition to these traditional surface EMG signals, new features can be extracted by wavelet packet transform. Wavelet packet transform is a signal analysis method developed on the basis of wave analysis theory. It has multi-scale analysis capability and good time–frequency localization characteristics. It can provide higher time-frequency resolution than wavelet transform, which is very suitable for non-stationary analysis.

Transient and time-varying characteristics of the signal

Wavelet packet transformation includes two processes: wavelet packet decomposition and wavelet packet reconstruction. The energy characteristic value of the surface muscle electrical signal is extracted by wavelet decomposition.

Wavelet decomposition algorithm:

$$ \left\{\begin{array}{c}{d}_m^{j+1,2n}=\sum \limits_l{d}_l^{j+1,2n}{h}_{l-2m}\\ {}{d}_m^{j+1,2n}=\sum \limits_l{d}_l^{j+1,2n}{g}_{l-2m}\end{array}\right\} $$
(5)

In which, hl − 2m and gl − 2m are two functions under scalej. With decomposition algorithm, the coefficient \( {d}_l^{j+1,n} \) can be used to calculate the coefficient \( {d}_m^{j+1,2n} \) and\( {d}_m^{j+1,2n} \).

Wavelet reconstruction algorithm:

$$ {d}_l^{j+1,n}=\sum \limits_m\left({d}_m^{j+1,2n}{h}_{k-2m}+{d}_m^{j+1,2n+1}{g}_{k-2m}\right) $$
(6)

With this, the coefficient \( {d}_m^{j+1,2n} \) and \( {d}_m^{j+1,2n+1} \) can be used to rebuild\( {d}_l^{j+1,n} \).

2.3 Extraction of fuzzy entropy features

In the process of muscle movement, muscle fatigue is a physiological function change that must occur, which has a relatively large impact on the performance of subsequent pattern recognition. At present, there is no uniform standard for the definition, production mechanism, and evaluation index of muscle fatigue. The commonly used methods of muscle fatigue test include oxygen content in the blood, heart rate, muscle strength, and surface electromyography signals. The surface EMG signal contains the state of muscle activity during muscle contraction, which provides information for the analysis of muscle fatigue status and reflects the fatigue state of the muscle. Muscle fatigue and surface EMG signals are a nonlinear mapping relationship, and thus fuzzy theory is used to evaluate and analyze muscle fatigue. Fuzzy entropy is a nonlinear method used in fuzzy theory. It is mainly used to characterize the complexity of the signal. The greater the complexity, the larger the entropy value. Compared with the sample entropy, it has a smaller dependence on the length of the acquired signal and has a better correlation. Compared with the approximate entropy, it can give full play to the quantitative and reliable advantages. Fuzzy entropy estimates the description of similarities in time series data sets.

For a given time series{u(i) : 1 ≤ i ≤ N}, the vector sequence\( \left\{{\mathbf{X}}_i^m\right\} \), i = 1, …, N − m + 1 should be obtained according to Eq. 7

$$ {\mathbf{X}}_i^m=\left\{u(i),u\left(i+1\right),\cdots, u\left(i+m-1\right)\right\}-u0(i) $$
(7)

In which, \( u0(i)=\frac{1}{m}\sum \limits_{j=0}^{m-1}u\left(i+j\right) \).

\( {\mathbf{X}}_i^m \)is a mdimensional vector, which is composed of normalized mconsecutive samples starting from u(i). The maximum distance between \( {\mathbf{X}}_i^m \) and \( {\mathbf{X}}_j^m\left(j\ne i\right) \) is defined as:

$$ {d}_{ij}^m=d\left[{\mathbf{X}}_i^m,{\mathbf{X}}_j^m\right]={\max}_{k\in \left(0,n-1\right)}\mid \left(u\left(i+k\right)-u0(i)\right)-\left(u\left(j+k\right)-u0(j)\right)\mid $$
(8)
$$ {D}_{ij}^m\left(n,r\right)=\mu \left({d}_{ij}^m,n,r\right) $$
(9)

In which, fuzzy function \( \mu \left({d}_{ij}^m,n,r\right) \) is exponential function

$$ \mu \left({d}_{ij}^m,n,r\right)=\exp \left(-{\left({d}_{ij}^m/r\right)}^n\right) $$
(10)
$$ {\phi}^m\left(n,r\right)=\frac{1}{N-m}\sum \limits_{i=1}^{N-m}\left(\frac{1}{N-m-1}\sum \limits_{j=1,j\ne i}^{N-m}{D}_{ij}^m\right) $$
(11)
$$ {\phi}^{n+1}\left(n,r\right)=\frac{1}{N-m}\sum \limits_{i=1}^{N-m}\left(\frac{1}{N-m-1}\sum \limits_{j=1,j\ne i}^{N-m}{D}_{ij}^{m+1}\right) $$
(12)

So, the FuzzyEn(m, n, r) of time series is:

$$ FuzzyEn\left(m,n,r,N\right)=\ln {\phi}^m\left(n,r\right)-\ln {\phi}^{n+1}\left(n,r\right) $$
(13)

3 Construction and training of adaptive fuzzy neural networks

3.1 Introduction of fuzzy neural network

Fuzzy logic is used to study fuzzy thinking, language forms, and their laws, which is close to human thinking and decision-making methods but needs the relatively rich prior knowledge [60,61,62,63]. As problems become more complex, it is difficult to establish effective and complete fuzzy inference rules. However, there is no limitation with the neural network which has the ability of adaptive learning to extract corresponding information from existing sample data. Thus, neural networks are generally suitable for processing unstructured data, while fuzzy systems are more effective for unstructured knowledge representation. Therefore, fuzzy logic and neural networks are properly combined to form a system with better performance. Figure 7 shows a basic fuzzy inference system block diagram, which mainly includes: fuzzy input, fuzzy inference mechanism, rule base, database, and defuzzification. The most important part is the generation of rule base.

Fig. 7
figure 7

Fuzzy logic system schematic

Full size image

Since the fuzzy inference system relies heavily on the experience and knowledge of experts or operators, the experience is difficult to connect more and more complex problems. The greatest advantage of neural networks lies in their ability to self-learn. This adaptive neural network technology is applied to the analysis and modeling of model features and is called adaptive neural network technology. A fuzzy system based on adaptive neural network technology can learn fuzzy membership functions and fuzzy rules from a large number of known data, avoiding the experience or conscious dependence on humans. Since the model of surface EMG signal and force is a very complex system, adaptive fuzzy neural network has important research significance for analyzing and solving this problem. A typical structural diagram of a fuzzy neural network is shown in Fig. 8. It greatly simplifies the mechanism of the traditional fuzzy inference system through the distributed neural network, and realizes the functions of self-tuning and self-learning of the fuzzy neural network. Since the fuzzy inference system relies heavily on the experience and knowledge of experts and operators, it is difficult to achieve the desired control effect without such experience. The greatest advantage of neural networks lies in their ability to self-learn. This adaptive neural network technology is applied to the analysis and modeling of model features and is called adaptive neural network technology. A fuzzy system based on adaptive neural network technology can learn fuzzy membership functions and fuzzy rules from a large number of known data, avoiding the experience or conscious dependence on humans. Since the model of surface EMG signal and force is a very complex system, adaptive fuzzy neural network has important research significance for analyzing and solving this problem. A typical structural diagram of a fuzzy neural network is presented in Fig. 8. It greatly simplifies the mechanism of the traditional fuzzy inference system through the distributed neural network, and realizes the functions of self-tuning and self-learning of the fuzzy neural network.

Fig. 8
figure 8

Schematic diagram of fuzzy neural network structure

Full size image

3.2 Adaptive fuzzy neural network modeling based on Takagi-Sugeno model (Figs. 9, 10, 11, 12, 13, and 14)

Neural network modeling refers to the use of neuro-fuzzy systems to approximate unknown nonlinear dynamics, thus approaching the entire system. Neural networks have shown good performance in the modeling of unknown nonlinearities. An adaptive fuzzy neural network modeling method based on the Takagi-Sugeno model is provided in the fuzzy logic toolbox of Matlab, in which the BP backpropagation algorithm and the least squares algorithm are used to model the input/output data pairs. In the modeling process, the method can extract the corresponding information (fuzzy rules) learning method from the data set, which is very similar to the neural network method. By self-learning to obtain the best membership function parameters, the designed Takagi-Sugeno-type fuzzy inference system can better simulate the desired input/output relationship. In order to ensure that the subject can output a stable force as much as possible, the subject experimenter has a little time to pre-train. The data used in this experiment had a hand output force of 10–100 N, and the interval between each value was 1 N and maintained for 10 s. The obtained data were randomly divided into training data and test numerical data according to 2:8, and 11 characteristic values of the surface electromyogram signal were extracted as a feature matrix, and the force was taken as an output. The final evaluation index is the maximum root mean square error.

Fig. 9
figure 9

Output of raw data

Full size image
Fig. 10
figure 10

Fuzzy neural network structure

Full size image
Fig. 11
figure 11

The fuzzy rule before training

Full size image
Fig. 12
figure 12

The fuzzy rule after training

Full size image
Fig. 13
figure 13

Training error

Full size image
Fig. 14
figure 14

Checking error

Full size image

4 Conclusion

In order to further promote the application of surface EMG signals in human–computer interaction, this paper proposes an adaptive fuzzy neural network-based hand grip assessment method. Based on the characteristics of traditional surface EMG signals, the fuzzy entropy value is introduced to reduce the influence of muscle fatigue on the performance and accuracy of the model, so that the recognition based on surface EMG signals can be better applied in real life. Because fuzzy reasoning mechanism and neural network have their own advantages and disadvantages, this paper combines them to construct an adaptive fuzzy neural network and evaluate the grip of the opponent. The contribution of this paper lies in the effective realization of the control force output of human hand, which can overcome the problem of unstable control of human hand force in the process of grasping. According to the experimental results, the hand grip strength evaluation based on the adaptive fuzzy neural network has better performance, and the error is about 10%, which satisfies the basic control requirements. Fuzzy theory is widely used in the control of robots, and it is very necessary to construct an adaptive fuzzy neural network system because it is difficult for human to achieve accurate control of grip force. The method benefits that the health status of patients during rehabilitation training can also be effectively monitored using sEMG. Because of the lack of experimental equipment, the model cannot be applied to the actual myoelectric false hand to further verify the reliability of the proposed method, and more experiments and researches are needed.