Keywords

1 Introduction

In recent years, as one of the key technologies for human behavior recognition, gait behavior recognition technology has attracted widespread attention in the academic community. Its goal is to distinguish individuals through individual walking postures. Gait is the inherent bio-characteristics of the human body [1]. It mainly refers to the behavioral characteristics of the human body when walking in a normal walking state. It is not only ubiquitous in a body weight, but also can not easily be stolen or imitated [2]. Studies in Medicine [3] and Psychology [4] have shown that the gait of the human body involves coordinated movements between various parts of the human body and joints [5]. Differences in gait characteristics generally include height, weight, step, and step speed factors Moreover, it has the advantages of far-reaching perception, difficulty in disguise, non-invasiveness, and small impact on the environment, and it has increasingly attracted the attention of researchers. Based on the above advantages, gait behavior recognition has a wide range of application prospects in fields such as mobile health, telemedicine, security monitoring, smart home, and smart city.

The gait behavior recognition technology utilizes the attitude of people walking to recognize the behavior of people, and through the classification and identification of gait behavior, it can further analyze the behavior of the moving human body. Since Niyogi and Adelson [6] used machine vision and pattern recognition theory to conduct research on gait behavior recognition technology in 1994, more and more researchers have devoted a lot of time and energy to gait behavior recognition technology. A lot of research work has started. Mantyjarvi et al. proposed the use of an acceleration sensor to perceive gait behavior data [7], which opened up a new way to obtain gait data. Sabatini et al. proposed a wearable sensor network centered on a smartphone to assess the risk of falls for the elderly. By collecting gait data from different sensor sources, a step-based gait stability index was evaluated in real time. Evaluate the fall risk, extract the accelerometer data using time and frequency domain features, and use the radial basis function of the support vector machine as the classifier algorithm. The support vector machine shows advantages in solving small sample, nonlinear and high dimensional pattern recognition. The method used in [8] is to first determine an acceleration threshold. If the acceleration of an axis exceeds this threshold, a dynamic time warping (DTW) algorithm is used to perform the discrimination. Fan Lin and others used smartphones to collect acceleration data of people’s motions such as motionlessness, walking, running, and going downstairs. In order to eliminate the influence of mobile phone carrying position on behavioral recognition efficiency, feature sets highly relevant to behaviors were extracted from different positional data. Build position-independent behavior recognition [9]. Second, there are still some studies that classify out-of-sync states through acceleration signals, such as upstairs, downstairs, walking, running, and other daily activities. In [10], five 5-axis accelerometers were fixed on the soles, thighs, wrists, arms, and buttocks of the human body, respectively, to obtain human behavior data. Although the behavioral data it obtains is more comprehensive, the system consumes a lot of energy, has a large amount of calculation, and the wearer’s comfort is poor, which seriously hampers human daily life. In addition, there are gait behavior recognition methods based on angles, pressures, etc. Professor Zhang Daqing proposed a gait calculation framework for distinguishing between Parkinson’s patients and normal people in 2016, gait phase identification, feature extraction and selection, and mode. The three parts of the classification constitute the framework, first proposed a sliding window to distinguish the four gait stages from the plantar data, and the second step to extract and select the gait features describe the gait stability, symmetry and coordination. Finally, a mixed classification model was used to identify the Parkinson’s gait [11].

Through a brief overview of the above literature, we can understand that the key technologies involved in gait behavior recognition systems include: data processing, feature value extraction, data classification, and so on. The gait behavior recognition mainly analyzes and deals with the time series containing human gait movements. Generally, the gait behavior recognition process includes data preprocessing, gait feature extraction, and gait classification and identification. The following section outlines the methods and algorithms for gait behavior recognition.

2 Gait Behavior Recognition Process

2.1 Data Preprocessing

The following sources of noise may be present when collecting human gait information:

  1. (1)

    In walking, the sensor will produce vibration and noise as the body shakes;

  2. (2)

    Associated noise occurs during the conversion of a signal from an analog signal to a digital signal;

  3. (3)

    Electromagnetic interference exists in the process of data transmission from the acquisition device to the host computer;

  4. (4)

    Power frequency interference caused by working power in the acquisition system.

The noise generated in the process of collecting data may be very close to the real signal. Therefore, the collected gait data must be denoised and preprocessed. The denoising preprocessing can be performed in the acquisition circuit, for example, by using hardware filtering methods such as inductance, capacitance, and LC composite filter circuit, and also by using MATLAB, FIR filter, wavelet filter, etc. in the upper computer [12,13,14,15] Software filtering method.

2.2 Gait Behavior Feature Extraction

The main task of feature extraction is to select the most effective features for classification and identification, which is the key to gait behavior identification. It can be divided into spatio-temporal parameters feature extraction methods [16] and [17], angle parameters feature extraction methods [18] and lower limb movement mechanics. Parameter feature extraction method.

2.2.1 Temporal and Spatial Parameter Characteristics

Spatio-temporal parameters refer to features such as gait cycle, support period, pace, pace, and step length.

Gait cycle: refers to the period of time between the landing of one leg from the foot to the next leg. According to the movement of the gait, the gait cycle can be divided into the support period and the swing period.

The stance phase (stance phase) refers to the period from the heel touchdown to the toe off the ground. The heel strikes the ground as the starting point of the gait cycle and lands on the heel and toe to support the mid-term. After that, the heel began to leave the ground, the body gradually moved forward, finally to the tiptoe off the ground, the end of the support period.

Swing phase: It refers to the period from the toe to the next heel strike after the support period. When the support period is over, that is, when the tiptoe is off the ground, the lower limbs stop by accelerating forward to reduce the swing speed, and the next support begins.

The support period lasts about 60% of the gait cycle. It can be divided into three stages: early support, middle support, and late support. The early support refers to the heel strike from the ground to the toe, and the middle support refers to the full support of the foot. The late period refers to the process from the heel off the ground to the toes off the ground.

Walking speed: The ratio of the distance traveled in the walking cycle to the duration of the gait cycle, ie .pace = step/gait cycle.

Stride length: It refers to the linear distance between two adjacent landings on the same side of the heel.

Step length: A linear distance between the measured heel and the other heel.

2.2.2 Angle Parameter Characteristics

The angle parameter features mainly include the angle changes of the three joints of the hip, knee, and ankle, such as the angle, angular velocity, and angular acceleration of each joint. The definition of lower limb joint angle [18] is as follows:

Hip angle parameter: The angle between the longitudinal trunk axis and the femur longitudinal axis.

Knee angle parameter: The angle between the extension of the femur longitudinal axis and the parallel line of the longitudinal axis of the tibia.

Ankle angle parameter: The angle between the fifth metatarsal and the midline of the lateral tibia is reduced by 90°.

2.2.3 Characteristics of Mechanical Parameters of Lower Limbs

The parameters of lower limb movement mechanics mainly include plantar pressure and joint torque.

Through the above description of the characteristics of gait parameters, it is clear that space-time parameters are relatively intuitive, and angle parameters and kinematics parameters are more detailed in describing gait details.

2.3 Recognition of Gait Behavior

The gait classification recognition process is to match the gait recognition after feature extraction with the known sequences trained in the sample database, and determine the category of the gait recognition classification criteria and the discrimination conditions. The gait classification recognition algorithm has a very close relationship with the construction of the entire gait behavior recognition system. Selecting an appropriate classification recognition algorithm and matching strategy is a key issue. Common gait behavior classifier algorithms are: k-Nearest Neighbor (KNN), Bayesian classifiers, Support Vector Machines (SVM), neural network classifiers, and hidden Mars Hidden Markov Model (HMM), etc. The common gait behavior classifier principle and characteristics are shown in Table 1.

Table 1. Principles and characteristics of common gait behavior classifiers.
Full size table

For the multi-perspective classification and identification research, when the identification is a big lock, a decision-based information fusion classification method is used. This method firstly classifies individual different features and obtains their own independent decision results. Association and comprehensive matching according to the fusion strategy, and finally get the classification recognition result. Decision-based fusion methods include Bayesian decision theory, artificial neural networks, genetic algorithms, and rough theory. In recent years, with the gradual deepening of research on gait behavior classification, a variety of algorithms combined with research methods and some improved algorithms based on classical algorithms have emerged gradually to improve the recognition of gait classification.

3 HMM

The Hidden Markov Model (HMM) is a statistical model used to describe a Markov process with implicit unknown parameters. Founded in the 1970 s, it was proposed by Baum et al. [41] and conducted rigorous mathematical arguments by Rabiner et al. [42]. It was first used to solve continuous speech recognition problems [43]. Bell Lab’s Rabiner et al. [44] made the HMM become a research hotspot in the field of voice processing that researchers are interested in through the promotion of HMM theory and the successful use of vocabulary recognition reports.

The Hidden Markov Model is based on the Markov model. In the Markov model, observations and states are in a one-to-one correspondence. However, practical problems are often more complex than the problems described. The observed events do not correspond to the states one by one. They are related by a set of observed probabilities. Such a model is called HMM [42]. It consists of two stochastic processes, one of which is the Markov model, which is a basic random process that describes the transitions between states; the other is the correspondence between states and observations. In other words, observers can only see observations and cannot see the state directly. Instead, they perceive the existence of states through a random process. Therefore, they are called “hidden” Markov models.

The following is a simple description of the HMM parameter description model:

  1. (1)

    Q: The number of states in the model of state sequences is \( \text{Q} = \left\{ {q_{1} ,q_{2} , \ldots ,q_{N} } \right\} \)

  2. (2)

    V: The set of observations corresponding to each state, denoted as \( \text{V} = \left\{ {v_{1} ,v_{2} , \ldots ,v_{M} } \right\} \)

  3. (3)

    π: The initial state probability distribution vector, denoted as \( \uppi = \left\{ {\uppi_{1} ,\uppi_{2} , \ldots ,\uppi_{\text{N}} } \right\} \),among them

$$ \uppi = P(i_{1} = \theta_{i} ),0 \le\uppi_{\text{i}} \le 1,\sum\limits_{i} {\uppi_{\text{i}} = 1,1 \le i \le N} . $$
(1)

It is the probability \( q_{i} \) of being in the state at time t = 1.

  1. (4)

    A: State transition probability matrix, \( \text{A} = \left[ {a_{ij} } \right]_{N \times N} \), among them

$$ a_{i,j} = P\left( {i_{i + 1} = q_{j} |i_{i} = q_{i} } \right),i = 1,2, \ldots ,N;j = 1,2, \ldots ,N. $$
(2)

It is the probability \( q_{i} \) of transitioning to the state at the next time t+1 under the condition of the state at time t.

  1. (5)

    B: Observation probability matrix, \( \text{B} = \left[ {b_{j} (k)} \right]_{N \times M} \),among them \( b_{j} (k) \) is

$$ P\left( {o_{t} = v_{k} |i_{t} = q_{j} } \right),\;k = 1,2, \ldots ,M;j = 1,2, \ldots ,N . $$
(3)

It is the probability \( q_{j} \) of generating observations \( v_{k} \) under the condition of state at time t.

The Hidden Markov Model is determined by the initial state probability distribution vector π, the state transition probability matrix A, and the observation probability matrix B. A and π determine the state sequence and B determines the sequence of observations. So Hidden Markov Models can be represented by ternary symbols, i.e.

$$ \lambda = \left( {A,B,\pi } \right) . $$
(4)

A, B, π are called the three elements of the hidden Markov model. Among them, the state transition probability matrix A and the initial state probability distribution vector π determine the hidden Markov chain, thereby generating an unobserved state sequence. The observation probability matrix B determines how to generate the observation sequence from the state, and the state sequence determines The resulting observation sequence.

The Hidden Markov Model mainly solves three basic problems. For this purpose, three basic algorithms are proposed. The questions are as follows:

Question I: Probability Calculation of Hidden Markov Models

Given the sequence of observations \( \text{Q} = \left\{ {q_{1} ,q_{2} , \ldots ,q_{N} } \right\} \) and the model \( \lambda = \left( {A,B,\pi } \right) \), calculate the probability \( P\left( {O|\lambda } \right) \) of occurrence of the sequence Q of observations under a given model.

Baum corresponds to the problem of a forward-backward algorithm. According to the composition of the HMM, the solution method is as follows [30]:

For a given state sequence \( S = \left\{ {q_{1} ,q_{2} , \ldots ,q_{N} } \right\} \), There have

$$ P\left( {O|S,\lambda } \right) = \prod\limits_{t = 1}^{t} {} P\left( {o_{t} |q_{t} ,\lambda } \right) = b_{q1} \left( {o_{1} } \right)b_{q2} \left( {o_{2} } \right) \ldots b_{qt} \left( {o_{t} } \right) . $$
(5)

Among them,

$$ b_{qt} \left( {o_{t} } \right) = b_{jk} |q_{t} = \theta_{j} ,o_{t} = v_{k} ,1 \le t \le N . $$
(6)

For a given \( \lambda \), the probability of producing S is

$$ p\left( {S|\lambda } \right) = \pi_{q1} a_{q1q2} \ldots a_{qT - 1qT} . $$
(7)

So the probability is

$$ p\left( {O|\lambda } \right) = \sum\limits_{s} {P\left( {O|S,\lambda } \right)p\left( {S|\lambda } \right)} . $$
(8)

Question 2: Prediction (Decoding) of Hidden Markov Models

Given the sequence of observations \( \text{Q} = \left\{ {q_{1} ,q_{2} , \ldots ,q_{N} } \right\} \) and the model \( \lambda = \left( {A,B,\pi } \right) \), the probability of the observed sequence \( P\left( {O|\lambda } \right) \) is the largest under the given model, and the corresponding state sequence is obtained. That is, given the sequence of observations, find the corresponding most likely sequence of states.

The second problem is to seek the “optimal” state sequence [42]. Refers to the state sequence determined when the maximum value is taken. The Viterbi algorithm actually uses dynamic programming to solve the prediction problem of Hidden Markov Models, that is, using dynamic programming to solve the optimal path. This process can be implemented using the Viterbi algorithm:

Input: model \( \lambda = \left( {A,B,\pi } \right) \) And observation sequences \( O = \left( {o_{1} ,o_{2} , \ldots ,o_{T} } \right) \);

Output: Optimal path \( I = \left( {i_{1} ,i_{2} , \ldots ,i_{T} } \right) \).

Initialization

$$ \delta_{1} \left( i \right) = \pi_{i} b_{i} \left( {o_{I} } \right),i = 1,2, \ldots ,N . $$
(9)
$$ \psi_{1} \left( i \right) = 0,i = 1,2, \ldots ,N . $$
(10)

Recursive, for \( t = 2,3, \ldots ,T \)

$$ \delta_{t} \left( i \right) = \mathop {\hbox{max} }\limits_{1 \le j \le N} \left[ {\delta_{t - 1} \left( j \right)a_{ji} } \right]b_{i} \left( {o_{t} } \right),i = 1,2, \ldots ,N . $$
(11)
$$ \psi_{1} \left( i \right) = \arg \mathop {\hbox{max} }\limits_{1 \le j \le N} \left[ {\delta_{t - 1} \left( j \right)a_{ji} } \right],i = 1,2, \ldots ,N . $$
(12)

Termination

$$ P = \mathop {\hbox{max} }\limits_{1 \le j \le N} \delta_{T} \left( i \right) . $$
(13)
$$ i_{T} = \arg \mathop {\hbox{max} }\limits_{1 \le j \le N} \left[ {\delta_{T} \left( i \right)} \right] . $$
(14)

Best path backtracking. For \( t = T - 1,T - 2, \ldots 1 \)

$$ i_{t} = \psi_{i + 1} \left( {i_{t + 1} } \right) . $$
(15)

Finding the optimal path \( I = \left( {i_{1} ,i_{2} , \ldots ,i_{T} } \right) \)

Question 3: Learning Problems of Hidden Markov Models

Given the sequence of observations \( \text{Q} = \left\{ {q_{1} ,q_{2} , \ldots ,q_{N} } \right\} \), the parameters of the model \( \lambda = \left( {A,B,\pi } \right) \), so that the probability of the sequence of observations under the given model is maximum, ie, the parameters of the model \( P\left( {O|\lambda } \right) \) are estimated using the method of maximum likelihood estimation. The Baum-Welch algorithm is a hidden Markov model for learning.

Input: Observation sequence \( O = \left( {o_{1} ,o_{2} , \ldots ,o_{T} } \right) \);

Output: Hidden Markov Model Parameters

Initialization

for n = 0,select \( a_{ij}^{(0)} ,b_{j} \left( k \right)^{(0)} ,\pi_{i}^{(0)} \), Get the model \( \lambda^{(0)} = \left( {A^{(0)} ,B^{(0)} ,\pi^{(0)} } \right) \)

Recursive. For n = 1, 2, … ,

$$ a_{ij} = \sum\limits_{t = 1}^{T - 1} {\xi_{t} \left( {i,j} \right)} /\sum\limits_{t = 1}^{T - 1} {\xi_{t} \left( i \right)} . $$
(16)
$$ b_{j} \left( k \right) = \sum\limits_{{t = 1,o_{t} = v_{k} }}^{T} {\gamma_{t} \left( j \right)} /\sum\limits_{t = 1}^{T} {\gamma_{t} \left( j \right)} . $$
(17)
$$ \pi_{i}^{(n + 1)} = \gamma_{1} \left( i \right) . $$
(18)

Termination. Get model parameters \( \lambda^{(n + 1)} = \left( {A^{(n + 1)} ,B^{(n + 1)} ,\pi^{(n + 1)} } \right). \)

As a statistical model of dynamic time series, Hidden Markov Model is the most effective recognition method for human gait behavior recognition [40]. The gait behavior movement can be regarded as a series of posture changes of the human body, determined by the structure of the human body itself, and has the characteristics of probability and statistics. The structure behind the gait behavior can become the hidden state of the hidden Markov model, and the walking posture Seen as the output of the structure, the observed vector value. Theoretical and practical experiments have shown that Hidden Markov Model can not only represent gait behavior well, but also has a good classification effect.

4 Conclusion

This article mainly describes the progress of gait behavior recognition research. The gait behavior recognition is mainly divided into three parts: gait data preprocessing, gait behavior feature extraction and gait behavior classification and identification. The classification and recognition is extremely important and the focus of this study. Firstly, the causes of the gait behavior data noise and the research status of the data pre-processing measures are analyzed. Secondly, the characteristic parameters of the gait behavior are evaluated from three angles: space-time, angular velocity, and motion mechanics. Finally, the gait behavior is classified. The recognition algorithm is characterized and compared, and the hidden Markov model with good classification effect is analyzed. This article describes the gait behavior recognition process intuitively, and provides valuable reference for related researchers; it facilitates researchers to better understand the characteristics of gait behavior recognition and broadens the research ideas; and identifies the gait behavior recognition system. The application has a certain theoretical guidance and practical value.