1 Introduction

Biometrics is increasingly becoming an important technology for human identification in various significant applications such as surveillance and access control security. It refers to the automatic identification of a person based on his/her unique physiological or behavioral characteristics (called also biometric modalities) [1]. Many biometric-based identification systems have been proposed using a wide variety of static modalities such as fingerprint, palmprint, face, iris and ear [2]. However, these systems, and despite their widespread applications, suffer for the two following disadvantages [3]: (i) failure to match in low-resolution images taken at a distance and (ii) necessitates user cooperation for accurate identification results. Other biometric systems have been proposed to overcome these limitations using dynamic modalities such as gait, which has recently received a considerable interest from the biometric recognition community and becomes an important biometric modality for human identification [4].

Indeed, gait describes the walking pattern of a person, which is unique if compared with other behavioral modalities such as speech. One of its main characteristics is that data can be taken at a distance if compared with other static modalities such as fingerprint or face, speaker and hand. Also, low-resolution data can be used successfully. In addition, human gait is unobtrusive as it can be deployed without any cooperation from the users. However, the effect of clothes and carrying conditions can affect the performances of human gait recognition. Recently, research in gait recognition has focused on developing a gait representation that is capable of capturing the relevant information under variable conditions including speed, clothing, carrying and view among others. Human gait recognition techniques can be split into two types of approaches: model-based approaches [5] and model-free approaches [4]. A considerable amount of literature published on gait recognition is based on model-free approaches where the gait features are obtained from the moving shape of the individual. Furthermore, these gait features are derived from spatio-temporal patterns of an individual gait [7], 2D optical flow field [6, 10], skeleton variance image [8], binary silhouettes [9, 18], variations of area within a particular region or Gait energy image (GEI) [4, 11] which is a spatio-temporal gait representation. GEI is the human silhouette image obtained by determining the skeleton from the body segments [12]. Several studies in the field of gait recognition have focused on the model-free approaches to make them robust against covariate factors such as clothing and carrying conditions. Among the approaches that have been proposed recently, one can cite, for example, gait representation using flow fields [13], gait entropy image [14] and gait recognition based on local binary pattern (LBP) descriptors [16]. On the other hand, model-based approaches aim to extract the characteristics of the human body’s movement using the torso and the legs. Methods of these approaches use static body parameters for identification, such as stride lengths [5]. Although these approaches focus on person recognition using walking only, Yam et al. [17] extended the concept to enable the recognition not only by the walking gait but also by the running gait by analysis of leg motion and the angles between the limbs.

Otherwise, the performance of the proposed gait recognition methods depends mainly on the quality of the extracted gait features. The inclusion of shape information in gait features can introduce variations that will hinder the recognition performance especially in the cases where the same person wears different clothes and has different carrying conditions. In this paper, we propose a novel gait recognition approach for human identification under variations of clothing and carrying conditions and also under different viewing angles, in order to overcome the above limitations.

The proposed approach is based on a supervised feature extraction method capable of selecting discriminating features from GEI under varying clothing and carrying conditions and hence to improve the recognition performance. The proposed method is based on Haralick features extracted locally from equal regions in the GEI and the RELIEF selection algorithm employed on the extracted features in order to select only the most relevant features with a minimum redundancy. The evaluation of the performances including a comparative analysis against a few recent and similar techniques is carried out using CASIA gait database.

The rest of the paper is organized as follows: Sect. 2 gives a review of related works, while Sect. 3 introduces the principle of gait recognition approach. The proposed approach is described in detail in Sect. 4, and the experimental results and their analysis using CASIA database are reported in Sect. 5. Finally, conclusion and future work are discussed in Sect. 6.

2 Related work

Various gait recognition approaches have been proposed during the last decade that can be split into two broad categories : (i) model-based approaches that fit a model to human body and represent gait using the parameters of the model which are updated over time [5, 14], and (ii) model-free approaches that use motion information directly extracted from silhouettes [4, 14]. Recent gait recognition researches seem to favor model-free approaches due mainly to their better performance compared to the model-based approaches, as well as to their robustness to noise and less computational cost. Bashir et al. [14] proposed a gait feature selection method referred to as GEI representing a significance measure of features (e.g., pixels with high entropy, which correspond to dynamic parts and are robust against appearance changes). The principle consists of computing Shannon entropy for each pixel over a gait cycle. In other terms, the technique aims to discriminate static and dynamic pixels of the GEI. In [16], the authors proposed a method of gait recognition system using GEI and LBP techniques to extract features from the gait representation. LBP operator is applied to extract the features from entire GEIs and the region bounded by legs (RBL). The process was applied in instances (covariate factors) of a gait such as a change in clothing, carrying a bag and different normal walking conditions. Boulgouris et al. [18] proposed an approach based on Radon transform of the binary silhouettes. In this approach the binary silhouettes used to calculate the gait cycle are subjected to Radon transform to generate a Radon template from which a set of features is extracted using linear discriminant analysis (LDA). These features are represented as a single feature vector to represent the entire gait sequence. Finally, recognition is achieved by comparing the feature vectors of reference sequences with the feature vectors of a given test/probe sequence. In [19], authors proposed an interesting feature selection method based on random forest rank features algorithm for gait recognition. In this paper, we propose a novel GEI-based gait recognition approach under different clothing and carrying conditions for different viewing angles, thus improving the human identification performance.

3 GEI-based gait recognition

Although a number of gait recognition approaches have been proposed in the literature, the algorithms share a common goal of ensuring the best trade-off between the recognition performance and the computational complexity. GEI is one of the most widely used methods from which to extract the relevant feature descriptors of human gait and has proven to be one of the most effective techniques.

3.1 Gait energy image

GEI is a representation of human walking using a single grayscale image obtained by averaging the silhouettes extracted over a gait cycle [4]. GEI can be seen as the sum of images of the walking silhouette divided by the number of images and is defined as follows:

$$\begin{aligned} G(x,y) = \frac{1}{N} \sum _{t=1}^{N} S(x,y,t) \end{aligned}$$
(1)

where N and t are the number of frames within complete gait cycle and the frame number in the gait cycle, respectively. S is the silhouette image whose pixel coordinates are given by x and y. Figure 1 shows an example of GEI of an individual under different conditions. Pixels with low intensity correspond to the dynamic parts of the body which are very useful for recognition and are not affected by the carrying and clothing conditions, referred to as “covariate factors.” On the other hand, pixels with high intensities correspond to the static parts of the body containing the body shape information useful for identification, but they can be affected by the covariate conditions (e.g., carrying-bag, wearing-coat) [14]. GEI is used for selecting informative gait features in our proposed approach.

Fig. 1
figure 1

An example of GEI of an individual for side view \(90^{\circ }\) under three different conditions/covariates. a Normal walking. b Carrying-bag. c Wearing-coat

Full size image

3.2 Gait recognition approach

Human gait recognition refers to verifying and/or identifying persons using their walking style under covariate factors (i.e., carrying and clothing conditions). GEI-based gait recognition is one of the most recent effective biometric systems, having high recognition rates with low computational complexity. Such a system includes the following three steps:

  1. 1.

    Feature extraction: extracts the discriminating features from the gait representation and is able to characterize the gait under variations of covariate factors such as clothing and carrying conditions.

  2. 2.

    Feature selection: selects a subset of relevant features from the GEI representation.

  3. 3.

    Classification: makes a decision about the recognition of the gait in question using a classification process with the selected feature vector.

Figure 2 illustrates the human gait recognition system diagram.

Fig. 2
figure 2

Human gait recognition system diagram

Full size image

4 Proposed method

We have visually analyzed the information contained in GEI and defined feature extraction methods for gait recognition under variations of clothing and carrying conditions. The main idea is to exploit locally the discriminating features that characterize these conditions by dividing horizontally and/or vertically the GEI in three (top, medium and bottom) and/or two equal (left and right) parts where each part (also called region of interest, ROI) represents the discriminative information for clothing and carrying conditions under different viewing angles considered in our study. For example, for the case of carrying conditions, the bag appears most often in the medium part of the horizontal division or the right part of the vertical division. Also, for the case of clothing conditions, the clothes appear most often in the top part of the horizontal division or the right part of the vertical division. An illustrative example is shown in Figs. 3, 4.

Fig. 3
figure 3

An example of ROIs extracted from a horizontal division of GEI of an individual for side view \(90^{\circ }\) under three different covariates: normal walking (first column), carrying-bag (second column) and wearing-coat (third column). a Top part, b medium part, c bottom part

Full size image
Fig. 4
figure 4

An example of ROIs extracted from a vertical division of GEI of an individual for side view \(90^{\circ }\) under three different covariates: normal walking (first column), carrying-bag (second column) and wearing-coat (third column). a Left part. b Right part

Full size image

In this work, we propose two supervised feature extraction methods for human gait recognition based on texture descriptors extracted from GEI. The proposed methods described below are capable of extracting the most discriminative features from GEI under different covariates or conditions, hence improving the recognition performances.

4.1 Haralick texture descriptors

The discriminative features proposed in our feature extraction method include the Haralick texture descriptors [21] extracted and computed from GEI. These features are computed from the gray-level co-occurrence matrix \(\texttt {GLCM}\) of GEI, denoted P, with dimension \(N_g \times N_g\) where \(N_g\) is the number of gray levels in the GEI. The co-occurrence matrix \(P_{d,\theta }\) can be defined as:

$$\begin{aligned} P_{d,\theta } (i,j) = \sum _{x=1}^{N_g} \sum _{y=1}^{N_g} \left\{ \begin{array}{rl} 1, &{} \text{ if } I(x,y)= i \\ &{} \text{ and } I(x+dx,y+dy)=j \\ 0, &{} \text{ otherwise } \end{array} \right. \end{aligned}$$
(2)

where d and \(\theta \) are, respectively, the offsets and the direction (phase). dx and dy specify the distance between the pixel of interest and its neighbor, along the x-axis and the y-axis of an image, respectively.

Haralick texture features are statistical entities defined to emphasize certain texture properties and calculated from P. Table 1 describes the proposed Haralick features allowing description of the texture in the GEI in order to recognize the observed human gait. These features comprise 14 statistics calculated from \(\texttt {GLCM}\). In this study, \(F_{14}\) was not calculated due to computational instability and only the features \(\{F_1,F_2,\ldots ,F_{13}\}\) are considered. The proposed Haralick texture descriptors are extracted and computed locally in four directions (i.e., \(\theta \) in \(\{0^{\circ },45^{\circ },90^{\circ },135^{\circ }\}\)) from different ROIs generated by dividing the GEI vertically into two equal ROIs (left and right parts) and/or horizontally into three equal ROIs (top, medium and bottom parts), where each part (or ROI) represents the relevant information for different conditions. RELIEF feature selection algorithm [20] is then employed on Haralick features computed in four directions in order to select only the most relevant features with a minimum redundancy. Algorithm 1 summarizes the proposed method.

Table 1 Haralick texture features extracted and computed from GEIs
Full size table
figure a

4.2 RELIEF selection algorithm

RELIEF, proposed by Kira and Rendell in [20], is used in the data-processing stage as a feature selection method. RELIEF-based algorithms are mainly divided into three principal parts:

  1. 1.

    Compute the nearest miss \(\mathcal {M}\) and nearest hit \(\mathcal {H}\).

  2. 2.

    Compute the weight of a feature.

  3. 3.

    Return a ranked list of features or the top k-features according to a given threshold.

The algorithm begins with initializing the weight vector and tuning the weight for every feature to 0. Then it randomly picks a learning sample X and computes the \(\mathcal {H}\) and \(\mathcal {M}\) from the same subfamily \(\mathcal {H}\) and one from the opposite subfamily \(\mathcal {M}\). The weight \(\mathcal {W}\) can be computed using Eq. 3:

$$\begin{aligned} \mathcal {W}[i] =\mathcal {W}[i] + \frac{ diff (X^{(i)}, {\mathcal {M}}_X^{(i)} )}{(\mathcal {S} \times K)} - \frac{ diff (X^{(i)}, {\mathcal {H}}_X^{(i)} )}{(\mathcal {S} \times K)} \end{aligned}$$
(3)

where \(\mathcal {S}\) is the number of learning instances X described by N features, and K is a number of iterations. The function \( diff \) is the difference of feature values between two instances (e.g., a and b) defined as follows:

$$\begin{aligned} diff (a,b) = \frac{a-b}{u} \end{aligned}$$
(4)

where u is a normalization unit to normalize the values of \( diff \) into the interval [0, 1]. Algorithm 2 summarizes the pseudo-code of the RELIEF algorithm [20] used in Algorithm 1. Figure 5 illustrates the diagram of the proposed feature extraction and selection method based on GEI Haralick texture features with RELIEF selection algorithm.

figure b
Fig. 5
figure 5

Block diagram of the proposed supervised feature extraction and selection approach based on GEI Haralick texture features with RELIEF selection algorithm

Full size image

5 Experimental results and discussion

5.1 Database and evaluation criteria

We have evaluated the proposed methods using CASIA gait database (dataset B) [15] which is a multi-view gait database [22]. This database was constructed from 124 subjects (93 men and 31 women) and 11 cameras around the left-hand side of the subject when they were walking. Thus, the data were captured from 11 different angles starting from \(0^{\circ }\) to \(180^{\circ }\) (i.e., the angle between two nearest view directions is \(18^{\circ }\) in the range of [0o, 180o]). Each subject has six normal walking sequences (SetA), two carrying-bag sequences (SetB) and two wearing-coat sequences (SetC). For our experiments, we have selected from this database the three first sequences from SetA and the first sequence from SetB and SetC to test the performance of the proposed method under the following three conditions: normal, carrying-bag and wearing-coat. The remaining sequences for all the 124 subject were assigned to the training set. Experiments are carried out under the following viewing angles: \(36^{\circ }\), \(72^{\circ }\), \(90^{\circ }\), and \(108^{\circ }\).

For evaluation criteria, k-NN classifier was used to quantitatively evaluate the classification performance. The highest identification rate (IR) at rank-1 that is defined as the percent of samples with a correct match in the first place of the ranked list is used to evaluate the classification performance.

5.2 Results and analysis

We have assessed the performance of our proposed method using the selected data from CASIA database for side view \(90^{\circ }\). Three covariates were considered: normal walking, carrying-bag and wearing-coat. Table 2 shows the comparison results in terms of IR at rank-1 (in %) of our proposed method against four other existing methods (i.e., methods proposed in [13, 14, 23] and [19]). By analyzing the results obtained, we make the following observations:

  • The use of horizontal GEI division allows to increase the recognition performance in terms of IR at rank-1 up to 80% than the vertical GEI division.

  • The proposed method yields comparable results for the cases: “normal walking” and “carrying-bag” conditions; also provides best IR at rank-1 for the case of “wearing-coat” condition compared to the rest of methods (e.g., an increase of up to 26, 31, 32 and 13% compared to the methods in [13, 14, 19, 23], respectively).

  • The proposed method outperforms all the state-of-the-art methods considered in our experiment. Compared to the best CCRs provided by the state-of-the-art methods which are in the range of 60.70–77.96%, our method achieves a better IR at rank-1 up to 80% for the side view \(90^{\circ }\).

Finally, experiment demonstrates that the proposed method improves significantly the recognition performance in the presence of the following covariates: normal walking, carrying-bag and wearing-coat and outperforms the state-of-the-art methods (e.g., an increase of up to 2% compared to the method in [19]).

Table 2 Comparing the proposed method with different state-of-the-art methods on CASIA database (dataset B) for side view \(90^{\circ }\)
Full size table

We have also assessed the performance of the proposed method using CASIA database (dataset B) under four side views: \(36^{\circ }\), \(72^{\circ }\), \(90^{\circ }\) and \(108^{\circ }\). Table 3 shows the performance results obtained in terms of IR at rank-1 and rank-5. By analyzing these results, it can be observed that the proposed method achieves a acceptable IR at rank-1 for both horizontal and vertical GEI divisions and for different viewing angles (up to 80 and 71.67% for horizontal and vertical division, respectively). The IR is increased at rank-5 up to 91.12 and 84.67% for horizontal and vertical division, respectively. This confirms that the proposed method allows recognizing gait under different viewing angles.

Table 3 Comparison of IR (in %) from the proposed method on CASIA database (dataset B) for four side views
Full size table

5.3 Additional experiment and future work

We have evaluated the proposed method on another database in order to assess its performances under other covariate factors such as shoe and walking surface. Table 4 shows the results obtained using the proposed method with the state-of-the-art method in [4] on USF Human ID gait database [24] that consist of persons walking under five covariates: viewpoints (left/right), two different shoe types, surface types (grass/concrete), carrying conditions (with/without a briefcase), time and clothing. Only two probes have been considered here: Probe A (grass walking surface+shoe type A+left camera viewpoint) and Probe C (grass walking surface+shoe type B+left camera viewpoint). We can notice that our proposed method provides encouraging results that remain comparable to the results of the method in [4]. The results obtained can be improved by a preprocessing the silhouette images (i.e., improvement of segmentation, effect of shadow, removing surface area to keep only shoe). This constitutes our future work.

Table 4 Comparison of IR (in %) from the proposed method with the method in [4] on USF Human ID gait database for Probe A and Probe C
Full size table

6 Conclusion and perspectives

This paper has proposed a novel gait recognition approach for human identification under variations of clothing and carrying conditions for different viewing angles. The proposed method based on Haralick features was evaluated on CASIA database and compared against some similar techniques. The results obtained have shown that the proposed features are relevant features for gait recognition under the effect of clothing and carrying conditions for different viewing angles. Our future work focuses on investigating the performance of the proposed approach under other covariate factors such as walking surface [25] and also to study other features capable of improving the performance of our proposed approach.