Keywords

1 Introduction and Related Works

Feature extraction is a crucial step which greatly affects the performance of the intended applications, and it is an open research problem of Computer Vision. Depending on the application, nature of the feature to be extracted varies. Feature extraction can be performed in two ways; namely local and global methods. Global methods represent the whole image with a single feature vector, whereas in local methods, a single image can have feature descriptors at multiple feature points [1]. These feature descriptors describe the local image neighborhood around the computed interest points. The features described by both global and local methods provide different information about the image as the support (neighborhood) used to extract the intended features is different. Global methods are compact representation of an image, and the feature vector of an image represented by a global method corresponds to a single point in higher dimensional space making it suitable for analysis. As global methods represent the entire image with a single feature, it assumes that the image contains only a single object and the method fails when the image has more than one object. This drawback is overcome by local methods.

Feature extraction using Gabor wavelets has many important applications in Computer Vision. Although the Gabor wavelets are used to extract features for various applications, but the procedure of feature extraction varies from one application to another. One important application is face recognition. In face recognition, Gabor wavelet-based feature extraction is implemented in two ways—Analytical methods and Holistic methods. In analytical methods, Gabor features are extracted from a pre-defined feature point, whereas Gabor features are used to represent the entire image in the holistic methods [2].

In facial expression recognition only a single frequency is selected or the interval between the neighboring frequencies is increased to reduce the dimensionality of Gabor feature for each of the orientations [3]. Dimensionality can be further reduced by using principal component analysis (PCA), and linear discriminant analysis (LDA). Most of the widely available Gabor-based feature extraction methods use magnitude information, but the method proposed in [4] only uses Gabor phase information for facial expression recognition. In [5], a spatially maximum occurrence model (SMOM) for facial expression recognition (FER) is proposed. Similarity comparisons between the images are performed by elastic shape-texture matching (ESTM) algorithm which is based on shape and texture information. Texture information in the high-frequency spectrum is obtained by applying Gabor wavelets on the edge images.

Traditional Gabor filter fails in rotation invariant texture analysis as the sinusoidal grating varies in one direction. In [6], a circular Gabor filter in which the sinusoid varies in all orientation is proposed.

Zhang and Ma used Gabor features for medical image retrieval [7]. The energy from the output of each Gabor filter is given as input to fuzzy sets, and features are extracted. To improve the retrieval rate, mean and standard deviation of the filter output is used instead of energy.

Chen and Liu proposed a face coding algorithm using Gabor wavelet network (GWN) [8]. GWN combines the local-feature and template-based characteristics. Feature extracted by GWN is converted to bit strings, and given them as input to genetic algorithm for coding.

Gabor wavelet is also used for determining stereo correspondence. Phase information of Gabor wavelet at different scales is used to find the corresponding matching pixels along the same scanline [9].

All the existing Gabor wavelet-based feature extraction methods either use Gabor magnitude or phase as features, which requires the computation of both real and imaginary coefficients. Subsequently PCA, LDA or statistical measures are used to reduce the dimensionality of the feature. Additionally, Gabor features are extracted at specific orientation in order to reduce the dimension of the feature. All these dimensionality reduction techniques affect the performance of an algorithm. Hence, the use of both real and the imaginary components of Gabor wavelets unnecessarily create a burden in terms of memory requirement and the computational complexity. To address this specific issue, we proposed a new local feature extraction method only using real coefficients of Gabor filter in spatial domain. It is experimentally verified that the real coefficients can produce significantly good results for most of the applications. The organization of the paper is as follows: Sect. 2 describes the proposed feature extraction method, Sect. 3 shows the experimental results, and Sect. 4 concludes the paper.

2 Proposed Local Gabor Feature Extraction Method

Feature extraction plays a crucial role in computer vision and image processing applications such as face recognition, facial expression recognition, texture classification and segmentation, image retrieval, face reconstruction, fingerprint recognition, iris recognition and stereo correspondence. Gabor wavelet is a widely used feature extraction tool in these areas. The motivation behind using Gabor wavelet Eq. (1) for feature extraction is as follows [10]:

  • Simple cells in the visual cortex of mammalian brains can be best modeled by Gabor function.

  • Gabor wavelet is a bandpass filter, and it is an optimal conjoint representation of images in both space and frequency domain that occupies the minimum area.

  • The orientation and scale tunable property of Gabor wavelet helps in detecting edge and bars which aids in texture feature extraction [11].

  • 2D Gabor wavelet has good spatial localization, orientation and frequency selectivity property.

  • Image perception by human visual system is similar to image analysis by Gabor function.

Gabor function is Gaussian modulated complex Eq. (2) sinusoids which is given by [12]

$$ g(x,y) = \frac{1}{{\sqrt {2\pi } }}e^{-{\tfrac{1}{8}\left( {4x^{2} + y^{2} } \right)}} \left[ {e^{i\kappa x} - e^{{ - \tfrac{{\kappa^{2} }}{2}}} } \right] $$
(1)

Gabor wavelet is referred as a class of self-similar functions generated by the process of orientation and the scaling of the 2D Gabor function which is given by

$$ \begin{aligned} & g_{mn} (x,y) = a^{ - m} g\left( {x_{a} ,y_{a} } \right),\;a > 1 \\ & x_{a} = a^{ - m} \left( {x\cos \theta + y\sin \theta } \right)\;{\text{and}} \\ & {\text{y}}_{a} = a^{ - m} \left( { - x\sin \theta + y\cos \theta } \right) \\ \end{aligned} $$
(2)

where, \( \theta = \frac{n\pi }{k} \), m and n are two integers and k is the total number of orientations.

To find the feature vector for a pixel of interest, a small image patch around the pixel is considered and convolved with Gabor wavelet with different orientations and scaling. The obtained filter outputs are concatenated to obtain the desired feature vector.

Figure 1 shows the block diagram of the proposed feature extraction method. Let us consider an image \( I \) of size \( P \times Q \). In order to find the feature vector for the pixel \( I(i,j) \), a certain neighborhood \( N(i,j) \) of size \( u \times v \) is considered, where \( (i,j) \) is the pixel coordinates. This patch is convolved with the Gabor filter kernel \( g_{mn} \) for different orientations and scaling. The features are then extracted by concatenating the obtained coefficients given by

Fig. 1
figure 1

Block diagram of the proposed local Gabor feature extraction method

Full size image
$$ \begin{aligned} & F(i,j) = {\text{concat}}\left( {\gamma_{mn} \left( {i,j} \right)} \right) \\ & \gamma_{mn} \left( {i,j} \right) = N\left( {i,j} \right) * g_{mn} \\ \end{aligned} $$
(3)

where, “\( concat \)” is the concatenation operator and “\( * \)” is the convolution operator. This procedure is repeated for all the pixels of the image. Since Gabor filter is a complex filter, we extracted three different features: only real, only imaginary, and both real and imaginary (magnitude) coefficients. Figure 2 shows the output (magnitude, real and imaginary part) of Gabor wavelet for teddy image.

Fig. 2
figure 2

Output of Gabor wavelet filtered image. a Input teddy image, b both the real and the imaginary coefficients (magnitude), c real part and d imaginary part

Full size image

In general, any function can be reconstructed by the linear superposition of its bases weighted by the wavelets. 2D Gabor wavelets are nonorthonormal bases i.e., they are nonorthogonal wavelets. The function can be approximately reconstructed when the family of wavelets is orthonormal basis [12]. This is achieved when \( 0 < A \le B < 2 \), and the function is reconstructed using the inversion formula given by

$$ f = \frac{2}{A + B}\sum\limits_{mn} {\left\langle {g_{mn} ,f} \right\rangle g_{mn} } $$
(4)

where, \( \left( {A + B} \right)/2 \) is a measure of the redundancy of the frame, \( B/A \) is a measure of the tightness of the frame. When \( A = B \), the frame is called a tight frame and the reconstruction by linear superposition using the above inversion formula is exact.

Reconstruction of the teddy image using the above inversion formula Eq. (4) is shown in Fig. 3. Similar to feature extraction, reconstruction of the original image is also performed by using only real coefficients, only the imaginary coefficients, and the magnitude. The reconstructed image using the extracted local features are compared with the original image, and the performance of the reconstruction is evaluated by using the metrics Mean Square Error (MSE), Peak Signal-to-Noise ratio (PSNR), and cross correlation (CC).

Fig. 3
figure 3

Reconstructed teddy image. a Input teddy image, b magnitude of the output, c real part and d imaginary part

Full size image

Extraction of Gabor (magnitude and phase) features is performed by convolving an image with the Gabor filter. Since Gabor wavelet is complex, feature extraction is done in two steps: (i) convolution of an image with real Gabor filter to obtain real coefficients, (ii) convolving an image with the imaginary Gabor filter to obtain the imaginary coefficients. In general, the computational complexity of Gabor filter is \( O\left( {PQh^{2} } \right) \), where \( P \times Q \) is the size of input image, \( h \times h \) is the Gabor kernel size. So, the computational complexity is significantly reduced in our method as we proposed to use only the real coefficients. Apparently, the memory requirement for storing of all the coefficients of Gabor filter is also reduced. This is because of the fact that both the real and imaginary coefficients have to be computed and stored in order to compute the magnitude and phase information. On the other hand, the proposed method requires half of the memory since it uses only the real coefficients. Hence, the applications which demand fast computation of features, and also the implementation with a reduced system memory, the proposed method can be effectively employed.

3 Experimental Results

The performance of the proposed feature extraction method is evaluated on the Middlebury stereo images [13, 14]. The feature vectors are extracted using the Eq. (2).

The original image is reconstructed from the real and imaginary feature vectors using the Eq. (4). The performance of the proposed feature extraction method is evaluated by comparing the reconstructed image \( R \) with the original input image. The metrics used to evaluate the performance are MSE, PSNR and CC. The performance of the proposed method is evaluated for different window size, number of orientations and scales. For all these cases, these three parameters are obtained. The performance of the proposed method is compared with the holistic Gabor features [2]. Tables 1, 2, 3, 4 and 5 shows the MSE, PSNR and CC for different window sizes, number of orientations and number of scales. Parameters used to obtain our experimental results are as follows: Number of scales—1 and 2, window sizes—\( 3 \times 3,5 \times 5,7 \times 7,9 \times 9,11 \times 11,13 \times 13 \) and \( {\kern 1pt} 15 \times 15 \), number of orientations—1, 2, 3, 4, 5, 6, 7 and 8. Apparently, low SNR, high PSNR and CC show better performance. In all these cases, we can see that the performance of the image reconstructed by using only the real coefficients outperforms the reconstruction using the imaginary coefficients and the magnitude of the coefficients. Additionally, reconstruction only by real coefficients is significantly better than the reconstruction by the magnitude of the coefficients of the holistic Gabor features. This is because, the receptive field profiles of human visual system can be best modeled by the product of a Gaussian and either a cosine or sine function [15]. Even receptive fields are bar detectors, while the odd receptive fields are edge detectors [16]. In following tables, H corresponds to holistic Gabor features, and the real, imag and mag denote the real, imaginary and magnitude information respectively.

Table 1 Comparison of MSE, PSNR and CC of the proposed method for different window sizes
Full size table
Table 2 Comparison of MSE and PSNR of the proposed method with the holistic approach for different number of orientations
Full size table
Table 3 Comparison of Correlation coefficients (CC) of the proposed method with the holistic approach for different numbers of scaling
Full size table
Table 4 Comparison of MSE and PSNR of the proposed method with the holistic approach for different numbers of scaling
Full size table
Table 5 Comparison of Correlation coefficients (CC) of the proposed method with the holistic approach for different numbers of scaling
Full size table

4 Conclusion

Feature extraction is a vital step in many Computer Vision applications. Due to some important characteristics of Gabor wavelet, it is one of the widely used filters for feature extraction. In this paper, we proposed a local Gabor feature extraction method. To evaluate the performance of the proposed method, the original input image is reconstructed with the help of the extracted feature vector, and subsequently, the performance of our method is evaluated using the metrics such as MSE, PSNR and CC. Experimental results show that the proposed feature extraction method can efficiently represent an image. Additionally, we found that the reconstruction of an image only using the real coefficients outperforms the other two reconstruction methods i.e., reconstruction only by imaginary coefficients, and reconstruction by using both the real and the imaginary coefficients (magnitude). The proposed method clearly has two important attributes—reduced computational complexity and the less memory requirement. So, the proposed method can be effectively used for the applications of image representation by a Gabor filter, which need less computation and also less memory.