Keywords

1 Introduction

Computer vision and Biometric systems have illustrated the significant improvement in recognizing and verifying faces in digital images. The face recognition methods that are performing well in constrained environments, includes principal component analysis [1], linear discriminant analysis [2], Fisherface [3], etc. In many applications, including facial image analysis, visual inspection, remote sensing, biometrics, motion analysis, etc. Mostly, these environments are not constrained. Therefore, we aim to develop an efficient method that accurately recognizes the individual from their unconstrained facial images.

In literature, the methods that work in unconstrained face images are mainly based on the texture descriptions. The local feature-based or multimodal approaches to face recognition have achieved attention in the scientific world [4, 5]. These local feature-based and multimodal methods are less sensitive to variations in pose and illumination than the traditional methods. In unconstrained environments, the local binary pattern (LBP) is one of the popular methods of face recognition. The intuition behind using the LBP operator for face description is that the face can be seen as a composition of various micro-patterns; and it is insensitive to variations, such as pose and illumination. Global description of the face image is obtained by consolidating these micro-patterns [6].

In literature, plenty of methods have been proposed to improve the robustness of the LBP operator in unconstrained face recognition. For example, Liao et al. [7] proposed dominant LBPs which make use of the frequently occurred patterns of LBP. Center-symmetric local binary pattern is used to replace the gradient operator used by the SIFT operator [8]. Multi-block LBP, replaces intensity values in the computation of LBP with the mean intensity value of image blocks [9]. Local ternary pattern (LTP) was initiated by Tan and Triggs [10], to add resistance to noise. However, LTP representation is still limited due to its hard and fixed quantization. Three-patch LBP and four-patch LBP utilize novel multi-patch sampling patterns to add sparse local structure into a composite LBP descriptor [11].

The prime issues of the LBP are that insensitive to the monotonic transformation of the gray-scale, they are still susceptible by illumination variations that generate non-monotonic gray-scale changes, such as self shadowing [4]. LBP may not work properly for noisy images or on flat image areas of constant graylevel. This is due to the thresholding scheme of the operator [12]. The remainder of the paper is organized as follows: Sect. 2, proposes the novel method for face recognition. Evaluation of the proposed method and their comparison with LBPs are presented in Sect. 3. Finally, the conclusions are outlined in the last section.

2 Proposed Method

In this section, we present a novel method that acts on the LBP, called augmented local binary pattern. Earlier work on the LBP have not given too much attention on the use of non-uniform patterns. They are either treated as noise and discarded during texture representation, or used in consolidation with the uniform patterns. The proposed method considers the non-uniform patterns and extract the discriminatory information available to them so as to prove their usefulness. They are used in consolidation to the neighboring uniform patterns and extract invaluable information regarding the local descriptors.

The proposed method uses a grid-based regions. However, instead of directly putting all non-uniform patterns into 59th bin, it replaces all non-uniform patterns with the mod of neighboring uniform patterns. For this, we have taken a kernel of size 3 × 3 that is moved on the entire LBP generated surface texture. In this filtering process, the central pixel’s value \((c_{p} )\) is replaced with the mode of a set in case of the non-uniformity of the central pixel. This set contains 8-closet neighbors of central pixel, in which non-uniform neighbors are substituted with 255. Here 255 is the highest uniform value.

The lookup table containing decimal values of 8-bit uniform patterns are U = {0, 1, 2, 3, 4, 6, 7, 8, 12, 14, 15, 16, 24, 28, 30, 31, 32, 48, 56, 60, 62, 63, 64, 96, 112, 120, 124, 126, 127, 128, 129, 131, 135, 143, 159, 191, 192, 193, 195, 199, 207, 223, 224, 225, 227, 231, 239, 240, 241, 243, 247, 248, 249, 251, 252, 253, 254, 255} [13]. The basics of filtering process is explained in Fig. 1.

Fig. 1
figure 1

a Neighboring non-uniform patterns are replaced with highest uniform pattern 255, b the central non-uniform pattern 36, replaced with mode value of 32, and c the next central pattern 64 is found to be uniform, therefore remains unchanged

Full size image

The classification performance of the proposed method is evaluated with Chi square (\(\chi^{2}\)) distance measure which are formally defined as follows:

$$\chi^{2} (p,q) = \sum\limits_{i = 1}^{N} {\frac{{(p_{i} - q_{i} )^{2} }}{{(p_{i} + q_{i} )}}}$$
(1)

where \(N\) is the dimensionality of the spatially enhanced histograms, \(p\) is the histogram of the probe image, \(q\) is the histogram of the gallery image, \(i\) represents the bin number and \(p_{i}\), \(q_{i}\) are the values of the ith bin in the histograms \(p\) and \(q\) to be compared.

The schematic of the proposed A-LBP method is shown in Fig. 2.

Fig. 2
figure 2

Schematic of a A-LBP face recognition system [14]

Full size image

3 Experimental Results

The efficiency of the proposed method is tested on the publicly available face databases, such as AT & T-ORL [15], extended Yale B [16], and Yale A [17]. These databases differ in the degree of variation in pose (p), illumination (i), expression (e) and eye glasses (eg) present in their facial images. The face recognition accuracy of the proposed A-LBP method is compared to the LBP method on the different face databases (see Table 1). The performance of the proposed A-LBP method is analyzed using equal error rate, which is an error, where the likelihood of acceptance assumed the same value to the likelihood of rejection of people who should be correctly verified. The performance of the proposed method is also confirmed by the receiver operating characteristic (ROC) curves. The ROC curve is a measure of classification performance that plots the genuine acceptance rate (GAR) against the false acceptance rate (FAR).

Table 1 Face recognition accuracies of methods on different face databases
Full size table

The face recognition accuracy of the proposed A-LBP method is compared to the LBP method on different face databases. The experimental results show that the A-LBP performs better than the LBP. For AT & T-ORL database, the A-LBP achieves a recognition accuracy of 95 %, whereas LBP reports an accuracy of 92.5 %. Similar trends are also observed for extended Yale B and Yale A databases. For extended Yale B database, proposed method performs better than LBP such as the accuracy values are reported to 81.22 % and 74.11 %, respectively. For Yale A database, the proposed method reported the better accuracy value of 73.33 % in comparison to LBP accuracy value of 61.19 % (see Table 1).

The ROC curve for AT & T-ORL database is plotted and shown in Fig. 3a. It shows that the GAR is found highest for the proposed A-LBP method and reported value of 78 %; when the FAR is strictly nil. As FAR increases, the GAR value is also increased. For example, the GAR is found 93 % for LBP, 96 % for A-LBP at 5 % of the FAR. The GAR is found maximum 100 % of 32 % FAR. The ROC curve for extended Yale B database is plotted and shown in Fig. 3b. It shows that the GAR is found highest for A-LBP method and reported value of 32 %; when the FAR is strictly nil. As FAR increases, the GAR value is also increased for all methods accordingly. For example, the GAR is found 62 % for LBP, 82 % for A-LBP of 20 % FAR. The GAR is found maximum 100 % at 83 % of FAR for LBP and 78 % of FAR for A-LBP. The A-LBP method achieves better recognition accuracy, because it is insensitive to changes such as illumination.

Fig. 3
figure 3

ROC curves representing the performance of different face recognition methods on the databases: a AT & T-ORL, b extended Yale B, and c Yale A

Full size image

The ROC curve for Yale A database is plotted and shown in Fig. 3c. It shows that the GAR is found highest for A-LBP method and reported value of 20 %; when the FAR is strictly nil. As FAR increases, the GAR value is also increased for all methods accordingly. For example, the GAR is found 50 % for LBP, 69 % for A-LBP at 20 % of the FAR. The GAR is found maximum 100 % at 90 % of FAR for LBP and 82 % of FAR for A-LBP. The A-LBP method achieves better recognition accuracy, because it is insensitive to changes such as illumination.

The histogram representation of  recognition performance achieved by the LBP and proposed A-LBP methods using different face databases, such as Yale A, extended Yale B, and AT & T-ORL is shown in Fig. 4.

Fig. 4
figure 4

Histogram of equal error rate (EER) of different databases

Full size image

4 Conclusion

This paper presents a novel method of face recognition under unconstrained environments. The proposed method namely, A-LBP has efficiently recognized the faces from challenging databases. A-LBP work on the consolidation of the principle of locality of uniform and non-uniform patterns where non-uniform patterns are replaced with the mod value of the uniform patterns. It consolidates the neighboring uniform patterns that extract more discriminatory information from local descriptors. The experimental results, have shown that the performance of the A-LBP method improved substantially with respect to LBP on different face databases. The accuracy values of LBP and A-LBP vary considerably with training databases and the distance metrics preferred. When there are more variations in illumination of facial images in the databases, the A-LBP has shown promising recognition accuracy than the LBP. Similar trends are also observed for the databases that have variations in pose and expressions.