1 Introduction

Face recognition biometrics is the process of automatically identifying a human being using the face features. It has the benefit of being a passive and non-intrusive system for verifying personal identity. It has gained much attention in recent years and has become one of the most successful areas in image analysis and understanding. Automatic face recognition can be categorized into the following two major areas: (i) Face identification: The process by which the system identifies the face image by performing a one-to-many search against the entire stored face images. (ii) Face verification or authentication: The process by which the system verifies the face image by matching a captured face image against the person’s stored face image.

This research investigates the task of automatic recognition of human faces in controlled environments. By concentrating on face recognition, this work covers only one of a larger set of techniques connected with the identification of face. Performing face recognition directly using raw images is an inefficient strategy due to high information redundancy in face images. To overcome this difficulty, feature extraction methods are used to transform pixel images into face features and these features are then used for analysis and recognition. There are two broad categories of feature extraction methods proposed in the literature (Zhao et al 2003): appearance based and local feature based methods.

Appearance of an object is the combined effect of its shape, reflectance properties, pose and illumination. Appearance-based approach represents the images in terms of their projection into generally a low-dimensional space, which captures the important characteristics of the object to be recognized. The most common approaches are Principal Component Analysis [PCA] (Turk & Pentland 1991; Kirby & Sirovich 1990), Linear Discriminant Analysis [LDA] and Independent Component Analysis [ICA] (Zhao et al 2003). Moments and moment-based functions are also used in many pattern recognition problems (Prokop & Reeves 1992). They are powerful tools for characterizing the image using region-based shape features and basic visual feature for image description and representation. Hu (1962) inspired the use of moments for image analysis by introducing two-dimensional geometric moment invariants for pattern recognition. Teague (1980) has suggested the notion of orthogonal moment (OM) to recover the image from moments based on the theory of orthogonal polynomials. OMs are used recently for the analysis of images and they were proved to be suitable for handling images with binary patterns such as character recognition, palm print verification, etc., under any geometric transformation (Terilion et al 2000). Recently, the moment-based features have been applied to face recognition also. In Haddadnia et al (2003), Pang et al (2004, 2006) the orthogonal Pseudo–Zernike moment features were used in face recognition. The Fisher LDA was used along with Pseudo–Zernike moment to enhance the discriminant nature (Pang et al 2006). The hybrid features, in which the moment-based method is combined with the statistical method, can improve the recognition rate. Foon et al (2004) used Zernike moment with wavelet and they showed a better recognition rate of 94.26% in Essex database. Arnold et al (2007) proposed Zernike moment as face features and proved that they were good discriminative features and possess robustness to rotation, scale, pose and illumination variation. Sheeba Rani et al (2007) proposed a hybrid approach using Pseudo–Zernike moment and curvilinear component analysis and showed that the performances of the rotated, tilted and noisy images were good. Mukundan et al (2001) introduced Tchebichef discrete OM to avoid the coordinate space normalization and computational complexity of continuous orthogonal moments.

In general, moments are global feature extractors. Their main characteristic is of being invariant to tilt and rotation and insensitivity to noise make it advantageous over the statistical methods in face recognition. Appearance-based method can affect the recognition result if the localization of the face is not properly done. Hair, shoulders, background may cause deviation in the feature vector representation. Local feature based method deals with local information such as local face features, their location and local statistics. Elastic bunch graph matching system based on Dynamic Link Architecture (Zhao et al 2003) is one of the successful approaches in local feature based method. The problem with the local feature based method is that they can be ill posed and may be brittle, subject to variation of scale and orientation. But they are less sensitive to variation as compared to appearance-based methods.

There are some approaches proposed to recognize face using hybrid method (Loon et al 1997; Phiasai et al 2001). Pentland et al (1994) proposed the modular eigenfaces approach which uses global eigenfaces and local eigenfeatures. In Penev & Atick (1996), PCA and Local Feature Analysis (LFA) were used to extract the hybrid features. In Lanitis & Taylor (2000) both shape and gray level information were used to identify the face. To overcome varying pose and illumination effect, Weyrauch et al (2003) proposed a component based recognition and 3D morphable models. Combination of different face classifier can integrate the complementary information, which gives a good recognition rate. In Huang et al (2000), ensembles of neural network classifier were used to classify the view specific eigenface features. Lu et al (2003) proposed combination of different face classifiers, where different component analyses were used for feature extraction. In Lawrence et al (1997), a hybrid neural network called convolutional neural network was proposed which is partially invariant to changes in the local image samples, scaling, and translation and has better classification than eigenface methods.

Successful object recognition system needs to use both global and local features for recognition. This research focuses on designing an efficient face recognition system using hybrid approach in feature extraction and classifier, which is invariant to size, rotation, illumination changes and poses. Krawtchouk discrete orthogonal moment is used to extract both the global features and local features of the image. Discrete orthogonal moments based on Krawtchouk polynomials (Yap et al 2003), has the ability to extract local features from any region of interest in an image in addition to the global feature extraction capability. This has motivated to investigate the Krawtchouk discrete orthogonal moment-based feature representation for hybrid face recognition. These characteristics are used to construct feature vectors which are invariant to scale, rotation and translation. Summed normalized distance classifier is used as classifier. The rest of the paper is organized as follows. The proposed face recognition system is presented in section 2. Section 3 presents the feature extraction techniques and classifier design. Section 4 presents the experimental results.

2 Proposed face recognition system

The proposed face recognition system is shown in figure 1. It first localizes the face region using feature invariant approach, and then it extracts the global features and local features using KM. Variations in illumination and expression will affect the recognition rate when global features alone are used. To solve the uneven illumination and pose problems, sub images are extracted which are left face, right face, lower face and upper face features using KM. The motivations in using sub images are (i) uneven illumination affects only some of the sub-images and when a face has a large depth rotation, roughly half of the face is occluded. Even for a frontal face image, if the image is captured even at a slight angle, the left half and right half of the face will not be symmetric affecting the global features. Summed distance fusion strategy is used to fuse the different features. Nearest neighbour classifier is employed for classification.

Figure 1
figure 1

Proposed system block diagram.

Full size image

2.1 Face localization

In this work, it is assumed that the given still image contains only one face, and the task is to localize the face portion which includes eyes, mouth and part of chin and forehead. A feature invariant approach is proposed in this work for face localization. The underlying assumption in feature invariant approaches is based on the fact that humans detect faces and objects in different poses and lighting conditions, and so, there exist properties or features which are invariant over this variability. The practical difficulty is in selecting features that are truly invariant with regard to all face patterns and the large number of variations associated with lighting conditions and pose. Most of the feature invariant algorithms in the literature are based on shape, colour and motion (Yang et al 2002). In this work, the elliptical shape of the face is considered as invariant feature to localize the face. The acquired image is subjected to gradient correction method to normalize the illumination effect. Next, the shape-based information of the face is extracted using edge detection technique. Using heuristic rules, the face contour point is detected from the edge information. The face region is extracted by fitting an ellipse using the contour point, and to precisely localize the face region, an inner ellipse is generated by reducing the width and height of the ellipse.

2.1.1 Edge detection

The edge detector, where thresholding is performed using statistical principles, is used in this work (Rishi et al 2004). In this method, the image is first smoothened using Gaussian kernel smoother and then, the gradient vector at each pixel of the image is calculated. The variation in the gradient vector at each pixel position is estimated using its variance covariance matrix based on standard statistical formulae. This variance covariance matrix is used to standardize the gradient vector at each pixel. This leads to a statistic, which is used to extract regions in the image where the gradient magnitudes are significantly large. Edge pixels are extracted from this region using algorithms similar to non-maxima suppression and thresholding with hysteresis.

2.1.2 Head contour detection

The edge-detected image is scanned from the centre of the top to find out an approximate top edge pixel (Cx, Cy) of the image with the assumption that only one face image is present in a plain image background, and is also scanned from the centre of the image bottom to find out an approximate bottom edge pixel. Then, using knowledge base information of the head, the image is scanned from the top edge pixel towards the right and then towards the left to find the upper-face contour points. Next, from the bottom edge pixel, the face image is scanned towards right and left to find the lower face contour. If an edge pixel is found in a particular row or column while scanning towards left (right), then it stops scanning that row and moves on to the next row and column to detect the edge pixel. Thus, the head contour points are extracted by scanning the edge-detected image from the pixel (Cx, Cy).

2.1.3 Ellipse fitting

Faces are characterized by elliptical shape and an ellipse can approximate the shape of a face. In this work, the fitting method based on optimization of an objective function (Fitzgibbon et al 1999) is used to fit an ellipse for the extracted contour points. The method gives the center, width, height and the angle of the ellipse for the given N contour points of the head. Bresenham’s ellipse generation algorithm (Newmann & Sproull 1979) is used to generate the ellipse using the estimated center, width, height and angle values.

To precisely localize the face portion, an inner ellipse is generated by reducing the width and height of the ellipse so that the elliptic major/minor ratio of the face is between 0.7 and 2. An elliptical mask is generated using the inner ellipse points. The face portion is segmented from the background, by logically AND the elliptical mask with the image.

2.1.4 Image normalization

Once a face has been localized and segmented within an image, the image itself must be standardized or normalized prior to further processing to improve the efficiency of matching. Image normalization is a process of adjusting, to allow particular areas in different images to line up when any two images are matched together. This could be as simple as a rescaling to some standard size, or as complex as remapping each pixel.

In this work, the angle of the ellipse is used to normalize the orientation of the image. Translation and scale invariance can be achieved by normalizing the image using the Cartesian moments (Mukundan and Ramakrishnan 1998). Translation invariance is achieved by moving the origin to the center of the image by using the centralized moments. Following this, scale invariance is produced by altering each object so that its area is GM 00 = β, where β is a predetermined value. Both invariance properties can be achieved using

$$ g\left( {x,y} \right)=f\left( {\frac{x}{A}+\overline x ,\frac{y}{A}+\overline y } \right), $$
(1)

where \(A=\sqrt {\frac{\beta }{GM_{00} }} \) and g(x, y) is the new translated scaled version and GM00 is the geometric moment of order (0,0). Then the image is scaled to a fixed size.

3 Krawtchouk moment for feature extraction

Feature extraction is the crucial task in object recognition. Feature extraction using invariant approach is a robust technique. Moments are powerful tools for characterizing the image using region-based shape features and basic visual feature for image description and representation. Regular higher order moments are sensitive to noise and have redundant information about an object feature because of the non-orthogonality function makes it less use. Here, OMs are used for extracting features from the face because orthogonal moments are less sensitive to noise, have good discriminating power and minimal information redundancy. In general, moments are invariant to shift, rotation and scale, but OM does not possess native invariance hence using affine transformation invariance can be achieved with respect to shift, rotation and scale.

3.1 Orthogonal moment transforms

Moments are characterized by their moment kernels. The moment kernels are used to transform the image in a scalar domain into a multidimensional moment space consisting of moments as points in the space.

The moment of an image can be calculated by the equation

$$ \Phi_{\rm{pq}} =\int \int\limits_{\rm c} {\mbox{f}\left({\mbox{x},\mbox{y}} \right)\upphi_{\rm {pq}} \left({\mbox{x},\mbox{y}}\right)\mbox{dxdy},} $$
(2)

where Φpq is the moment of order (p + q) over an image f(x, y) in the region specified by the moment function ϕ pq (x, y) known as the moment weighting kernel (i.e., \(\left\{{\phi_{pq} \left( {x,y} \right)} \right\}\) is the basis set). To characterize all of the information contained in an image segment requires a potentially infinite number of moment values. But practically the goal is to select a meaningful subset of moment values that contain sufficient information to uniquely characterize the image for a specific application.

Moment function with orthogonal polynomial set as kernel, additionally yields global characteristics of the image shape and provides lot of information about different types of geometrical features of the image. OM produce features that are less correlated than their non-orthogonal counterparts. Further, the orthogonal property enables simple and accurate signal reconstruction from the generated moments. Moments of orthogonal polynomial basis have proven to be less sensitive to noise, are inherently invariant to linear transformations, can be effectively used for image representation. Basically, OM is classified into two types: Continuous OM and Discrete OM. Krawtchouk polynomials are discrete in nature which gives discrete orthogonal moment.

3.2 Discrete orthogonal Krawtchouk polynomial

The nth order classical Krawtchouk polynomial is defined (Yap et al 2003) as

$$ K_n \left( {x;p,N} \right)=\sum\limits_{k=0}^N {a_{k,n,p} x^k} =\,_2 F_1 \left({-n,-x;-N;\frac{1}{P}}\right), $$
(3)

where x, n = 0,1,2......N, N > 0, p ∈ (0,1), n is the order of the Krawtchouk polynomial and N is the number of spatial points, p is the parameter of the binomial distribution, 2 F 1 is the hyper geometric function which is defined as

$$ { }_2F{ }_1\left( {a,b;c;z} \right)=\sum\limits_{k=0}^\infty {\frac{\left( a \right)_k \left( b \right)_k z^k}{\left( c \right)_k k!}} , $$
(4)

where (a) k is the pochammer symbol defined as

$$ \left( a \right)_k =a\left( {a+1} \right).......\left( {a+k+1} \right). $$
(5)

Krawtchouk’s polynomials occur classically as orthogonal polynomials with respect to the binomial distribution. To obtain numerical stability, the Krawtchouk polynomial is defined as

$$ \overline K_n \left( {x;p,N} \right)=K_n \left( {x;p,N} \right)\sqrt {\frac{w\left( {x;p,N} \right)}{\rho \left( {n;p,N} \right)}} $$
(6)

and satisfies the orthogonality condition

$$ \sum\limits_{x=0}^N {\overline K_n \left( {x;p,N} \right)\overline K_m \left( {x;p,N} \right)} =\delta_{nm} . $$
(7)

The set of (N + 1) Krawtchouk polynomials {\(\mathop K\nolimits_n (x;p,N)\)} forms a complete set of discrete basis function with weight function

$$ w\left( {x;p,N} \right)=\left( {{\begin{array}{*{20}c} N \\ x \\ \end{array} }} \right)p^x\left( {1-p} \right)^{N-x}, $$
(8)

where n, m = 1, 2........N and

$$ \rho \left( {n;p,N} \right)=\left( {-1} \right)^n\left( {\frac{1-p}{p}} \right)^n\frac{n!}{-N_n }. $$
(9)

The discrete equivalent of weighted Hermite polynomials (Wu & Shen 2005) are the weighted Krawtchouk polynomials which posses most of the characteristics of it.

3.3 Krawtchouk moment (KM)

KM is a set of moments formed by using Krawtchouk polynomials as the basis function set (Yap et al 2003). The KM of order (m + n), for an image with intensity function, f(x, y) is defined as

$$ Q_{nm} =\sum\limits_{x=0}^{N-1} {\sum\limits_{y=0}^{M-1} {\overline K_n } } \left( {x;p_1 ,N-1} \right)\overline K_m \left( {y;p_2 ,M-1} \right)f\left( {x,y} \right), $$
(10)

where p 1, p 2 ∈(0, 1).

The weighted mass of the image is obtained by substituting n  = m  =  0 in (2)

$$ Q_{oo} =\sum\limits_{x=0}^N {\sum\limits_{y=0}^M {\sqrt {w\left( {x;p_1 ,N-1} \right)w\left( {x;p_2 ,N-1} \right)} } } f\left( {x,y} \right). $$
(11)

To reduce the computation complexity, recurrence relationship is used to calculate the moments. 2D KM can be calculated by the cascade of 1D moment which reduces the computational complexity.

3.4 Spatial behaviour of Krawtchouk moment

To characterize the image by the use of moments, different spatial modes of the image are extracted efficiently. Orthogonal moment base functions of different orders having different number of zero crossings and varying in shapes can better separate image features based on different modes i.e., orders. The moment-based features are extracted by projecting the image on the moment base function space. The spatial domain behaviour of the KM are analysed and compared with geometric moments. Figure 2 shows the 1D moment functions of different orders of Krawtchouk moment and figure 3 shows the 1D moment functions of geometric moment.

Figure 2
figure 2

Spatial plot of 1D KM function \(\overline K_n \left( {x;\,0.5,\,100} \right)\) of orders n = 0, 1, 2, 3.

Full size image
Figure 3
figure 3

Spatial plot of 1D geometric moment function \(\phi \left( x \right)=x^p\), where p = 0,1,2,3.

Full size image

From figure 3, it is observed that the geometric moment function has no zero crossings and possess similar shape independent of the moment order. Thus, geometric moments can only characterize low frequency image features using different spatial modes. But in KM functions, with the increase of the order, the oscillations of the moment kernels also increase, which gives rise to different spatial modes for image characterization. Figure 4 shows the 2D KM functions of lower orders and figure 5 shows the 2D KM functions of higher orders. KMs are linear combinations of the derivatives of the image filtered by binomial filter. So, the local discriminated characteristics of the image can be extracted well. It is a smoothened orthogonal moment, so the noise sensitivity of the kernel is nullified. The binomial weighing kernel which acts as a smoothing function is responsible for this. In literature, it is proved that Krawtchouk moment is less sensitive to noise compared to all the OMs (Zhu et al 2007).

Figure 4
figure 4

2D plot of KM function \(\overline K_{mn} \left( {x,y;\,0.5,\,100} \right)\) of lower orders.

Full size image
Figure 5
figure 5

2D spatial plot of Krawtchouk moment function \(\overline K_{mn} \left( {x,y;0.5,100} \right)\) of higher orders.

Full size image

3.5 Frequency domain behaviour of Krawtchouk moment

The orthogonal KM functions produce more oscillations as the moment order (n) increases, so they contain more frequency. It can be justified by taking the Fourier transform of the orthogonal moment functions, and it is compared with geometric moment. Figure 6 shows the frequency response of 1D geometric moment of lower orders. From the figure, it is strongly inferred that all the geometric moment functions give a low pass filter response independent of order. Figures 7 and 8 show the frequency response characteristics of lower and higher order KM functions shown in figures 4 and 5. The frequency domain behaviour of the discrete Krawtchouk kernel behaves similar to a band pass kernel when the order increases. So, to effectively characterize or to represent an image these features are good enough.

Figure 6
figure 6

Fourier amplitude spectra of lower order 1D geometric moment function.

Full size image
Figure 7
figure 7

Fourier amplitude spectra of lower orders 2D KM function.

Full size image
Figure 8
figure 8

Fourier amplitude spectra of higher order 2D KM function.

Full size image

3.6 Local feature extraction property

KM has the property of extracting features from the region of interest by varying p 1 and p 2. If the value of p 1 = p 2 = 0.5, the region of interest lies in the centre of the image. If the values deviate from 0.5 by Δp 1 and Δp 2, the Krawtchouk polynomial are approximately shifted by NΔp 1 and MΔp 2, and it shifts the region of interest horizontally and vertically. This property is utilized in this paper for extracting local features by varying the values of p 1 and p 2.

3.7 Classification method

3.7.1 Fusion of global and local features for classification

For any given sample, five d dimensional discriminated feature vectors are obtained. Hence, it is necessary to fuse them in the decision level. Here, a summed normalized distance fusion strategy is used. Take a pattern \(z=\left[ {z^1,\,z^2,\,z^3,\,z^4,\,z^5} \right]\), where z 1 is the global feature vector, z 2, z 3, z 4, z 5 are local feature vectors. The summed normalized distance between sample z and training sample \(z_i =\left[ {z_i^1 ,\,z_i^2 ,\,z_i^3 ,\,z^4,\,z^5} \right]\), is

$$ g\left( {z,z_i } \right)=\frac{\left\| {z^1-z_i^1 } \right\|}{\sum\limits_{j=1}^L {\left\| {z^1-z_j^1 } \right\|} }+\frac{\left\| {z^2-z_i^2 } \right\|}{\sum\limits_{j=1}^M {\left\| {z^2-z_j^2 } \right\|} }+\frac{\left\| {z^3-z_i^3 } \right\|}{\sum\limits_{j=1}^M {\left\| {z^3-z_j^3 } \right\|} }+\frac{\left\| {z^4-z_i^4 } \right\|}{\sum\limits_{j=1}^M {\left\| {z^4-z_j^4 } \right\|} }+\frac{\left\| {z^5-z_i^5 } \right\|}{\sum\limits_{j=1}^M {\left\| {z^5-z_j^5 } \right\|} }, $$
(12)

where i = 1,2.......L, L is the total number of training samples.

3.7.2 Nearest neighbour classifier

The nearest neighbour classification rule is defined as follows

$$ g\left({z,z_k}\right)=\mathop{\min}\limits_j \,g\left( {z,z_j } \right)\to z\in \omega_k {.} $$
(13)

The sample vector z is classified to the class of the minimum distance based on the similarity measure g that is z j belongs to class k, and then z belongs to class k.

4 Experiments

To demonstrate the effectiveness of the proposed approach, experimental studies are conducted on two well-known face image databases ORL and Yale. The ORL database has been used to evaluate the performance of the system under variation in pose. The Yale database has been used to test the performance of the system when both facial expression and illumination are varied.

4.1 Databases

4.1.1 ORL database

The ORL database (http://www.camorl.co.uk/facedatabase.html) developed at the Olivetti Research Laboratory, Cambridge; (UK) contains images from 40 individuals in different states, taken between April 1992 and April 1994. The total number of images for each person is 10. None of the 10 images is identical to any other. They vary in position, rotation, scale and expression. The changes in orientation have been accomplished by each person rotating a maximum of 20 degrees in the same plane, as well as each person changing his/her facial expression in each of the 10 images (e.g., open/close eyes, smiling/not smiling). The changes in scale have been achieved by changing the distance between the person and the video camera. For some individuals, the images were taken at different times, varying facial details (glasses/no glasses). All the images were taken against a dark homogeneous background. Each image was digitized and presented by a 112 × 92 pixel array whose gray levels ranged between 0 and 255.

4.1.2 Yale database

The Yale database (http://eve.yale.edu/projects/yalefaces/yaleface.html) was taken from the Yale Center for Computational Vision and Control. It consists of images from 15 different people, using 11 images from each person, for a total of 165 images. The images contain variations with the following total expressions or configurations: center-light, with glasses, happy, left light, without glasses, normal, right-light, sad, sleepy, surprised and wink.

4.2 Experiments on the ORL database

Among the 400 images, 200 images are used to train, and another 200 are used to test. Each training set consists of 5 randomly chosen images from the same class in the training stage. There is no overlap between the training and test sets. The images are preprocessed and the face part is extracted from all the training images using the face localization as explained in section 2.1. The extracted face images for a few samples are shown in figure 9. Next, the localized face images are normalized to obtain scale, translation and orientation invariant images. Then, using the local feature extraction property of KM, the upper, lower, right and left parts of the face are extracted using p 1 = 0.5 and p 2 = 0.1, p 1 = 0.5 and p 2 = 0.9, p 1 = 0.95 and p 2 = 0.5, p 1 = 0.05 and p 2 = 0.6. The extracted images are shown in figure 10. KMs are calculated for full face using p 1 = 0.5 and p 2 = 0.5 of order 12, 19, 29. The hybrid features are given as input to the summed normalized distance classifier.

Figure 9
figure 9

Face localization steps. (a) Original image, (b) edge detected image, (c) contour detection, (d) ellipse fitting, (e) localized face, (f) localized face using inner ellipse, (g) and (h) normalized face.

Full size image
Figure 10
figure 10

Local face features extracted using Krawtchouk moment. (a) Lower face, (b) left face, (c) upper face, (d) right face, (e) full face.

Full size image

The first experiment aims at addressing the performance of the hybrid approach compared with the global approach with and without face localization. The results are shown in table 1.

Table 1 Effect of moment order with recognition rate.
Full size table

The recognition rate greatly gets reduced without localization in both Krawtchouk global and hybrid extraction methods. But, the performance improves immensely as the moment order increases. It is because of the characteristics of the Krawtchouk orthogonal polynomial. The energy content is not condensed in one area as that of the other orthogonal transforms but it is spread out to some extent which is shown in figure 11. Krawtchouk hybrid gives the highest recognition rate of 95%.

Figure 11
figure 11

(a) Transformed coefficient using discrete Krawtchouk polynomial transform. (b) Reconstructed image using 320 discrete Krawtchouk coefficients.

Full size image

The second experiment aims at comparing the performance of KM of order 19 with that of 2DPCA (Yang & Zhang, 2003) of dimension 112 × 5. In this comparison, two experimental strategies were adopted: First, only frontal images for training and the remaining non-frontal images are used for testing. The second is leave-one-out strategy, in which, the image of one person is removed from the data set and all the remaining images are used for training. The results are shown in table 2.

Table 2 Comparison of Krawtchouk moment with 2DPCA.
Full size table

Krawtchouk global and local features are good in identifying non-frontal images even though frontal images were used in training.

Next, the system was tested by keeping 19 as the moment order and varying the number of classes for the two strategies. Table 3 shows the effect of number of classes with their recognition rate. From the test result it is inferred that as the number of classes increase, the recognition rate slowly decreases for randomly used strategy, but the recognition rate is improved for frontal image strategy. Krawtchouk hybrid is better in recognizing non-frontal images and 2DPCA is good in recognizing images when random images are used for testing and training.

Table 3 Effect of number of classes on recognition rate.
Full size table

4.2.1 Effect of recognition rate with different combination of local features

In the third experiment the global features of moment order 19 and the various combinations of local features of the same order are combined. The recognition rate is improved when all the features are considered. Significantly, the recognition rate is quite good for the combination of full face + upper face which is less variant to facial expression. The results are presented in table 4.

Table 4 Effect of recognition rate with different combination of local features moment order 19.
Full size table

4.2.2 Effect of noise on recognition rate

The fourth experiment is done to evaluate the performance of the noise corrupted images. Here, the test images are added with Gaussian noise of variance 0.1 and 0.5 before giving it to the classifier. The performance is good for hybrid features as compared to global and 2DPCA features even though the test images are corrupted by noise. From this, it is justified that lower order KM are less sensitive to noise. Table 5 shows the effect of noise on recognition rate.

Table 5 Effect of noise on recognition rate.
Full size table

4.2.3 Computation time for extracting the features

One of the minimal requirements of a face recognition system is the ability to give the result quickly. Hence, computational time is an important factor to be considered. A face should be recognized within a minimum duration of time. The time required by the different feature extraction methods to extract the face features from a single image for order 19 is given in table 6. The results show that KM takes very less time for extracting the features as compared to 2DPCA and the 2DPCA takes very much less time as compared to one-dimensional PCA, whereas, for the hybrid combination, the computation time is high as compared to other methods because of the increase in features.

Table 6 Comparison of CPU time (s) for feature extraction in ORL (cpu: pentium iv, 2 GHz, 256 MB RAM).
Full size table

4.3 Experiments with Yale database

The Yale database images are illumination normalized before localization. The normalized images are localized and resized to 100 × 80 pixels. The developed localization method is evaluated for noisy images also. The images are added with speckle or salt and pepper noises. The noisy images are subjected to the face localization. It is found that the localization method performed well even when the images are corrupted by noise. Figure 12 shows the localized image of a noisy image.

Figure 12
figure 12

Face localization of noisy image.

Full size image

The performance of the recognition system is evaluated using random strategy and leave-one-out strategy. In random strategy, randomly 6 images are used for training, and 5 images are used for testing. In leave-one-out strategy, 1150 images are used for training in each trial and 15 images are used for testing. The full, upper, lower, right and left part of the face features are extracted using KMs. Figure 13 shows the sample feature extracted images.

Figure 13
figure 13

(a) Sample Yale images. (b) Edge detected images, (c) localized images.

Full size image

The proposed Krawtchouk local feature extraction method is compared with Krawtchouk global and 2DPCA.

The recognition rate is tabulated in table 7. From the table it is inferred that the performance of Krawtchouk global moment and 2DPCA based approaches are same. But the recognition rate has improved when hybrid Krawtchouk features are used. It is found that the use of local features makes the uneven illumination effect less variant to the face features so that the recognition rate is improved.

Table 7 Comparison of Krawtchouk moment with 2DPCA using Yale database.
Full size table

5 Conclusion

In this paper, a hybrid face recognition system using discrete Krawtchouk orthogonal moment is proposed, and the potential of local feature extraction property for face recognition is investigated. It is found that the Krawtchouk moment can recognize non-frontal images even though it is trained up with frontal images. A good recognition rate is obtained using Krawtchouk moment when the image is corrupted by noise. The effect of uneven illumination is somewhat compensated because of the fusion of local features along with the global features. This work can be improved further, using RBF neural network classifiers and fuzzy integral as fusion strategy.