1 Introduction

In the last decades, the increasing interest in security and the evaluation of the robustness of biometric systems has shown to be a major field of research. Earlier, biometric systems were designed to only recognize identities without concern to spoof attacks. Biometric systems are vulnerable to several types of treats grouped by sensor tampering, database tampering, attacking the channel between the database and matching and many other attacks described in [1].

Among the different types of treats analyzed, the increasing interest in the study of the so-called direct or spoofing attacks, specifically print and replay attacks, has led to initiatives focused on this major field of research. Print attack is based on printed modality images of an identity to spoof 2D recognition systems, while replay attack is carried out by replaying a video sequence of a live identity on a screen that is either fixed or hand-held to evade liveness detection.

Detection of spoofing attacks is still a big challenge in the field biometrics and has motivated the biometric community to study the vulnerabilities against this type of fraudulent actions in modalities such as the fingerprint [2], the face [3, 4] and even the gait [5].

In order to counter spoofing in the recognition systems, techniques are generally divided into texture, motion and liveness analyses [6]. Texture analysis techniques mainly detect texture patterns such as print failures, and overall image blur to detect attacks. Motion analysis refers to motion features such as optical flow which are used to get over the dependency on certain texture patterns [7, 8]. However, motion-based methods have limitations when there is low motion information in a video due to static subject motion and low-resolution cameras. On the other hand, liveness cues depend on the vitality signs of a biometric trait by analyzing spontaneous movements that cannot be detected in photographs. Eye blinking, lips movement and changes in facial expression can be considered as these cues for 2D face recognition. As a result, one solution may not always be generalized to other attack methods.

Furthermore, the quality of digital images could visibly degrade, since they are subject to distortions during acquisition, capturing, processing, transmission and reproduction. For example, palmprint images captured from a printed paper are more likely to present local acquisition artifact such as spots and patches; face images captured from an electronic device will probably be over- or underexposed. Recently, a significant amount of research has gone into the development of quality assessment methods that take advantage of known characteristics of the human visual system. Therefore, using a wide range of image quality methods (IQM) should also detect the quality differences between real and fake samples.

In this work, we focused on both printed photograph and replayed video attacks to unimodal 2D face and palmprint recognition systems separately which are easy to be reproduced and has potential to succeed. The contribution of this paper can be summarized as follows: Our first contribution is to introduce the use of our proposed pipeline for countering spoof attacks in face and palmprint recognition systems. In order to counter both printed photograph and replayed video attacks, fusion of different texture-based and IQA-based methods is proposed. We constructed a palmprint spoof database including 50 subjects made by printed palmprint photograph using a high-quality camera. It allows us to evaluate the ability of different palmprint spoof detection algorithms. We present results of both face and palmprint spoof detection methods using three public-domain face spoof databases (Idiap Print-Attack, Replay-Attack and MSU MFSD) and palmprint spoof database.

The rest of the paper is organized as follows: Sect. 2 presents different research works that have been reported in biometrics anti-spoofing. Sections 3 and 4 give a brief description of texture-based methods and image quality assessment metrics. Section 5 describes the proposed anti-spoofing framework. In Sect. 6, we describe the databases and the experimental results obtained using different methods. In the last section, we conclude this study.

2 Prior works

Recently, different types of countermeasures based on motion and texture analysis have been considered for face anti-spoofing [9,10,11]. Face spoofing detection from single images using micro-texture analysis was implemented in [9] to emphasize the differences of micro-texture in the feature space. The proposed approach analyzes the texture of the facial images using multi-scale local binary patterns (LBP) and encodes the micro-texture patterns into an enhanced feature histogram. Support vector machine (SVM) classifier is then used to determine whether there is a live person in front of the camera or not. The proposed simple LBP \(+\) SVM method achieved a comparable result both on NUAA and Idiap databases. In [10], a novel physics-based method is proposed in order to detect images recaptured from printed paper using only a single image. Features extracted from micro-textures presented in printed paper are then fed to a linear SVM classifier. The experimental results demonstrate the robustness of the proposed approach with a low equal error.

A novel approach is proposed to detect face spoofing using the dynamic texture extensions of the local binary pattern operator [11]. The proposed method detects the structure and the dynamics of the facial micro-textures that characterize real faces but not fake ones. In order to evaluate the proposed approach, two publicly available databases, namely Replay-Attack database and CASIA Face Anti-Spoofing Database, are used.

A number of comparative studies have been reported to suggest motion information that can cover a wide range of attacks targeting the 2D face recognition systems. In [12], an efficient face spoof detection algorithm based on image distortion analysis (IDA) is proposed. Specular reflection, blurriness, chromatic moment, and color diversity features are extracted to form the IDA feature vector. In order to extract facial dynamic information [3], modified dynamic mode decomposition (DMD) was used to capture the complex dynamics of head movements, eye blinking and lip movements found in a typical video sequence containing face images.

The use of image quality assessment for liveness detection has been studied for image manipulation detection. A novel software-based fake detection method is presented in [13] to detect different types of fraudulent access attempts. The proposed approach used 25 general image quality features extracted from one image to distinguish between real and impostor samples.

3 Baseline methods

3.1 Texture-based methods

Texture-based methods focus on textural differences between the live and counterfeit biometric images. In this study, local binary pattern (LBP), difference of Gaussians (DoG) and histograms of oriented gradients (HOG) have been implemented for analyzing and measuring the texture quality and determining whether degradations due to recapturing process. Local binary patterns (LBP), a powerful image texture descriptor, extracts texture information that is invariant to local grayscale variations [14,15,16]. Difference of Gaussian (DOG) is a band-pass filter originally proposed by [17, 18]. It removes high-frequency components representing noise by constructing a Gaussian pyramid from the input image. Histograms of oriented gradients (HOG) is a feature descriptor technique that counts occurrences of gradient orientation in localized portions of an image [19].

3.2 Image quality assessment metrics

Image quality metrics can be classified depending upon the availability of a reference image, with which the distorted image is to be compared. Most existing techniques are well known as full-reference (FR) measures, meaning that the quality of a test image is evaluated by comparing it with a reference image that is assumed to have perfect quality. No-reference (NR) metrics try to assess the quality of a test image without any reference to the original one.

In this paper, well-known full-reference objective measurements are used to evaluate image quality including pixel difference measures such as peak signal-to-noise ratio (PSNR) [20], mean-squared error (MSE) [21], maximum difference (MD), normalized absolute error (NAE) and average difference (AD) [22], structural similarity measures such as similarity index measure (SSIM) [23] and correlation-based measures such as normalized cross-correlation (NXC) [22].

Pixel difference measures compute the distortion between two images on the basis of their pixelwise differences. The nonstructural distortions in an image that come from illumination variations, such as contrast or brightness changes, should be treated differently from structural ones. Structural similarity index measure (SSIM) has achieved prevalent popularity in view of its properties. On the other hand, the similarity between two digital images can be quantified in terms of the correlation function such as normalized cross-correlation (NXC).

4 Proposed anti-spoofing framework

Our proposed anti-spoofing approach is presented in Fig. 1. Additionally, the following steps explain the detailed stages employed in the proposed method for face and palmprint spoof detection:

Step 1 Image preprocessing is performed on face and palmprint images separately. First, a video frame of face samples in Idiap databases is decomposed to single frames in order to produce single-image inputs. Then, all produced palm and face images are resized to \(60\times 60\) prior to the anti-spoofing experiments. Following this stage, all images undergo histogram equalization technique and then normalized with mean variance normalization.

Step 2 All the entire images are then filtered with LBP or HOG which produce the best results in Replay-Attack and Print-Attack databases, and they are divided into several blocks (each image divided into \(5\times 5\) blocks) to produce 25 blocks. Features are extracted from each biometric image to produce a global feature vector of each modality.

Fig. 1
figure 1

The proposed anti-spoofing Pipeline

Full size image

Step 3 Due to high-dimensional feature space, before producing the scores, PCA and LDA are employed to find a proper feature subset of each biometric source by removing irrelevant and redundant information.

Step 4 In order to avoid degradation in accuracy, the matching scores have to be transformed into a unique domain before classification. In this study, the normalized scores are obtained by using tanh-normalization method which is robust and highly efficient.

Step 5 Euclidean distance measure is used to measure the similarity between real and fake images. The classification is performed using nearest neighbor classifier due to its simplicity and its generally good performance reported in classification.

Step 6 On the other hand, the original images are used prior to the computation of the IQ features. Normalized absolute error (NAE) is then used as the best image quality assessment metric according to our experimental results in order to provide a general quality score. This has allowed us to compare the image quality between a genuine image and fake one by considering the minimum error of a real or fake test sample with all real and fake images stored in training set.

Step 7 In Step 5 and Step 6, each classifier is applied separately in order to decide if the input is real or fake and the final decision is postponed to the end of the fusion process in order to take advantage of each method. In the fusion step, the decision provided by the different texture-based methods in Step 5 and image quality assessment metric in Step 6 are fused using logical OR operator. The decision real is returned if either or both decisions are real and fake decision otherwise.

5 Experimental results

This section presents our experimental analysis on the proposed texture-based and IQA-based methods using the Print-Attack, Replay-Attack, MSU MFSD face databases and our palmprint spoof database. These experiments are performed to demonstrate the robustness of the proposed schemes against spoofing attacks. The results are presented on the tables for both proposed schemes (using LBP, DoG and HOG) with and without performing PCA and LDA on the images. In addition, we examined the effect of image quality toward anti-spoofing accuracy. A biometric spoofing detection system is subjected to two different types of errors that are used in recent publication [13]: the False Genuine Rate (FGR), which is the number of false samples that are incorrectly classified as real, and the False Fake Rate (FFR), which is the number of genuine samples being considered as fake. The widely used performance measure is Half Total Error Rate (HTER), defined as half of the sum of the False Genuine Rate (FGR) and False Fake Rate (FFR). Hence, in all cases, performance of the palmprint and face spoof detection systems has been reported in terms of FFR, FGR and HTER.

Fig. 2
figure 2

Typical palmprint sample belongs to the same person in our constructed palmprint spoof database. a Spoof palmprint generated for printed photograph attack, b a fake palmprint image, c a genuine palmprint image

Full size image

5.1 Databases

In this section, we provide a brief summary of three face spoof databases, namely Idiap Print-Attack and Replay-Attack databases which are publicly available from the Idiap Research Institute [4] and MSU Mobile Face Spoofing database (MSU MFSD) [12] in order to facilitate spoof detection research on mobile phone applications. In addition, we present constructed palmprint spoof database made by printed palmprint photograph using the camera.

5.1.1 Palmprint spoof attack—printed photograph database

PolyU database [24] provided by the Hong Kong Polytechnic University is used to create a palmprint spoofing database. All the original grayscale images of size \(150\times 150\) pixels are cropped and resized to \(50\times 50\). Due to non-availability of spoof palmprint database, we constructed a spoof database made by printed paper. In order to generate a printed photograph for attack, KONICA MINOLTA 554eSeriesPCL printer (\(1200\times 600\) dpi) is used to print palmprint images on a plain A4 paper from PolyU database. Canon 6D camera is then used to capture a HD picture (\(5472\times 3648\)), which is then stored in spoof palmprint database. The average standoff for the printed photograph attack is 50 cm. Figure 2 shows example images of genuine and spoof palmprint of one subject in the PolyU database. In this study, we prepared a palmprint dataset using 50 individuals, each including 10 real samples and 10 attack samples randomly selected from left and right hands. In general, for validating the performance under spoofing attacks the whole database of 50 individuals is divided into two sets. The train and test sets distribution is indicated in Table 1.

Table 1 Number of samples available in each real and fake subset in palmprint spoof database
Full size table
Table 2 Results of face in HTER (%) obtained on the Print-Attack database by the proposed texture-based methods
Full size table

5.1.2 Face spoof database

The Idiap Replay-Attack database extended the Idiap Print-Attack database [25] by augmenting the spoof face samples with a wider range of variability (replayed video and photograph attacks). Idiap Print-Attack and Replay-Attack databases involve 200 short video clips of printed photograph and 1300 video clips of photograph and video for both valid-access and attack attempts of 50 different subjects under different lighting conditions.

5.1.3 MSU MFSD database

The MSU MFSD database consists of 440 video clips of photograph and video attack attempts of 55 subjects. Two types of cameras were used in collecting this database: (i) built-in laptop camera; (ii) front-facing Android phone camera. Genuine subject presents his face close to the camera, and a genuine face video is recorded using both the Android and laptop cameras.

Table 3 Results of face in HTER (%) obtained on the Print-Attack, Replay-Attack, palmprint spoof and MSU face databases by the proposed image quality assessment metrics
Full size table

5.2 Results

We evaluated our experiments on face databases only when the face regions are considered and the background is not included. For this reason, the input face image is first aligned based on two eyes locations and then is detected by Viola–Jones face detection algorithm which is widely used for face detection [26]. Idiap databases are divided into 3 subsets for training, development and testing. Identities for each subset were chosen randomly without any overlap. In this study, the proposed method is trained using data from the print, mobile and high-definition scenarios (coming from 15 subjects in training subset) and tested on grandtest dataset using samples from the all types of attacks (containing 35 subjects coming from development and testing subsets) as it is used in [13]. We extracted the live and fake face images from the corresponding videos. In particular, for each subject, we extracted 10 live face images and 10 spoofed face images collected in grandtest set. In the grandtest experiments, the protection method is trained using data from the print, mobile and high-definition scenarios, and tested on three types of attacks in test set. This is probably the most realistic attack case while we cannot know a priori the type of artifact (paper, mobile phone or tablet) that the attacker will use to try to enter into the system.

5.2.1 Texture-based protection systems

The first experiments analyze the results of the implementation of three types of spoof detection feature vectors such as LBP, DoG and HOG with and without PCA and LDA for dimensionality reduction. In Table 2, we show the results obtained on Print-Attack database by the proposed LBP, DoG and HOG texture-based methods using a standard classifier based on principal component analysis (PCA) and linear discriminant analysis (LDA). The results for LBP, DoG and HOG using PCA–LDA features give an HTER of 81.6, 50 and 8.9%, respectively. We have also conducted experiments at frame level. Similar experimental setup has been followed for Replay-Attack database, and the results are listed in Table 2. LBP \(+\) PCA–LDA and DoG \(+\) PCA–LDA features demonstrated an HTER of 8.8 and 43.5%, while HOG \(+\) PCA–LDA showed an HTER of 49.5%. In the case of Replay-Attack scenario, LBP \(+\) PCA–LDA has obtained the best classification accuracy with an HTER of 8.8%. Hence, it can be an appropriate protection approach to increase the security of biometric system.

Moreover, in order to test the effectiveness of our proposed system we conducted experiments on our constructed palmprint spoof database. In this case, attacker is assumed not to have previous knowledge about recognition algorithm and tries to access by only displaying printed palmprint photograph of the attacked person to the input camera. Furthermore, a printed palmprint image which is directly injected to the communication channel before the feature extraction step will most likely lack some of the properties found in natural images. As it is shown in Table 2, the results for LBP, DoG and HOG using PCA–LDA features give an HTER of 45.1, 16.9 and 11.1%, respectively.

In order to further indicate the potential significance of the proposed method, we have conducted a set of experiments on MSU face database. Similar experimental protocols have been followed as above, and the results are listed in Table 2. As it is recorded, LBP, DoG and HOG features followed by PCA–LDA give an HTER of 27.2, 50.5 and 21.7%, respectively, on the grandtest set. Once more, it is observed that the HOG \(+\) PCA–LDA pipeline obtains the best performance, in spite of the error rates which are considerably higher than those reported on the Print-Attack and palmprint spoof databases. The results of the HOG feature extractor with PCA–LDA for the cut photograph attacks on palmprint and Idiap Print-Attack and MSU face databases clearly show significantly better classification performance with the lower classification error rate.

5.2.2 Image quality assessment for fake biometric detection

The second set of experiments are performed on a multi-attack protection method using seven general image quality measures, which aims to evaluate different image quality assessment (IQA) metrics in order to overcome certain type of spoofs. IQA-based method is based on a single-image input. Hence, each frame of the videos in the Print-Attack and Replay-Attack databases is considered as an independent input sample. Therefore, classification of real or fake is done on a frame-by-frame basis and not per video. Our experiments on Print-Attack and Replay-Attack databases also show the strength of the employed image quality assessments metrics. As shown in Table 3, it is evident that the PSNR, SSIM and NAE methods have achieved better results rather than other methods. For both Print-Attack and Replay-Attack datasets, NAE produces the minimum error rate with HTER of 9% which is consistently selected as the best feature set for all the measured scenarios in the whole group of seven quality measures. We have repeated the same experiments on the constructed palmprint spoof and MSU face databases shown in Table 3, in order to test the widely used general image quality approaches showing performance for different applications. The lowest detection error rate for palmprint spoof and MSU face databases is obtained once we used MSE by recorded HTER of 7.1 and 27%, respectively. In addition, PSNR, SSIM and NAE have also reported good performances with HTER of 27.9, 23.6 and 27.9%, respectively, for palmprint spoof database. In the case of MSU face database, PSNR, SSIM and NAE have reported performances with HTER of 27.2, 28.7 and 33.2%, respectively.

Table 4 Results in HTER (%) obtained on the Print-Attack, Replay-Attack and palmprint spoof databases by the proposed protection method
Full size table

5.2.3 Proposed spoof detection method

In order to further improve the overall performance, the proposed protection system employs normalized absolute error (NAE) as the best performing feature subset, and histograms of oriented gradients (HOG) and local binary patterns (LBP) as the best local feature descriptors. A comparable result is obtained in comparison with the state-of-the-art systems as shown in Table 4. Our proposed anti-spoofing approach which utilizes the fusion of both texture-based method and image quality assessment (IQA) achieves an improvement with HTER of 5 and 1.2%, respectively, compared to the single-model systems considered for the texture-based methods and image quality assessment metrics on Print-Attack and Replay-Attack datasets. Compared to the results of texture-based and IQA-based algorithms on palmprint spoof and MSU face databases, the proposed fusion approach achieves a performance improvement as reported in Table 4, HOG \(+\) NAE with HTER of 5.8 and 5.8%, respectively.

On the other hand, in order to demonstrate the effectiveness of the proposed protection approach, a comparison is presented with the state-of-the-art methods. Similar experimental protocol with the protocol used in the state-of-the-art methods is followed, and the results are shown in Table 5 that are obtained by different texture-based detection methods on the face compared to the performance of our proposed method. It is observed that an HTER of 1.6% was recorded on the development set of the Replay-Attack dataset and an HTER of 1% on the test set. For the Print-Attack dataset, we recorded an HTER of 4.3 and 4.7% on development and test sets, respectively. The performance of our proposed system is significantly better than that of most of the proposed spoof detection systems. Furthermore, different LBP-based anti-spoofing approaches were tested following the same protocol used in the present study. A comparison between texture-based and IQA-based protection methods is also presented in Table 5 in which all results are reported on the grandtest scenario.

Table 5 Comparison results in HTER (%) on Replay-Attack and Print-Attack databases for different state-of-the-art methods
Full size table

6 Conclusion

This paper presents a novel protection method based on the fusion of texture-based methods and image quality assessment metrics. For this purpose, we considered three types of spoof detection feature vectors such as LBP, DoG and HOG with and without feature selection methods such as PCA and LDA. Additionally, feature space of seven complementary image quality measures is considered. We have combined texture-based algorithms and IQA metrics with a simple classifier to detect real accesses and fraudulent attacks. The proposed fusion protection scheme is able to be generalized for different databases and scenarios. It is able to be adopted to different types of attacks. It is also able to perform a high level of security for different biometrics traits. We also constructed a palmprint spoof database made by printed palmprint photograph using camera to evaluate the ability of different palmprint spoof detection algorithms. The experiments are conducted on face and palmprint spoof detection methods using three public-domain face spoof database (Idiap Print-Attack, Replay-Attack and MSU face) and constructed palmprint spoof database. The experimental results of the proposed scheme demonstrated considerable improvement in classification error rate compared to the state-of-the-art systems.