Keywords

1 Introduction

Biometric identification techniques can be done by two ways, either using physical characteristics or using behavior characteristics. Identification techniques that are based on physical characteristics are more difficult to counterfeit than behavior methods. Physical characteristic identification includes face recognition, voice recognition, vein recognition, fingerprint recognition, and iris recognition. Face recognition has a benefit that it does not require direct physical interaction with machine. Face recognition can be done in visual and thermal domain; each one has its advantages. Currently, most researchers are tending to use the thermal domain as thermal face characteristics are unique for a person [1] and represent a heat pattern emitted by an object.

Thermal mid-wave infrared (MWIR) is one of the partitions of electromagnetic (EM) spectrum that can fix problems like light illumination that may appear in visual domain. Also, any fake object present over face could be detected easily as they have different emissivity than human face. Socolinsky et al. [2, 3] suggested that thermal images of human faces can be a valid biometric and is superior as compared to visual images. Moreover, a thermal domain technique is not affected by scattering and absorption of smoke or dust and gives satisfactory result in complete darkness as well.

There are appearance-based techniques mostly used in thermal face recognition like principal component analysis (PCA) [4], linear discriminant analysis (LDA) [4], Kernel component analysis [5], local binary pattern [6], etc. Utilizing anatomical information of human face one can extract vascular information and can be used in recognition [7]. However, these techniques face problems in thermal domain due to amorphous nature of images and lack of sharp boundaries which makes object boundary extraction challenging [8]. It is observed that if thermal face image background consists of different objects, the recognition becomes difficult. The thermal pattern of background objects affects the performance. In this work a face localized scale-invariant feature transform (FLSIFT) based on scale-invariant feature transform (SIFT) [9] is proposed to address this issue. SIFT is a very robust feature descriptor for object recognition and matching [9].

The main contribution of this work includes development a new Indian thermal face database (ITFDB), evaluation of existing blood vessel pattern (BVP) technique [7] on ITFDB and development of FLSIFT face recognition system. The Sect. 2 briefly describes about thermal face recognition using BVP. Section 3 elaborates detailed setup about ITFDB creation. The proposed thermal face recognition using FLSIFT is detailed in Sect. 4. The experimental results are discussed in Sect. 5.

2 A Brief on Thermal Face Recognition Using BVP

This section briefly discusses the thermal face recognition system using BVP [7]. In this system, a thermal pattern is analyzed for the recognition obtained by superficial blood vessels pattern present over the bones and muscles. A typical thermal pattern is caused due to temperature gradient of warm blood flowing through the superficial vessels against the surrounding tissues. By knowing the thermal characteristics; pattern of blood vessels can be extracted. The implementation is done in four steps as shown in Fig. 1. In the first step, a reference image of a subject is used to do registration of three other images of the same subject. In second step, the face region is extracted by region growing segmentation algorithm with predefined seed points. For enhancing edge information, a standard Persona-Malik anisotropic diffusion filter [10] is applied over all images.

Fig. 1
figure 1

Thermal face recognition using BVP

Full size image

Extraction of enhanced edges is required for thermal pattern generation. White top-hat segmentation is used for this purpose. This gives an individual signature pattern for a subject as shown in Fig. 2. For each subject, four signatures are created using four different images. This signature is used to match with signatures of other images using distance-based similarity metric, such as Euclidean distance and Manhattan distance [7, 11]. Figure 3 shows the overlay of signature of a subject signature with signature of itself and with other subject.

Fig. 2
figure 2

a Thermal image b Extracted signature

Full size image
Fig. 3
figure 3

Overlay of signature over a same subject b Different subject

Full size image

3 Development of a New Indian Thermal Face Database

To evaluate performance of thermal face recognition systems in the Indian conditions, a need of an Indian thermal face dataset with complex background was felt. This work proposes ITFDB consisting of thermal face images of 113 subjects. The database is created using TESTO-875-1i thermal camera equipped with an uncooled detector with a spectral sensitivity range from 8 to 14 µm and provided with a standard optical lens. The thermographic camera provides a resolution of 160 × 120 pixels for thermal.

The dataset was generated at a room temperature. For each subject, four frontal views were taken. Specific arrangement was maintained for the creation of dataset; camera was mounted on tripod stand, a chair was fixed at a distance of 1 m from tripod stand. Each subject was asked to seat straight in front of thermal camera, looking straight into the lens and snapshots were captured, as detailed in [7]. Database is available online at [12]. Other database details are as follows:

Figure 4 shows sample images from ITFDB. The frontal face images were captured with simple background, i.e. wall and with complex background; which contains glass, wood, iron material, etc. As the different material has different emissivity and reflection it creates different temperature gradients in thermal image which can be clearly observed (Table 1).

Fig. 4
figure 4

Dataset Samples a simple background, b and c complex background

Full size image
Table 1 Dataset details
Full size table

It may be noted that I2BVSD dataset with occluded thermal faces without complex background is proposed in [13, 14].

4 Thermal Face Recognition Using FLSIFT

In this work, a thermal face recognition system using face localized SIFT is proposed. The detail flow of the system is given in Fig. 5. Proposed system consists of five steps for the recognition. These include (1) face localization, (2) key point extraction, (3) descriptor Extraction, (4) descriptor matching, and (5) decision-making based on the maximum matching score of descriptors. The details of the proposed system are depicted in Fig. 5.

Fig. 5
figure 5

Thermal face recognition using FLSIFT

Full size image

4.1 Face Localization

The first step, localizes face in the given thermal image is done. For a robust system, this is a crucial stage so that the disturbances due to background are minimized. The key points of these localized face regions are extracted using SIFT [9]. These robust features are invariant to affine transform, rotation, scale, and having distinctive features which are highly required in recognition was invented by Lowe [9]. To extract the face region, the thermal image is first binaries and using connected component labeling the larger connected area having same labeling is cropped and extracted as a face [15] the result is as shown in Fig. 6b.

Fig. 6
figure 6

a Original thermal image b Extracted face

Full size image

4.2 FLSIFT Feature Extraction

After face localization, the features of faces are extracted using SIFT [9]. Extraction of feature using SIFTS have three steps: (1) Scale space key point selection, (2) Key point localization, (3) Orientation assignment. Scale space key point is selected, where local maxima and minima of difference-of-Gaussian function in the scale space is present. The convolution result of variable Gaussian function with the image gives scale space of an image. If G (x, y, σ) is a variable Gaussian function and I (x, y) is input image then scale space function L (x, y, σ) is—

$$ L\left( {x,y,\sigma } \right) = G\left( {x,y,\sigma } \right)*I\left( {x,y} \right) $$
(1)

With

$$ G\left( {x,y,\sigma } \right) = \frac{1}{{2\pi \sigma^{2} }}e^{{ - \left( {x^{2} + y^{2} } \right)/2\sigma^{2} }} $$
(2)

The difference-of-Gaussian functions is derived as follows –

$$ D\left( {x,y,\sigma } \right) = L\left( {x,y,\sigma } \right) - L\left( {x,y,k\sigma } \right) $$
(3)

with two nearby scales separated by a multiplicative factor (k).

At this stage many key points are obtained which either present in all scale space or in some of them. Final key points are selected which are present in all scale space, and a detail model created for location and scale determination. For invariance to rotation every key point assigned gradient orientation by the gradient magnitude. The Eqs. (4) and (5) give the detail information about the gradient magnitude and gradient orientation.

$$ m\left( {x,y} \right) = \sqrt {\left( {L\left( {x + 1,y} \right) - L\left( {x - 1,y} \right)} \right)^{2} + \left( {L\left( {x,y + 1} \right) - L\left( {x,y - 1} \right)} \right)^{2} } $$
(4)
$$ \theta \left( {x,y} \right) = \tan^{ - 1} \left( {\left( {L\left( {x,y + 1} \right) - L\left( {x,y - 1} \right)} \right)/\left( {L\left( {x + 1,y} \right) - L\left( {x - 1,y} \right)} \right)} \right) $$
(5)

4.3 FLSIFT Descriptor

The computation of descriptor converts these key points into vector which is used further as a feature vector. A rectangular area of (16 × 16 pixels) is centered on each key point; and then it is divided into 4 × 4 subregions which is characterized by 8-bin orientation histogram [9]. A 128 element descriptor vector is created using 8-bin orientation over 16 subregions.

In the proposed approach, the system localizes face in the thermal image and then computes descriptors for the face localized region. We call these descriptors as FLSIFT descriptors. The extracted descriptors for the facial images of all the subjects are stored and used for matching.

4.4 FLSIFT Descriptor Matching

The nearest neighbor distance metric is used for matching. Match is declared if and only if the Euclidean distance between the closed key point descriptor is less than 0.6 times the next closest key point descriptor. Figure 7 shows the FLSIFT feature matching between same subject and different subject.

Fig. 7
figure 7

FL-SIFT Feature Matching between same subject and different subject

Full size image

5 Experimental Results and Discussion

The experiments are performed with following objectives, (1) To evaluate importance of FLSIFT over SIFT, (2) To compare results of proposed system with BVP [7] technique.

5.1 Recognition on ITFDB

From ITFDB dataset Images of randomly chosen 57 subjects for the training set and the images from the remaining 54 subjects are used for testing. The training set thus contains 109 images similarly; the testing set consists of 108 images. Further the testing set is divided into two parts gallery and probe.

For each subject, one complex background image is taken as gallery and one complex background image, and one simple background images are constitute the query set. For evaluating recognition experiment, the combination of train set and query set are tested as considering gallery set as trained set.

Using this configuration experimental results are reported in terms of receiver operating characteristics (ROC) curves. This experiment is carried out on ITFDB dataset mentioned above. Figure 8 shows the performance of proposed approach compared with SIFT operator on thermal face images from ITFDB. The ROC curves indicate that thermal image recognition does not get affected much due to background information in FLSIFT.

Fig. 8
figure 8

ROC curve on ITFDB dataset

Full size image

5.2 Identification

This experimentation deals with problem of face identification. For identification comparison of the proposed FLSIFT and BVP [7] technique is done on ITFDB dataset. The dataset consists of images with simple background as well as complex background. The comparative results on both the background are as follows:

Simple background.

In the proposed ITFDB dataset, there are 452 (113 × 4) images of 113 subjects with simple background, i.e. four images per subject. These images are divided into four subsets as, S-1, S-2, S-3, and S-4. To test the performance of FLSIFT and BVP technique [7], one subset is used for training (consisting of 113 images) and remaining sets are used for testing. The obtained results are as shown in Table 2.

Table 2 Comparison over Different Training Set
Full size table

Table 3 demonstrates the average identification accuracy for both the systems on the subsets S1-S4. The proposed FLSIFT gives 100 % recognition accuracy on both training and test sets. Whereas, recognition accuracy of BVP technique on test dataset is 79.28 and 100 % on trained dataset with signature.

Table 3 Comparison Result of Average Accuracy of Recognition
Full size table

By adding the four signature of same subject one unique template for individual subject is created. When all signature were tested using BVP considering this template as a training set we got 86.9 % recognition accuracy. It may be noted that with formation of templates for each face using four signatures of the same subject, performance of BVP technique is found to be improved as compared to its performance on signature of faces.

Complex background.

Thermal images also have temperature gradient because of background. As discussed in Sect. 3, the background consists of different objects having different emissivity and reflection coefficients results in a disturbed thermal characteristic in the image. Experimental results show that FLSIFT method is sustainable with this type of complex background as well. The following Table 4. Shows the comparative results of FLSIFT, SIFT [9], and BVP systems.

Table 4 Comparison Result of Accuracy of Recognition
Full size table

The training set consists of 162 (54 × 3) images of 54 subjects with a combination of two images of simple background and one image from complex background per subject. During testing of the systems, a set of another 54 images with complex background is used.

Table 4 describes the detail comparison of average accuracy of the complex background for both the methods. It is observed that BVP technique performed satisfactorily on training dataset, however its performance on complex dataset is very poor. Note that proposed recognition rate of FLSIFT is 98.14 % even on complex background. To ensure the effect of face localization on performance, results without face localization are also reported in Table 4. This clearly shows importance of face localization.

6 Conclusion

This work evaluates the performance of existing thermal face recognition BVP technique. It is observed from experimental results that BVP performs poorly on complex thermal background. A dataset for thermal facial images is developed in Indian environmental conditions. To overcome the difficulty in face recognition in complex thermal background, a novel FLSIFT method is proposed. Its performance is compared empirically with BPV on simple and complex background. It may be concluded from this work that Human Thermal Face recognition is a good biometric for identification and proposed FLSIFT is found to be a suitable for the task.