Abstract
Face extraction is considered a very important step in developing a recognition system. It is a challenging task as there are different face expressions, rotations, and artifacts including glasses and hats. In this paper, a face extraction model is proposed for thermal IR human face images based on superpixel technique. Superpixels can improve the computational efficiency of algorithms as it reduces hundreds of thousands of pixels to at most a few thousand superpixels. Superpixels in this paper are formulated using the quick-shift method. The Quick-Shift’s superpixels and automatic thresholding using a simple Otsu’s thresholding help to produce good results of extracting faces from the thermal images. To evaluate our approach, 18 persons with 22,784 thermal images were used from the Terravic Facial IR Database. The Experimental results showed that the proposed model was robust against image illumination, face rotations, and different artifacts in many cases compared to the most related work.
Access provided by Autonomous University of Puebla. Download conference paper PDF
Similar content being viewed by others
Keywords
1 Introduction
There is a great volume of research efforts in the area of the face recognition using the visible spectrum mechanism. However, the visible spectrum-based face recognition suffers from the problem of the variations of light [1]. To address this problem, 3D face recognition [2] or a combination between visible and Infrared (IR) spectrum have been suggested [3]. There is always a need for more robust security systems which does not affect by variations of light. As the IR spectrum is not affected by variations of light, this have increased the rise to develop face recognition system based only on the infrared spectrum.
It is reported that IR spectrum could offer a promising alternative face recognition systems to visible spectrum specifically in case of variations in the face appearance causing by illumination changes [4, 5]. In particular, Jain et al. [6] reported that IR spectrum provides an ability for human identification under different lighting conditions even in the total darkness. Wolff et al. [7] have concluded that IR spectrum is nearly invariant to any change in ambient illumination. Thus, IR-based human recognition systems have the potential to offer simpler and yet robust solutions which achieve a good performance in uncontrolled environment.
Segmentation is an important step in recognition systems. Recognition rates of most recognition approaches can be improved by a good segmentation technique as it enables the utilization of the face shape in the recognition process [8, 9]. There are limited studies about the segmentation approaches for thermal face image. Here, we will give an overview about these studies.
Aglika et al. [10] proposed a segmentation approach using an elliptical mask to be put over the face image to remove the background, align and scale the faces. However, this approach is applicable only for frontal and centered faces. Pavlidis et. al. [11] suggested a face segmentation method based on Bayesian approach. This method is based on both of the models of skin and the background pixel intensities. Thus, clothes pixels was included as skin pixels while ignoring other skin pixels and considering them as a background. In another study, Cho et al. in [12] proposed a segmentation method for the IR face images using contours and morphological operations. The Sobel edge detector was used for the edge detection then the morphological operations was applied to the contour to connect open contours and remove small areas. Recently, Filipe et al. [13] proposed two segmentation methods which make use of the active contour approaches and the statistical modeling of pixel intensities. The two methods are robust against face pose, expression, and rotation. In addition, they addressed the problem of considering the clothes as part of the face, thus enabling the segmentation of the face shape to be used recognition methods.
Superpixels can improve the computational efficiency of algorithms as it reduces hundreds of thousands of pixels to at most a few thousand superpixels. Algorithms for generating superpixels can be categorized as either graph based [14–17] or gradient-ascent based [18–22]. Quick-shift is a common image segmentation method as a gradient-ascent based method [18]. The quick-shift’s superpixels are not fixed in size or number and preserve most of the boundaries in the original image. The quick-shift parameters are usually determined by segmenting a few training images. Generating superpixels by quick shift are controlled by three parameters of Ratio, Kernel Size, and Distance.
In this paper, a face extraction model is proposed based on superpixel technique for thermal IR human face images. Superpixels formation using quick-shift helps to get more accurate face extractions. The Quick-Shift parameters’ values and automatic thresholding, using a simple Otsu’s thresholding, help to produce good results of extracting faces from the thermal images. The Terravic Facial IR Database is used to evaluate our approach. The Experimental results showed that the proposed model was robust against image illumination, face rotations, and different artifacts. Comparing to the most related work, our model was found better in many cases.
2 Theoretical Background
2.1 Quick-Shift Method
The quick-shift method [18] is used to extract superpixels from the thermal face image. The superpixels, in this method, depend on three different parameters of ratio, kernel size, and maximum distance. Determining the quick-shift parameters successfully makes the resulted image more meaningful and easier to be used to extract the thermal face superpixels. In this paper, the three parameters’ values are determined by segmenting a few training images by hand until we find a set that shows a good segmentation result for nearly all of the face boundaries and had the largest possible average segment size. In practice, the quick-shift algorithm is not too much sensitive to the choice of parameters, thus a quick tuning by hand is somewhat sufficient for thermal face extraction.
In summary, the superpixels extracted by quick-shift depends on the following parameters:
-
Ratio, Ratio: It is a tradeoff between spatial and intensity consistency.
-
Kernel size, KernalSize: It is the parameter that controls the scale at which the density is estimated.
-
Max-distance, MaxDist: It is the distance between two pixels that the method considers when building the tree.
2.2 Otsu’s Thresholding Method
Converting a greyscale image to a binary image is a common task in image processing. Otsu’s segmentation method [23] is usually used to automatically perform clustering-based image thresholding [24]. This method converts a grayscale image to binary image. The algorithm assumes that the image contains two classes of pixels (foreground and background pixels). Thresholding tries all possible threshold values to separate the pixels that either fall in foreground or background. The optimum threshold value minimizes the sum of foreground and background spreads.
3 Proposed Thermal Face Extraction Model
A model is proposed to extract human faces based on their thermal images. The model makes use of the Quick-Shift algorithm to produce superpixels and the Otsu’s method for automatic thresholding. The proposed model steps are as shown in the Algorithm 1. This model works as follows.
Firstly, a thermal face image \(I_i\) is selected, where \(I_i\) represents the ith input image of the total number of images N in this group for \(i=1,2,3, \ldots ,N\). The Terravic Facial IR Database is used for the proposed model. The Quick-Shift method is applied with its initial parameters, ratio, kernel size, and maximum distance, to produce superpixels. The Otsu’s thresholding method is then applied to the produced superpixels image. Thus, each superpixels image is converted to a binary image \(B_i\) based on the optimum threshold. Finally, the relevant pixel values from the original thermal image are extracted. The Quick-Shift parameters’ values with the automatic thresholding one can get the best results of extracting faces from the thermal images.
4 Experimental Results and Discussion
To evaluate our approach, the Terravic facial IR database [25] was used. This database consists of 20 persons each of one of them has a different number of images with various variations (front, left, right; indoor/outdoor; glasses, hat). Its images’ format is 8-bit grayscale JPEG with the size of \(320 \times 240\) pixels. 18 persons with 22,784 thermal images were used from this database in our experiments. Table 1 shows the distribution of the images for each class.
Two main scenarios were designed to evaluate our proposed model:
-
The first scenario was to check the accuracy of extracting thermal faces using only Otsu’s automatic threshold.
-
The second scenario was to evaluate the accuracy of extracting thermal faces using Quick-Shift based automatic threshold.
To show the evaluation of the two scenarios, we use classes (01), (12), and (16) which represents various poses and variations required for face recantation. All of these classes contain different poses (front, left, right). The class of (1) contains indoor images while (12) and (16) includes outdoor images. For the glasses and hat poses, class (1) contains glasses whereas (12) and (16) include glasses and hat.
Figure 1 shows the results for class ‘face16’ for the two scenarios. This class of the thermal images were captured outdoor from front, left, right direction with glasses, hat, and both. From this figure, its clear that both methods (Otsu’s and Quick-Shift) achieve good results for this class because there is clear different in the brightness between the face area and the object of clothes, hate, glass, and other surroundings.
On the other side, Fig. 2 shows the results of the class ‘face01’ where both methods (i.e. scenarios) are totally different. As the thermal images of class ‘face01’ were captured indoor with different front, left, right direction with glasses, the Otsu’s method did not succeed to extract the face area because of the brightness of the clothes.
For images of class ‘face12’, which were captured outdoor from front, left, right direction and containing glasses, hat, both the Quick-Shift accomplished some good results and other not good as shown in Fig. 3. The good or the bad the results were noticed that they depend on the face direction.
To show the effectiveness of the proposed method, a face segmentation approach based on active counters [26] and suggested by Filipe et al. [13] was implemented and its results were compared with our model’s results. This comparison was conducted with the same database (the Terravic facial IR database) and its results are illustrated in Fig. 4. From this figure, it can be noticed that the proposed method was more robust than Filipe’s approach. Although both methods could effectively extract human faces when the intensity clothes are high, still our method showed results better than Filipe’s one.
From the above results, it can be concluded that the proposed method can successfully extract face from thermal images which were taken indoor/outdoor under various variations, e.g. different directions (left, right, front), glasses, clothes, and hat. On the other side, with high-intensity clothes in the images, the model needs refinement.
5 Conclusion
In this paper, we proposed a face extraction model from IR human face images. This model made use of Otsu and Quick-Shift methods. Based on an extensive experimental results using 18 persons with 22,784 thermal images from the Terravic Facial IR Database, it was concluded that Quick-shift can improve the face extraction results. Our model achieved excellent results for extracting faces from thermal images which were taken under various variations, e.g. different directions, glasses, clothes, and hat. Comparing with the related work, our model’s results were better in all different variations. As for the future work, we plan to (a) make a refinement for the case where there is high intensity in clothes in the images and (2) explore the effectiveness of the proposed model for object detection and extracted in different types of thermal databases such as Terravic Motion IR Database, Terravic Weapon IR Database, and Thermal Infrared Video Benchmark for Visual Analysis.
References
Ross, A., Nandakumar, K., Jain, A.: Handbook of Multibiometrics (International Series on Biometrics). Springer, New York (2006)
Quy, N.H. et al.: 3D human face recognition using sift descriptors of face’s feature regions. In: New Trends in Computational Collective Intelligence. Springer International Publishing, pp. 117–126 (2015)
Akhloufi, M., Bendada, A., Batsale, J.: State of the art in infrared face recognition. Quant. Infrared Thermogr. J. 5(1), 3–26 (2008)
Ramaiah, N.P., Ijjina, E.P., Mohan, C.K.,: Illumination invariant face recognition using convolutional neural networks. In: IEEE International Conference on Signal Processing, Informatics, Communication and Energy Systems (SPICES), 2015, pp. 1–4 (2015)
Chen, C.-L., Jian, B.-L.: Infrared thermal facial image sequence registration analysis and verification. Infrared Phys. Technol. 69, 1–6 (2015)
Jain, A., Bolle, R., Pankanti, S.: Biometrics: Personal Identification in Networked Society. Kluwer Academic Publishers, London (1999)
Wolff, L., Socolinsky, D., Eveland, C.: Quantitative measurement of illumination invariance for face recognition using thermal infrared imagery. In: IEEE Workshop on Computer Vision Beyond the Visible Spectrum: Methods and Applications, Hawaii (2001)
Pantofaru, C.: Studies in using image segmentation to improve object recognition. Ph.D. thesis, Robotics Institute, Carnegie Mellon University, Pittsburgh (2008)
Segundo, M.P., Silva, L., Bellon, O.R.P., Queirolo, C.C.: Automatic face segmentation and facial landmark detection in range images. IEEE Trans. Syst. Man Cybern. Part B Cybern. 40(5), 1319–1330 (2010)
Gyaourova, A., Bebis, G., Pavlidis, L.: Fusion of infrared and visibleimages for face recognition. In: Computer Vision-ECCV 2004, pp. 456–468. Springer, Berlin(2004)
Pavlidis, I., Tsiamyrtzis, P., Manohar, C., Buddharaju, P.: Biometrics: face recognition in thermal infrared. In: Biomedical Engineering Handbook, 3rd edn., Chap. 29, pp. 1–15. CRC Press, Boca Raton
Cho, S., Wang, L., Ong, W.: Thermal imprint feature analysis for face recognition. IEEE Int. Symp. Ind. Electron, pp. 1875–1880 (2009)
Filipe, S., Alexandre, L.A.: Algorithms for invariant long-wave infrared face segmentation: evaluation and comparison. Pattern Anal. Appl. 17(4), 823–837 (2014)
Ren, X., Malik, J.: Learning a classification model for segmentation. IEEE Proc. ICCV 1, 10–17 (2003)
Mori, G., Ren, X., Efros, A., Malik, J.: Recovering human body configurations: combining segmentation and recognition. IEEE Proc. CVPR 2, 326–333 (2004)
Felzenszwalb, P., Huttenlocher, D.: Efficient graph-based image segmentation. Int. J. Comput. Vis. 59(2), 167–181 (2004)
Moore, A., Prince, S., Warrell, J., Mohammed, U., Jones, G.: Superpixel Lattices. IEEE Proc. CVPR, pp. 1–8 (2008)
Vedaldi, A., Soatto, S.: Quick shift and kernel methods for mode seeking. In:Proceedings of the European Conference on Computer Vision (ECCV) (2008)
Levinshtein, A., Stere, A., Kutulakos, K., Fleet, D., Dickinson, S., Siddiqi, K.: Turbopixels: fast superpixels using geometric flows. IEEE Trans. Pattern Anal. Mach. Intell. 31(12), pp. 2290–2297 (2009)
Arbelaez, P., Maire, M., Fowlkes, C., Malik, J.: From contours to regions: an empirical evaluation. IEEE Proc. CVPR, pp. 2294–2301 (2009)
Achanta, R., Shaji, A., Smith, K., Lucchi, A., Fua, P., Ssstrunk, S.: SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Trans. Pattern Anal. Mach. Intell. 34(11), 2274–2282 (2012)
Ren, C.Y., Reid, I.: gSLIC: a real-time implementation of SLIC superpixel segmentation. Department of Engineering Science, University of Oxford (2011)
Otsu, N.: A threshold selection method from gray-level histograms. IEEE Trans. Syst. Man Cybern. 9(1), 62–66 (1979)
Sezgin, M., Sankur, B.: Survey over image thresholding techniques and quantitative performance evaluation. J. Electron. Imag. 13(1), 146–165 (2004)
IEEE OTCBVS WS Series Bench; Roland Miezianko, Terravic Research Infrared Database
Chan, T.F., Vese, L.A.: Active contours without edges. IEEE Trans. Image Process. 10(2), 266–277 (2001)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this paper
Cite this paper
Ibrahim, A., Gaber, T., Horiuchi, T., Snasel, V., Hassanien, A.E. (2016). Human Thermal Face Extraction Based on SuperPixel Technique. In: Gaber, T., Hassanien, A., El-Bendary, N., Dey, N. (eds) The 1st International Conference on Advanced Intelligent System and Informatics (AISI2015), November 28-30, 2015, Beni Suef, Egypt. Advances in Intelligent Systems and Computing, vol 407. Springer, Cham. https://doi.org/10.1007/978-3-319-26690-9_15
Download citation
DOI: https://doi.org/10.1007/978-3-319-26690-9_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-26688-6
Online ISBN: 978-3-319-26690-9
eBook Packages: Computer ScienceComputer Science (R0)