Abstract
Facial expression recognition has widely been investigated in the literature. The need of accurate facial alignment has however limited the deployment of facial expression systems in real-world applications. In this paper, a novel feature extraction method is proposed. It is based on extracting local binary patterns (LBP) from image key points. The face region is first segmented into six facial components (left eye, right eye, left eyebrow, right eyebrow, nose, and mouth). Then, local binary patterns are extracted only from the edge points of each facial component. Finally, the local binary pattern features are collected into a histogram and fed to an SVM classifier for facial expression recognition. Compared to the traditional LBP methodology extracting the features from all image pixels, our proposed approach extracts LBP features only from a set of points of face components, yielding in more compact and discriminative representations. Furthermore, our proposed approach does not require face alignment. Extensive experimental analysis on the commonly used JAFFE facial expression benchmark database showed very promising results, outperforming those of the traditional local binary pattern approach.
Chapter PDF
Similar content being viewed by others
References
Zhao, G., Pietikäinen, M.: Dynamic Texture Recognition Using Local Binary Patterns with An Application to Facial Expressions. IEEE Trans. Pattern Anal. Mach. Intell. 29(6), 915–928 (2007)
Huang, X., Zhao, G., Zheng, W., Pietikäinen, M.: Spatiotemporal Local Monogenic Binary Patterns for Facial Expression Recognition. IEEE Signal Processing Letters 19(5), 243–246 (2012)
Feng, X., Pietikäinen, M., Hadid, A.: Facial Expression Recognition with Local Binary Patterns and Linear Programming. Pattern Recognition and Image Analysis 15(2), 546–548 (2005)
He, L., Zou, C., Zhao, L., Hu, D.: An enhanced LBP feature based on facial expression recognition. In: Proc. Ann. Int. Conf. Eng. Med. Biol. Soc., pp. 3300–3303 (2005)
Liao, S., Fan, W., Chung, A.C.S., Yeung, D.Y.: Facial expression recognition using advanced local binary patterns, tsallis entropies and global appearance features. In: Proc. IEEE Int. Conf. Image Process., pp. 665–668 (October 2006)
Feng, X., Lv, B., Li, Z., Zhang, J.: A novel feature extraction method for facial expression recognition. In: Proc. Joint Conf. Inform. Sci. Issue Adv. Intell. Syst. Res., Kaohsiung, Taiwan, pp. 371–375 (2006)
Cao, J., Tong, C.: Facial expression recognition based on LBP-EHMM. In: Proc. Congr. Image Signal Process. (2008)
Fu, X., Wei, W.: Centralized binary patterns embedded with image Euclidean distance for facial expression recognition. In: Proc. Int. Conf. Neural Comput., pp. IV: 115–119 (October 2008)
Ahmed, F., Hossain, E., Bari, A., Shihavuddin, A.: Compound Local Binary Pattern (CLBP) for Robust Facial Expression Recognition. In: International Conference of Soft Computing and Pattern Recognition (SoCPaR), pp. 391–395 (2011)
Lyons, M., Akamastu, S., Kamachi, M., Gyoba, J.: Coding facial expressions with Gabor wavelets. In: Proc. Third IEEE Conf. Face and Gesture Recognition, Nara, Japan, pp. 200–205 (April 1998)
Nilsson, M., Nordberg, J., Claesson, I.: Face Detection Using Local SMQT Features and Split up SNOW Classifiers. In: IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP (2007)
Valstar, M.F., Pantic, M.: Biologically vs. logic inspired encoding of facial actions and emotions. In: Proc. of IEEE Intl. Conf. on Multimedia and Expo (ICME), pp. 325–328 (2006)
Bartlett, M.S., Littlewort, G., Frank, M., Lainscsek, C., Fasel, I., Movellan, J.: Recognizing facial expression: machine learning and application to spontaneous behavior. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR (2005)
Shan, C., Gong, S., McOwan, P.W.: Facial expression recognition based on Local Binary Patterns: A comprehensive study. Image and Vision Computing 27, 27803–27816 (2009)
David, D.G.: Lowe, Distinctive Image Features from Scale-Invariant Points. International Journal of Computer Vision 60(2), 91–110 (2004)
Ojala, T., Pietikäinen, M., Mäenpää, T.: Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Transactions on Pattern Analysis and Machine Intelligence 24(7), 971–987 (2002)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Feng, X., Lai, Y., Mao, X., Peng, J., Jiang, X., Hadid, A. (2013). Extracting Local Binary Patterns from Image Key Points: Application to Automatic Facial Expression Recognition. In: Kämäräinen, JK., Koskela, M. (eds) Image Analysis. SCIA 2013. Lecture Notes in Computer Science, vol 7944. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-38886-6_33
Download citation
DOI: https://doi.org/10.1007/978-3-642-38886-6_33
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-38885-9
Online ISBN: 978-3-642-38886-6
eBook Packages: Computer ScienceComputer Science (R0)