Abstract
Traditionally, facial expression recognition (FER) issues have been studied mostly based on modalities of 2D images, 2D videos, and 3D static models. In this paper, we propose a spatio-temporal expression analysis approach based on a new modality, 3D dynamic geometric facial model sequences, to tackle the FER problems. Our approach integrates a 3D facial surface descriptor and Hidden Markov Models (HMM) to recognize facial expressions. To study the dynamics of 3D dynamic models for FER, we investigated three types of HMMs: temporal 1D-HMM, pseudo 2D-HMM (a combination of a spatial HMM and a temporal HMM), and real 2D-HMM. We also created a new dynamic 3D facial expression database for the research community. The results show that our approach achieves a 90.44% person-independent recognition rate for distinguishing six prototypic facial expressions. The advantage of our method is demonstrated as compared to methods based on 2D texture images, 2D/3D Motion Units, and 3D static range models. Further experimental evaluations also verify the benefits of our approach with respect to partial facial surface occlusion, expression intensity changes, and 3D model resolution variations.
Chapter PDF
Similar content being viewed by others
Keywords
- Facial Expression
- Linear Discriminative Analysis
- Recognition Rate
- Facial Expression Recognition
- Facial Surface
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Ekman, P., Friesen, W.: The Facial Action Coding System. Consulting Psychologists Press, San Francisco (1978)
Tong, Y., Liao, W., Ji, Q.: Facial action unit recognition by exploiting their dynamic and semantic relationships. IEEE Trans. on PAMI 10, 1683–1699 (2007)
Pantic, M., Rothkrantz, L.: Automatic analysis of facial expressions: the state of the art. IEEE Trans. PAMI (2000)
Yang, P., Liu, Q., Metaxas, D.: Boosting coded dynamic features for facial action units and facial expression recognition. In: CVPR 2007 (2007)
Bartlett, M., et al.: Fully automatic facial action recognition in spontaneous behavior. In: FGR 2006, pp. 223–228 (2006)
Donato, G., Bartlett, M., Hager, J., Ekman, P., Sejnowski, T.: Classifying facial actions. IEEE Trans. PAMI 21(10), 974–989 (1999)
Lien, J., et al.: Subtly different facial expression recognition and expression intensity estimation. In: CVPR 1998 (1998)
Pantic, M., Patras, I.: Detecting facial actions and their temporal segments in nearly frontal-view face image sequences. In: IEEE Int’l Conf. on Systems, Man and Cybernetics 2005, pp. 3358–3363 (2005)
Cohen, I., Sebe, N., Garg, A., Chen, L., Huang, T.: Facial expression recognition from video sequences: temporal and static modeling. Journal of CVIU 91 (2003)
Sebe, N., et al.: Authentic facial expression analysis. Image Vision Computing 12, 1856–1863 (2007)
Zeng, Z., et al.: Spontaneous emotional facial expression detection. Journal of Multimedia 5, 1–8 (2006)
Chang, Y., Hu, C., Turk, M.: Probabilistic expression analysis on manifolds. In: IEEE Inter. Conf. on CVPR 2004 (2004)
Zhao, G., Pietikainen, M.: Dynamic texture recognition using local binary patterns with an application to facial expressions. IEEE Trans. on PAMI (2007)
Lyons, M., et al.: Automatic classification of single facial images. IEEE Trans. PAMI 21, 1357–1362 (1999)
Zalewski, L., Gong, S.: Synthesis and recognition of facial expressions in virtual 3d views. In: FGR 2004 (2004)
Wang, J., Yin, L., Wei, X., Sun, Y.: 3d facial expression recognition based on prmitive surface feature distribution. In: IEEE CVPR (2006)
Wang, P., Verma, R., et al.: Quantifying facial expression abnormality in schizophrenia by combining 2d and 3d features. In: CVPR 2007 (2007)
Wang, Y., Huang, X., Lee, C., et al.: High resolution acquisition, learning and transfer of dynamic 3d facial expressions. In: EUROGRAPHICS 2004 (2004)
Di3D, I. (2006), http://www.di3d.com
Chang, Y., Vieira, M., Turk, M., Velho, L.: Automatic 3d facial expression analysis in videos. In: IEEE ICCV 2005 Workshop on Analysis and Modeling of Faces and Gestures (2005)
Yin, L., Wei, X., Sun, Y., Wang, J., Rosato, M.: A 3d facial expression database for facial behavior research. In: IEEE FGR 2006 (2006)
Yeasin, M., et al.: From facial expression to level of interest: a spatio-temporal approach. In: CVPR 2004 (2004)
Phillips, P., Flynn, P., et al.: Overview of the face recognition grand challenge. In: CVPR 2005 (2005)
Yin, L., Chen, X., Sun, Y., Worm, T., Reale, M.: A high resolution 3d dynamic facial expression database. In: IEEE FGR 2008 (2008)
Sun, Y., Yin, L.: 3d face recognition using two views face modeling and labeling. In: IEEE CVPR 2005 Workshop on A3DISS (2005)
Cootes, T., Edwards, G., Taylor, C.: Active appearance models. IEEE Trans. PAMI 23 (2001)
Rabiner, L.: A tutorial on hidden markov models and selected applications in speech recognition. Proceedings of IEEE, 77(2) (1989)
Othman, H., Aoulnasr, T.: A separable low complex 2d hmm with application to face recognition. IEEE PAMI 25 (2003)
Saul, L., Roweis, S.: Think globally, fit locally: Unsupervised learning of low dimensional manifolds. Journal of Machine Learning Research 4, 119–155 (2003)
Martinez, A., Kak, A.: Pca versus lda. IEEE Trans. on PAMI 23, 228–233 (2003)
Wallraven, C., et al.: Psychophysical evaluation of animated facial expressions. In: Proc. of the 2nd Symposium on Applied Perception in Graphics and Visualization (2005)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Sun, Y., Yin, L. (2008). Facial Expression Recognition Based on 3D Dynamic Range Model Sequences. In: Forsyth, D., Torr, P., Zisserman, A. (eds) Computer Vision – ECCV 2008. ECCV 2008. Lecture Notes in Computer Science, vol 5303. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-88688-4_5
Download citation
DOI: https://doi.org/10.1007/978-3-540-88688-4_5
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-88685-3
Online ISBN: 978-3-540-88688-4
eBook Packages: Computer ScienceComputer Science (R0)