Abstract
The way a person is moving his/her head and facial parts (such as the movements of the mouth when a person is talking) defines so called facial dynamics and characterizes personal behaviors. An emerging direction in automatic face analysis consists of also using such dynamic cues, in addition to facial structure, in order to enhance the performance of static image-based methods. This is inspired by psychophysical and neural studies indicating that behavioral characteristics do also provide valuable information to face analysis in the human visual system. This survey article presents the motivations, reviews the recent developments and discusses several other important issues related to the use of facial dynamics in computer vision. As a case study of using facial dynamics, two LBP-based baseline methods are considered and experimental results in different face-related problems, including face recognition, gender recognition, age estimation and ethnicity classification are reported and discussed. Furthermore, remaining challenges are highlighted and some promising directions are pointed out.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
References
Ahonen T., Hadid A., Pietikäinen M.: Face description with local binary patterns: application to face recognition. IEEE Trans. Pattern Anal. Mach. Intell. 28(12), 2037–2041 (2006)
Baluja S., Rowley H.: Boosting sex identification performance. Int. J. Comput. Vis. 71, 111–119 (2007)
Bassili J.: Emotion recognition: the role of facial movement and the relative importance of upper and lower areas of the face. J. Pers. Soc. Psychol. 37, 2049–2059 (1979)
Bowyer, K., Chang, K., Flynn, P.: A survey of approaches to three-dimensional face recognition. In: International Conference on Automatic Face and Gesture Recognition, vol. 1, pp. 358–361 (2004)
Chellappa, R., Wilson, C.L., Sirohey, S.: Human and machine recognition of faces: a survey. In: IEEE, vol. 83, no. 5, pp. 705–740 (1995)
Chen, L.F., Liao, H.-Y.M., Lin, J.C.: Person identification using facial motion. In: International Conference on Image Processing, 2001, vol. 2, pp. 677–680 (2001)
CRIM: http://www.crim.ca/
Doucet A., Godsill S., Andrieu C.: On sequential monte carlo sampling methods for bayesian filtering. Stat. Comput. 10(3), 197–208 (2000)
Fasel B., Luettin J.: Automatic facial expression analysis: a survey. Pattern Recognit. 36, 259–275 (2003)
Foucher, S., Lalibert, T.F., Boulianne, G., Gagnon, L.: A dempster-shafer based fusion approach for audio-visual speech recognition with application to large vocabulary french speech. In: IEEE International Conference on Acoustics, Speech and Signal Processing, 2006 (ICASSP 2006), pp. I597–I600 (2006)
Freund Y., Schapire R.: A decision-theoretic generalization of on-line learning and an application to boosting. J Comput Syst Sci 55(1), 119–139 (1997)
Geng X., Zhou Z.H., Smith-Miles K.: Automatic age estimation based on facial aging patterns. IEEE Trans. Pattern Anal. Mach. Intell. 29(12), 2234–2240 (2007)
Gross, R., Shi, J.: The CMU motion of body (mobo) database. Technical report (2001)
Hadid, A., Pietikäinen, M.: Combining motion and appearance for gender classification from video sequences. In: 19th International Conference on Pattern Recognition (ICPR 2008), p. 4 (2008)
Hadid A., Pietikäinen M.: Combining appearance and motion for face and gender recognition from videos. Pattern Recognit. 42(11), 2818–2827 (2009)
Hadid, A., Pietikäinen, M., Li, S.Z.: Learning personal specific facial dynamics for face recognition from videos. In: IEEE International Workshop on Analysis and Modeling of Faces and Gestures (in conjunction with ICCV 2007), pp. 1–15 (2007)
Heisele B., Ho P., Wu J., Poggio T.: Face recognition: component based versus global approaches. Comput. Vis. Image Underst. 91(1–2), 6–21 (2003)
Hill H., Johnston A.: Categorizing sex and identity from the biological motion of faces. Curr. Biol. 11(11), 880–885 (2001)
Hjelmas E., Low B.K.: Face detection: a survey. Comput. Vis. Image Underst. 83(3), 236–274 (2001)
Huang, K.S., Trivedi, M.M.: Streaming face recognition using multicamera video arrays. International Conference on Pattern Recognition, vol. 4, p. 40213 (2002)
Kanade, T., Cohn, J.F., Tian, Y.: Comprehensive database for facial expression analysis. In: IEEE International Conference on Automatic Face and Gesture Recognition, pp. 46–53 (2000)
Kittler, J., Nixon, M.S. (eds.): Audio- and video-based biometrie person authentication. Lecture Notes in Computer Science, vol. 2688. Springer (2003)
Knight B., Johnston A.: The role of movement in face recognition. Vis. Cognit. 4, 265–274 (1997)
Koreman, J., Morris, A., Jassim, S., Sellahewa, H., Chollet, G., Aversano, G., Salicetti, S., Allano, L. (eds.): Multi-modal biometric authentication on the Secure Phone PDA (2006)
Lanitis A., Taylor C., Cootes T.: Towards automatic simulation of aging affects on face images. IEEE Trans. Pattern Anal. Mach. Intell. 24, 442–455 (2002)
Lee, B., Hasegawa-Johnson, M., Goudeseune, C., Kamdar, S., Borys, S., Liu, M., Huang, T.: Avicar: Audio-visual speech corpus in a car environment. In: Proceedings of International Conference on Spoken Language (2004)
Lee K.C., Ho J., Yang M.H., Kriegman D.: Visual tracking and recognition using probabilistic appearance manifolds. Comput. Vis. Image Underst. 99(3), 303–331 (2005)
Li B., Chellappa R.: Face verification through tracking facial features. J. Opt. Soc. Am. 18, 2969–2981 (2001)
Li B., Chellappa R.: A generic approach to simultaneous tracking and verification in video. IEEE Trans. Image Process. 11(5), 530–544 (2002)
Li, S.Z., Jain, A.K. (eds): Handbook of Face Recognition. Springer, New York (2005)
Li, Y.: Dynamic face models: construction and applications. Ph.D. thesis, Queen Mary, University of London (2001)
Liu J.S., Chen R.: Sequential monte carlo methods for dynamic systems. J. Am. Stat. Assoc. 93, 1032–1044 (1998)
Liu, X., Chen, T.: Video-based face recognition using adaptive hidden markov models. In: IEEE International Conference on Computer Vision and Pattern Recognition, pp. 340–345 (2003)
Matta, F., Dugelay, J.L.: Person recognition using human head motion information. In: Articulated Motion and Deformable Objects, pp. 326–335 (2006)
Matta, F., Dugelay, J.L.: Video face recognition: a physiological and behavioural multimodal approach. In: ICIP 2007, 14th IEEE International Conference on Image Processing, 16–19 Sept 2007, San Antonio, USA (2007)
Matta F., Dugelay J.L.: Person recognition using facial video information: a state of the art. Image Vis. Comput. 20(3), 180–187 (2009)
Matta, F., Saeed, U., Mallauran, C., Dugelay, J.L.: Facial gender recognition using multiple sources of visual information. In: MMSP 2008, 10th IEEE International Workshop on MultiMedia Signal Processing, 8–10 Oct 2008, Cairns, Queensland, Australia (2008)
Meng, H., Ching, P.C., Lee, T., Mak, M.W., Mak, B., Moon, Y.S., Siu, M.H., Tang, X., Hui, H., Lee, A., Lo, W.K., Ma, B. (eds.): The Multi-Biometric, Multi-Device and MultiLingual (M3) Corpus (2006)
Messer, K., Matas, J., Kittler, J., Lüttin, J., Maitre, G.: XM2VTSDB: The extended M2VTS database. In: Audio- and Video-Based Biometric Person Authentication, AVBPA’99, March 1999, 16 IDIAP–RR 99-02, pp. 72–77. Washington, DC (1999)
Moghaddam B.: Principal manifolds and probabilistic subspaces for visual recognition. IEEE Trans. Pattern Anal. Mach. Intell. 24, 780–788 (2002)
Moghaddam B., Yang M.H.: Learning gender with support faces. IEEE Trans. Pattern Anal. Mach. Intell. 24(5), 707–711 (2002)
Murase H., Nayar S.K.: Visual learning and recognition of 3-d objects from appearance. Int. J. Comput. Vis. 14(1), 5–24 (1995)
Ojala T., Pietikäinen M., Harwood D.: A comparative study of texture measures with classification based on feature distributions. Pattern Recognit. 29, 51–59 (1996)
Ojala T., Pietikäinen M., Mäenpää T.: Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans. Pattern Anal. Mach. Intell. 24, 971–987 (2002)
O’Toole A.J., Roark D.A., Abdi H.: Recognizing moving faces: a psychological and neural synthesis. Trends Cognit. Sci. 6, 261–266 (2002)
Pantic M., Rothkrantz L.: Automatic analysis of facial expressions: the state of the art. IEEE Trans. Pattern Anal. Mach. Intell. 22(12), 1424–1455 (2000)
Pigeon, S., Vandendorpe, L.: The M2VTS multimodal face database (release 1.00). In: AVBPA’97: Proceedings of the First International Conference on Audio- and Video-Based Biometric Person Authentication, pp. 403–409. Springer, London, UK (1997)
Popovici, V., Thiran, J., Bailly-Bailliere, E., Bengio, S., Bimbot, F., Hamouz, M., Kittler, J., Mariethoz, J., Matas, J., Messer, K., Ruiz, B., Poiree, F.: The BANCA database and evaluation protocol. In: 4th International Conference on Audio- and Video-Based Biometric Person Authentication, Guildford, UK. Lecture Notes in Computer Science, vol. 2688, pp. 625–638. SPIE (2003)
Saeed, U., Matta, F., Dugelay, J.L.: Person recognition based on head and mouth dynamics. In: MMSP 2006, IEEE International Workshop on Multimedia Signal Processing, 3–6 Oct 2006, Victoria, Canada (2006)
Samal A., Iyengar P.: Automatic recognition and analysis of human faces and facial expression: a survey. Pattern Recognit. 25(1), 65–77 (1992)
Sanderson C., Paliwal K.K.: Noise compensation in a person verification system using face and multiple speech feature. Pattern Recognit. 36(2), 293–302 (2003)
Shan, C., Gong, S., McOwan, P.: Learning gender from human gaits and faces. In: IEEE International Conference on Advanced Video and Signal based Surveillance, pp. 505–510 (2007)
Tistarelli M., Bicego M., Grosso E.: Dynamic face recognition: From human to machine vision. Image Vis. Comput. 27(3), 222–232 (2009)
Yang M.H., Kriegman D.J., Ahuja N.: Detecting faces in images: a survey. IEEE Trans. Pattern Anal. Mach. Intell. 24, 34–58 (2002)
Zhao G., Pietikäinen M.: Dynamic texture recognition using local binary patterns with an application to facial expressions. IEEE Trans. Pattern Anal. Mach. Intell. 29(6), 915–928 (2007)
Zhao W., Chellappa R., Phillips P.J., Rosenfeld A.: Face recognition: a literature survey. ACM Comput. Surv. 34(4), 399–458 (2003)
Zhou, S., Chellappa, R.: Probabilistic human recognition from video. In: European Conference on Computer Vision, pp. 681–697 (2002)
Zhou S., Chellappa R., Moghaddam B.: Visual tracking and recognition using appearance-adaptive models in particle filters. IEEE Trans. Image Process. 13, 1434–1456 (2004)
Zhou, S., Krueger, V., Chellappa, R.: Face recognition from video: a condensation approach. In: IEEE International Conference on Automatic Face and Gesture Recognition, pp. 221–228 (2002)
Zhou S., Kruger V., Chellappa R.: Probabilistic recognition of human faces from video. Comput. Vis. Image Underst. 91, 214–245 (2002)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Hadid, A., Dugelay, JL. & Pietikäinen, M. On the use of dynamic features in face biometrics: recent advances and challenges. SIViP 5, 495–506 (2011). https://doi.org/10.1007/s11760-011-0247-3
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11760-011-0247-3