Skip to main content
SpringerLink
Log in

Learning Approaches for Facial Expression Recognition in Ageing Adults: A Comparative Study

  • Chapter
  • First Online:
Advances in Data Science: Methodologies and Applications

Part of the book series: Intelligent Systems Reference Library ((ISRL,volume 189))

Abstract

Average life expectancy has increased steadily in recent decades. This phenomenon, considered together with aging of the population, will inevitably produce in the next years deep social changes that lead to the need of innovative services for elderly people, focused to improve the wellbeing and the quality of life. In this context many potential applications would benefit from the ability of automatically recognize facial expression with the purpose to reflect the mood, the emotions and also mental activities of an observed subject. Although facial expression recognition (FER) is widely investigated by many recent scientific works, it still remains a challenging task for a number of important factors among which one of the most discriminating is the age. In the present work an optimized Convolutional Neural Network (CNN) architecture is proposed and evaluated on two benchmark datasets (FACES and Lifespan) containing expressions performed also by aging adults. As baseline, and with the aim of making a comparison, two traditional machine learning approaches based on handcrafted features extraction process are evaluated on the same datasets. Experimentation confirms the efficiency of the proposed CNN architecture with an average recognition rate higher than 93.6% for expressions performed by ageing adults when a proper set of CNN parameters was used. Moreover, the experimentation stage showed that the deep learning approach significantly improves the baseline approaches considered, and the most noticeable improvement was obtained when considering facial expressions of ageing adults.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 199.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 199.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. United Nations Programme on Ageing. The ageing of the world’s population, December 2013. http://www.un.org/en/development/desa/population/publications/pdf/ageing/WorldPopulationAgeing2013.pdf. Accessed July 2018

  2. Zeng, Z., Pantic, M., Roisman, G.I., Huang, T.S.: A survey of affect recognition methods: audio, visual, and spontaneous expressions. IEEE Trans. Pattern Anal. Mach. Intell. 31(1), 39–58 (2009). https://doi.org/10.1109/tpami.2008.52

    Article  Google Scholar 

  3. Pantic, M., Rothkrantz, L.J.M.: Automatic analysis of facial expressions: the state of the art. IEEE Trans. Pattern Anal. Mach. Intell. 22(12), 1424–1445 (2000). https://doi.org/10.1109/34.895976

    Article  Google Scholar 

  4. Fasel, B., Luettin, J.: Automatic facial expression analysis: a survey. Pattern Recogn. 36(1), 259–275 (2003). https://doi.org/10.1016/s0031-3203(02)00052-3

    Article  MATH  Google Scholar 

  5. Carroll, J.M., Russell, J.A.: Do facial expressions signal specific emotions? Judging emotion from the face in context. J. Pers. Soc. Psychol. 70(2), 205 (1996). https://doi.org/10.1037//0022-3514.70.2.205

    Article  Google Scholar 

  6. Ekman, P., Rolls, E.T., Perrett, D.I., Ellis, H.D.: Facial expressions of emotion: an old controversy and new findings [and discussion]. Philoso. Trans. R Soc. B Biolog. Sci. 335(1273), 63–69 (1992). https://doi.org/10.1098/rstb.1992.0008

    Article  Google Scholar 

  7. Shbib, R., Zhou, S.: Facial expression analysis using active shape model. Int. J. Sig. Process. Image Process. Pattern Recogn. 8(1), 9–22 (2015). https://doi.org/10.14257/ijsip.2015.8.1.02

  8. Cheon, Y., Kim, D.: Natural facial expression recognition using differential-AAM and manifold learning. Pattern Recogn. 42(7), 1340–1350 (2009). https://doi.org/10.1016/j.patcog.2008.10.010

    Article  MATH  Google Scholar 

  9. Soyel, H., Demirel, H.: Facial expression recognition based on discriminative scale invariant feature transform. Electron. Lett. 46(5), 343–345 (2010). https://doi.org/10.1049/el.2010.0092

    Article  Google Scholar 

  10. Gu, W., Xiang, C., Venkatesh, Y.V., Huang, D., Lin, H.: Facial expression recognition using radial encoding of local Gabor features and classifier synthesis. Pattern Recogn. 45(1), 80–91 (2012). https://doi.org/10.1016/j.patcog.2011.05.006

    Article  Google Scholar 

  11. Shan, C., Gong, S., McOwan, P.W.: Facial expression recognition based on local binary patterns: a comprehensive study. Image Vis. Comput. 27(6), 803–816 (2009). https://doi.org/10.1016/j.imavis.2008.08.005

    Article  Google Scholar 

  12. Chen, J., Chen, Z., Chi, Z., Fu, H.: Facial expression recognition based on facial components detection and hog features. In: International Workshops on Electrical and Computer Engineering Subfields, pp. 884–888 (2014)

    Google Scholar 

  13. Guo, G., Guo, R., Li, X.: Facial expression recognition influenced by human aging. IEEE Trans. Affect. Comput. 4(3), 291–298 (2013). https://doi.org/10.1109/t-affc.2013.13

    Article  Google Scholar 

  14. Wang, S., Wu, S., Gao, Z., Ji, Q.: Facial expression recognition through modeling age-related spatial patterns. Multimedia Tools Appl. 75(7), 3937–3954 (2016). https://doi.org/10.1007/s11042-015-3107-2

    Article  Google Scholar 

  15. Malatesta C.Z., Izard C.E.: The facial expression of emotion: young, middle-aged, and older adult expressions. In: Malatesta C.Z., Izard C.E. (eds.) Emotion in Adult Development, pp. 253–273. Sage Publications, London (1984)

    Google Scholar 

  16. Malatesta-Magai, C., Jonas, R., Shepard, B., Culver, L.C.: Type A behavior pattern and emotion expression in younger and older adults. Psychol. Aging 7(4), 551 (1992). https://doi.org/10.1037//0882-7974.8.1.9

    Article  Google Scholar 

  17. Malatesta, C.Z., Fiore, M.J., Messina, J.J.: Affect, personality, and facial expressive characteristics of older people. Psychol. Aging 2(1), 64 (1987). https://doi.org/10.1037//0882-7974.2.1.64

    Article  Google Scholar 

  18. Lozano-Monasor, E., López, M.T., Vigo-Bustos, F., Fernández-Caballero, A.: Facial expression recognition in ageing adults: from lab to ambient assisted living. J. Ambi. Intell. Human. Comput. 1–12 (2017). https://doi.org/10.1007/s12652-017-0464-x

  19. LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436–444 (2015). https://doi.org/10.1038/nature14539

    Article  Google Scholar 

  20. Yu, D., Deng, L.: Deep learning and its applications to signal and information processing [exploratory dsp]. IEEE Signal Process. Mag. 28(1), 145–154 (2011). https://doi.org/10.1109/msp.2010.939038

    Article  Google Scholar 

  21. Xie, S., Hu, H.: Facial expression recognition with FRR-CNN. Electron. Lett. 53(4), 235–237 (2017). https://doi.org/10.1049/el.2016.4328

    Article  Google Scholar 

  22. Li, Y., Zeng, J., Shan, S., Chen, X.: Occlusion aware facial expression recognition using cnn with attention mechanism. IEEE Trans. Image Process. 28(5), 2439–2450 (2018). https://doi.org/10.1109/TIP.2018.2886767

    Article  MathSciNet  Google Scholar 

  23. Lopes, A.T., de Aguiar, E., De Souza, A.F., Oliveira-Santos, T.: Facial expression recognition with convolutional neural networks: coping with few data and the training sample order. Pattern Recogn. 61, 610–628 (2017). https://doi.org/10.1016/j.patcog.2016.07.026

    Article  Google Scholar 

  24. Goodfellow, I.J., Erhan, D., Carrier, P.L., Courville, A., Mirza, M., Hamner, B., …, Zhou, Y.: Challenges in representation learning: a report on three machine learning contests. In: International Conference on Neural Information Processing, pp. 117–124. Springer, Berlin, Heidelberg (2013). https://doi.org/10.1016/j.neunet.2014.09.005

  25. Kahou, S.E., Pal, C., Bouthillier, X., Froumenty, P., Gülçehre, Ç., Memisevic, R., …, Mirza, M.: Combining modality specific deep neural networks for emotion recognition in video. In: Proceedings of the 15th ACM on International Conference on Multimodal Interaction, pp. 543–550. ACM (2013)

    Google Scholar 

  26. Liu, M., Wang, R., Li, S., Shan, S., Huang, Z., Chen, X.: Combining multiple kernel methods on riemannian manifold for emotion recognition in the wild. In: Proceedings of the 16th International Conference on Multimodal Interaction, pp. 494–501. ACM (2014). https://doi.org/10.1145/2663204.2666274

  27. LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998). https://doi.org/10.1109/5.726791

    Article  Google Scholar 

  28. Viola, P., Jones, M.J.: Robust real-time face detection. Int. J. Comput. Vision 57(2), 137–154 (2004). https://doi.org/10.1023/b:visi.0000013087.49260.fb

    Article  Google Scholar 

  29. Zuiderveld, K.: Contrast limited adaptive histogram equalization. Graphics Gems 474–485 (1994). https://doi.org/10.1016/b978-0-12-336156-1.50061-6

  30. Hubel, D.H., Wiesel, T.N.: Receptive fields and functional architecture of monkey striate cortex. J. Physiol. 195(1), 215–243 (1968). https://doi.org/10.1113/jphysiol.1968.sp008455

    Article  Google Scholar 

  31. Glorot, X., Bordes, A., Bengio, Y.: Deep sparse rectifier neural networks. In: Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, pp. 315–323 (2011)

    Google Scholar 

  32. Bottou, L.: Large-scale machine learning with stochastic gradient descent. In: Proceedings of COMPSTAT ’2010, pp. 177–186. Physica-Verlag HD (2010). https://doi.org/10.1007/978-3-7908-2604-3_16

  33. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)

    Google Scholar 

  34. Smith, L.N.: Cyclical learning rates for training neural networks. In: 2017 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 464–472 IEEE (2017). https://doi.org/10.1109/wacv.2017.58

  35. Milborrow, S., Nicolls, F.: Active shape models with SIFT descriptors and MARS. In: 2014 International Conference on Computer Vision Theory and Applications (VISAPP), vol. 2, pp. 380–387. IEEE (2014). https://doi.org/10.5220/0004680003800387

  36. Ebner, N.C., Riediger, M., Lindenberger, U.: FACES—a database of facial expressions in young, middle-aged, and older women and men: development and validation. Behav. Res. Methods 42(1), 351–362 (2010). https://doi.org/10.3758/brm.42.1.351

    Article  Google Scholar 

  37. Minear, M., Park, D.C.: A lifespan database of adult facial stimuli. Behav. Res. Methods Instru. Comput. 36(4), 630–633 (2004). https://doi.org/10.3758/bf03206543

    Article  Google Scholar 

  38. Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., …, Kudlur, M.: Tensorflow: a system for large-scale machine learning. In: OSDI, vol. 16, pp. 265–283 (2016)

    Google Scholar 

  39. Zhang, C., Zhang, Z.: A survey of recent advances in face detection (2010)

    Google Scholar 

  40. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556 (2014)

  41. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016). https://doi.org/10.1109/cvpr.2016.90

  42. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., …, Rabinovich, A.: Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–9 (2015). https://doi.org/10.1109/cvpr.2015.7298594

Download references

Author information

Authors and Affiliations

  1. National Research Council of Italy, Institute for Microelectronics and Microsystems, Via Monteroni c/o Campus Universitario Ecotekne-Palazzina A3, 73100, Lecce, Italy

    Andrea Caroppo, Alessandro Leone & Pietro Siciliano

Authors
  1. Andrea Caroppo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alessandro Leone
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Pietro Siciliano
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Andrea Caroppo .

Editor information

Editors and Affiliations

  1. Loyola University Maryland, Baltimore, MD, USA

    Gloria Phillips-Wren

  2. Dipartimento di Psicologia, Università della Campania “Luigi Vanvitelli”, Caserta, Italy

    Anna Esposito

  3. Centre for Artificial Intelligence, University of Technology Sydney Broadway,, Sydney, NSW, Australia

    Lakhmi C. Jain

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Caroppo, A., Leone, A., Siciliano, P. (2021). Learning Approaches for Facial Expression Recognition in Ageing Adults: A Comparative Study. In: Phillips-Wren, G., Esposito, A., Jain, L.C. (eds) Advances in Data Science: Methodologies and Applications. Intelligent Systems Reference Library, vol 189. Springer, Cham. https://doi.org/10.1007/978-3-030-51870-7_15

Download citation

Publish with us

Policies and ethics

Search

Navigation