Skip to main content
SpringerLink
Log in

A Survey on Facial Emotion Recognition for the Elderly

  • Conference paper
  • First Online:
Digital Technologies and Applications (ICDTA 2023)

Part of the book series: Lecture Notes in Networks and Systems ((LNNS,volume 668))

Included in the following conference series:

  • 858 Accesses

Abstract

The elderly have a sensitive period of life in terms of physical and mental health and require close assistance or a caregiver. Medical assistance has the ability to recognize the emotional states of older adults through facial expressions and take care of them in real time. This paper provides a comprehensive Facial Emotion Recognition (FER) review, especially for the elderly. Several studies have been conducted on the facial emotion recognition of young and middle-aged adults. Very few studies have focused on automatic emotion recognition for the elderly. Aging comes with a decline in the ability to recognize emotions and impacts emotion perception in humans. Furthermore, older people are suffering from cognitive impairment worldwide, which leads to abnormal emotional patterns. This paper is a literature review of FER techniques in computer vision; FER approaches, and FER databases, and discusses the main challenge of facial expression recognition across age and lifespan.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 229.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 299.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. United Nations, Department of Economic and Social Affairs, Population Division, World population ageing 2020 Highlights: living arrangements of older persons (2020)

    Google Scholar 

  2. Khanal, S., Reis, A., Barroso, J., Filipe, V.: Using emotion recognition in intelligent interface design for elderly care. In: Rocha, Á., Adeli, H., Reis, L.P., Costanzo, S. (eds.) WorldCIST’18 2018. AISC, vol. 746, pp. 240–247. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-77712-2_23

    Chapter  Google Scholar 

  3. Lopes, N., et al.: Facial emotion recognition in the elderly using a SVM classifier. In: 2018 2nd International Conference on Technology and Innovation in Sports, Health and Wellbeing (TISHW) and Facial Expression Recognition in Older Adults using Deep Machine Learning, Thessaloniki, pp. 1–5 (2018).

    Google Scholar 

  4. Dou, S., Feng, Z., Yang, X., Tian, J.: Real-time multimodal emotion recognition system based on elderly accompanying robot. J. Phys. Conf. Ser. 1453(1), 012093 (2020). https://doi.org/10.1088/1742-6596/1453/1/012093

    Article  Google Scholar 

  5. Ma, K., Wang, X., Yang, X., Zhang, M., Girard, J.M., Morency, L.-P.: ElderReact: a multimodal dataset for recognizing emotional response in aging adults. In: 2019 International Conference on Multimodal Interaction, Suzhou, China, pp. 349–357 (2019)

    Google Scholar 

  6. Anderson, G.F., Hussey, P.S.: Population aging: a comparison among industrialized countries: populations around the world are growing older, but the trends are not cause for despair. Health Affairs 19(3), 191–203 (2000). https://doi.org/10.1377/hlthaff.19.3.191

    Article  Google Scholar 

  7. Spezialetti, M., Placidi, G., Rossi, S.: Emotion recognition for human-robot interaction: recent advances and future perspectives. Front. Rob. AI 7, 145 (2020). https://doi.org/10.3389/frobt.2020.532279

    Article  Google Scholar 

  8. Ekman, P.: An argument for basic emotions. Cogn. Emot. 6(3–4), 169–200 (2008). https://doi.org/10.1080/02699939208411068

    Article  Google Scholar 

  9. Chafale, D., Pimpalkar, A.: Review on developing corpora for sentiment analysis using plutchik’s wheel of emotions with fuzzy logic. Int. J. Comput. Sci. Eng. IJCSE 2, 14–18 (2014)

    Google Scholar 

  10. Canal, F.Z., et al.: A survey on facial emotion recognition techniques: a state-of-the-art literature review. Inf. Sci. 582, 593–617 (2022). https://doi.org/10.1016/j.ins.2021.10.005

    Article  Google Scholar 

  11. Kumar, R.J.R., Sundaram, M., Arumugam, N.: Facial emotion recognition using subband selective multilevel stationary wavelet gradient transform and fuzzy support vector machine. Vis. Comput. 37(8), 2315–2329 (2021). https://doi.org/10.1007/s00371-020-01988-1

    Article  Google Scholar 

  12. Michael Revina, I., Sam Emmanuel, W.R.: A survey on human face expression recognition techniques. J. King Saud Univ. Comput. Inf. Sci. 33(6), 619–628 (2021). https://doi.org/10.1016/j.jksuci.2018.09.002

    Article  Google Scholar 

  13. Kuruvayil, S., Palaniswamy, S.: Emotion recognition from facial images with simultaneous occlusion, pose and illumination variations using meta-learning. J. King Saud Univ. Comput. Inf. Sci. 34(9), 7271–7282 (2022). https://doi.org/10.1016/j.jksuci.2021.06.012

    Article  Google Scholar 

  14. Ebner, N.C., Riediger, M., Lindenberger, U.: FACES—a database of facial expressions in young, middle-aged, and older women and men: development and validation. Behav. Res. Methods 42(1), 351–362 (2010). https://doi.org/10.3758/BRM.42.1.351

    Article  Google Scholar 

  15. Kas, M., Merabet, Y.E., Ruichek, Y., Messoussi, R.: New framework for person-independent facial expression recognition combining textural and shape analysis through new feature extraction approach. Inf. Sci. 549, 200–220 (2021). https://doi.org/10.1016/j.ins.2020.10.065

    Article  MathSciNet  Google Scholar 

  16. Stone, J.V.: Independent component analysis: an introduction. Trends Cogn. Sci. 6(2), 59–64 (2002). https://doi.org/10.1016/S1364-6613(00)01813-1

    Article  Google Scholar 

  17. Xanthopoulos, P., Pardalos, P.M., Trafalis, T.B.: Linear discriminant analysis. In: Xanthopoulos, P., Pardalos, P.M., Trafalis, T.B. (eds.) Robust Data Mining, pp. 27–33. Springer New York, New York (2013). https://doi.org/10.1007/978-1-4419-9878-1_4

    Chapter  MATH  Google Scholar 

  18. Kurita, T.: Principal component analysis (PCA). In: Computer Vision: A Reference Guide, pp. 1–4. Springer International Publishing, Cham (2019). https://doi.org/10.1007/978-3-030-03243-2_649-1

    Chapter  Google Scholar 

  19. Fukumizu, Y., Takano, T., Oshima, Y., Terada, T., Yamauchi, H.: A gabor pseudo fisherface based face recognition algorithm for LSI implementation. In: 2009 Fifth International Conference on Intelligent Information Hiding and Multimedia Signal Processing, Kyoto, Japan, pp. 1342–1345 (2009). https://doi.org/10.1109/IIH-MSP.2009.236

  20. Kas, M.: Development of handcrafted and deep based methods for face and facial expression recognition, p. 160 (2021)

    Google Scholar 

  21. Lekdioui, K.: Reconnaissance d’états émotionnels par analyse visuelle du visage et apprentissage machine, p. 190 (2018)

    Google Scholar 

  22. Ghazouani, H.: A genetic programming-based feature selection and fusion for facial expression recognition. Appl. Soft Comput. 103, 107173 (2021). https://doi.org/10.1016/j.asoc.2021.107173

    Article  Google Scholar 

  23. Algaraawi, N., Morris, T.: Study on Aging Effect on Facial Expression Recognition, p. 7 (2016)

    Google Scholar 

  24. Maithri, M., et al.: Automated emotion recognition: current trends and future perspectives. Comput. Methods Programs Biomed., 106646 (2022). https://doi.org/10.1016/j.cmpb.2022.106646

  25. Schirmer, A., Adolphs, R.: Emotion perception from face, voice, and touch: comparisons and convergence. Trends Cogn. Sci. 21(3), 216–228 (2017). https://doi.org/10.1016/j.tics.2017.01.001

    Article  Google Scholar 

  26. Ekundayo, O.S., Viriri, S.: facial expression recognition: a review of trends and techniques. IEEE Access 9, 136944–136973 (2021). https://doi.org/10.1109/ACCESS.2021.3113464

    Article  Google Scholar 

  27. Ge, H., Zhu, Z., Dai, Y., Wang, B., Xuedong, W.: Facial expression recognition based on deep learning. Comput. Methods Prog. Biomed. 215, 106621 (2022). https://doi.org/10.1016/j.cmpb.2022.106621

    Article  Google Scholar 

  28. Fei, Z., Yang, E., Leijian, Y., Li, X., Zhou, H., Zhou, W.: A Novel deep neural network-based emotion analysis system for automatic detection of mild cognitive impairment in the elderly. Neurocomputing 468, 306–316 (2022). https://doi.org/10.1016/j.neucom.2021.10.038

    Article  Google Scholar 

  29. Montembeault, M., et al.: Multimodal emotion perception in young and elderly patients with multiple sclerosis. Multiple Sclerosis Relat. Disord. 58, 103478 (2022). https://doi.org/10.1016/j.msard.2021.103478

    Article  Google Scholar 

  30. Hwang, S., Hwang, J., Jeong, H.: Study on associating emotions in verbal reactions to facial expressions in dementia. Healthcare 10(6), 1022 (2022). https://doi.org/10.3390/healthcare10061022

    Article  Google Scholar 

  31. Ochi, R., Midorikawa, A.: Decline in emotional face recognition among elderly people may reflect mild cognitive impairment. Front. Psychol. 12, 664367 (2021). https://doi.org/10.3389/fpsyg.2021.664367

    Article  Google Scholar 

  32. Chuang, Y.-C., et al.: An exploration of the own-age effect on facial emotion recognition in normal elderly people and individuals with the preclinical and demented alzheimer’s disease. J. Alzheimers Dis. JAD 80(1), 259–269 (2021). https://doi.org/10.3233/JAD-200916

    Article  Google Scholar 

  33. Ferreira, B.L.C., de Fabrício, D., Chagas, M.H.N.: Are facial emotion recognition tasks adequate for assessing social cognition in older people? a review of the literature. Arch. Gerontol. Geriatr. 92, 104277 (2021). https://doi.org/10.1016/j.archger.2020.104277

    Article  Google Scholar 

  34. Grondhuis, S.N., Jimmy, A., Teague, C., Brunet, N.M.: Having difficulties reading the facial expression of older individuals? blame it on the facial muscles, not the wrinkles. Front. Psychol. 12, 620768 (2021). https://doi.org/10.3389/fpsyg.2021.620768

    Article  Google Scholar 

  35. Belkhiria, C., Vergara, R.C., Martinez, M., Delano, P.H., Delgado, C.: Neural links between facial emotion recognition and cognitive impairment in presbycusis. Int. J. Geriatr. Psychiatry 36(8), 1171–1178 (2021). https://doi.org/10.1002/gps.5501

    Article  Google Scholar 

  36. Liu, Y., Wang, Z., Yu, G.: The effectiveness of facial expression recognition in detecting emotional responses to sound interventions in older adults with dementia. Front. Psychol. 12, 707809 (2021). https://doi.org/10.3389/fpsyg.2021.707809

    Article  Google Scholar 

  37. Ko, H., et al.: Changes in facial recognition and facial expressions with age. Behav. Sci. (2021). https://doi.org/10.20944/preprints202104.0542.v1

  38. Aktürk, T., İşoğlu-Alkaç, Ü., Hanoğlu, L., Güntekin, B.: Age related differences in the recognition of facial expression: evidence from EEG event-related brain oscillations. Int. J. Psychophysiol. 147, 244–256 (2020). https://doi.org/10.1016/j.ijpsycho.2019.11.013

    Article  Google Scholar 

  39. Caroppo, A., Leone, A., Siciliano, P.: Facial expression recognition in older adults using deep machine learning, p. 14 (2017)

    Google Scholar 

  40. Minear, M., Park, D.C.: A lifespan database of adult facial stimuli. Behav. Res. Methods Instrum. Comput. 36(4), 630–633 (2004). https://doi.org/10.3758/BF03206543

    Article  Google Scholar 

  41. Huang, W.: Elderly depression recognition based on facial micro-expression extraction. Trait. Signal 38(4), 1123–1130 (2021). https://doi.org/10.18280/ts.380423

  42. Lakshmi, D., Ponnusamy, R.: Facial emotion recognition using modified HOG and LBP features with deep stacked autoencoders. Microprocess. Microsyst. 82, 103834 (2021). https://doi.org/10.1016/j.micpro.2021.103834

    Article  Google Scholar 

  43. Murugappan, M., Mutawa, A.: Facial geometric feature extraction based emotional expression classification using machine learning algorithms. PLOS ONE 16(2), e0247131 (2021). https://doi.org/10.1371/journal.pone.0247131

    Article  Google Scholar 

  44. Hajarolasvadi, N., Bashirov, E., Demirel, H.: Video-based person-dependent and person-independent facial emotion recognition. SIViP 15(5), 1049–1056 (2021). https://doi.org/10.1007/s11760-020-01830-0

    Article  Google Scholar 

  45. Hernández-Luquin, F., Escalante, H.J.: Multi-branch deep radial basis function networks for facial emotion recognition. Neural Comput. Appl. 2021, 1–15 (2021). https://doi.org/10.1007/s00521-021-06420-w

    Article  Google Scholar 

  46. Devi, D.A.S., Satyanarayana, C.: An efficient facial emotion recognition system using novel deep learning neural network-regression activation classifier. Multimedia Tools Appl. 80(12), 17543–17568 (2021). https://doi.org/10.1007/s11042-021-10547-2

    Article  Google Scholar 

  47. Arabian, H., Wagner-Hartl, V., Geoffrey Chase, J., Möller, K.: Image pre-processing significance on regions of impact in a trained network for facial emotion recognition. IFAC-PapersOnLine 54(15), 299–303 (2021). https://doi.org/10.1016/j.ifacol.2021.10.272

    Article  Google Scholar 

  48. Chowdary, M.K., Nguyen, T.N., Hemanth, D.J.: Deep learning-based facial emotion recognition for human–computer interaction applications. Neural Comput. Appl. 2021, 1–18 (2021). https://doi.org/10.1007/s00521-021-06012-8

    Article  Google Scholar 

  49. Do, L.-N., Yang, H.-J., Nguyen, H.-D., Kim, S.-H., Lee, G.-S., Na, I.-S.: Deep neural network-based fusion model for emotion recognition using visual data. J. Supercomput. 77(10), 10773–10790 (2021). https://doi.org/10.1007/s11227-021-03690-y

    Article  Google Scholar 

  50. Georgopoulos, Markos, Panagakis, Yannis, Pantic, Maja: Modeling of facial aging and kinship: a survey. Image Vision Comput. 80, 58–79 (2018). https://doi.org/10.1016/j.imavis.2018.05.003

    Article  Google Scholar 

  51. Ricanek, K., Tesafaye, T.: MORPH: a longitudinal image database of normal adult age-progression. In: 7th International Conference on Automatic Face and Gesture Recognition (FGR06), Southampton, UK, pp. 341–345 (2006) https://doi.org/10.1109/FGR.2006.78

  52. Panis, G., Lanitis, A.: An overview of research activities in facial age estimation using the FG-NET aging database. In: Agapito, L., Bronstein, M.M., Rother, C. (eds.) ECCV 2014. LNCS, vol. 8926, pp. 737–750. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-16181-5_56

    Chapter  Google Scholar 

  53. Moschoglou, S., Papaioannou, A., Sagonas, C., Deng, J., Kotsia, I., Zafeiriou, S.: AgeDB: the first manually collected, in-the-wild age database. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Honolulu, HI, USA, pp. 1997‑2005 (2017). https://doi.org/10.1109/CVPRW.2017.250

  54. Rothe, R., Timofte, R., Gool, L.V.: DEX: deep expectation of apparent age from a single image. In: 2015 IEEE International Conference on Computer Vision Workshop (ICCVW), Santiago, Chile, pp. 252–257 (2015). https://doi.org/10.1109/ICCVW.2015.41

  55. Lucey, P., Cohn, J.F., Kanade, T., Saragih, J., Ambadar, Z., Matthews, I.: The Extended Cohn-Kanade Dataset (CK+): a complete dataset for action unit and emotion-specified expression. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops, San Francisco, CA, USA, pp. 94–101 (2010). https://doi.org/10.1109/CVPRW.2010.5543262

  56. Lyons, M.J.: ‘‘Excavating AI’’ Re-excavated: Debunking a Fallacious Account of the JAFFE Dataset, p. 20 (2021)

    Google Scholar 

  57. Li, S., Deng, W., Du, J.: Reliable crowdsourcing and deep locality-preserving learning for expression recognition in the wild. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, pp. 2584–2593 (2017). https://doi.org/10.1109/CVPR.2017.277

  58. Özdemir, M., Elagöz, B., Alaybeyoglu, A., Sadighzadeh, R., Akan, A.: Real Time Emotion Recognition from Facial Expressions Using CNN Architecture (2019). https://doi.org/10.1109/TIPTEKNO.2019.8895215

  59. Zhao, G., Huang, X., Taini, M., Li, S.Z., Pietikäinen, M.: Facial expression recognition from near-infrared videos. Image Vis. Comput. 29(9), 607–619 (2011). https://doi.org/10.1016/j.imavis.2011.07.002

    Article  Google Scholar 

  60. Calvo, M.G., Lundqvist, D.: Facial expressions of emotion (KDEF): identification under different display-duration conditions. Behav. Res. Methods 40(1), 109–115 (2008). https://doi.org/10.3758/BRM.40.1.109

    Article  Google Scholar 

Download references

Acknowledgments

We would like to thank Teamnet and Kompai Robotics for being our industrial partners in this project. As a reminder, this publication is part of the work undertaken by different partners composed of MAScIR (Moroccan Foundation for Advanced Science, Innovation and Research), ENSIAS (Ecole Nationale Supérieure d'Informatique et d'Analyse des Systèmes), ENSAO (Ecole Nationale des Sciences Appliquées d'Oujda) and USPN (Université de Sorbonne Paris Nord) within the framework of the ‘‘Medical Assistant Robot’’ project. This project has been supported by the Moroccan Ministry of Higher Education, Scientific Research and Innovation, the Digital Development Agency (DDA) and the National Center for Scientific and Technical Research of Morocco (CNRST) through the “Al-Khawarizmi project”, besides Kompai, Teamnet and MAScIR. (Research project code: Alkhawarizmi/2020/15).

Author information

Authors and Affiliations

  1. IRDA Group, ADMIR Lab. Rabat IT Center, ENSIAS, Mohammed V University of Rabat, Rabat, Morocco

    Nouhaila Labzour & Sanaa El Fkihi

  2. Embedded Systems and Artificial Intelligence Department, MAScIR, Rabat, Morocco

    Nouhaila Labzour, Soukayna Benaissa, Yahya Zennayi & Omar Bourja

Authors
  1. Nouhaila Labzour
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sanaa El Fkihi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Soukayna Benaissa
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yahya Zennayi
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Omar Bourja
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nouhaila Labzour .

Editor information

Editors and Affiliations

  1. Ecole Nationale des Sciences Appliquées, Fez, Morocco

    Saad Motahhir

  2. Faculty of Sciences, Sidi Mohamed Ben Abdellah University, Fez, Morocco

    Badre Bossoufi

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Labzour, N., Fkihi, S.E., Benaissa, S., Zennayi, Y., Bourja, O. (2023). A Survey on Facial Emotion Recognition for the Elderly. In: Motahhir, S., Bossoufi, B. (eds) Digital Technologies and Applications. ICDTA 2023. Lecture Notes in Networks and Systems, vol 668. Springer, Cham. https://doi.org/10.1007/978-3-031-29857-8_57

Download citation

Publish with us

Policies and ethics

Search

Navigation