Abstract
Psychology suggests highly synchronized expressions of emotion across different modalities. Few experiments jointly studied the relative contribution of facial expression and body posture to the overall perception of emotion. Computational models for expressive virtual characters have to consider how such combinations will be perceived by users. This paper reports on two studies exploring how subjects perceived a virtual agent. The first study evaluates the contribution of the facial and postural expressions to the overall perception of basic emotion categories, as well as the valence and activation dimensions. The second study explores the impact of incongruent expressions on the perception of superposed emotions which are known to be frequent in everyday life. Our results suggest that the congruence of facial and bodily expression facilitates the recognition of emotion categories. Yet, judgments were mainly based on the emotion expressed in the face but were nevertheless affected by postures for the perception of the activation dimension.
Access provided by Autonomous University of Puebla. Download to read the full chapter text
Chapter PDF
Similar content being viewed by others
Keywords
References
Albrecht, I., Schröder, M., Haber, J., Seidel, H.-P.: Mixed feelings: Expression of non-basic emotions in a muscle-based talking head. Special issue of Journal of Virtual Reality on Language, Speech & Gesture 8(4) (2005)
Ballin, D., Gillies, M., Crabtree, B.: A Framework For Interpersonal Attitude And Non-Verbal Communication In Improvisational Visual Media Production. In: The First European Conference on Visual Media Production IEE, London (2004)
Bänziger, T., Scherer, K.: Using Actor Portrayals to Systematically Study Multimodal Emotion Expression: The GEMEP Corpus. In: Paiva, A.C.R., Prada, R., Picard, R.W. (eds.) ACII 2007. LNCS, vol. 4738, pp. 476–487. Springer, Heidelberg (2007)
Bianchi-Berthouze, N., Kleinsmith, A.: A categorical approach to affective gesture recognition. Connection Science 15(4), 259–269 (2003)
Buisine, S., Abrilian, S., Niewiadomski, R., Martin, J.C., Devillers, L., Pelachaud, C.: Perception of blended emotions: From video corpus to expressive agent. In: Gratch, J., Young, M., Aylett, R.S., Ballin, D., Olivier, P. (eds.) IVA 2006. LNCS (LNAI), vol. 4133, pp. 93–106. Springer, Heidelberg (2006)
De Silva, R., Bianchi-Berthouze, N.: Modelling human affective postures: an information theoretic characterization of posture features. Computer Animation and Virtual Worlds 15(3-4), 269–276 (2004)
Dittman, A.T.: The role of body movement in communication. In: Nonverbal behavior and communication. Lawrence Erlbaum, Mahwah (1987)
Ekman, P.: An argument for basic emotions. Cognition and Emotion 6, 169–200 (1992)
Ekman, P., Friesen, W.: Manual for the facial action coding system. Consulting Psychology Press, Palo Alto (1978)
Ekman, P., Friesen, W.: A new pan-cultural facial expression of emotion. Motivation and Emotion 10, 159–168 (1986)
Ekman, P., Friesen, W.V.: Nonverbal behavior and psychopathology. In: Friedman, R.J., Datz, M.M. (eds.) The psychology of depression: contemporary theory and research, pp. 203–232. Winston & Sons, Washington (1974)
Ekman, P., Friesen, W.V.: Unmasking the face. A guide to recognizing emotions from facial clues. Prentice-Hall Inc., Englewood Cliffs (1975)
Ekman, P., Friesen., W.: Unmasking the face: a guide to recognizing emotions from facial clues. Prentice-Hall, Englewood Cliffs (1975)
Fontaine, J.R., Scherer, K.R., Roesch, E.B., Ellsworth, P.: The world of emotion is not two-dimensional. Psychological Science 18(2), 1050–1057 (2007)
Gosselin, P., Kirouac, G.: Le décodage de prototypes emotionels faciaux. Canadian journal of experimental psychology 49(3), 313-329 (1995)
Gunes, H., Piccardi, M.: Bi-modal emotion recognition from expressive face and body gestures. Journal of Network & Computer Applications 30(4), 1334–1345 (2007)
Hall, J.A., Matsumoto, D.: Gender differences in judgments of multiple emotions from facial expressions. Emotion 4(2), 201–206 (2004)
Harrigan, J.A.: Proxemics, kinesics, and gaze. In: Harrigan, J.A., Rosenthal, R., Scherer, K. (eds.) The new handbook of methods in nonverbal behavior research. Series in Affective Science, pp. 137–198. Oxford University Press, Oxford (2005)
Hietanen, J.K., Leppänen, J.M.: Judgment of other people’s facial expressions of emotions is influenced by their concurrent affective hand movements. Scandinavian Journal of Psychology 49, 221–230 (2008)
Hietanen, J.K., Leppänen, J.M.: Judgment of other people’s facial expressions of emotions is influenced by their concurrent affective hand movements. Scandinavian Journal of Psychology 49, 221–230 (2008)
Izard, C.: Human Emotions. Plenum, New York (1977)
Kleinsmith, A., Bianchi-Berthouze, N.: Recognizing affective dimensions from body posture. In: Paiva, A.C.R., Prada, R., Picard, R.W. (eds.) ACII 2007. LNCS, vol. 4738, pp. 48–58. Springer, Heidelberg (2007)
Kleinsmith, A., De Silva, R., Bianchi-Berthouze, N.: Cross-Cultural Differences in Recognizing Affect from Body Posture. Interacting with Computers 18(6), 1371–1389 (2006)
Meeren, H.K.M., van Heijnsbergen, C., de Gelder, B.: Rapid perceptual integration of facial expression and emotional body language. Proceedings of the National Academy of Sciences of the USA, 16518–16523 (2005)
Mota, S., Picard, R.W.: Automated posture analysis for detecting learner’s interest level. In: Workshop on Computer Vision and Pattern Recognition for Human-Computer Interaction (2003)
Niewiadomski, R.: A model of complex facial expressions in interpersonal relations for animated agents. PhD Thesis, University of Perugia (2007)
Pelachaud, C.: Multimodal expressive embodied conversational agents. In: Proceedings of the 13th annual ACM international conference on Multimedia. ACM, Hilton (2005)
Plutchik, R.: A general psychoevolutionary theory of emotion. In: Plutchik, R. (ed.) Emotion: Theory, research, and experience, Theories of emotion, Academic, New York. vol. 1, pp. 3–33 (1980)
Russell, J.A.: A circumplex model of affect. Journal of Personality and Social Psychology 39(6), 1161–1178 (1980)
Russell, J.A., Mehrabian, A.: Evidence for a three-factor theory of emotions. Journal of Research in Personality 11, 273–294 (1977)
Scherer, K.R.: The nonverbal dimension: A fad, a field, or a behavioral modality? The social dimension: European developments in social psychology, 160–183 (1984)
Scherer, K.R.: Analyzing Emotion Blends. In: Fischer, A. (ed.) Proceedings of the Xth Conference of the International Society for Research on Emotions, pp. 142–148 (1998)
Scherer, K.R.: Emotion. In: Stroebe, M.H.W. (ed.) Introduction to Social Psychology: A European perspective, pp. 151–191. Blackwell, Oxford (2000)
Scherer, K.R., Ellgring, H.: Are facial expressions of emotion produced by categorical affect programs or dynamically driven by appraisal? Emotion 7(1), 113–130 (2007)
Scherer, K.R., Ellgring, H.: Multimodal Expression of Emotion: Affect Programs or Componential Appraisal Patterns? Emotion 7(1) (2007)
Wallbott, H.G.: Bodily expression of emotion. European Journal of Social Psychology 28, 879–896 (1998)
Woo, W., Park, J., Iwadate, Y.: Emotion Analysis from Dance Performance Using Time-Delay Neural Networks. In: JCIS-CVPRIP 2000, pp. 374–377 (2000)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Clavel, C., Plessier, J., Martin, JC., Ach, L., Morel, B. (2009). Combining Facial and Postural Expressions of Emotions in a Virtual Character. In: Ruttkay, Z., Kipp, M., Nijholt, A., Vilhjálmsson, H.H. (eds) Intelligent Virtual Agents. IVA 2009. Lecture Notes in Computer Science(), vol 5773. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-04380-2_31
Download citation
DOI: https://doi.org/10.1007/978-3-642-04380-2_31
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-04379-6
Online ISBN: 978-3-642-04380-2
eBook Packages: Computer ScienceComputer Science (R0)