Abstract
This paper presents research using full body skeletal movements captured using video-based sensor technology developed by Vicon Motion Systems, to train a machine to identify different human emotions. The Vicon system uses a series of 6 cameras to capture lightweight markers placed on various points of the body in 3D space, and digitizes movement into x, y, and z displacement data. Gestural data from five subjects was collected depicting four emotions: sadness, joy, anger, and fear. Experimental results with different machine learning techniques show that automatic classification of this data ranges from 84% to 92% depending on how it is calculated. In order to put these automatic classification results into perspective a user study on the human perception of the same data was conducted with average classification accuracy of 93%.
Access provided by Autonomous University of Puebla. Download to read the full chapter text
Chapter PDF
Similar content being viewed by others
References
Kapoor, S.M., Picard, R.W.: Towards a Learning Companion that Recognizes Affect. In: Proc. Emotional and Intelligent II: The Tangled Knot of Social Cognition, AAAI Fall Symposium, North Falmouth, MA (November 2001)
Fernandez, R., Picard, R.W.: Modeling Driver’s Speech under Stress. In: Proc. ISCA Workshop on Speech and Emotions, Belfast (2000)
Pantic, M.: Toward an Affect-Sensitve Multimodal Human-Computer Interaction. In: Proc of the IEEE 91(9) (September2003)
Kang, B.S., Han, C.H., Lee, S.T., Youn, D.H., Lee, C.: Speaker dependent emotion recognition using speech signals. In: Proc ICSLP, pp. 383–386 (2000)
Cowie, R., Douglas-Cowie, E., Tsapatsoulis, N., Votsis, G., Kollias, S., Fellenz, W., Taylor, J.G.: Emotion recognition in human-computer interaction. IEEE Signal Processing Mag. 18, 32–80 (2001)
Ververidis, D., Kotropoulos, C., Pitas, I.: Automatic Emotional Speech Classification. In: Proc ICASSP, pp. 593–596 (2004)
Schiano, D.J., Ehrlich, S.M., Rahardja, K., Sheridan, K.: Face to interface: Facial affect in human and machine. In: Proc. CHI, pp. 193–200 (2000)
Essa, I., Pentland, A.: Coding analysis interpretation recognition of facial expressions. IEEE Trans. Pattern Anal. Machine Intell. 19, 757–763 (1997)
Blackand, M.J., Yacoob, Y.: Recognizing facial expressions in image sequences using local parameterized models of image motion. Int. J. Conmput. Vis. 25(1), 23–48 (1997)
Chen, L.S., Huang, T.S., Miyasato, T., Nakatsu, R.: Multimodal Human Emotion/Expression Recognition. In: Proc Third International Conference on Automatic Face and Gesture Recognition, Nara, Japan (1998)
De Silva, L.C., Miyasato, T., Nakatsu, R.: Facial emotion recognition using multimodal information. In: Proc FG, pp. 332–335 (2000)
Yoshitomi, Y., Kim, S., Kawano, T., Kitazoe, T.: Effect of sensor fusion for recognition of emotional states using voice, face image and thermal image of face. In: Proc. ROMAN, pp. 178–183 (2000)
Picard, R.W.: Towards Computers that Recognize and Respond to User Emotions. IBM System Journal 39, 705–719 (2001)
Picard, R.W., Healey, J.: Affective Wearables. Personal Technologies 1(4), 231–240 (1997)
Pollick, F.E., Paterson, H., Bruderlin, A., Sanford, A.J.: Perceiving affect from arm movement. Cognition 82, B51–B61 (2001)
Vines, M., Wanderley, M., Krumhansl, C., Nuzzo, R., Levirin, D.: Performance Gestures of Musicians: What Structural and Emotional Information do they Convey? In: Gesture-Based Communication in Human-Computer Interaction - 5th International Gesture Workshop, Genova, Italy (2003)
Wallbott, H.G.: Bodily expression of emotion. European Journal of Social Psychology 28, 879–896 (1998)
DeMeijer, M.: The contribution of general features of body movement to the attribution of emotions. Journal of Nonverbal Behavior 13, 247–268 (1989)
Woolard, A.: Vicon 512 User Manual, Vicon Motion Systems, Tustin CA (January 1999)
Ian, H., Frank, E., Kaufmann, M.: Data Mining: Practical machine learning tools with Java implementations, San Francisco (2000)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Kapur, A., Kapur, A., Virji-Babul, N., Tzanetakis, G., Driessen, P.F. (2005). Gesture-Based Affective Computing on Motion Capture Data. In: Tao, J., Tan, T., Picard, R.W. (eds) Affective Computing and Intelligent Interaction. ACII 2005. Lecture Notes in Computer Science, vol 3784. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11573548_1
Download citation
DOI: https://doi.org/10.1007/11573548_1
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-29621-8
Online ISBN: 978-3-540-32273-3
eBook Packages: Computer ScienceComputer Science (R0)