Abstract
Recently, there has been a significant amount of work on the recognition of emotions from visual, verbal or physiological information. Most approaches to emotion recognition so far concentrate, however, on a single modality while work on the integration of multimodal information, in particular on fusing physiological signals with verbal or visual data, is scarce. In this paper, we analyze various methods for fusing physiological and vocal information and compare the recognition results of the bimodal recognition approach with the results of the unimodal approach.
Access provided by Autonomous University of Puebla. Download to read the full chapter text
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Picard, R., Vyzas, E., Healy, J.: Toward machine emotional intelligence: Analysis of affective physiological state. IEEE Trans. Pattern Anal. and Machine Intell. 23, 1175–1191 (2001)
Cowie, R., Douglas-Cowie, E., Tsapatsoulis, N., Votsis, G., Kollias, S., Fellenz, W., Taylor, J.G.: Emotion recognition in human-computer interaction. IEEE Signal Processing Mag. 18, 32–80 (2001)
Nwe, T.L., Wei, F.S., Silva, L.C.D.: Speech based emotion classification. In: IEEE Region 10 International Conference on Electrical Electronic Technology, vol. 1, pp. 297–301 (2001)
Chen, L.S.: Joint processing of audio-visual information for the recognition of emotional expression in human-computer interaction. Ph.D. thesis, University of Illinois at Urbana-Champaign, Dept. of Electrical Engineering (2000)
Batliner, A., Zeissler, V., Frank, C., Adelhardt, J., Shi, R.P., Nöth, E.: We are not amused-but how do you know? user states in a multi-modal dialogue system. In: EUROSPEECH 2003, Geneva, pp. 733–736 (2003)
Nasoz, F., Alvarez, K., Lisetti, C., Finkelstein, N.: Emotion recognition from physiological signals for presence technologies. International Journal of Cognition, Technology, and Work - Special Issue on Presence 6(1) (2003)
Wagner, J., Kim, J., André, E.: From physiological signals to emotions: Implementing and comparing selected methods for feature extraction and classification. In: ICME 2005, Amsterdam (2005)
De Silva, L.C., Ng, P.C.: Bimodal emotion recognition. In: IEEE International Conf. on Automatic Face and Gesture Recognition, pp. 332–335 (2000)
Chen, L.S., Huang, T.S.: Emotional expressions in audiovisual human computer interaction. In: ICME 2000, pp. 423–426 (2000)
Zeng, Z., Tu, J., Liu, M., Zhang, T., Rizzolo, N., Zhang, Z., Huang, T.S., Roth, D., Levinson, S.: Bimodal HCI-related affect recognition. In: ICMI 2004 (2004)
Kim, J., André, E., Rehm, M., Vogt, T., Wagner, J.: Integrating information from speech and physiological signals to achieve emotional sensitivity. In: INTERSPEECH 2005, Lisbon, Portugal, pp. 809–812 (2005)
Lang, P.: The emotion probe: Studies of motivation and attention. American Psychologist 50(5), 372–385 (1995)
Douglas-Cowie, E., Devillers, L., Martin, J.C., Cowie, R., Savvidou, S., Abrilian, S., Cox, C.: Multimodal Databases of Everyday Emotion: Facing up to Complexity. In: InterSpeech, Lisbon (2005)
Haag, A., Goronzy, S., Schaich, P., Williams, J.: Emotion recognition using bio-sensors: First step towards an automatic system. In: Affective Dialogue Systems, Tutorial and Research Workshop, Kloster Irsee, Germany (2004)
Kim, K.H., Bang, S.W., Kim, S.R.: Emotion recognition system using short-term monitoring of physiological signals. Med. Biol. Eng. Comput. 42, 419–427 (2004)
Pan, J., Tompkins, W.: A real-time qrs detection algorithm. IEEE Trans. Biomed. Eng. 32 (1985)
Busso, C., Deng, Z., Yildirim, S., Bulut, M., Lee, C.H., Kazemzaden, A., Lee, S., Neumann, U., Narayanan, S.: Analysis of emotion recognition using facial expression, speech and multimodal information. In: ICMI 2004, State College, Pennsylvania, USA, pp. 205–211 (2004)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Kim, J., André, E. (2006). Emotion Recognition Using Physiological and Speech Signal in Short-Term Observation. In: André, E., Dybkjær, L., Minker, W., Neumann, H., Weber, M. (eds) Perception and Interactive Technologies. PIT 2006. Lecture Notes in Computer Science(), vol 4021. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11768029_6
Download citation
DOI: https://doi.org/10.1007/11768029_6
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-34743-9
Online ISBN: 978-3-540-34744-6
eBook Packages: Computer ScienceComputer Science (R0)