Abstract
This paper presents ongoing research on the modelling of expressive gesture in multimodal interaction and on the development of multimodal interactive systems explicitly taking into account the role of non-verbal expressive gesture in the communication process. In this perspective, a particular focus is on dance and music as first-class conveyors of expressive and emotional content. Research outputs include (i) computational models of expressive gesture, (ii) validation by means of continuous ratings on spectators exposed to real artistic stimuli, and (iii) novel hardware and software components for the EyesWeb open platform (www.eyesweb.org), such as the recently developed Expressive Gesture Processing Library. The paper starts with a definition of expressive gesture. A unifying framework for the analysis of expressive gesture is then proposed. Finally, two experiments on expressive gesture in dance and music are discussed. This research work has been supported by the EU IST project MEGA (Multisensory Expressive Gesture Applications, www.megaproject.org) and the EU MOSART TMR Network.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Bobick, A.F., Davis, J.: The Recognition of Human Movement Using Temporal Templates. IEEE Transactions on Pattern Analysis and Machine Intelligence 23(3), 257–267 (2001)
Bradsky, G., Davis, J.: Motion segmentation and pose recognition with motion history gradients. Machine Vision and Applications 13, 174–184 (2002)
Cadoz, C., Wanderley, M.: Gesture – Music. In: Wanderley, M., Battier, M. (eds.) Trends in Gestural Control of Music (Edition électronique), IRCAM, Paris (2000)
Camurri, A., Lagerlof, I., Volpe, G.: Emotions and cue extraction from dance movements. International Journal of Human Computer Studies 59(1-2), 213–225 (2003)
Camurri, A., De Poli, G., Leman, M.: MEGASE - A Multisensory Expressive Gesture Applications System Environment for Artistic Performances. In: Proc. Intl Conf CAST, GMD, St Augustin-Bonn, pp. 59 – 62 (2001)
Camurri, A., Hashimoto, S., Ricchetti, M., Trocca, R., Suzuki, K., Volpe, G.: EyesWeb – Toward Gesture and Affect Recognition in Interactive Dance and Music Systems. Computer Music Journal 24(1), 57–69 (2000)
Canazza, S., De Poli, G., Drioli, C., Rodà, A., Vidolin, A.: Audio morphing different expressive intentions for Multimedia Systems. IEEE Multimedia 3, 79–83 (2000)
Chi, D., Costa, M., Zhao, L., Badler, N.: The EMOTE model for Effort and Shape. In: ACM SIGGRAPH 2000, New Orleans, LA, pp. 173–182 (2000)
Clarke, E.F., Davidson, J.W.: The body in music as mediator between knowledge and action. In: Thomas, W. (ed.) Composition, Performance, Reception: Studies in the Creative Process in Music, pp. 74–92. Oxford University Press, Oxford (1998)
Cowie, R., Douglas-Cowie, E., Tsapatsoulis, N., Votsis, G., Kollias, S., Fellenz, W., Taylor, J.: Emotion Recognition in Human-Computer Interaction. IEEE Signal Processing Magazine (1) (2001)
Friberg, A., Colombo, V., Frydén, L., Sundberg, J.: Generating Musical Performances with Director Musices. Computer Music Journal 24(3), 23–29 (2000)
Hashimoto, S.: KANSEI as the Third Target of Information Processing and Related Topics in Japan. In: Camurri, A. (ed.) Proceedings of the International Workshop on KANSEI: The technology of emotion. AIMI (Italian Computer Music Association) and DIST-University of Genova, pp. 101–104 (1997)
Kilian, J.: Simple Image Analysis By Moments. OpenCV library documentation (2001)
Krumhansl, C.L.: A perceptual analysis of Mozart’s piano sonata K.282: Segmentation, tension and musical ideas. Music Perception 13(3), 401–432 (1996)
Krumhansl, C.L.: Can dance reflect the structural and expressive qualities of music? A perceptual experiment on Balanchine’s choreography of Mozart’s Divertimento. Musicae Scientiae 1(15), 63–85 (1997)
Kurtenbach, Hulteen: Gesture in Human-Computer Interaction. In: Laurel, B., (Ed.) The Art of Human-Computer Interface Design (1990)
Laban, R., Lawrence, F.C.: Effort. Macdonald&Evans Ltd., London (1947)
Laban, R.: Modern Educational Dance. Macdonald & Evans Ltd., London (1963)
Lagerlof, I., Djerf, M.: On cue utilization for emotion expression in dance movements. Manuscript in preparation, Department of Psychology, University of Uppsala (2001)
Liu, Y., Collins, R.T., Tsin, Y.: Gait Sequence Analysis using Frieze Patterns. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2351, pp. 657–671. Springer, Heidelberg (2002)
Lucas, B., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: Proceedings of the International Joint Conference on Artificial Intelligence (1981)
McNeill, D.: Hand and Mind: What Gestures Reveal About Thought. University Of Chicago Press, Chicago (1992)
Palmer, C.: Music Performance. Annual Review of Psychology 48, 115–138 (1997)
Pollick, F.E., Paterson, H., Bruderlin, A., Sanford, A.J.: Perceiving affect from arm movement. Cognition 82, B51–B61 (2001)
Scherer, K.R.: Why music does not produce basic emotions: pleading for a new approach to measuring the emotional effects of music. In: Proc. Stockholm Music Acoustics Conference SMAC 2003, KTH, Stockholm, Sweden, pp. 25–28 (2003)
Sloboda, J.A., Lehmann, A.C.: Tracking performance correlates of changes in perceived intensity of emotion during different interpretations of a Chopin piano prelude. Music Perception 19(1), 87–120 (2001)
Timmers, R.: Freedom and constraints in timing and ornamentation: investigations of music performance. Shaker Publishing, Maastricht (2002)
Wanderley, M., Battier, M. (eds.): Trends in Gestural Control of Music (Edition électronique). IRCAM, Paris (2000)
Zhao, L.: Synthesis and Acquisition of Laban Movement Analysis Qualitative Parameters for Communicative Gestures. Ph.D Dissertation, University of Pennsylvania (2001)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Camurri, A., Mazzarino, B., Ricchetti, M., Timmers, R., Volpe, G. (2004). Multimodal Analysis of Expressive Gesture in Music and Dance Performances. In: Camurri, A., Volpe, G. (eds) Gesture-Based Communication in Human-Computer Interaction. GW 2003. Lecture Notes in Computer Science(), vol 2915. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-24598-8_3
Download citation
DOI: https://doi.org/10.1007/978-3-540-24598-8_3
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-21072-6
Online ISBN: 978-3-540-24598-8
eBook Packages: Springer Book Archive