Abstract
Backchannel feedback is an important kind of nonverbal feedback within face-to-face interaction that signals a person’s interest, attention and willingness to keep listening. Learning to predict when to give such feedback is one of the keys to creating natural and realistic virtual humans. Prediction models are traditionally learned from large corpora of annotated face-to-face interactions, but this approach has several limitations. Previously, we proposed a novel data collection method, Parasocial Consensus Sampling, which addresses these limitations. In this paper, we show that data collected in this manner can produce effective learned models. A subjective evaluation shows that the virtual human driven by the resulting probabilistic model significantly outperforms a previously published rule-based agent in terms of rapport, perceived accuracy and naturalness, and it is even better than the virtual human driven by real listeners’ behavior in some cases.
Access provided by Autonomous University of Puebla. Download to read the full chapter text
Chapter PDF
Similar content being viewed by others
References
Huang, L., Morency, L.-P., Gratch, J.: Parasocial Consensus Sampling: Combining Multiple Perspectives to Learn Virtual Human Behavior. In: Proceedings of 9th International Conference on Autonomous Agents and Multiagent Systems (2010)
Morency, L.-P., de Kok, I., Gratch, J.: Predicting Listener Backchannels: A Probabilistic Multimodal Approach. In: Prendinger, H., Lester, J.C., Ishizuka, M. (eds.) IVA 2008. LNCS (LNAI), vol. 5208, pp. 176–190. Springer, Heidelberg (2008)
Gratch, J., Wang, N., Gerten, J., Fast, E., Duffy, R.: Creating Rapport with Virtual Agents. In: Pelachaud, C., Martin, J.-C., André, E., Chollet, G., Karpouzis, K., Pelé, D. (eds.) IVA 2007. LNCS (LNAI), vol. 4722, pp. 125–138. Springer, Heidelberg (2007)
Ward, N., Tsukahara, W.: Prosodic features which cue backchannel responses in English and Japanese. J. Pragmatics 23, 1177–1207 (2000)
Gratch, J., Okhmatovskaia, A., Lamothe, F., Marsella, S., Morales, M., Werf, R.J., Morency, L.-P.: Virtual Rapport. In: Gratch, J., Young, M., Aylett, R.S., Ballin, D., Olivier, P. (eds.) IVA 2006. LNCS (LNAI), vol. 4133, pp. 14–27. Springer, Heidelberg (2006)
Gratch, J., Wang, N., Okhmatovskaia, A., Lamothe, F., Morales, M., Morency, L.-P.: Can Virtual humans be more engaging than real ones? In: Jacko, J.A. (ed.) HCI 2007. LNCS, vol. 4552, pp. 286–297. Springer, Heidelberg (2007)
Bailenson, J.N., Yee, N.: Digital Chameleons: Automatic assimilation of nonverbal gestures in immersive virtual environments. Psychological Science 16, 814–819 (2005)
Bailenson, J.N., Yee, N., Merget, D., Schroeder, R.: The Effect of Behavioral Realism and Form Realism of Real-Time Avatar Faces on Verbal Disclosure, Nonverbal Disclosure, Emotion Recogition, and Copresence in Dyadic Interaction. PRESENCE: Teleoperators and Virtual Environments 15(4), 359–372 (2006)
Lee, J., Marsella, S.: Learning a Model of Speaker Head Nods using Gesture Corpora. In: 8th International Conference on Autonomous Agents and Multiagent Systems (2009)
Jonsdottir, G.R., Thorisson, K.R., Nivel, E.: Learning Smooth, Human-Like Turntaking in Realtime Dialogue. In: Prendinger, H., Lester, J.C., Ishizuka, M. (eds.) IVA 2008. LNCS (LNAI), vol. 5208, pp. 162–175. Springer, Heidelberg (2008)
de Melo, C., Gratch, J.: Expression of Moral Emotions in Cooperating Agents. In: Ruttkay, Z., Kipp, M., Nijholt, A., Vilhjálmsson, H.H. (eds.) IVA 2009. LNCS, vol. 5773, pp. 301–307. Springer, Heidelberg (2009)
Bickmore, T., Puskar, K., Schlenk, E., Pfeifer, L., Sereika, S.: Maintaining Reality: Relational Agents for Antipsychotic Medication Adherence. J. Interacting with Computers special issue on Mental Health (2010)
Kang, S.-H., Gratch, J., and Watts, J. The Effect of Affective Iconic Realism on Anonymous Interactants’ Self-Disclosure. In: Proceedings of Interaction Conference for Human-Computer Interaction (2009)
Wang, N., Gratch, J.: Can a Virtual Human Build Rapport and Promote Learning? In: Proceedings of 14 International Conference on Artificial Intelligence in Education (2009)
Kipp, M., Neff, M., Kipp, K.H., Albrecht, I.: Towards natural gesture synthesis: Evaluating gesture units in a data-driven approach to gesture synthesis. In: Pelachaud, C., Martin, J.-C., André, E., Chollet, G., Karpouzis, K., Pelé, D. (eds.) IVA 2007. LNCS (LNAI), vol. 4722, pp. 15–28. Springer, Heidelberg (2007)
Cassell, J., Sullivan, J., Prevost, S., Churchill, E.F.: Embodied Conversational Agents. MIT Press, Cambridge (2000)
Gratch, J., Rickel, J., Andre, E., Badler, N., Cassell, J., Petajan, E.: Creating Interactive Virtual Humans: Some Assembly Required. IEEE Intelligent Systems, 54–63 (July/August 2000)
Vinayagamoorthy, V., Gillies, M., Steed, A., Tanguy, E., Pan, X., Loscos, C., Slater, M.: Building Expression into Virtual Characters. In: Eurographics 2006 (2006)
Horton, D., Wohl, R.R.: Mass communication and parasocial interaction: Observation on intimacy at a distance. Psychiatry 19, 215–229 (1954)
Levy, M.R., Watching, T.V.: News as parasocial interaction. J. Broadcasting 23, 60–80 (1979)
Heylen, D.: Understanding Speaker-Listener Interactions. In: Proceedings of 10th Annual Conference of the International Speech Communication Association (2009)
Bavelas, J.B., Coates, L., Johnson, T.: Listener Responses as a Collaborative Process: The Role of Gaze. J. Communication 52(3), 566–580 (2006)
Bavelas, J.B., Coates, L., Johnson, T.: Listeners as co-narrators. J. Personality and Social Psychology 79(6), 941–952 (2000)
Bernieri, F.J., Gillis, J.S., Davis, J.M., Grahe, J.E.: Dyad Rapport and the Accuracy of Its Judgment Across Situations: A Lens Model Analysis. J. Personality and Social Psychology 71(1), 110–129 (1996)
Gifford, R.: A Lens-Mapping Framework for Understanding the Encoding and Decoding of Interpersonal Dispositions in Nonverbal Behavior. J. Personality and Social Psychology 66(2), 398–412 (1994)
Morency, L.-P., et al.: Contextual Recognition of Head Gestures. In: Proceedings of 7th International Conference on Multimodal Interactions (2005)
Montare, A.: The simplest chronoscope: group and interindividual differences in visual reaction time. J. Perceptual and motor skills 108(1), 161–172 (2009)
Reaction time, http://en.wikipedia.org/wiki/Reaction_time
Houlberg, R.: Local television news audience and the para-social interaction. J. Broadcasting 28, 423–429 (1984)
Rubin, A.M., Perse, E.M., Powell, R.A.: Loneliness, para-social interaction, and local television news viewing. Human Communication Research 12, 155–180 (1985)
Yngve, V.: On Getting a Word in Edgewise. In: 6th Regional Meeting of the Chicago Linguistic Society, pp. 567–577.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Huang, L., Morency, LP., Gratch, J. (2010). Learning Backchannel Prediction Model from Parasocial Consensus Sampling: A Subjective Evaluation. In: Allbeck, J., Badler, N., Bickmore, T., Pelachaud, C., Safonova, A. (eds) Intelligent Virtual Agents. IVA 2010. Lecture Notes in Computer Science(), vol 6356. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-15892-6_17
Download citation
DOI: https://doi.org/10.1007/978-3-642-15892-6_17
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-15891-9
Online ISBN: 978-3-642-15892-6
eBook Packages: Computer ScienceComputer Science (R0)