Abstract
This work proposes a Bayesian approach to learn the behavior of human characters that give advice and help users to complete tasks in a situated environment. We apply Bayesian Inverse Reinforcement Learning (BIRL) to infer this behavior in the context of a serious game, given evidence in the form of stored dialogues provided by experts who play the role of several conversational agents in the game. We show that the proposed approach converges relatively quickly and that it outperforms two baseline systems, including a dialogue manager trained to provide “locally” optimal decisions.
Access provided by Autonomous University of Puebla. Download to read the full chapter text
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Levin, E., Pieraccini, R., Eckert, W.: A stochastic model of human-machine interaction for learning dialog strategies. IEEE Transactions on Speech and Audio Processing 8(1), 11–23 (2000)
Rieser, V., Lemon, O.: Reinforcement learning for adaptive dialogue systems. Springer (2011)
Pietquin, O., Dutoit, T.: A probabilistic framework for dialog simulation and optimal strategy learning. IEEE Transactions on Audio, Speech, and Language Processing 14(2), 589–599 (2006)
Cuayáhuitl, H.: Hierarchical reinforcement learning for spoken dialogue systems. PhD thesis, Citeseer (2009)
Williams, J.D., Young, S.: Partially observable markov decision processes for spoken dialog systems. Computer Speech & Language 21(2), 393–422 (2007)
Paek, T., Pieraccini, R.: Automating spoken dialogue management design using machine learning: An industry perspective. Speech Communication 50(8), 716–729 (2008)
Ng, A.Y., Russell, S.J.: Algorithms for inverse reinforcement learning. In: Icml, pp. 663–670 (2000)
Ramachandran, D., Amir, E.: Bayesian inverse reinforcement learning. Urbana 51, 61801 (2007)
Michini, B., How, J.P.: Improving the efficiency of bayesian inverse reinforcement learning. In: IEEE International Conference on Robotics and Automation (ICRA), pp. 3651–3656. IEEE (2012)
Walker, M.A.: An application of reinforcement learning to dialogue strategy selection in a spoken dialogue system. Journal of Artificial Intelligence Research 12, 387–416 (2000)
Young, S., Gašić, M., Keizer, S., Mairesse, F., Schatzmann, J., Thomson, B., Yu, K.: The hidden information state model: A practical framework for pomdp-based spoken dialogue management. Computer Speech & Language 24(2), 150–174 (2010)
Tetreault, J.R., Litman, D.J.: A reinforcement learning approach to evaluating state representations in spoken dialogue systems. Speech Communication 50(8), 683–696 (2008)
Rojas Barahona, L.M., Lorenzo, A., Gardent, C.: An end-to-end evaluation of two situated dialog systems. In: Proceedings of the 13th Annual Meeting of the Special Interest Group on Discourse and Dialogue, pp. 10–19. Association for Computational Linguistics, Seoul (2012)
Abbeel, P., Ng, A.Y.: Apprenticeship learning via inverse reinforcement learning. In: Proceedings of the Twenty-First International Conference on Machine Learning, p. 1. ACM (2004)
Abbeel, P., Coates, A., Ng, A.Y.: Autonomous helicopter aerobatics through apprenticeship learning. The International Journal of Robotics Research 29(13), 1608–1639 (2010)
Chandramohan, S., Geist, M., Lefevre, F., Pietquin, O., et al.: User simulation in dialogue systems using inverse reinforcement learning. In: Proceedings of the 12th Annual Conference of the International Speech Communication Association, pp. 1025–1028 (2011)
Boularias, A., Chinaei, H.R., Chaibdraa, B.: Learning the reward model of dialogue pomdps from data. In: NIPS Workshop on Machine Learning for Assistive Techniques, Citeseer (2010)
Zhifei, S., Joo, E.M.: A review of inverse reinforcement learning theory and recent advances. In: 2012 IEEE Congress on Evolutionary Computation (CEC), pp. 1–8. IEEE (2012)
Rojas-Barahona, L.M., Lorenzo, A., Gardent, C.: Building and exploiting a corpus of dialog interactions between french speaking virtual and human agents. In: Proceedings of the 8th International Conference on Language Resources and Evaluation (2012)
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Rojas-Barahona, L.M., Cerisara, C. (2014). Bayesian Inverse Reinforcement Learning for Modeling Conversational Agents in a Virtual Environment. In: Gelbukh, A. (eds) Computational Linguistics and Intelligent Text Processing. CICLing 2014. Lecture Notes in Computer Science, vol 8403. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-54906-9_41
Download citation
DOI: https://doi.org/10.1007/978-3-642-54906-9_41
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-54905-2
Online ISBN: 978-3-642-54906-9
eBook Packages: Computer ScienceComputer Science (R0)