Abstract
Recently, discriminative models using recurrent neural networks (RNNs) have shown good performance for dialog state tracking (DST). However, the models have difficulty in handling new dialog states unseen in model training. This paper proposes a fully data-driven approach to DST that can deal with unseen dialog states. The approach is based on an RNN with an attention mechanism. The model integrates two variants of RNNs: a decoder that detects an unseen value from a user’s utterance using cosine similarity between word vectors of the user’s utterance and that of the unseen value; and a sentinel mixture architecture that merges estimated dialog states of the previous turn and the current turn. We evaluated the proposed method using the second and the third dialog state tracking challenge (DSTC 2 and DSTC 3) datasets. Experimental results show that the proposed method achieved DST accuracy of 80.0% for all datasets and 61.2% for only unseen dataset without hand-crafted rules and re-training. For the unseen dataset, the use of the cosine similarity-based decoder leads to a 26.0-point improvement from conventional neural network-based DST. Moreover, the integration of the cosine similarity-based decoder and the sentinel mixture architecture leads to a further 2.1-point improvement.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Henderson M, Thomson B, Williams J (2014) The second dialog state tracking challenge. In: Proceedings of the 15th annual meeting of the special interest group on discourse and dialogue (SIGDIAL), p 263
Henderson M, Thomson B, Williams J (2014) The third dialog state tracking challenge. In: Spoken language technology workshop (SLT). IEEE, pp 324–329
Henderson M, Thomson B, Young S (2013) Deep neural network approach for the dialog state tracking challenge. In: Proceedings of the 14th annual meeting of the special interest group on discourse and dialogue (SIGDIAL), pp 467–471
Henderson M, Thomson B, Young S (2004) Word-based dialog state tracking with recurrent neural networks. In: Proceedings of the 15th annual meeting of the special interest group on discourse and dialogue (SIGDIAL), pp 292–299
Henderson M, Thomson B, Young S (2014) Robust dialog state tracking using delexicalised recurrent neural networks and unsupervised adaptation. In: Spoken language technology workshop (SLT). IEEE, pp 360–365
Hori T, Wang H, Hori C, Watanabe S, Harsham B, Roux JL, Hershey JR, Koji Y, Jing Y, Zhu Z et al (2016) Dialog state tracking with attention-based sequence-to-sequence learning. In: Spoken language technology workshop (SLT). IEEE, pp 552–558
Jang Y, Ham J, Lee BJ, Chang Y, Kim KE (2016) Neural dialog state tracker for large ontologies by attention mechanism. In: 2016 IEEE spoken language technology workshop (SLT), pp 531–537
Kim S, D’Haro LF, Banchs RE, Williams JD, Henderson M, Yoshino K (2016) The fifth dialog state tracking challenge. In: Proceedings of the 2016 IEEE workshop on spoken language technology (SLT)
Lee S (2013) Structured discriminative model for dialog state tracking. In: Proceedings of the 14th annual meeting of the special interest group on discourse and dialog (SIGDIAL), pp 442–451
Merity S, Xiong C, Bradbury J, Socher R (2016) Pointer sentinel mixture models. CoRR arXiv:abs/1609.07843
Pennington J, Socher R, Manning CD (2014) Glove: global vectors for word representation. In: Empirical Methods in Natural Language Processing (EMNLP), pp 1532–1543
Shi H, Ushio T, Endo M, Yamagami K, Horii N (2017) Convolutional neural networks for multi-topic dialog state tracking. In: Dialogues with social robots. Springer, Berlin, pp 451–463
Thomson B, Young S (2010) Bayesian update of dialogue state: a POMDP framework for spoken dialogue systems. Comput Speech Lang 24(4):562–588
Tokui S, Oono K, Hido S, Clayton J (2015) Chainer: a next-generation open source framework for deep learning. In: Proceedings of workshop on machine learning systems in the 29th annual conference on neural information processing systems (NIPS)
Wang Z, Lemon O (2013) A simple and generic belief tracking mechanism for the dialog state tracking challenge: on the believability of observed information. In: Proceedings of the 14th annual meeting of the special interest group on discourse and dialog (SIGDIAL), pp 423–432
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Yoshida, T., Iwata, K., Fujimura, H., Akamine, M. (2019). Dialog State Tracking for Unseen Values Using an Extended Attention Mechanism. In: D'Haro, L., Banchs, R., Li, H. (eds) 9th International Workshop on Spoken Dialogue System Technology. Lecture Notes in Electrical Engineering, vol 579. Springer, Singapore. https://doi.org/10.1007/978-981-13-9443-0_7
Download citation
DOI: https://doi.org/10.1007/978-981-13-9443-0_7
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-13-9442-3
Online ISBN: 978-981-13-9443-0
eBook Packages: Literature, Cultural and Media StudiesLiterature, Cultural and Media Studies (R0)