Abstract
This paper introduces a new model of attentional state in task-oriented human-machine interaction. It integrates three lines of research: (i) neurocognitive understanding of the focus of attention in working memory, (ii) the notion of attention related to the theory of discourse structure in the field of computational linguistics, and (iii) investigation of a corpus that comprises recordings of spontaneous speech-based human-machine interaction. The underlying idea was to make a computationally appropriate representation of attentional information that imitates the function of a focus of attention in human perception. The introduced model addresses both the research questions of storage and processing of attentional information. Finally, the paper illustrates the model for concrete interaction domains, and discusses its implementation within a prototype spoken dialogue system.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
References
Awh E, Jonides J (2001) Overlapping mechanisms of attention and spatial working memory. Trends Cogn Sci 5(3):119–126
Atkinson RC, Shiffrin RM (1968) Human memory: A proposed system and its control processes. In: Spence KW, Spence JT (eds) The psychology of learning and motivation: Advances in research and theory, vol 2. Academic Press, New York, pp 89–195
Baddeley AD (1993) Working memory or working attention? In: Baddeley AD, Weiskrantz L (eds) Attention: Selection, awareness, and control. A tribute to Donald Broadbent. Oxford University Press, New York, pp 152–170
Baddeley AD, Hitch GJ (1974) Working memory. In: Bower GH (ed) The psychology of learning and motivation: Advances in research and theory, vol 8. Academic Press, New York, pp 47–89
Baddeley AD, Logie RH (1999) Working memory: The multiple component model. In: Miyake A, Shah P (eds) Models of working memory. Cambridge University Press, New York, pp 28–61
Bledowski C, Rahm B, Rowe BR (2009) What ‘works’ in working memory? Separate systems for selection and updating of critical information. J Neurosci 29:13735–13741
Bledowski C, Kaiser J, Rahm B (2010) Basic operations in working memory: Contributions from functional imaging studies. Behav Brain Res 214(2):172–179
Broadbent DE (1958) Perception and communication. Pergamon Press, New York
Campbell N (2006) On the structure of spoken language. In: Proceedings of the 3rd international conference on speech prosody 2006, Dresden, Germany, 4 pages
Campbell N (2007) Towards conversational speech synthesis; lessons learned from the expressive speech processing project. In: Proceedings of the sixth ISCA workshop on speech synthesis (SSW6), Bonn, Germany, pp 22–27
Cowan N (1988) Evolving conceptions of memory storage selective attention, and their mutual constraints within the information processing system. Psychol Bull 104(2):163–191
Cowan N (2001) The magical number 4 in short-term memory: A reconsideration of mental storage capacity. Behav Brain Sci 24:87–185
Cowan N, Fristoe NM, Elliot EM, Brunner RP, Saults JS (2006) Scope of attention, control of attention, and intelligence in children and adults. Mem Cogn 34:1754–1768
Engle RW, Kane MJ (2004) Executive attention, working memory capacity, and a two-factor theory of cognitive control. In: Ross B (ed) The psychology of learning and motivation, vol 44. Elsevier, New York, pp 145–199
Engle RW, Cantor J, Carullo JJ (1992) Individual differences in working memory and comprehension: A test of four hypotheses. J Exp Psychol Learn Mem Cogn 18:972–992
Ericsson KA, Kintsch W (1995) Long-term working memory. Psychol Rev 102:211–245
Gnjatović M (2009) Adaptive dialogue management in human-machine interaction. Verlag Dr Hut, Munich
Gnjatović M, Rösner D (2007) An approach to processing of user’s commands in human-machine interaction. In: Proceedings of the 3rd language and technology conference (LTC’07), Adam Mickiewicz University, Poznan, Poland, pp 152–156
Gnjatović M, Rösner D (2008) Adaptive dialogue management in the NIMITEK prototype system. In: Proceedings of the 4th IEEE tutorial and research workshop perception and interactive technologies for speech-based systems (PIT’08), Lecture notes in computer science, vol 5078. Springer, Berlin, pp 14–25
Gnjatović M, Rösner D (2010) Inducing genuine emotions in simulated speech-based human-machine interaction: The nimitek corpus. IEEE Trans Affect Comput 1:132–144
Gnjatović M, Pekar D, Delić V (2011) Naturalness, adaptation and cooperativeness in spoken dialogue systems. In: Toward autonomous, adaptive, and context-aware multimodal interfaces, theoretical and practical issues, Lecture notes in computer science, vol 6456. Springer, Berlin, pp 298–304
Griffin IC, Nobre AC (2003) Orienting attention to locations in internal representations. J Cogn Neurosci 15(8):1176–1194
Grosz B, Sidner C (1986) Attention, intentions, and the structure of discourse. Comput Linguist 12(3):175–204
Halliday M (1994) An introduction to functional grammar, 2nd edn. Edward Arnold, London
Hovy E, McCoy K (1989) Focusing your RST: A step toward generating coherent, multisentential text. In: Proceedings of the 11th annual conference of the cognitive science society, Ann Arbor
Jokinen K (2009) Constructive dialogue modelling: Speech interaction and rational agents. Wiley, New York
Jokinen K, Tanaka H, Yokoo A (1998) Context management with topics for spoken dialogue systems. In: Proceedings of COLING-ACL 1998, pp 631–637
Just MA, Carpenter PA (1992) A capacity theory of comprehension: Individual differences in working memory. Psychol Rev 99:122–149
Kane MJ, Conway ARA, Hambrick DZ, Engle RW (2007) Variation in working memory capacity as variation in executive attention and control. In: Conway ARA, Jarrold C, Kane MJ, Miyake A, Towse JN (eds) Variation in working memory. Oxford University Press, Oxford, pp 21–48
Kari L, Rozenberg G (2008) The many facets of natural computing. Commun ACM 51(10):72–83
Kirschner M (2007) Applying a focus tree model of dialogue context to interactive question answering. In: Proceedings of the ESSLLI’07 student session, Dublin, Ireland, pp 135–147
Lecœuche R, Robertson D, Barry C, Mellish C (2000) Evaluating focus theories for dialogue management. Int J Hum-Comput Stud 52(1):23–76
Li S-T, Tsai F-C (2010) Constructing tree-based knowledge structures from text corpus. Appl Intell 33(1):67–78
McCoy K, Cheng J (1991) Focus of attention: Constraining what can be said next. In: Paris CL, Swartout WR, Moore WC (eds) Natural language generation in artificial intelligence and computational linguistics. Kluwer Academic, Norwell, pp 103–124
Mendonça M, de Arruda LVR, Neves F (2011) Autonomous navigation system using event driven-fuzzy cognitive maps. Appl Intell. doi:10.1007/s10489-011-0320-1
Miller GA (1956) The magical number seven, plus or minus two: Some limits on our capacity for processing information. Psychol Rev 63:81–97
Moeller J-U (1996) Domain related focus-shifting constraints in dialogues with knowledge based systems. In: Adorni G, Zock M (eds) Trends in natural language generation: An artificial intelligence perspective. Springer, Berlin, pp 188–204
Mogle JA, Lovett BJ, Stawski RS, Sliwinski MJ (2008) What’s so special about working memory? An examination of the relationship among working memory, secondary memory, and fluid intelligence. Psychol Sci 19:1071–1077
Mohammad Y, Nishida T (2010) Controlling gaze with an embodied interactive control architecture. Appl Intell 32(2):148–163
Moubaiddin A, Obeid N (2009) Partial information basis for agent-based collaborative dialogue. Appl Intell 30(2):142–167
Nobre CA, Coull TJ, Maquet P, Frith DC, Vandenberghe R, Mesulam MM (2004) Orienting attention to locations in perceptual versus mental representations. J Cogn Neurosci 16(3):363–373
Oberauer K (2002) Access to information in working memory: Exploring the focus of attention. J Exp Psychol Learn Mem Cogn 28(3):411–421
Oberauer K, Lange EB (2009) Activation and binding in verbal working memory: A dual-process model for the recognition of nonwords. Cogn Psychol 58(1):102–136
O’Reilly RC, Braver TS, Cohen JD (1999) A biologically-based computational model of working memory. In: Miyake A, Shah P (eds) Models of working memory. Cambridge University Press, New York, pp 375–411
Rahwan I, McBurney P (2007) Argumentation technology. IEEE Intell Syst 22(6):21–23
Roulet E (1992) On the structure of conversation as negotiation. In: Mey JL, Parret H, Verschueren J (eds) (On) Searle on conversation. John Benjamins, Philadelphia, pp 91–99
Salthouse TA (1996) The processing-speed theory of adult age differences in cognition. Psychol Rev 103:403–428
Searle J (1992) Conversation. In: Mey JL, Parret H, Verschueren J (eds) (On) Searle on conversation. John Benjamins, Philadelphia, pp 7–29
Shah P, Miyake A (1999) Models of working memory: An introduction. In: Miyake A, Shah P (eds) Models of working memory: Mechanisms of active maintenance and executive control. Cambridge University Press, New York, pp 1–26
Stede M, Schlangen D (2004) Information-seeking chat: Dialogue management by topic structure. In: Proceedings of the 8th workshop on semantics and pragmatics of dialogue, CATALOG 04, Barcelona
Stoltzfus ER, Hasher L, Zacks RT (1996) Working memory and aging: Current status of the inhibitory view. In: Richardson JTE, Engle RW, Hasher L, Logie RH, Stoltzfus ER, Zacks RT (eds) Working memory and human cognition. Oxford University Press, New York, pp 66–88
Unsworth N, Engle RW (2007) The nature of individual differences in working memory capacity: Active maintenance in primary memory and controlled search from secondary memory. Psychol Rev 114:104–132
Unsworth N, Spillers GJ (2010) Working memory capacity: Attention control secondary memory, or both? A direct test of the dual-component model. J Mem Lang 62(4):392–406
Wheeler ME, Treisman AM (2002) Binding in short-term visual memory. J Exp Psychol Gen 131:48–64
Xu Y, Ohmoto Y, Okada S, Ueda K, Komatsu T, Okadome T, Kamei K, Sumi Y, Nishida T (2010) Formation conditions of mutual adaptation in human-agent collaborative interaction. Appl Intell. doi:10.1007/s10489-010-0255-y
Zhong N, Liu J, Yao Y (2010) Introduction to brain informatics. Cogn Syst Res 11(1):1–2
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Gnjatović, M., Janev, M. & Delić, V. Focus tree: modeling attentional information in task-oriented human-machine interaction. Appl Intell 37, 305–320 (2012). https://doi.org/10.1007/s10489-011-0329-5
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10489-011-0329-5