Abstract
Goal-directed action selection is the problem of what to do next in order to progress towards goal achievement. This problem is computationally more complex in case of joint action settings where two or more agents coordinate their actions in space and time to bring about a common goal: actions performed by one agent influence the action possibilities of the other agents, and ultimately the goal achievement. While humans apparently effortlessly engage in complex joint actions, a number of questions remain to be solved to achieve similar performances in artificial agents: How agents represent and understand actions being performed by others? How this understanding influences the choice of agent’s own future actions? How is the interaction process biased by prior information about the task? What is the role of more abstract cues such as others’ beliefs or intentions?
In the last few years, researchers in computational neuroscience have begun investigating how control-theoretic models of individual motor control can be extended to explain various complex social phenomena, including action and intention understanding, imitation and joint action. The two cornerstones of control-theoretic models of motor control are the goal-directed nature of action and a widespread use of internal modeling. Indeed, when the control-theoretic view is applied to the realm of social interactions, it is assumed that inverse and forward internal models used in individual action planning and control are re-enacted in simulation in order to understand others’ actions and to infer their intentions. This motor simulation view of social cognition has been adopted to explain a number of advanced mindreading abilities such as action, intention, and belief recognition, often in contrast with more classical cognitive theories - derived from rationality principles and conceptual theories of others’ minds - that emphasize the dichotomy between action and perception.
Here we embrace the idea that implementing mindreading abilities is a necessary step towards a more natural collaboration between humans and robots in joint tasks. To efficiently collaborate, agents need to continuously estimate their teammates’ proximal goals and distal intentions in order to choose what to do next. We present a probabilistic hierarchical architecture for joint action which takes inspiration from the idea of motor simulation above. The architecture models the casual relations between observables (e.g., observed movements) and their hidden causes (e.g., action goals, intentions and beliefs) at two deeply intertwined levels: at the lowest level the same circuitry used to execute my own actions is re-enacted in simulation to infer and predict (proximal) actions performed by my interaction partner, while the highest level encodes more abstract task representations which govern each agent’s observable behavior. Here we assume that the decision of what to do next can be taken by knowing 1) what the current task is and 2) what my teammate is currently doing. While these could be inferred via a costly (and inaccurate) process of inverting the generative model above, given the observed data, we will show how our organization facilitates such an inferential process by allowing agents to share a subset of hidden variables alleviating the need of complex inferential processes, such as explicit task allocation, or sophisticated communication strategies.
Chapter PDF
Similar content being viewed by others
References
Sebanz, N., Bekkering, H., Knoblich, G.: Joint action: bodies and minds moving together. Trends Cogn. Sci. 10(2), 70–76 (2006)
Newman-Norlund, R.D., Noordzij, M.L., Meulenbroek, R.G., Bekkering, H.: Exploring the brain basis of joint action: Co-ordination of actions, goals and intentions. Social Neuroscience 2(1), 48–65 (2007)
Sebanz, N., Knoblich, G.: Prediction in joint action: What, when, and where. Topics in Cognitive Science 1, 353–367 (2009)
Pickering, M.J., Garrod, S.: An integrated theory of language production and comprehension. Behavioral and Brain Sciences (fort)
Fong, T., Thorpe, C., Baur, C.: Collaboration, dialogue, human-robot interaction. Robotics Research, 255–266 (2003)
Breazeal, C.: Designing sociable robots. The MIT Press (2004)
Sun, R.: Cognition and multi-agent interaction: From cognitive modeling to social simulation. Cambridge University Press (2005)
Goodrich, M.A., Schultz, A.C.: Human-robot interaction: a survey. Foundations and Trends in Human-Computer Interaction 1(3), 203–275 (2007)
Cohen, P.R., Levesque, H.J.: Teamwork. Nous, 487–512 (1991)
Breazeal, C.: Social interactions in hri: the robot view. IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews 34(2), 181–186 (2004)
Baker, C.L., Saxe, R., Tenenbaum, J.B.: Action understanding as inverse planning. Cognition 113(3), 329–349 (2009)
Yoshida, W., Dolan, R.J., Friston, K.J.: Game theory of mind. PLoS Comput. Biol. 4(12), e1000254+ (2008)
Anderson, J.R., Bothell, D., Byrne, M.D., Douglass, S., Lebiere, C., Qin, Y.: An integrated theory of the mind. Psychological Review 111(4), 1036 (2004)
Rizzolatti, G., Craighero, L.: The mirror-neuron system. Annual Review of Neuroscience 27, 169–192 (2004)
Frith, C.D., Frith, U.: How we predict what other people are going to do. Brain Research 1079(1), 36–46 (2006)
Kilner, J.M., Friston, K.J., Frith, C.D.: Predictive coding: An account of the mirror neuron system. Cognitive Processing 8(3), 159–166 (2007)
Pezzulo, G., Candidi, M., Dindo, H., Barca, L.: Action simulation in the human brain: Twelve questions. New Ideas in Psychology (2013)
Grush, R.: The emulation theory of representation: motor control, imagery, and perception. Behavioral and Brain Sciences 27(3), 377–396 (2004)
Gardenfors, P.: Mind-reading as control theory. European Review 15(2), 223–240 (2007)
Wolpert, D.M., Ghahramani, Z.: Computational motor control. In: Gazzaniga, M. (ed.) The Cognitive Neurosciences III, pp. 485–494. MIT Press (2004)
Wolpert, D.M., Doya, K., Kawato, M.: A unifying computational framework for motor control and social interaction. Philos. Trans. R Soc. Lond. B Biol. Sci. 358(1431), 593–602 (2003)
Jordan, M.I., Wolpert, D.M.: Computational motor control. The Cognitive Neurosciences 601 (1999)
Fitzpatrick, P., Metta, G., Natale, L., Rao, S., Sandini, G.: Learning about objects through action-initial steps towards artificial cognition. In: Proceedings of IEEE International Conference on Robotics and Automation, ICRA 2003, vol. 3, pp. 3140–3145. IEEE (2003)
Dindo, H., Schillaci, G.: An Adaptive Probabilistic Approach to Goal-Level Imitation Learning. In: Proc. of the 2010 IEEE/RSJ International Conference on Intelligent RObots and Systems (IROS), October 18-22, pp. 4452–4457 (2010), doi:10.1109/IROS.2010.5654440
Dindo, H., Zambuto, D., Pezzulo, G.: Motor simulation via coupled internal models using sequential monte carlo. In: Proceedings of IJCAI 2011, pp. 2113–2119 (2011)
Pezzulo, G., Dindo, H.: What should I do next? using shared representations to solve interaction problems. Experimental Brain Research 211(3), 613–630 (2011)
Demiris, Y., Khadhouri, B.: Hierarchical attentive multiple models for execution and recognition (hammer). Robotics and Autonomous Systems Journal 54, 361–369 (2005)
Tomasello, M., Carpenter, M., Call, J., Behne, T., Moll, H.: Understanding and sharing intentions: the origins of cultural cognition. Behav. Brain Sci. 28(5), 675–691 (2005); discussion 691–735
Csibra, G., Gergely, G.: Obsessed with goals: Functions and mechanisms of teleological interpretation of actions in humans. Acta Psychologica 124, 60–78 (2007)
Cuijpers, R.H., van Schie, H.T., Koppen, M., Erlhagen, W., Bekkering, H.: Goals and means in action observation: a computational approach. Neural Netw. 19(3), 311–322 (2006)
Vesper, C., Butterfill, S., Knoblich, G., Sebanz, N.: A minimal architecture for joint action. Neural Networks 23(8-9), 998–1003 (2010)
Knoblich, G., Sebanz, N.: Evolving intentions for social interaction: from entrainment to joint action. Philos. Trans. R Soc. Lond. B Biol. Sci. 363(1499), 2021–2031 (2008)
Murphy, K.P.: Machine Learning: A Probabilistic Perspective. MIT Press (2012)
Doucet, A., Johansen, A.M.: A tutorial on particle filtering and smoothing: fifteen years later. In: Handbook of Nonlinear Filtering, pp. 656–704 (2009)
Friston, K.: Hierarchical models in the brain. PLoS Computational Biology 4(11), e1000211 (2008)
Rao, R.P., Ballard, D.H.: Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. Nat. Neurosci. 2(1), 79–87 (1999)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Dindo, H., Chella, A. (2013). What Will You Do Next? A Cognitive Model for Understanding Others’ Intentions Based on Shared Representations. In: Shumaker, R. (eds) Virtual Augmented and Mixed Reality. Designing and Developing Augmented and Virtual Environments. VAMR 2013. Lecture Notes in Computer Science, vol 8021. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-39405-8_29
Download citation
DOI: https://doi.org/10.1007/978-3-642-39405-8_29
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-39404-1
Online ISBN: 978-3-642-39405-8
eBook Packages: Computer ScienceComputer Science (R0)