Keywords

1 Introduction

It seems reasonable to assume that agents will be more effective at achieving their goals during interactions if they understand the other entities involved. Understanding others requires the capability of modelling and reasoning about other agents’ minds. These characteristics are intrinsic to Theory of Mind (ToM) [10]. ToM is the ability of humans to ascribe elements such as beliefs, desires, and intentions, and relations between these elements to other human agents. In other words, it is the ability to form mental models of other agents.

The Multi-Agent Systems (MAS) community is showing increased interest in ToM [6, 7, 24]. One reason for this interest might be that ToM could boost the quality of communication between agents that need to exchange information in order to make decisions and reach meaningful agreements. By meaningful agreements we mean agreements that result from a mutual understanding. We consider mutual understanding to be represented by a certain set of shared beliefs reached through communication.

Various studies have investigated the use of ToM in MAS. Among them, [6, 7] investigated the advantages of using different levels of ToM in games played by agents, and [1, 11, 12, 20, 29], even though ToM is not mentioned, show the advantages of modelling the opponent when considering strategies in argumentation-based dialogues. All that work shows that modelling other agents’ minds is an important topic of research, and the results are important contributions to the MAS literature. However, as described in [32], most of the work on modelling other agents’ minds assume ToM as given. This is an understandable assumption, but it is nevertheless unrealistic given that there are no readily-available, practical techniques for developing such agents. Also, as a result of relying on such unrealistic assumption, the question of how agents acquire the model of other agents’ minds has not been fully investigated. In this work, we propose a formal semantics for updates that agents can effect to their ToM based on the communication that they have with other agents, thus allowing them to acquire a ToM.

Communication plays an important role in MAS [34], and takes places on multiple levels. Communicating content is only one part of the process of communication. It also includes forming the message in a way that will make the sender’s purpose of communication clear to the receivers [8]. In order to make the sender’s purpose clear, agent communication languages, such as FIPA-ACL [9] and KQML [8], have been proposed based on speech act theory. Both languages format message to include performatives in such a way that the sender’s purpose will be clear to the agent that is receiving the communication, facilitating the correct interpretation of the content of that communication. In this work we show that, based on the semantics of the agent communication languages, agents are able to infer the likely model of other agents’ minds, i.e., ToM, considering the meaning of each communication exchange. Using ToM acquired from communication, agents are able to reason and make decisions using other agents’ models.

The main contributions of this paper are: (i) an operational semantics, formally defined, for updates that agents carry out on their ToM during communication—to the best of our knowledge, our work is the first to propose a formal model of how agents acquire and update ToM during communication in multi-agent systems, particularly in the practical context of an Agent-Oriented Programming Language (AOPL) based on the BDI architecture; (ii) an approach for agent reasoning and decision making, and, in particular, we show how agents can reach shared beliefs more efficiently than when they are not able to model ToM.

2 Background

2.1 Agent Communication Languages

Agent communication languages have been developed based on speech act theory [30]. Speech act theory is concerned with the role of language as actions. In speech act theory, a speech act is composed by (i) a locution, which represents the physical utterance; (ii) an illocution, which provides the speaker intentions to the hearer; and (iii) the perlocution, which describes the actions that occur as a result of the illocution. For example, “I order you to shut the door” is a locution with an illocution of a command to shut the door, and the perlocution may be that the hearer shuts the door. Thus, an illocution is considered to have two parts, the illocutionary force and a proposition (content). The illocutionary force describes the type speech act used, e.g., assertive, directive, commissive, declarative, expressive.

Among the agent communication languages which emerged based on speech act theory, FIPA-ACL [9] and KQML [8] are the best known. In this work, for practical reasons, we choose KQML, which is the standard communication language in the Jason Platform [3], the multi-agent platform we choose to implement this work.

The Knowledge Query and Manipulation Language (KQML) was designed to support interaction among intelligent software agents, describing the message format and message-handling protocol to support run-time agent communication [8, 17]. In order to make KQML broadly applicable, in [16] a semantic framework for KQML was proposed. Considering the speech act semantics, they argue that it is necessary to consider the cognitive state of the agents that use these speech acts. Defining the semantics, the authors provided an unambiguous interpretation of (i) how the agents’ states change after sending and/or receiving a KQML performative, as well as (ii) the criteria under which the illocutionary point of the performative is satisfied (i.e., the communication was effective).

2.2 Agent Oriented Programming Languages

Among the many AOPLs and platforms, such as Jason, Jadex, Jack, AgentFactory, 2APL, GOAL, Golog, and MetateM, as discussed in [2], we chose the Jason platform [3] for our work. Jason extends the AgentSpeak language, an abstract logic-based AOPL introduced by Rao [28], which is one of the best-known languages inspired by the BDI architecture.

Besides specifying BDI agents with well-defined mental attitudes, the Jason platform [3] has some other features that are particularly interesting for our work, for example, strong negation, belief annotations, and (customisable) speech-act based communication. Strong negation helps the modelling of uncertainty, allowing the representation of things that the agent: (i) believes to be true, e.g., \(\mathtt {about(paper1,tom)}\); (ii) believes to be false, e.g., \(\mathtt {\lnot about(paper2,tom)}\); (iii) is ignorant about, i.e., the agent has no information about whether a paper is about \(\mathtt {tom}\) or not. Also, Jason automatically generates annotations for all the beliefs in the agents’ belief base about the source from where the belief was obtained (which can be from sensing the environment, communication with other agents, or a mental note created by the agent itself). The annotation has the following format: \(\mathtt {about(paper1,tom)[source(reviewer1)]}\), stating that the source of the belief that \(\mathtt {paper1}\) is about the topic \(\mathtt {tom}\) is \(\mathtt {reviewer1}\). The annotations in Jason can be easily extended to include other meta-information, for example, trust and time as used in [19, 21]. Another interesting feature of Jason is the communication between agents, which is done through a predefined (internal) action. There are a number of performatives allowing rich communication between agents in Jason, as explained in detail in [3]. Furthermore, new performatives can be easily defined (or redefined) in order to give special meaning to themFootnote 1, which is an essential characteristic for this work.

3 Running Example

As a running example, we will consider a scenario with five agents in a university. The first agent, named John, plays the role of a professor in the university, and the other agents, named Bob, Alice, Nick, and Ted, play the role of students. John has a relation of supervisor to the students. Also, John is responsible for distributing some tasks to the students. In order to distribute the tasks, John maintains information about the students, so as to distribute tasks to students that have the required knowledge for each task.

Our model can be described as \(\langle Ag, \mathcal {T}, \mathcal {A}, \mathcal {S} \rangle \), in which Ag represents the set of agents, \(\mathcal {T}\) the set of tasks of the kind \(\mathcal {T} \subseteq \mathcal {A} \times \mathcal {S}\), representing an action from \(\mathcal {A}\), requiring knowledge about a subset of subjects from \(\mathcal {S}\), that might be executed to achieve the task \(\mathcal {T}\). In our example, we consider the following actions, subjects, and tasks:

  • \(\mathcal {A} = \{ \mathtt {write\_paper}, \mathtt {review\_paper}, \mathtt {paper\_seminar} \}\)

  • \(\mathcal {S} = \{ \mathtt {mas}, \mathtt {kr}, \mathtt {tom} \}\)

  • \(\mathcal {T}\) = \( \left\{ \begin{array}{l} \mathtt {task(write\_paper,[mas,tom])} \\ \mathtt {task(review\_paper,[kr])} \\ \mathtt {task(paper\_seminar,[tom,mas])} \end{array} \right\} \)

For example, the task for writing a paper on the subjects multi-agent systems and theory of mind, \(\mathtt {task(write\_paper,[mas,tom])}\), requires competence on both subjects (\(\mathtt {mas}\) and \(\mathtt {tom}\)). Thus, this task should be assigned to a student (or a group of students) who knows both subjects.

4 Semantics for ToM in Agent Communication

4.1 The Basis for the Operational Semantics

To define the semantics for the updates agents execute in their ToM, we extend the original operational semantics of AgentSpeak [33], which is based on a widely used method for giving semantics to programming languages [27]. It is important to mention that we are interested in the operational semantics for the updates agents execute in their ToM, which considers the performatives (locutions) as computational instructions that operate successively on the states of agents [18]. The operational semantics is given by a set of inference rules. These inference rules define a transition relation between configurations represented by the tuple \(\langle ag ,C,M,T,s\rangle \), originally defined in [33], as follows:

  • \( ag \) is a set of beliefs \( bs \), a set of plans \(ps\), and a set of theories of minds ToM.

  • An agent’s circumstance C is a tuple \(\langle I, E, A \rangle \) where:

    • I is a set of intentions \(\{i, i', \ldots \}\). Each intention i is a stack of partially instantiated plans.

    • E is a set of events \(\{ ( te ,i), ( te ', i'), \ldots \}\). Each event is a pair \(( te ,i)\), where \( te \) is a triggering event and i is an intention—a stack of plans in case of an internal event, or the empty intention \(\mathsf {T}\) in case of an external event. An example is when the belief revision function (which is not part of the AgentSpeak interpreter but rather of the agent’s overall architecture), updates the belief base, the associated events—i.e., additions and deletions of beliefs—are included in this set. These are called external events; internal events are generated by additions or deletions of goals from plans currently executing.

    • A is a set of actions to be performed in the environment.

  • M is a tuple \(\langle In, Out \rangle \) whose components characterise the following aspects of communicating agents (note that communication is typically asynchronous):

    • In is the mail inbox: the multi-agent system runtime infrastructure includes all messages addressed to this agent in this set. Elements of this set have the form \(\langle mid , id , ilf , cnt \rangle \), where \( mid \) is a message identifier, \( id \) identifies the sender of the message, \( ilf \) is the illocutionary force of the message, and \( cnt \) its content: a (possibly singleton) set of AgentSpeak predicates or plans, depending on the illocutionary force of the message.

    • Out is where the agent posts messages it wishes to send; it is assumed that some underlying communication infrastructure handles the delivery of such messages. Messages in this set have exactly the same format as above, except that here \( id \) refers to the agent to which the message is to be sent.

  • When giving semantics to an AgentSpeak agent’s reasoning cycle, it is useful to have a structure which keeps track of temporary information that may be subsequently required within a reasoning cycle. In this particular work, we consider only \(T_\iota \), which records a particular intention being considered along the execution of one reasoning cycle.

  • The current step within an agent’s reasoning cycle is symbolically annotated by \(s \in \{ \mathsf {ProcMsg}, \mathsf {SelEv}, \mathsf {RelPl}, \mathsf {ApplPl}, \) \( \mathsf {SelAppl}, \mathsf {AddIM}, \mathsf {SelInt}, \mathsf {ExecInt}, \mathsf {ClrInt}\}\). These labels stand for, respectively: processing a message from the agent’s mail inbox, selecting an event from the set of events, retrieving all relevant plans, checking which of those are applicable, selecting one particular applicable plan (the intended means), adding the new intended means to the set of intentions, selecting an intention, executing the selected intention, and clearing an intention or intended means that may have finished in the previous step.

  • The semantics of AgentSpeak makes use of “selection functions” which allow for user-defined components of the agent architecture. We use here only the \(S_{M}\) function, as originally defined in [33]; the select message function is used to select one message from an agent’s mail inbox.

In the interests of readability, we adopt the following notation in the semantics rules:

  • If C is an AgentSpeak agent circumstance, we write \(C_E\) to make reference to the E component of C, and similarly for other components of the multi-agent system and of the configuration of each agent.

  • We write \(b[s( id )]\) to identify the origin of a belief, where \( id \) is an agent identifier (s refers to source)

4.2 Tell Performative

It is important to note that when we consider agents that are able to model other agents’ minds during communication, both sides, sender and receiver, execute updates in their ToM. The sender will be able to infer the likely model of the receiver’s mind after receiving the message, and the receiver will be able to infer the likely model of the sender based on the message received. In the semantics presented in [33], there are separate semantic rules for sending and receiving a message. We follow the same approach here.

Considering the \( Tell \) performative, when the sender agent sends a message to a receiver agent \( sid \) with the content \(\varphi \), first the sender checks if the receiver will believe that information , using a function \( func\_send \) (which we assume as given and is domain dependent), based on ToM it already has about the receiver and the relevant beliefs in its belief base \( ag _{ bs }\). The sender will also annotate this ToM belief with a label \(\gamma \) that represents, for example, the likelihood of the belief (i.e., a certainty on the expected state of mind). Note that \(\gamma \) represents an estimation of the uncertainty given that no absolute inference is possible in regards to an agent’s private state of mind.

figure a

After the agent updates its mail outbox \(M_{ Out }\) with the message, it updates its current intention to \( i[head\leftarrow h] \) (considering the action \(\mathtt {.send}( sid , Tell ,\varphi )\) that has already been executed), then it updates its ToM with the prediction of a belief , creating an event that may be treated in a later reasoning cycle, possibly forming a new goal for the agent based on this new information.

Conversely, when a receiver agent receives a \( Tell \) message from an agent \( sid \), first it checks whether the sender believes \(\varphi \) based on its previous ToM about the sender and the relevant information in its belief base. This expectation of a state of mind results from function \( func\_rec \). A label \(\gamma \) is used to annotate relevant information such as the confidence on the projected state of mind.

figure b

After that, the agent updates its mail inbox \(M_{ In }\), its belief base \( ag _{ bs }\) with this new information \(\varphi [s( sid )]\) (following the original semantics of AgentSpeak [33]), and it updates its ToM about the sender with . Both of these updates (on the ToM and the belief base) generate events to which the agent is able to react.

Note that the predictions resulting from \( func\_send \) and \( func\_rec \) can be different from the actual state of mind of the other agents. Therefore, a good prediction model, considering both the ToM and relevant information from the agents’ belief base, plays an important role when modelling ToM based on agent communication. Such models might consider the uncertainty present in agent communication, agents’ autonomy and self interest, trust relations, reliability, etc. Thus, there are many different ways to instantiate such a model, and our approach allows different models to be implemented through the user-defined \( func\_send \) and \( func\_rec \) functions. Proposing a particular model for uncertainty on ToM is out of the scope of this work. Therefore, we will omit \(\gamma \) in our examples. A model for uncertain ToM can be found in our work presented in [31].

Example: Considering the scenario introduced in Sect. 3, imagine that John meets his students every week in order to supervise their work. In a particular meeting with Alice, Alice asks John about the definition of ToM, and John responds Alice with the following message: \(\langle alice ,\mathtt {tell},\mathtt {definition(tom},\) “an approach to model others’ minds”\({\mathtt {)}}\rangle \). At that moment, John is able to model that Alice believes the definition of Theory of Mind as “an approach to model others’ minds”, i.e., John models according to the SndTell semantic rule. Also, when Alice receives the message, Alice is able to model that John believes on that definition for ToM, i.e., Alice models according to the Tell semantic rule.

4.3 Achieve Performative

Considering the \( Achieve \) performative, when a sender agent sends a message with the content \(\varphi \), it expects that the receiver agent will likely desire \(\varphi \). It can predict this result using its previous ToM about the receiver, , and the relevant information in its belief base, \( ag _{ bs }\), resulting in (where again \(\gamma \) is an estimation of how likely the receiver is to adopt that goal).

figure c

The sender agent updates its mail outbox \(M_{ Out }\), its current intention, its ToM about the receiver with the prediction , and an event is generated from the update in its ToM.

On the other hand, when a receiver agent receives an \( Achieve \) message, it can safely conclude that the sender desire \(\varphi \) itself, using its previous ToM about the sender and the relevant information from its belief base.

figure d

The receiver agent updates its mail inbox \(M_{ In }\) and its ToM about the sender, which generates an event . Also, another event \(+!\varphi \) is generated, and the agent is able to autonomously decide whether to achieve \(\varphi \) or not. In case it decides to achieve \(\varphi \), then the agent will look for a plan that achieves \(\varphi \) and make that plan one of its intentions.

Example: Continuing our scenario, imagine that during a meeting with Bob, John realises that it could be interesting for Bob to read a paper about multi-agent systems, so John sends the following message to Bob: \(\langle bob ,\mathtt {achieve},\mathtt {read(bob,paper\_mas)}\rangle \). At that time, John is able to model that Bob desires to read the paper, i.e., according to the SndAchieve semantic rule. Also, Bob is able to model that John desires that Bob reads the paper, i.e., according to the Achieve semantic rule. Bob is able to react to the event \(\mathtt {+!read(bob,paper\_mas)}\), searching for a plan to achieve that goal and turning the plan into one of Bob’s intentions. A simple plan, written in Jason, that Bob could use to achieve this goal is shown below:

figure e

The plan above says that, when an event of the type +!read(Ag,Paper) is generated, then if Ag unifies with the name of the agent executing this plan (obtained with .my_name(Ag)), and if the agent believes that its supervisor desires that it reads that paper (desires(Sup,read(Ag,Paper)) and supervisor(Sup,Ag)), then the agent will proceed to execute the action read(Paper). Note that the Achieve semantic rule provides the context (precondition) necessary for Bob to execute this plan, considering the unification \(\mathtt {\{ Ag \mapsto bob, \ Paper \mapsto paper\_mas, \ Sup \mapsto john \}}\) and that desires(john,read(bob,paper_mas)) is the code representation for .

4.4 Ask-If Performative

Considering the \( AskIf \) performative, when the sender agent sends a message with the content \(\varphi \), the only inference the agent can make is that the other agent will believe that the sender desires to know \(\varphi \), i.e., .

figure f

The sender agent updates its mail outbox \(M_{ Out }\), its current intention and its ToM about the receiver with the prediction , thus an event is generated from the update in its ToM. Conversely, when a receiver agent receives the message, it is able to infer that the sender desires to know \(\varphi \). After that, in both cases the agent updates its mental state similarly to the other semantic rules.

figure g

Example: Continuing our scenario, imagine that during a group meeting, John asks all students if they like paper seminars, using the following message: \(\langle \{bob, alice, nick, tom\} ,\mathtt {AskIf},\mathtt {like(Ag,paper\_seminar))}\rangle \). At that moment John considers that all students believe that John desires to know who likes paper seminars, , according to the SndAskIf semantic rule. Also, all students think that John desires to know who likes paper seminars, , according to the AskIf semantic rule. Two simple plans, written in Jason, that students could use to react the event generated by adding to their ToM is shown below:

figure h

The plans above say that an agent will tell John that it likes a particular task if it likes the task. Otherwise, an agent will tell John that it does not like that task. For example, Alice likes paper seminars, answering John with the following message: \(\langle john ,\mathtt {tell},\mathtt {like(alice,paper\_seminar)}\rangle \). In this case, John will update its ToM stating that Alice likes paper seminars, and Alice will update its ToM stating that John believes that she likes paper seminars , according to the Tell and SndTell semantic rules. In the future, as John has this information, it would be able to allocate a task to a student who likes that task.

5 Reaching Shared Beliefs Using ToM

In [33], the authors showed how agents are able to reach shared beliefs. That approach for agents reaching shared beliefs starts with an agent \(ag_i\), which believes in \(\varphi \), sending to another agent \(ag_j\) a tell message with the content it desires to became a shared belief, i.e., \(\langle ag_j, \mathtt {tell}, \varphi \rangle \). Thus, following the semantics in [33], agent \(ag_j\) will receive the message and update its belief base with \(\varphi [\mathtt {source(ag_i)}]\). Then, agent \(ag_i\) needs to send a message to agent \(ag_j\) to achieve that shared belief, i.e., \(\langle ag_j, \mathtt {achieve}, \varphi \rangle \), thus the agent \(ag_j\) is able to execute the same procedure, sending a tell message to the agent \(ag_i\) with \(\varphi \), i.e., \(\langle ag_i, \mathtt {tell}, \varphi \rangle \). Finally, agent \(ag_i\) receives this message and updates its belief base to \(\varphi [\mathtt {source(itself), source(ag_j)}]\), reaching the state of shared beliefs.

Definition 1

(Shared Beliefs [33]). An agent \(ag_i\) will reach a state of shared beliefs with another agent \(ag_j\) when, for a belief \(\varphi [S]\) with S the different sources of \(\varphi \), both \( itself \) and \(ag_j\) are sources of \(\varphi \), i.e., \(\mathtt {source(self), \ source(ag_j) \in S}\).

Considering agents that are able to model ToM, we are able to redefine the idea of shared beliefs, including the model of other agents’ minds, i.e., a ToM.

Definition 2

(Shared Beliefs using ToM). An agent \(ag_i\) will reach a state of shared beliefs with another agent \(ag_j\) when, for a belief \(\varphi \), it is able to match its own belief \(\varphi \) with a ToM about \(ag_j\) believing \(\varphi \), i.e., , with \(\gamma \) the parameter describing, for example, the certainty on ToM required to consider \(\varphi \) a shared belief.

When we assume that agents are cooperative, they trust each other, and the network infrastructure guarantees that messages will reach their intended receivers, we also are able to assume that there is no uncertainty of the ToM agents model about each other. Thus, we are able to ignore the label \(\gamma \), which aims to model uncertainty of ToM.

Proposition 1

(Reaching Shared beliefs—ToM without Uncertainty). Without uncertainty of ToM, agents able to model ToM are able to reach a state of shared beliefs faster (with fewer messages) than agents without this ability.

Proof

(sketch). Following the semantic rule SndTell, when an agent \(ag_i\) believes in \(\varphi \) and it is able to model ToM, then it is able to reach a state of a shared belief \(\varphi \) with another agent \(ag_j\) communicating a single message \(\langle ag_j, \mathtt {tell}, \varphi \rangle \) to \(ag_j\). When the agent \(ag_i\) sends this message, it updates its ToM with , reaching the state of shared beliefs according to the Definition 2. Agents that are not able to model ToM will need at least two messages, i.e., a tell message each, according to the semantics from [33] and Definition 1.

Example: Following the scenario introduced in Sect. 3, imagine that during the meetings John has had with his students, the students tell John which subjects they know more about, and John has the following information of his students, according to the Tell semantic rule:

$$\begin{aligned} \begin{array}{l} \left\{ \begin{array}{ll} \mathtt {knows(alice,tom)} &{} \mathtt {knows(bob,mas)} \\ \mathtt {believes(alice,knows(alice,tom))} &{} \mathtt {believes(bob,knows(bob,mas))} \\ \mathtt {knows(nick,kr)} &{} \mathtt {knows(ted)} \\ \mathtt {believes(nick,knows(nick,kr))} &{} \mathtt {believes(ted,knows(tom,[tom,mas]))} \end{array} \right\} \end{array} \end{aligned}$$

Given this knowledge and the tasks John wants to allocate to his students, John decides to assign the tasks as follows: \(\mathtt {task(write\_paper,[mas,tom])}\) to Ted, who knows about both subjects needed for completing that task, \(\mathtt {task(review\_paper,[kr])}\) to Nick, who is the only student able to execute that task, and grouping Alice and Bob for the task \(\mathtt {task(paper\_seminar,[tom,mas])}\). If Bob only knows mas and Alice only knows tom, then they need to share their knowledge in order to successfully perform the task.

Reaching Shared Beliefs: Alice and Bob need to work together to accomplish this particular task, which requires the subjects mas (Multi-Agent Systems) and tom (Theory of Mind). Bob only knows the subject of mas and Alice only knows the subject of tom. Considering that together Alice and Bob know both topics in order to help each other during the paper seminar, they decide to exchange knowledge about these topics. Thus, they might reach some shared beliefs (knowledge) about both topics. Note that, in this scenario, Alice and Bob assume that both are cooperating and both are rational. Thus, Alice starts the dialogue telling Bob that “Theory of Mind is an approach to model others’ minds”, i.e., \(\langle bob ,\mathtt {tell},\mathtt {def(tom,}\) “an approach to model others’ minds”\({\mathtt {)}}\rangle \). At that moment, following the semantic rule SndTell, Alice updates its ToM with the following information . When Bob receives this message, following the semantic rule Tell, Bob updates its belief base with the following information def(tom, “an approach to model others’ minds”), as well as its ToM about Alice with . By now, both Alice and Bob have reached a state of shared belief about the definition of tom, according to Definition 2. They proceed sharing the relevant information about each topic until they both feel confident about both topics. Reaching shared beliefs (knowledge) is important for this particular task, in which, when the audience asks them questions about the topics tom and mas, both Alice and Bob are able to answer the questions because they both have sufficient knowledge about the topics.

6 Future Work

The relation of trust between agents [19, 25, 26] is an interesting property agents could consider in a model for uncertain ToM. Our approach allows us to model uncertainty through the functions \( func\_rec ()\) and \( func\_send ()\), labelling the uncertainty of that information using \(\gamma \). Even though our approach allows us to model ToM that reflects uncertainty, we believe that the modelling of uncertain ToM is a task that falls beyond the scope of this particular paper and we thus leave it as future work.

Another aspect of ToM to be considered in future work is that ToM can also be inferred by agents from the environment by observing other agents’ actions. The modelling of ToM based on these aspects is part of our ongoing research and it faces some more complex issues such as the ones mentioned in [5]: “the slamming of a door communicates the slammer’s anger only when the intended observer of that act realises that the slammer wanted both to slam the door in his face and for the observer to believe that to be his intention”. This means that there is both uncertainty about the slammer’s intentions and uncertainty about the act of slamming the door, which could be caused either by an accidental shove or by natural means, which would not represent a communicative act and, therefore, observing such an event occur should not cause the observer to make any inference about the slammer’s mental state.

7 Related Work and Conclusions

As mentioned before, to the best of our knowledge, there is no work that explicitly and formally describes how agents acquire and update ToM during communication. However, our work is inspired by others who have investigated agents that use models of other agents in reasoning and decision making, e.g., [1, 6, 7, 11, 12, 20, 29]. Also, we took some inspiration from the STAPLE language, that seems to have ceased to be used. The STAPLE (Social and Team Agents Programming Language) language has its logic semantics based on joint intention theory [13]. STAPLE has the goal of reaching a fault-tolerant approach to programming teamwork, in which the authors argue that a team is more than a collection of individuals working together to achieve a common goal. The agents in a team must have shared goals as well as a shared mental state [15]. Thus, STAPLE enables agents to specify the models of other agents, as well as temporal properties of actions and events, allowing them to reason about group beliefs, team intentions, and team commitments [14]. Note that our approach is more general than that, in which ToM could be used to implement similar approaches to teamwork and scalable cooperation, which is a likely research direction for our work.

In this paper, we have defined the formal semantics for updates agents execute on their ToM during communication. The formal semantics uses components based on the BDI model and it is, therefore, broadly applicable to any BDI based AOPL. To the best of our knowledge, our work is the first to address a formal model for ToM in agent communication. We have showed not only how agents acquire and update ToM based on agent communication, but we have also shown how agents reason and make decisions using ToM through an illustrative scenario. The modelling, the implementation and the study of agents that are able to model other agents’ minds (i.e., ToM) goes beyond the current interests of the AI community, in which the main research scope is to implement rational and efficient software agents that are able to reason and make decisions in order to simulate and study the social behaviour of intelligent entities [24]. ToM is also regarded as important by other research communities that engage in the interdisciplinary study of communication, negotiation, social behaviour, and developmental psychology [4]. We consider that it would be very useful for these interdisciplinary communities to have the possibility to use AOPLs in order to study ToM or any other problems in which ToM plays a significant role.