Keywords

1 Introduction

The motivation mechanisms inspired from drives, personality traits, and emotions modulate the cognitive system of an agent to make it function better in a complex, unpredictable environment than it could with its cognitive system alone, i.e., to allow the agent to make better decisions, to learn more effectively, to interact more appropriately with others [1, 2].

The cognitive science implies the cognition (computation, “information processing psychology”, manipulation of data structures stored in memory, formal operations carried out on symbol structures), perception, and action [3,4,5]. Its basic aim is identifying the functional architecture of cognition, in terms of rules and representations as well as a form that is more analog and more biologically plausible, that mediate thought.

Researchers from artificial intelligence, computer science, brain and cognitive science, and psychology have been oriented, by the end of 1980s, towards a new field to build intelligent machines called embodied cognitive science or new artificial intelligence or behavior-based artificial intelligence [6].

The brain does not run ‘programs’: it does something entirely different, i.e., it does not do mathematical proofs, but controls behavior, to ensure our survival [6]. The researchers from these various disciplines agreed that intelligence always manifests itself in behavior and consequently that we must understand the behavior [6]. In fact, a particular attention must be given on thinking and high-level cognition focusing on the interaction with the real world. This interaction is always mediated by a body, i.e., the intelligence needs to be ‘embodied’. This, has rapidly changed the research disciplines of artificial intelligence and cognitive science towards a new research field which is exerting more and more its influence on psychology, neurobiology, and ethology, as well as engineering science.

By another way, throughout recorded human intellectual history, there has been active debate about the nature of the role of emotions or “passions” in human behavior [7], with the dominant view being that passions are a negative force in human behavior [8]. By contrast, some of the latest research has been characterized by a new appreciation of the positive functions served by emotions [9]. An appreciation for the positive functions is not entirely new in behavioral science. Darwin, in 1872, was one of the first to hypothesize the adaptive mechanisms through which emotion might guide human behavior [10].

The great interest and large investigations in decision-making research associated with the emergence of behavioral decision theory, then, largely ignored the role played by the irrationality part (related to affect, in general) in decision-making [7]. However, with the research developments particularly psychology-related fields from 1990s, a great interest have been oriented towards the role of the irrationality part (related to affect) in decision-making.

The aim of this paper is to suggest a framework for adaptive agent decision-making modeling of biological regulation and psychological mechanisms. It integrates drives, personality traits, and emotions in order to:

  • use this framework as a test bed to test, analyze, and compare different pertinent models of drives developed from biological regulation and survival of social organisms, personality traits developed from the field of personality psychology, and emotions, central aspects of biological regulation,

  • analyze the impacts (effects), to assess the variation consequences of different personality and emotion aspects on the decision agents make,

  • emphasize the adaptive behaviors emerging from agent-environment interactions.

This paper is an extended version of the research works, suggesting such a framework for adaptive agent decision-making modeling of biological regulation and psychological mechanisms, developed in [11]. In this extended version more details and numerous references are added throughout all the paper. In addition, in order to highlight important problems of the suggested framework and to detail the ways they are solved, a discussion is added. This discussion is mainly related to the adaptive decision-making behavior, the common currency that should have each system, and the suggested neural architecture.

Thus, first, a perception-action cycle scheme for the agent-environment interactions and the deduced framework for adaptive agent decision-making modeling are developed in Sect. 2. Second, motivation systems: drives, personality traits, and emotions are developed in Sect. 3. Third, a neural architecture implementation of the framework is suggested in Sect. 4. Then, in Sect. 5 first tests related to a stimulation-drive, for two different agent personalities, and the activation level of emotions are presented and analyzed. Finally, a discussion is given, in Sect. 6, in order to highlight important problems related to the adaptive decision-making behavior, the common currency that should have each system in the suggested framework, and the neural architecture, as well as to detail the ways they are solved.

2 Agent-Environment Interactions (Perception-Action Cycle)

The perception-action cycle scheme for the agent-environment interactions, suggested in Fig. 1, is inspired from [12].

Fig. 1.
figure 1

Agent-environment interactions (perception-action cycle).

Full size image

The agent and environment are coupled in two ways:

  • an informational function (which maps properties of the agent-environment system into informational variables, in accordance with laws of ecological perception-action approach to the control of behavior):

$$ {\mathbf{i}} =\uplambda\left( {\mathbf{e}} \right), $$
(1)

where i is a vector of informational variables, e is a vector of environmental variables;

  • an effector function (which transforms the vector of action variables into muscle activation patterns that produce forces in the environment, action is thus characterized as a relation defined over the agent, causal forces, and the environment):

$$ {\mathbf{f}} =\upbeta\left( {\mathbf{a}} \right), $$
(2)

where f is a vector of external forces, and a is a vector of agent state variables (which describes the current state of the action system).

Thus, the adaptive, goal-directed behavior emerges from these local interactions between an agent governed by the control laws Ψ and an environment governed by the physical laws Φ such as:

$$ \begin{aligned} {\dot{\mathbf{a}}} & =\Psi ({\mathbf{a,i}}\text{),} \\ {\dot{\mathbf{e}}} & =\Phi ({\mathbf{e}} ,{\mathbf{f}} ).\\ \end{aligned} $$
(3)

Indeed, adaptive behavior emerges from task dynamics or behavioral dynamics (information-based approach to perception, dynamical systems approach to action).

From this, the deduced framework for agent decision-making modeling in behavior-based systems is then suggested in Fig. 2.

Fig. 2.
figure 2

A framework for agent decision-making modeling in behavior-based systems.

Full size image

This framework integrates particularly the motivation systems: drives (homeostatic regulation), personality traits (five-factors model), and emotions (basic emotions).

3 Motivation Systems

In this Section, the motivation systems consists of:

  • an homeostatic regulation system implementing the drives of the agent,

  • an emotion system implementing the emotions of the agent,

  • a personality system implementing the personality traits of the agent.

3.1 Drives (Homeostatic Regulation)

The design of homeostatic regulation system is inspired by ethological views of the analogous process in animals [13, 14]. However, it is a simplified and idealized model of those discovered in living systems. The drive features are: its temporally cyclic behavior with three regimes (under-stimulated, homeostatic, and overwhelming), i.e., if no stimulation, a drive will tend to increase in intensity unless it is satiated. This is analogous to an animal’s degree of hunger or level of fatigue, both following a cyclical pattern [9, 15, 16].

3.2 Personality (Five-Factor Model)

Social psychologists believe that human behavior is determined by both a person’s characteristics and the social situation. They also believe that the social situation is frequently a stronger influence on behavior than are a person’s characteristics, [17]. From this purpose, it is very important to integrate the personality traits in the suggested framework.

The model in personality which appears to represent a major conceptual and empirical advance in the field of personality psychology is the five-factor model in personality (Extraversion, Agreeableness, Conscientiousness, Neuroticism, and Openness to experience) developed in [18].

3.3 Emotions (Basic Emotions)

Emotions are not a luxury, they play a role in communicating meanings to others, and they may also play the cognitive guidance role [9].

In fact, emotions are another important motivation system for complex organisms. They seem to be centrally involved in determining the behavioral reaction to environmental (often social) and internal events of major significance for the needs and goals of a creature [19, 20].

4 Neural Architecture Implementation

In the design and achievement of the architecture of the suggested framework for agent decision-making modeling, first, carry out the aspects of computation and cognition (issues in the foundations of cognitive science) related to cognitive functional architecture, respecting the biological and psychological nature under the cognitive impenetrability condition (non influence by purely cognitive factors as goals, beliefs, inferences, tacit knowledge, …) is of great importance [3, 4]. This allows the fixed capacities of mind (called its functional architecture) avoiding the particular representations and algorithms used on specific occasions. This in turn requires that the fixed architectural function and the algorithms be independently validated in order to examine the fundamental distinction between a behavior governed by rules and representations, and a behavior that is merely the result of the causal structure of the underlying biological system.

Second importance is related to the neural aspect of functional architecture which is highly distributed network of interacting neurons [5, 16, 21].

The basic computational process, implemented as a value based system (influences graded in intensity, instead of simply being “on” or “off”), is modeled by its activation level Ai which is computed by Eq. (4).

$$ \text{A}_{\text{i}} = (\sum\limits_{{{\text{j}} = 1}}^{\text{n}} {\upomega_{\text{ji}} \, \text{i}_{\text{j}} } ) + {\text{b,}} $$
(4)

where the inputs ij are integer values, the weights ωji, and the bias b, over the number of inputs n. The process is active when the activation level Ai exceeds a threshold T.

The weights can be either positive or negative; a positive weight corresponds to an excitatory connection and a negative weight corresponds to an inhibitory connection.

Each perceptual unit, drive, emotion, personality, behavior, and motor process is modeled as a different type that is specifically tailored for its role in the overall architecture.

4.1 Perceptual Units

The antecedent conditions come through the perceptual system where they are assessed with respect to the agent’s “well being” and active goals. Thus, the perceptual system, which is inspired from [14], is built of perceptual units:

  • related to stimulus (from moving object, another agent, human) as shown in Fig. 3(a), i.e., there is a set of perceptual units defined for each stimulus, that indicate its presence (time: short, medium, long), absence (time: short, medium, long), nature (moving object, another agent, or human), quality (intensity of stimulus: too low, just right, too high), desirability (desired or not desired),

    Fig. 3.
    figure 3

    (a) Perceptual unit implementation related to stimulus. (b) Perceptual unit implementation related to drives.

    Full size image

  • related to drives (stimulation drive, social drive, …) as shown in Fig. 3(b), i.e., there is a perceptual unit defined for each regime (under-stimulated, homeostatic, overwhelming) of each drive, to represent how well each drive is being satiated,

  • related to behaviors as shown in Fig. 4(a), i.e., there is a set of perceptual units defined for each behavior, that indicate whether its goal has been achieved or not, and if not, then for how long.

    Fig. 4.
    figure 4

    (a) Perceptual unit implementation related to behaviors. (b) Drive implementation (AStimulation-Drive).

    Full size image

Note that the perceptual unit implementation is designed for each regime k (under-stimulated, homeostatic, overwhelming) of each drive i (stimulation-drive, social-drive, …).

4.2 Drives

The drive model given in Fig. 4(b) concerns the Stimulation-Drive leading to a cyclic behavior of a drive, where t is a temporal input; given no stimulation, a drive will tend to increase in intensity (AStimulation-Drive) unless it is satiated (homeostatic regime). Note that a similar drive model will be implemented in future concerning Social-Drive.

4.3 Personality

The five-factor model in personality, suggested in Fig. 5, is inspired from [18, 22, 23] implying five broad dimensions which are used to describe human personality: Openness to experience (Op), Agreeableness (Ag), Conscientiousness (Co), Extraversion (Ex), and Neuroticism (Ne).

Fig. 5.
figure 5

Personality model.

Full size image

Thus, combined personality value Pi affecting a behavior is defined in Eq. (5) as:

$$ {\text{P}}_{\text{i}} = (\sum\limits_{{{\text{j}} = 1}}^{5} {\upomega_{\text{ji}} {\text{ T}}_{\text{j}} } ) + {\text{b}}, $$
(5)

where Tj denotes the intensity of each personality parameter, and ωji the influence (inverse influence–1, no influence 0, direct influence +1) on a particular behavior.

4.4 Emotions

The relations between emotions and behavioural responses, i.e., under what conditions certain “emotions” and behavioural responses arise, are given in Table 1. This table is derived from the evolutionary, cross-species, and social functions hypothesized by [10, 19, 20]. Then, the perceptual behavioral or motivational information is tagged (arousal, valence, stance) with affective information [9]. Note that the stance is related to how approachable the percept is to the agent. Moreover, each regime of a drive biases arousal and valence differently, contributing to the activation of different emotions.

Table 1. Relations between emotions and behaviors under antecedent conditions.
Full size table

5 Tests, and Results

In this Section, first tests related to a Stimulation-Drive, from a Stimulus of a moving object, presented in Fig. 6, for two different agent personalities, and the activation level of emotions are presented and analyzed. Influence and intensity parameters presented in Table 2 concern two different personalities (since personality trait of Neuroticism has been found to influence avoidance behavior [24], Escape in Table 1 in our concern, and Agreeableness to inversely influence it):

Fig. 6.
figure 6

The used Stimulus (A-satiatory-stimulus).

Full size image
Table 2. Influence and intensity of personality parameters, Openness to experience (Op), Agreeableness (Ag), Conscientiousness (Co), Extraversion (Ex), and Neuroticism (Ne).
Full size table

In Fig. 8, the activation of emotions: Sadness (corresponding to prolonged absence of a desired stimulus, as shown in Table 1), followed by Interest (appearance of a desired stimulus), then Fear (presence of a threatening, corresponding to overwhelming stimulus, as shown in Fig. 7), followed by Interest (appearance of a desired stimulus), and finally Sadness (prolonged absence of a desired stimulus).

Fig. 7.
figure 7

Stimulation-drive results (from the stimulus, Fig. 6).

Full size image

Note that In Fig. 8, Fear appears and crosses Interest the first time, around t = 65 s, for Agent-Personality1.

Fig. 8.
figure 8

Activation of emotions in case of Agent-Personality1 (from stimulation-drive Fig. 7).

Full size image

In Fig. 9, Fear appears and crosses Interest the first time, around t = 63 s, for Agent-Personality2. Note that in this case Fear appears and crosses Interest earlier than in the case of the Agent-Personality1; moreover, Fear in the case of the Agent-Personality2 reaches more intensity than in the case of the Agent-Personality1.

Fig. 9.
figure 9

Activation of emotions in case of Agent-Personality2 (from the same stimulus, Fig. 6).

Full size image

This can be explained by the following:

  • in Personality 1, from Eq. (5):

$$ {\text{P}}_{\text{i}} = \, \left( { - 1} \right)\;*\;\left( 2 \right)\, + \,\left( { + 1} \right)*4\, + \,{\text{b}}\, = \,2\, + {\text{b }}; $$

  • in Personality 2, from Eq. (5):

$$ {\text{P}}_{\text{i}} = \,\left( { - 1} \right)*\left( 3 \right)\, + \,\left( { + 1} \right)*7\, + \,{\text{b}}\, = \,4\, + \,{\text{b}}. $$

This means that Personality 2 will demonstrate the avoidance behavior, Escape, (earlier and with great intensity) than in the case of Personality 1.

Note that, as personality traits of one agent, remain invariant throughout execution, the corresponding behavioral Pi is computed only once at the beginning of execution.

Thus, the personality effect is indirect, i.e., it influences emotion generation rather than the behaviors themselves.

6 Discussion

In this Section, a discussion is given in order to highlight important problems related to the adaptive decision-making behavior, the common currency that should have each system in the suggested framework, and the neural architecture, as well as to detail the ways they are solved.

6.1 Adaptive Decision-Making Behavior

In a changing, unpredictable, and more or less threatening environment, the behavior of an animal is adaptive so long as the behavior allows the animal to survive. Under the same conditions, the behavior of an agent (robot) is considered to be adaptive so long as the agent (robot) can continue to perform the functions for which it was built [25]. Under these circumstances, it is obvious that one can associate with an animat a certain number of state variables upon which its survival or successful operation depends, and that each of these state variables has a specific range within which the animat’s continued survival or operation is preserved. Ashby referred to such variables long ago as essential variables [16, 26].

Indeed, Ashby proposes the definition that: a form of behavior is ‘adaptive’ if it maintains the essential variables within physiological limits. The thesis that ‘adaptation’ means the maintenance of essential variables within physiological limits is thus seen to hold not only over the simpler activities of primitive animals but over the more complex activities of the ‘higher’ organisms.

The ranges of such essential variables describe a zone of viability inside the given state space, allowing the animat to be referenced at any instant by a point within this zone [25]. Under the influence of environmental or behavioral variations affecting the animat, the corresponding reference point moves and may at times approach the limits of the viability zone. In this case, the animat’s behavior can be called adaptive so long as it avoids transgressing the boundary of viability [26, 27].

Such behavior can be generated by means of several different or complementary abilities and architectures [25]. For example, the laws governing the animat’s operation may rely upon various homeostatic mechanisms thanks to which, if the reference point alluded to earlier moves away from an adapted point of equilibrium (adapted because it is suitably located within the viability zone), this process tends to return it to its original position, thereby decreasing the risk that it will pass outside the limits of the zone. Other ways in which to lower this risk involve the use of high-quality sensory organs or motor apparatus that allows the animat to detect as early as possible that it is approaching these limits and/or to move away from them quickly and effectively. In this line of reasoning, it is obvious that the equivalent of a nervous system is mandatory in order to connect the animat’s perceptions with its actions and that reflex circuits activated as quickly as possible increase the adaptive nature of its behavior.

6.2 Common Currency

Ethology, comparative psychology, and neuroscience have shown that observable behavior is influenced by internal factors (i.e., motivations, past experience, …) and by external factors (i.e., perception) [2]. This demands that different types of systems be able to communicate and influence each other despite their different functions and modes of computation.

This has led ethologists such as McFarland and Bosser [13] and Lorenz [28] to suggest that there must be a common currency, shared by the perception, motivation, and behavior systems. In this scheme, the perception system generates values based on environmental stimuli, and the motivation system generates values based on internal factors. Both sets of values are passed to the behavior system, where competing behaviors use them to compute their relevance and then compete for expression (action system) based on this value. Within different systems, each can operate on their own currencies.

Furthermore, as the system becomes more complex, it is possible that some agents may conflict with others (such as when competing for shared resources). If each agent computes its relevance in terms of a shared currency, conflicting agents can compete based on this value.

An interesting theory is proposed in [29] according to which pleasure is such a common currency needed in order to achieve the ranking of motivations in case of conflict between two or more of them. In fact, the perception of pleasure, as measured operationally and quantitatively by choice behavior (in the case of animals), or by the rating of the intensity of pleasure or displeasure (in the case of humans) can serve as such a common currency. The tradeoffs between various motivations would thus be accomplished by simple maximization of pleasure.

6.3 Neural Architecture

It is common for biologically inspired architectures [2] to be constructed from a network of interacting elements such as subsumption architecture [30], neural networks [31], or agent architectures [32].

Inspired by models of intelligence in natural systems, the design of the neural architecture features both a cognitive system and a motivation system including drives, personality traits, and emotions. Both operate in parallel and are deeply intertwined to foster appropriately adaptive functioning of the agent in the environment as it interacts with people.

Thus, the suggested neural architecture is inspired from [2] and implemented as an agent architecture where each computational element is conceptualized as a specialist [32]. Hence, each drive, behavior, perception unit, emotion process is modeled as a different type of specialist that is specifically tailored for its role in the overall system architecture.

Hence, although the specialists differ in function, they all follow a basic activation scheme. In fact, units are connected to form networks of interacting processes that allow for more complex computation. This involves connecting the outputs of one unit to the inputs of other units. When a unit is active, besides passing messages to the units connected to it, it can also pass some of its activation energy. This is called spreading activation and is a mechanism (originally conceptualized by Lorenz [28]) by which units can influence the activation or suppression of other units. Such an activation mechanism, in the suggested neural architecture, is inspired by ethological models similar to that developed in [33].

7 Conclusion

In this paper, a framework for adaptive agent decision-making modeling, deduced from a perception-action cycle scheme, has been suggested. Then, motivation systems: drives (homeostatic regulation), personality traits (five-factor model), and emotions (basic emotions) have been developed. Afterwards, a neural architecture implementation of the framework has been suggested. Finally, a discussion has been given in order to highlight important problems related to the adaptive decision-making behavior, the common currency that should have each system in the suggested framework, and the neural architecture, as well as to detail the ways they are solved.

The first tests related to a stimulation-drive (from a moving object), for two different agent personalities, and the activation level of emotions are presented and analyzed. The obtained results demonstrate how the personality and emotion of the agent can be used to regulate the intensity of the interaction; predicting a promising result in future: to demonstrate how the nature of the interaction (stimulation-drive, social-drive, …) influences the agent behavior which could be very interesting for cooperative agents.

After investigating the behavior system, the action system, and the social-drive to test the interactions agent-agent and human-agent, it is interesting to investigate different cooperative strategies with different emotion regulation strategies [34], and the learning from interaction [6, 35].