Keywords

1 Introduction

Robot soccer is considered a multi-robot system that includes uncertainties and hostile environment, where robots are working in a coordinated manner on a real challenging problem. Coordination is an important issue for robot soccer, since undesirable behaviors can arise in uncoordinated teams, for example, teammates blocking the ball, obstructing each other or failing to block opposing players [1]. In robot soccer, strategy is defined as the plan of the robot soccer team expecting to win a match, tactic is defined as the organization of the team for the game. Roles are defined as a list of behaviors to perform by the robot. Behaviors are the basic sensorimotor skills of the robot, such as shoot the ball, to block an opponent or to intercept the ball [2].

Robot soccer leagues are divided into two main categories. One is the centralized leagues, using one decision-making body, with global perception and global motion control [3]. The other corresponds to the distributed leagues, which use fully autonomous robots, with local perception and autonomous control [4]. Many studies have been published on robot soccer strategies, looking for team coordination. Some other are focused on machine learning, for example using case-based approach for coordinated action selection in distributed robot soccer teams [5]. Or presenting a novel model of reinforcement learning algorithm applied to learning behavior [6]. In [7] is used fuzzy neural networks to task planning and action selection. In [8] is training the decision-making system by Bayesian SOM. Other papers proposed learning machine algorithms using robot soccer strategies for validation purpose [9].

One way to design robot soccer strategies is by using different computational models, such as coordination graphs for role assignment [10]. Finite state machines to switch tactics combined with Petri Net Plans for tasking execution are presented in [11]. Other example of using hierarchical state machines for robot soccer architectures is presented in [12]. Other research uses finite state machines for tactic selection and behavior selection [13]. Collaborative Filtering techniques combined with set-plays are also presented [14]. The use of Hierarchical Finite State Machines is presented in [15]. Synchronization has been used in different robot soccer distributed architectures in order to coordinate autonomous robots [16], focused on communication protocols. Other example of synchronization in computational models for robot soccer architectures is presented in [17], introducing the concept of Multi Robot Plans. In [18] a team strategy is modeled based on the discrete events system theory. In [19] it is proposed dynamic role assignment based on action utility prediction.

In [20] a robot soccer strategy based on tactics, roles and behaviors is presented, which uses a hierarchical state machines. Nevertheless the state machines are based on the game conditions, without a feedback of the behaviors selected by their teammates. This strategy was used to control the operation mode in an islanded ac microgrid [21]. Subsequently, in [22] a robot soccer strategy based on roles was proposed, in this work one role is assigned to each player and a behavior is assigned depending on game conditions, however a behavior feedback selected by their teammates is neither presented. This situation is similar to other strategies related above where synchronization is not presented. In this paper, is proposed a strategy for centralized robot soccer architectures based on a hierarchical finite state machine for tactic selection, dynamic roles assignment and behaviors selection. The behavior selection layer is performed by synchronized state machines, for the purpose of coordinate collaborative behaviors between players. In this case, the behavior selected by one player depends on a function activated by a state of its teammate. This model is designed for a robot soccer team with two-wheeled robots, with a central computer used for computational processes. This strategy is tested using the SimuroSot 5 vs. 5 league simulation platform, and programmed using C++. The proposed strategy has different advantages such as easy programming and low computational costs, and is scalable and easily adaptable to changes in the decision-making system.

2 Hierarchical Team Architecture Using Synchronization Functions

In centralized architecture there is a top-view vision system, which provides full global knowledge of the game field, showing the position of the players and the ball. The image processing is performed by the host computer. Coordinates of each robot and the ball are supplied to the decision-making body which selects the tactic, roles and behaviors. Once behaviors are selected, the decision-making body calculates and transmits the signals control for each robot. State machines have often been used in the robot soccer domain, as an alternative for team architectures. In this paper is proposed a decision-making system based on hierarchical state machine, which is divided into levels. The highest level is presented in Fig. 1(a), where a hierarchical finite state machine (HFSM) selects a tactic from a set of n predefined tactics T = {t1,t2,…,tn}, using environmental conditions Ev as transitions.

Fig. 1.
figure 1

(a). Highest level of HFSM. (b). Behavior selection using synchronization function.

Full size image

Once the tactic has been selected, the next layers are activated. At this level one role is assigned to each player. Players with roles perform specific behaviors controlled by a lower layer of the state machine, using environmental conditions St as transitions. To carry out the behaviors, the lowest layer, namely the motion control layer is activated. Once the tactic has been selected, it is assigned a set of roles R = {r1,r2,… ,rm} to a finite number of agents A = {a1,a2,…,am}, such that i ≠ j→Ri ≠ Rj. Note that there is the same number of agents and roles. One role has been assigned to each agent, without repeating roles. There is a set of behaviors B = {b1,b2,…,bk}, which will be performed by the agents. Each role r has been associated to one or more behaviors that will be executed by agent a in the game, according to game conditions. There is a set of characteristics G = {g1,g2,…,gp} that describes the changes in the game field. The highest level of the HFSM uses set G to select T, and then is used set G to assign R to A. Subsequently, the second layer of the HFSM selects one behavior bk to be executed by each a. Control of the robot motion is implemented in the lowest layer of the HFSM. The procedure is performed as follows:

figure a

It is proposed synchronization functions to feedback the state of the other player to synchronize collaborative behaviors between players. As is shown in Fig. 1(b), the player with the role m triggers the behavior k when the conditions St2k or ST1k are triggered. The player with the role n may trigger the behavior 1 when the condition St1 k is activated and the agent with the role m plays its respective behavior k and the condition of the role m triggers the function fm n(Stk1). This happens when the player with the role m players its behavior k. This strategy is implemented in a robot soccer team with five wheeled robots. One is the goalkeeper, which is the only constant role.

Two parameters have been chosen for tactic selection. The first one refers to ball localization on the game field. For this purpose the field is divided into three different zones defined as follows: The defensive zone is the zone where the goalkeeper of the own team is located. The middle zone is in the half of the playing field, and the offensive zone is where the opponent’s goal post is located. Ball zone corresponds to the zone where the ball is located. It is defined the ball possession as the minimal distance between a robot and the ball, where the robot can shoot the ball. This distance is obtained experimentally as an appropriate distance at which a robot can shoot the ball.

Four tactics are defined as follows:

  • Defensive defense tactic: This is selected when the ball is located in the defensive zone or the middle zone, and one’s own-team does not keep ball possession.

  • Defensive attack tactic: This is selected when the ball is located in the defensive zone or the middle zone, and own-team keeps ball possession.

  • Offensive defense tactic: This is selected when the ball is located in the offensive zone, and own-team does not keep ball possession.

  • Offensive attack tactic: This is selected when the ball is presented in the offensive zone, and own-team keeps ball possession.

Four tactics correspond to states, whereas the environmental conditions (ball zone, and ball possession), correspond to the transitions. For role assignment, there are five players per team. One of them is the goal-keeper ap, which keeps a constant role. Thus, there is a finite number of agents A = a1; a2; a3; a4 that perform a set of roles R = rad; rpd; rs; ra, where rad corresponds to active defender, rpd corresponds to passive defender, rs is supporter, and ra is attacker. The decision-making body executes the algorithm to assign roles, depending on the tactic previously selected. It is important to highlight that the role assignments is similar between defensive tactics (defensive defense and defensive attack tactics), and offensive tactics (offensive defense and offensive attack tactics).

For defensive tactics, the role assignment is reaches by both defensive defense and defensive attack tactics. Once each tactic is activated, the role selection algorithm is carries out. In these tactics, the player closest to the ball becomes the active defender rad, and this player attempts to take the ball. The second nearest player to the ball becomes the passive defender rpd, which blocks an opponent player. The third nearest player to the ball is the attacker ra, which goes to the offensive zone, waiting for the ball. The last player becomes the supporter rs, which goes to offensive zone.

For the offensive tactics, the role assignment is reaches by both offensive defense and offensive attack tactics. Similarly, once each tactic is activated, the role selection algorithm is carries out again. The player nearest to the ball becomes the attacker ra, and this player attempts to take possession of the ball, in order to score a goal. The second nearest player to the ball becomes the supporter rs, which supports the attacker. The third nearest player to the ball is the active defender rad, which is in the middle zone in case of counter attack. The last one is the passive defender rpd, which supports the active defense.

2.1 Behavior Selection

Many roles were implemented to execute different behaviors, depending on the tactic chosen. Goalkeeper is the only static role in this strategy. The behaviors of the goalkeeper are presented in Fig. 2. There is a goalkeeper zone, which is an area in front of the own-team’s goal post. When goalkeeper behavior is triggered, the player checks its location. If it is within the goalkeeper zone, the robot localizes and follows the ball in a line running parallel to the goal line. If the player is not located in the goalkeeper zone, the robot must become properly located and subsequently checks its location and continues playing.

Fig. 2.
figure 2

Goalkeeper behaviors.

Full size image

For the tactic defensive defense, the active defender role goes by the ball as is shown in the Fig. 3(a). Firstly, the ball is located. If the ball is in the goalkeeper zone, the robot follows the ball outside, to avoid own-goals or block the goalkeeper. If the ball is outside of the goalkeeper zone, the active defender goes to catch the ball. The Fig. 3(b) shows the behaviors in the tactic defensive attack. Once the tactic is activated the ball is located. If the ball is inside the goalkeeper zone, then the active defender follows the ball outside. If the ball is not located in the goalkeeper zone, then the active defender checks the ball possession, if it has ball possession, the robot shoots the ball to offensive zone. If it has not the ball possession, the active defender tries to catch the ball. In the Fig. 3(c) is presented the behaviors of the active defender for both offensive tactics. There is an active defender zone, which corresponds to a zone behind the half-way line. When the tactics are activated, the active defender checks its location, if it is not inside the active defender zone, the robot goes to its zone. If the robot is located in its zone, the player follows the ball in line with the half-way line.

Fig. 3.
figure 3

Active defender behaviors

Full size image

The behaviors of the role passive defender for both defensive tactics are shown in the Fig. 4(a). In these tactics, the passive defender checks its nearest opponent and goes to block without touch the player, in order to prevent possible opponent passes avoiding collisions with the player. In the Fig. 4(b) are shown the behaviors of the passive defender for both offensive tactics. For this purpose there is a passive defender zone, behind the active defender zone, where the passive defender covers the other side that the active defender blocks. Thus, when the offensive tactics are activated, the passive defender check its location, if the robot is not located in its respective zone, it goes to its zone. If the robot is located in its respective zone, the player covers in line with the half-way line the other side where the ball is located.

Fig. 4.
figure 4

Behaviors of passive defender.

Full size image

For the defensive tactics, the behaviors of the supporter and attacker are presented in the Fig. 5. In the case of the attacker (Fig. 5(a)), there is an attacker zone in the offensive zone, where the robot follows the ball in line with the half-way line, waiting for the ball. Thus, when the defensive tactics are activated, coordinates of attacker are checked, if the robot is in the attacker zone, the robot follows the ball in line. If the attacker is not properly located, the robot goes to its zone. In the Fig. 5(b), it is presented the behaviors for the supporter. It is defined the supporter zone, located in front of the attacker zone, where supporter covers the other side where attacker blocks. If the supporter is inside its zone, the player covers the zone in line with the half-way line. If not, the supporter goes to its zone.

Fig. 5.
figure 5

Behaviors of the supporter and attacker in defensive tactics.

Full size image

2.2 Synchronization Functions in Coordinate Attack Behaviors

For the tactic offensive attack, the main purpose of the attacker is to shoot the ball to opponent’s goal post, seeking score a goal. As is shown in the Fig. 6(a), ball possession is checked in the attacker, if the attacker not has the ball possession, player goes to catch the ball. If the robot has the ball possession, it is checked if there is an opponent player different from the goalkeeper, which blocks the trajectory to opponent’s goal post. If the trajectory is blocked, attacker activates a synchronization function to supporter, in order to coordinate a ball pass. In the Fig. 6(b), is presented the behaviors of the supporter. Initially, the supporter checks if the attacker activates the synchronization function. If there is not activated, supporter works as is described in the Fig. 5(b). If the synchronization function is activated, the supporter receives a ball pass of the attacker, when the tactic changes and new role assignment is reached, supporter becomes the new attacker.

Fig. 6.
figure 6

Behaviors of the supporter and attacker in offensive attack tactic

Full size image

In the Fig. 7, are shown the different zones presented to perform the different roles (goalkeeper zone, active defender zone, passive defender zone, attacker zone, supporter zone and lateral zones).

Fig. 7.
figure 7

Zones in game field.

Full size image

3 Simulation and Results

In order to test and validate the proposed model, the strategy is implemented in FIRA’s SimuroSot and programing in C++. In order to validate the performance of the proposed strategy, there were carried out 100 games against a team strategy provided by default in the simulator and programming in LINGO (hereinafter called LINGO strategy). LINGO strategy consists of five constant roles which are the goalkeeper, two defenders that blocking the ball in parallel to the goal line, and two attackers which go to the ball and shoot to their opponent’s goal. All games were conducted applying the rules of FIRA SimuroSot 5 vs. 5 League.

All games were won by the strategy with synchronization, in many cases with ample goal difference. In Table 1 is presented the comparison of the different results obtained in the 100 games by each team. The average of goals scored, the average of ball possession and the average of successful passes is significantly higher in the team with the proposed strategy. This is because a greater number of collaborative behaviors, due to the synchronization between attacker and supporter. It should also be noted that in all games the ball remained in the offensive zone in the many of cases. The variations of goals scored, successful passes or ball possession observed in different games are due to an incremental odometry error induced by the simulator. However, it shown that team with the strategy proposed presented a better adaptation to game conditions than LINGO strategy. In all games was observed that team with LINGO strategy shown different behaviors that produce faults or own-goals.

Table 1. Comparison of the parameters of 100 games against team with LINGO strategy
Full size table

In Table 2 is presented a comparison among different results obtained in games between different teams against LINGO strategy. The comparison includes the results previously reported of the team with the proposed strategy, and the results obtained by other strategies against LINGO strategy reported in [9, 15, 18–20, 22]. As can be seen the team with the proposed strategy presented a higher average of goals by game than the strategies previously published in [18–20, 22], the goal difference is higher than reported in [15], the only strategy which presented considerably higher average of goals is the strategy presented in [9] however, this work is focused on machine learning. Strategies which reported ball possession showed similar percentages. N.R. corresponds to data not reported. Goals av., refers to goals average. Ball poss., refers to ball possession. Standard deviation is similar in the cases when it was reported is similar (excepting [20], where were only performed 10 games), this is because an incremental error of odometry induced by the simulator, in order to perform more realistic simulations.

Table 2. Comparison of results reported of other strategies against team with LINGO strategy
Full size table

In the Fig. 8 is presented an example of synchronization of behaviors between attacker and supporter, including a change in the roles. In the Fig. 8(a) the player 2 (attacker) detects that an opponent player blocks the trajectory of the ball to the goal, player 1 (supporter) accompanies. In the Fig. 8(b) player 2 activates the synchronization function and sends the ball to player 1, who, in turn goes to catch the ball. When player 2 do not have ball possession a new tactic is selected and new roles are assigned, player 1 becomes attacker and player 2 becomes supporter. In the Fig. 8(c) the new attacker catches the ball and try to score a goal, meanwhile player 2 supports.

Fig. 8.
figure 8

Example when an opponent player blocks the goal, with synchronization function.

Full size image

4 Conclusions and Future Works

Finite state machines for multi-agent coordination in centralized robot soccer architectures allow an intuitive design of the strategy keeping in mind the league rules. Given that robot soccer is a multi-agent system, coordination among player is necessary and given that, synchronization function merge as alternative to improve the performance of collaborative behaviors such as successful passes. The proposed strategy was compared with other reported strategies that played games against a basic strategy provided by the simulator. The strategy proposed showed better results that those strategies, excepting one case where machine learning where implemented. Even though the use of computational models as finite state machines have displayed satisfactory results in coordination in robot soccer, the use of synchronization functions among players allow to obtain better results in cooperative behaviors that other similar strategies that not use synchronization functions obtaining, for example, more goals scored, more number of successful passes or more percentage of ball possession. The Synchronization functions usage do not represent any increasing in strategy complexity, neither a relevant increasing in source code or computation cost. Likewise these synchronization functions can be also applicable to other architectures, such as distributed architectures or using other computational models.

In future works will be proposed synchronization function implemented in distributed strategies seeking better results in cooperative behaviors. As well nondeterministic finite state machines and probabilistic finite state machines to design coordination strategies and the making decision systems in robot soccer leagues.