Keywords

1 Introduction

The use of groups of mobile Unmanned Vehicles (UVs) is in the focus of attention of many researchers and corresponds to the global trend of increasing of use UVs both in specially prepared conditions (e.g. highway vehicles), and in difficult conditions, that require self-organization groups of robots [7].

The greatest difficulties are encountered in solving the problems of self-organization of heterogeneous groups of robots, because their tactical and technical characteristics may vary over a wide range of values: in the types of their activity, functionality, on – Board equipment, etc., and, in addition, from a tactical point of view they may differ in a structure of the interconnection to ensure environmental safety. Today, there are a number of studies, that discuss the problem of the collision avoidance control in scenarios involving the activity of AMR of heterogeneous forms  [2, 8, 17,18,19,20].

In these works the formalized models are combined with elements of heuristic approach, and set a role of HO as an active element for inclusion in management of a heterogeneous group, because HO in the role of “Manager – Observer” can dynamically form the risk criteria and explicitly set limits on the activity of both the whole group and separately considered robots in related areas of their functioning.

Further, taking into account the formerly proposed design solutions for the lunar missions, the potential application of a heterogeneous group of AMRs in joint work on the lunar surface as part of the lunar mission will be discussed. One of the possible approaches to organize the functioning of AMR may base on the use of cognitive capabilities of a person, who perceives each of the tasks for AMR as a separate cognitive component of decision-making [6, 11]. This approach, based on the principles of “Task-based Guidance” [21], can be used in the development of the domain ontology and in the construction of a computer model of a current situation.

In this regard, the requirements to be applied to the on-Board equipment of the AMR and to the software environment that allows maintaining situation awareness of HO are considered [5].

In the visual display (indication) of the current situation at the workplace of the cosmonaut, who remotely monitors the operation of the group (as an observer-dispatcher) a special role can be played by the induced reality technology, which is currently being applied for modeling and visualization in virtual environments [26]. Heuristic techniques, which HO can use to perform monitoring to reduce the collisions risks in the group by apply limits and standing orders on the tasks of AMR, are also of interest.

2 The Usage of Autonomous Mobile Robots During the Crew’s on-Planet Activity on Lunar Surface Outside the Stationary Lunar Base

Currently, much attention is being paid to the prospects of human exploration of the Moon and planets and to the peculiarities of activities in extreme conditions [1, 3, 4, 14, 16].

The expected types of activity of cosmonauts (in accordance with cosmonaut’s crew functional duties), when performing lunar missions during on-planet activities in the proposed design solutions, are the following [13, 22]:

  • the solution of target tasks (implementation of a program of scientific research and experiments; development of new space technologies, extraction and processing of lunar resources);

  • the participation in the construction of space infrastructure (on the lunar surface);

  • the work with the delivered equipment (unloading and loading operations, delivered and removed cargo inventory tracking, transportation on the lunar surface);

  • the control of on-planet robotic systems, including anthropomorphic types;

  • ensuring the safety of on-planet activity;

  • providing medical assistance to the injured cosmonaut in case of a medical emergency or incident, that led to loss of efficiency, including the failure of protective equipment, life support systems and others.

In a series of works it is noted that mobile robotics systems capable of performing transport, scientific and a number of other tasks on the lunar surface will play a special role in lunar missions. It is assumed that the first stages will be carried out with the help of automatic-only vehicles, and then with the use of automatic and manned moon-rovers. However, further steps to do a systematic and deep study of the Moon can be carried out only with the participation of cosmonauts, supported by the robots designed to work in hazardous and extreme conditions.

According to the works [1, 14], the “industrial” stage of the Moon exploration should be preceded by studies on its surface using a network of stationary and self-propelled scientific stations.

For example, thematic geological survey and preliminary geological exploration, according to [22] are an important and necessary stage of the Moon exploration.

At this stage, it is expected not only to collect extensive scientific information about the Moon itself, but also to gather the necessary information for placement of the long-term or periodically visited lunar base, which is a first step towards the deployment of the primary industrial infrastructure.

In general, it can be assumed that various modifications of autonomous mobile robots with different functionality will be tested with the development of robotics. The crew will be safe located in a stationary lunar base and primarily ensure the safety of work and remote monitoring of groups of robots. Thus, there is a new direction of cosmonauts activity associated with monitoring the activity of AMR, which can be considered as a kind of dispatching functions, supplemented by operational planning and analysis of prerequisites for incidents related to potential conflicts and problems of joint operation of groups of robots [11].

3 Distribution of Functions During Remote Monitoring of the Activity of a Group of Robots by Cosmonauts

The problem of distribution of functions in Human-machine Interactions is one of the key in ensuring the safety of complicated technical systems. At the same time, this problem has not been sufficiently studied and highlighted in the literature with regard to space robotics, especially for the use of groups of robots under the control of the crew in lunar missions. The undeniable progress in the field of increasing the autonomy of mobile robots and their adaptive capabilities to extreme environments does not remove questions about the control of their activity on the part of a human, when joint participation of the crew and robots in extravehicular on-planet activity is planned. Unlike a robot, which is assigned for narrow and particular tasks, a human has the ability to understand these tasks at the mission level, that is, to understand the current situation in general [11]. To do that he must know how: (1) plan the execution of tasks taking into account the possible collisions in the conflict of “interests”, (2) monitor the activity of a group of robots, (3) get data about their exact location and free movement limits in designated zones, (4) identify the risk of collision when moving in the intersecting routes of mobile robots, (5) use communication equipment to control the tasks, and (6) anticipate the actions, performed by robots, that can affect the performance of tasks by group members, etc.

At present, it is not possible to achieve full automation that would realize certain cognitive functions inherent in adaptive human behavior in problem situations.

Thus, the problem is transferred to the plane of division of functions at the level of planning and execution of tasks in flight operations, and the role of “Supervisor” in a complex system of monitoring the activity of a heterogeneous group of agents is assigned to a human. For fulfilling these demands it is necessary to provide mechanisms for setting regulations for AMR’s activity and control of compliance with these regulations by HO. In fact, this means the need to build a unified information base on the monitoring area in which the group operates (in the form of electronic maps), using the tools of localizing all participants at specific time periods. In addition HO needs the development of means of introducing restrictions on robots mobility in order to separate traffic routes over space-time parameters, etc. To some extent, in this case, we can talk about the need to use information tools similar to those used in the activities of dispatchers, which found its justification in [12, 15, 23].

However, the most significant issues of HRI design are related to the use of a single semantic space for design the Dialogue for Human Robot Interaction during monitoring a group’s activity, and to the use of a virtual environment, that simulates real-time events for full control of the situation by HO and for its constant “inclusion” in the situation on the basis of visual representations, which is covered by the concept of situation awareness.

4 Tools for Visualization the Current Situation, Focused on the Cognitive Components of HO Activity During the Monitoring of the AMR Group

In order to create the conditions for the actualization of the cognitive capabilities of the HO, it is necessary to design the information support. In particular to supplement the traditional types of information about the parameters of AMR movement in the way that contribute to the constant involvement of a HO in the situation, his full situation awareness and the rapid detection of signs indicative of a risk of contingency and collisions of AMR. As noted in [15, 23], the basic principle of building an information display system for HO, performing the functions of Manager – Supervisor of the group of active agents is the use of multimode displays that allow representing visual images with different semantic content.

One of the widely used forms of displaying information about objects in a controlled territorial zone is a multilayered electronic map, layers of which represent various characteristics of the territory, including: (1) the nature of the terrain and the designation of natural obstacles to the movement of mobile vehicles, (2) the location of stationary artificial objects that can serve as landmarks for navigation of active agents and the coordinates of which are used to adjust the dynamically measured parameters; (3) coordinate grid for more precise visual control of the mutual position of mobile objects; (4) designations assigned to different routes of vehicles in accordance with the provided scenarios, etc.

Management of the AMR group, taking into account the composition of tasks, may require the introduction of new visualization elements that facilitate the actualization of cognitive functions during the monitoring of the AMR group for trouble-free interaction. For this purpose it is offered to enter differentiation and indication (marking) of the working zones connected with accomplishment of concrete tasks by these or those types of robotic complexes. This makes it possible to clearly identify the zone of manifestation of AMR activity, and during the active operation of the robot to monitor the penetration of other robots into this zone. Practically, this means the visualization of regions according to the principle of association tasks, that they are executed in specific periods. In addition to this information layer of the electronic map, a data frame may be provided for HO, which shows a time chart of the AMR loading with specific tasks in a particular area of the controlled territory. On this basis HO can control adjacent zones of activity of AMR, and if necessary allocate space-apart or “to compartmentalize” in time this activity for minimization collisions risks. These heuristic methods of planning and control should be at the disposal of the HO within its competence, and the event display system should show how these or those events relate to the time and place of manifestation.

Another version of the visual representation is associated with the necessity to build a picture of events in the egocentric coordinate system associated with a particular AMR. In fact, created conditions for monitoring the situation are based on the information that is available from on-Board means of the concrete AMR, including the measurement of its location relative to landmarks and boundaries of the permissible area of active movement, the detection of moving objects near the control zone and others. This type of display corresponds to a circular overview indicator, in the center of which there is a label corresponding to the position of the concrete AMR, from which it is possible to count the distance to other mobile objects.

The most difficult option for HO is monitoring, associated with the passage of a particular AMR through the set of zones allocated for the tasks of other members of the AMR group. In this case, it may be necessary to provide management functions to the AMR itself (as a “Leader”), and at the same time on the part of the HO, it is necessary to ensure the passage of the transition team to the subordination of the AMR – the “Leader” of those AMR, whose areas of responsibility intersect when passing the route. Visual indication of this type of scenario can be performed using the “External Observer” technology, and for this it is possible to apply modeling and visualization of the virtual environment of the AMR – “Leader” [15].

In other words, different scenarios of interaction between robots among themselves may require initialization of different cognitive components of the activity of HO.

5 Software and Information Environment for Information Support the Human in Remote Monitoring the Group of AMRs

To provide situation awareness of HO in remote monitoring the group of AMR, we need to provide software and informational tools to keep up to date the information required to manage tasks for a group of AMR and monitoring of progress. The main goal facing the HO is to achieve a coherent, conflict-free functioning of the heterogeneous group of AMR. The general structure can be described as follows.

The first component is a distributed hardware – software complex for the collection of raw data from the AMR, implemented: (1) as part of on-Board devices that are equipped with AMR for navigation in the environment; (2) in the telemetry and communication system for data exchange and for accumulation of all available information at the workplace.

As the second component, it is possible to identify the tools that should be available to HO for integration and visualization information in an understandable form of the entire set of data that is promptly received through AMR’s communication channels and delivered to the workplace of the HO.

The purpose of the third component is to support the management mechanism of the group by setting objectives of the AMR in the framework of the formed ontology. As part of this component is implemented HRI, which allows to build a visual image corresponding to the mental representations of the person about how to interact with AMR, so that their activity meets the principles of accident-free and conflict prevention.

Possible principles and methods of construction of the environment that meets the goals of modeling such a complex technical system are discussed in a number of papers [9, 10, 19].

In [10] questions of construction of interaction of humans and robots and improvement of situation awareness of HO on the basis of agent-oriented approach of the mixed team of AMR and group of cosmonauts in protective equipment on the surface of the moon in the controlled territorial area are considered.

In [19] for a similar situation of construction of HRI it is offered to use ontology to provide semantic interoperability of robots and humans at their interaction.

An example of building HRI in monitoring a group of AMRs is shown in Fig. 1. According to the proposed approach, ontologies store knowledge about the tasks that need to be performed, as well as knowledge about the functionality of robots and the current situation. This knowledge determines the tasks that based on the current state of the active agents. Ontologies are published in the intellectual space, allowing active agents to carry out indirect interaction on the basis of operational information about the current situation. To do this, all active agents generate and publish tasks when additional resources are needed to complete them.

Fig. 1.
figure 1

Building HRI in monitoring a group of AMRs.

Full size image

As mentioned above, the realization of human cognitive capabilities in such a slow information environment is possible with the fulfillment of certain prerequisites and agreements.

The main proposal is that when the system “team of cosmonauts – group of robots” is projected, it is assumed that the degree of autonomy of robots allows them not only to perform fairly complex assign actions, but also to independently navigate the obstacle area and to adapt dynamically changing environmental conditions.

In the literature on the problem of constructing a Dialog of Humans and Robots as a HRI form [24, 25], this range of methodological problems is considered from the standpoint of finding ways to interact on a single semantic basis (for example, in the form of ontology), available for expression in human categories of space and time in relation to the object of the external environment.

In this work, as one of possible solutions, we propose to consider the agent-oriented building of HRI, in which “virtual proxy agents” on behalf of the real intelligent agents, operating in a real environment, can act both as sources and as consumers of information transmitted between the participants of team interaction. This approach develops, to some extent, the ideas of induced virtual reality. It is aimed at replacing existing active agents with their virtual analogues (with which HO interacts, using the visual representation of the interactive virtual environment), which are able to collect, process, summarize, purposefully deliver and display the agreed source data to the recipient on the basis of a common understanding and a single semantic interpretation of the current situation.

In other words, in addition to the actual active agents, the “virtual agents” or proxy agents that are shown on the HO’s display can support the course of such interaction, organize it, and provide situation awareness to the participants of the dialogue. This is based on the knowledge of the natural limitations for each real agent to collect information about the environment; participate in the solution of specific tasks and the preparation of a common task; the necessary and available resources, etc. A key feature of remote control of a group of robots and the tasks of monitoring the situation is that humans and robots, interacting in a mixed team, coordinate their actions through a dialogue about their current capabilities, immediate goals and effectiveness of solving previous problems.

6 Conclusions

The paper deals with the peculiarities of the use of AMR in manned lunar exploration programs. According to the available design solutions, the success of the flights will largely depend on the ability of specialized robotic systems to work on the lunar surface and perform many different tasks.

At the same time, unlike the researches devoted to the application of homogeneous groups of ARM in terrestrial conditions, under review the new issues have to be included that concern the variant of the heterogeneous group of AMR, when a cosmonaut, located at a long distance in the lunar base, has to conduct monitoring of the activity of AMR.

In the course of a specific operation HO should perform the following functions:

  1. (1)

    analyze the functioning of the AMR as a part of the complex organizational and technical system and make decisions based on a priory data on flight operations, e-maps of lunar surface regions, various AMR task chains and their resources;

  2. (2)

    promptly and timely receive the data on the successfully completed tasks from all members of the group, the location of all members of the group and other data (e.g. environmental data).

Human-Robot Interaction (HRI) is generally the object of interdisciplinary research, since the ARM group should perform an increasing number of functions in lunar expeditions. Cosmonauts in addition to the implementation of narrow operator tasks (manual control and emergency intervention), should additionally be able to plan the work of a group of robots and ensure their safe joint work. This requirement created new problems to HRI that can be solved using the “Human – Robot” Form of the Dialogue with a semantic context.

Assessing the prospects for the further development of this approach, we can state the need to pay attention to the design of complex activities related to cognitive tasks to assess complex tactical situations arising from planetary activities, when the support of a cosmonaut by ground teams will be severely limited, in contrast to the current practice of flying orbital space stations. These issues are closely related to the prospects for developing an “Electronic Adviser” for a cosmonaut, designed to develop a plan of further action for each of the following cycles of the group’s activities. This work should be carried out by the cosmonaut a priory, with a special emphasis on finding a possible way out of conflict situations and failures of robotics products.