Design and Evaluation of a Mixed-Initiative Planner for Multi-vehicle Missions

Abstract

The command and control of multiple vehicles in highly dynamic scenarios by a single operator require high situation awareness and can result in excessive workload. In this article, we argue why the introduction of automated planners instead induces new human factors related problems such as complacency, opacity, and loss of situation awareness. In order to avoid such issues, we propose a mixed-initiative approach. The article describes our concept and the technical implementation of a mixed-initiative multi-vehicle mission planner. The planner serves as cognitive agent and supports a human operator du-ring planning and re-planning processes. The article focuses on the interaction concept. A first experimental evaluation of the described interaction concept is presented. Our application comprises the teaming of manned and unmanned helicopters in complex military missions.

1 Introduction

The mission planning and re-planning of multiple vehicles by a single operator under time constraints can result in excessive mental workload (MWL) conditions. Algorithms for automated planning and scheduling were developed that can solve such complex problems in reasonable time. However, these algorithms are often not directly suitable for use in incremental, user-centered collaborative planning [1]. Rather, the usage of such automated planners may result in loss of competences [2] and loss of plan awareness and plan comprehension. Additionally, most automated planners require programming exactly then when workload is already increased due to changes of the situation [3] which results in even more workload. Finally, the usage of fully automated planners may result in an inversion of hierarchy during mission execution as the human operator is obliged to execute a fully automation-generated plan.

In order to counteract such human factors issues, we propose a cooperative planning approach between human operators and cognitive agents based on incremental planning. The collaboration and cooperation of human operators with cognitive agents to solve a common planning problem is known as mixed-initiative (MI) planning. Although multiple mixed-initiative approaches were already presented in [4,5,6] the introduction of such systems in real-world applications is still missing. The challenge is now to integrate such a system into complex multi-agent human-machine systems. Thereby, in order to bring an actual benefit, the focus of research must address an efficient interaction concept between human and agent. The design of the agent’s intervention policy is of essential significance.

In this article, we present a mixed-initiative planning associate for helicopter onboard multi-aircraft mission (re-)planning. The agent is specifically designed to reduce automation-induced errors and to enable a helicopter pilot to solve multi-vehicle planning problems with sufficient quality in reasonable computation time. The proposed planning associate monitors the tactical situation and the pilot and intervenes during the planning process whenever necessary. The planning agent is integrated in a full military two seated helicopter mission simulation. The purpose of this article is to describe our concept, the implementation and evaluation of the planning associate. Thereby, the article focuses on the interaction aspects of the system.

2 Related Work

Allen and Ferguson present the design of a mixed-initiative agent that collaborates with a human operator in order to solve a common planning problem [6]. Thereby, they describe an integrated framework of several planning and reasoning functions and give a short evaluation. In [7] Chen et al. present experimental results on a mixed-initiative agent named RoboLeader which helps a human operator to coordinate a team of multiple UxVs. Roth et al. [8] describe the evaluation of a mixed-initiative system for multi-UAV mission management. The evaluation focuses on human-factors questions such as the measure of situation awareness, workload, and plan comprehension. Further research can be found in [9].

3 Application Manned-Unmanned Teaming

Our research application comprises the teaming of a manned two-seated helicopter with multiple unmanned aerial vehicles (MUM-T) in military helicopter transport missions. Characteristics of these missions are reduced preplanning time, highly dynamic mission environments (i.e. rapid changes of the tactical situation) as well as landings in hostile territory. Thereby, the unmanned systems are used as detached sensor platforms to increase the sensor range of the manned helicopter and to meet information demands for the pilots. The UAVs are used to reconnoiter the primary route of the manned helicopter and to find alternatives routes. Furthermore, the UAVs are able to locate suitable landing points in hostile territory. In critical mission phases, such as approach, ground operations and departure of the helicopter, the UAVs can provide protection. In this setup, the pilot non-flying is fully responsible for the tactical planning and re-planning of the manned/unmanned team during the mission. Mission planning tasks include helicopter route planning, contingency planning, and UAV task planning. The pilot uses a tactical map display to sketch plans and to command tasks to the unmanned systems. Thereby, he is assisted by our mixed-initiative planning agent, in order to increase performance and to reduce workload and human factors related issues. The agent proposes reconnaissance tasks, such as the identification of targets, and task assignment to an aircraft depended on available resources. It identifies flaws and conflicts in the current plan and offers solutions. Furthermore, it helps optimizing a given plan. The interaction between pilot and agent is dialog-based using either text boxes on the tactical map display or voice interactions. However, the proposed concept is not restricted to our domain. Rather, it can be transferred to various applications with single or multi-vehicle planning by a single operator, such as the air traffic domain.

4 Concept

4.1 Work System

We define mixed-initiative as cooperation between human and agent, in which both can take initiative over the panning process and direct the process. Thereby, mixed-initiative systems can include adaptable and adaptive components. Other definitions can be found in [10,11,12].

The conceptual design is presented in Fig. 1 in work system notation [13]. The work system notation differentiates between the worker on the left-hand side and the tools on the right-hand side. The worker has knowledge of the overall work objective and tries to reach that objective by own initiative. He is furthermore authorized to change this work objective by own initiative. To achieve that work objective, the worker uses given tools which are shown on the right hand side of the work system. The tools are subordinates, i.e. hierarchically degraded with respect to the worker.

Fig. 1.

Design pattern in the work system notation [13]

In our application, the worker is represented by a single human pilot. Multiple vehicles and their planning interfaces and algorithms, for example semi-automated planner, serve as tools to the pilot. A cognitive agent onboard each unmanned aircraft (UAV) serves as delegate agent. Each delegate agent is controlled by the human worker using a task-based-guidance approach [14]. The agent controls the underlying conventional automation in a supervisory control relationship. The introduction of such a delegate agent reduces the workload of the human worker.

Finally, we introduce a cognitive agent as worker on the left hand side which re-presents the mixed-initiative planning associate. This agent has all characteristics of a human worker. That means, it is also aware of the mission objective and pursues it by own initiative. However, in contrast to the pilot, he is not allowed to define or modify the work objective. The agent has knowledge about the mission domain, available tools and given resources as well as the human pilot. The task of the cognitive planning agent is to assist the pilot in all situations which require mission planning and re-planning.

4.2 Behavior Rules of the Mixed-Initiative Agent

The task of the agent is to enable the pilot to solve multi-vehicle planning problems with sufficient quality in reasonable time. The agent shall increase planning performance and reduce workload of the human-agent team. Thereby, the agent shall help to mitigate human factors problems such as reduced situational awareness and reduced plan comprehension. In order to achieve these goals, we formulated the following behavior rules for the mixed-initiative agent:

• leave as much work to the pilot as possible,

• intervene as late as possible,

• intervene as little as possible, and

• intervene incremental, rather than complex, whenever possible.

In order to determine which information is required for the pilot at a given time we considered psychological aspects of human pilots. In organizational psychology in the area of planning and decision making Herbert Simon proposed the “Satisficing Principle” [15] which describes the behavior of decision makers. The principle underlines that the human in general does not try to find an optimal solution to a given problem. Instead, he stops working on the problem as soon as the solution is sufficient to his personal level of aspiration. We transferred Simon’s principle into our concept. Our agent does not try to reach an optimal solution. Rather, it stops intervening as early as possible. If the agent intervenes, it guides the pilot through the problem step by step. Thereby, it is designed to leave as much work as possible to the pilot.

Simon states that the human’s level of aspiration depends on the current situation. This means that in critical situations a low-quality solution is sufficient. In contrast, in very uncritical situations, the personal aspiration level of the human pilot might be much higher. We mapped these levels of aspiration into situation criticality and workload. If the pilot’s mental workload is excessive, his aspiration level might be much lower compared to a low workload situation. Similar, if the pilot finds himself in a critical tactical situation, his aspiration level is probably reduced. These assumptions result in further behavior rules for the agent:

• aspire after a sufficient plan, rather than an optimal plan,

• adapt the aspiration level to the current tactical situation, and

• adapt the aspiration level to the current human mental workload.

4.3 Interaction Concept

In the following, we describe the concept for interaction between human pilot and cognitive planning agent: in a first step, the pilot provides information about a mission goal to our cognitive planning agent. In the following course, the pilot uses planning tools to (re-)plan or to modify the mission plan whenever required. Figure 2 visualizes such a planning process; initial state and final state of the planning problem are highlighted. A plan is defined as a sequence of actions (operators) which transforms the initial state into the final state. In general, for a given problem multiple solutions exist. The partial plan denotes the currently implemented part of the full plan. Constraints, such as maximum route distance, minimum fuel on board, or time of arrival, can reduce the solution space. The pilot needs to find a solution, which does not violate any constraints.

Fig. 2.

Visualization of the planning process starting at an initial state

The planning agent monitors the pilot and the situation and intervenes if required. There are three reasons for intervention:

• 1. The pilot does not continue with planning activities, even though it is required: If next planning steps are required and the pilot does not take actions, the agent informs about the problem and proposes next planning steps incrementally.

• 2. The pilot plan is erroneous: if (mission specific) constraints were violated in the implemented plan, the system informs about consequences and proposes alternative options.

• 3. The pilot plans suboptimal: if the plan can be significantly improved, the agent proposes enhancements. However, since a valid solution already exists, the agent shall intervene only if the aspiration level of the pilot is high, i.e. the situation is uncritical and the pilot’s workload is low.

The determination of the pilot’s activities is required to shape the intervention policy. If the pilot is about to plan the helicopter route, we do not need to inform him about the necessity for planning. Furthermore, if the pilot works on the helicopter route problem, the agent should contribute to rather this problem than to less related problems.

Not only the agent has the possibility to initiate a dialog with the human, but also the pilot is able to initiate a dialog with the planning agent. The pilot can assign tasks to the automated planning associate in high workload situations or request information about future options and consequences. For example, he can ask the agent for optimizations of his mission plan. This allows human pilot to interact on own initiative with the associate agent which marks a fundamental difference to previous work.

4.4 Functional Architecture

In order to work with a team member on a mutual problem, first of all, both must identify the mutual goal. Secondly, a human team member must have knowledge about the work domain, activities of team members, and their mental states. In order to develop a cognitive agent, we transferred these key capabilities to our concept. We identified four system pillars which serve as key enablers for our mixed-initiative planner:

• 1. planning and plan reasoning,

• 2. pilots’ activity determination,

• 3. pilots’ mental plan progress assessment, and

• 4. intervention.

The first key enabler is the capability to plan and reason about plans. This capability is required to infer options and determine next planning steps for incremental planning. Furthermore, this pillar is required to reason about human generated plans, their shortcomings, and conflicts. Therefore, the knowledge about goals and planning domains is required. The second key enabler is the ability to determine the pilot’s activities. Knowledge about the current pilots’ activities is required to enable planner interventions which do not disturb the conversation flow. The third key enabler is a human mental plan progress assessment. Such an assessment is the accumulation of the pilot planning activities over time. If the pilot for example has noticed a plan conflict in the past, but not reacted to it so far, we do not need to inform him about the conflict later again, but maybe better support his reaction by any means. Finally, the fourth key enabler is the intervention generation component. This component forms the bidirectional interface to the pilot and interacts based on the behavior rules (Sect. 4.2). This component receives data from the three previously described components, generates an interaction strategy, and interacts as required to influence the environment. These four pillars allow for an incremental and mixed-initiative planning process. Figure 3 shows the corresponding functional architecture of the agent. Our system can intervene on two different levels of automation. On the first level, it plans incremental and therefore requires more user interaction. On the second level, the agent intervenes with complex interactions rather than incremental. Thus, less user interaction is required. On the one hand, this reduces the mental workload of the pilot. On the other hand, it may also reduce pilot plan awareness. These two levels of automation can be adjusted by a workload-adaptive associate system, described in [16].

Fig. 3.

Functional Architecture of the Mixed-Initiative Associate

5 Implementation

This chapter describes the implemented system. The first subchapter shows the system architecture of our planning agent. The second subchapter describes the implemented HMI. The overall system architecture is presented in Fig. 4. The figure shows the important components of our agent on the left side as well as the implemented parts of the HMI and other relevant components on the tool side.

Fig. 4.

Components of the implemented system in work system notation

5.1 Agent System Architecture

Mission Planning.

The first component, mission planning, represents planning and plan reasoning capabilities. It requires substantial domain knowledge in the area of military mission planning because otherwise the agent cannot assist the pilot adequately. For this reason, we conducted knowledge acquisition experiments with German military helicopter pilots to model our planning domain. The domain contains a model of tools, i.e. the manned helicopter and the unmanned systems. For logical planning and re-planning purposes, we modelled our planning problem in PDDL which is an action-centered language to solve planning problems [17]. Core of PDDL are actions with pre- and post-conditions that describe the applicability and the effects of actions. Figure 5 shows a graph-based visualization of the domain and their implemented actions. These actions comprise helicopter and UAV specific actions. We use a PDDL planner which works based on our mission domain and a problem file. Furthermore, a CPLEX planner is used for rapid task assignment, optimization and scheduling [18].

Fig. 5.

Mission planning domain

On runtime, information about capabilities of the UAVs (i.e. sensor equipment, transit speed, reconnaissance speed) is provided to the planner. Thereon, the planner generates a mission plan, based on the mission goal, the information demands by the pilots and available resources. However, usually the demand for reconnaissance information outreaches the available resources. Then, the planner prioritizes demands based on their criticality to the mission success. Most important are the detection of landing points, the protection during ground operations, approach and departure, as the helicopter is most vulnerable in these flight phases. Less critical are information demands, concerning alternative route segments. The generated mission plan is not directly forwarded to the pilot.

Activity Determination.

The second component is the pilots’ activity determination. Therefore, the pilot’s interactions with automated cockpit functions are observed. Interactions can be manual or visual. In the second step, these observations generate evidences for hypotheses, which represent certain activities. Finally, these evidences are combined using the Dempster-Shafer theory. The result of the reasoning is the continuous determination of the pilot’s current activities. The detailed concept and realization of the activity determination is presented in [19].

Plan Progress Assessment.

The third component is called mental plan progress assessment and represents the pilot’s mental state regarding the planning process. Therefore, it gathers data provided by the activity determination and accumulates the data to a mental plan progress. The gathered data comprise pilot helicopter planning activities and UAV planning activities.

Intervention Generation.

The fourth pillar deals with the intervention generation. We modelled the interaction domain in PDDL as well. Interaction planning is handled using a logical contingency planner. The next sub-chapter will discuss the implementation of our intervention component in more detail. The purpose of the cooperative planning approach is to generate a mission plan which satisfies all constraints and is agreeable by the human pilot as well as by the agent. The purpose of the agent’s intervention generation component is to derive an intervention strategy, i.e. a sequence of actions which can be used to transform an insufficient mission plan into a plan which is agreeable by all team members.

To enable cooperative behavior, we need to model the agent’s behavior. We modelled an interaction domain in PDDL. The domain contains the environment and actions which can influence the environment if applicable. The environment consists of three pillars:

• 1. the specification of the tactical environment,

• 2. the description of the pilot’s mental state, and

• 3. the description of mission related facts, such as mission plans, flaws, alternatives and optimizations.

Furthermore, the domain contains actions. An action can be applied to influence the environment if the preconditions for this action can be met. The actions represent all possibilities of the cognitive agent and can be used to determine the interaction strategy. Four different types of actions enable the planner to generate such a mixed-initiative interaction strategy. Types of actions are:

• 1. Prioritizations to determine the most important issue. For example, an enemy unit that threatens the mission plan must be prioritized higher than an optimization of the mission plan regarding the task assignment. The pilot’s activities influence the prioritization.

• 2. Observations to wait for further activities of the pilot. For example, the system can decide to observe the pilot’s next interaction with the tactical display before intervening.

• 3. Pilot interactions to communicate with the pilot. This type of interaction is most important for cooperation between agent and pilot. The interaction of the agent is dialog-based. It includes informative dialogs, proposals for future routes and target candidates for reconnaissance or proposals for optimizations.

• 4. System interactions to modify the state of a technical system. These include the modification of tools such as the route planner or the UAV-planner (Fig. 4).

On each simulation step, the interaction planner determines the most important planning problem and then generates a proper handling strategy to solve the problem. As described in the previous chapter, it is most important to avoid flooding the pilot with information. Rather, only mission relevant information and proposals shall be communicated to the pilot. This represents Simon’s Satisficing principle which is modeled as an optimization function. An example of such a generated interaction strategy is displayed in Fig. 6.

Fig. 6.

Interaction strategy generated by the contingency-based intervention planner during runtime

5.2 Pilot Planner Interface

We developed an HMI that satisfies requirements for mission planning and communication between pilot and agent. The HMI has the following components (also displayed in Fig. 5):

• 1. a mission interface which is used by the pilot to specify the mission objective,

• 2. a tactical map display, which is used by the pilot to sketch the mission plan and command tasks to the UAVs (this display can be also used by the agent to visualize plan proposals and alternatives),

• 3. a dialog interface for the agent to communicate with the pilot using text boxes, and

• 4. a dialog interface for the pilot to initiate a communication directly with the agent.

The tactical map display is shown in Fig. 7. The left side shows the pilot’s interface (an object-oriented context menu) to sketch a mission plan and command tasks to the UAVs. The right side shows an agent initiated dialog to solve a problem. The text box on the upper right is used by the agent to explain the problem and to point out a solution. Additionally, the solution is visualized on the map display in magenta.

Fig. 7.

IFS Helicopter simulator cockpit

Fig. 8.

Results for workload and planning time for configurations A and B

Fig. 9.

IFS helicopter simulator cockpit

Fig. 10.

Results of performance increase for the secondary tasks for all three configurations

Fig. 11.

Results of performance increase for the secondary task for configurations B and C

Furthermore, we developed a pilot-agent dialog voice interface. This interface allows the pilot to interact on own initiative with the associate agent which marks a fundamental difference to our previous work. The interface can be used to request information and proposals from the agent and to add constraints to the mission plan. Therefore, we implemented the following grammar for commands:

• Commands: [Keyword] [Affected Vehicle] [Command] [ContextVariable]

• Requests: [Keyword] [Request] [Affected Vehicle] [ContextVariable]

The implemented interface is based on the software SIMON which is an open source front-end for speech recognition.

6 Experimental Evaluation of Agent-Human Interactions

In an experimental campaign, we examined the impact of our mixed-initiative agent-human interactions on the pilot’s workload and the overall planning performance. We hypothesize that during a planning process:

• plan segment proposals of the mixed-initiative agent reduce the workload of the pilot, and

• planning performance can be increased using plan segment proposals.

6.1 Configurations and Research Setup

In order to examine the hypotheses, a comparative experiment was conducted with four different configurations.

• Configuration A: The planning problem is simple (1 UAV, 3 Targets). The pilot uses a tactical map display to task the UAV manually.

• Configuration B: The planning problem is simple (1 UAV, 3 Targets). The planning agent proposes a task assignment. The pilot has to accept or decline the proposal after verification.

Our experimental hypotheses state that configuration B increases performance and reduces MWL compared to configuration A. The performance was operationalized using the time required to fulfill a given task. More specifically, based on a task with a given complexity, the performance is operationalized by the inverse of the time used to execute the planning task. The MWL construct was operationalized using the NASA-TLX [20] questionnaire as a subjective measure.

As experimental design we chose a within-subjects design. Each subject had to perform each configuration 5 times. The sequence of configurations was randomized between all participants in order to eliminate sequence effects.

In order to enforce adequate proposal verification by the subjects, one of the agent’s proposals was incorrect for each subject. The subjects were informed about the possibility of incorrect proposals.

6.2 Participants and Experimental Conditions

The experimental test sample consisted of 21 cadets of the German Armed Forces. The participants aged between 18 and 29 years (M_age = 22.6, SD_age = 2.2). During the experiment the participants planned one UAV on a single tactical map display. The display showed the UAV and its equipment as well as possible targets. All subjects were trained with the software previously to the experiment.

6.3 Results

Experimental data set was generated in [21]. Figure 8 shows the results for the workload and performance evaluation. The assessment of overall workload showed differences between configurations A and B. Compared to configuration A (M = 32.12; SD = 16.91), the workload could be reduced by 28.18% in configuration B (M = 23.07; SD = 11.20, p = 0.053). The assessment of overall performance showed significant differences between configurations A and B. The time required to perform the simple planning task manually (M = 15.3; SD = 5.97) could be reduced with assistance in Configuration B (M = 10.08; SD = 4.63; p = 0.032) by 34.14%. The results show that both hypotheses can be accepted.

Fig. 12.

Deviation from specified altitude over configurations A, B, C

7 Experimental Evaluation of Human-Agent Interactions

In order to evaluate the effectiveness of the developed human-agent interaction concept, we conducted another experiment in our helicopter mission simulator. The experiment was conducted to evaluate if pilot initiated dialogs with the mixed-initiative agent can reduce the workload and increase the performance of the human-agent team.

Therefore, we developed an experiment in which a single pilot has to fly a simplified MUM-T mission and is additionally responsible for helicopter route planning and UAV task assignment. We hypothesize that the pilot flying the helicopter benefits from the speech interface. Furthermore, we hypothesize that pilot initiated interactions with the mixed-initiative agent will affect the work result of the system in a positive way and increases performance and reduce mental workload of the pilot.

7.1 Configurations and Research Setup

For the evaluation of the system, a comparative experiment was conducted, in which three missions (mission I, II and III) were performed. In the experiment we evaluated the described interface of the mission planner. Thereby, we compared three configurations:

• Configuration A: the planning process is executed using a tactical map display without speech interaction. The pilot works with his tools using manual interactions.

• Configuration B: Planning is done using basic speech commands, which are equal to the planning commands in Configuration A. The pilot works with his tools using speech interactions.

• Configuration C: Planning is done with advanced speech interaction directly with the mixed-initiative agent. The pilot works with the planning agent using speech interaction.

The experimental hypothesis says that configuration B increases performance compared to configuration A; configuration C increases performance compared to configuration B. Furthermore, it can be assumed that Configuration B reduces workload compared to Configuration A and that configuration C reduces workload even more. In the experiment, all participants were exposed to the three configurations of the system. Therefore, the experimental design is a within-subjects design. In order to derive mental workload of the pilot, we added a secondary task. To eliminate sequence effects, we conducted the experiment with two test groups and switched the sequence of the configurations between both groups. Test group one conducted the configurations in ascending order. For test group two we reversed the sequence.

The missions were designed as follows. The subject had to fly a helicopter at an assigned altitude and speed in our helicopter mission simulator. The simulator cockpit is shown in Fig. 9. During each flight five re-planning situations occurred. To re-plan the mission, the subjects could use either the tactical map display or voice interaction, as according to the experimental condition under evaluation. To ensure comparability, all missions had an identical layout.

In order to prove the hypotheses, the constructs (MWL and performance) were operationalized using the following dependent measures. We operationalized MWL using a secondary task. Here, the subject had to classify possible targets on a display, whenever possible. To determine the workload of the primary task, we evaluated how much interaction time was spent in the secondary task similar to [22]. We operationalized performance as deviation in altitude and speed from the intended flight path.

7.2 Participants

The sample consists of 10 persons recruited from the University of the Bundeswehr Munich. Participants include 8 officers of the German Armed Forces and 2 Engineers. Participants include 9 male and one female. The participants aged between 22 and 31 (M_age = 27.2, SD_age = 2.6).

7.3 Results

Results for the secondary task are presented in the following. The average time required to classify an object in configuration A (M = 14.7; SD = 13.47) could be reduced in configuration B (M = 9.4; SD = 5.66) and configuration C (M = 7.8; SD = 5.4). Figure 10 shows the results for the performance increase in the secondary task. The graph shows the averaged performance increase. Thereby, configuration A serves as a baseline measure. The averaged performance in configuration B is 25% higher and in configuration C 40% increased, compared to configuration A. These results indicate that the workload in the primary tasks could be reduced. Therefore, the hypothesis that states that MWL can be reduced using voice interactions can be accepted.

Figure 11 visualizes results of the increase of performance in the secondary task in boxplot format for configurations B and C with regard to configuration A.

The evaluation shows that the standard deviation for altitude, referred to the overall mission, could also be reduced, displayed in Fig. 12. The standard deviation in altitude could be reduced in configuration B by 40.7% and in configuration by 41.7% compared to configuration A which served as reference. This shows that voice interaction in general could increase planning performance. However, the direct agent interactions could not increase performance further.

In general, it must be said that the implemented speech recognition rate was depended on the subject. The results show that workload could be reduced using voice interaction instead of manual display interactions if the pilot is flying the helicopter. However, our experience shows that if the pilot is not flying the aircraft, manual display interactions are preferred.

8 Conclusion

This article describes our concept for a mixed-initiative planning agent, integrated in a two seated military helicopter mission simulation. The agent shall ensure correct mission planning and re-planning of multiple unmanned vehicles in high workload conditions. The pilot and the cognitive agent cooperate and solve mission (re-) planning problems incrementally. Thereby, the cooperation between pilot and agent is dialog-based (text or voice). During runtime, our agent monitors the pilot’s activities, as well as the tactical situation and the given partial plan. Based on the currently implemented partial plan, the current tactical situation and the pilot’s activities the agent generates an interaction strategy which is used to assist the pilot. The evaluation shows that the implemented interfaces reduce workload and increase performance in onboard re-planning situations. Further research is required in order to demonstrate the benefits of the agent’s interaction strategy. We are currently in the preparation phase for full mission experiments with German military helicopter pilots.