Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Introduction

As part of the digitalization of the battlespace, the Command & Control systems (C2 systems) are already widespread in the army. These information and communication systems provide an operator with a view on an operational situation, which typically shows a map with symbols representing units. The system gathers information from various sources and allows users to interact and give orders directly. C2 systems are also used or being adopted in non-military areas, for example in civil safety or by large private operators. In the TACTIC project, three complementary sources of information commonly are used together: spatial coordinates of objects (the field map), temporal coordinates (the chronology) and socio-technical coordinates (ODB). It seems appropriate to consider them as a cognitive tridimensional referential in which the events take place in a specific context.

Operator-Simulator interaction goes through an interface (the place of cognitive interaction with the operator) and the screen (the place of physical interaction for the visualization of the simulation). During a simulation, the operator has to face three interrelated challenges: (1) Collecting relevant data and information from several sources; (2) Translating data and information into knowledge to produce contextual understanding of the events and behaviors of interest in relation to particular goals, capabilities, and policies of the decision makers; and (3) Using that knowledge for making relevant decision in the working context. As a consequence, operators must deal with an interpretation of the domain (the simulation resulting of the operator-simulator interaction) intertwined with an interpretation of the interface functioning for translating actions on the simulation into commands to the interface.

By focusing on the process that leads to an action (including the decision-making part), rather than the result of the action execution only, it is possible to make explicit the context in which the operator works effectively. Generally, context is used to eliminate or reduce ambiguity, detect inconsistencies, explain observations, and constrain processing [4].

A context is associated with the operator’s focus of attention (a task realization, a new event) during the simulation. It contains two types of knowledge, namely, the contextual knowledge that is more or less related to the current focus in a flat way and the external knowledge that has nothing to do with the focus at the time at which the operator considers the focus. The proceduralized context is a structured subset of contextual knowledge that is explicitly used to address the focus of attention. The focus of attention evolving, its context evolves too: there are exchanges between contextual knowledge and external knowledge and, thus, the frontier between them is porous.

Brézillon [5] introduces the notion of contextual element for representing context information coming from heterogeneous sources in a uniform way. A contextual element is an “element of the nature” for which it is necessary to know its value in the current focus (i.e. its instantiation). Contextual elements come from different highly heterogeneous sources like the operator, the task, the situation and the local environment where available resources are. The distinction between a contextual element and its values is important for the reuse of experience because a difference between a past context and the working context can be a difference of either contextual elements (e.g. a contextual element only exists in one context) or instances (e.g. the same contextual element has different instantiations in the two contexts).

Hereafter, the paper is organized in the following way. The next section discusses the type of knowledge that is used in task realization and the way to represent it. The following section presents the modeling of reasoning in task realization, especially when operational knowledge intervenes. The section after positions this work with related works and emphasizes the role of context.

2 Knowledge Representation in Task Realization

2.1 Mental Representation

Experts rely on a highly compiled experience because they generally act under temporal pressure and are very concerned by the consequences of their decision. Experience reuse is never direct because the context of any decision-making is unique, and thus any experience must be adapted in a process of contextualization-decontextualization-recontextualization [4] to be efficient in another context. Fan et al. [11] used this process in finding a scientific workflow (SWF) in virtual screening: A researcher extracts from a repository a SWF close to his working context (phase of contextualization), extracts the SWF model (phase of decontextualization), and, finally, looks for instantiating the SWF model in his working context (phase of recontextualization). Thus, the experience acquired by the researcher during this process (and thus experience management) relies on context management. As a consequence, it is more important to model task realization than a task model. This context-based modeling makes operators’ behavior explicit.

An operator receives a lot of information on events occurring in his environment, but only a small number of events enter operator’s focus of attention. Most of events concern what the operator is doing and thus are processed automatically (often unconsciously). A good example of the importance of the focus of attention on the selection of events judged relevant by the operator is given on the video of cognitive blindnessFootnote 1 with the gorilla. Events in the focus of attention correspond to either information proactively searched by the operator (e.g. information about the mission of a specific unit) or unpredicted events (e.g. an action of the enemy) not explained in his mental representation of the focus, the situation, the resources available. The goal is to integrate the corresponding information in the mental representation of the task realization. Other events are put in the periphery of operator’s attention because they are not directly related to the focus of attention.

2.2 Expert Maps

A mental representation depends on expert’s experience with the realization of tasks attached to his role. Experience contains knowledge accumulated by the expert during his practical use of the domain knowledge along a number of realizations of the same task in different contexts. Thus, experience relies more on operational knowledge than domain knowledge and, thus, experience is highly contextual. We hypothesize that the mental representation corresponds to contextual knowledge, which is related to the operator (the expert), the task at hand, the situation of the work, and the local environment in which resources are available. By expressing contextual knowledge with contextual elements (and thus mental representation too), it is possible to externalize the mental representation with a classical knowledge-management tool like FreemindFootnote 2 as a cognitive map, that is, a semi-structured expression of the mental representation.

Operators may start from a domain map to develop their expert map. The domain map is a representation of the whole context (i.e. the contextual and external knowledge). A domain map can be obtained by different ways, from a state-of-the-art on the use of domain knowledge, or by developing a glossary to fix the terminology among experts [1]. All elements of the domain map belong to operators’ knowledge, but operators use only the part that is operational for task realization.

The expert map corresponds to the selection of the operational part of the domain knowledge effectively used by operators during the realization of their tasks. As a result, the expert map is a tree representation of the elements considered by operators. In terms of context, the expert map is a representation of the contextual knowledge (the part of the context) that operators relate more or less directly to their focus (i.e. task realization).

Figure 1 presents the general shape of an expert map expressing the organization of contextual elements in the mental representation of an operator of the experiment of the TACTIC project. This expert map is strongly inspired by the domain map proposed initially as a bootstrap. (Some operators introduced links between different leaves of the map).

Fig. 1.
figure 1

General shape of the expert map of a modeler

Full size image
Fig. 2.
figure 2

General shape of the expert map of a project manager

Full size image

Operators established their expert map with respect to all the tasks they have to realize. The (contextual) elements not used in the specific context of the session are represented in grey and with a red cross (see Figs. 1 and 2). The other part corresponds to the proceduralized context. The modeling of operators’ reasoning as contextual graphs uses these contextual elements kept in the expert maps (see next section).

Figure 2 shows another general shape of an expert map made by a project developer (i.e. a vendor of the simulator). The operator has a clear idea of what may interest future users of the simulator, and developed an expert map limited to the essential contextual elements (i.e. corresponding to his interaction with users), knowing that he will have at any moment the opportunity to retrieve information in his external-knowledge part of the context of his task realization.

The expert map is centered on the core of operator’s task realization with only 41 contextual elements. In term of context, the operator makes an efficient management of the ratio contextual knowledge versus external knowledge while other operators preferred to rely on the domain map.

An operator interprets a focus of attention based on a number of elements in his knowledge and experience with his task realization. These contextual elements constitute the mental representation that is associated with the focus of attention. In our approach, the elements of the mental representation are contextual elements that “do not intervene directly in the operator’s task but constrain how the task will be performed” [8]. Thus a failure in the mental representation (some important elements are missing) can lead the operator to not recognize (or not see) the event for what it is.

The expert map, being designed by the operator himself, is a good approximation of the expert’s mental representation of his expertise field. It is the operator’s signature. In some sense, an expert map is a kind of ontology of operational knowledge of the domain. However, the difference with usual ontology-based context (e.g. see [17]) is to do not deal with a domain-specific ontology. Indeed, if the task model (or procedure) is unique for all operators, each operator develops specific ways for his task realizations (practices) that include an explicit contextualization process. These contextual variants appear as soon as the degree of freedom increases in task realization. The reason is that the context of a task realization includes elements at the level of the operator (e.g. his preferences), of the task (e.g. selection of moments in the scenario), of the situation (e.g. many events occurs simultaneously), and of the local environment (e.g. movement of the enemy).

A decision-maker reasons in two steps [7]. The first step concerns a phase of data gathering, and the second step is the decisional phase. Generally, the first step corresponds to a global reasoning and the second step to a local reasoning. The global step of data gathering corresponds to the identification of the relevant contextual elements and their “instantiation” in the mental representation with respect to the current working context. For example, in Fig. 2 “Field” has four possible values, namely “map”, “tactical issue”, “tactical objects” and “shooting”. The gathering phase in a given context consists of the identification of the value of interest for the focus at hand (i.e. its instantiation). If the operator focuses on information retrieval about a given unit, “tactical objects” will be the instantiation of “Field”. Note that “tactical object” is itself a contextual element with different values (including “unit”). Thus, the expert map is assimilated to a search space where the operator looks for instantiations of relevant contextual elements in real time.

2.3 Example: Modeling Operator-Simulator Interaction

Interaction is a phenomenon between a user and a computer that is controlled by the user interface running on the computer. Designing interaction rather than interfaces implies that interfaces are the means, not the end [3]. This supposes to combine and understand the context of use with a particular attention to the details of the interaction. Different users work differently and a given user applies different interaction patterns according to the context of use [13]. As a result, no single interaction technique works identically in all contexts, and the best solution is to provide a range of interaction techniques and let users decide which one must be used according to the working context, although users may have context-aware support for choosing an interaction technique.

Figure 3 shows the two main changes in order to simplify operator’s task realization with a simulator. The first one concerns a clear distinction of the interface with operator-simulator interaction. The consequence is the separation of “domain_ actions” and “interface_actions” and a simple mechanism of translation between domain-actions and interface-actions by shifting the main problem of translation at the level of the exchange of interfaces. (This part will be described in another paper). This would facilitate the change of computer (say, from a PC to a touchpad). The second change is to compare the expert map of the operator with the “expert map” of the simulator for making them compatible, even across a translation in terms of “interface_actions”, for transmitting commands to the simulator and, conversely, for presenting results to the operator.

Fig. 3.
figure 3

A model of user-simulator interaction

Full size image

Thus, domain_actions will be more easily associated with interface_actions by taking into account the (shared) context of interaction, resulting in greater flexibility of the interface, not only with respect to the operator, but also with respect of task realization. This leads us to propose a “task realization centered approach” for designing the interface as a shared space (or shared context) between the operator and the simulator.

3 Reasoning Modeling in Task Realization

3.1 Introduction

The effective application of a procedure supposes to account for the working context in which the task must be realized. This leads operators to establish practices that are tailored to specific contexts. A practice is the way in which operators contextualize a procedure for taking into account their preferences, the particularities of the task to realize, the situation where the task is realized and the local environment where resources are available. The essence is to understand and model how work actually goes done (i.e. the practice or task realization), not what is supposed to happen (i.e. the procedure or a task model). This means to identify which contextual elements are important, and what their values are for the current focus of attention.

Contextual elements structure experiences (practices) differently, on the one hand, from the knowledge bases of expert systems represented in a flat way because context is not represented explicitly, and, on the other hand, from knowledge organization in an ontology where links between concepts depend on the domain (is-a, kind-of, etc.) while elements in our context model concern the operator, the task, the situation and the local environment.

A practice is developed jointly with the building of a proceduralized context, i.e. a context-specific model [6]. Thus, there are simultaneously the development of the practice (by looking for instantiations of contextual elements orienting the choice of the path to follow) and the realization of the task. Finding a good practice consists of the progressive assembling of the components that are relevant in the working context during the development of the practice. A “best practice” thus has a meaning only in a specific working context.

3.2 Representing Operators’ Practices in the Contextual-Graphs Formalism

A Contextual Graph (CxG) is a context-based formalism for representing all the different practices developed in different working contexts for a task realization [5]. Operator’s experience is represented by an organization of practices that are structured by contextual elements. Thus, a contextual graph is similar to an experience base focusing on the realization of a given task. This supposes, first, to use a formalism allowing a uniform representation of knowledge, reasoning and context, and, second, to have support systems with powerful functions for processing such a representation.

Formally, contextual graphs are acyclic and series-parallel due to the time-directed representation that garanties algorithm termination. Each contextual graph has exactly one root and one end node because the decision-making process starts in a state of affairs (i.e. a working context) and ends in another state of affairs and the branches express only different contextually dependent ways to achieve this goal. Each path in a contextual graph corresponds to a practice effectively developed in a working context leading to a specific solution.

Rogova [15] describes the Contextual-Graphs formalism as incorporating action and context nodes (variables and relationships) as well as paths through them. Although a contextual graph is not free of weaknesses, e.g. there is a problem with the lack of direct time representation. However, the formalism offers certain advantages over other approaches since it allows a representation of knowledge and reasoning in a way that is directly comprehensible by users. Thus, information in contextual graphs is useful and usable for users.

3.3 Modeling Reasoning

A practice represents the result of the application of reasoning with its choices (instantiation of contextual elements) and actions executed. Generally graph traversal in reasoning is lead between a “depth-first” strategy and a “breadth-first” strategy. The “depth-first” strategy goes to the finest possible granularity on a line of reasoning in order to anticipate the course of events. It assumes that we know what to do and how to get there quickly. This strategy allows studying the technical feasibility of an approach as well as the needs in terms of resources, and, in a second step, gradually expands this approach. Figure 2 gives an example of an expert map of this kind.

Conversely, the breadth-first strategy is applied when it is necessary to consider all possible situations first. The breadth-first strategy is observed in expert maps of operators evolving at a strategic decisional level that maintain important contextual elements, even if not directly necessary in the realization of their tasks. Figure 1 gives an example of expert map of this kind. For example, “environment” was considered as a part of “situation”, even if environment is the main source of contextual elements on the battlefield map. Indeed, the operator considers environment in expert maps through what the operator needs to extract of it for realizing his task, that is, operational knowledge. For example, it is only when the focus is on a zone of interest that operator performs a global reasoning, switching from a global reasoning (e.g. finding a zone of interest) to a local reasoning (e.g. exploring the zone of interest for detecting relevant features). The operator thus is interested by an external event more through its effect on his task rather than by event origin.

The CxG formalism allows the representation of a system at the tactical and operational levels. The contextual graph represents at the tactical level the different practices used to realize a task in various contexts, while the development of a practice in a specific context is made at the operational level.

3.4 An Example of Modeling “Manage a Unit”

Figure 4 gives the contextual graph of the mission “Giving an order of recognition”. It has three actions and three activities. The activity “unit manager” is found in two locations (pink ovals 62 and 67). The reason is that an operator initially chose a shielded but in seeking to define the scope of recognition around enemy, the operator realized that the enemy were in a city that was not screened the most appropriate unit for what he wanted to do (blue contextual element 65).

Fig. 4.
figure 4

Modeling of “Giving an order of recognition” in the Contextual-Graphs formalism (Color figure online).

Full size image

Figure 5 represents the modeling of the activity “Manage a unit” in Fig. 4 as a contextual graph accompanied by its legend. “Manage a unit” is rather a sequential activity beginning by the choice of an area where to select a unit (first contextual element), followed by the manner to choose the unit (second contextual element), continuing by checking if the selected unit may realize the required recognition mission, and finishing by positioning the unit in the center of the window of the field map. Brézillon [5] shows that such a sequential structure of a contextual graph corresponds to a pragmatic approach, while a parallel structure is closer of a procedure (like self-diagnose of a piece of equipment in its user manual).

Fig. 5.
figure 5

The contextual graph of the activity « Manage a unit »

Full size image

4 Related Works

The challenge to address concerns what actors are doing effectively, that is, their activity (and not the task) taking into account the actor, the task, the situation and the local environment where are available resources. A task is associated with a set of objectives, which are prescribed by managers, and assigned to the actor that must realize the task. The actor develops an activity to perform the task that includes his mobilization, the task at hand, the situation features, the objectives and the technical and organizational resources available to the actor. It is the well-known problem of separation of task and activity [10, 12], of procedures and practices [5], and of logic of functioning and logic of use [14]. Making context explicit as contextual elements allows us to consider all these heterogeneous elements of context in a uniform way and thus to handle practices.

In a classical case-based reasoning (CBR) scenario, a case consists of a problem description and a solution. A case contains a set of (structured) information entities, and optional artifacts. Structured information is represented as (attribute – value pairs), while the optional meta-information contains unstructured textual information. Atzmueller [2] uses stored cases (experiences) for selecting an appropriate task and method, reusing stored task-configurations that are similar to a (partially) defined characterization. The process of capturing and reusing complex task-experiences is lead in four main steps: experience retrieval, task instantiation, task evaluation and deployment, and experience maintenance. Thus, a case is recalled as a whole and its characterization is then adapted to the context at hand. In the Contextual-Graphs approach the practice (the equivalent of the case) is identified and developed during its use. The main difference here is that cases are represented in a relatively flat way in the base, while practices are structured by contextual elements in the experience base. In the CBR, the approach is “result-oriented” while the approach is “reasoning-oriented” in the Contextual-Graph formalism.

Clancey [9] proposed that solving a particular problem (e.g. diagnosing a patient) involves creating situation-specific models. “Situation-specific” refers to a particular case, setting, or scenario. “Situation-specific” is not “situated cognition” that refers to how people are conceiving and thus coordinating their identity, values, and activities in an ongoing process enabled by high order consciousness. In the CxG approach, context concerns an operator accomplishing a task in a particular situation in a specific local environment. The development of a practice is associated with the progressive building of a “context-specific model”. The “situation-specific model” is embedded in the problem solving as a static model-based description fixed initially and filled progressively during the problem solving. Conversely, the context-specific model (i.e. the proceduralized context) is built in parallel with the practice development with the movement of contextual elements entering and leaving the proceduralized context.

The hierarchical task analysis (HTA) [16] is a basic ergonomic approach used for thirty years in a wide range of applications, such as the design and evaluation of interfaces. The key idea is the decomposition of a task into subtasks at granularities finer and the rules relating to know whether a task should be performed or not. HTA is very similar to what is done in the formalism of contextual graphs. Indeed, a detailed example in [16] —the passage of a customer at the checkout of a supermarket—was the subject of a translation of the HTA in a contextual graph (one activity being actually a finer granularity of sub-task) with rules translated in terms of contextual elements.

5 Conclusion

Taking into account the end-user in the design loop is not sufficient. The design loop must also integrate end-user variability. Our work on expert maps—as expressions of mental representations—shows that each expert map is unique (the operator’s signature) and corresponds to a specific view on operational knowledge used in a task realization. Thus, the expert map can help to make a real contextualization of the interface to meet the needs of each operator (as illustrated for operator-simulator interaction in the TACTIC project). Expert maps can be extended by introducing domain_actions as instantiations that the operator has the habit of applying to each object in the domain. This would provide a description of the domain_actions linked to the operational knowledge on operator-side (in his expert map) and help a more or less automatic translation (at least for ranking them along operator’s preference) of domain_actions into interface_actions.

Operators’ reasoning, as shown in the contextual graph presented in Fig. 4, relies on operators’ operational knowledge for contextualizing the task “Give a recognition mission”. Operators contextualize at the operational level the procedures coming from the tactical level (for example, the various means used to monitor the progress of the mission). If different operators have specific views on a task realization (a result of our study with the large spectrum of expert maps), a contextual graph, which is the accumulation of practices developed by all operators realizing the task, may be used as a tool for sharing experiences among operators performing the same task, explanation generation purposes and a training tool for future operators. This has already been done in other areas where contextual graphs are used (e.g. see [1]).

The change from “operator communicating with the simulation” to “operator communicating with the simulator about the simulation” allows giving the interface the role of a flexible communication medium equivalent to a shared context through which the operator (with domain_actions in his reasoning) and the simulator (with domain_actions in his model of the battlefield) communicate with a simple translation in interface_actions (essentially, mouse clicks actually). In the TACTIC project, three complementary sources of information are commonly used together: spatial coordinates of objects (the field map), temporal coordinates (the chronology) and socio-technical coordinates (ODB). It seems appropriate to consider them as a cognitive tridimensional referential in which events take place. Thus, clicking on a unit, all information related to this unit would be extracted automatically for presentation according to the operator’s request. In the cognitive referential, a unit would be associated with a knowledge network with close information, such as the life bar, and other more distant information such as belonging to an automaton. Such a knowledge network should allow different views adapted to the desired level of aggregation of the context.

An important lesson is that design and development of an interface would be made along a “task-realization oriented” rather than “user-oriented” approach, thanks making context explicit. This finding is interesting because it would be easier to design a task-oriented interface than to develop a generic user-oriented or multiple user-oriented interfaces for the presentation of information.

A next step would be to model the relationships between domain_actions and interface_actions. A path to explore is to develop an “interface map” for representing operator’s mental representation of the interface in a similar way to the expert map that represents operator’s mental representation of the domain.

An expert map corresponds to the operational knowledge used by an actor in a task realization. There are as many expert maps as actors, but all expert maps have a large nonempty intersection. This intersection corresponds to the context of the task realization that is shared by actors. There are two lessons to retain. First, the common part of expert maps could be assimilated to a generic expert map for training new actors and may be the basis for developing a support system. Second, private parts of expert maps (i.e. the not shared parts) correspond to actors’ personal experience. However, the differences of organization of expert maps have to be studied first.

Depth-first and breadth-first strategies are another challenging aspect for modeling reasoning. Expert maps based on a breadth-first strategy are interesting for detecting weak signal because the expert keeps an “open mind” when analyzing a situation. His focus is not limited to the task realization in an isolated way, but replaced in the context of the task realization. Such actors reason at a strategic level and do not consider details of lower levels of the task realization. Conversely, expert maps based on a depth-first strategy correspond to experts that are able to decide rapidly at the tactical level which reasoning must be held (units managed by the simulator are at the operational level).