Keywords

1 Introduction

The graphic representation of information for the emergency management decision-making presents challenges because of the need to provide expert decision makers, subsidies to understand the reality of the situation awareness in a real-time scenario [1].

To support SAW, the information display has been used as a manner to present products and by-products of processing steps of the emergency intelligence, called situation assessment. In each set of information produced, the expert observes and takes guidance in the light of visualizations, and then makes the decision and acts according to the level of SAW.

These visualizations can be jeopardized when quality problems resulted in the acquisition, processing and interpretation of information are added to the SAW process. One of these problems is the uncertainty.

Pang et al. [2], it is described the uncertainty as a concepts which comprises inconsistency, doubt, reliability, inaccuracy and error, and may include statistics variations, errors, differences, noises, or missing data.

However, there is a consensus that uncertainty does exist and it is known, so it should be viewed and transmitted to the experts.

Thus, information visualization analysis is intended not only for evaluating available information and the way in which they are organized, but should also help guiding the expert as for the uncertainty of such information. To this end, quality metadata is used (metacues) to qualify the information. Such metadata can help inform the experts and help them measure the influence that each information has in decision making. When there is a lack of quality metadata, the visualization may allow some misunderstanding [1].

The state of the art in visualization uncertainties for situation assessment indicates that in order to graphically represent the information necessary for the SAW process, authors often describe visualization techniques for the dimensions specific to domains. However, the need to portray quality as a representation spread over several modules of acquisition, processing, and visualization of information in systems aimed at situation awareness, is still an area to be explored.

Every uncertainty information can directly change the decision-making process. This is due to the fact that there is no pattern to follow when it comes to issues of quality aggregated to information. The same thing happens with the data visualization process that does not follow any standard that indicates ideal data visualization for cognition under adverse conditions of uncertainty [2].

The present article introduces a new framework to orientate the creation of visualizations from uncertain information, propagated by processes of a group authorship situation assessment system. This framework also aims at demonstrating the impact of uncertainty on the graphical representation of information for the guidance of experts. To this end, it is evaluated methods of graphic representation of known uncertainties and new alternative proposals to make the framework feasible, specifically under the dimensions of completeness and dimensions related to time.

A case study in the analysis of context information for uncertain situation awareness of crime events is described. In this scenario, the framework is applied to the construction of visualizations in synergy with uncertain information propagated in the process, which is useful to the SAW process.

In Sect. 2 situation awareness applied to emergency management field is presented, as well as the importance of abstraction of situation information to the decision-making process. Section 3 presents the quality of data and information applied to the emergency management, as well as dimensions of quality and uncertainty. In Sect. 4, advances in visual representation of uncertainty are discussed. Section 5 illustrates in detail the processes that make up the framework. Section 6 presents the results of an implementation case study of the proposed framework for the design of uncertainty visualizations to the emergency. Finally, in Sect. 7, the findings of the case study results are presented and discussed, illustrating the contributions and future work.

2 Situation Awareness in the Emergency Management Systems

Situation Awareness (SAW) is a cognitive model which explains the understanding by an personal to dynamic and complex decision making systems. SAW can be modeled in three levels: perception, understanding, and projection.

The perception level is characterized by the identifying entities relevant of the environment. The level of understanding goes beyond the perception of the elements, including understanding the meaning and the evolution of states of these elements in view of the objectives. And the level of projection is characterized by the ability to project actions of the environmental features in the near future [3].

Since reaching the SAW state is a process that occurs in the operator’s mind, it is very important to understand the level of knowledge they obtained in the use of a system, so that failures in cognition do not affect the decision-making process. In the representation level of information, the goal is to provide visualizations in order to assist in the understanding of what is happening in the scene for the user to abstract the current status in relation to the problem, so that the projection of the sequence of activities is faithful in what concerns the environment [3].

SAW is the basis for decision-making, and its performance is commonly applied to the emergency call situations. The higher the level SAW maintained, the greater the effectiveness of the decision-making.

These systems aiming to assist the understanding of the situation are dependent on the quality of the information so that the information provided to system operators is the best in relation to the current situation, thereby assisting in cognition and decision-making. The system operator can be subjected to uncertainty when uncertain information is provided to the SAW system, affecting the decision-making process.

Given that uncertainty can be contained in the information, the visualization of uncertain data must provide the user with a resource for the acquisition, maintenance and recovering of situation awareness. The goal is to provide the decision-maker with visualizations that help the abstraction of the current state of situation so that the projection of the activities sequence of resources is accurate related to the environment [4].

So that decision-makers acquire SAW of an specific mission by means of visual media, it is important to use charts, graphs, timelines, and visual distortion of the scene in the composition, in an integrated manner or in fusion, by adding glyphs, and visual metrics able to help prioritize future actions, then making useful visualizations to support the acquisition /maintenance of SAW and decision-making by reducing the analysis time of relevant information.

3 Quality of Information in Emergency Field

Quality is considered one of the most important factors in the decision-making systems, since information with quality problems reduce the effectiveness of a support system for cognition, hinders the formation of the mental model and affects SAW process, resulting in a wrong decision-making.

The quality of data is a multidisciplinary concept. In the emergency field, limitations are commonly reported regarding the completeness of size, accuracy, consistency, and time [5]. Emergency management systems, has quality of data as a decisive factor, since the process of awareness of the operator’s situation is made based on the information that the system displays in real time. Such information, when lacking of quality, impairs creation of the mental model of the situation, which makes the projection of the activities sequence imperfect.

Although there is no consensus in general dimensions and metrics to be adopted in different fields that require knowledge of the quality of information, experts agree that both the presence of imperfect information and the lack of familiarity with quality meta-information knowledge can sacrifice all decision-making process, resulting in a decreased reliance on these systems.

According to Mecella et al. [6], you can define a basic set of dimensions that make up a definition of the quality of data and information in the emergency field, including the syntax accuracy, completeness, consistency, and time dimensions.

Pang et al. [2], describe uncertainty as a complex concept that encompasses several other concepts such as inconsistency, doubt, reliability, inaccuracy, and error. It can also include statistical variations, errors, differences, noise or missing data. More generally, the uncertainty of definitions implies that there are imperfections in the knowledge of users about a data set, process or outcome.

Among main sources of uncertainty in assessing situations, the main ones are the processes of acquisition and transformation of the data, steps where the data undergoes change from its initial shape [7].

The uncertainty in the emergency systems field is a process that takes place in the mind of an expert system due to quality problems above mentioned: imprecision, incompleteness, inconsistency, and time dimensions. When these factors are presented in the analysis of a situation, they can influence the system operator to an inaccurate and inconsistent opinion. Additionally, we consider uncertainty a generalization of such dimensions; an overall measure of quality information.

4 Advances in the Visualization of Uncertainties

Gouin and Evdokiou [4], present an advanced visualization developed for the program of the Future Command Post, called ‘Circular Blobs’, consisting of a three-dimensional terrain where circles represent the deployment force of the weapons base, the thickness of the line represents its strength, and the circle diameter represents the range of weapons. In this example, features such as color, size, shape, edges and thickness are used to represent the information.

Fricker and Macklin [8] introduce a map-based visualization in which resources are represented in different bases. All information is presented concerning quantity, resources and power level compared to other bases. In this example, colors in shapes and sizes are used to assist the representation, allowing the operator to easily understand the situation. The authors map routes that take the resources from allied bases to enemy bases. The information is presented in relation to criticality, amount of resources, power level (strength) of the base and whether it is running. This example conveys more information to the decision-maker since it uses color in the ground structure to transmit the degree of security of each region, which facilitates the recognition of the area to which the base belongs [8].

Pang et al. [2] present the results of the development of uncertainty visualization methods, among them radiosity, animation, interpolation, flow visualization, adding glyphs, adding geometry, geometry modification, attribute modification, sonification, and the use of psycho-visual techniques.

5 Uncertainty Visualization Framework for Improving Situational Awareness in Decision Making System

As part of a complete model of situation assessment, this paper aims to improve the process of SAW. To fulfill this objective, it was structured a framework that specifies the steps for the construction of visualizations aware of uncertainty which meet situation awareness requirements in management systems of emergency situations.

The framework was defined in order to support the phases of a situation assessment system, considering how each phase impacts on the representation of field information and on the observation /orientation of the expert.

The solution to be described aims to solve quality problems in emergency call decision-making, besided contributing to the acquisition, maintenance and recovering SAW state.

In order to do so, it is necessary that visualization assists the perception and understanding of the whole situation. Figure 1 shows how other assessment processes interact with the visual representation processes. In orange color, see the fundamental modules for the information visualization.

Fig. 1.
figure 1

Uncertainty visualization framework as part of a situation assessment process

Full size image

At first, it is necessary to relate specific sub-processes of information visualization with the acquisition phase at the beginning of the stream. This approach is relevant for two reasons: visually encode the raw data obtained from heterogeneous data sources, initially in transcribed audio of phone calls to the São Paulo State Police (PMES), or graphically represent objects and attributes identified in this step by natural language processing methods (NLP). This step of acquisition is best described by Junior et al. [9].

The visualization of raw data may be relevant to help the expert to infer entities not inferred by automated processes. In turn, encoding of objects and attributes is an essential part of SAW process, identified as a priority in the analysis of visualization requirements. According to [10] the needed objects to attend a robbery report are: criminal, victim, stolen object and event spot which are identified by the NLP [9]

When there is the need to visualize raw data, they are directly brought to the interface, without any prior review or quality assessment. In the case of the visualization of objects and relevant attributes identified via NLP and that are of interest to the expert, a JavaScript Object Notation (JSON) schema first transports them to an assessment of the quality of the inferred information, seeking a preliminary trial information.

In the primary evaluation phase, there is the analysis of synctactic accuracy, in which algorithms identify misspellings which can impair the assessment of the completeness dimension, measured by the attributes present in the complaint call. In addition to the completeness, timeliness is also inferred and scored. The return of this information is also in a JSON schema, this time also carrying quality indicators, useful for setting the visualization in the graphically represented quality [10].

The output generated by the acquisition and evaluation is regarded as a situation knowledge that should be represented. In the complete evaluation system, it was opted for the semantic model of ontologies to represent such knowledge. To date, objects, their attributes and possible relationships between objects are known. In situations assessment systems, it is appointed L1 and L2 levels of assessment, also corresponding to the levels of perception and understanding of SAW. This knowledge must be encoded in visualizations.

The process of abstraction and coding in visualizations is complicated when the representation contains a lot of information, since it may lead the expert to not notice all the elements in a given time interval. To mitigate these problems, a process of information filtering was used. Filtering is a process that helps determine which information should be represented, when it should be represented, how and where in SAW interface. This feature provides the user access to the high-level when available, and the low-level information when needed.

This abstraction is achieved by using graphics variables such as color, shape, size, animation, among others. When combined, these techniques allow the experts to easily get to the awareness of the situation due to the possibility to draw attention to the information that they consider important because of its quality score over others.

In addition to the acquisition processes for the generation of entities to be encoded, minings and integrations can be performed using the existing objects. This process, known as information fusion, is capable of producing, in a lesser extent, new objects and new semantic relations between them, which shall also be represented. The product of this step, called situation, is also subject to evaluations, enriches the existing situation awareness, and then encoded. Figure 2 shows an example of JSON schema to be encoded.

Fig. 2.
figure 2

JSON schema generated by the previous steps to be encoded in visualizations

Full size image

The encoding step results in visualizations of objects detected in the time of acquisition, fusion and evaluation, mapped in the visualization by using the above mentioned techniques of visualization of uncertain information such as colors, shapes, size, position, border, and transparency.

The requirements for the encoding of visualizations were defined based on the state of the art in visualization of uncertainties, and by mans of the rationale design methodology. This methodology helps the designer justify the interfaces and visualizations of design solutions for a given purpose. In addition, the authors have also been guided by the analysis result of the requirements of the PMESP in which the informational priorities have been defined to improve cognition. SAW global requirements were also considered, as described by Endsley [3].

The colors used in the representation of the instance vary according to the quality score of the information. The closer to 100 % sure, the darker the edge of the instance. The warning connotation that this type of visualization impose on the data allows the expert to better understand it [12].

The variation of colors of objects are presented as in the use of red-green scale described by Ware and Jiang [11, 12], where the purest shade of red is the uncertainty, and green stands for certainty. Metrics of vibrant and dark colors and such as red, blue, shades of black, etc., represent the uncertainty of information, and green, yellow, and shades of white present the visualization of certainties.

The size of representations indicate the value of the data quality score entered into the information. The greater the size compared with the rest, the higher the quality score of the represented information.

The application of the transparency in information helps to identify the degree of reliability of the information. The greater the transparency score, the lower the quality. The application of transparency in instances of objects does not mean they do not have quality, but is a warning that there is a problem of quality added to information. Figure 3 shows the visualizations developed from the framework described, in two modes of representation: overlays in geo-referenced maps and in hierarchical graph.

Fig. 3.
figure 3

Visualizations developed from the framework described, in two modes of representation: (a) overlays in geo-referenced maps and (b) situation attribute objects hierarchical graph.

Full size image

The use of overlays in geo-referenced map is because PMESP operations dependent on the location attributes, especially to determine how to approach an occurrence. Thus, other objects that make up a situation complete the information with other objects and attributes such as information about criminals, victims, and stolen objects, each with its description.

The adoption of the graph structure is justified by the need of hierarchical knowledge about the formation of information objects and situations. It is necessary that PMESP operators know how each situation was composed, which objects and attributes.

The use of the graph structure is justified by the exploration of hierarchical knowledge about the formation of information objects and situations. It is necessary that PMESP operators know how each situation was composed, which objects and attributes. This hierarchy was also obtained by analyzing the requirements by the PMESP. The situation is the central entity, constituted by the relationship between objects and their attributes, thus making its branches. Objects can have no relationship and, in this way, they may belong to independent hierarchies.

In our case study, a situation is a robbery event, and objects that constitute the situation are the victims, criminals, place, and stolen object, each with a set of characteristics called attributes

6 Case Study

The present case study aims to contribute to the decision-making process in the emergency field, more specifically to the decision-making process in the PMESP. The main objective of this study is to represent, by using visualizations, information that enable the identification of entities and situation compression of criminal charges, and also confirm the reported events. Since SAW is obtained, the expert has better subsidies to allocate resources and develop a better strategic plan of action. The acquisition of SAW in this context involves the identification of objects: criminal, victim, place, and stolen object and its attributes described in the complaint calls.

This case study specifically addresses situations of robbery, in which people make complaints by means of social networking posts or emergency calls. Such information is subject to the acquisition process, information fusion and quantification of quality. As an output of these processes prior to the visualization encoding, it was obtained the JSON schema shown in Fig. 4.

Fig. 4.
figure 4

JSON schema generated by the previous steps which must be encoded in visualizations of the case study.

Full size image

In visualization, situation is composed of edges that enable a relevant identification from the scores calculated by primary processing process.

The color information of the situation circle is applied in accordance with the total quality score received, where colors such as green, yellow, and shades of white are applied to represent the quality on a scale ranging from 0 to 100, where the closer to 100, the more quality; and the closer to 0, colors of blue, red, and shades of black in dark shades represent uncertainty.

The internal information regarding situation – instances of objects: victim, criminal, place, and stolen object – is classified under the same color scales described, as well as the use of size and edge, to which are also added transparency and shape.

In order to represent the fusion of information, visualization is attributed a higher score of opacity in color, indicating incidence of most significant amount. The use of shading around the representation is used to highlight the presence of information resulting from the fusion. To present the results of this process, a graph was created, as shown in Fig. 5, where the edges that concatenate the knots create an indicative of synergy information with potential to fuse. In this, attributes of quality are applied in the same way, using the sizes and colors of the knots to quantify the quality, as well as the use of edges.

Fig. 5.
figure 5

Visualizations developed from the framework described and using case study information, in two modes of representation: (a) overlays in geo-referenced maps, (b) situation attribute objects hierarchical graph.

Full size image

The use of filters is aimed at showing or hiding information to or from the map. In visualization, this technique is applied by the expert, who has the option to visualize or not the data which was not processes in the visualization. Figure 5 shows the results of visualizations with the case study information.

7 Conclusion

From the product resulting from the assessment steps, specifically acquisition and processing/fusion, it was possible to develop the framework in order to define uncertain information considering the phases of situation assessment and its impact on the graphical representation for the guidance of experts. Uncertain data visualization techniques were used, and the information generated by the use of the framework was encoded.

With this framework, it is expected that views dedicated to emergency rule can be created in synergy with the other steps of situation assessment.

So far, the results proved to be valid regarding the generation of such views in line with the assessment of the quality of information and information fusion.

The next steps include assessing the views generated regarding the acquisition and maintenance of SAW, together with decision makers from various levels of PMESP experience and expertise.