Keywords

1 Introduction

The process of building a multicriteria decision model encompasses several relevant issues. One of the most important of these issues is that of evaluating the criteria (or attributes) weights [1]. This is particularly relevant when the aggregation procedure is based on an additive model, given the meaning of these weights in the context of MAVT (Multi-Attribute Value Theory) [2].

The elicitation of the criteria weights may be considered as a central issue in MCDM/A (Multi-Criteria Decision Making/Aid) process, as mentioned in a recent survey [3]. There many studies related to this matter and broad overview on the elicitation procedures of weights for additive models can be seen in [4].

Previous behavioral studies on the main elicitation procedures in the context of MAVT, considering aggregation of criteria with additive models have shown several issues regarding inconsistencies in the process of preference modeling [5, 6]. Possibly due to these issues, several MCDM/A methods for dealing with additive models have been proposed in the literature [1]. One of these methods is the FITradeoff (Flexible and Interactive Tradeoff), which may reduce inconsistencies in the elicitation process [1], when compared with the traditional tradeoff procedure [2].

One of the main features of the FITradeoff method is the use of a flexible elicitation procedure [7] in order to reduce the timing spent in the elicitation process for establishment of criteria weights [1].

One of the mechanisms for introducing this flexibility in FITradeoff is the use of graphical visualization for analysis of intermediate results in the decision process.

In the preference elicitation process, FITradeoff selects a subset of potential optimal alternatives (POA), based on the current partial information available. The flexibility of FITradeoff allows the Decision Maker (DM) the possibility of interrupting the elicitation process in order to analyzing the partial results (POA) at each step in the process. This analysis may be conducted based on graphical visualization of performance of POA.

This process of visualization analysis is emphasized in this paper and illustrated with an application of FITradeoff for selecting a weapon to be incorporated in a navy ship [8]. It is shown that the visualization analysis may abbreviate the decision process, reducing the time spent by the DM. It is also shown that the decision could be based on the visualization information provided.

Therefore, evaluating the confidence of this visualization analysis is an important matter. Also, finding means for improving this confidence is relevant when preparing analysts for supporting DMs. Furthermore, information for the designing of this visualization is relevant in the process of building DSS (Decision Support Systems). This paper shows how these issues can be approached based on cognitive neuroscience, particularly when using eye tracking resources.

2 Application of FITradeoff DSS with Visualization

In this section the FITradeoff method [1] is briefly described followed by an application for selecting a weapon to be incorporated in a navy ship [8]. This paper shows how the use of the graph visualization can be improved in the method described in reference [1].

The decision process is considered with an emphasis to the use of graphical visualization in the analysis phase. The graph visualization for the elicitation questions is not considered in this paper, since it involves other issues to be analyzed.

2.1 FITradeoff Method

The FITradeoff method supports decisions in multicriteria problems in the context of MAVT, with partial information. This method uses a flexible and interactive approach [1] in such way that graphical visualization could be applied in order to shorten the modeling process.

Figure 1 shows the decision process with FITradeoff, in which preference statements are collected from the DM in step 1, in the preference elicitation process. These preference statements consist of choices the DM makes regarding a pair of consequences, which are well known for the traditional tradeoff procedure [2].

Fig. 1.
figure 1

FITradeof decision process.

Full size image

Since the very beginning, for each input from the DM, a set of LPP (linear programming problems) are applied in step 2, in order to select a subset of potential optimal alternatives (POA). A more detailed explanation as to how the initial POA is obtained is given in [1]. This POA reflects the partial result with the new preference statement obtained and added to the current partial information.

If a unique solution is found (in step 3), then, the process is finalized (in step 7). Otherwise, the DM may conduct a graphical visualization analysis in steps 5 and 6. If the DM makes a decision of not conducting the visualization analysis, in step 4, and confirming to continue the elicitation process, in step 8, then a new preference statement is collected, returning to step 1.

The graphical visualization analysis in step 5 can be conducted at any cycle in the decision process and may result in a decision, in step 6, if the DM chooses one of the current POA as a solution for the decision problem. If the DM considers that the graphical visualization is not enough to make a choice the process carries on going to step 8. Step 9 shows that the DM can get a partial result, if that is satisfactory.

Therefore, the DM has the flexibility of interrupting the elicitation process in order to analyzing the partial results by means of a graphical visualization of the performance of the POA. The performance in each criterion is shown for each alternative (POA) and the DM can check if it is possible to compare and distinguish their global performance, according to the tradeoff amongst criteria.

This graphical visualization can be conducted at any cycle in the decision process, in such a way that this flexibility of FITradeoff method is available at each interaction accomplished by the DM in the process.

As illustrated in Fig. 1, the flexibility is a key feature in the FITradeoff DSS. The steps 4, 6 and 8 show this very well. The DM has the flexibility of changing the usual sequence of the elicitation procedure in any of these steps. Then, the DM can follow another sequence, in which a decision on how to proceed is given. Furthermore, even when the usual elicitation procedure is followed by the DM, the FITradeoff DSS introduces the possibility of finding a solution (step 3) before finishing the steps that would be required in the traditional tradeoff procedure. This depends on the distribution of weights and on the topology of the alternatives [1].

Therefore, the DM may make a decision before go further in the elicitation process. This contributes for reducing the DM’s cognitive effort. Consequently, it may increase the confidence of the decision process, since several possibilities of inconsistencies and bias are avoided. It is worthy to emphasize that the FITradeoff method already introduces a marked reduction in inconsistencies in the elicitation process [1] when compared with the traditional tradeoff procedure [2]. That is why evaluating the confidence of this visualization analysis is important for the decision process supported by FITradeoff method.

2.2 An Application with FITradeoff DSS

In order to illustrate the use of the graphical visualization for analysis of POA an application of FITradeoff is given. This application is based on realistic cases and is related to selecting a weapon to be incorporated in a navy ship [8]. There are 30 alternatives to be analyzed and the following criteria are considered: Hit rate, Range, Fire rate, MTBF (mean time between failures), MTTR (mean time to repair), and Cost. All criteria are to be maximized, but MTTR and cost. This application has applied the FITradeoff DSS, which is available by request (www.fitradeoff.org).

Since the first phase of the elicitation procedure, in which the criteria weights are ranked, four alternatives are selected as POA. Then, the process continues and DM answer questions in the elicitation process in order to reduce the weight space and obtaining the solution to the problem. The questions asked to reduce the number of alternatives consist of the standard kind of question in the elicitation procedure, although in this case it is more specifically related to the scope of the traditional tradeoff procedure [2]. This is detailed in [1].

The number of questions will depend on the relative distribution of performance of alternatives and the distribution of weights [1]. The minimum number of questions in the traditional tradeoff procedure is expected to be 3(n – 1), where n is the number of criteria [1]. In this case 15 questions would be expected. However, the problem is solved after the DM answer the fourth question.

The number of POA with questions answered by the DM is:

  • after ranking the weights: 4 POA (alternatives: 20, 21, 23 and 28);

  • after first question: 4 POA, same previous result;

  • after second question: 3 POA (20, 23, 28);

  • after third (till eighth) question: same result;

  • after ninth question: 2 POA (23 and 28);

  • after tenth question: 1 alternative - solution found to be alternative 28.

Therefore, the visualization analysis can be conducted between these questions and the DM may be able to choose one of the POA as a solution to the problem. This could reduce the timing for the decision process. That is, if the DM is able to make a decision with the graph visualization, no further question is required any more. It should be noted that, although this particular decision problem in the navy context has a reduced number of questions until a solution is found (see steps 3 and 7 of Fig. 1), it could take longer. Also, in general the DM has not an idea of how long this process would take. That is why estimations about reducing times cannot be provided. In the following section the visualization analysis for this problem is illustrated.

2.3 Visualization with FITradeoff DSS

The Graphical visualization of FITradeoff DSS considers several types of graphs that are available to the DM at any stage of the interactive process. These graphs are commonly applied in different multicriteria methods, which may have additional different types of graphs. For the purpose of this analysis, subsequently shown, these kinds of graphs are suitable for comparing alternatives and have been widely applied. Following Figures shows two types of graphs most applied in different stages of the decision problem previously described.

Figure 2 shows the bar graph in an early stage, when the ranking of weights is obtained. In this graph, the set of four POA are grouped by criterion. The criteria are shown in order of weights from the left to the right. The performances of alternatives are normalized in a ratio scale (0 to 1, the maximum performance).

Fig. 2.
figure 2

Graphical visualization with FITradeof after weights ranking and first question - type bar graph.

Full size image

In Fig. 2, it can be evaluated if the DM is able to compare the four alternatives with this graphical visualization. It might not be possible for a DM to analyze a set of information with four alternatives in six criteria.

As it can be seen, for the first criterion with the greatest weight, the four alternatives have almost the same performance. For the second criterion alternative 21 has a performance a bit above 0.6. For this second criterion the alternative 20 and 28 have the greatest performance (close to 1), followed by 23 (around 0.9). In the third criterion, the alternatives have diversified performance and with 23 has the best performance.

The DM could consider that at the very beginning of the process it is not worthy to make a decision with so many items to be considered (4 alternatives × 6 criteria).

Figure 3 shows another type of graphical visualization with similar information given in the Fig. 2. Same situation is observed in Fig. 3. The amount of information make not easy to the DM to make a decision.

Fig. 3.
figure 3

Graphical visualization with FITradeof after weights ranking and first question - type radar graph.

Full size image

Figure 4 shows the bar graph after the DM has answered the second question, when the set of POA is reduced to three alternatives.

Fig. 4.
figure 4

Graphical visualization with FITradeof after second question - type bar graph.

Full size image

In Fig. 4, compared with previous graph, it seems easier to the DM to be able to compare the three alternatives with this graphical visualization. The analyst should discuss with the DM about the confidence of this analysis and the timing to be spent in the elicitation process.

Figure 5 shows the bar graph after the DM has answered the ninth question, when only two alternatives are selected as POA.

Fig. 5.
figure 5

Graphical visualization with FITradeof after third question - type bar graph.

Full size image

Figure 6 shows the radar type of graphical visualization with similar information to that given in the Fig. 5.

Fig. 6.
figure 6

Graphical visualization with FITradeof after third question - type radar graph.

Full size image

Now, in both graphical visualizations, the DM may see that alternative 28 has a better performance compared to alternative 23 in four criteria. This means that, for this specific problem, a decision made at this point in the process might be reasonable. One cannot wonder if a decision with two POA would often be possible. However, it depends on how the performances of the alternatives are distributed by the criteria. This has to be evaluated case by case.

3 Cognitive Neuroscience for Improving FITradeoff DSS

As demonstrated in previous section the flexibility of FITradeoff for interrupting the elicitation process and analyzing the partial result (POA) by graphical visualization might be an important resource in order to abbreviate the decision process and spend less time. However, as it has been shown in previous visualization, the differentiation amongst alternatives could be a hard process to the DM distinguishes. In that case the analyst should always consider with the DM, the tradeoff between the confidence of this process and time to be spent in the elicitation process.

Therefore, evaluating how confident this visualization analysis can be is important information to be presented to the DM by the analyst. This can be done by considering the hit rate of the best alternative chosen by the subject in the experiment.

Moreover, information regarding this confidence is useful for designing the DSS, with regard to this graphical visualization. In this section it is shown how these issues could be approached based on cognitive neuroscience. A particular focus is given on an experiment based on eye tracking resources.

3.1 Cognitive Neuroscience and Neuroeconomics

Cognitive neuroscience is concerned with cognition aspects and the biological processes, considering neural connections in the brain and its association with mental processes. Cognitive neuroscience is related to several fields and mainly to psychology and neuroscience [9]. It may have connections with many other areas, such as linguistics, philosophy, and mathematics. Experimental procedures from cognitive psychology are one of the methods employed in cognitive neuroscience that concerns this work. In association with cognitive neuroscience, neuroeconomics is closer related to the present study. It seeks to explain decision making in association with mental processes, considering how neuroscience findings are associated to decision models and economics.

Several different procedures and tools are used to study the biological basis of decision making and economic behavior. Analyzing which areas of the brain are most active while processing specific steps in the decision process may be useful to explain mental decision process and support the design of DSS. For that purpose neural imaging can be applied in human subjects, which can be done by fMRI [10] or EEG (Electroencephalography), for instance. These devices can give details of the brain with information on specific structures involved in a step of the decision process. The use of this kind of information and other related, such as response times, eye-tracking and neural signals that DM generate in decision making process, can be associated to the decision analysis process.

Associated to Cognitive neuroscience and neuroeconomics, other topics have been mentioned in the literature, which includes: Decision Neuroscience [11], Neuro-information system, Neuromanagement, and Neuromarketing.

Regarding the graphical visualization in FITradeoff the use of EEG may contribute to the investigation of areas of brain that are activated during the elicitation process, comparing for different types of visualization. It is important to evaluate which kind of visualization stimulates either an intuitive or a rational process, for instance. It can be explored the mechanism behind visualization decision-making, for elicitation and for analysis, examining how it interchanges with system 1 and system 2 [12] in the decision-making process.

In a recent study, eye-tracking experiment has been applied in order to identifying causal effects of intuition and deliberation [13]. One of the intended results is to identify how thinking mode affects the pattern for information collection in decision making.

Response times and fixation time in different regions of a graphical visualization are analyzed with eye-tracking in order to get insights for improvement of the design for the FITradeoff DSS.

Regarding response time one of the main features found to be analyzed in literature is the Hick’s Law [14]. According to Hick’s Law the time spent by the DM to make a choice among items is expected to linearly increase with the logarithm of the number of items.

In MCDM/A the items are not presented in the same way as it is in the previous neuroscience studies for Hick’s Law. For those studies the choice is made by comparing single options, such as in a study [15] related to food choices. In that study they applied an eye-tracking experiment in which subjects have chosen snack food items.

In graphical visualization for the FITradeoff DSS the items are not presented as single choices. The items are presented in a combination of alternatives and criteria. That is, the alternatives to be analyzed are decomposed in criteria. It has to be investigated how the Hick’s Law works in such a situation and how this can be applied to designing of DSS and recommendations for supporting analysts in their interaction with DMs.

There is another issue in graphical visualization for the FITradeoff. How should be considered the number of items, since there is a combination of alternatives times criteria? For instance, in Fig. 6 there are 4 alternatives and 6 criteria. One may wonder what would be the meaning of the number of items in Fig. 6. Would it be four alternatives? It could be considered 24 items (4 × 6)? Actually, it seems that the DM analyzes 4 items (alternatives), each one of them composed of 6 sub-items. Another possibility is that the DM analyzes how the alternatives perform for each criterion. For instance, which is the best alternative or the two best of them, for each criterion; or only for part of the criteria with higher weight?

3.2 Neuroscience Analysis for Visualization of FITradeoff DSS

An experiment has been design for exploring the use of cognitive neuroscience for evaluating the visualization for decision support in FITradeoff method. This experiment is following described with some preliminary results. So far, it has been observed an enormous potentiality with this kind of experiments in order to bring insights and new understanding on the decision process with visualization, which could be of interest for both: designing visualization in DSS and preparing the analyst for advising the DMs in a more proper way.

At this stage, the experiment considers visualization with bar graphs predominantly and including a few questions with other kind of graphical visualization.

The experiment has recorded data throughout the visualization analysis by the subjects, regarding eye position, duration in regions of interest and eye-movements. These recordings are done while performing different tasks. The basic tasks consists in choosing the best alternative from a set of alternatives, whose performance are shown for a set of criteria in graphs similar of those in Figs. 2, 3 and 4. In order to make their choices, the subjects have had as much time as they wished.

The set of alternatives are organized in such a way that they are ranked with a difference in performance of 5%, considering an additive model for aggregating the criteria.

Still, it has been thought about performing different tasks. For difference tasks it has been applied different number of alternatives × criteria, which included a combination of 3, 4 and 5 for both. This has made 9 different graphs to be analyzed. Another issue for differentiating the tasks is the distribution of weights of criteria, using a set of 9 graphs with equal weights and another set with skewed weights (decreasing form right to left order of visualization). It is worthy to note that these weights have been applied to design a set of alternatives, ranking them and computing their performance with 5% of difference.

3.3 Results of Eye Tracking Experiment

In the preliminary phase of the experiment several observations have been made and will be applied for designing other steps of this research. Nevertheless, some of these results are still being applied for designing changes in the DSS visualization and for instruction to the analysts regarding the use of visualization analysis in FITradeoff. Some of these observations are shortly described.

One of the information of this experiment regards the hit rate of the subjects, which can show how the confidence of graphical analysis changes with the number of items, for instance.

Concerning to the total number of items (alternatives × criteria), it has not been observed, so far, a correlation with the hit rate. This may confirm the conjecture on the association of timing and difficulty with the number of alternatives grouped by criteria or criteria grouped by alternatives. Similarly, Hick’s Law does not seem to work in such a situation with the total number of items, although additional investigation is needed relating to this issue.

Regarding the two sets of graphs with different distribution of weights, following results have been found, from equal weights set to that of skewed weights: (a) the hit rate decreased in around 20%; (b) the mean time of fixation has increased of around 15%; (c) there is a high correlation between hit rate and scholar degree and degree of knowledge on MCDM/A methods.

Regarding the pupil size it has been noted a constriction of the pupil for the experiment of the set of graphs with skewed weights, when compared the set with equal weights.

Those subjects with lower hit rate have shown pupil with higher dilatation. As regard to the left and right pupillary response, it has been observed that there are not significant differences in variation all along the experiment. In general the size of left pupil is higher than the right one.

The experiment has considered regions of interest to be analyzed. The graph has been divided mainly in three regions: left, middle and right. A fourth region of interest related to x-axis has been investigated, but practically no interest has been demonstrated by the subjects.

There was not a significant difference of areas of interest in the graphs for the two types of set of graphs with different distribution of weights (equal and skewed weights). The middle region is the area of most interest by the subjects, when analyzing the graphs. It seems that for the first set of graphs with equal weights, there is not a common pattern, when the number of alternatives varies, although in most cases there is predominance for seeking the middle region of the graph.

The preliminary results are being applied for formulating more specific research questions in order to advance in this research. Also, some findings can still be considered for improving the graphical visualization in decision processes. For instance, the quality of data visualization and its use for decision making can be shown with the analysis of the hit rate of each combination of number of alternatives × criteria. This analysis can give basis to the analyst on how advising the DM in making decisions with particular number of items in a graph visualization.

A tradeoff has to be made in the use of visualization, considering two factors: (a) its use reduces the timing in the decision process and may contribute to its quality; (b) the confidence of a straight analysis of those graphs by the DM. The latter indicates that the confidence decreases with the amount of items in the graph. The former is a well know result in behavioral decision making [5, 6] the probability of inconsistences increases with the number of questions put forward to the DM.

4 Conclusions

This paper has focused on the process of visualization analysis of FITradeoff method, which is an important feature for its flexibility. In order to illustrate the use of graphical visualization in intermediate steps of analysis with FITradeoff, an application has been considered. As it has been shown, the visualization analysis may contribute to reduce the timing in the decision process, which may contribute to its quality. This is straightforward as it can be seen in Fig. 1, since a decision with the visualization (step 6) stops the elicitation process not requiring additional question (step 8) to the DM. In this sense this work deals with quality of data visualization and its use for decision making/support.

Therefore, evaluating the confidence of this visualization analysis is an important matter. Then, finding means for improving this confidence is relevant in the designing of this visualization within the process of building DSS. Some considerations are given on how to approach these matters, based on ‘decision neuroscience’ field, which is correlated to cognitive neuroscience, and neuroeconomics.

The study has been focused in a particular MCDM/A method based on a compensatory rationality by aggregating the criteria with the additive model. This method uses visualization as an important feature for analysis, incorporating more flexibility in the decision process, although these tools can be dismissed by following the interactive elicitation process.

For future work, the neuroscience analysis of graph visualization for the elicitation questions is going to be considered, since it contains other matters that require specific examination. Also, it can be considered a non-compensatory rationality [16], although an evaluation of the most appropriate kind of rationality (compensatory or non-compensatory), for each DM, remains to be made.