1 Introduction

Multi-criteria decision making/aiding (MCDM/A) problems are characterized as problems with two or more alternatives evaluated in two or more attributes (Keeney and Raiffa 1976a, b; Belton and Stewart 2002; Figueira et al. 2005). These problems are very common in personal and professional situations, such as: selecting maintenance policies (Wang et al. 2007; de Almeida et al. 2015), outsourcing and supplier selection (de Almeida 2007; Chai et al. 2013), selecting locations (Zolfani et al. 2013; Demirel et al. 2017); and selecting equipment (Bazzazi et al. 2009; Lashgari et al. 2012). Thus, because these problems can be complex and therefore there being a need for approaches to tackle this, some multi criteria decision making/aiding (MCDM/A) methods have been developed, widely applied and reported in the literature.

In this context, related to MCDM/A methods to support the decision making process, graphical visualization can be used as a tool to complement them and help decision makers (DMs) reach a better understanding of the problem. According to Miettinen (2014), different forms of graphical visualization can be used in solving multiple criteria decision-making problems based on understanding the performances of different alternatives.

The graphical visualization in the flexible interactive tradeoff (FITradeoff) method (de Almeida et al. 2016) is the focus of this research, and neuroscience tools are used to understand DMs’ behavior. The purpose of this research is to investigate how DMs understand graphical visualization and, therefore, how this leads them to selecting the best alternative. A neuroscience experiment was developed using eye-tracking equipment, and the results were analyzed in order to improve the FITradeoff decision support system (DSS) and suggest insights to the analyst into the behavior of DMs (de Almeida and Roselli 2017; Roselli et al. 2018).

This paper is organized as follows. Section 2 offers a brief description of the neuroscience approach to decision-making; Sect. 3 describes the FITradeoff method; Sect. 4 presents the neuroscience experiment; Sect. 5 reports on the analysis developed from the experiment, while Sect. 6 discusses the findings of this analysis. Final remarks are made and lines for future research studies are suggested in Sect. 7.

2 The neuroscience approach to decision-making

Neuroscience takes a broad approach and was developed with a view to reaching a better understanding of the neural system and the mechanisms of functions of the human body. Regarding the latter, this approach can be considered to be multidisciplinary and has been used by many academic areas to analyze human behavior and to improve systems (Smith and Huettel 2010).

Due to the importance of neuroscience as a support tool to understand human behavior, several kinds of equipment have been developed and used in experiments. These measure body variables and induce conclusions about human behavior when the tools are used in studies to monitor interactions between humans and systems.

In this context, Sanfey et al. (2003) presented an experiment to analyze the limitations of classical economics and to explore models that may provide a real representation of a decision-making process. In order to conduct this research, functional magnetic resonance imaging (fMRI) was used to analyze brain activation during the conduct of a economics experiment called the Ultimatum Game. Similarly, Goucher-Lambert et al. (2017) and Khushaba (2013) presented an experiment to investigate how consumers choose sustainable products and types of crackers by seeking to evaluate consumers’ preference judgments.

In addition to fMRI apparatus, neuroscience also uses eye-tracking equipment, which measures ocular movements. Thus, using eye-tracking, Ares et al. (2014) presented an experiment to evaluate the influence of rational and intuitive thinking styles when yogurt labels were analyzed. And Guixeres (2017) presented an experiment using Super Bowl TV commercials to investigate the effectiveness of a new ad on YouTube.

Finally, using an electroencephalograph (EEG), which measures electric signals transmitted between neurons, Slanzi et al. (2016) presented an experiment to analyze brain activities when users observed information on websites.

Therefore, what has become ever more apparent is the relevance of neuroscience in providing insights into human behavior for many areas of knowledge, such as: economics, psychology, political science, consumer theory, marketing, and information systems. Thus, not only has neuroscience been applied to these areas of knowledge but specific approaches have been developed for them, e.g., neuroeconomics (Glimcher and Rustichini 2004; Fehr and Camerer 2007; Rangel et al. 2008; Mohr et al. 2010), NeuroIS (Riedl et al. 2014; Slanzi et al. 2016), consumer neuroscience (Khushaba 2013; Goucher-Lambert et al. 2017), neuromarketing (Morin 2011; Guixeres 2017).

With regard to MCDM/A problems, the neuroscience approach can also be considered an important ally that seeks to provide a fuller understanding of DMs’ behavior. However, despite the relevance of this approach, a review of the literature has revealed that there are few papers which relate multi-criteria decision making problems to neuroscience approaches.

As to papers found after searching the literature, neuroscience was cited in studies for understanding the multi-attribute relation and as the subject of a future research study (Kothe and Makeig 2011; Sylcott et al. 2013; Hunt et al. 2014; Brookes et al. 2015). However, it was not found, until now, papers in the literature on applying neuroscience experiments in MCDM/A methods so as to improve these methods by the understanding of DMs’ behavior.

Therefore, this paper presents a neuroscience experiment in order to evaluate how DMs understand graphical visualization in the FITradeoff method. A brief description of the FITradeoff method and the background to it is given in the next section.

3 FITradeoff method

In the context of Multi-Attribute Value Theory—MAVT (Keeney and Raiffa 1976a, b; Belton and Stewart 2002), the flexible interactive tradeoff method—FITradeoff (de Almeida et al. 2016) was developed in order to elicit criteria scaling constants. This method has the same axiomatic structure as the traditional tradeoff procedure (Keeney and Raiffa 1976a, b). It has two main steps: to rank the criteria weights and to elicit the values of the criteria weights.

In the first step, the DM ranks the criteria weights by considering the range of values of the consequences in each criterion, just as in the traditional tradeoff procedure. The output of this step is the inequality in (1), where \(k_{i}\) is the scaling constant of criterion \(i\), and \(n\) is the number of criteria.

$$k_{1} > k_{2} > \cdots > k_{n}$$
(1)

The second step is the elicitation of criteria weights, which is conducted based on the DM’s comparison of the hypothetical consequences. In this step, the DM states strict preference relations for each pair of consequences compared. The comparisons are made based on pairs of adjacent criteria, i.e., from (1) \(k_{1}\) and \(k_{2}\)\(k_{n - 1}\) and \(k_{n}\), in such a way that the best outcome of the worst criterion of the pair (\(k_{2}\)) is compared to the lowest outcome of the best criterion (\(k_{1}\)) (de Almeida et al. 2016). After each comparison has been made, i.e. DM express his/her strict preference, the linear programing problem (LPP) model in (2) runs for each alternative \(j\), in order to test the potential optimality of each alternative.

$$\begin{aligned} & Max_{k1,k2.k3, \ldots ,kn} \mathop \sum \limits_{i = 1}^{n} k_{i} v_{i} (x_{ij} ) \\ & s.t \\ & k_{1} > k_{2} > \cdots > k_{n} \\ & k_{i} v_{i} \left( {x'_{i} } \right) \ge k_{i + 1} \\ & k_{i} v_{i} \left( {x^{\prime\prime}_{i} } \right) \le k_{i + 1} \\ & \mathop \sum \limits_{i = 1}^{n} k_{i} v_{i} \left( {x_{ij} } \right) \ge \mathop \sum \limits_{i = 1}^{n} k_{i} v_{i} (x_{iz} ),\quad z = 1,2, \ldots ,m;\quad j \ne z \\ & \mathop \sum \limits_{i = 1}^{n} k_{i } = 1 \\ & k_{i} \ge 0,\quad i = 1,2,3, \ldots ,n \\ \end{aligned}$$
(2)

In (2), the objective function tries to maximize the global value of alternative \(j\), which is given by the weighted sum of the scaling constant of criterion \(i \left( {k_{i} } \right)\) by the value of alternative \(j\) in criterion \(i\), \(v_{i} \left( {x_{ij} } \right)\). The first constraint is the inequality obtained from ranking the criteria weights in (1). The second and third constraints are obtained by comparing consequences based on pairs of adjacent criteria, where \(x'_{i}\) and \(x^{\prime\prime}_{i}\) are arbitrary outcomes of criterion \(i\) and \(x'_{i}\) is higher than \(x^{\prime\prime}_{i}\). The fourth constraint guarantees that alternative \(j\) is potentially optimal, i.e., the global value of alternative \(j\) can be greater than or equal to the global values of all other alternatives \(z = 1,2, \ldots ,m;\,j \ne z\). Criteria weights are normalized and non-negative, as guaranteed by the fifth and sixth constraints. In the LPP model in (2), if alternative \(j\) has no feasible solution, then alternative \(j\) cannot be optimal for this problem, and therefore is eliminated from the process; otherwise, alternative \(j\) stays in the process as a potentially optimal alternative (POA), i.e., as a candidate to be the optimal alternative at the end of the process.

The process is interactive, and as the DM compares consequences, more constraints are included in the LPP model, which runs again at each interaction. The elicitation process continues until a unique alternative is found to be potentially optimal for the problem.

The process for eliciting criteria weights in FITradeoff is different from that of the traditional tradeoff procedure. In the latter, the DM compares consequences, seeking to find the exact point that makes the two consequences indifferent for him/her. According to Weber and Borcherding (1993), identifying indifference points is a difficult task for DMs, which leads to 67% of the inconsistencies in the results. In FITradeoff, finding indifference points is not necessary, since the method works based on inequalities obtained from strict preference statements. FITradeoff requires information from the DM that he/she finds cognitively easier to provide than in the traditional procedure, which leads to less inconsistency in the results.

Another advantage of the FITradeoff method is that it provides information that can be viewed as graphs. In FITradeoff, bar graphs, bubble graphs and spider graphs are available for the DM to view in the DSS in order to make the decision-making process easier. The DM can view them at any moment of the process. They illustrate the performance of the current subset of potentially optimal alternatives, in such way that the DM can compare them and even make a global evaluation, with the possibility of finishing the elicitation process at that point rather than only after considering all POAs. Examples of FITradeoff DSS graphs are given in Fig. 1.

Fig. 1
figure 1

Graphics presented in FITradeoff DSS

In Fig. 1, each color represents an alternative of the subset of POAs. The bars, tips of the stars and bubbles represent the performance of the alternatives in each criterion, normalized on a 0–1 scale.

FITradeoff is considered a flexible method because the DM can use graphical visualization to select one of the POAs and interrupt the elicitation process at any time. This consequently leads to time and effort saving for DMs. The FITradeoff process is summarized in Fig. 2, and the DSS is available by request to the authors at www.fitradeoff.org.

Fig. 2
figure 2

FITradeoff process

4 Design of an behavioral experiment with neuroscience tools

According to Kasanen et al. (1991), visualization systems were developed to transform numerical data into graphical images. In this context, based on FITradeoff process, the graphical visualization can be used to represent the POA in a global evaluation with the possibility of finishing the elicitation process if the DM’s desire. Thus, in order to evaluate how DMs understand graphical visualization and select the best alternative, a neuroscience experiment was constructed. The implicit objective of this experiment was to evaluate how the participants deal with decision problems which present only graphical information. The results of this experiment were used to support the use of graphical visualization in the FITradeoff method, i.e. improve the FITradeoff DSS, and suggest insights to the analyst about graphical visualization recommendation for MCDM/A problems, since they are in MAVT (Keeney and Raiffa 1976a, b; Belton and Stewart 2002) scope. The focus of the research was in the specific step of graphical visualization in the FITradeoff method, presented in blue on Fig. 2.

Therefore, the experiment conducted in the present research consists of presenting different types of graph used in FITradeoff method. The graphs presented to the participants were coming from the FITradeoff method and were organized in such a way that covered a broad range (see the range of visualizations in Appendix 1) of possibilities given by FITradeoff method. If we consider a specific decision problem as an instance, it would not be useful for a more general result. Instead of that, this broad range of possible graphs that can be generated by FITradeoff DSS had been carefully organized to be used in this experiment. Therefore, in this way, we can have a more consistent and general result to be obtained. Since the FITradeoff is built in the scope of MAVT (Keeney and Raiffa 1976a, b; Belton and Stewart 2002) with the Additive Model for aggregating the criteria, this has been consistently applied to compute the best alternative in order to evaluate the HR.

Thus, each visualization represented a possible MCDM/A problem, which is part of that range mentioned. In order to have a more general result, the graphics had been presented out of particular contexts. That is, the participant should consider only the performance given in that particular visualization. For instance, if a bar graph is considered, the information used consists in comparing their height for different alternatives. Alternatively, if a particular decision context is taken into account for comparing those alternatives, then a bias would be introduced.

In order to build this broad range of MCDM/A problems, different decision matrices were compiled with different combinations of items (alternatives vs. criteria) and different scale constants [same weights (S) and different weights (D)]. From these matrices, twenty-four different graphics were developed which had three, four or five alternatives and criteria.

Eighteen of the twenty-four graphics were Bar Graphs (G), which were split into two groups of nine. The first group had the same weights for the criteria and the second group had different weights for the criteria. Each type of visualization has an acronym, which is fully presented in a glossary in Appendix 1. For example, GS4A5C was the acronym developed for the bar graphic with the same weights, 4 alternatives and 5 criteria and GD4A5C was the acronym developed for the bar graphic with different weights, 4 alternatives and 5 criteria.

Additionally, more types of visualization than currently existing in FITradeoff method have been included in this experiment, so that an evaluation of other possibilities can be done. The others six visualization were split into: one bubble graph (GBubble4A5C), one spider graph (GSpider4A5C), two tables (T3A5C and T4A5C) and two bar graphs with table (GT3A5C and GT4A5C). All these graphics had the same weights for the criteria.

In order to evaluate different profiles, the twenty-four graphics developed were mixed into three distinct sequences. These sequences were constructed to analyze how DM evaluated these graphics when they were placed in a different order, which was related to the number of items and degrees of difficulty.

So, the first sequence (S1) presented the bar charts with same weights first and then, the bar charts with different weights, ranging from the easiest combination of items (GS3A3C) to the hardest combination of items (GD5A5C). The second one (S2) presented the bar charts with different weights first, then the bar charts with same weight, from the hardest combination of items (GD5A5C) to the easiest combination of items (GS3A3C). The last sequence (S3) presented the bar charts in a random way. Finally, the other six charts were presented in the middle for each sequence.

Based on these sequences, three similar experiments were developed and conducted using the X120 eye-tracker by Tobbii Studio in the NeuroScience for Information and Decision (NSID) laboratory. At the end of interviews, thirty-six recordings of the eye movements of graduate and postgraduate management engineering students of Federal University of Pernambuco were used in the experiment. This sample was composed for fifteen women and twenty-one men, 16 graduate students, 10 master students and 10 Ph.D. students, there are distributed with following range by age: 23 are below 26 years old and 13 above. A sample of twelve recordings was used for each sequence, which were scheduled at the convenience of the researcher. Finally, the research project was approved by the Ethics Committee of the University before the data was collected. The experiment consist in the presentation of an instruction (‘Analyze the following graphics and answer, for each one, the question: Which is the best alternative?’), the graphical visualization and it respective questionnaire. This sequence repeat until all the 24 visualization was showed. The Fig. 3 present the GS4A5C and GSpider4A5C and the Fig. 4 and illustration of how the experiment was applied using the equipment.

Fig. 3
figure 3

The GS4A5C and GSpider4A5C

Fig. 4
figure 4

A participant in the experiment

5 Results of the experiment

In this section, some of the variables collected in the experiment are illustrated and analyzed to investigate human behavior by using graphs and thereby to reach some conclusions for the purposes of this research study. The variables description and collection process are presented in Appendix 2.

The fixation duration (FD) and fixation count (FC) variables were collected directly from the eye-tracking recordings. The first variable (FD) represents the time in milliseconds that each participant spent looking at each graphical image. And the second variable (FC) represents the number of fixations that each participant made when looking at each graphical image. In this context, to simplify future analysis, a final value of FD and FC for each graph in each sequence was obtained from the median, mean and standard deviation measures. The results are shown in Tables 1 and 2.

Table 1 FD values for each graphical visualization
Table 2 FC values for each graphical visualization

The diameter of the pupil of the left eye (LEPD) variable was also collected directly from eye-tracking recordings and represents the size, in millimeters, of each participant’s pupil, when he/she analyzed each graphical image. In this research, only data on the left eye data were selected. This is supported by results in the literature which presents indifference by analyzing either eye (Sharma and Gedeon 2012).

Exactly as for FD and FC, to simplify future analyses, two aggregation measures were calculated. The first one was calculated for each participant in each graphic image, which represents the average size of the pupil per graphical visualization. And the second one was calculated for the twelve mean values of LEPD, which represents a unique value (median) for each graphic in each sequence. By using these simplifications, it was observed that LEPD ranged between 4.08 and 4.70 mm for all the visualization charts.

The hit rate (HR) variable was developed for this experiment specifically. The HR was obtained by comparing the participants’ answers to each graphic with the Additive Model answer, obtained by the researcher. Thus, this variable corresponds to the ratio of the number of correct answers to the total number of answers and can be used to express the percentage of success for each graphic. The HR results are shown in Table 3.

Table 3 Hit rate values

The quality interval (QI) was also implemented specifically for this research. This quality interval was constructed using five categories to classify the graphs in accordance with the maximum percentage of wrong answers, the limits for this classification has been chosen based of their meaning for advising a DM on their choice of whether using or not those visualizations, based on practical considerations. For instance, 50% of wrong answers is the limit for the last class, which is assumed to be unacceptable. On the other hand, the first class has a limit of 5% of wrong answers, which is assumed to be acceptable; that is very good. These five categories as shown in Table 4.

Table 4 Quality interval with five categories

After collecting these variables, some analyses were developed. The first was characterized as a descriptive analysis constructed from HR and QI. Directly related to the second purpose of this research, this analysis was developed to support the analyst in his recommendation about which was the specific graphic to use in MCDM/A problems, in the MAVT (Keeney and Raiffa 1976a, b; Belton and Stewart 2002) context.

In this context, the researcher estimated the minimum confidence level (MCL) to help the analyst formulate his recommendation. This MCL was developed initially from the mean of HR values for similar graphs. Thus, for bar graphs with equal weights compared to those with different weights, the MCL was obtained. This is shown in Table 5. Also for the others six types of visualization, the MCL was obtained by comparing each one with the corresponding bar graph, as shown in Table 6.

Table 5 Minimum confidence level for bar graphs
Table 6 Minimum Confidence Level for Six other types of visualization

The second analysis, which is a statistical one, is to investigate the relationship between the variables collected directly from the eye-tracking (FD, FC and LEPD) and HR. In this context, the non-parametric Spearman correlation test was developed to evaluate the intensity of causality between these variables, as shown in Table 7, and to suggest some conclusion about human behavioral which is discussed in the next section.

Table 7 Results of the Spearman correlation test

Finally, the last analysis was developed using variables collected in areas of interest (AOI) designed by eye-tracking software. Areas of interest are areas designed in the eye-tracking software, which are necessary to collect variables related to eye-movements. In this research, these AOI were designed in some graph images to collect some specific variables. For example, for bar graphics with different weights, AOIs were designed in each criterion for the purpose of collecting FD and FC values and investigating the differences in how criteria are visualized. Figure 5 shows how AOIs are designed.

Fig. 5
figure 5

AOI’s design for a bar graphic

In this context, before data collection, to simplify AOI analysis, the Spearman Correlation test was conducted to analyze the relationship between FD and FC variables. Based on the results obtained, a correlation value above 90% between FD and FC was observed for all sequences. Therefore, only FD values were collected and compared for all AOI regions by the researcher. The criterion most looked at (ML) and the second most looked at (SML), for each sequence, is presented in Table 8.

Table 8 AOI analysis

The next section discusses the results obtained from the analysis presented in this section. This discussion is undertaken so as to provide explanations about variables used and results obtained, and thereby seeks to provide some conclusions for the two main purposes of this research study.

6 Discussion of results

The neuroscience experiment was developed for two main purposes, namely: to improve FITradeoff DSS and to give insights to the analyst about the visual analysis for MCDM/A problems, in the scope of MAVT (Keeney and Raiffa 1976a, b; Belton and Stewart 2002). In this context, the forms of analysis developed and presented in the previous section offer some suggestions for these purposes. Therefore, in this section, the results obtained above are discussed, starting with the first purpose.

The first purpose is to improve FITradeoff DSS, for which it was relevant to develop the AOIs analysis and conduct the Spearman Correlation test. The analysis of the AOIs, the results of which are given in Table 8, provided evidence that criteria 1 and 2, which received the highest value of weights, were the most visualized, which highest FD, in bar graphics. These criteria were positioned in the left and central region of the graphics. Thus, this analysis can confirm the previous design of MCDM/A graphics in the FITradeoff DSS, where higher weights are properly positioned (left to right) in the visual analysis of MCDM/A problems. Although, this seemed to be obvious since the beginning, it was not found evidence in the literature supporting more formally this result.

The second analysis, the Spearman Correlation test, was developed to explore the relationship between the HR and eye movement variables (FD, FC and LEPD), thereby seeking to find some evidence about degrees of difficulty in interpreting the graphics developed and presented in FITradeoff DSS. These variables were collected by using the eye-tracker, based on evidence in the literature that higher values of fixation and pupil diameter are generated by higher mental efforts (Porter et al. 2007; Laeng et al. 2012; Kim et al. 2012; Slanzi et al. 2016; Bault et al. 2016). However, because of the absence of a correlation between the variables evaluated, this analysis cannot provide any conclusion based on statistic evidences, about the difficulty related to interpreting graphical visualization in FITradeoff DSS. Even this result does not provide a directly recommendation, it is importance to yield more knowledge about eye movement variables and promote suggestions for future research.

With regard to the second purpose, which is to give insights to the analyst about graphical visualization recommendation for MCDM/A problems, the analysis of MCL can directly provide a guide as to the success in using some types of visualization in multi-criteria decision problems. In other words, it should be observed in Fig. 2 that the DM can skip the elicitation process and make a decision by means of a visual analysis, in which he/she chooses one of the alternatives among the POA available, assuming that the chosen alternative would be the best. At this point, this results can support the analyst in making recommendations whether this visual analysis is or not reasonable confident. For instance, for a situation with 5 alternatives and 4 criteria the analyst can state to the DM that there is probability of 75% that this visual analysis will provide the best solution, assuming that the HR found in this experiment is a reasonable estimator for that probability.

Moreover, the analysis of MCL can be used as a complement to the Spearman Correlation test in order to compare some types of graphical visualization and suggest improvements to FITradeoff DSS. Thus, based on the results in Table 6, it was observed that Tables T3A5C and T4A5C obtained a higher HR than other types of visualization. Therefore, the inclusion of Tables in FITradeoff DSS can be a relevant suggestion, as this adds more flexibility to this method since extra items of information in addition to graphical information are included.

Figure 6 synthetized the neuroscience experiment constructed. In this figure a flowchart was built to connect the phases of this research, such as: the research question, the variables collected, the analysis developed and the suggestions presented in this section.

Fig. 6
figure 6

Conclusion about neuroscience experiment

7 Conclusion

Multi-criteria decision making/aiding (MCDM/A) problems are strongly present in people’s personal and professional lives. Because these problems can be complex, many MCDM/A methods have been developed and published in the literature. The neuroscience approach, which is about studying how the neural system functions, is an important resource which can be used to improve data visualization in systems based on a better understanding of human behavior.

In this context, related to the relevance and complexity of multi-criteria decision problems and the benefits that can be derived by using the neuroscience approach, this research study was developed to evaluate graphical visualization in the FITradeoff method because it makes FITradeoff more flexible and easier to use.

According to Miettinen (2014), graphical visualization is a very important tool in decision support systems (DSS), so this research had two main purposes: to improve FITradeoff DSS and to give insights to the analyst as a result of using graphical visualization to support MCDM/A problems, in the scope of MAVT (Keeney and Raiffa 1976a, b; Belton and Stewart 2002).

Therefore, to satisfy these two purposes, a neuroscience experiment was developed, which presents some types of visualization and evaluates how DM understands them. It was constructed using eye-tracking equipment and was applied to graduate and postgraduate students in Management Engineering. Based on the variables collected and analyses developed, some suggestions were obtained and are presented in Fig. 6.

So, related to the first purpose, improving FITradeoff DSS, it was confirmed that the position of weights is adequate in graphical visualization for MCDM/A problems. Moreover, the inclusion of tables in FITradeoff DSS was suggested. With regard to the second purpose, this paper has presented a minimum level of confidence (MCL) for each type of visualization that was a directly recommendation to the analyst about the use of graphical visualization in MCDM/A problems.

Despite of the importance of neuroscience approach for enriching of MCDM/A techniques, the construction and application of a neuroscience experiment have scope, time and sample limitations. In this context, future researches are necessary for the solid investigation of the graphical visualization evaluation in MCDM/A problems. For this reason, it is already planned to add electroencephalograph (EEG) in order to investigate differences in DM profiles. Moreover, other types of graphics can be included to produce different experiment design’s to strengthen the application of graphical visualization in MCDM/A problems.