Keywords

1 Introduction

In vast majority of real-life situations, making decisions requires a thorough and time-consuming trade-off analysis of the performances of many alternatives with respect to many and usually conflicting criteria. Therefore, a variety of multiple criteria decision aiding (MCDA) methods are proposed to support decision maker (DM) in their decision analyses [11, 32]. Various MCDA methods differ in the philosophy of preference elicitation, the aggregation algorithms they use, and the information their require from DMs [30], which often makes it difficult to choose the one that will be suitable to support DM in a particular decision making problem. Therefore, some researchers even propose the guidelines for selecting an appropriate MCDA technique, depending on the type and context of the decision making problem or the requirements imposed on the process of the preference elicitation [13, 16, 27], to make sure that the preference analyses and the final results will be sound and reliable.

There are, however, some other factors, such as the behavioral ones, that may affect the results the MCDA methods produce [21, 25]. Some of them are related to the decision making skills and cognitive abilities of DMs, i.e. to the thinking/cognitive style and the way of analyzing of the decision-making processes [8, 10]. DMs that think fast and do not analyze the facts thoroughly are more prone to make some information processing errors that result from using intuition and heuristics [12, 33]. Consequently, they may misuse the decision support tools and produce false conclusions based on the recommendations provided by the decision support systems. Some empirical results show the potential relationship between cognitive capabilities and the decision making process and its results [7, 22]. Therefore, it seems vital to analyze, what decision support mechanisms fit the thinking styles the best (i.e. do not require an effort higher than the cognitive capabilities of DMs), and hence offer the facilitation that increases the chances for more accurate preference elicitation and generate final recommendations that are reflecting the intrinsic preferences of DMs best.

However, understanding the relationships between information processing style and the preferable decision aiding tools may be biased by the perspective used in describing the nature of style constructs themselves [10, 28]. Depending on which approach is used, i.e. unitary (e.g. parallel-competitive) or dual (i.e. orthogonal, e.g. default-interventionist) one, the conclusions on how the style affects the decision making effects may be different. In the former approach, the conclusions may be formulated based on the single index describing the single bipolar rationality-intuition dimension, e.g. the highly rational DMs are perceived simultaneously as little intuitive. In the second approach, both constructs form separate dimensions and hence the style is perceived as a mix of various intensities of both constructs, e.g. DM can be considered as both highly rational and intuitive.

Taking the above issues into account, the goal of this paper is twofold. First, we aim at investigating how the cognitive style affects: (1) the DM’s expectations towards the form of the support mechanisms offered in the decision support system while analyzing the multiple criteria decision making problem, and (2) their actual selection of the MCDA method as most useful in solving real-world decision making problems. In particular, we analyze the DMs opinion regarding the most preferable and efficient way of preference representation that can be offered in the support system in quantitative or qualitative way; e.g. by means of numbers, linguistic terms, verbal descriptions or pictograms, and confront these opinions with the final choice of the decision support mechanisms that they recommend. Second, we wish to verify if the dual/orthogonal approach for measuring the cognitive style allows for better (more detail) description of the opinions and choices made by the DMs when evaluating the MCDA techniques.

In our analyses we use the results of the experiment conducted in online survey system (OSS). In the experiment, the decision making problem was predefined and three selected MCDA methods were implemented to support DMs, namely: AHP, SMART and TOPSIS [26]. Having completed the decision making phase, the respondents evaluated the OSS itself as well as the methods and their interfaces and provided the opinion regarding the most preferable, adequate and informative design of decision support mechanism. To describe cognitive profile of the participants, the Rational-Experiential Inventory (REI) test was used [17], which allows to identify two dimensions of such a profile: rationality and experientiality [8]. To identify the relationship between the cognitive style and the expectations regarding the most preferable way of support the correspondence analysis was used, which maps the relationships from multi-dimensional matrix into a two-dimensional space (using the aggregation that keeps as much of original information as possible) and hence allows to analyze them based on the notion of distances.

The rest of the paper is organized as follows. In the Sect. 2 the behavioral aspects of decision support are discussed and research questions asked. In Sect. 3 we described the experiment we had conducted, while in Sect. 4 the results are presented. We summarize the results in Conclusions and answer the research questions.

2 Behavioral Factors and Decision Support

2.1 Investigating an Impact of Thinking Styles on Decision Process

There were early works of Simon, Kahneman and Tversky that paid attention to the behavioral issues of decision making processes and the cognitive limitation of DMs [29, 33]. The postulates of including some notions of behavioral analysis into the process of verifying the effects of decision support and the usefulness of decision aiding tools were and still are raised by the MCDM methodologists [21, 25, 34]. Some experimental works show that various behavioral elements may affect the decision making and decision support, among others the cognitive style of DMs, which is defined as the consistent individual differences in preferred ways of organizing and processing information and experience [24].

Green and Hughes [14] experimentally confirm the existence of interaction between the cognitive style and the type of training, which affected the decision maker initial use of a decision support system (DSS). The manager’s cognitive style was measured by them by means of Myers-Briggs Type Indicator (MBTI) [3].

Engin and Vetschera [7] experimentally studied the relationship between the cognitive style and decision quality when using tabular or graphical representations of information. The error rate, which measured the discrepancy between a reference ranking of alternatives and the one built by the DM, seemed to decrease when the analytical orientation of DM’s increased for tabular representation, while it increased for graphical one. Moreover, the effect of cognitive style was stronger for tables than for graphical representation. To described an individual’s position they used competitive analytic-intuitive scale measure by Cognitive Style Index [2].

Lu et al. [22] analyzed the effects of cognitive style (measured by means of MBTI) and type of decision support model on the decision support acceptance. Three different support models were considered: the fuzzy weighted-sum model, Saaty’s analytic hierarchy process, and the linear weighted-sum model. They inconclusively observed, that the cognitive style allowed to described the relationships among different acceptance measure only for one of these models (the fuzzy weighted-sum one), but not for others.

Chakraborty et al. [5], on the other hand, examined the DMs’ acceptance of new technology using Technology Acceptance Model [6]. They showed that cognitive style has significant direct effects on perceived usefulness, perceived ease of use, and subjective norms, and both perceived usefulness and subjective norms affect actual technology usage. The individual’s cognitive style was measured by means on Kirton Adaption–Innovation Inventory [4], which is an instrument for measuring the style on an adaptor–innovator continuum.

All these works show that cognitive style can affect the decision making process and results, however, the cognitive style itself can be differently perceived and interpreted. Thus, the question arises, what can be the simplest yet sufficiently informative way of measuring the cognitive style.

2.2 Rational/Analytical and Experiential/Intuitive Thinking Modes

There is an extensive discussion among the psychologists on how to define the cognitive style best. An approach based on the dual-processing distinction seems to be most grounded among the researchers, and it stems from the early works in a field of psychology and decision making [33, 36]. It makes a distinction between two thinking modes: rational/analytical and experiential/intuitive ones. For the terminology issues Stanovich simply calls them System 1 and System 2 [31], but we will use these terms interchangeably.

Within the cognitive psychology literature, it has been suggested that rationality and intuition are two coexisting information-processing systems, however there is lack of consensus about the theoretical relation between them [2, 9]. As was pointed out [35]: “models of individual differences in cognition differ as to whether intuition and analysis are viewed as bipolar opposites or as two independent unipolar dimensions. The distinction concerns whether one can be as follows: (i) either intuitive or analytical or (ii) both intuitive and analytical in orientation. The first implies a negative relation between the constructs, whereas the second implies no relation between intuition and analysis”.

The cognitive style, that is an orthogonal mix of the rational and intuitive approaches, can be determined by the Rational-Experiential Inventory (REI) [8]. This is a psychometric test consisting of the series of questions regarding the way of thinking and reasoning that the responder usually implements in their everyday life. However, some experimental results indicate various problems with understanding a 40-item original inventory [20]. Hence, some modifications of original REI test are proposed, such as a shorten version of REI test that consists of 20 items, i.e. the REI-A test [23].

According to REI results and some aggregation principles, the DMs can be assigned into one of four classes that differ in the combination of scores for rational and experimental modes [1, 18]. Those highly rational and highly experiential (HRHE) are called cognitively versatile, and are considered to have the skills to consider the problems both in details and as a big picture, when required. DMs from class HRLE are detail conscious (or rational) and have tendency to approach problem step by step in systematic way, while in contrast those from the class LRHE are big picture conscious (or experiential) use mostly intuition and are able to detect emergent issue ahead. Finally, those from the LRLE class (non-discerning or indifferent) seems not to be willing to engage their own cognitive resources in information processing (neither analyze nor base on the intuition) but rather rely on the opinions of others.

2.3 Research Questions

Taken into account all the issues raised in the previous two subsections some research questions may be formulated. First, following the extensive discussion regarding the competing approaches to the definition of the cognitive style [10] we would like to answer the following question:

Q1::

Does the orthogonal definition of the cognitive styles allow to define the cognitive style classes that differ significantly in DMs’ most preferred ways of preference representation and declaration (preference representation schema)?

Taking into account earlier studies an experiments (see e.g. [19]), the detail conscious DMs may be perceived as rational in bilateral perspective, and we may expect that they prefer more quantitative ways of preference representations. Similarly, the big-picture conscious ones can be called experimental in bilateral perspective and hence they would probably prefer to operate with non-numerical representation of preferences. We will try to confirm these two theses, but we are more interested in finding:

Q2::

What are the most preferred preference representation schemas by versatile or indifferent DMs?

To answer Q1 and Q2 we will use the dataset from the decision making experiment, in which the REI test was implemented. We will cluster the DMs into four groups as defined in Sect. 2.2, and analyze the answers they gave in the post-task questionnaires regarding: the way in which they wish to declare the preferences the most; and the most preferable representation of the results of the decision making process and the evaluation of alternatives. These are however, the self-reported general declarations and we would like to confront them with the DMs’ final recommendations regarding the most preferable decision support method, i.e. one of three different ones that they used in the experiment. Therefore, we ask:

Q3::

Do the cognitive style and the corresponding preference representation schema correlate with the choice of the best decision aiding method?

This would allow us to check whether DMs are truly interested in using the decision aiding tool that fits their cognitive capabilities the most, or they would rather opt, for instance, for a method that is quick, less time-consuming or has a nicer user interface.

3 Decision Making Experiment

To find the answers for the research questions posed in Sect. 2.2, we organized the decision making experiment in OSS. In the experiment the hypothetical problem of choosing a flat to rent was consisted for which five predefined alternatives were defined, each describing the resolution levels for five evaluation criteria. Since the participants were 413 students of four Polish universities, the problem was stylized to their decision making context. Table 1 presents the full decision matrix of the problem under consideration.

Table 1. Decision matrix in OSS experiment
Full size table

The experiment consisted of several steps that were related to the process of preference elicitation and decision support. At the beginning the respondents read the case and set an individual ranking of alternatives using the holistic approach. Using the boxes that visualized the alternatives and the drag-and-drop mechanism, they organized the boxes in an order that were supposed to reflect their individual subjective preferences (no instruction about the references were given to the participants).

In the next steps the decision analysis was conducted, which started from elicitation of criteria weights, where the participants used both AHP-based pair-wise mechanism and linguistic evaluation. Unsatisfied with the results produced by these two methods, they could also assign the weights directly themselves. Then the consequences of the alternatives were evaluated using three implemented MCDA methods: AHP, SMART and TOPSIS that differed in the preference elicitation schema. To each method the corresponding user interface was designed that was supposed to fit the cognitive requirements of the method itself. For AHP the sliders were used for each compared pair with accompanying verbal description of evaluation set by the slider. For SMART the tables were presented, which had to be filled by DMs directly with numbers representing their preferences. Since TOPSIS evaluates the quantitative criteria automatically using the notion of distances, there was a need for implementing a method for evaluation of two qualitative criteria in our problem, namely no. of rooms and equipment. Here, the DMs declared the preferences using pictograms, i.e. for each option seven empty stars were assigned and DMs colored in yellow as many of them as required to express the option performance. The screen-shots of the interface for each of MCDA method are presented in Fig. 1.

Fig. 1.
figure 1

User interface used for preference declarations in each MCDM method (Color figure online)

Full size image

Finally, the rankings of alternatives obtained by means of these three MCDA methods were presented to the respondents, including the scores they obtained and their graphical representations (five stars rating for TOPSIS and SMART or colored circles for AHP), as shown in Fig. 2.

Fig. 2.
figure 2

Display of the results of MCDA process in OSS (Color figure online)

Full size image

In the series of post-decision making questionnaires the respondents had evaluated the whole decision support process offered in OSS. They were asked to evaluate each support method, their ease of use, interface, reliability etc. We also asked about their opinion regarding some aspects related to the optimal design of the decision support mechanism and software support, such as the way of representing the results, describing the alternatives scores in final rankings, and best possible way of declaring the preferences in the preference elicitation process.

Finally, the respondents filled the REI test, which allowed us to determine their cognitive profiles and link them with their evaluation and expectations toward the decision support. As our respondents were non-native English speakers, we were afraid of misunderstanding problems related to some language nuances, hence in this study we used a shorten version of REI test that consists of 20 items, i.e. the REI-A test.

Correspondence analysis was used to describe the relationships among the factors under analysis [see e.g. 15].

4 Results

4.1 The Cognitive Styles and Cognitive Profiles

Confirmatory factor analysis with Varimax rotation with Kaiser normalization for the REI test allowed to defined the decision-making profiles at the satisfactory level. The Kaiser-Meyer-Olkin (KMO) measure (=0.852) confirms sample adequacy. The KMO values for individual items between 0.791 and 0.887 are also satisfactory. Bartlett’s test shows that correlations between questions were large enough to perform a factor analysis [χ2(190) = 2284.122; p < 0.001].

There is a strong (>0.99) and statistically significant (p < 0.01) correlation between the factor loadings of the thinking styles and the corresponding average values of answers for questions describing this style. Therefore, we used the average values from questions 1–10 as the scalar measure of rational mode (R), and from questions 11–20 to describe their experiential mode (E). These average values were used to classify the respondents within each mode into two classes: L (less or equal to average) and H (above the average). The combination of these two classes for two modes make four different categories of cognitive profiles: Versatile (HRHE), Rational (HRLE), Experiential (LRHE), and Indifferent (LRLE). The correlation coefficient between average values of rational and experiential thinking modes is equal −0.054. This is a first indicator that REI’s dimensionality should rather be expressed by two interacting but independent (orthogonal) rational and experimental systems, not the unimodal one, and suggests a positive answer for Q1.

4.2 Expectations Towards the Declaration of Preferences

First, we analyzed the respondent’s choices regarding the most preferred way of defining their preferences in the preference elicitation process. The numbers and fractions of choices are shown in Table 2 and visualized in Fig. 3a.

Table 2. Decision making profiles vs preferred forms of preference impartation (N = 413).
Full size table
Fig. 3.
figure 3

Correspondence analysis for cognitive profiles in OSS (1)

Full size image

Taking into account the relatively low fractions in choosing the forms of preference declarations other than three most frequently chosen classes we removed the latter form the correspondence analysis to make the visualization more readable.

It seems that the Versatile and Experiential DMs are relatively close to each other (fractions in Table 2 are quite similar), but far from Rational and Indifferent. Among all the classes most DMs choose numerical, next pictorial and finally verbal description of preferences. This is a dominant choice (69.6%) for DMs with Rational profile. However, Experiential and Versatile DMs are also quite frequently choosing pictorial way of preference declaration, while the Indifferent ones choose the verbal declarations most frequently than others.

The differences in fractions for Rational profile and two other classes with low rationality index (Indifferent, Experiential) are significant (p < 0.072). The DMs with higher rationality index (Rational and Versatile) differ significantly in the choice of pictorial declarations (p < 0.017). Also, Indifferent and Experiential DMs differ significantly (p < 0.05) in choosing the verbal declarations among themselves. This is another premise to answer Q1 positively.

4.3 Expectation Towards the Representation of Final Results of Decision Analysis

Next, we have analyzed the DMs’ expectations about the forms of the representation of the results obtained in the decision analysis phase. The fractions of user’s expectations regarding the cardinal quantitative representation of the results (ratings) vs. ordinal representation by means of rankings only or other types of representations are presented in Table 3 and visualized by means of correspondence analysis in Fig. 3b.

Table 3. Decision profiles classes vs forms of representing results by the system (N = 413).
Full size table

The respondents with high experiential index (HE: Versatile, Experiential) prefer the results to be presented in a form of ranking only. Their fractions of choosing the rankings (65.1% and 63.6% respectively) differ significantly from those determined for DMs with low intuitive index (51.4% and 47.3%, respectively).

The respondents with low experiential index (LE: Indifferent or Rational) prefer mostly the rating as the best way of representing the results of the decision analysis. There is as much as 45.8% of Indifferent DMs and 50.0% of Rational who choose rating, while only 34.1% of Versatile and 33.0% of Experiential do so.

The differences in choices between LE and HE are significant (p < 0.048), and this is in fact the only situation in our study, where unimodal approach to the definition of cognitive style could be used to describe the differences in sufficiently accurate way.

4.4 Expectations Towards the Representation of Offers’ Evaluation

We have also analyzed the responses of the DMs for more detailed question regarding the most preferred representation of the alternatives evaluation in the final ranking. Various combinations of representations suggested by the respondents were clustered into three classes: pure numerical representation, non-numerical representation, or the mixed one that joins the advantages of all three forms. In Table 4 the numbers of respondents selecting each representation of alternatives’ evaluations are provided and the results are visualized in Fig. 4a.

Table 4. Preferred forms of presentation of alternatives evaluation in the final ranking (N = 413).
Full size table
Fig. 4.
figure 4

Correspondence analysis for cognitive profiles in OSS (2)

Full size image

Here we see that Indifferent and Experiential profiles are relatively close to each other (unlike to previous results), but far from Versatile and Rational. The DMs from all the classes prefer most the numerical evaluation, then mixed one and the least – the non-numerical representation of offers’ evaluation. Rational DMs are the only ones to prefer numerical scores in majority (56.2%). The Versatile also like the numerical evaluations of offers but combined with some alternative representations (verbal and/or pictorial), while the DMs with low rationality index (Experiential, Indifferent) are more satisfied with the non-numerical representation of offers’ values than DMs from other classes.

The differences in fraction of choices made by the Rational and Versatile DMs are significant (p < 0.023) for numerical and mixed representations. They similarly frequently choose non-numerical representation, but the fraction of such choices is relatively low (<19.3%). It seems, however, that Indifferent and Rational profiles differ significantly in choosing the numerical scores (p < 0.05).

4.5 Recommendation of MCDA Method

Finally, we have analyzed what are the recommendations towards using a specific MCDA method by the DMs from different classes. This way we wished to confront the DMs most preferred ways of handling the preferences with their real choices made as the DSS users, who were offered the software implementations of particular MCDA techniques. The results are presented in Table 5, and visualized in Fig. 4b.

Table 5. Recommended methods (N = 413).
Full size table

Surprisingly, TOPSIS appeared to be a choice of relative majority of DMs within each class of cognitive profiles. The highest percentage of TOPSIS choices was in Versatile (47.7%) and Experiential (46.3%) classes. This is an interesting finding, since the respondents from all cognitive profiles had previously preferred in majority the numerical forms of preference declarations (see Table 2).

Indeed, Rational DMs appeared to choose SMART more frequently than others, Versatile more frequently than others chose TOPSIS, and the Indifferent – AHP. The general profile of choices of Experimental DMs makes them similarly close to TOPSIS and AHP but definitely far form SMART.

5 Conclusions

In this study we tried to analyze what are the DMs’ willingness and expectations for operating with various types of preference information during the process of decision analysis conducted in the decision support system. Some of our results confirm the general propositions formulated by the behavioral theory of decision making, that link the thinking styles with some schemas of DMs activities focused on solving the decision making problems. For instance, the Rational DMs prefer in majority the usage of ratings, numerical only description of alternatives and numerical declaration of preferences more frequently than respondents from other classes. They also prefer not to use the pictorial declaration preferences more than respondents from other classes of decision making profiles. Contrary, the Experiential ones do not process information extensively and hence need no precise data to be provided. This simply overlap with the general description of rational and experiential actors provided by Epstein [8].

However, trying to find the answers for the research questions we were able to shed a new light on some nuances of the potential impact of cognitive styles on the expectations and use of the decision support tools. It seems that the question Q1 can be answered positively, as the orthogonal definition of the cognitive styles was absolutely necessary to described the differences in DMs’ most preferred ways of preference representation and declaration. When we look at the groups of similar profiles encircled at Figs. 3 and 4, the Rational and Experiential ones always constitute separate groups, while the Versatile and Indifferent classes, each independently, sometimes are more alike to Rational, and sometimes to the Experiential one. A good summary of our results, that allow to formulate the answer for question Q2 is presented in Table 6.

Table 6. Profiles and nearest choices according to correspondence analysis.
Full size table

The Versatile on general seem to be more like Experimental in preference declaration and results representation (no quantitative representation required). However, they would like to obtain a little more specific information about differences in alternative performance that would be somehow linked to numbers. This matches with the support method they are closest to, which appears to be TOPSIS. The Indifferent, on the other hand, prefer to define their preferences verbally (like no other), and wish to have a cardinal information as the result (rating as Rational). Interestingly, they would like to receive such cardinal information about alternatives in non-numerical way (as Experiential). This explains their general choice of AHP.

The answer for question Q3 is not so evident. Looking at Table 6 one may say that some generalized outcomes provided by correspondence analysis allow to answer Q3 positively. Cognitive style and choices regarding preference representation match the methods the DMs recommend to use. Rational like numbers and hence choose SMART, while Experiential dislike them and they opt for either TOPSIS or AHP. However, if we look at the profiles of choices (Table 5), we will see that in each class the relative majority of DMs choose TOPSIS. If we confront it with another finding, that the absolute majority in each class prefers to declare preferences by means of numbers (Table 2), a complicated picture of some contradictory expectations and needs of DMs is given, that needs further and deeper investigation.

A need of more detailed research arises, that would identify the preferences towards using particular decision support algorithms accompanied by some additional visualization techniques, with a measure of the efficiency of their use. This could help to develop the support mechanisms that are fit to the cognitive capabilities of the decision makers, reduce the potential errors they could make using the heuristics typical to the profile of their thinking style, and provide them with a type of information that they are able to process and use efficiently to make the best decisions.