1 Introduction

In multicriteria decision problems, the inter-criteria evaluating parameter is usually called weights. This is one of the fundamental modeling issues widely discussed in the literature. The additive model is widely used in the literature (Palha et al. 2016; Pergher and de Almeida 2017), and in such kind of models using weights means using scale constant or tradeoffs, and declaring which committed relationship the decision maker (DM) is willing to establish between two criteria. In other methods, weights may be about the relative importance among the criteria.

The literature discusses why eliciting parameter is a complex task (Pöyhönen and Hämäläinen 2001; Bottomley and Doyle 2001; Sarabando and Dias 2010) and states that it makes strong cognitive demands on the DM (Barron and Barrett 1996; Larichev 1992).

Usually the method for eliciting weights requires a level of precision that, in many cases, the DM is not able to provide. To determine the weight of criteria accurately using specific procedures is a task subject to errors caused by the DM having an uncertainty preference or lack of understanding of the procedure (Borcheding et al. 1991; Jimenez et al. 2003; Pöyhönen and Hämäläinen 2001; Jia et al. 1998; Walley 1991; Ben Amor et al. 2016).

This paper focuses on the tradeoff method to elicit weights which is considered in the literature as having the strongest theoretical foundation (Weber and Borcherding 1993). However, the use of this method is complex and requires considerable cognitive effort from the DM.

In this context, a new procedure is presented for the flexible elicitation of weights when an additive model is used, this procedure being an alternative to minimize the effort required from the DM on eliciting scale constants in an additive model. Distinctly from the classic tradeoff method for elicitation, the flexible elicitation procedure proposed is based on the DM’s preference relation judgments on different consequences.

First of all, this paper discusses previous studies that have contributed to this research and different studies that have addressed similar problems in order to highlight the differences amongst these and the contribution of this work. Then, the elicitation procedure described by de Almeida et al. (2016) is formally presented as are its assumptions, propositions and properties. Finally, the results are given from empirical tests conducted to validate the procedure presented by de Almeida et al. (2016), recently applied in Gusmão and Medeiros (2016).

2 Related methodologies

The literature contains many studies that seek to reduce the effort required from a DM to provide multicriteria model parameters such as weights, specifically. It is noteworthy that weights can be specified in many different ways and with different precision levels (Puerto et al. 2000).

In the context of multi-attribute utility theory, dominance and potential optimality are frequently explored with a view to reducing the level of accuracy in the information required from a DM. Hazen (1986) explored additive models with unknown weights through dominance and potential optimality relations. The literature points out that to obtain precise weights is such a difficult task (Kirkwood and Sarin 1985; Barron and Barrett 1996; Bottomley et al. 2000) that a natural way to deal with this matter is to relax some assumptions and instead of determining weight values, to work with approaches based on weight intervals (Puerto et al. 2000).

Riabacke et al. (2012) argue that, in the literature, there have been a number of methods suggested for assessing criteria weights and that the numbers elicited can be heavily dependent on the assessment method. They divide the most prominent weight elicitation methods into two groups: (1) ratio weight procedures that include direct rating (DR) (Bottomley et al. 2000), SMART (Edwards 1977), SWING (Winterfeldt and Edwards 1986) and the tradeoff method (Keeney and Raiffa 1976; 2) imprecise weight elicitation that includes the ROC method in the SMARTER method (Edwards and Barron 1994), the AHP method (Saaty 1980), Interval methods (Walley 1991; Danielson and Ekenberg 2007), an example of an interval-based method is the SMART/SWINGinterval proposed by Mustajoki (2005).

The flexible elicitation procedure presented in this paper is based on the tradeoff approach. In the classical tradeoff elicitation model (Keeney and Raiffa 1976), a consequence C has the result \(x_{i}\) for criterion i, and the worst result for all other criteria \(w_{i}\). The DM is asked to report results for which \(x_{i}\) would make the consequences C and another consequence A indifferent, and so on. By considering all criteria i, the scale constants are defined. This procedure can use results around \(x_{i}\), assuming that scale constants are ordered, as \(k_{1}>k_{2}>\cdots>k_{i},\ldots >k_{n}\). In this article, \(x_{i}^{I}\) will be used to represent the value that establishes the indifference relation between C and A. For a criterion i, assuming that \(x_{i}^{'}>x_{i}^{I}>x_{i}^{''}\), then using the result \(x_{i}^{'}\) for a consequence related to criterion i would be preferable to the result \(x_{i}^{I}\) and the result \(x_{i}^{I}\) would be preferable to the result \(x_{i}^{''}\). What makes this approach distinct from the classic elicitation model is that the flexible elicitation procedure proposed seeks to obtain \(x_{i}^{'}\) and \(x_{i}^{''}\), based on the DM’s preference relation judgments.

Many approaches were developed to introduce the use of imprecise information about weights in multi-criteria decision making, usually by exploring dominance relations and optimality aspects. With regard to the use of imprecise or partial information, these methods can be grouped as: (a) Interval weights (Steuer 1976; Park 2004; Mustajoki 2005; Li et al. 2012; b) Partial/Incomplete information on weights (Barron 1992; Salo and Hämäläinen 2001; Mármol et al. 2002; Salo and Punkka 2005; Punkka and Salo 2013); and (c) unknown weights (Hazen 1986; Lotfi et al. 1992).

Steuer (1976) put forward a novel analysis regarding interval weight with the purpose of evaluatingthe weights space vector in multiple objective linear programming problems and considered interval weights. This study discussed two techniques for finding the optimal solution for a multiple objective linear programming (MOLP) problem: the weighted-sum technique and the vector-maximum method. The author proposes to solve the family of weighted-sum problems for all maximizing extreme points (the subset of the set of all efficient extreme points that correspond to the interval weights). The author claims that in this way the DM can incorporate any uncertainties he has about his/her criterion weight judgment.

In multi-attribute value theory (MAVT), Park (2004) argues that many indifference judgments are required from the DM in order to obtain weights and the value function. The author discusses the extent to which the relaxation of such precise judgment is a way to reduce the gap between theoretical research and practical needs.

Li et al. (2012) point out that in multi-attribute problems it is never easy to obtain exact values for the attribute weights. The DM is only willing or able to provide incomplete information, such as an interval value. In this situation, the computation of this information always leads to linear optimization problems or binary relation methods based on pair wise dominance between any two alternatives. The authors comment that many studies focus on the concepts of dominance and potential optimality and that one class of methods always needs to deal with many linear or nonlinear optimization problems. According to these authors, to confirm the dominance relation between any two alternatives and to obtain the nondominated alternatives is usually an easy task. On the other hand, attempts to determine the best alternative or rank alternatives satisfactorily usually fail (Li et al. 2012).

Salo and Hämäläinen (1992) presented the PAIRS method, based on interval judgments to elicit a weights interval from a DM. They later developed the PRIME method (Salo and Hämäläinen 2001), thereby extending the results they had obtained in their earlier paper. They claim that all preference statements should be modeled as linear constraints based on preference statements that can make a holistic comparison between real and/or theoretical alternatives through ordinal strength statements or evaluations based on ratio relations.

Since then Salo and Punkka (2005) have proposed another method (rank inclusion in criteria hierarchies—RICH) in which the DM(s) can provide incomplete rank-ordering information about the relative importance of attributes for hierarchical weighting models. This method also explores dominance relations and decision rules in order to provide recommendations for decision problems.

Another model that does not require exact estimations of attribute weights and values is presented in Eum et al. (2001). In this model information on the decision parameters is only assumed to be in the form of arbitrary linear inequalities which represent constraints in the model. The authors considered two criteria, dominance and potential optimality, to check whether or not what alternatives are outperformed for a fixed feasible region denoted by the constraints.

Mustajoki (2005) studied the use of weight intervals in the simple multi-attribute rating technique (SMART) and SWING weighting methods and presented interesting results regarding the practical implications of applying an interval approach in these methods.

Mustajoki (2012) also addressed problems based on the hierarchical model structure and considered the effects of imprecise information for such problems by simulating various weighting schemes to identify dominated alternatives. Mustajoki (2012) discussed these findings with regard to the weighting process and suggested that it is more difficult to identify dominances in a hierarchical model than in a non-hierarchical one. However, the cognitive benefits to the DM from this approach could overcome its disadvantage.

More recently, Punkka and Salo (2013) developed a model using incomplete ordinal information about both the value of alternatives and the relative importance of the attributes.

Published papers on procedures for eliciting weights point out that this is still a relevant topic in the literature on multi-attribute decision making (Li et al. 2012; Mustajoki 2012; Punkka and Salo 2013) and use different approaches. For instance, Chen (2014a) addresses diversiform preference types, representing the decision maker’s various forms of preference structures and assesses the criteria weights using incomplete information by the concept of using the likelihood of fuzzy preference relation to compare weights of the interval-valued intuitionistic fuzzy (IVIF) number type. In Chen (2014b) the prioritization between the criteria is modeled by assessing their IVIFs. Fuzzy Approach has been used when decision maker has difficulties in expressing their preferences (Campos et al. 2015; Parreiras et al. 2012, 2010).

Thus, similarly to previous studies in the literature, the flexible elicitation procedure proposed in this paper also seeks to reduce the level of accuracy in the information required from a DM by using concepts of dominance and dominance tests to reduce the amount of information required from the DM by providing a procedure that enables scale constant properties to be incorporated as in the traditional tradeoff method, which differentiates this approach from others in the literature.

Unlike other studies that introduce imprecise or partial information on the elicitation of weights, the procedure described by de Almeida et al. (2016) reduces the information required from a DM by using his/her judgment over consequences as preference relations, which are explored by dominance tests through a linear programming problem. This is presented in Sect. 3.

3 Flexible and interactive tradeoff elicitation procedure (FITradeoff)

In the flexible elicitation procedure presented it is assumed that:

  1. I.

    DM is not able to specify \(x_{i}^{I}\) or

  2. II.

    That DM’s indifference statement on \(x_{i}^{I}\) may not be reliable information

  3. III.

    The information provided from preference statements on \(x_{i}^{'}\ge x_{i}^{I}\ge x_{i}^{''}\) are more reliable than indifference statements from assumption II.

If assumptions (I or II) and III are verified then the proposed procedure may be considered as a method for choice problems based on a multi-attribute additive value function.

3.1 Procedure and assumptions

Thus, let A be the set of alternatives, represented by \(A=\left\{ X_{1},X_{2},\ldots ,X_{j},\ldots ,X_{m} \right\} \), and let \(X_{j}\) be the consequence vector of alternative j, such as \(X_{j}=(x_{1j},x_{2j},\ldots ,x_{{\textit{ij}}}, \ldots ,x_{nj})\), \(x_{ij}\) representing the result of each attribute i from the n attributes. To simplify \(X_{j}\) also will represent the alternative j.

For each of the n attributes there is a value function \(v_{i}(x_{{\textit{ij}}})\) for the evaluation of the result \(x_{{\textit{ij}}}\) in the attribute i for the alternative j, such that \(x_{{\textit{ij}}}{\textit{ a component of }}X_{j} \in A\), where \(v_{i}\left( x_{ij} \right) \rightarrow ,\{0,1\}\). To simplify notation throughout the text, \(v_{i}(x_{ij})\) will also be represented by \(y_{ij}\).

Regarding the scale constants, let \(K=\left\{ k_{1},k_{2},\ldots ,k_{i}, \ldots ,k_{n} \right\} \), where \(k_{i}\in \{0,1\}\), \(\sum \nolimits _{i=1}^n {k_{i}=1}\) and \(K\in \varphi \), where \(\varphi \) represents the vector space for all admitted scale constants based on the partial information elicited from the DM about his/her preferences.

Thus, the multi attribute value function for a DM shall be written as \(\vartheta (X_{j},K)=\sum \nolimits _{i=1}^n {k_{i}v_{i}(x_{ij})} \), where \(\vartheta \left( X_{j},K \right) \rightarrow \{0,1\}\), with \(D\left( \vartheta \right) =\left\{ X_{j},K\vert X_{j}\in {\textit{ A and }} K\in \varphi \right\} \), which shall be a function of the set of alternatives and the scale constant vector space.

Within this structure, scale constants values are explored as per (1):

$$\begin{aligned} K^{P}=\left[ k_{1}^{P},k_{2}^{P},\ldots ,k_{i}^{P},\ldots ,k_{n}^{P} \right] , {\textit{where}} \left\{ {\begin{array}{c} \sum \nolimits _{i=1}^n k_{i}^{P} =1 \\ k_{1}^{P}y_{1}^{''}\le k_{2}^{P}\le k_{1}^{P}y_{1}^{'}, \\ \vdots \\ k_{i}^{P}y_{i}^{''}\le k_{i+1}^{P}\le k_{i}^{P}y_{i}^{'} \\ \vdots \\ k_{n-1}^{P}y_{n-1}^{''}\le k_{n}^{P}\le k_{n-1}^{P}y_{n-1}^{'} \\ \end{array}} \right. \end{aligned}$$
(1)

Considering the additive model given by \(\vartheta (X_{j},K)\), which is a global value function and \(X_{j}=(x_{1j},x_{2j},\ldots ,x_{ij}, \ldots ,x_{nj})\) as a consequence vector that considers n criteria associated with an alternative in the multi criteria decision problem. Let \(X_{A}\) be a consequence vector such that \(X_{A}=(b_{1A},w_{2A},w_{3A},\ldots ,w_{nA})\) where \(b_{1A}\) is the best consequence for the criterion \(i=\textit{1}\) and \(w_{iA}(i=\textit{2,\ldots ,n)}\) is the worse consequence for the other i criterion, \(i\ne 1\).

Assuming that \(v_{i}\left( b_{ij} \right) =1\) and \(v_{i}\left( w_{ij} \right) =0\), \(\forall i =\textit{ 1, 2,\ldots ,n}\) ; it is easy to see that \(\vartheta \left( X_{A},K^{P} \right) =k_{1}^{P}v_{1}\left( b_{1A} \right) =k_{1}^{P}\). Similarly, let \(X_{B}\) be a consequence vector such that \(X_{B}=(w_{1B},b_{2B},w_{3B},\ldots ,w_{nB})\). Therefore \(\vartheta \left( X_{B},K^{P} \right) =k_{2}^{P}v_{2}\left( b_{2B} \right) =k_{2}^{P}\). So if DM prefers consequence \(X_{B}\) to \(X_{A}\) \((X_{B}PX_{A})\), then \(\vartheta \left( X_{B},K^{P} \right)>\left( X_{A},K^{P} \right) \Rightarrow k_{2}^{P}>k_{1}^{P}\).

Let \(X_{C}\) be the consequence vector \(X_{C}=(w_{1C},w_{2C},b_{3C},\ldots ,w_{nC})\) such that the DM is indifferent between consequence \(X_{A}\) and \(X_{C}(X_{A}IX_{C})\). Therefore \(\vartheta \left( X_{A},K^{P} \right) =\vartheta \left( X_{C},K^{P} \right) \Rightarrow k_{1}^{P}=k_{3}^{P}v_{3}\left( b_{3C} \right) =k_{3}^{P}\).

If there is a scale constant rank such as \(k_{1}^{P}>k_{2}^{P}>\cdots>k_{i}^{P},\ldots >k_{n}^{P}\) , and there are \(y_{i}^{'}\) and \(y_{i}^{''}\) such that \(y_{2}^{'}>y_{2}^{I}>y_{2}^{''}\), then there is \(\hbox {a}X_{C^{'}}=(w_{1C'},x'_{2C'},w_{3C'},\ldots ,w_{nC'})\) such that (\(X_{C^{'}}PX_{A})\), and similarly there is a \(X_{C^{''}}=(w_{1C''},x_{2C''}^{''},w_{3C''},\ldots ,w_{nC''})\) that \((X_{A}PX_{C^{''}})\).

For the new flexible procedure suppose that \({v}_{2}\left( b_{2j} \right)>v_{2}\left( x_{2j}^{'} \right)>v_{2}\left( x_{2j}^{I} \right)>v_{2}\left( x_{2j}^{''} \right) >v_{2}\left( w_{2j} \right) \), as \(v_{i}(b_{ij})=\textit{1}\) and \(v_{i}(w_{ij})=\textit{0}\). It follows that \({1>v}_{2}\left( x_{2j}^{'} \right)>v_{2}\left( x_{2j}^{I} \right)>v_{2}\left( x_{2j}^{''} \right) >0\).

Thus let the vectors of consequence be \({X}_{A}=(w_{1A},w_{2A},b_{3A},\ldots ,w_{nA})\), \(X_{B}=(w_{1B},x_{2B}^{'},w_{3B},\ldots ,w_{nB}), X_{C}=(w_{1C},x_{2C}^{''},w_{3C},\ldots ,w_{nC})\) so \(\exists x_{2B}^{'},x_{2C}^{''}|X_{B}PX_{A}\) so \(\vartheta \left( X_{B},K^{P} \right) >\vartheta \left( X_{A},K^{P} \right) \) and \(X_{A}PX_{C}\) so \(\vartheta \left( X_{A},K^{P} \right) >\vartheta \left( X_{C},K^{P} \right) \), and further \(\exists \,\mathrm {x}_{2}^{I}=x'_2=x''_{2}\,\vert X_B\,{\textit{IX}}_A\) and \(X_{A}\,{\textit{IX}}_{C}\) so \(\vartheta \left( X_{A},K^{P} \right) =\vartheta \left( X_{B},K^{P} \right) =\vartheta \left( X_{C},K^{P} \right) =k_{3}^{P}\). Hence \(\vartheta \left( X_{B},K^{P} \right) =k_{2}^{P}v_{2}\left( x_{2}^{'}\right) >k_{2}^{P}v_{2}\left( x_{2}^{I} \right) =k_{3}^{P}\). Without loss of generality the relation amongst \(y_{i}^{'},y_{i}^{''},y_{i}^{I}\) and \(k_{i+1}^{P}\) is proved.

In the flexible elicitation procedure for each alternative \(X_{j}\), with \(j=\textit{1,\ldots ,m ,}\exists \) a consequence vector \(X_{j}=\left\{ x_{1j},x_{2j},\ldots ,x_{ij}, \ldots ,x_{nj} \right\} \), and with the values of \(y_{{\textit{ijs}}}^{'}\) and \(y_{{\textit{ijs}}}^{''}\) a linear programming problem (LPP) can be used to test alternative \(X_{j}\) regarding its dominance relations with the m-1 alternatives of the set of alternatives, based on the elicited information elicited about \(K^{P}\). If the problem stated in (2) has at least one solution, then \({\exists \, K}^{P}\in \varphi _{s}^{P}\) that enables \(X_{j}\) as a non dominated alternative. If the LPP from (2) has no feasible solution, then it means that \(\not \exists \, K^{P}\in \varphi _{s}^{P}\) that enables \(X_{j}\) as a non dominated alternative. Thus \(X_{j}\) shall be considered a dominated alternative from iteration s.

Therefore the dominance test proposed is defined by the LPP stated below in (2):

$$\begin{aligned} {\begin{array}{*{20}c} max: &{} \vartheta \left( X_{j},K^{P} \right) &{} &{} \\ s.t. &{} &{} &{} \\ &{} &{} \vartheta \left( X_{j},K^{P} \right) \ge \vartheta \left( X_{z},K^{P} \right) , &{} \forall z\ne j;z=1,2,\ldots ,m\\ &{} &{} k_{i}^{P}v_{i}\left( x_{ij}^{'} \right) \ge k_{i+1}^{P}, &{} \forall i=1,\ldots ,n-1\\ &{} &{} k_{i}^{P}v_{i}\left( x_{ij}^{''} \right) \le k_{i+1}^{P}, &{} \forall i=1,\ldots ,n-1\\ &{} &{} \sum \nolimits _{i=1}^n {k_{i}=1} &{} \\ &{} &{} k_{i}^{P}\ge 0 , &{} \forall i\\ &{} &{} &{} \\ &{} &{} &{} \\ \end{array}} \end{aligned}$$
(2)

Assuming that the DM may not feel comfortable about specifying a \(x_{ij}^{I}\) it shall be a premise that it would be much easier for a DM to deal with a flexible elicitation protocol, which asks for less information to build the tradeoff preference relation.

Although this may represent less information to be evaluated in an ex post perspective, there are no fixed values for the marginal rate of substitution of the attribute. However, this allows the DM to fulfill the aim of a choice problem by evaluating preferences through a vector space, which evaluates all possibilities for \(k_{i}\) using a robust procedure. According to de Almeida (2006), choice problems are frequently faced in literature when considering any activity outsourcing, which has been a topic of interest of many recent works in the literature.

Thus this vector space that represents the DM’s preferences shall be represented as \(\varphi _{s}^{P}\subset \varphi \), which is the set of scaling constants admitted by DM after s iterations of the flexible elicitation procedure. All during flexible elicitation \(\varphi _{s}^{P}\) is reduced ass increases until sufficient information is acquired to find a solution from the set of alternatives. This means that the DM can answer with values that carry less uncertainties or hesitation.

Throughout the flexible elicitation procedure the DM will be questioned in order to build \(\varphi _{s}^{P}\), defined as the DM preferential vector space of scale constants. Thus, all alternatives shall be evaluated in terms of any \(K^{P} \in \varphi _{s}^{P}\), which is a scale constant vector within DM preferences.

Considering for instance that the scale constant \((k_{1}^{P}>k_{2}^{P}>\cdots>k_{i}^{P},\ldots >k_{n}^{P})\) is already one part of the traditional tradeoff procedure approach, thus the flexible elicitation procedure can be a replacement for the tradeoff traditional elicitation steps of a tradeoff after ordering the scale constants, which means that all comparisons with the proposed procedure shall be done with its equivalent steps.

In a convergence perspective, the proposed flexible procedure considers the reduction in a weights vector space reduction depending on how much information each specific problem requires from the DM to point to the chosen alternative according to the DM’s preference over the set of criteria. Thus, the algorithm for the method starts with an initial vector space \(\varphi _{0}^{P}\) with n dimensions (criteria) where the index 0 states that this is the space before the first iteration of the flexible elicitation procedure. In other words, after the ranking of the scale constants is obtained and \(\varphi _{0}^{P}\) is defined as per (3):

$$\begin{aligned} \varphi _{0}^{P}=\left\{ {K^{P}\in {\mathfrak {R}}^{+n}} \vert {k_{1}^{P}\ge k_{2}^{P}\ge \cdots \ge k_{i}^{P}\ge \cdots \ge k_{n}^{P}{\textit{ and }} \sum \nolimits _{i=1}^n {k_{i}^{P}=1} }\right\} \end{aligned}$$
(3)

Throughout the elicitation procedure, the hyperplane defined by \(\varphi _{0}^{P}\) is reduced initially to \(\varphi _{1}^{P}\) which is the weight vector space after the first flexible elicitation iteration, defined as per (4):

$$\begin{aligned} \varphi _{1}^{P}=\left\{ {K^{P}\in {\mathfrak {R}}^{+n}} \vert {\varphi _{1}^{P}\subset \varphi _{0}^{P}\,\,{\textit{and }}\,\, k_{i}^{P}y_{i1}^{''}\le k_{i+1}^{p}\le k_{i}^{P}y_{i1}^{'}\,\, {\textit{ for all }}i}\right\} \end{aligned}$$
(4)

where \(y_{ij1}^{'}\) and \(y_{i1}^{''}\) are the single attribute values of the respective result \(x_{ij1}^{'}\) and \(x_{ij1}^{''}\) obtained in the flexible elicitation process regarding criterion i for alternative j in the iteration 1. After new constraints were added to the initial weight vector space after the elicitation iteration then \(\varphi _{1}^{P}\subset \varphi _{0}^{P}\). Thus after s iterations, the vector space of the n dimension weights becomes \(\varphi _{s}^{P}\) defined as per (5):

$$\begin{aligned} \varphi _{s}^{P}=\left\{ {K^{P}\in {\mathfrak {R}}^{+n}} \vert {\varphi _{s}^{P}\subset \varphi _{s-1}^{P}\,\,{\textit{and }} k_{i}^{P}y_{ijs}^{''}\le k_{i+1}^{p}\le k_{i}^{P}y_{ijs}^{'}\,\, {\textit{ for all }} i}\right\} \end{aligned}$$
(5)

The weight vector space shall be reduced until convergence in iteration r, which is after the last iteration that enables the chosen alternative with the weight vector space \(\varphi _{r}^{P}\) to be pointed out as presented in (6):

$$\begin{aligned} \varphi _{r}^{P}=\left\{ {K^{P}\in {\mathfrak {R}}^{+n}} \vert {\varphi _{r}^{P}\subset \varphi _{r-1}^{P}\,\,{\textit{and }} k_{i}^{P}y_{ijr}^{''}\le k_{i+1}^{p}\le k_{i}^{P}y_{ijr}^{'}\,\, {\textit{ for all }} i}\right\} \end{aligned}$$
(6)

The weight vector space is reduced in each iteration by cutting off a portion of \(\varphi _{0}^{P}\) using for the first iteration the hyper plane \(\alpha =\left\{ Y\in {\mathfrak {R}}^{+n}\vert k_{i}^{P}y_{i1}^{''}\le k_{i+1}^{p}\le k_{i}^{P}y_{i1}^{'}\,\,{\textit{ for all }}i \right\} \), therefore \(\varphi _{r}^{P}\subset \cdots \subset \varphi _{s}^{P}\subset \cdots \subset \varphi _{1}^{P}\subset \varphi _{0}^{P}\) which means that the weight vector space is updated with partial information and reduced by each iteration of flexible elicitation.

Then as long as \({ k}_{i}^{P}y_{ijs}^{''}\le k_{i+1}^{p}\le k_{i}^{P}y_{ijs}^{'}\) is true, then the condition of (7) is also true:

$$\begin{aligned}&k_{i}^{P}y_{ij1}^{''}\le \cdots \le k_{i}^{P}y_{ijs}^{''}\le \cdots \le k_{i}^{P}y_{ijr}^{''}\le k_{i+1}^{p}\le k_{i}^{P}y_{ijr}^{'}\le \cdots \nonumber \\&\quad \le k_{i}^{P}y_{ijs}^{'}\le \cdots \le k_{i}^{P}y_{ij0}^{'} {\textit{ for all i}} \end{aligned}$$
(7)

Thus, as \(s\rightarrow r\), \(k_{i}^{P}y_{ijs}^{'}\rightarrow k_{i+1}^{P}\) and \(k_{i}^{P}y_{ijs}^{''}\rightarrow k_{i+1}^{P}\), this assures the convergence of this flexible elicitation procedure to achieve the DM’s preferential weights through requiring minimum partial information requirements and respecting additive tradeoff assumptions. This result is similar to the convergence of cutting plane algorithms such as those found in Dabbene et al. (2010) and Franc and Sonnenburg (2009).

3.2 Dominance and optimality analysis

The sets defined by (1), (3), (4), (5) and (6) are convex sets, therefore, the solution of LPP given by (2) depends on the dominance relation as presented below.

Definition

Let the set of alternatives be A with \(\vartheta (X_{j},K^{P})=\sum \nolimits _{i=1}^n {k_{i}^{P}v_{i}(x_{ij})}\) and \(\varphi _{s}^{P}\) the weight space given by (5).

  1. 1.

    \(X^{d+}\in A \) is an optimal alternative in A, if \(X^{d+}\) is an unique solution of LPP given by (2), \(\forall \,K^{P}\in \varphi _{s}^{P}\). It follows that \(\vartheta \left( X^{d+},K^{P} \right) >\vartheta \left( X_{j},K^{P} \right) ,\forall \,X_{j}\in A\). \(X^{d+}\) is a optimal alternative.

  2. 2.

    \(X^{d-} \in A\) is a dominated alternative in A, if \(X^{d-}\) is a non-potentially optimal alternative, \(\forall \,K^{P}\in \varphi _{s}^{P}\). It follows that \(\vartheta \left( X^{d-},K^{P} \right) <\vartheta \left( X_{j},K^{P} \right) \), for any \(X_{j}\in A\).

  3. 3.

    \(X^{nd} \in A\) is a non-dominated alternative in A, if \(X^{nd}\) is a potentially optimal alternative, if \(\exists \quad K^{P}\in \varphi _{s}^{P}\) such that \(\vartheta \left( X^{nd},K^{P} \right) >\vartheta \left( X_{j},K^{P} \right) \),for \(\hbox {any}X_{j}\in A\).

Proposition 1

(Optimal alternative) If \(X^{d+}\) is an optimal alternative in \(A,\forall \,K^{P}\in \varphi _{s}^{P}\), the LPP given by (2) has solution only for the alternative \(X^{d+}\).

Proof

Let \(X^{d+}\) be an optimal alternative in A and \(X_{j}\in A\) any other alternative. Given \(K^{'},K^{''}\in \varphi _{s}^{P}\), there are two different weight vectors as possible solutions for the LPP given by (2), for \(X^{d+}\) and \(X_{j}\) respectively. \(\square \)

From the definition of the dominance relation definition for \(X^{d+}\), \(\forall \,K^{P}\in \varphi _{s}^{P}\)including \(K^{'}\)and \({K}^{''}\), the condition in (8) should hold.

$$\begin{aligned} \sum \nolimits _{i=1}^n {k_{i}^{P}v_{i}\left( x_{i}^{d+} \right) } >\sum \nolimits _{i=1}^n {k_{i}^{P}v_{i}(x_{ij})} , \forall \, X_{j}\in A. \end{aligned}$$
(8)

Thus, if the condition in (8) is true, (9) and (10) are also true.

$$\begin{aligned}&\sum \nolimits _{i=1}^n {k_{i}^{'}v_{i}\left( x_{i}^{d+} \right) } >\sum \nolimits _{i=1}^n {k_{i}^{'}v_{i}(x_{ij})} , \forall \, X_{j}\in A. \end{aligned}$$
(9)
$$\begin{aligned}&\sum \nolimits _{i=1}^n {k_{i}^{''}v_{i}\left( x_{i}^{d+} \right) } >\sum \nolimits _{i=1}^n {k_{i}^{''}v_{i}(x_{ij})} , \forall \, X_{j}\in A. \end{aligned}$$
(10)

By the LPP first constraint in (2), if there is a feasible solution for \(X^{d+}\) and \(X_{j}\) with respectively weight vectors \(K^{'},K^{''}\in \varphi _{s}^{P}\), this means that the condition presented in (11) would be valid. If (11) condition is valid, it violates the dominance relation definition for \(X^{d+}\), as stated in (8).

$$\begin{aligned} \sum \nolimits _{i=1}^n {k_{i}^{''}v_{i}\left( x_{ij} \right) } >\sum \nolimits _{i=1}^n {k_{i}^{''}v_{i}\left( x_{i}^{d+} \right) } , \forall \, X_{j}\in A. \end{aligned}$$
(11)

Thus, if there is an optimal alternative in A, then the LPP presented in (2) has a solution only for \(X^{d+}\).

Proposition 2

(Dominated alternative) If \(X^{d-}\) is a dominated alternative in A, then \(\forall \,K^{P}\in \varphi _{s}^{P}\,\not \exists \) solution for the LPP given by (2) associated with \({\forall \,X}^{d-}\in A\).

Proof

Let \(X^{d-}\) be an alternative dominated in A, and \(X_{j}\in A\) any other alternative. Suppose that \(K^{'}\in \varphi _{s}^{P}\) is a weight vector solution for the LPP given by (2) for \(X^{d-}\). \(\square \)

By establishing a dominance relation of \(X^{d-},\forall \,K^{P}\in \varphi _{s}^{P}\), the condition presented in (12) should hold.

$$\begin{aligned} \sum \nolimits _{i=1}^n {k_{i}^{P}v_{i}\left( x_{i}^{d-} \right) } <\sum \nolimits _{i=1}^n {k_{i}^{P}v_{i}(x_{ij})} , \forall \, X_{j}\in A \end{aligned}$$
(12)

If (12) holds, this also holds for \(K^{'}\). Thus if for the LPP presented in (2) there is a solution for \(X^{d-}\), then the LPP first constraint can not be violated, and the condition stated in (13) should hold.

$$\begin{aligned} \sum \nolimits _{i=1}^n {k_{i}^{'}v_{i}\left( x_{i}^{d-} \right) } >\sum \nolimits _{i=1}^n {k_{i}^{'}v_{i}(x_{ij})} , \forall \, X_{j}\in A. \end{aligned}$$
(13)

However, if the condition stated in (13) holds, this violates the dominance relationship of \(X^{d-}\), thus \(K^{'}\in \varphi _{s}^{P}\)cannot be a solution for the LPP given by (2) associated with \(X^{d-}\).

Proposition 3

(Non-dominated alternative) If \(X^{nd}\) is a non-dominated alternative in A, then \(\exists \, K^{P}\in \varphi _{s}^{P}\) which is a solution for the LPP given by (2) associated with \(X^{nd}\in A\).

Proof

Let \(X^{nd}\) be a non-dominated alternative in A and \(X_{j}\in A\) any other alternative. Given \(K^{'},K^{''}\in \varphi _{s}^{P}\), there are two different vectors as possible solutions for the LPP given by (7), respectively for \(X^{nd}\) and \(X_{j}\).

From the first constraint of the LPP given by (2), the conditions stated in (14) and (15) are valid, thus \(\exists \,K^{P}\in \varphi _{s}^{P}\) such that \(\vartheta \left( X^{nd},K^{P} \right) >\vartheta \left( X_{j},K^{P} \right) \), \(\forall \, X_{j}\in A\) and the LPP given by (2) has a feasible solution with a \(K^{P}\in \varphi _{s}^{P}\). \(\square \)

$$\begin{aligned}&\sum \nolimits _{i=1}^n {k_{i}^{'}v_{i}\left( x_{i}^{nd} \right) } >\sum \nolimits _{i=1}^n {k_{i}^{'}v_{i}(x_{ij})} , \forall \, X_{j}\in A. \end{aligned}$$
(14)
$$\begin{aligned}&\sum \nolimits _{i=1}^n {k_{i}^{''}v_{i}\left( x_{ij} \right) } >\sum \nolimits _{i=1}^n {k_{i}^{''}v_{i}(x_{i}^{nd})} , \exists \, X_{j}\in A. \end{aligned}$$
(15)

3.3 Preferential properties

Regarding the preference properties, we considered the traditional preference structure of the additive based model, such that preferences are complete in the intra criterion evaluation (total order), which means that the DM is able to evaluate and rank outcome \(x_{ij}\) over attribute i and also that the preferences are transitive and monotonic.

Given the dominance definitions used for this flexible elicitation process, defined in Sect. 3.2, the results provided are consistent with the preferential properties. Based on the dominance definitions presented, the preference relations that may be considered are P, I and H, respectively preference, indifference and non-defined.

The inability to establish a preferential definition amongst alternatives (non-defined) may be a temporary condition while the DM is formalizing and providing his/her preferential information. For as long as the elicitation process gathers more preferential information the arguments for non-defined amongst a set of alternatives may disappear to favor a preference or indifference statement. Thus non-defined in this text shall be distinct as the situation where there are not sufficient arguments to state whether there is a preference or indifference. From this point of view, a non-dominated alternative has non-defined preferential to other non-dominated alternatives, depends on the parameter of the corresponding interaction, resulting from the imprecision inherent in the limited amount of information acquired.

It is important to emphasize that, throughout the flexible elicitation application, the DM is also allowed not to provide a preferential statement if he/she does not feel confident in providing such a statement and such questions are not included as scale constant vector space constraints. Thus questions different from those already asked are put to the DM and these are considered as new constraints on the scale constant vector space.

Thus, \(X^{nd}PX^{d-},X^{d+}PX^{d-}\) and \(X^{nd}HX^{nd}\). Note that there is no preference relation amongst \(X^{d+}\) and \(X^{nd}\), since \(X^{d+}\) exists if, and only if at the current elicitation stage, there is only one single non-dominated alternative from A, the set of alternatives. This set of alternatives A, which represents initially \(\left( A_{0}^{nd}\cup A_{0}^{d-} \right) \), is the union of the set of non-dominated alternatives and dominated alternatives based only on \(\varphi _{0}^{P}\), which is the initial scale constant space vector that considers only the ranking of the scale constants defined in (3). As the iterations of the flexible elicitation procedure are fulfilled, new pieces of information are acquired with regard to the DM’s preferences thereby enabling the preference relations amongst alternatives that could not be compared to be defined.

Thus the evolution of \(\varphi _{s}^{P}\) as s increases is represented by the diagram in Fig. 1. The process of enabling the preference relations amongst alternatives that could not be compared to be defined in the sequence of elicitation defines one set of non-dominated and one set of dominated alternatives for each iteration s, respectively \(A_{s}^{nd}\) and \(A_{s}^{d-}\), while each set is updated by dominance checks after the scale constants vector space is updated with partial preferential information.

Fig. 1
figure 1

The set of alternatives during the procedure

Full size image

The diagram in Fig. 1 denotes how the flexible elicitation establishes relations amongst alternatives. Once the set of alternatives A is equal to the union between \(A_{0}^{nd}\) and \(A_{0}^{d-}\), which represents the first dominance relations that can be established with the information available on the DM’s preferences. As long as partial information is added in the process, the preferential relation amongst alternatives may be established until \(\aleph \left( A_{s}^{nd} \right) =1\), which means that \(A_{s}^{nd}=A_{s}^{d+}\), where \(A_{s}^{d+}\) is the unitary set that contains the optimal alternative. However, if after providing a number of statements the DM is unable to give reliable statements and finishes the elicitation process, the results shall be based on the set of non-dominated alternatives from the last iteration, which probably shall have a small number of non-dominated alternatives.

The preferential results that can be obtained from the use of the flexible elicitation procedure proposed in this paper are defined based on the definition below for Preference, Indifference, Non-definition, and Monotonicity, which is obtained as a secondary result after \(A_{s}^{d+}\) is found.

Proposition 4

(Preference) If there are alternatives \(X_{A},{X}_{B}\), \(X_{C}\) such that \(X_{A},{X}_{B}\in A_{s}^{nd}\) , and \(X_{C}\in A_{s}^{d-}\) after the \(s^{th}\) iteration of the flexible elicitation procedure where the current scale constant space vector is \({\varphi }_{s}^{P}\). Then \(X_{A}PX_{C}\) and \(X_{B}PX_{C} \forall K^{P}\in { \varphi }_{s}^{P}\)

Proof

Let \(A_{s}^{d-}=\{ X^{d-}\in A\vert \forall \,K^{P}\varphi _{s}^{P}\,\not \exists \) solution for the LPP given by \((2){\forall \,X}^{d-}\in A \}\), it follow \({\textit{that }}\max {\vartheta \left( X^{d-},K^{P} \right) <\min {\vartheta \left( X_{j},K^{P} \right) }}\forall \,X_{j}\in A, X_{j}\ne X^{d-}{\textit{ and }}K^{P}\in \varphi _{s}^{P}\) and \(A_{s}^{nd}=\left\{ X^{nd}\in A\vert \max {\vartheta \left( X^{nd},K^{P} \right) }>\vartheta \left( X_{j},K^{P} \right) \forall \,X_{j}\in A, X_{j}\ne X^{nd}{\textit{ and }}K^{P}\in \varphi _{s}^{P} \right\} \). If \(X_{C}\in A_{s}^{d-}{\textit{ and }}X_{A},X_{B} \in A_{s}^{nd} {\textit{ then }} \vartheta \left( X_{A},K^{P} \right)>\vartheta \left( X_{C},K^{P} \right) {\textit{ and }} \vartheta \left( X_{B},K^{P} \right) >\vartheta \left( X_{C},K^{P} \right) ,\forall \,K^{P}\in \varphi _{s}^{P}\), thus \(X_{A}PX_{C} {\textit{ and }} X_{B}PX_{C} \forall K^{P}\in \varphi _{s}^{P}\). \(\square \)

Proposition 5

(Indifference) Let \(X_{A}, X_{B} \in A=\left( A_{0}^{nd}\cup A_{0}^{d-} \right) \) and \(K^{P}\in \varphi _{s}^{P}\), with \(X_{A}I X_{B}\) if \(X_{B}I X_{A}\) for \(s\left( i \right) \). If \(\left( i \right) \) holds, then \(X_{A}I X_{B}\,\,{\textit{and}}X_{B}I X_{A}\,\, \forall \, s\) and \(X_{A}= X_{B}\).

Proof

\(X_{A}I X_{B}\) in s if \(\vartheta \left( X_{A},K^{P} \right) =\vartheta \left( X_{B},K^{P} \right) \forall \,K^{P}\in \varphi _{s}^{P}\), then \(\sum \limits _{i=1}^n {k_{i}^{P}y_{iA}} =\sum \limits _{i=1}^n {k_{i}^{P}y_{iB}}\) and \(y_{iA}=y_{iB}\), thus for all \(v_{i}\left( x_{ij} \right) =y_{ij}\) that \(v_{i}\left( x_{ij} \right) \) is a bijective \(\forall \,i\) and \(x_{ij}\) a consequence on attribute i of alternative \(X_{j}\in A\), if \(\sum \limits _{i=1}^n {k_{i}^{P}y_{iA}} =\sum \limits _{i=1}^n {k_{i}^{P}y_{iB}} \forall \,k\in \varphi _{s}\) then \(x_{iA}=x_{iB}\), thus \(X_{A}= X_{B}\) and \(X_{A}I X_{B}\) and \(X_{B}I X_{A}\) for all s. \(\square \)

Proposition 6

(Non-defined) \({\textit{Let }} X_{A}, X_{B} \in A_{s}^{nd} {\textit{ then }} X_{A} H X_{B}{\textit{ and }}{ X}_{B}H X_{A}\). This Non-defined binary relation is required since the specific values for the additive value model are not explicitly defined yet. Thus, once more information is provided with regard to the scale constants and the space of weights updated, such binary relation between \(X_{A}, X_{B}\) may be established in terms of preference or indifference.

Proof

Given \(X_{A}, X_{B} \in A_{s}^{nd}\) and \(K^{'},K^{''}\in \varphi _{s}^{P}\), and \(K^{'},K^{''}\) are respectively solutions for the LPP defined by Eq. (7) when \(X_{A}, X_{B}\) are considered, then \(\vartheta \left( X_{A},K^{'} \right) >\vartheta \left( X_{B},K^{'} \right) \) and \(\left( X_{B},K^{''} \right) >\vartheta \left( X_{A},K^{''} \right) \). Thus as \(X_{A}\) is not preferable to \(X_{B}\), and \(X_{B}\) is not preferable to \( X_{A}\), since \(X_{A}\ne X_{B}\) then \(X_{A}\) and \(X_{B}\) are not indifferent, thus \(X_{A} H X_{B}\) and \({X}_{B}H X_{A}\), same results for \(X_{A},X_{B}\in A_{s}^{d-}\) and \(X_{A}\ne X_{B}\). \(\square \)

Proposition 7

(Monotonicity (Rank Reversal)) Let \(X_{A}, X_{B} \in A_{s}^{nd}\) and \(X_{C}\in A_{s}^{d-}\). For a new alternative \(X_{E }\in \left( A_{s}^{nd}\cup A_{s}^{d-} \right) \) now considered, if conditions established for preference and non-defined hold, thus \(X_{A}PX_{C}, X_{B}PX_{C}\) and \(X_{A}HX_{B}\). \(X_{E}\) will be related to \(X_{A}, X_{B}\) and \(X_{C}\) depending only on how \(X_{E}\) relates to \({A}_{s}^{nd}\) and \(A_{s}^{d-}\), hence relations amongst \(X_{A}, X_{B}\) and \(X_{C}\) will remain the same.

Proof

Let \(w_{i}=\{ \min \left( x_{ij} \right) \vert x_{ij}\) in \(X_{j} \in (A_{0}^{nd}\cup A_{0}^{d-}) \} , b_{i}=\left\{ \max \left( x_{ij} \right) \vert x_{ij} \, {\textit{in}} \, X_{j}\right. \left. \in (A_{0}^{nd}\cup A_{0}^{d-}) \right\} \) and \(X_{E}=\left\{ x_{1E},x_{2E},\ldots , x_{iE},\ldots ,x_{nE} \right\} \). \(\square \)

Given \(K^{'},K^{''}\in \varphi _{s}^{P}\), and \(K^{'},K^{''}\) are respectively solutions for the LPP defined by Equation (7) when \(X_{A}, X_{E}\) are considered, then \(\vartheta \left( X_{A},K^{'} \right) >\vartheta \left( X_{E},K^{'} \right) \) and \(\left( X_{E},K^{''} \right) > \vartheta \left( X_{A},K^{''} \right) \), then \(X_{A}, X_{E}\in A_{s}^{nd}, X_{A}PX_{C}{\textit{ and }} X_{E}PX_{C}\).

If \(\not \exists \,K^{P}\in \varphi _{s}^{P}\) such that \(\vartheta \left( X_{B},K^{P} \right) >\vartheta \left( X_{E},K^{P} \right) \) then the LPP defined by Eq. (7) has no solution for \(X_{B}\) when \(X_{E} \in \left( A_{s}^{nd}\cup A_{s}^{d-} \right) \), then \(X_{E}PX_{B}\), thus with \(X_{B}, X_{C}\in A_{s}^{d\mathrm {-}}, X_{B}H X_{C}\). As \(\vartheta \left( X_{B},K^{P} \right) >\vartheta \left( X_{C},K^{P} \right) \forall K^{P}\in \,\varphi _{s}^{P}, X_{C}PX_{B}\) is still not possible and the previous rank statement is never violated. As long as \(\exists \,X_{E} \in \left( A_{s}^{nd}{\cup }\,A_{s}^{d-} \right) \) then \(X_{B}HX_{C}{\textit{ and }} X_{C}HX_{B}{\textit{ as }}X_{B}, X_{C}\in A_{s}^{d-}\).

If \(x_{iE}\notin \{w_{i}, b_{i}\}\forall \,i\), a new \(\varphi _{1}^{P}\) should be elicited in order to be consistent with new scale boundaries according to the tradeoff conditions.

4 Empirical evidence of performance

In this section, a performance experiment is conducted for the proposed method for flexible elicitation. The experiment is based on a Decision Analysis MSc course activity when students were asked to formulate a real problem and given two weeks solve it using the proposed method for elicitation. The number of statements required using this new approach is compared with benchmarks for the traditional tradeoff elicitation procedure described by Keeney and Raiffa (1976), which are related only with the number of criteria of the problem. Therefore, statistical tests are performed to evaluate if the number of statements required when using this new procedure is less than the traditional benchmarks.

4.1 Benchmark definition

The proposed procedure is meant to replace part of the tradeoff scale constant elicitation procedure after the scale constants rank is obtained, which is the part of the procedure that is frequently called into question in the literature. Thus we shall consider the flexible elicitation procedure comparison with the tradeoff elicitation approach given the scale constant rank. Thus, the benchmark considered for the proposed method is the number of questions asked usually in a tradeoff elicitation procedure for a multi-attribute additive value function.

If there are n attributes then the traditional tradeoff elicitation procedure requires a minimum of \(n-1\) indifference questions to define the value of the scale constants if DM is asked directly about \(x_{ij}^{I}\), without any calibration questions (Keeney and Raiffa 1976).

For this performance evaluation, two benchmark definitions will be considered based on the traditional tradeoff when eliciting scale constants. The first benchmark will be defined, as \(n-1\) indifference statements to find \(x_{ij}\) which is the minimum number of questions the DM will be asked (Keeney and Raiffa 1976). The second benchmark will be defined as \(3(n-1)\) statements, corresponding to \(2(n-1)\) preference statements to identify reference points such as \(x_{ij}^{''}\) and \(x_{ij}^{'}\) in order to define the boundaries for each scale constant as a calibration process before asking the DM \(n-1\) indifference statements in order to find \(x_{ij}^{I}\) as suggested by Keeney and Raiffa (1976). Other such as Riabacke et al. (2012) indicate that n(n\(-1\)) statements may be required when including consistency check, but this would not be a reasonable benchmark for this study.

One of the objectives of these tests are highlight the contribution of the proposed procedure and to provide evidence that it requires the DM to make less effort. Moreover, as per the premises of the flexible elicitation procedure, it is likely there will be fewer errors during the elicitation process. These results will also strengthen the contribution of the proposed dominance tests used throughout the procedure.

4.2 Sample characteristics

The sample comprises real problems structured by real DMs, who enrolled on a postgraduate course for professionals. Thus, there is a broad range business organizational decision problems, such as selecting candidates for Human Resources internship, selecting industrial equipment, selecting a maintenance strategy for an electrical power network, selecting suppliers, and so forth. Each DM after taking a decision analysis course framed a real problem from his/her organization and solved it using an additive tradeoff based approach. In total the sample is based on 23 problems which used a range of from 3 to 8 criteria and from 3 to 27 alternatives.

The main aspect to answer with this test is whether the proposed procedure is able to help a DM to solve a decision problem within the same amount of preferential statements compared to other tradeoff elicitation procedures. To answer that question, a pair wise comparison can be made amongst the number of preferential statements required by the proposed procedure and the number of preferential statements required from each of the two benchmarks considered by using a test for two dependent samples. In this case, the Wilcoxon matched pairs test (Siegel and Castellan 1988) was chosen since it does not require t test assumptions. However, if these assumptions are met, then this test is almost as powerful as the t test (Siegel and Castellan 1988).

Another important aspect to verify is whether or not the performance of the proposed procedure is affected by the number of criteria considered in the problem. To address this issue, the Wilcoxon test shall be conducted depending on the size of the sample, based on the number of criteria. The sample distribution based on the number of criteria is given with a histogram in Fig. 2.

4.3 Results

Table 1 shows how the benchmarks and the results of the proposed procedure were evaluated, and the descriptive statistics over the number of statements required in each approach.

Fig. 2
figure 2

Histogram of the number of sample criteria

Full size image

From the statistics of Table 1, it can be seen that the flexible elicitation procedure has almost the same performance as the Benchmark I. Although there is a greater standard deviation in the results of the proposed procedure, it has the lowest mode and same median as Benchmark I, which is less than those of Benchmark II.

When comparing each sample, the Wilcoxon matched pairs test indicates that the performance of the proposed procedure is statistically the same as the Benchmark I performance. Thus the null hypothesis stating that the two variables are not different from each other was not rejected with a p value of 0.658 and since 4 of 23 cases have the same number of statements, the valid number of cases for this test was 19. The performance results for this are displayed in a Box and Whisker plot given in Fig. 3a.

Table 1 Descriptive statistics of the number of statements required
Full size table
Fig. 3
figure 3

Box and Whisker plot for number of statements required for Benchmark I and II versus the proposed procedure

Full size image

The comparison with Benchmark II revealed that the performance of the proposed procedure has a better result. The result obtained from the Wilcoxon matched pairs test indicates that the proposed procedure requires fewer statements than Benchmark II does. The null hypothesis stating that the two variables are not different from each other was rejected in favor of the alternative hypothesis stating that the proposed procedure requires fewer statements than Benchmark II with a p value of 0.0033, with only 1 of 24 cases with the same number of statements. The Box and Whisker plot for this test is presented in Fig. 3b.

With regard to the second question, if the number of criteria affects the performance of the proposed procedure when compared with the benchmarks, the Wilcoxon matched pairs test was performed only for the cases which had 3 and 4 criteria due to the availability of cases in the sample. These results are summarized in Table 2.

Table 2 Wilcoxon matched pairs test results for cases with 3, 4 and 5 criteria
Full size table

From the results obtained from Table 2 it can be seen that the results for the comparison of the proposed procedure and Benchmark I remains the same, and that there still are high p values to support this result.

When comparing with Benchmark II, there is contradicting result for the subsample of problems with 3 criteria. However, this subsample has a small number of valid cases compared with the subsample of problems with 4 criteria.

Although there were three in four tests that supported the results of the previous performance tests, from the available data, we cannot state that the performance of the proposed procedure is not affected by the number of criteria considered in the problem, although this result was obtained only from one test based on a quite small sample.

The Box and Whiskers plot for this performance test is presented below in Fig. 4.

Fig. 4
figure 4

Box and Whisker plot for the number of statements required in the subsample of problems with 3 criteria and 4 criteria

Full size image

It is interesting to remark that despite having a potential outlier on the sample, the choice for keeping this observation is to show that there may be such situations when a high level of precision is required for defining the weights, thus, the parameters are very sensitive. In other words, the alternatives are so close in performance that in fact, it is difficult for the DM to differentiate amongst the best one, and another aspect may be observed in practice such as errors and/or biases that may occur when establishing alternatives performance value as well. Such situation has also been widely explored within the literature such as in Medeiros et al. (2016).

5 Discussion

There are several methods for eliciting weights dedicated to different contexts as discussed in Sect. 2. However, most of the procedures available do not incorporate tradeoff characteristics and requirements in the elicitation process that enable weights to be elicited with regard to the range and importance of the attributes scale. Thus these do not represent scale constants.

Although simplifications during the elicitation process are acceptable as long as this reduces the DM’s cognitive effort or mistakes, thereby improving the decision process, it is important to consider features of the scale of attributes when defining the scale constant. On seeking to address this matter, the flexible elicitation procedure proposed in this paper tries to harmonize the quality of the decision process by reducing the cognitive effort required from DM and by taking the assumptions and particularities of tradeoff elicitation into account. Based on these premises a new procedure was proposed and its performance was tested within real problems with real alternatives and solved using the preferences of real DMs.

Given that there are no right or wrong answers for such problems, these really rely on the DM’s preference structure. The core contribution of this paper is not to assist the DM in how to find a specific scale constant vector \(K^{P}\), but to assist in how to find the subspace \(\varphi _{s}^{P}\) that has greater chances of containing the right answer for \(K^{P}\). Allied to this, the proposed procedure seeks to require as minimum a cognitive effort as possible, no matter how large \(\varphi _{s}^{P}\) may be, if this is enough to find an optimal alternative.

As expected, the performance tests showed that in many real situations the number of preferential statements required from a DM can be reduced by applying a dominance check whenever the DM provides an “update” on \(\varphi _{s}^{P}\); in other words, the DM’s preferences.

Thus, this paper presented a procedure that avoids indifference statements from DM, which deals with partial information acquired from preference statements based on the range of the attributes scale, defined from the set of alternatives, thereby enabling the DM to provide preference statements with flexibility such that the DM is not required to provide statements if there is any doubt about such a statement. Thus, this characteristic enables the DM to make more reliable statements.

However when comparing the proposed approach with traditional tradeoff elicitation procedures, such as those proposed by Keeney and Raiffa (1976), there is a limitation which is the interpretation of the scale constants as a substitution rate amongst attributes. This flaw in the proposed procedure is compensated for by its offering more reliable results and because it does not require a sensitivity analysis since this has already been performed in a robust way during the elicitation process.

To provide an idea of the substitution rate amongst attributes the DSS built to perform the proposed procedure presents a graphical report with the range defined by \(\varphi _{s}^{P}\) for each scale constant. Thus if the DM would like to evaluate the substitution rate amongst attributes, this would have to be based on the ranges admitted for each scale constant.

Despite the limitation on the number of samples, the performance test presented in Sect. 4 enabled interesting results to be found that provide evidence that the proposed procedure would require less information and lead to fewer errors as it is not seeking specific values for \(K^{P}\), but uses \(\varphi _{s}^{P}\) instead.The test results support the claim that the approach of the flexible elicitation procedure requires the DM to make less cognitive effort because it indicates that the number of statements required from this new proposed procedure is statistically the same as the minimum required by the traditional tradeoff method and even less than the number of statements required from the traditional tradeoff with calibration questions.

5.1 Future studies

From the results obtained and the perspective of the contribution of this paper, there are several questions to be addressed, especially for the problems that correspond to the region above the upper quartile of the number of required statements.

For this class of problems there is still much work to do in the sense of determining the maximum admitted cost relation amongst preference statements and indifference statements in order to assure that the proposed procedure can also overcome also this class of problems. A definition for a cost related to each type of cognitive statement could also be used to evaluate different elicitation procedures available in the literature, thus enabling these procedures to be classified based on a “cost-effectiveness” perspective of the effort required during the decision process.

Regarding the relation amongst the set of alternatives and \(\varphi _{s}^{P}\), experiments can be design to predict the typology of the problem and how many statements it would be fair enough to require in order to find an optimal solution.

Another issue that shall be addressed in future research is how to improve and assure that the sequence of questions that are made to the DM is minimized given previous knowledge of the problem typology defined by the set of alternatives and \(\varphi _{s}^{P}\) properties.