Keywords

1 Introduction

The main difficulty of comprehensive evaluation is the weight determination of each attribute of evaluated objects by evaluators. The characteristic of evaluation method of attribute coordinate is that the evaluation way is very close to the normal thinking pattern of people and the corresponding preference curve can be accurately constructed according to the preference of the evaluator. Hence, this method can not only learn about the experience of experts but also give full play to the advantages of machine learning [1,2,3,4,5]. After more than ten years of research and application, the evaluation method of attribute coordinate has made a certain achievements in many fields [6,7,8,9,10,11,12,13,14]. But after the new evaluation samples are added, the evaluation model needs to be reconstructed rather than the dynamic update. There has been no progress on the study on how to dynamically update the evaluation model. This paper proposes a kind of update model to solve this problem. The remaining parts of this paper are organized as follows. Section 2 introduces the related work including the introduction to updating strategies and evaluation method of attribute coordinate. Section 3 elaborates the algorithm through the flow chart, which combines the evaluation method with strategy III. Section 4 carries out the experiments and validates the updating algorithm. Section 5 comes to the conclusion.

2 Related Work

2.1 Classification on Updating Strategy

  1. (1)

    The updating of evaluation model

After the completion of evaluation model, if new evaluation sample data are collected, the evaluation model needs to be correspondingly updated. So the updating process refers to the process changing from the evaluation on old samples to that on both new and old samples. For the comprehensive evaluation, the updating of the evaluation model is an essential process. The purpose and effect of evaluation model update are that the original understanding is amended by the latest information to decrease the subjective or objective inaccuracy caused by the imperfect understanding. Moreover, the latest information could be various and contain other information in the comprehensive evaluation, such as the expert experiences and preferences.

The updating mode of evaluation model of attribute coordinate is shown in Fig. 1. The figure indicates that, in the evaluation depending on the weight obtained from samples, when new samples are added, they are likely to have a certain impact on old samples like changing the weight of samples, which further gives rise to the change of evaluation result. Old samples and new samples need to work together to generate new samples in light of a certain weight calculation method.

Fig. 1.
figure 1

Updating mode of samples with weight

Full size image
  1. (2)

    Updating Strategy

We divide the updating methods of evaluation model into three strategies: Strategy I, Strategy II and Strategy III.

Strategy I

Strategy I refers to the algorithm that after the new data are added to the original data, apply the original evaluation model to update the evaluation results. Its main characteristic is that both old sample data and comprehensive evaluation model are known. Although the updating method is easy, the result may be beyond accuracy, thus the updating is meaningless.

Strategy II

Strategy II refers to the algorithm that all the data join to calculate the weight. Its main characteristic is that the new and old data are thought of as the original data to reconstruct the new evaluation model. Strictly, this kind of updating is the standard to verify the effectiveness of other updating algorithms. However the drawback is it might take enormously long time.

Strategy III

Strategy III refers to the algorithm that the newly collected data are used to supplement and affect the original comprehensive evaluation model. Its core is how to impose the new weight on the latest information.

Obviously, from Strategy I to Strategy III, as the constraint conditions are more and more complex, the updating algorithm will be more and more difficult.

2.2 Introduction to Attribute Coordinate Evaluation

Comprehensive evaluation is to evaluate whether the evaluated objects with multi attributes are good or not. After attribute values with certain unified dimension are given to all the evaluated objects, the optimal evaluated object is the solution A = (10,…,10) that each attribute value is full score (assuming the full score is 10). However, there does not exist the optimal solution in the comprehensive evaluation but generally exists the satisfactory solution. Hence, the satisfactory solution can be obtained instead of the optimal solution in the practical decisions. Thus the comprehensive evaluation could only require the most satisfactory solution meeting some weighted conditions. When evaluators evaluate multi-attribute objects, they often think that some attributes are important, namely, attributes can be given certain weights in the evaluation model. The importance of different attributes would change with the degree to which good evaluated objects or bad evaluated objects belong, namely, the weight of an attribute would dynamically change in the evaluation model.

  1. (1)

    Solution of attribute barycentric coordinate

It is assumed that \( S_{T} = \{ x_{i} = \left( {x_{i1} , \ldots ,x_{im} } \right)\left| {\sum\limits_{j = 1}^{m} {x_{ij} } = T} \right.\} \) is a hyper plane with the total score T. \( x_{i} = \left( {x_{i1} , \ldots ,x_{im} } \right) \) are supposed to be independent attribute values. The intersection of \( S_{T} \) and X (S T ∩X) is a (n-1)-dimensional simplex (shown in Fig. 2, e.g. \( S_{{100}} \cap X~ = {\varDelta}ABC \), and △A’B’C’ is a two-dimensional simplex that the total score hyperplane \( S_{T} \) of 10 < T < 10 × n intersects X, (S T ∩X) =△A’B’C’).

Fig. 2.
figure 2

Two-dimensional simplex from ST∩X = △A’B’C’

Full size image

Let {xk,k = 1,…,s}⊆ST∩X be the sample solution set and \( Z = \left\{ {z_{ 1} , \cdots ,z_{n} } \right\} \) be the evaluator set. The decision maker \( z_{i} \) selects t sets of satisfactory solutions {x h , h = 1,…,t}, and each solution is marked as w h (x h ). Because the solution space X is a convex set with respect to score w h (x h ), the barycenter \( b\left( {\left\{ {x_{{^{h} }} \left( {z_{i} } \right)} \right\}} \right) \) of \( \left\{ {x_{{^{h} }} \left( {z_{i} } \right)} \right\} \) can be solved as formula (1) by the weighted average method (w h (x h ) is taken as the weight).

$$ b\left( {\left\{ {x_{h} \left( {z_{i} } \right)} \right\}} \right) = \left( {\frac{{\sum\limits_{h = 1}^{t} {w_{h}^{i} x_{1h} } }}{{\sum\limits_{h = 1}^{t} {w_{1h}^{i} } }}, \cdots ,\frac{{\sum\limits_{h = 1}^{t} {w_{mh}^{i} x_{mh} } }}{{\sum\limits_{h = 1}^{t} {w_{mh}^{i} } }}} \right) $$
(1)
  1. (2)

    Solution of attribute barycentric curve

Obviously, when we have large enough training sample set {x k } and more enough training times, the solution sets selected by decision makers \( z_{i} \) are enough, the barycenter \( b\left( {\left\{ {x_{h} \left( {z_{i} } \right)} \right\}} \right) \) would be close to the local most satisfactory solution x*|T, namely, \( \mathop {\lim }\limits_{h \to \infty } \) b({x h (z)})→x*|T. When the number of decision makers \( Z \) is more than 1, \( \mathop {\lim }\limits_{h \to \infty } b^{{\prime }} \left( {\left\{ {x_{h} \left( Z \right)} \right\}} \right) \) is the local most satisfactory solution of decision makers \( Z \) in ST∩X, namely, the barycenter of decision makers. The set of all local most satisfactory solutions \( \left\{ {b^{{\prime }} \left( {\left\{ {x_{h} \left( Z \right)} \right\}} \right)\left| {T \in \left[ {10,10 \times m} \right]} \right.} \right\} \) can be obtained after T covers the interval \( \left[ {10,10 \times m} \right] \). The set would be a line, written as \( L\left( {b^{{\prime }} \left( {\left\{ {x_{h} \left( Z \right)} \right\}} \right)} \right) \), and this line is called as the local most satisfactory linear solution of decision makers Z. The line can be obtained by polynomial curve fitting method such as the following polynomial function (2):

$$ G(T) = a_{0} + a_{1} T + a_{2} T_{2} $$
(2)

In this situation, three local most satisfactory solutions of decision makers in S100 ∩ X, ST ∩ X and S10 × m ∩ X are taken as three interpolation points to be substituted into Lagrange interpolation formula (3) to calculate the local most satisfactory solution line.

$$ g_{i} (T) = \frac{{(T - x_{1}^{*} )(T - x_{2}^{*} )}}{{(x_{0}^{*} - x_{1}^{*} )(x_{0}^{*} - x_{2}^{*} )}}a_{i0} + \frac{{(T - x_{0}^{*} )(T - x_{2}^{*} )}}{{(x_{1}^{*} - x_{0}^{*} )(x_{1}^{*} - x_{2}^{*} )}}a_{i1} + \frac{{(T - x_{0}^{*} )(T - x_{1}^{*} )}}{{(x_{2}^{*} - x_{0}^{*} )(x_{2}^{*} - x_{1}^{*} )}}a_{i2} $$
(3)

2.3 The Work of the Paper

After the calculation of barycentric point \( b^{'} \left( {\left\{ {x_{h} } \right\}} \right) \), if the new sample data are collected again, Strategy II is used to substitute both new and old sample data into the formula (3) for recalculation. The advantage of this method is that the calculation results are very accurate, but the disadvantages are that repeated marks on old samples are unavoidable. Besides, the calculation will not be feasible if the old data are lost. However, if Strategy III is used, only the marks on new samples are needed, hence this algorithm is simple and effective. In the paper, we integrate the Strategy III into the evaluation model of attribute coordinate to accomplish the updating of the model in the case of new sample data being added into the current model. The combined algorithm is described as follows.

3 Updating Algorithm of Evaluation Model of Attribute Coordinate

3.1 Algorithm Process

Figure 3 illustrates the process of the algorithm in the way of flow chart.

Fig. 3.
figure 3

The process of the algorithm

Full size image

3.2 Updating of Attribute Barycentric Coordinates

After calculating old barycentric coordinates, if t new samples are added and they are marked as \( w_{t} \left( {\left\{ {x_{h} } \right\}} \right) \), respectively, the formula (1) can be updated as formula (4). Where s is the number of old samples, \( x_{1} \),…, \( x_{m} \) are the old barycentric coordinates.

$$ b^{u} = \left[ {\frac{{\frac{{\sum\limits_{h = 1}^{t} {w_{h}^{i} x_{1h}^{{}} } }}{{\sum\limits_{h = 1}^{t} {w_{1h}^{i} } }}{ + }sx_{1} }}{t + s}, \cdots ,\frac{{\frac{{\sum\limits_{h = 1}^{t} {w_{mh}^{i} x_{mh}^{{}} } }}{{\sum\limits_{h = 1}^{t} {w_{mh}^{i} } }} + sx_{m} }}{t + s}} \right] $$
(4)

The new barycentric coordinate figured out is \( b^{\text{u}} \left( {\left\{ {x_{h} \left( {z_{i} } \right)} \right\}} \right) \), and the barycentric coordinate of old model is \( b^{{u}} \left( {\left\{ {x_{h} \left( {z_{i} } \right)} \right\}} \right) \). Their norm can be solved according to the formula (5) and be calculated by Euclidean distance, as green segment shown in Fig. 4.

Fig. 4.
figure 4

The old and updated barycentric curve (Color figure online)

Full size image
$$ e = \left\| {b^{\text{u}} \left( {\left\{ {x_{h} \left( {z_{i} } \right)} \right\}} \right){ - }b^{{\prime }} \left( {\left\{ {x_{h} \left( {z_{i} } \right)} \right\}} \right)} \right\| $$
(5)

Figure 4 also shows the old psychological barycenter curve L(x) and the updated psychological barycentric curve L’(x). T represents a certain total score obtained from Attribute1 and Attribute2.

3.3 Satisfaction Calculation after Updating Barycentric Coordinates

When there exists the new psychological standard point \( b^{u} \left( {\left\{ {x_{h} \left( {z_{i} } \right)} \right\}} \right)\left| T \right. \) of Z in ST∩X, taking \( b^{u} \left( {\left\{ {x_{h} \left( {z_{i} } \right)} \right\}} \right)\left| T \right. \) as the standard, the global satisfaction function (6) is used for the satisfaction evaluation of all the solutions in ST∩X.

$$ sat(x,Z) = \lambda \left( {x,Z} \right) *\exp \left( { - \frac{{\sum\limits_{j = 1}^{m} {w_{j} \left| {x_{ij} - b_{ij}^{u} (\{ x_{h} (Z))} \right|} }}{{\sum\limits_{j = 1}^{m} {\delta_{j} (b_{ij}^{u} (\{ x_{h} (Z)) - \delta_{j} )} }}} \right) $$
(6)

Where \( sat\left( {x_{i,} Z} \right) \) is the satisfaction of evaluated object \( x_{i} \) from evaluator Z, whose value is expected to be between 0 and 1. \( x_{ij} \) is each attribute value. \( \left| {x_{ij} - b_{ij}^{u} (\{ x_{h} (Z))} \right| \) is to measure the difference between each attribute value and the corresponding barycentric value. \( w_{j} \) and \( \delta_{j} \) are used as the factor which can be adjusted to make the satisfaction comparable value in the case where the original results are not desirable. It is found from the above learning that all the solutions in \( L\left( {b^{u} \left( {\left\{ {x_{{^{h} }} \left( Z \right)} \right\}} \right)} \right) \) are determined by taking the local most satisfactory solution as the standard in different simplexes ST∩X as well as calculating the maximal satisfaction by formula (6). The correction coefficient \( \lambda \left( {x,Z} \right) \) is described as \( \lambda \left( {x,Z} \right) = \left( {\frac{{\sum\limits_{i = 1}^{m} {x_{ij} } }}{{\sum\limits_{j = 1}^{m} {X_{j}}}}}\right)^{S} \), Where \( \sum\limits_{j = 1}^{m} {X_{j} } \) is the sum of X j with each attribute value the full score,\( \sum\limits_{ij = 1}^{m} {x_{ij} } \) is the sum of each attribute value xij of solution xi, and S = \( \left( {\frac{{\sum\limits_{j = 1}^{m} {Xj} }}{{\sum\limits_{i = 1}^{m} {xij} }}} \right)^{{}} \). Formula (6) can reach consistent satisfaction \( sat(x,Z) \) in the entire solution space, so λ(x,Z) is called the global consistent coefficient.

4 Experiments

The data set comes from the scores of college entrance examination of a certain city in 2015 and contains more than 4,000 objects to be evaluated. Moreover, each object contains four attributes: Chinese, mathematics, English and comprehensive discipline. This simulation adopts the first three attributes in order to get a more intuitive three-dimensional chart.

4.1 Experiments of Updated Barycentric Coordinates

According to evaluation model of attribute coordinates, firstly, a hyper plane with the total score of 370 is selected, on which there are 26 sample points. Then the 26 sample points are marked by five experts. It is assumed that 18 sample points are initial samples and the rest (8 sample points)are the supplementary samples. The above attribute coordinate updating algorithm is used to update the barycentric coordinates of attributes.

  1. (a)

    Calculate the original barycentric coordinates. According to formula (1), the barycentric coordinates of 18 sample points are figured out, as black points shown in Fig. 5.

    Fig. 5.
    figure 5

    Comparison among original and updated barycentric coordinates with Strategy II and III (Color figure online)

    Full size image
  2. (b)

    Calculate the new barycentric coordinates by the updating method of Strategy II. According to formula (1), the barycentric coordinates of 26 sample points are figured out, as green points shown in Fig. 5.

  3. (c)

    Calculate the new barycentric coordinates by the updating method of Strategy III. The barycentric coordinates integrated with the other 8 sample points are obtained according to formula (4), as red points shown in Fig. 5.

Table 1 shows the results of barycentric coordinates in terms of the above three cases.

Table 1. Comparison of original and updated barycentric coordinates
Full size table

As we had stated in Sect. 2, the updated barycentric coordinates by Strategy III provides more accurate result as the standard to verify the effectiveness of other updating algorithms. It can be seen from Table 1 that the updated barycentric coordinates by Strategy III cannot be completely equal to the updated barycentric coordinates by Strategy II but be close to it.

Figure 5 illustrates the position of the three barycentric coordinates in three dimentional space, each point including three attributes indicated with CN (Chinese), MATH (mathematics), and EN (English).

In Fig. 6, the black point is the barycentric coordinates which are figured out from the original samples. Then the satisfaction curve illustrates the degrees of satisfaction of samples according to formula (6), as the red line shown in Fig. 6. The barycentric coordinates of 8 sample points evaluated by experts are shown as the blue point. The new barycentric coordinates updated by formula (4) is shown as the red point. Finally, the updated satisfaction curve is figured out as the yellow line (solid line).

Fig. 6.
figure 6

Comparison of original and updated barycentric curves (Color figure online)

Full size image

From Fig. 6, the satisfaction curve after adding sample is basically close to the original satisfaction curve (dotted line), which indicates that our updating algorithm is effective. Compared with the Strategy II which needs to calculate the barycentric coordinates for all the samples again, this algorithm greatly avoids the repeated evaluation on old samples and effectively reduces the time of evaluation.

5 Conclusions

This paper studies how to update the existing evaluation model based on the evaluation of attribute coordinate after the new sample data are added and presents the updating algorithms of barycentric coordinates and satisfaction function. The simulation of comparison is carried out according to the real data. It can be clearly seen from the calculation results that the updating strategy of Strategy III is used to figure out the new barycentric coordinates and then to calculate the new satisfaction for all samples by calculating the barycentric coordinates of supplementary samples. This can not only realize the effective update of evaluation system but also greatly decrease the calculation work. In the further work, we would conduct the multi-expert evaluation on the increased samples, integrate the supplementary samples into the old evaluation model, and then observe how they affect the evaluation results.