Keywords

1 Introduction

Comprehensive evaluation is used for evaluating the evaluated objects good or bad. When certain uniform-dimension attribute value is endowed to all the evaluated objects, the optimum evaluated object is A = (10, …, 10) with each index value full score (the full score is assumed as 10). However, the optimum principle is not usually adopted in comprehensive evaluation, instead the satisfaction principle is usually adopted. That is, in actual decision making, we usually obtain the satisfactory solution rather than the optimum solution. That also explains why comprehensive evaluation tends to explore the most satisfactory solution meeting some conditions of weight [1,2,3,4,5].

The attribute coordinate comprehensive model features allowing the evaluator to subjectively weigh the evaluated objects by scoring the given samples, which reflects the evaluator’s preference upon some indexes (attributes), and then calculate the satisfactory value of each evaluation index [6,7,8,9,10,11,12,13]. However, the method has its disadvantage. If there are too many evaluation indexes and samples, it is usually difficult for the evaluator to accurately give the score to each sample, so such result may not well reflect the psychological weight of the evaluator, thus influence the subsequent process and accordingly the accuracy of the final evaluation. The paper improves the weighing method for the evaluator in the way that the evaluator is not required to give the specific scores to the samples but rank the given samples, and then the ranking values are converted into the evaluator’s psychological weight. Thus it is much easier for the evaluator to rank the samples than rate them, and also more accurate to obtain the evaluator’s psychological weight.

In the paper, firstly, the attribute coordinate comprehensive evaluation model is briefly introduced; then, the improved weighing method is described; finally, the evaluation results obtained by the two methods before and after improvement are compared through the simulation experiment.

2 Brief Introduction to Attribute Coordinate Comprehensive Evaluation Model

2.1 Local Satisfactory Solution

When evaluating the multi-attribute object, the evaluator usually thinks that some attributes are more important than others, and should be endowed with more weights. The importance of attributes can possibly change along with the advantageous or disadvantageous degree of the evaluated objects, and such change reflected in the evaluation model is the dynamic change of the weight of one attribute.

One of the main characteristics of the attribute coordinate comprehensive evaluation is to endow attributes with different weights according to the evaluator’s preference. The specific method is to give ratings on the given samples. Suppose T0 is the critical total scores, and Tmax is the maximum total scores. In the interval \( \left[ {T_{0} ,T_{max} } \right] \), several scores \( T_{1} ,T_{2} , \ldots ,T_{n - 1} \) are uniformly selected according to the requirement for curve fitting. For the total score \( T_{i} \left( {i = 1,2,3 \ldots n - 1} \right) \), several samples are selected for the evaluator and rated according to the evaluator’s psychological preference. Then, the barycentric coordinate with total score of \( T_{i} \left( {i = 1,2,3 \ldots n - 1} \right) \) is calculated by Formula (1).

$$ b\left( {\left\{ {v^{h} \left( z \right)} \right\}} \right) = \left( {\frac{{\sum\limits_{h = 1}^{t} {v_{1}^{h} f_{1}^{h} } }}{{\sum\limits_{h = 1}^{t} {v_{1}^{h} } }}, \cdots ,\frac{{\sum\limits_{h = 1}^{t} {v_{m}^{h} f_{m}^{h} } }}{{\sum\limits_{h = 1}^{t} {v_{m}^{h} } }}} \right) $$
(1)

Where, \( b\left( {\left\{ {vh\left( z \right)} \right\}} \right) \) is the psychological barycentric coordinate of the evaluator z; \( \left\{ {x_{k} ,k = 1, \ldots ,s} \right\} \) is the set of all the samples with total score of Ti, and each sample has m indexes, with the values respectively of \( f_{i} ,\,\,\text{i = 1} \ldots \text{m} \). The evaluator selects t sets of samples \( \left\{ {f_{h} ,h = 1, \ldots ,t} \right\} \) which are believed thereby to be satisfactory, and respectively rates as \( v^{h} \left( {f^{h} } \right) \), which is taken as the evaluator’s psychological weight; then, the weighted average method is adopted to find the psychological barycentric coordinate of the evaluator for the total score Ti, and also seen as the evaluator’s local satisfactory solution. The process is called as the learning of the evaluator’s psychological weight.

2.2 Satisfactory Solution Curve

Obviously, with plenty of training samples and training times, the barycenter \( b\left( {\left\{ {v^{h} \left( z \right)} \right\}} \right) \) will be gradually approximate to the local most satisfactory solution x*|T in total score T, namely: \( \lim\limits_{h \to \infty } b\left( {\{ vh\left( z \right)\} } \right) \to x * |T \). Through the learning of each total score \( T_{i} \left( {i = 1,2,3 \ldots n - 1} \right),b_{i} \left( {\left\{ {vh\left( z \right)} \right\}} \right) \) can be obtained. After T traverses the interval \( \left[ {T_{0} ,T_{max} } \right] \), the set \( \left\{ {b^{\prime } \left( {\{ v^{h} \left( z \right)\} } \right)|T \in \left[ {T_{0} ,T_{\hbox{max} } } \right]} \right\} \) for all local most satisfactory solutions can be obtained. Generally speaking, the psychological criteria of the evaluator z on different total score Ti are consistent with each other. In other words, \( \left\{ {b^{\prime } \left( {\left\{ {v^{h} \left( z \right)} \right\}} \right)|T \in \left[ {T_{0} ,T_{\hbox{max} } } \right]} \right\} \) can form a continuous curve, recorded as \( L\left( {b^{\prime } \left( {\{ v^{h} \left( z \right)\} } \right)} \right) \), which is called as the local most satisfactory solution curve of the evaluator z. \( L\left( {b^{\prime } \left( {\{ v^{h} \left( z \right)\} } \right)} \right) \) can be obtained by polynomial curve fitting. For example, three local most satisfactory solutions are taken as the interpolation points and input into Lagrange interpolation Formula (2) to calculate and the most satisfactory solution curve:

$$ g_{i} (T) = \frac{{(T - x_{1}^{ * } )(T - x_{2}^{ * } )}}{{(x_{0}^{ * } - x_{1}^{ * } )(x_{0}^{ * } - x_{2}^{ * } )}}a_{i0} + \frac{{(T - x_{0}^{ * } )(T - x_{2}^{ * } )}}{{(x_{1}^{ * } - x_{0}^{ * } )(x_{1}^{ * } - x_{2}^{ * } )}}a_{i1} + \frac{{(T - x_{0}^{ * } )(T - x_{1}^{ * } )}}{{(x_{2}^{ * } - x_{0}^{ * } )(x_{2}^{ * } - x_{1}^{ * } )}}a_{i2} $$
(2)

Normally, when T value is larger, the evaluator is better satisfied with the local most satisfactory solution \( b^{\prime } \left( {\{ v^{h} \left( z \right)\} } \right) \) corresponding to T in \( L\left( {b^{\prime}\left( {\{ v^{h} \left( z \right)\} } \right)} \right) \).

2.3 Calculate the Satisfaction Degree for Each Object

With many satisfactory solutions which construct satisfactory solution curve L \( \left( {b\left( {\left\{ {f^{h} \left( z \right)} \right\}} \right)} \right) \), we can calculate the global satisfaction degree according to (3) for each object to reflect how satisfactory it is compared with the satisfactory solution of the total plane to which the object belongs.

$$ sat\left( {f,z} \right) = \left( {\frac{{\sum\limits_{i = 1}^{m} {f_{ij} } }}{{\sum\limits_{j = 1}^{m} {F_{j} } }}} \right)^{{\left( {\frac{{\sum\limits_{i = 1}^{m} {f_{j} } }}{{3(\sum\limits_{j = 1}^{m} {f_{ij} } )}}} \right)_{ * } }} \exp \left( { - \frac{{\sum\limits_{j = 1}^{m} {w_{j} \left| {f_{j} - b(f^{h} (z_{j} )} \right|} }}{{\sum\limits_{j = 1}^{m} {w_{j} \delta_{j} } }}} \right) $$
(3)

Where, \( sat\left( {f,Z} \right) \) is the satisfaction of evaluated object f, whose value is expected to be between 0 and 1. \( f_{j} \) is the value of each index. \( \left| {f_{j} - b(f^{h} (z_{j} )} \right| \) is to measure the difference between each attribute value and the corresponding barycentric value (satisfactory solution). \( w_{j} \) and \( \delta_{j} \) are used as the factor which can be adjusted to make the satisfaction comparable value in the case where the original results are not desirable. \( \sum\limits_{j = 1}^{m} {F_{j} } \) is the sum of Fj with each index value full score. \( \sum\limits_{ij = 1}^{m} {f_{ij} } \) is the sum of the values of all the indexes Fij of Fi.

3 Improvement of Attribute Coordinate Comprehensive Evaluation Model

3.1 Improvement of Rating Mode on Samples

As mentioned in Sect. 2, the evaluator is required to rate the samples to reflect this psychological weight upon some indexes. The scores of 9 courses of 4 sample students with the total score of about 770 are shown in Table 1. Specifically, 9 courses include liberal-arts courses (Chinese, English, politics, history, geography) and science courses (maths, physics, chemistry, biology), and the evaluator needs to rate each student and give the psychological weight according to the scores in order to present his preference upon the students with good liberal-arts scores or science scores.

Table 1. Sample data ranking
Full size table

With so many indexes, if the 10-score system is adopted, it is difficult for the evaluator to provide the score which can accurately reflect his psychological weight according to the sample difference. Therefore, there exists the possibility of rating randomly in practice. Instead, if the evaluator only needs to qualitatively rank the evaluated object, it becomes much easier for the evaluator to do the judgement.

For example, if an evaluator prefers science, he or she may provide a reasonable ranking for the samples shown in the column “Ranking (Science Preference)” of Table 1.

The evaluator preferring liberal-arts courses may provide the reasonable ranking as shown in the column “Ranking (Liberal-arts Preference)” of Table 1.

3.2 Conversion from Ranking Value into Weight Value

After obtaining the evaluator’s qualitative evaluation upon the samples, it is necessary to convert the ranking value into weight value, namely qualitative evaluation (ranking) into quantitative value (weight), which is done by inverse qualitative mapping method [14]. Specifically, various conversion functions can be applied in the inverse qualitative mapping method. Here, we believe that the samples ranked in front are more important, so they should have larger weight when calculating barycentric coordinates. Therefore, we adopt \( y = {1 \mathord{\left/ {\vphantom {1 {\left( {nx} \right)}}} \right. \kern-0pt} {\left( {nx} \right)}} \) function (as shown in Fig. 1) to calculate the evaluator’s psychological weight. If n is the number of the samples and x is the ranking value of a certain sample, the weight y of the sample is shown in Table 2.

Fig. 1.
figure 1

Conversion function for converting ranking value into weight value

Full size image
Table 2. Conversion from ranking value to weight value (Sample volume n = 4)
Full size table

4 Simulation Experiment

In order to verify the reasonability of the improvement method, we have carried out the simulation experiment. We took the test scores of certain senior high school as the sample data, including 1,200 samples in total. We selected a small amount of samples to illustrate the local satisfactory solution, the local satisfactory solution curve and the satisfaction in order to find the difference in results before and after improvement.

4.1 Difference Between Local Satisfactory Solutions

Firstly, the original method is adopted by the evaluator to score the samples, as shown in Table 3. It is a little difficult for the evaluator to provide the exact score with so many indexes and samples to be looked through. It can be seen from the ratings that some same scores exist, and the scores are in 6 to 10 range. Such ratings are not plausible enough to present exactly the evaluator’s psychological weight.

Table 3. Sample ratings and rankings based on psychological weight
Full size table

Whereas, it is much easier for the evaluator to rank the samples (as shown in the last column in Table 3), without duplicate scores.

Table 4 shows the local satisfactory solutions before and after improvement, obtained according to Formula (1). From the result, it is obvious to reflect that the evaluator has stronger preference for science courses after improvement than before improvement.

Table 4. Local satisfactory solutions before and after improvement (Total score plane 790)
Full size table

Figure 2 illustrates the two local satisfactory solutions. The upper point represents the local satisfactory solution after improvement, and the lower one represents the local satisfactory solution before improvement. Obviously, the former solution is superior to the latter solution.

Fig. 2.
figure 2

Local satisfactory solutions before and after improvement (Total score plane 790)

Full size image

Likewise, we respectively obtain the local satisfactory solutions of total score plane 736 as shown in Table 5 and Fig. 3. The improved local satisfactory solution can better represent the evaluator’s psychological preference for science courses. In Fig. 3, the upper point is the local most satisfactory solution after improvement while the lower point is the local most satisfactory solution before improvement. Obviously, the former solution is superior to the latter solution.

Table 5. Local satisfactory solutions before and after Improvement (Total score plane 736)
Full size table
Fig. 3.
figure 3

Local satisfactory solutions before and after improvement (Total score plane 736)

Full size image

4.2 Calculation of Most Satisfactory Solution Curve

The most satisfactory solutions before and after improvement are interpolated according to Interpolation Formula (2) to obtain the mathematically most satisfactory solution curve, as shown in Fig. 4. Specifically, the full curve represents the most satisfactory line obtained by the original algorithm, and the dashedcurve represents the most satisfactory line obtained by the improved algorithm. Graphically, the improved algorithm can better represent the evaluator’s preference to science courses, so more weight is given to such courses as maths.

Fig. 4.
figure 4

Most satisfactory solution (Barycentric) curves before and after improvement

Full size image

4.3 Satisfaction Comparison

Finally, the satisfaction degree for each object before and after the improvement is illustrated according to Formula (3). On the premise that the evaluator prefers to science score, No. 115 student and No. 116 student have the same total score, and we can see No. 116 student has better science scores, however if the original method is applied, it is unreasonable that No. 115 student obtains higher satisfaction than No. 116 student (showed in the column “Satisfaction (before)” Table 6). Whereas, if the improved algorithm is applied, higher satisfaction is given to No. 116 student rather than No. 115 student (showed in the last column in Table 6), thus indicating that the improved algorithm can better present the evaluation preference and is more effective.

Table 6. Satisfaction before and after improvement
Full size table

5 Conclusion

The paper aims at researching how to simply and accurately weigh the indexes subjectively when the evaluated object has too many indexes in the attribute coordinate comprehensive evaluation method. Specifically, the quantitative rating on samples is converted into qualitative ranking, and then is converted into the evaluator’s psychological weight. The most satisfactory solution and the most satisfactory solution curve obtained thereby are superior to those obtained by the original scoring method, so the evaluation result becomes more reasonable and can better reflect the evaluator’s psychological preference and better present the advantages of the attribute coordinate comprehensive evaluation method.