1 Introduction

Using a service-oriented networked manufacturing approach, cloud manufacturing [1] has developed into a new mode of service manufacturing and smart manufacturing [2]. Its manufacturing services are both highly integrated and highly modular, which offer new opportunities in the development of the related industries.

Over the whole life cycle of cloud manufacturing activities, the cloud manufacturing platform brings together a large number of manufacturing services resources [3]. By breaking down the manufacturing tasks, analyzing requirements and matching functional requirements, these integrated manufacturing services resources can be divided into a number of different independent service portfolios, and the “centralized use of decentralized resources, decentralized service of centralized resources” is realized [4]. Due to the dynamic, diverse and discrete nature of the manufacturing services, the service capability in cloud manufacturing is faced with high requirements. The comprehensive evaluation of manufacturing services can not only enhance the competitiveness of different services, but can also concentrate advantageous resources, reduce production costs and achieve additional advantages for cloud manufacturing service resources. It is therefore very important to evaluate cloud manufacturing services under networked manufacturing resources

Several research achievements have been made in cloud manufacturing service evaluation. In terms of cloud service evaluation index systems, Li et al. evaluate the reputation of a cloud service [5], and put forward six kinds of evaluation indices, including time control, economic efficiency, processing quality, service attitude, business scale, and logistics efficiency. Li and Bardi propose four kinds of second-level indices for cloud service security, including control measures and so on, and 16 kinds of third-level indices, including security management and so on [6]. In the study of Yan et al. a credit evaluation index [7] is divided into three categories, including 16 kinds of second-level indices involving time and quality, etc. In terms of evaluation methods for services, in order to meet the needs of the users, Lartigau and Xu propose a quality evaluation model for cloud manufacturing services based on an artificial honey bee algorithm [8]. Strunk uses a genetic algorithm to evaluate service quality, and optimal service is obtained [9]. In order to make use of cloud service resources effectively, a cloud resource evaluation model based on entropy optimization and ant colony clustering is proposed [10]. Taking into consideration the complexity of service quality, Setiawan and Sarno propose a service evaluation model based on the Fuzzy Analytic Hierarchy Process (FAHP) [11, 12] and the Technique for Order Preference by Similarity to an Ideal Solution (TOPSIS) [13, 14].

It can be seen from the above works that the establishment of an evaluation index system is the basis for and key to the evaluation of cloud manufacturing services. However, in the current comprehensive evaluation index system of cloud manufacturing services, the evaluation indices are different for each category, some are qualitative, some are quantitative, and have characteristics of diversity, hierarchy and fuzziness; in addition, there are not only qualitative and quantitative indices, but also positive and inverse indices. Furthermore, the evaluation indices proposed by different researchers are very different for the evaluation of the same cloud service, and are not universal and extensible. Cloud manufacturing services have fuzziness and complexity, and although many methods have been combined to implement a comprehensive evaluation, it is difficult to obtain the reasonable evaluation results. It is therefore very important to establish a general, flexible and comprehensive evaluation model of cloud manufacturing services.

Cloud manufacturing service evaluation involves a large number of complex factors. Fuzzy mathematics [15] has the advantages of both qualitative and quantitative evaluation, and can quantify certain factors that have unclear boundaries and are not easy to quantify. It can intuitively reflect the membership degree of an evaluation index, and can therefore be more objective and accurate in reflecting the true situation. In view of the above, and combination with the theory of fuzzy mathematics, this paper establishes a model of fuzzy comprehensive evaluation [16]. The evaluation model is verified using several examples, and has a certain reference value for further selection of cloud manufacturing services.

2 The Multi-level Fuzzy Comprehensive Evaluation Model

Fuzzy comprehensive evaluation method is based on fuzzy mathematics. This method takes a fuzzy object and fuzzy concept as a fuzzy set; it establishes the appropriate fitness function [17] and through the relevant operations of fuzzy set theory [18, 19] carries out a quantitative analysis of the fuzzy object. This fuzzy comprehensive evaluation model can give a comprehensive summary of the opinions of each evaluated subject and can reflect the quality of the evaluated objects. The flow chart of multi-level fuzzy comprehensive evaluation model of cloud manufacturing service established in this paper is shown in Fig. 1.

Fig. 1
figure 1

Flow chart of multi-level fuzzy comprehensive evaluation model for cloud manufacturing service

Full size image

2.1 Establishing the Multi-level Comprehensive Evaluation Index System

A cloud manufacturing service is a complex service composition system. It is very important to establish a scientific, reasonable and reliable evaluation index system. Since there are many factors involved, this paper focuses on representativeness and comprehensiveness in order to select an effective evaluation index. At the same time, the evaluation system can be increased or reduced according to the real-world situation. Based on the above principles, combined with the characteristics of cloud manufacturing services, this paper establishes three types of first-level evaluation indices: a manufacturing service capability evaluation index, a service quality evaluation index and a service transaction security evaluation index. Table 1 presents the comprehensive cloud manufacturing services evaluation indices.

Table 1 Comprehensive cloud manufacturing services evaluation indices
Full size table

2.2 Establishment of the Multi-level Evaluation Factor Set

As shown in Table 1, the comprehensive cloud manufacturing service evaluation is a three-level index evaluation system. The evaluation of the first level indices, manufacturing service capability, service quality and service transaction security, is a fuzzy evaluation process. In other words, the third-level indices corresponding to the first level are first comprehensively evaluated; then, the second-level indices are evaluated; and finally, the first-level indices are evaluated. Thus the layer-by-layer evaluation can obtain the final evaluation results. Multi-level evaluation factor structure is shown as Fig. 2.

Fig. 2
figure 2

Multi-level evaluation factor structure chart

Full size image

The resulting multi-level evaluation factor sets are as follows:

  1. (1)

    First-level evaluation factor set:

    $$E = \left\{ {C_{1} , \, Q_{1} , \, S_{1} } \right\}$$
  2. (2)

    Second level evaluation factor set:

    $$C_{1} = \left\{ {C_{21} , \, S_{22} , \, C_{23} , \, C_{24} } \right\},\quad Q1 = \left\{ {Q_{21} , \, Q_{22} , \, Q_{23} } \right\},\quad S_{1} = \left\{ {S_{21} , \, S_{22} , \, S_{23} } \right\}$$
  3. (3)

    Third-level evaluation factor set:

    $$C_{21} = \left\{ {C_{231} , \, C_{232} , \, C_{233} , \, C_{234} , \, C_{235} } \right\},\quad C_{22} = \left\{ {C_{236} , \, C_{237} , \, C_{238} } \right\}$$
    $$\begin{aligned} C_{23} & = \left\{ {C_{239} , \, C_{2310} , \, C_{2311} , \, C_{2312} ,C_{2313} , \, C_{2314} } \right\}, \\ C_{24} & = \left\{ {C_{2315} , \, C_{2316} , \, C_{2317} , \, C_{2318} , \, C_{2319} , \, C_{2320} , \, C_{2321} } \right\}, \\ Q_{21} & = \left\{ {Q_{231} , \, Q_{232} , \, Q_{233} } \right\},\quad Q_{22} = \left\{ {Q_{234} , \, Q_{235} , \, Q_{236} } \right\}, \\ Q_{23} & = \left\{ {Q_{237} , \, Q_{238} , \, Q_{239} , \, Q_{2310} , \, Q_{2311} , \, Q_{2312} } \right\}, \\ S_{21} & = \left\{ {S_{231} , \, S_{232} , \, S_{233} , \, S_{234} , \, S_{235} ,S_{236} } \right\} \\ S_{22} & = \left\{ {S_{237} , \, S_{238} } \right\},\quad S_{23} = \left\{ {S_{239} , \, S_{2310} , \, S_{2311} , \, S_{2312} , \, S_{2313} , \, S_{2314} } \right\} \\ \end{aligned}$$

2.3 Establishment of Evaluation Criteria for the Third-level Indices

Since the value range of the third-level evaluation indices is different, and the types of indices are different, a unified standard is needed. Based on the actual situation and the relevant data, the evaluation standards for indexes can be divided into five grades: “excellent”, “good”, “medium”, “qualified” and “poor”. Within these, there are 33 criteria for the qualitative evaluation of cloud manufacturing services, and 14 criteria for quantitative evaluation, as shown in Table 2.

Table 2 Standard value of cloud manufacturing service evaluation index
Full size table

2.4 Establishment of Multi-level Fuzzy Relation Matrix R

According to Tables 1 and 2, the indices first undergo fuzzy processing, and the evaluation indices are then quantified for each factor individually; in this way, the membership degree between the evaluation index and its corresponding fuzzy subset is determined. Based on this idea, this paper presents a fuzzy trapezoidal distribution function to calculate the membership degree of each evaluation index. This function is simpler and more practical than the triangular membership function. For some evaluation indices, a higher value represents a better result, and for others the opposite is true, in view of this, for the first type of index that, a fuzzy ascending trapezoidal equation is adopted, as shown in Eqs. (1) to (5), and for the second type, a fuzzy descending trapezoidal equation is adopted, as shown in Eqs. (6) to (10).

  • Fuzzy ascending trapezoid equation

    $$r_{1} = \left\{ \begin{gathered} {0}\quad x_{i} \le v_{{2}} \hfill \\ \frac{{x_{i} - v_{2} }}{{v_{1} - v_{2} }}\quad v_{2} < x_{i} < v_{1} \hfill \\ 1\quad x_{i} \ge v_{1} \hfill \\ \end{gathered} \right\}$$
    (1)
    $$r_{2} = \left\{ \begin{gathered} {0}\quad x_{i} \ge v_{1} {\kern 1pt} {\kern 1pt} or{\kern 1pt} {\kern 1pt} x_{i} \le v_{3} \hfill \\ \frac{{x_{i} - v_{3} }}{{v_{2} - v_{3} }}\quad v_{3} < x_{i} < v_{2} \hfill \\ 1 - r_{i} \quad v_{2} \le x_{i} < v_{1} \hfill \\ \end{gathered} \right\}$$
    (2)
    $$r_{3} = \left\{ \begin{gathered} 0\quad x_{i} \ge v_{2} {\text{ or }}x_{i} \le v_{4} \hfill \\ \frac{{x_{i} - v_{4} }}{{v_{3} - v_{4} }}\quad v_{4} < x_{i} < v_{3} \hfill \\ 1 - r_{2} \quad v_{3} \le x_{i} < v_{2} \hfill \\ \end{gathered} \right\}$$
    (3)
    $$r_{4} = \left\{ \begin{gathered} 0\quad x_{i} \ge v_{3} {\text{ or }}x_{i} \le v_{5} \hfill \\ \frac{{x_{i} - v_{5} }}{{v_{4} - v_{5} }}\quad v_{5} < x_{i} < v_{4} \hfill \\ \;1 - r_{3} \quad v_{4} \le x_{i} < v_{3} \hfill \\ \end{gathered} \right\}$$
    (4)
    $$r_{5} = \left\{ \begin{gathered} 0\quad x_{i} \ge v_{4} \hfill \\ 1 - r_{4} \quad v_{5} < x_{i} < v_{4} \hfill \\ \;1\quad x_{i} \le v_{5} \hfill \\ \end{gathered} \right\}$$
    (5)

  • Fuzzy descending trapezoid equation

    $$r_{1} = \left\{ \begin{gathered} 1\quad x_{i} \le v_{1} \hfill \\ \frac{{v_{2} - x_{i} }}{{v_{2} - v_{1} }}\quad v_{1} < x_{i} < v_{2} \hfill \\ 0\quad x_{i} \ge v_{2} \hfill \\ \end{gathered} \right\}$$
    (6)
    $$r_{2} = \left\{ \begin{gathered} 1 - r_{1} \quad v_{1} < x_{i} \le v_{2} \hfill \\ \frac{{v_{3} - x_{i} }}{{v_{3} - v_{2} }}\quad v_{2} < x_{i} < v_{3} \hfill \\ 0\quad x_{i} \le v_{1} \;or\;\;x_{i} \ge v_{3} \hfill \\ \end{gathered} \right\}$$
    (7)
    $$r_{3} = \left\{ \begin{gathered} 1 - r_{2} \quad v_{2} < x_{i} \le v_{3} \hfill \\ \frac{{v_{4} - x_{i} }}{{v_{4} - v_{3} }}\quad v_{3} < x_{i} < v_{4} \hfill \\ 0\quad x_{i} \le v_{2} \;or\;x_{i} \ge v_{4} \hfill \\ \end{gathered} \right\}$$
    (8)
    $$r_{4} = \left\{ \begin{gathered} 1 - r_{3} \quad v_{3} < x_{i} \le v_{4} \hfill \\ \frac{{v_{5} - x_{i} }}{{v_{5} - v_{4} }}\quad v_{4} < x_{i} < v_{5} \hfill \\ 0\quad x_{i} \le v_{3} \;or\;x_{i} \ge v_{5} \hfill \\ \end{gathered} \right\}$$
    (9)
    $$r_{5} = \left\{ \begin{gathered} 1\quad \in x_{i} \ge v_{5} \hfill \\ 1 - r_{4} \quad v_{4} < x_{i} < v_{5} \hfill \\ 0\quad x_{i} \le v_{4} \hfill \\ \end{gathered} \right\}$$
    (10)

where V1, V2, V3, V4 and V5 are the corresponding standard values of excellent, good, medium, qualified and poor, respectively; xi is the actual value of the evaluation factor set corresponding to the index; and r1, r2, r3, r4 and r5 are the membership values of the five grades after the fuzzy operation, where \(\alpha_{i} = \frac{{w_{i} \beta_{i} }}{{\sum\nolimits_{i = 1}^{n} {w_{i} \beta_{i} } }}\).

The membership values of each evaluation index are derived from Eqs. (1) to (5) and (6) to (10); based on these, the fuzzy membership degree subset Ri can be structured, and the fuzzy matrix or the evaluation matrix is then obtained as follows:

$$\alpha_{i} = \frac{{w_{i} \beta_{i} }}{{\sum\nolimits_{i = 1}^{n} {w_{i} \beta_{i} } }}$$
(11)

2.5 Weights of Evaluation Indices

2.5.1 Comprehensive Weight of the Third-Level Evaluation Index

The analytic hierarchy process (AHP) is a method combining both qualitative and quantitative methods. It determines the weights of evaluation indices, which contain the knowledge and experience of decision makers and expert opinion. The entropy weight method is an objective method for determining the weight of an evaluation index. The entropy weight method fully reflects the information contained in the original data; however, although the results are objective, they cannot reflect expert knowledge and experience and the views of decision makers. Sometimes, the weights obtained by the entropy weight method may be inconsistent with the actual importance of the index and may deviate from this. By comprehensively analyzing the advantages and disadvantages of the two methods, this paper combines the results of both the AHP and entropy weight methods to obtain the final weight values.

Since the first- and second-level evaluation indices have no corresponding actual value, the weight value is determined by AHP. The third-level evaluation index has an actual value, so its actual weight can be obtained using entropy weight method, and the subjective weight then obtained by AHP. In this way, through a combination of subjective and objective approaches, the comprehensive weights of the third-level evaluation indices can be obtained, as shown in Eq. (12).

$$\alpha_{i} = \frac{{w_{i} \beta_{i} }}{{\sum\nolimits_{i = 1}^{n} {w_{i} \beta_{i} } }}$$
(12)

where wi is the weight obtained by AHP, βi is the weight obtained by the entropy weight method, αi is the comprehensive weight.

2.5.2 Multi-level Fuzzy Comprehensive Evaluation Result Matrix F

In practice, the most commonly used method is the maximum membership principle; however, the disadvantage of this method is the high loss of information in some cases, which may lead to unreasonable evaluation results. A method of finding the membership grade by means of a weighted average is therefore proposed; a number of items to be evaluated can be sequenced according to their grade positions. In this paper, the weighted average model is adopted for the fuzzy comprehensive evaluation. The comprehensive weight i and the fuzzy relation matrix R are combined for all levels of the evaluated items, and the fuzzy comprehensive evaluation result vector F of the evaluated items can then be obtained for all levels, as shown in Eq. (13).

$$F = \alpha *R = \left\{ {\alpha_{1} ,\alpha_{2} ,...,\alpha_{n} } \right\}*\left[ {\begin{array}{*{20}c} {r_{11} } & {r_{12} } & {...} & {r_{1m} } \\ {r_{21} } & {r_{22} } & {...} & {r_{2m} } \\ {...} & {...} & {...} & {...} \\ {r_{n1} } & {r_{n2} } & {...} & {r_{nm} } \\ \end{array} } \right]$$
(13)

where i is the comprehensive weight, R is the fuzzy relation matrix, and F is the comprehensive fuzzy evaluation result vector.

3 Classifying the Evaluation Results

Since this approach uses a three-level comprehensive evaluation index system, according to the steps of the multi-level fuzzy comprehensive evaluation model the first-level indices are evaluated first; then, the second-level indices are evaluated; and the third-level indices are evaluated last. From this multi-level evaluation, the final evaluation result is achieved. In addition, cloud manufacturing services are diverse, and different service requirements may place different importance on the same indices; thus, it is necessary to bring the comprehensive evaluation model more into line with the actual situation.

For the first-level evaluation index, three kinds of weights are obtained by AHP; first is WS > WQ > WP, the second is WP > WQ > WS, and the third WQ > WP > WS, where WS is the weight value of the manufacturing service capability, WQ is the weight value of the service quality, and WP is the weight value of the service transaction guarantee.

Based on the above situation, the standard matrix of evaluation results is established, N = [100,90,80,65,55], corresponding to “excellent”, “good”, “medium”, “qualified” and “poor”. The total evaluation result Z is obtained from Eq. (14). Finally, based on the value of the result Z, the grade classification in Table 3 can be obtained.

$$Z = F * N$$
(14)

where: N is the evaluation result standard matrix, F is the fuzzy comprehensive evaluation result vector, Z is the total evaluation result value.

Table 3 Grade classification
Full size table

4 Case Study

In order to verify the effectiveness of the fuzzy comprehensive evaluation model, we select 30 groups of cloud manufacturing services as an example for evaluation. The evaluation results are shown in the following table.

(1) First type of weight: WS > WQ > WP.

where WS = 0.5714, WQ = 0.2857, WP = 0.1429, obtained by AHP.

As shown in Table 4, the cloud manufacturing services with an “excellent” grade are 1, 3, 5, 6, 7, 8, 9, 10, 13, 14, 16, 19 and 22. Those with a “good” grade are 2, 4, 12, 15, 17, 18, 20 and 21; those with a “medium” grade are 23, 24, 25 and 28; those with a “qualified” grade are 26, 27 and 29; and the only one with a “poor” grade is 30.

Table 4 Evaluation results for 30 cloud manufacturing services
Full size table

(2) Second type of weight: WP > WQ > WS.

where WS = 0.1429, WQ = 0.2857, WP = 0.5714, obtained by AHP.

As shown in Table 5, the cloud manufacturing services with an “excellent” grade are 2, 5, 6, 7, 8, 13 and 19. Those with a “good” grade are 1, 3, 4, 9, 10, 11, 12, 14, 15, 16, 17, 18, 20, 21, 22, 23 and 24; those with a “medium” grade are 25, 28 and 29; those with a “qualified” grade are 26, 27; and the only one with a “poor” grade is 30.

Table 5 Evaluation results for 30 cloud manufacturing services
Full size table

(3) Third type of weight: WQ > WP > WS.

where WS = 0.1429, WQ = 0.5714, WP = 0.2857, obtained by AHP.

As shown in Table 6, the cloud manufacturing services with an “excellent” grade are 2, 4, 5, 6, 7, 8, 13, and 19. Those with a “good” grade are 1, 3, 10, 11, 12, 14, 15, 16, 17, 18, 20, 21, 22, 23 and 24; those with a “medium” grade are 25, 26, 27, 28 and 29; and the only one with a “poor” grade is 30.

Table 6 Evaluation results for 30 cloud manufacturing services
Full size table

Tables 4, 5 and 6 show the number of cloud manufacturing services corresponding to these three weights, as shown in Fig. 3 For the first type of weight, the numbers of excellent, good, medium, qualified and poor cloud manufacturing services are 13, 9, 4, 3 and 1, respectively. For the second type of weight, the numbers are 7, 17, 3, 2 and 1, respectively. For the third type of weight, the numbers are 9, 15, 5, 0 and 1, respectively.

Fig. 3
figure 3

Numbers of cloud manufacturing services corresponding to three different weights

Full size image

From the above, it can be seen that the comprehensive fuzzy evaluation grade classification model can classify the cloud manufacturing services and achieve the desired purpose more effectively; thus, the needs of different service users can be more closely met.

5 Conclusions

This paper proposes a hierarchical evaluation model using AHP, entropy weight and fuzzy comprehensive evaluation, based on the theory of fuzzy mathematics and the characteristics of the cloud manufacturing service. Using 30 examples, the evaluation model is shown to be both reasonable and effective. The main results are as follows:

  1. (1)

    According to the characteristics of the cloud manufacturing service, a representative three-level evaluation index system is constructed. At the same time, this evaluation system can be modified according to the actual needs of the service users.

  2. (2)

    The evaluation index and fuzzy membership function of the cloud manufacturing service can be used to analyze the qualitative and quantitative indexes.

  3. (3)

    The use of the AHP and entropy methods to obtain the comprehensive evaluation index weight value means that the result of the comprehensive fuzzy evaluation is more scientific and reliable.

In summary, the comprehensive fuzzy evaluation hierarchy classification model has the following characteristics: a reasonable evaluation index weight; clear evaluation steps; simple evaluation rules, quantitative indices and easy data processing; and very strong versatility. This model can therefore be applied to a variety of cloud manufacturing services, and can easily evaluate the key problems of different evaluation objects. It can therefore provide a more valuable reference for users, which is of great significance in the further improvement of cloud manufacturing services.

In our future work, we will study a variety of intelligent service evaluation theories and methods. Combined with a variety of evaluation methods of intelligent optimization theory, the evaluation has the characteristics of intelligence, autonomous learning, high fault tolerance and high security, so it can freely deal with complex and changeable cloud services.