1 Introduction

With the exponential proliferation of service computing, cloud computing has emerged as a new upcoming paradigm that changed the way of computing and service solution [8, 31]. The notion of cloud computing is originated from many state-of-the-art technologies and has some feature in common with other computing models. In terms of price and efficiency, cloud computing provides significant advantages compared to conventional computing models [3, 29]. According to the customer’s need, the cloud computing model offers three levels of services i.e., Software as a service (SaaS), Platform as a service (PaaS) and Infrastructure as a service (IaaS). Cloud computing allows cloud users to fulfill their IT needs with virtualized resources via the Internet rather than owning their own computing infrastructure. This is advantageous for cloud customer since they just have to pay per use instead of paying all the costs of hardware and software as in other computing paradigms. Furthermore, with respect to in-house IT infrastructure, cloud computing eliminates various administrative overheads as well as technical complexities. These superiority have forged cloud computing a great alternative for a company to manage its business. As a result, facilities provided by cloud computing have galvanized many organizations migrate their business from in-house to cloud based computing.

As a result, facilities provided by cloud computing have galvanized many leading IT organizations e.g., Microsoft, Google, Amazon, and individuals drift their business from in-house to cloud based computing. Because of this, many services are now being deployed over the cloud, and there exist several cloud service providers worldwide. Most importantly, the various cloud service providers offer similar services with different feature sets at different prices and performance levels. With all this variety and uncertainties, cloud user who decides to use the cloud services face the issue of choosing the best cloud service provider which can satisfy their diverse need. One big challenge is how to select a suitable cloud service with high efficiency and accuracy for potential service consumers based on their customized needs.

In this regard, the key factors for ranking and selecting different cloud service providers form a similar set are QoS criteria. QoS is a set of functional and non-functional characteristics [15, 22, 45]. The cloud customer uses these QoS characteristics to compare the different services available on the cloud. Cloud service selection is a typical scenario where a cloud customer or a decision-maker selects the most suitable cloud service depending on multiple conflicting QoS criteria [23, 24]. Also, the cloud service selection process involves many factors such as different QoS criteria, cloud service alternatives, interdependent relationship between criteria and the synthesis all information obtained corresponding to each criterion [4, 41]. Therefore, cloud service selection problem fits naturally into the category of multi-criteria decision making (MCDM) problem (Fig. 1) [9, 12].

Fig. 1
figure 1

A common MCDM based cloud service selection scenario

Full size image

Several research efforts have been made in the recent past with respect to cloud service selection [6, 13, 46]. They have some advantages and disadvantages in terms of their environments, applications, and restrictions. A good number of research work have used cloud broker architecture in which cloud customer contact to cloud brokers with their requirements and cloud brokers recommend a ranked list of potential cloud services [1, 9, 12, 20]. These research works utilize AHP or ANP method for calculation of QoS criterion’s weights. Although these methods are time tested but have certain limitations like low consistency of comparison, lengthy and arduous comparison system. This process may be too long and the consistency of the results is not always guaranteed. However, the low consistency and laborious comparison system increase the overall computational complexity, which is still huge problem in the cloud service selection problem.

This article discusses a new methodology to evaluates cloud service providers to deal with the aforesaid problem and examines their inconvenience in using AHP method for the cloud service selection problem. The main objective of this paper is to find a better, effective and novel cloud service selection method with consideration of the various limitations of the existing method. In this study, firstly, we propose a novel framework called Optimal Service Selection and Ranking of Cloud Computing Services (CCS-OSSR). The proposed cloud service selection framework consists four major components. Secondly, to deal with low comparison consistency, arduous comparison system of existing methods and to produce an accurate result, we introduced a hybrid MCDM methodology. First, to assign the weight to the QoS criteria, we apply the BWM (Best Worst Method). BWM is an alternative to AHP, which is the most common method for determining the weight of QoS parameters. AHP normally uses n(n − 1)/2 comparisons to find the appropriate weights of the criteria. On the other hand, BWM needs only 2n − 3 comparisons, which makes it easier to use and reduce the risk of inconsistency occurrences during pairwise comparisons [39]. Finally, we apply the TOPSIS method to rank the eligible cloud services with respect to QoS criteria. This paper has the following specific novelties:

  1. 1.

    A novel framework called CCS-OSSR is introduced to evaluate the cloud service based on customer’s preferences.

  2. 2.

    We use the Best-Worst method in conjunction with the TOPSIS method to rank the cloud service provider based on cloud customer’s preference.

  3. 3.

    We perform an experiment based on a real-world dataset which demonstrates the effectiveness and the efficiency of the proposed methodology.

  4. 4.

    To check the uncertainty and robustness of our introduced approach, we conduct the sensitivity analysis.

The remainder of this paper is organized as follows. Section 2 offers a comprehensive literature review. Section 3 explains the formulation and motivation behind the cloud service selection problem. Section 4 defines important QoS metrics for the cloud. Our proposed CCS-OSSR framework is introduced in Sect. 5. Our suggested cloud service selection methodology are presented in Sect. 6. A case study with real dataset validated the proposed methodology in Sect. 7. A comprehensive analysis presented in Sect. 8. Finally, the conclusion remark is given in Sect. 9.

2 Related work

In this section, we conduct a comprehensive survey on various MCDM methods which are used in cloud service problem. From the past few years, a good amount of work has been focused in this context [2, 13, 42, 46]. In this section, we concentrate on the notable studies that has been carried out over the previous couple of years. In paper [12], authors have introduced SMICLOUD framework for comparing and ranking three IaaS cloud services based on service measurement index (SMI) criteria of cloud services. The authors used the AHP approach for calculating the weight of SMI criteria based on user’s preferences and assess the three IaaS cloud services using calculated weights. In this study, many measurable key performance indicators (KPIs) for QoS criteria are introduced by CSMIC and various cloud service providers are compared using these KPIs. Additionally, the adoption of AHP allows to calculate attribute weights based on user preference as well as estimates interdependencies between criteria. In paper [40], author has utilized AHP method to find the best cloud database provider based on cloud customer requirements and preferences. The proposed work decomposes the MCDM problem into a hierarchy which has three main criteria and seven sub criteria. For an IaaS cloud evaluation, AHP method was used with genetic algorithm in [32]. This work proposed CloudGenius framework which evaluates and find the optimal IaaS service provider using 15 QoS criteria. AHP has been carried out in many studies in recent years to assess various SaaS cloud services [10, 21], for IaaS cloud evaluation [37, 38] and for general cloud service evaluation [17, 26]. Moreover, all these assessment methodologies are used to assist cloud users to deal with cloud service selection issues. These AHP based cloud service selection methods was commonly used Saaty’s basic 1–9 scale to support cloud customers in criteria weighting and ranking cloud service alternatives. These methods were used “Saaty basic 1–9 scale” to find criteria weight in cloud service selection problem. Another common feature of these proposed cloud service selection techniques is that a list of cloud services is generated in ascending order.

Some authors developed hybrid MCDM techniques, combining different MCDM techniques with the AHP method for measuring cloud services performance. For instance, a hybrid MCDM strategy proposed to analyze and rank distinct PaaS clouds using AHP and LSP (logic scoring of preference). Here, logic scoring of preference was used for building logical relationships between criteria for evaluation. Some other works [44] integrated AHP (to calculate criteria weights using pairwise comparison) and TOPSIS (to obtain the cloud service’s final rank) to measure the performance of cloud services. An AHP ranking solution is introduced in [49] for Cloud Service Provider(CSP) and the corresponding Service Requesting Customer(SRC) based on QoS criteria. In this study, a TRCSM framework is introduced to handle the QoS requirement of CSPs and SRCs. A trust evaluation framework is proposed in [43] with AHP and TOPSIS methods. Here, AHP is utilized to evaluates the criterion weight and the trustworthiness of cloud services are evaluated using the TOPSIS method. A compliance-based multi-dimensional model is proposed for evaluating trustworthiness of cloud service providers in [44]. In this paper, author calculates the trustworthiness of different cloud services with improved TOPSIS method. A fuzzy AHP approach was developed for evaluating various cloud services based on several QoS characteristics in [35]. Here, the Service Measurement Index model (CSMIC, 2011) was extended to build a hierarchy structure and include “reputation” as a new high standard criterion. A fuzzy user focused methodology was introduced in paper [47]. In this paper, AHP method is used to compute the QoS criteria weight and the fuzzy TOPSIS method is used to evaluate the cloud service alternatives. A cloud service assessment method has been developed with six evaluation criteria in [16], which assist cloud customers to evaluates cloud services provider. Weight of different criteria was calculated using the AHP technique and the TOPSIS method was used for evaluating the cloud services providers.

A thorough analysis of the related study shows that the rank based cloud service selection problem has emerged as an important problem. The literature review shows that various researchers have suggested a number of MCDM methods based on AHP/ANP for optimal service selection problem (Table 1). In fact, these techniques claim to be reliable and effective in dealing with these problems. However, the enormous complexity and subjectivity of suggested MCDM techniques raise questions about their suitability and practical applicability. These methods have some limitations as follows:

  • These methods are extremely complicated in terms of execution and computation with a large number of QoS criteria.

  • Inconsistency during pairwise comparison.

  • Challenging to achieve the highly consistent results.

  • Rank reversal problem.

To address the shortcomings of existing work and motivated via a number of similar studies, we have suggested a novel framework for selecting optimal cloud services from a pool of eligible cloud services. The proposed framework is called Optimal Service Selection and Ranking of Cloud Computing Services (CCS-OSSR). CCS-OSSR helps cloud customers to find the most suitable cloud service based on their QoS requirements. Furthermore, we introduce a hybrid MCDM methodology based on the BWM and TOPSIS techniques to execute the selection and ranking service of the CCS-OSSR framework.

Table 1 Identified gaps of related work based on Cloud Service Selection using MCDM method
Full size table

Our study offers not only a novel approach to select a suitable cloud service but can also overcome the shortcomings of existing research. In short, the proposed framework offers: (i) highly consistent and authentic results (ii) reduces implementation and computational complexity (iii)solve rank reversal issue.

3 Formalization and motivation

In this section, we have formalized the QoS aware cloud service selection problem in cloud and afterwards, an motivational example described the complexities of cloud service selection problem.

3.1 Cloud service selection problem formalization

Cloud service in current cloud market can be provided by different cloud service provider based on various parameter. A cloud service selection problem is formulated as quadruple (CSP, QOS, Const, W), Where

  • \(CSP=\{CSP_1, \ldots , CSP_n\}\) denotes the n candidate cloud service which could satisfies the cloud customer’s functional requirement.

  • \(QoS=\{C_1,\ldots ,C_j, \ldots , C_m\}\) indicate QoS criteria of a cloud service. There are totally m criteria and \(Q_j\) indicate the \(j_{th}\) QoS criteria of a cloud service provider.

  • \(Const= \{Const_1,\ldots ,Const_j,\ldots ,Const_m\}\) represents set of QoS constraints given by cloud customer. \(Const_j\) indicate the user’s quality constraints for \(j_{th}\) QoS criteria. For a negative QoS criteria the constraints impose a upper bound and for positive QoS criteria, the constraints impose a lower bound.

  • \(W=\{W_1,\ldots ,W_j,\ldots ,W_n\}\) be a set of weight provided by user for each QoS criteria. \(W_j\in [0,1]\) indicate the cloud customer preference(weight) for \(Q_j\) QoS criteria with \(\sum _{j=1}^{n} W_j=1.\)

3.2 Motivation

In this subsection, an example is introduced to motivate our work in practice. Suppose that an imaginary a big company, ABC which offer various services to customers beside health services. This company planning to attract more customer by improving its service efficiency. To provide various services with high efficiency, high security, customer information privacy and low maintenance costs, the company plans to transfer its services to the public cloud for the following reasons:-

  • Better cost control: By using cloud services, ABC can significantly reduce manpower, maintenance, IT and infrastructure costs.

  • Save time & effort: With cloud solutions, no technical expertise is needed for software updates, maintenance and data security. All employees are free to shift their focus from system to customers.

  • Appealing to today’s customer: In order to request different services, today’s customers expect to use their own tablets and smart phones. Customers and employees can interact with the service anytime and anywhere with a cloud - based solution.

  • Stay ahead of market trends: The cloud service helps ABC to stay ahead in the on-line market business with real- time distribution. It offers real opportunities for business growth by offering new features and services.

In addition, the ABC is a customer - based enterprise, it provides its customer with a high - quality and comprehensive sales service that is highly desirable. For implementing its services, ABC is looking for a cloud service provider. In current market there are many service provider which offers several service varying in QoS criteria. ABC compares all QoS criteria and analyzes the efficiency of each service to select a suitable cloud service from among the available ones. Typically, a cloud service has faster computing and memory operation (e.g. availability and response time) and has lower maintenance costs would be chosen. To select a service provider must be based upon the company’s specific requirement. Company explain their preferences and requirement and based on the that weight of each QoS criteria can be assigned. The best performing service provider will be chosen by the company.

In this example, to obtain high satisfaction degree of ABC it is necessary to consider their individual preferences and requirement. Since a optimal service selection without customer’s expectation is insufficient. Therefore, a service selection approach is needed that takes into account and accurately estimates ABC’s preference. In our approach, we take the customer’s preference into account and assign weight of each QoS criterion.

4 QoS criteria of cloud services

In order to evaluate the different cloud services, we need to find the QoS evaluation matrices. QoS criteria represents the functional and non-functional properties of service. In the absence of standard QoS model, it become a hurdle task to select a right option. CSMIC [11] consortium solve this problem by proposing a set of business KPI which provides a standardize method for evaluation and comparing the cloud services. QoS criteria have to be identified based on the requirements of cloud services. Some QoS criteria are essential for particular cloud services. For example, the installability criteria are more relevant for IaaS services rather than SaaS services because there is almost no customer end installation in SaaS. Additionally, some QoS criteria are evaluated by existing user experience, such as stability and suitability. Therefore, creating a list of service measurement criteria that cover all aspects is a complicated task. In order to address these problems, this work divides the QoS criteria into two classes: service dependent and customer dependent (see Table 2). Also, we classified these QoS criteria into two types: positive and negative. For positive criteria, the higher value of QoS criteria is higher the quality and for negative criteria, the higher the value of QoS criteria is lower the quality. Moreover, these QoS are also classified into quantitative or qualitative criteria. Quantitative criteria are measured by using software and qualitative criteria are inferred based on user’s experience.

Table 2 QoS criteria of Service Measurement Index
Full size table

5 Proposed framework

In this section, an Optimal Service Selection and Ranking of Cloud Computing Services (CCS-OSSR) framework is proposed which deals with cloud customer preferences. The proposed framework is a decision-making tool with facilities such as service selection and ranking, taking into account the QoS requirement of the cloud customer. In particular, it provides an output to the cloud customer as a sorted order of cloud service. Figure 2 exhibits the proposed framework. The following are the key component of the suggested framework:

  • Cloud service repository This component store various information about service providers like assets, artifacts, types of services and their features. It also stores the QoS criteria information of cloud service providers. This repository is utilized by cloud service discovery engine for matchmaking and filtering the cloud services.

  • Cloud service discovery engine Its primary role is to communicate with clients and discover a list of suitable cloud services. In general it has two key functionality: (1) cloud service matchmaker and (2) cloud service filter. The primary functionality of this component is to verify whether the cloud customer’s requirement matches with the offered cloud services based on their QoS criteria values. Further it also verifies the compatibility of the cloud services. It filters out a list of eligible cloud services from the cloud service repository after matching process.

  • Cloud service selection engine This core component has two tasks i.e., cloud service selection and QoS value conversion. With the help of our proposed methodology, this component evaluate all eligible cloud services depend upon customer QoS requirement and identifying a list of optimal cloud services using proposed MCDM methodology (see Sect. 6).

  • Cloud service prioritizing engine This engine ranks the selected cloud services by sorting them either in ascending or in descending order of their QoS values. Finally it returns this ranking to the cloud customer.

A three phase methodology has been applied

Fig. 2
figure 2

Proposed CCS-OSSR framework

Full size image

6 Proposed cloud service selection methodology

The proposed hybrid BWM-TOPSIS method for identifying, prioritizing and ranking cloud service alternatives has been introduced in this section. Figure 3 illustrates the proposed schematic diagram. The proposed schematic diagram has three important phases as follows:

  • Determine the evaluation criteria and cloud service alternatives used in service selection process

  • Prioritize the QoS criteria using the BWM method.

  • Employ TOPSIS method and determines the solution for cloud service selection problem.

Fig. 3
figure 3

Schematic diagram for phases of proposed approach

Full size image

6.1 Establishing decision matrix

Assume that a cloud customer submits its requirement to the cloud broker for selecting the most suitable cloud service. The cloud broker service finds the m eligible cloud service alternatives denoted by (\(CSA_i\)), where \(i=1,\ldots ,m\) as per the cloud customer’s requirement. Each \(CSA_i\)’s performance score is evaluated on the basis of n QoS criteria denoted by \(C_j\), where \(j=1,\ldots ,n\). Here, we developed an evaluation decision matrix \(DM= (x_{ij})_{m*n}\) of m cloud service alternatives and their n QoS criteria values for all the eligible cloud service alternatives with the same functionality. It is shown below:

$$\begin{aligned} DM=\begin{array}{cc} \begin{array}{c} CSA_1\\ CSA_2\\ \vdots \\ CSA_m \end{array}&\begin{array}{c} \begin{array}{cccc} C_1 &{}\quad C_2 &{}\quad \ldots &{}\quad C_n \end{array}\\ \begin{bmatrix} x_{11} &{}\quad x_{12} &{}\quad \ldots &{}\quad x_{1n} \\ x_{21} &{}\quad x_{22} &{}\quad \ldots &{}\quad x_{2n} \\ \ldots &{}\quad \ldots &{}\quad \ldots &{}\quad \ldots \\ x_{m1} &{}\quad x_{m2} &{}\quad \ldots &{}\quad x_{mn} \end{bmatrix} \end{array} \end{array} \end{aligned}$$
(1)

where \(x_{ij}\) denotes the value of the j th criteria of the i th cloud service alternative.

6.2 Calculate QoS criteria weight using best worst method

As stated earlier, the selection of services takes into account the preferences of the cloud customers of various QoS criteria. For each cloud service alternative there exists a n-dimensional preference vector since we have n number of QoS criterion. Each entry of this vector shows the preferences of the cloud customer with respect to each criterion. In particular, we employ BWM method to calculate the criteria weight of each QoS criterion. The Best-Worst method is one of the novel MCDM approach which is proposed by Razaei in 2015 [39]. The steps used to calculate weight of QoS criteria are:

Step 1 With the support of decision makers, chose the best and worst criteria among all criteria. The most important criteria select as best criteria (\(C_B\)) and the least important criteria select as worst criteria (\(C_W\)).

Step 2 Using a semantic standardized 9-point scale presented in Table 3, the decision maker determine the preference of the best QoS criterion over all other QoS criteria. The resultant best-to-others (BO) vector shown as below:

$$\begin{aligned} A_{B}= (a_{B1}, a_{B2},\dots ,a_{Bn}) \end{aligned}$$
(2)

Where, \(a_{Bj}\) shows the priority of most important criterion over other j th criteria. Here, \(a_{BB}=1\)

Table 3 Pairwise comparison scale for BWM preferences
Full size table

Step 3 Again, utilizing Table 3, the decision maker determine the preference of the other QoS criteria over the worst QoS criterion. The resultant other-to-worst (OW) vector shown as below:

$$\begin{aligned} A_{W}= (a_{1W}, a_{2W},\dots ,a_{nW})^T \end{aligned}$$
(3)

Where, \(a_{jW}\) shows the priority of other other j th criteria over least important QoS criterion and \(a_{WW}=1\)

Step 4 Weights of the each criterion are calculated such that the maximum absolute difference for all j are minimized for the set of \(\mid w_B - w_j*a_{Bj}\mid\), \(\mid w_j - w_W*a_{jW}\mid\). This can be formulates as

$$\begin{aligned} min max\{\mid w_B - w_j*a_{Bj}\mid \mid w_j - w_W*a_{jW}\mid \} \end{aligned}$$
(4)

s.t. \(\varSigma _jw_j = 1\) where \(w_j\) \(\ge\) 0, for all j.

$$\left| {\frac{{W_{B} }}{{W_{j} }} - a_{{Bj}} } \right| \le \xi \quad for~all~j{\text{ }}$$
(5)
$$\left| {\frac{{W_{j} }}{{W_{w} }} - a_{{jW}} } \right| \le \xi \quad for~all~j{\text{ }}$$
(6)

The optimal weight \((w_{1}^*,w_{2}^*,\dots ,w_{n}^*)\) and optimal value \(\xi\) are calculated by solving the Eqs. 5 and 6. The optimal value \(\xi\) close to zero means high consistency.

Step 5 In this step, we check the consistency of the comparison matrix similar to the AHP method to ensure overall consistency. The comparison is fully consistent when \(a_{Bj}a_{jW}=a_{BW} (j = 1, 2, \ldots , n)\). Here, \(a_{BW}\) is the preference of the best criterion over the worst criterion. The consistency ratio(CR) of obtained criteria weight determined using Eq 7.

$$\begin{aligned} CR= \frac{\xi }{CI} \end{aligned}$$
(7)

The related consistency index(CI) are shown in the Table 4. Consistency ratio closer to 0 indicates more consistent solution [39].

figure d
Table 4 Consistency Index (CI) value
Full size table

6.3 Apply TOPSIS method

In this phase, we obtained final rank the cloud service alternatives using TOPSIS method. TOPSIS is well known multi-criteria decision-making (MCDM) method and it is developed by hwang and hoon [18]. The basic idea of this method the best solution should have the shortest geometric distance from the ideal solution and the furthest geometric distance from the anti-ideal solution [25, 30, 34].

The main steps of TOPSIS approach are as follows.

Step 1 Construct a normalized decision matrix of cloud services and QoS criteria: Here, we normalize the decision matrix (DM). Since QoS criteria has difference in terms of measurement units, ranges, and their meanings, it generally causes inconsistency during comparison. Therefore, the value of each QoS criterion can be normalized using Eq. 8.

$$\begin{aligned} nx_{ij}= \frac{x_{ij}}{\sqrt{\sum _{i=1}^{m}{x^2_{ij}}}} i= 1,2, \ldots , m; j= 1,2, \ldots , n. \end{aligned}$$
(8)

Step 2 Finding the weighted normalized decision matrix: We determine the weighted normalized matrix by multiplying the normalized matrix with the combined normalized weights as shown Eq. 9:

$$\begin{aligned} v_{ij}= w_{j} * nx_{ij} \end{aligned}$$
(9)

where, \(v_{ij}\) represents the weighted normalized value of j th criterion of the i th cloud service, \(w_{j}\) represents j th criteria weight obtained using BWM method and \(nx_{ij}\) denotes the normalized criteria value.

figure e

Step 3 Find the positive (\(v_{j}^{+}\) ) and negative ideal solution (\(v_{j}^{-}\)) as:

\(v_{j}^{+}\) = max \(\{v_{1j},\) ...\(v_{mj}\}\) and \(v_{j}^{-}\) = min \(\{v_{1j},\) ...\(v_{mj}\}\) for beneficial criteria

\(v_{j}^{+}\) = min \(\{v_{1j},\) ...\(v_{mj}\}\) and \(v_{j}^{-}\) = max \(\{v_{1j},\) ...\(v_{mj}\}\) for non-beneficial criteria

Step 4 Calculate the Euclidean distance for each cloud service alternatives from \(v_{j}^{+}\) and \(v_{j}^{-}\) as :

$$\begin{aligned}&D_{i}^{+}= \sqrt{\sum _{j=1}^{n}(v_{ij}-v_{j}^{+})^2} \quad i = 1, 2, \ldots , m, \quad j = 1, 2, \ldots , n \end{aligned}$$
(10)
$$\begin{aligned}&D_{i}^{-}= \sqrt{\sum _{j=1}^{n}(v_{ij}- v_{j}^{-})^2} \quad i = 1, 2, \ldots , m, \quad j = 1, 2, \ldots , n \end{aligned}$$
(11)

where \(D_{i}^{+}\) and \(D_{i}^{-}\) denote the distance between i th cloud service to positive ideal solution and distance between i th cloud service to negative ideal solution respectively.

Step 5 Determine closeness coefficient as:

$$\begin{aligned} CC_{i}= \frac{D_{i}^-}{D_{i}^- + D_{i}^+} \end{aligned}$$
(12)

Finally, by comparing \(CC_{i}\) values, we evaluate and rank the cloud service alternatives. Obviously, the higher \(CC_{i}\) value of an cloud service alternative, the more reliable it is.

7 Case study example

The efficiency of the suggested methodology is evaluated using a real world QoS dataset in this section. Since there is no availability of any benchmark dataset on cloud service and cloud service share common realization with web service, especially in QoS properties, we have used publicly available web services QoS dataset, i.e., QWS dataset. Note that this dataset has been developed by Eyhab Al-Masri from Guelph University [5]. The QWS dataset has been widely accepted across the research community and used in evaluation studies based on QoS service selection problem [27, 33, 36]. A case study based on the QWS data set is illustrated in the following subsection.

7.1 Cloud service alternatives and criteria selection for cloud service selection

We considered 11 different cloud services with the same functionality (SMSservice) from the QWS dataset for this case study. The SMS services are evaluated based on five quality criteria, i.e., response time(c1), reliability(c2), throughput(c3), best practices(c4), and cost(c5). We consider criteria c1 and c5 are negative criteria, and others are positive criteria. For ease of description, 11 cloud services have been encoded as CSP1, CSP2, ..., CSP11. A decision matrix based on ten cloud service alternative with eight QoS criteria, is illustrated in the Table 5.

Table 5 Decision matrix for eleven Cloud services
Full size table

7.2 Apply BWM method in determining weight of criteria

At this stage, we have applied BWM method (Algorithm 1) to find the weights of eight QoS criteria. With the help of cloud customer, the most and least important criteria are chosen from all QoS criteria. The response time(\(C_1\)) is chosen as the most important criteria and the throughput(\(C_3\)) is chosen as the least important criteria. After identifying the best and worst criteria, cloud customer provide relative preference for the most important(best) to other criteria (\(C_1, C_2, C_3, C_4, C_5\)) and similarly other criteria over the worst criteria on a scale 1 to 9, as shown in Table 6. We calculated each criterion weight using Eqs. 5 and 6. The calculated weights of each criterion are \(C_1 =0.475, C_2 =0.123, C_3 =0.057, C_4 =0.098, C_5 =0.246\), and \(\varepsilon =0.016\). The consistency ratio is calculated with the help of Eq. 7 and get CR=0.01.

Table 6 Best-to-others (BO) and others-to-worst (OW) pairwise comparison
Full size table

7.3 Apply TOPSIS method and get the final rank

Here, TOPSIS method is applied to rank of each cloud service alternatives. In particular, we use vector normalization technique to eliminate the inconsistency among QoS criteria and normalize the decision matrix using Eq 8. The normalized matrix is shown in Table 7. Next, we obtained the weighted normalized decision matrix by applying the BWM weight of each QoS criterion in Eq. 9. In next step, we identified the best and the worst service solutions i.e., calculate the positive ideal solution (PIS) and the negative ideal solution (NIS). In further step, we calculated the distance of each cloud service alternative from the PIS and NIS using Eqs. 10 and 11. According to the Eq. 12, we get the relative closeness degree (\(CC_i\)) to the ideal solution of each service alternatives. Finally, Table 8 shows the rank of each cloud service alternatives according to \(CC_i\) values. Based on the values of \(CC_i\) values, we concluded that the \(CSA_2\) is the best cloud service alternative among all the other alternatives.

Table 7 Normalized decision matrix
Full size table
Table 8 The overall result
Full size table

8 Comprehensive analysis

In this section, we conduct an extensive analysis to analyze the performance of proposed methodology considering a rank conformance analysis, comparative analysis, computational cost,Time Complexity, and adequacy under changes in alternatives.

8.1 Rank conformance analysis

To analyze the conformity of the proposed methodology with other current MCDM techniques, we compared the ranks acquired using our suggested methodology with other cloud service selection methodology such as AHP [12] and AHP-TOPSIS [43]. We considered the same datasets used in our case studies for this comparison. After the comparison, we have noted that both techniques have matched the efficiency with the suggested methodology. A close similarity between the ranks of the proposed methodology and AHP-TOPSIS methodology has been observed. For example, as shown in Fig. 4, all methods shows full consensus on \(A_2\) is the best cloud service alternative. Similarly, as shown in Fig. 4, 100% of considered methods agree on 2nd, 3rd, 4th, 5th, 7th, 8th and 9th rank. The results show that the proposed methodology’s performance is consistent with other MCDM methods.

Fig. 4
figure 4

Ranking of cloud service alternatives with different methods

Full size image

8.2 Comparative analysis

In this subsection, we compare BWM and AHP for determining the degree to which the results improve concerning the computation of criteria weight. We employ the same QoS data set as provided in our case study to make a distinction between AHP and BWM. In the cloud service selection problem, many current approaches use AHP for pairwise comparison. In contrast, our study utilized the BWM method for the same, which outperforms AHP. The reason behind that BWM needs only two comparison vectors, while AHP needs a whole matrix of comparison number [39]. In case of BWM, we only required \(2n-3\) comparisons, while in AHP we needed \(n(n-1)/2\) comparisons (See Fig. 5).

Fig. 5
figure 5

Pairwise comparisons in BWM and AHP

Full size image

Consistency ratio (CR) is commonly used to measure the reliability of the produced results of an MCDM method. For comparison, we carried out an experiment to compute and measure the consistency ratio from both BWM and AHP method. To achieve this goal, we performed 20 different comparisons and determined QoS criteria weight using the BWM and AHP method. Using this comparison(See Fig. 6), we can conclude that in terms of the consistency ratio, BWM outperforms AHP. This comparative analysis also shows that BWM is more consistent, authentic and reliable than AHP for the cloud service selection problem.

Fig. 6
figure 6

Consistency ratio analysis between BWM and AHP method

Full size image

8.3 Sensitivity analysis

In order to determine the evaluation efficiency, it is necessary to examine how the ranking of cloud service provider may change under different weight values. Sensitivity analysis is calculated to investigate the effect of QoS criteria weights in the final ranking result of different CSPs and checked the robustness of the ranking solution. In this case, we varied the degree of QoS attribute weights to analyze the changes in the final results. If the ranking of cloud service providers in most the of cases remains unchanged, the result is called sensitive, otherwise it is robust. In this experiment, we exchange the weight of each criterion with other selection criterion weight one at a time. We conducted 10 experiments in order to analyzed the effect of weights in selecting optimal cloud service provider. The values of \(CC_i\) are obtained for each experiment and each calculation is given a different name. For instance, C1–C4 denotes the exchange of the weight of criteria C1 and C4. Details of the sensitivity assessment test are shown in Table 9 and Fig. 7. Figure 8 shows that CSA2 is the best cloud service provider in 10 experiments out of 10 experiments and CSA3 is second best solution. Finally, based on experimental result, we can say that our proposed decision making methodology is robust and relatively insensitive to the criteria weights.

Fig. 7
figure 7

Result of the CCi

Full size image
Fig. 8
figure 8

Result of the sensitivity analysis

Full size image
Table 9 Weight sensitivity analysis
Full size table

8.4 Time complexity

The proposed methodology consists of three phases. Let us assume that n evaluation criteria and m cloud services alternatives have participated in evaluating a service. Unlike previous studies, we used the BWM method in phase 2 of the proposed methodology to calculate the QoS criteria weight. In terms of calculating QoS criteria weight, BWM significantly reduces the implementation and computational complexities of the suggested solution in this phase. The time complexity of calculating user-assigned weight using the AHP method is \(O(n^{3})\), i.e., the complexity to identify the maximum eigenvalue and the corresponding eigenvector. On the other hand, the BWM method need only O(n) to calculate weight. 3rd phase uses TOPSIS method to evaluates and finds the final rank of services against QoS criteria. In this phase, we established a decision matrix against each QoS criteria and normalized the decision matrix using linear scale transformation. Then, calculated QoS criteria weight using the TOPSIS method is multiplied with normalized decision matrix, and the time complexity is \(O(n^{2}m)\). Therefore, the overall time complexity of finding and rating the top-K cloud service in worst case is \(O(n+ n^{2}m)\). Contrariwise, the final time complexity to finding and rating the top-K cloud service using AHP and AHP-TOPSIS is \(O(n^{3}+ nm)\) and \(O(n^{3}+ n^{2}m)\) respectively. Thus, we can say that our suggested approach takes less time compare to other.

8.5 Suitability under changes in alternatives

To evaluate different cloud services appropriately, a decision-maker can require that certain requirements or alternatives be included or excluded. Selecting suitable cloud services requires a consistent selection criteria preferences. We performed several experiments to test the suitability of the proposed solution to changes in available alternatives. We used our datasets (Table 5) to carry out these experiments. We carried out experiments for this dataset by adding other alternatives and removing/reducing the number of decisions criteria. It is important to note that all decision criteria have been assigned equal weight. We observed for every experiment the effect of alternatives change on the final cloud service ranks. As a consequence of these experiments, we found that the proposed methodology is realistic and logical for cloud service selection problem.

9 Concluding remarks

9.1 Discussion and conclusion

Cloud computing has gained tremendous popularity as a promising Internet technology in the last few years due to its financial and technical superiority over conventional computing models. With cloud computing, users can acquire computer resources when they need them on a pay-as-you-go business model. Because of this, many services are now being deployed over the cloud, and there exist several cloud service providers worldwide. Most importantly, the various cloud service providers offer similar services with different feature sets at different prices and performance levels. With all this variety and uncertainties, cloud user who decides to use the cloud services face the issue of choosing the best cloud service provider which can satisfy their diverse need. The cloud service selection is currently being received close attention because of the growing demand and commercially availability of cloud service.

In recent years, a lot of research work has been introduced to the cloud services selection problem. This paper focuses on consumer preferences in order to always find the best service. Through this research work, we tried to obtain a high degree of satisfaction for cloud customers. This Paper develops a framework where a cloud user can find the best service based on their preferences. In this paper, three-phase methodology is proposed based on BWM–TOPSIS method for the selection of the optimal cloud service.

  • In the first phase, determine the evaluation criteria and cloud service alternatives used in service selection process.

  • In the second phase, calculate the weight of evaluation criteria using the BWM method.

  • In the third phase, using the TOPSIS technique evaluate the final rank of cloud service alternatives.

We designed an algorithm that can be applied to an actual cloud service selection problem. With the proposed algorithms, cloud customer can determine the weights according to their quality preference easily and ranks various cloud service alternatives based on their performance. The feasibility of the proposed methodology is confirmed via the real QoS data set. Besides, we test the proposed methods to verify its robustness and consistency in terms of sensitivity analysis.

9.2 Limitation and scope of future work

Since this study was carried out in-depth, but like other research, it has some limitations. The major limitation of this study is that it is unable to handle frequent and continuous changes in cloud customer requirements. Another important limitation of this study is that we did not consider interrelationship among the criteria. However, we also believe that this work opens a number of research opportunities. Some of the potential research directions where our study can be extended are discussed below:

  • The proposed framework can also be extended for non-quantifiable QoS criteria and interrelated criteria.

  • There is also the possibility to extend our ranking algorithm to cope with variation in QoS criteria by utilizing uncertainty as a fuzzy set and stochastic programming.

  • Apply this BWM with other MCDM methods in the cloud service selection problem and compare the results.