Introduction and Related Studies

Water is one of the essential natural resources for all living things to survive. Water is also an indispensable needs in terms of human use, industries, energy production, economy, and in short, almost every aspect of society. However, only 3% of the world’s water reserves are freshwater, and 70% of this freshwater is known to be unusable and available in glaciers, soil, atmosphere, and groundwater (Akın and Akın 2007). This situation indicates that people will experience water shortages with increasing population and industrialization, and scarcity of available water. Current economic conditions allow many investments required by urbanization. Demand for water used in many fields such as agriculture, industry, food processing, environmental services, energy production, and domestic use is increasing. But at the same time, water resources are rapidly decreasing around the world. The freshwater sources among these resources have decreased and reached to the point of exhaustion. Wastewater that is discharged to the environment without treatment causes pollution of decreasing water resources, posing further threat. For this reason, wastewater must be treated in order not to pollute the existing water resources and to use water resources more efficiently.

Ensuring effective delivery of water to end users is increasingly important, due to many factors such as insufficient freshwater resources and increasing population. Also, keeping water sources clean and reusing wastewater is vital throughout the cycle of water recovery and use. Turkish Ministry of Labour, Social Services and Family summarized the reasons for the reuse of wastewater as follows (Özcan 2014): (1) To allocate limited natural water resources as a drinking water; (2) To prevent environmental pollution problems, which may arise due to wastewater discharges; (3) Very high levels of pure water in domestic wastewater; (4) Insufficient water resources in cities due to increasing in-city population.

The water treated in wastewater treatment plants (WWTPs) can be used in agriculture, industry, fire fighting, and homes with the improvements in treatment technology. These important investments were made in Turkey starting in 1994, when the ratio of the number of municipalities served by WWTPs to the total number of municipalities in Turkey was 3%, which increased to 42% by 2016 (Turkish Statistical Institute 2018). So, decision-makers (DMs) should measure the effectiveness of WWTPs in almost every city to and compare their performance to develop strategies about WWTPs.

Literature Review

The problem of WWTPs performance evaluation is an area that has been handled by many different researchers for a long time and has not become outdated. Balkema et al. (2002) propose a multi-objective optimization model for selection of sustainable wastewater treatment systems. Hernández-Sancho and Sala-Garrido (2009) focus on the efficiency of WWTPs. Data Envelopment Analysis (DEA) is used to evaluate 338 different plants located in Valencia, Spain. Hernández-Sancho and Sala-Garrido (2009) determine that costs are the most important factors to explain the differences between WWTPs. Karagiannidis and Perkoulidis (2009) apply the elimination and choice of expressing reality (Electre) method for comparing and ranking alternative anaerobic digestion technologies, which can be used in WWTPs. Gaulke et al. (2010) determine the criteria to evaluate the implementation and sustainability of the wastewater treatment system. Hernández-Sancho et al. (2011) use nonradial DEA to calculate the efficiency of WWTPs in Spain, according to their energy consumption amounts. Bottero et al. (2011) employ analytic hierarchy process (AHP) and analytic network process (ANP) to select the best of sustainable wastewater treatment technologies. Sala-Garrido et al. (2011) use DEA to calculate the techno-economic efficiencies of WWTPs. Ilangkumaran et al. (2014) use AHP and gray rational analysis for the wastewater treatment planning problem. Lorenzo-Toja et al. (2015) propose a combined methodology consisting of life cycle assessment and DEA to determine the efficiency of WWTPs considering environmental gains. Longo et al. (2016) focus on energy consumptions of WWTPs. They define three different key performance indicators according to the literature review to evaluate the efficiency of WWTPs. The technology, layout, and country of location are determined as important factors for energy consumption. Castellet and Molinos-Senante (2016) extend DEA by a weighted slacks-based measure model to evaluate the efficiency of WWTPs. The potential economic savings are determined to develop strategies about WWTPs. Ang et al. (2019) develop an optimization model for wastewater plants considering disposal and reuse alternatives. Their study examines the variability of water quality and quantity. Ghaderi et al. (2019) use mixed integer programming technique to determine the best location and system for the WWTPs, which is used for subway stations. Five different stations are evaluated, and the best one is specified for Tehran subway line 2. Gomes et al. (2020) evaluate four different waste materials as adjuvants for two different temperatures. Environmental, technical, and economic criteria are taken into account to evaluate waste materials. Jiang et al. (2020) employ DEA to make a comprehensive analysis of the efficiencies of WWTPs. Operation cost, electricity consumption, and labor are determined as input variables and four different chemical variables are determined as outputs one of which is determined as undesirable output. While chemical oxygen demand removal rate, reclaimed water yield and ammonia nitrogen (NH3–N) removal rate are desirable outputs, dry sludge yield is determined as undesirable output. Mauricio-Iglesias et al. (2020) design a new index for energy efficiency of WWTPs, and 44 different WWTPs are evaluated via this index. Li et al. (2020) develop a two-stage DEA with undesirable inputs to determine the efficiency of industrial waste treatment and recycling facilities. The model developed in this study is based on a circular economy structure for China. Pan et al. (2020) employ bootstrap DEA method to determine the efficiency of urban WWTPs. The data between 2008 and 2017 are used with Malmquist index model to evaluate WWTPs located on Yangtze River Economic Belt, China. They show no city has full efficiency for 2017.

Study Motivation

As can be seen from the literature review given in the previous subsection, efficiency evaluation of WWTPs from different aspects is one of the topics handled in the current literature. But, there is a very limited number of studies examining this problem as an integrated multi-criteria decision-making (MCDM) problem. Therefore, a comprehensive set of criteria is defined, and these criteria are weighted by experts in this study to be used as input or output variables in DEA. Further, the methodology is applied to Turkey for the first time. To the best of our knowledge, there is no study for determining the efficiency of WWTPs according to cities in Turkey, and this study includes the first integration of stepwise weight assessment ratio analysis (SWARA) and DEA.

In the first step of integrated methodology, the importance weights of all candidate inputs and outputs are computed via SWARA method to determine the most appropriate and related parameters to use in DEA. There are many methods used in weight determination, such as AHP, ANP, entropy, the Best Worst method, etc. in the literature (Zolfani et al. 2013). SWARA, which is one of the weight determination methods, is an expert-oriented method. SWARA method, which is used in this study has many advantages compared to other weight determination methods. It is possible to decide for criteria weights simultaneously using opinions of different experts in SWARA method. In this way, less time is consumed in the weight determination process (Zolfani et al. 2013). Also, the method is important in collecting information from experts and gathering it together. Unlike the methods such as AHP and ANP, the rating process can be performed according to the priorities of the alternatives without evaluating or sorting the criteria. Thus, since the opinions of the experts are taken into consideration, SWARA method can be used for more subjective evaluations (Hashemkhani Zolfani et al. 2015). Since there is no comparison between all criteria for weight determination in SWARA method, it is less time consuming than AHP method. SWARA can directly decide on the criteria and priorities, so it is also suitable for situations where the criteria weights are known. In short, SWARA method is used as a weight determination method in this study because it is easier to implement, takes less time, and gives DMs more space to prioritize criteria.

Efficiency scores of WWTPs are calculated using DEA models in the second step of integrated methodology. DEA is an useful tool frequently used in performance measurement. It is possible to use a large number of input and output variables in DEA. These variables also can have different units. There is no need to make various transformations assumptions to measure inputs and outputs in the same unit. In DEA, it is not necessary to establish a functional relationship between inputs and outputs except in linear form. It has many advantages, such as comparing fully efficient decision-making units (DMUs) whose efficiency is under review, rather than average efficiency units.

In this study, the efficiency scores of WWTPs in Turkey’s metropolitan municipalities are determined using DEA method models. Metropolitan municipalities are used as DMUs in DEA. As a result of literature review and MCDM application, Daily Wastewater Amount per Person Discharged in Municipalities, WWTP Capacity and Number of WWTPs are used as inputs; Amount of Wastewater Treated in WWTPs and Municipal Population Served by WWTPs are used as the outputs to determine plant efficiencies. Figure 1 shows the variables used for performance evaluation of metropolitan municipalities.

Fig. 1
figure 1

The variables of DEA

Full size image

This study is organized as follows: Section “The Proposed Methodology” presents the proposed integrated methodology. Section “Case Study” presents the numerical application of the proposed methodology. The sensitivity analysis is detailed in Section “Conclusion”. Finally, the last section shows the conclusions and future recommendations.

The Proposed Methodology

This paper explores the determination of the efficiency scores of WWTPs. In this study, a two-step methodology is established to determine efficiency scores. In the first step, to determine the most appropriate and related parameters, the importance weights of all candidate input parameters and output parameters are computed using the extended SWARA method. Then, the inputs and outputs are chosen according to their importance weights. Efficiency scores of WWTPs are calculated using output-oriented DEA models in the second step. Finally, the results are presented and discussed by sensitivity analysis. The proposed methodology levels are given in Fig. 2. The levels of the methodology are detailed in the following sections.

Fig. 2
figure 2

The proposed methodology

Full size image

The Extended SWARA Method

SWARA is one of the MCDM methods used for determining the criteria weights and ranking them in importance order. SWARA is developed by Keršulienė et al. (2010) as an expert-oriented subjective decision-making method (Ijadi Maghsoodi et al. 2019). The method allows DMs to choose their own priorities taking into account the current environmental and economic conditions. The role of experts identified as DMs is even more critical in this method (Zavadskas et al. 2019), because the main feature of the method is the use of expert opinions in determining the criteria weights. The method can be used to solve many different decision-making problems. Example studies using SWARA method are summarized in Table 1.

Table 1 Remarkable studies using SWARA method
Full size table

In SWARA, DMs rank all criteria from most important to least important and remove the unimportant criteria from the decision process. In the next stage, the remaining criteria are reordered from important to unimportant, and the most important criterion is assigned a value of “1.00”. Then, based on this most important criterion, the importance of the remaining criteria is determined on a 0.05 point interval. For example, if the second criterion is 5% less important than the most important criterion, 0.95 is assigned to the second criterion. The relative importance of weight is obtained since all criteria are evaluated according to the first (most important) criterion (Keršuliene and Turskis 2011). Hashemkhani Zolfani et al. (2018) extend SWARA with the testing reliability of DMs and extend SWARA to improve the quality of the MCDM process by incorporating the reliability evaluation of DMs’ opinions about criteria. The steps for the extended version of the method are as follows (Hashemkhani Zolfani et al. 2018).

Step 1. Ranking of the criteria

The criteria are ranked according to their expected importance, with the most important being the first. The importance of each criterion is determined via the expert judgment methodology presented by Zavadskas et al. (2007).

Step 2. Calculating priority values \((\bar t_j)\)

The tjk values are obtained via interviews with DMs according to their preferences and the priority values \((\bar t_j)\) for each criterion are calculated using Eq. 1.

$${\bar{t}}_{j} = \frac{{\mathop {\sum }\nolimits_{{k} = 1}^{r} {t}_{{jk}}}}{{r}},$$
(1)

where tjk is the ranking of criterion j by DM k, and r represents the number of DMs.

Step 3. Calculating weights (qj)

The attribute weights are calculated via Eq. 2.

$${q}_{\mathrm{j}} = \frac{{{\bar{t}}_{j}}}{{\mathop {\sum }\nolimits_{{j} = 1}^{n} {\bar{t}}_{j}}},$$
(2)

where n represents the number of criteria.

Step 4. Calculating dispersion of experts (α2) and the variations (βj)

$${\upalpha}^2 = \frac{1}{{{r} - 1}}\mathop {\sum }\nolimits_{{k} = 1}^{r} \left( {{t}_{{jk}} - {\bar{t}}_{\mathrm{j}}} \right)^2.$$
(3)
$${\upbeta}_{j} = \frac{{\upalpha }}{{{\bar{t}}_{\mathrm{j}}}}.$$
(4)

Step 5. Determining value for concordance (W)

$${W} = \frac{{12{S}}}{{{r}^2\left( {{n}^3 - {n}} \right) - {r}\mathop {\sum }\nolimits_{{k} = 1}^{r} {T}_{k}}},$$
(5)

where S is the total square deviation of rankings of each criterion, Tk represents the index of reiterated ranks. S is calculated via Eq. 6.

$${S} = \mathop {\sum }\limits_{{j} = 1}^{n} \left( {\mathop {\sum }\limits_{{k} = 1}^{r} {t}_{{jk}} - \frac{1}{{n}}\mathop {\sum }\limits_{{j} = 1}^{n} \mathop {\sum }\limits_{{k} = 1}^{r} {t}_{{jk}}} \right)^2,$$
(6)

Step 6. Calculating χ2 value

The concordance value’s significance is determined.

$$\mathop {\upchi}\nolimits_{\upalpha,{v} }^2 = \frac{{12{S}}}{{{rn}\left( {{n} + 1} \right) - \frac{1}{{{n} - 1}}\mathop {\sum }\nolimits_{{k} = 1}^{r} {T}_{k}}}.$$
(7)

If the value of \(\chi _{\alpha ,v}^2\) is larger than the tabular value of χ2 for the predetermined α value, then the hypothesis about the consensus of ‘judgments’ of DMs is not rejected.

Step 7. Determination of the relative importance levels

The relative importance levels are determined for each criterion (j) starting from the second criterion. The criterion (j) is compared with the previous criterion (j − 1) for this. This ratio is called as “comparative significance of the mean value” and denoted by sj.

Step 8. Determination of the coefficients

The coefficient for each criterion kj is determined using Eq. 8.

$${k}_{\mathrm{j}} = \left\{ {\begin{array}{*{20}{c}} {1}\qquad\,\,{{\mathrm{j}} = 1} \\ {{s}_{\mathrm{j}} + 1\,\,\,\,{\mathrm{j}} \,\ne\, 1} \end{array}} \right.$$
(8)

Step 9. Recalculation of importance levels

New importance levels for each criterion qj are determined using Eq. 9.

$${q}_{\mathrm{j}}\, = \,\left\{ {\begin{array}{*{20}{c}}{1}\qquad \,\,\,\,\,\,{\mathrm{j} = 1} \\ \,{\frac{{{k}_{{\mathrm{j}} - 1}}}{{{k}_{\mathrm{j}}}}\qquad{\mathrm{j}}\, \ne \,1} \end{array}} \right.$$
(9)

Step 10. Determination of criterion weights

Criterion weights Wj are determined using Eq. 10.

$${w}_{j} = \frac{{{q}_{j}}}{{\mathop {\sum }\nolimits_{{j} = 1}^{n} {q}_{j}}}.$$
(10)

The DEA Method

DEA is first proposed by Debreu (1951). After, the method is developed by Farrell (1957) and consolidated by Charnes et al. (1978). In DEA, many similar input–outputs are evaluated using mathematical programming techniques. DEA is used to measure the productive efficiency of DMUs. One of the advantages of DEA is that the DMs determine the input and output weights. In addition, DEA is particularly suitable where researchers are concerned with converting multiple inputs into multiple outputs, investigating the efficiency of the units, and there are a few observations hinder a parametric analysis. The studies on different subjects using DEA are given in Table 2.

Table 2 DEA related studies
Full size table

It can be understood from Table 2 that DEA can be used in many different research areas such as health, supply chain, production, etc. There are many models used in DEA, which assumes that each system will select its input and output weights to maximize its effectiveness. DEA is basically divided into two groups as constant returns to scale and variable returns to scale. Charnes, Cooper, Rhodes (CCR) model (Charnes et al. 1978) is applied in case of constant returns to scale. Banker, Charnes, Cooper (BCC) model (Banker et al. 1984) is applied in case of variable returns to scale. CCR and BCC models are used in DEA. DEA models can be prepared in three different ways; input-oriented, output-oriented, and non-oriented (Özden 2008). An output-oriented model with little (or no) control over inputs; if there is little or no control over the outputs, an input-oriented model should be used. In input-oriented models it is tried to use the minimum input amount to produce the current output amount. In output-oriented models, the maximum output amount is tried to be obtained by using the available input amount. Lastly, non-oriented models are used if it is desired to produce the maximum output amount with the minimum input amount. Output-oriented models used in this study are as follows (Dinc and Haynes 1999).

Output-oriented CCR model

$${E}_{k} = {\mathrm{min}}\mathop {\sum }\nolimits_{{r} = 1}^{m} {v}_{\mathrm{i}}{X}_{{ik}}.$$
(11)
$$\mathop {\sum }\nolimits_{{r} = 1}^{p} {u}_{r}{Y}_{{rk}} = 1.$$
(12)
$$\mathop {\sum }\nolimits_{{r} = 1}^{p} {u}_{r}{Y}_{{rj}} - \mathop {\sum }\nolimits_{{r} = 1}^{m} {v}_{i}{X}_{{ij}} \le 0.$$
(13)
$${u}_{r},{v}_{i} \ge {\upvarepsilon}.$$
(14)

Output-oriented BCC model

$${E}_{k} = {\mathrm{min}}\mathop {\sum }\nolimits_{{i} = 1}^{m} {v}_{\mathrm{i}}{X}_{{ik}} - {\upmu}_0.$$
(15)
$$\mathop {\sum }\nolimits_{{r} = 1}^{p} {u}_{r}{Y}_{{rk}} = 1.$$
(16)
$$\mathop {\sum }\nolimits_{{r} = 1}^{p} {u}_{r}{Y}_{{rj}} - \mathop {\sum }\nolimits_{{i} = 1}^{m} {v}_{i}{X}_{{ij}} + {\upmu}_0 \le 0.$$
(17)
$${u}_{r},{v}_{i} \ge {\upvarepsilon},{\upmu}_0:{\mathrm{free}}.$$
(18)

Ek defines efficiency score of DMU k, Vi defines the weight of input i and Xik defines the number of input i for DMU k. ur and Yrk define the weights of output r and number of output r for DMU k, respectively. ε is a small positive number (i.e., 0.0000001) for the CCR model and a small number for the BCC model. Equations 11 and 15 are objective functions for CCR and BCC models, respectively. Equations 12, 13, 16 and 17 are constraints for determining efficiency scores of DMUs. Equations 14 and 18 are constraints related to decision variables used in mathematical models.

Case Study

In the study, first 30 metropolitan municipalities are determined as DMUs. Subsequently, input and output parameters used in the performance evaluation of the WWTPs are determined by literature review and expert interviews. Then, the number of parameters to be used in the study is reduced using the extended SWARA method to make more consistent performance analysis. Finally, efficiency scores of metropolitan municipalities are calculated using DEA.

Data Collection

The proposed model is applied for the measurement of the effectiveness of all WWTPs in the 30 metropolitan municipalities located in Turkey. The data used in the study are obtained from Turkish Statistical Institute (TUIK). The metropolitan municipalities located in Turkey are shown in Fig. 3.

Fig. 3
figure 3

Metropolitan municipalities in Turkey

Full size image

Determining the Variables Using SWARA

In this study, for the performance evaluation of the WWTPs six different input and four different output parameters, as can be seen in Table 3, are determined by a literature review.

Table 3 Input and output parameters
Full size table

In the first step of methodology, six different input parameters are specified to be evaluated by SWARA for efficiency determination. WWTPs have different capacities according to their size and technology. This is one of the most important factors determining the amount of water the facility can treat, so the total Capacity of the plants in the city is added into the proposed MCDM model. Two parameters related to the population are determined as input variables to evaluate the wastewater treatment performance objectively: The ratio of Municipal Population Served by Sewerage to Total Municipal Population (%) and Municipal Population Served by Sewerage. Considering that the cultural and economic structures of the cities may vary, it is assumed that the amount of clean water consumed will be different. In this context, Daily Wastewater Amount per Person Discharged in Municipalities (liter/person-day) is included in the study in order to get a more fair evaluation for 30 metropolitan cities with populations ranging between 750,000 and 15 million. The climate and geographical structure of a city are important in the construction of a WWTP. There are different numbers of WWTPs in the cities. To reflect this situation, Number of WWTPs is integrated into the proposed decision-making model as a parameter. There is no sewerage service in some settlements. This is an issue that should be taken into account in the performance evaluation of WWTPs. Finally, Number of Municipalities Served by Sewerage refers to the number of district municipalities that have sewerage infrastructure in the metropolitan municipality.

There are different output parameters considered in the literature for WWTP performance evaluation. Since this study focuses on public services of WWTPs, parameters based on treated water and population served are determined as output parameters. The ratio of Municipal Population Served by WWTP to Total Municipal Population (%) is included in the proposed model to make an objective comparison. Amount of Wastewater Treated in WWTP (thousand m3/year) is used as another output variable, as most of the studies in the literature. Number of Municipalities Served by WWTP and Municipal Population Served by WWTP are included in the model in order to evaluate the benefit and usage level of treated water to the society.

After determining the candidate parameters, suitable parameters to be used in DEA are selected by interviewing five experts (DMs). DMs are determined based on their experience in wastewater treatment processes. While determining the parameters to be used, the importance levels of the parameters are calculated. The extended SWARA method is applied to determine the importance (weight) of each input and output parameters. First, five different DMs sort criteria based on parameter weights for both input and output variables. Table 4 gives the order of each parameter according to DMs.

Table 4 Criteria rankings by DMs
Full size table

After the rankings are gained, the extended SWARA’s steps from 1 to 7 are performed using the equations given in these steps. The calculation process is performed separately for input and output parameters. Table 5 gives the results for input parameters from Step 1 to Step 6.

Table 5 Results of DM judgments for input parameters
Full size table

First, the hypothesis of the consensus of DMs’ ‘judgments’ is not rejected for this application according to \(\chi _{\alpha ,v}^2\) value. The DMs then rank the criteria from most important to least, using their knowledge and skills. While weighting the criteria, the opinions of experts are taken and average values are used. Weight calculation steps and weights for input parameters are given in Table 6.

Table 6 Weight calculation steps and weights of input parameters
Full size table

Three input parameters with the degrees of importance greater than 20% are used in DEA. These parameters are as follows; I3-Daily Wastewater Amount per Person Discharged in Municipalities (liter/person-day), I1-WWTP Capacity (thousand m3/year) and I4-Number of WWTPs.

The same procedure is repeated to determine importance levels of output parameters. Table 7 shows that the hypothesis about the consensus of DMs’ ‘judgments’ is not rejected.

Table 7 Results of DM judgments for output parameters
Full size table

\(\chi _{\rm{tabular}}^2\) value is greater than \(\chi _{\alpha ,v}^2\). So the weight calculation of output parameters can be performed with DMs. Weight calculation steps and weights for output parameters are given in Table 8.

Table 8 Calculation steps and the weights of output parameters
Full size table

Two output parameters with the importance degrees greater than 20% are used in DEA. These parameters are as follows: O3-Amount of Wastewater Treated in WWTP (thousand m3/year) and O4-Municipal Population Served by WWTP.

3.3. Efficiency Measurement by DEA

The variables used in DEA are determined with the extended SWARA method. Based on the expert opinions, the parameters determined as input variables are as follows: Daily Wastewater Amount per Person Discharged in Municipalities (I3), WWTP Capacity (I1) and Number of WWTPs(I5); and the parameters determined as output variables are s follows: Amount of Wastewater Treated in WWTPs (O3) and Municipal Population Served by WWTPs (O4). The data used in DEA models are official data from TUIK. These data are the most recent data published by TUIK about WWTPs, so this data are taken into account when evaluating the performance of WWTPs. All data used in the study are given in Table 9.

Table 9 Input and output data used in DEA
Full size table

While analyzing the wastewater treatment performances of metropolitan municipalities, it is aimed to provide as much output as possible with existing inputs. For this reason, the performances of metropolitan municipalities are determined by using the output-oriented models of DEA. The total efficiency scores (performance evaluations) of metropolitan municipalities are determined using the CCR model and divided into components with the BCC model. Output-oriented models are used in both stages. The technical efficiency scores of 30 metropolitan municipalities are calculated through the BCC model, and scale efficiency scores are obtained by proportioning the total efficiency scores to the technical efficiency scores. The efficiency scores of 30 metropolitan municipalities are given in Table 10.

Table 10 Efficiency scores of metropolitan municipalities
Full size table

As can be seen from Table 10, 14 metropolitan municipalities have total efficiency, 19 metropolitan municipalities have technical efficiency, and 21 metropolitan municipalities have scale efficiency. Adana, Ankara, Antalya, Aydın, Bursa, Denizli, Diyarbakır, Eskişehir, Gaziantep, İstanbul, İzmir, Kocaeli, Muğla, and Ordu are metropolitan municipalities with technical efficiency. According to the analysis, the total average efficiency is 80.2% among 30 metropolitan municipalities in Turkey; while average technical efficiency is 85.1% and average scale efficiency is 94.4%. Mardin has the worst total and scale efficiency scores with 0.007, and Kahramanmaraş has the worst technical efficiency score with 0.154.

Target outputs for inefficient metropolitan municipalities are determined by DEA. Then, the values to be provided to metropolitan muncipalities to become efficient in wastewater treatment processes are calculated. As a result of CCR and BCC models, reference sets and λ values are determined for reference sets. The target values that can give an idea to the management to become effective in ineffecient metropolitan municipalities are calculated with reference set. Table 11 shows the target values for each inefficient metropolitan municipality.

Table 11 Reference sets for inefficient DMUs
Full size table

As a result of the analysis, it has been determined which outputs and how much they will be increased to make the inefficient metropolitan municipalities efficient. Considering Konya, one of the inefficient metropolitan municipalities, it is seen that in Table 10, the efficiency score of the CCR model is determined as 0.789. The reference set of this WWTP consists of Ankara and İzmir, according to Table 11. The λ values are determined as 0.04 and 0.31 for Konya, respectively. The output values required for Konya to be effective can be calculated as follows:

$${\mathrm{Output}}_{{\mathrm{inefficient}}\,{\mathrm{DMU}}} = \mathop {\sum}\limits_i {\lambda _i{\rm{Output}}_i}.$$
(19)
$${\rm{Output}}_{{\rm{Konya}}} = 0.04 \times {\rm{Output}}_{{\rm{Ankara}}} + 0.31 \times {\rm{Output}}_{\dot{\mathrm{Izmir}}}.$$
$${\rm{Output}}_{\rm{Konya}} = \left( {\rm{O}}_3;{\rm{O}}_{4} \right) = 0.04\left( {303442;5142657} \right) + 0.31\left( {301087;4223545} \right) = \left( {105475;1515005} \right).$$

Konya must increase Amount of Wastewater Treated in WWTPs (O3) to 105,475 from 79,025 and Municipal Population Served by WWTPs (O4) to 1,515,005 from 1,192,298 in order to be an efficient WWTP for CCR model. Projection values for inefficient metropolitan municipalites are determined by Eq. 19 and given in Table 12.

Table 12 Projection values for inefficient DMUs
Full size table

As an example; Mardin, which is the worst among these 30 metropolitan municipalities, can become efficient if it increases both output parameters by 13,705%. In other words, Mardin must increase the Amount of Wastewater Treated in WWTPs (O3) to 33,547 and Municipal Population Served by WWTPs (O4) to 1,379,154 in order to be an efficient DMU according to CCR model. This analysis shows that there are issues need to be improved specifically for metropolitan municipalities. Another city, Kahramanmaraş, should increase its output values by 614% in order to become efficient for CCR model and should increase Amount of Wastewater Treated in WWTPs (O3) by 549% and Municipal Population Served by WWTPs (O4) by 556% to become efficient for BCC model.

Sensitivity Analysis

Comparison with AHP

The results of the proposed extended SWARA methodology for the parameter weighting are compared with the approach of AHP. AHP is used for solving MCDM problems. The method is one of the most used criteria weighting methods. AHP is based on criteria pairwise comparison. Thanks to the proper structuring and adjustment of the priorities in AHP, it can give the expected results for real-life problems. AHP has been used for criterion weighting in many different areas such as credit ranking (Ayyildiz et al. 2020), ATM site selection (Yildiz et al. 2020), land suitability modeling (Tashayo et al. 2020), swine sewage management (Vizzari and Modica 2013), hazard susceptibility assessment (Karlsson et al. 2017), etc. The steps of AHP are as follows:

Step 1. Define the decision problem and parameters.

Step 2. Construct a pairwise comparison matrix with experts. The experts use the scale given in Table 13 (Saaty 1977) to evaluate pairwise comparisons.

Table 13 Index scale for AHP proposed by Saaty
Full size table

Step 3. Examine the consistencies of pairwise comparison matrices. To find the consistency ratio (CR) of a matrix use crisp values proposed by Saaty (1977), first find the matrix consistency index (CI) using;

$${\rm{CI}} = \frac{{\lambda _{\rm{max}} - n}}{{n - 1}}.$$
(20)

Then calculate the CR using;

$${\rm{CR}} = \frac{{\rm{CI}}}{{\rm{RI}}}.$$
(21)

λmax is the largest or principal eigenvalue of the A decision matrix of pairwise comparison. Random index (RI) depends on matrix order (n) and is calculated using the table proposed by Saaty (1977). If CR is ≤0.1, then the matrix is considered as acceptable.

Step 4. Normalize the pairwise comparison matrix.

$$n_{ij} = \frac{{a_{ij}}}{{\mathop {\sum }\nolimits_{i = 1}^m a_{ij}}}.$$
(22)

Let aij be the pairwise comparison of parameter i and j. And m represents the number of parameters.

Step 5. Calculate the weights.

$$w_{ij} = \left( {\frac{1}{m}} \right)\mathop {\sum }\limits_{i = 1}^m n_{ij}$$
(23)

The consolidated pairwise comparisons of the input parameters are evaluated by experts. The pairwise comparison matrix of input parameters is shown in Table 14.

Table 14 Pairwise comparisons of input parameters
Full size table

After pairwise comparisons of input parameters are constructed by experts, AHP steps are employed to determine the weights. However, before this step, it is investigated whether the evaluations of experts are consistent, and if the pairwise comparisons are not consistent the experts are asked to re-evaluate. If the CR is calculated <0.1, the relevant matrix is considered consistent, and the weight calculation step is started. Once the pairwise comparison matrices are determined consistently, the weights of the parameters are calculated. As a result of the abovementioned calculations, the weights of the six input parameters are determined as 0.24, 0.09, 0.30, 0.20, 0.10, and 0.07, respectively. Thus, the parameter ranking order can be stated as I3 > I1 > I4 > I5 > I2 > I6. The ranking order is the same as the result of extended SWARA.

Then, the consolidated pairwise comparisons of the output parameters are evaluated by experts. The pairwise comparison matrix of output parameters is shown in Table 15.

Table 15 Pairwise comparisons of output parameters
Full size table

The same procedures are repeated for output parameters, and their weights are determined as 0.15, 0.16, 0.36, and 0.33, respectively. Thus, the parameter ranking order can be stated as O4 > O3 > O2 > O1. The ranking order is the same as the result of extended SWARA. This is important to show the validity of the method.

Determining the effects of variables

Since output-oriented models are used in the application, the effect of change in output variables on the effectiveness of WWTP is examined. Thus, the importance of each output is determined for each metropolitan municipality. For this purpose, one of the output variables is removed from the analysis, respectively, and efficiencies are determined. In this way, it is determined which output variables are more important for efficient metropolitan municipalities. This analysis is performed for CCR and BCC models. The results obtained from this analysis are presented in Table 16.

Table 16 Results of sensitivity analysis
Full size table

When the results of the sensitivity analysis are examined for a sample DMU (Adana), presented in Table 16, the following evaluations can be made:

According to the analysis made when the variable of O3-Amount of Wastewater Treated in WWTP (thousand m3/year) is removed, the efficiency score of Adana is decreased to 0.885 in CCR model and 0.864 in BCC model. In this case, it can be said that O3 is an important variable for Adana in terms of efficiency. When efficiency scores are determined by removing the variable O4-Municipal Population Served by WWTP, Adana becomes efficient again in both models. For Adana, this variable is less important than the other variables.

While only one variable is important for some of the DMUs, two variables can be important for some others. In this context, an analysis is made on which is the most important variable for DMUs. According to this analysis; while O3 is important for 19 DMUs (Adana, Ankara, Aydın, Balıkesir, Bursa, Denizli, Erzurum, Hatay, İzmir, Kayseri, Kocaeli, Konya, Manisa, Muğla, Ordu, Samsun, Tekirdağ, and Van), O4 is an important variable for four DMUs (Gaziantep, Malatya, Şanlıurfa, and Trabzon). Also, both variables are important for three DMUs (Kahramanmaraş, Mardin, and Sakarya). Four DMUs (Antalya, Diyarbakır, Eskişehir, and Antalya) are determined as efficient for each analysis.

Performing DEA for different years

A sensitivity analysis is performed to analyze and discuss the proposed methodology results. In the analysis, comparisons are made with the most recent data published in 2016. Accordingly, data published in 2014 and 2012 are taken into account in determining the efficiency levels of municipalities from the past to the now. With the help of this analysis, future projections are presented for the effectiveness of metropolitan municipalities, which is important as a guiding resource for DMs. For this purpose, firstly, data for the year of 2014 are taken into consideration and given in Table 17.

Table 17 Input and output data for the year of 2014
Full size table

Then, DEA is performed to determine the total, technical, and scale efficiency scores of metropolitan municipalities for the year of 2014, and the results are presented in Table 18.

Table 18 Efficiency scores of the metropolitan municipalities for the year of 2014
Full size table

The proposed methodology’s behavior against the changes of parameter’s value is observed and analyzed in detail for discussion. Results of this analysis would help the user in determining priorities and making easier the analysis process. Note that the efficiency scores of Mardin and Kahramanmaraş are determined as 0 because there are no WWTP in Mardin and Kahramanmaraş in 2014. The numbers of municipalities that have total efficiency, scale efficiency, and technical efficiency are determined as seven, thirteen, and six, respectively. According to the analysis, the average of total efficiency decreases to 68.3% from 80.2%, the average of technical efficiency decreases to 77.3% from 85.1%, and the average of scale efficiency decreases to 80.4% from 94.4%.

In the second analysis, DEA is performed with data for the year of 2012 (see Table 19), and the results are presented in Table 20.

Table 19 Input and output data for the year of 2012
Full size table
Table 20 Efficiency scores of metropolitan municipalities for the year of 2012
Full size table

The numbers of municipalities that have total efficiency, scale efficiency, and technical efficiency are determined as 7, 13, and 6, respectively. For Erzurum, its total efficiency score is determined as 0.002 in both analyzes. The reason is that WWTPs in Erzurum do not work actively during these years.

The total efficiency scores of metropolitan municipalities for each year are compared in Fig. 4.

Fig. 4
figure 4

Result of sensitivity analysis

Full size image

Kahramanmaraş and Mardin have the worst values for total efficiency scores for all years, according to Fig. 4. Şanlıurfa also has low efficiency. Antalya, Eskişehir, Diyarbakır, İstanbul, and Kocaeli have 1 as the total efficiency score for all years.

Managerial and Social Implications

The findings obtained as results of the WWTP effectiveness analysis of the metropolitan municipalities are examined together by five experts (one from the private sector, two from the public sector, and two academicians). All experts work on WWTP related issues. First of all, experts are informed about the purpose of DEA method, mathematical expressions, and input/output variables. Then, they are asked to evaluate the results by comparing them with the current situation of the metropolitan municipalities. The general evaluation of the experts is that the findings obtained are consistent with the actual situation. It is emphasized that the results for Mardin should be examined in more detail. The details of some evaluations made by experts can be listed as follows:

  • As a result of the analysis, six metropolitan municipalities (İstanbul, Ankara, İzmir, Bursa, Antalya, and Adana) with the highest population in the country are found to be efficient. It is stated that this situation is due to the high number of outputs.

  • Mardin has the lowest effectiveness. The reason for this situation is that the WWTP in Mardin has started to serve in late 2017.

  • Aydın, Denizli, Eskişehir, Kocaeli, and Muğla are determined as efficient in both models. It is understood that these cities, located in the western part of the country, are efficient thanks to their climate conditions and technological infrastructure.

  • Hatay, Van, and Şanlıurfa are determined as inefficient. Hatay, Van, and Şanlurfa should increase their performances by taking into account the increasing number of refugees in the recent years. Likewise, Diyarbakır and Gaziantep, which are efficient, should continue their performances.

  • Sakarya, Tekirdağ, and Trabzon are determined as inefficient. It is challenging to build a proper WWTP due to the geographical conditions of the regions.

Experts also made some suggestions for the implementation process. The first is that more effective evaluations can be made when the income/expense parameters of WWTPs are added. Another suggestion is that the results can be compared by calculating multiple models with different variables. So the input(s)/output(s) will be more consistent. Finally, it is suggested that WWTPs can be evaluated separately.

Conclusion

In order to meet the increasing water demand, the public and private sectors make significant investments in infrastructure. In this context, WWTP investments, which play an important role in the recycling of used water, are also increasing. Since every investment creates costs for the investor, it is crucial to use the available resources in the most appropriate way. Increasing the amount of treated wastewater and the number of people served are factors that increase the satisfaction of the public and the trust in the investors. At this point, it is one of the primary objectives of investors to provide services with the most appropriate planning within the available resources. Therefore, in this study, evaluations are made especially on the output variables together with the variables determined as input.

In this study, first, input and output parameters are determined to be used in evaluating WWTP performances of metropolitan municipalities. Then, these parameters are evaluated, and the number of parameters are reduced to be taken into account in the study by ye extended SWARA method with expert interviews. Subsequently, efficiency scores of 30 metropolitan municipalities are determined using the output-oriented versions of CCR and BCC models, which are the two basic models of DEA method. Within the scope of sensitivity analysis, DEA computations are conducted for different years. As a result of this study, it is determined that whether a metropolitan municipalities are efficient or not.

The contributions of the paper to the literature and application area can be specified as follows: (1) DEA method is adapted to the problem of performance evaluation of WWTPs; (2) The most important parameters on the problem of performance evaluation of WWTPs are determined and classified as inputs and outputs; (3) These parameters are evaluated by SWARA method and the importance levels of each parameter are obtained; (4) Parameters are eliminated according to their importance levels; (5) Considering Turkey, 30 metropolitan municipalities are evaluated by output-oriented DEA models and their efficiency scores are determined; (6) A real application is presented to show the applicability and reliability of the methodology; (7) It is aimed that, the proposed method will be used by firms to improve their strategies; (8) To the best of our knowledge, this study is the first real case of DEA method adaption to the problem of performance evaluation of WWTPs.

As a future direction, other performance evaluation methods or heuristics can be included in the methodology to ensure a more integrated and comparative study. Alternatives in different regions can be compared by using the proposed methodology in this study. The WWTPs may be evaluated separately. Also, this study can be expanded by interviewing with more experts.