1 Introduction

The efficiency of a dam reservoir to fulfill its two major roles of water regulation and flood protection, depends directly on the ability of its management approach to define the optimal releases at each time-step. Nonetheless, even if this ability is acquired and validated, it requires an update of its guidelines notes and parameters, in order to adapt it to new contexts. In fact, the optimization methods, if they are well configured and their results well evaluated, can lead to an optimized management approach. In this objective, Hashimoto et al. (1982) proposed an adaptation of the performance criteria, already used in other disciplines (Fiering and Holling 1974; Haimes and Hall 1977; Holling 1973), to the context of water resources assessment. Since, these criteria treating essentially the properties of reliability, resilience and vulnerability, have been subject to diverse discussion, and modifications, in order to enhance their representativeness of the performance aspect targeted (Jinno et al. 1995; Srinivasan et al. 1999; Vogel and Bolognese 1995). Other authors introduced new criteria representing new aspect of the performance, such as: sustainability (Loucks 1997; McMahon et al. 2006; Ray et al. 2010; Sagar and Najam 1998), variability of water allocations (Cai et al. 2002; Hirsch 1979) and the severity of deficit (Moy et al. 1986). However, and despite the long development of these performance criteria, criticism emerged addressing their ability to resume, significantly and correctly, in a unique value, the performance of a given water management system, particularly under varying application conditions. For instance, Kjeldsen and Rosbjerg (2004) concluded that the fact of basing the calculation of these criteria on a simulation period not sufficiently long, made the values obtained less representative of the system performance. Jain and Bhunya (2008) found that only one criterion can never describe the positive and negative points of a studied management method. And Sandoval-Solis et al. (2011) declare that the sustainability criterion is more influenced by low values of its parameters than high values.

In this paper, we studied the ability of six performance criteria to assess and compare 6 reservoir management approaches, under three different contexts. This comparison was done using the following six criteria: The released allocated volume, the released lost volume, the reliability, the resilience, the vulnerability and the sustainability.

2 Materials and Methods

2.1 The Studied Hydraulic Complex

The hydraulic complex of Bin El Ouidane-Ait El Ouarda-Afourer (called hereafter “BEO-AEO”) was constructed to satisfy the water demand of irrigation, drinking water and hydroelectricity. The complex contains two hydroelectric stations: one in BEO and another in Afourer. This permits to double valorize energetically the releases.

2.2 Description of the Six Management Approaches

In addition to the current management approach of BEO-AEO, the five analyzed management approaches concern five different cases of optimization of both the flood section (FS) and the released volume (Table 1). In fact, to establish these five approaches, a double optimization was realized: the first concerns the search, among five cases, for the optimized formula of FS calculation; and the second uses the optimized formula resulting from the first optimization, and focuses on the optimization of the released volume from BEO, by testing five different formulae.

Table 1 Different cases tested in the optimization process
Full size table

Concerning the objective function (OF), it focuses in the minimization of the drained volume form the complex via the bottom outlets.

The optimization is assured by the Genetic Algorithm (GA) method. The phase of optimization was preceded by a pre-optimization phase which permitted to define the best GA configuration, and the limits of variation of the optimized parameters.

2.3 The Performance Criteria

In the rest of this paper, the notions “satisfaction” and “dissatisfaction” of water demand indicate that the deficit is zero or different from zero respectively.

The performance criteria calculated are:

  • The total allocated volume: it represents the portion of the volume released from BEO, which flows from BEO to the hydroelectric station of Afourer via the connection pipe.

  • The total lost volume: it refers to the portion of the released volume which is evacuated by the AEO’s bottom outlets and/or spillway.

  • The reliability: is the probability that the allocated volumes meet, without deficit, the water demand during the simulation period (Hashimoto et al. 1982). In this paper we have calculated it based on satisfaction duration method proposed by McMahon et al. (2006) (Eq. 4):

$$ \operatorname{Re} li.=\frac{Nb\cdot of. times.{D}_t=0}{n} $$
(4)

Where, Dt is the deficit at time step t; n is the number of time steps in the simulation period.

  • The resilience: is the probability that a time step characterized by a state of demand dissatisfaction, is followed by another on which we have a satisfaction of demand (Sandoval-Solis et al. 2010) (Eq. 5):

$$ \operatorname{Re}s.=\frac{Nb. of. times.{D}_{t+1}=0. follows.{D}_t>0}{Nb. of. times.{D}_t>0} $$
(5)

  • The vulnerability: is the likely value that the deficit can take in case the system is in the case of a demand dissatisfaction (Hashimoto et al. 1982). Among the different methods to calculate it, we have used the one aiming to estimate an average value of likely deficit, as shown in Eq. 6 (Sandoval-Solis et al. 2011):

$$ Vul.=\frac{\raisebox{1ex}{$\sum \limits_{t=1}^n{D}_t$}\!\left/ \!\raisebox{-1ex}{$ Nb. of. times.{D}_t>0$}\right.}{water. demand} $$
(6)

  • The sustainability: is a criterion which attempts to unify in a unique value the information driven by more than one performance criterion. Sandoval-Solis et al. (2011) reformulated this formula so that it overcomes some issues of content, scale and flexibility that the formula of Loucks (1997) shows. The new formula is described in Eq. (7):

$$ Sus.=\sqrt[3]{\left[\operatorname{Re}l.\times \operatorname{Re}s.\times \left(1- Vul.\right)\right]} $$
(7)

2.4 The Contexts or the Calculation Conditions

In this paper, we studied the effect of three contexts:

  • The water demand: we compared two scenarios: a normal water demand scenario and a high water demand scenario. But, because we did not have access to the water demand data in BEO-AEO complex for the studied period, we have proceeded to the elaboration of hypothetical series which represent the daily water demand during a year.

  • The period of simulation: In fact, the effect of the simulation period is not limited only to its pluviometry, but also to its temporal length. The length effect was reported by Kjeldsen and Rosbjerg (2004) and Kundzewicz and Kindler (1995). In this paper, we have tested three periods differentiable by their length and pluviometry: (1) 1986-2008 (long period, moderate pluviometry); (2) 1986-2005 (short period, low pluviometry); (3) 2005-2008 (short period, high pluviometry). For detailed information about the mentioned decomposition, please refer to Ahbari et al. (2017).

  • The time step (annually, monthly and daily): Given that the majority of studies in the literature focused on the evaluation of performance based on an annual time step, we wanted to analyze the effect of time step modification on the evolution of performance criteria values. Therefore, we have tested three time steps: annually, monthly and daily.

The analysis of the effect of the three contexts mentioned was done by assessing, at each time, the effect of a combination “demand scenario-simulation period-time step” of the three contexts, on one management approach and one performance criteria (one from the five last criterion). The two first performance criteria are analyzed separately. The Fig. 1 presents the studied combinations.

Fig. 1
figure 1

The 18 combinations “demand scenario-simulation period-time step” studied

Full size image

As illustrated in Fig. 1, we have attributed to each combination a letter to facilitate their citation in the text. The analysis of the performance criteria evolution in function of the combination selected, and the comparison of the management approaches, was performed using the Principal Component Analysis (PCA). Indeed, for each performance criteria, two PCA were calculated: (1) based on the combinations as variables and the approaches as observations (named hereafter “PCA-A”); (2) based on the approaches as variables and the combinations as observations (named hereafter “PCA-B”).

3 Results and Discussion

3.1 FS Optimization

The results of optimization of the predefined five cases are presented in Fig. 2a. Indeed, we notice a decrease of the allocated volumes of 0.6% for the five cases, in comparison to the current approach (hereafter called “Initial”). This presumes that instead of enhancing the valorization of the releases, the optimization deteriorated the performance of the management. Concerning the total lost volume during the period of simulation, it experienced a sensitive variation of its value depending on the case selected. In fact, oppositely to the allocated volumes, the optimization succeeded to decrease the lost volumes from the system BEO-AEO. The total lost volume diminishes by 1.7% for case 4 (from 509.59 Mm3 to 501.09 Mm3). This is comprehensible, since the OF focused on the minimization of the lost volumes.

Fig. 2
figure 2

Evolution of volumes released, allocated and lost in function of the case of: a FS calculation; b releases calculation

Full size image

Despite the limited diminution of water allocations (0.6%), the performance improvement by only 1.7% after the optimization, do not question the efficiency of the Initial management approach. Concerning the evolution of the FS volume, we have found that this parameter evolution for the Initial approach is cyclic: a maximum of 225 Mm3 between October and February, and 0 for the rest. On the other hand, the same parameter evolution in case 4 is also cyclic, but it has roughly a “bell curve” form changing from year to year. This change affects more the cycle peak (between 225 and 290 Mm3) and the decrease after the peak. However, the observations that raise the suspicion regarding a probable dysfunction of the Initial approach are not clearly spotted, unless we zoom in on one cycle. Figure 3 compares the FS volume of the two approaches during one cycle.

Fig. 3
figure 3

Comparison of the daily FS volume of the Initial and case 4 management approaches, during the cycle 01/03/1987-31/09/1988

Full size image

The key observations extracted from Fig. 3 are: (1) While the FS in the Initial approach is deactivated during August and September, the optimized approach plans to allocate a volume ready to receive the summer storms characteristic of the region; (2) During the period October-February, the Initial approach defines a 225 Mm3 FS, activated until the end of the period. This may deprive the BEO reservoir of an important amount of water supply occurring in the autumn-winter period, since a portion of the water supply should be inevitably drained from the reservoir, to keep the FS activated. On the other hand, the optimized approach minimizes that portion, by limiting the FS of 225 Mm3 to months which show, historically, a high frequency and intensity of floods; (3) After the period October-February, and in opposition to the Initial approach, the optimized approach defines a FS volume, not as big as for the period October-February, but it permits to contain the spring rainfall events accentuated by snowmelt.

These three observations, undetectable while comparing the six approaches via the two criteria “allocated and lost volumes”, would not emerge until a detailed analysis of the results was performed. Hence, the manager should read with precaution the information driven by the allocated and lost volumes criteria values. To summarize, we can say that it was difficult to assess correctly the performance of the compared management approaches, using just the values of those two criteria. In fact, without the analysis of the results behind those values, the comparison suggested that the Initial approach does not show any sign of dysfunction.

The optimized values of the FS volume in case 5 were used in the optimization of the releases which is presented in sub-section 3.2. Nevertheless, the case 4 formula is the optimal equation we recommend to the manager for the adaptation of the Initial management approach.

3.2 Releases Optimization

3.2.1 “Total Allocated and Lost Volumes” Criteria

During the optimization of the releases, we have used the formula of case 5 to calculate the daily FS volume. In fact, in the last sub-section we have seen that case 5 and despite the optimization, it augments the lost volumes compared to the Initial approach. Therefore, and for analysis purpose we have chosen to use the worst case to see if the equation in case 5 is really not appropriate, or maybe it is again a failure of the criteria “allocated and lost volumes” to resume the performance into two simple values. In the rest of this paper, the use of: (1) case 5 formula to calculate the FS, without optimizing the releases is called “Case 0”; (2) the use of the FS values of the Initial approach is called “Initial”; (3) And cases 1, 2, 3, 4 and 5 are cases where we have optimized the releases (Table 1). Figure 2b illustrates the evolution of released, allocated and lost volumes of BEO-AEO complex, in function of the releases optimization case.

The Fig. 2b shows a significant decrease of the lost volumes (from 60% in case 1, to 93% in cases 4 and 5) in comparison to the loss provoked by the use of the Initial approach (509.6 Mm3). Additionally, this is associated with an increase of allocated volumes (from 3% in case 1 to 4.2% in cases 4 and 5). The augmentation of allocations and diminution of loss should decrease the deficits of irrigation, drinking water and hydroelectricity. Also, the increase of released volumes (from 3.5% in case 1 to 3.9% in cases 4 and 5) should diminish increasingly the hydroelectricity deficit, since all the releases will be turbinated at least once at the BEO hydroelectric station.

To conclude, the criteria “allocated and lost volumes” indicate that: the optimization of the FS and releases permitted to converge to performance better than those realized using the Initial approach.

The descending order of the studied approaches, in function of the performance criteria “allocated and lost volumes” is: Case 4 et 5, Case 3, Case 2, Case 1, Initial then Case 0.

According to the allocated and lost volumes criteria, the best rectification we recommend to enhance the performance of BEO-AEO current management is to determine the releases depending on the stocked volume at time t and water supply at t-1 (Case 2). Indeed, since the improvement after case 2 is not considerably important (a maximum of 0.2% allocations and − 31.8% loss), we do not judge necessary recourse to methods of water supply prediction with all the uncertainties they will provoke. In the rest of this paper, we will omit case 5 because it presents the same performance as case 4.

3.2.2 Reliability Criterion

After calculating the reliability criterion for the six management approaches under 18 combinations of factors that may affect this criterion, a PCA-A was run on the matrix of the obtained results. Figure 4a presents the biplot variables-observations of the first plane of the PCA type A.

Fig. 4
figure 4

a Biplot variables-observations of a the PCA-A first plane of reliability; b the PCA-B first plane of reliability; c the PCA-A first plane of resilience; d the PCA-B first plane of resilience

Full size image

La Fig. 4a shows that the first plane of the PCA expresses 87.68% of the total inertia. The first plane dimensions are approximatively represented by the standard-deviation of the reliability for each management approach (“SD” in Fig. 4a, r = 0.984), and the average reliability for each management approach (“Average” in 4 (a), r = 0.797). Consequently, the reliability of the six approaches can be sorted according to two principal components: (1) those located at the right of the dimension 1 have an important reliability standard-deviation compared to those in the left; (2) those placed at the top of dimension 2 have an average reliability higher than those in the bottom. Certainly, the approach which have the highest average value, and the lowest SD value is the most efficient. Thus, the descending order of approach suggested by the reliability criterion is: Case 3, Case 2, Case 4, Case 1, Initial then Case 0. As it is obvious from the ranking, the importance was given to the classification according to SD value for two reasons: (1) it is the dimension that explain better the differences between approaches (65% of inertia); (2) the minimal SD means that a kind of stability of performance is installing despite the change of contexts conditions.

Regarding the evolution of reliability in function of combination, Fig. 4b resumes the results of the PCA-B realized. It indicates that the first PCA plane containing 99.85% of inertia, is particularly focused on dimension 1 representing alone 99.29% of variability. The first dimension is approximated by the supplementary variable “average reliability calculated for each combination” (“Average” in Fig. 4b, r = 1). Furthermore, this PCA plane permits to identify 5 groups of combinations having similar effect on approaches: group (D, B and F), group (L, H and J), group (R), group (N, E, A, K, C and G) and group (I, M, O, P and Q).

Thereby, the combinations situated to the right of dimension 1 made the average reliability of an approach higher than those to the left. This means that the common factors between the combinations in the right tend to overestimate the reliability of the studied system, while the situation is inversed in the left of the axis. Therefore, the descending order of influence of the studied conditions on the reliability criterion is: Firstly, it is the demand scenario type, a normal scenario causes a higher reliability value than when using the high demand scenario. Then, came the influence of the calculation time step, which being short it provokes a high reliability and vice versa. Finally, the pluviometry of the simulation period is the less influencing condition. In addition, the observed effect of factor “time step” can be related also to the number of time steps considered. Hence, it is possible that a time step of 1 year, for example, repeated X times, converges to a reliability value as high as the one resulting from a time step of 1 day repeated also X times. This hypothesis is difficult to verify since if we take two simulation periods of X times daily and annual time steps, we will found inevitably that the two periods have different pluviometry.

3.2.3 Resilience Criterion

The PCA-A of resilience (Fig. 4c) shows an inertia of 91.75% for the first plane. The variability of combinations effect on management approaches is particularly explained by the first axis (approximated here by the supplementary variable “Median”, r = 0.982).

The dimension 1 of the PCA-A plane permits to sort the approaches according to the median of resilience values outputted by the use of different combinations (“Median” in Fig. 4c, r = 0.952). Therefore, we found in the right of axis 1 the approaches having a high resilience, and in the left those with low resilience values. Consequently, the descending order of approaches in function of their resilience is: Case 0, Initial, Case 3, Case 1, Case 2 then Case 4. Two remarks are deducted from this new ranking: (1) Case 0, considered before as the bad case, is the best management approach according to the resilience criterion; (2) The approaches for which we have not optimized the releases succeeded to perform better than those with optimized releases. The first remark indicates that the order of the best approaches can change depending on the criterion tested, which is the reason why we recommend to test several criteria before making a final decision. Nevertheless, the use of a unique criterion, either in the optimization or in the assessment of an approach, can be understood if the manager is seriously concerned about one aspect of performance (reliability, resilience…). In fact, this is the case highlighted by the second remark, which may prove that the current manager of BEO-AEO, is more concerned about the resilience of his system than other aspects of performance. This zero tolerance toward resilience compared to other aspects of performance, is more likely related to the fact that the management of the BEO-AEO complex is part of the Oum Er-Rabia basin management strategy. Indeed, the big portion of water allocations are directed to irrigation, but since the perimeters have other sources to fulfill their water demand (Tadla and Tassaout Aval aquifers), the manager of BEO-AEO is more tolerant toward the deficits that his system can manifest. On the other hand, the manger is more oriented to keep his complex able to return as fast as possible to the satisfaction state, because the other sources of water are not permanently available (due to: aquifer over exploitation, limited allocated volumes, …). However, the manager can improve the resilience of BEO-AEO complex by adopting the formulae of FS calculation proposed in Case 0.

Regarding the PCA-B (Fig. 4d), dimension 1 of the first PCA plane explains 95.95% of the total inertia. The supplementary variable “average resilience caused by each combination” (“Average” in Fig. 4d) tends to represent axis 1 (r = 1). Therefore, this axis classifies the combinations from those responsible of high resilience (in the right), to those leading to zero resilience (in the left).

According to the combinations distribution presented in Fig. 4d, the descending order of factors’ influence on the value of resilience is: Firstly, the water demand scenario, which when it is normal, it permits to the system to increase its resilience. Secondly, came the time step, but this time the monthly converges to better performance than the annual and the daily. And finally, the pluviometry of the simulation period, and here again we have an inversed order compared to the reliability criterion. Hence, we notice that during the period with low pluviometry, the system was more resilient than in the period with important pluviometry. This may sound contradictory, but it reflects perfectly the current management approach. Indeed, the high resilience of the BEO-AEO complex is obtained when using a monthly time step, and a simulation period with low pluviometry. This confirms the current practiced management approach of BEO-AEO: a monthly management oriented to decrease deficits during dry periods. In fact, to converge to the current management approach, the manager had very likely used a dry simulation period and a monthly time step, to optimize the parameters during the first calibration of the approach. Concerning the influence of the calculation conditions (time step, pluviometry of the period and demand scenario), it was the same as obtained for the reliability criterion, a part from some changes in the influence inside the variants of two factors (time step and pluviometry of the period).

3.2.4 Vulnerability Criterion

The analysis of the PCA-A of the matrix vulnerability calculated for six approaches under 18 combinations gave the results illustrated in Fig. 5a. The percentage of inertia overpassing 99% is formed essentially by dimension 1. Axis 1 is represented by the supplementary variable called “average vulnerability for each approach under different combinations influence” (“Average” in Fig. 5a, r = 1). Thus, the ascending order of approaches in function of their vulnerability is: Case 3, Case 2, Case 4, Case 1, Case 0 then Initial.

Fig. 5
figure 5

a Biplot variables-observations of a the PCA-A first plane of vulnerability; b the PCA-B first plane of vulnerability; c the PCA-A first plane of sustainability; d the PCA-B first plane of sustainability

Full size image

This ranking proves that the optimization procedure of both the releases and the FS helped to converge to a system less vulnerable to the augmentation of the water demand and to the change of pluviometry. Moreover, with the same releases as the Initial approach, Case 0 FS led to the reduction of the vulnerability of the BEO-AEO complex during periods of low pluviometry. The high performance of Case 0 compared to the Initial approach in terms of vulnerability and resilience, but not in terms of allocated and lost volumes shows the necessity to use several criteria to assess and compare management approaches.

Concerning the influence of calculation conditions on the value of vulnerability, Fig. 5b reveals the disposition of combinations in the first plane of the PCA-B. The plane permitted to represent 99.07% of the total variability on axis 1, symbolized here by the supplementary variable “average vulnerability provoked by each combination” (“Average” in Fig. 5b, r = 1). The distribution of combinations in the PCA plane allowed us to extract the following intriguing observations: (1) The vulnerability diminishes with the augmentation of the time step (from daily to annual); (2) The extremes values of vulnerability were manifested when we have used the normal water demand; (3) when the pluviometry decreases, the vulnerability follows. In reality, as we have mentioned in the section before, the Initial approach was primarily calibrated to address performance during dry periods. Thus, the current methods of determination of releases and FS volumes seem to be unable to adapt to a new context with more pluviometry. Consequently, instead of taking advantage of the supplementary pluviometry and decrease the deficits and therefore the vulnerability, the manager is obliged to make unvalued releases, in order to keep a useless activated FS during several months. This explains the substantial volumes lost using the Initial approach mentioned in Fig. 2b. On the other hand, the optimized approaches (Cases 0-4) were able to acquire this ability of adaptation to dry and wet periods, during the optimization process. Additionally, and due to the profound influence of the calculation conditions on the vulnerability value, we note the necessity to report the value of this criterion always associated with the conditions of its calculation.

3.2.5 Sustainability Criterion

Based on the sustainability matrix of six management approaches under 12 combinations of influencing factors (combinations leading to a zero value were omitted), a PCA-A was calculated (Fig. 5c). The resulting first PCA plane shows that the approaches are aligned along the axis 1 (84.32% of inertia), represented by the supplementary variable “range of the sustainability values for each management approach” (“Range” in Fig. 5c, r = 0.942).

Therefore, the approaches located to the right of axis 1 have a high range, and by consequent an instability of the criterion vulnerability when changing the calculation conditions. Indeed, if there is an instability of the value, then there is some form of failure that alters the performance of a given approach. According to that, the descending order of approaches depending on their sustainability is: Case 0, Initial, Case 3, Case 2, Case 1 then Case 4. The resulted ranking permits to notice that on one hand, the Case 0 outperforms again the Initial approach, which proves that the results obtained while comparing using allocated and lost volumes are questionable. On the other hand, the other optimized cases were outperformed by the Initial approach. This is intriguing since the optimized cases were better than the Initial approach in two of the three criteria composing the sustainability. The new performance ranking led us to assume the following hypothesis regarding the sustainability criterion: This criterion is more influenced by low values of its parameters (reliability, resilience and vulnerability) than by high values. Although the calculation formula seems to give the same weight to the three parameters, the low values of resilience of the optimized cases (1-4) compared to the Initial approach, appear to weight more than their high reliability and low vulnerability values. This should incite the manager to interpret carefully the value of sustainability outputted by a given management approach.

For the influence of factors combinations on the sustainability criteria, Fig. 5d resumes the results of the PCA-B. As it is observable, the first plane contains 99.58% of inertia, with dimension 1 which represents more than 97% of the sustainability variability. Like the PCA-B of the pre-discussed criteria, axis 1 is very correlated (r = 1) with the supplementary variable “average value of sustainability caused by each combination (“Average” in Fig. 5d).

By observing the combinations distribution along axis 1 of the PCA plane, we can remark that it copies perfectly the combinations distribution of the PCA-B of the resilience (Fig. 4d). This joins the hypothesis cited before claiming that the parameter of sustainability criterion that have the low value, possesses an implicit weight coefficient, which make the resulting value of sustainability follows it. Therefore, we can say that the change of the calculation conditions affects the sustainability and the resilience criteria in the same manner.

4 Conclusions

Overall, the presented results show that the optimized approaches converge to better performance than the Initial one. Indeed, among the six criteria tested, five indicate an outperformance of the approaches for which we have optimized both the FS and the releases (Cases 1-4). For the remaining criteria (resilience and sustainability), the Case 0 which optimized only the FS was able to give an efficiency more important than the current management approach of BEO-AEO. The comparison based on six criteria, and under 18 combinations of calculation conditions permitted to detect some aspects of failure of the Initial approach, such as: (1) the questionable management of the summer floods, the rainy period and the spring floods; (2) The instability of water allocations to users; (3) the incapacity to manage period with high pluviometry; (4) the reasons behind the importance of lost volumes from BEO-AEO; (5) and the pronounced instability of performance while changing the calculation conditions (pluviometry of the simulation period, time step and water demand scenario). Then, this should incite the manager of BEO-AEO to update the guidelines and the parameters of the current management approach.

Regarding the ability of the performance criteria to reflect with precision the efficiency of the studied hydraulic system, and the influence of calculation conditions on the values of those criteria, several conclusions and hypotheses were spotlighted. Firstly, the comparison of approaches according to one unique criterion converges very likely to mistaken evaluation. In fact, the order of efficiency of the approaches change brutally depending on the used criterion. This is partially related to the objectives that a system manager had fixed during the establishment of the management guidelines. For instance, if the OF he considered is oriented to the augmentation of the resilience, he can only guaranty that his approach could be classified first among the approaches according to the criterion resilience. This remark of performance order change in function of the criterion, can be mitigated by integrating several criteria in the OF, during the calibration process. Secondly, the sustainability criterion tends to follow strictly the parameter having the lowest value. In our case, in spite of the high positive values of reliability and vulnerability, the low values of resilience affected more the values of sustainability. Indeed, this implicit weight favoring the low performances was reported by Sandoval-Solis et al. (2010) when they discussed the formula suggested by Loucks (1997). So, they proposed to modify it into another formula that we have used in this analysis. Nevertheless, this modification did not permit to overcome this issue, which necessitates a special attention while interpreting the values taken by this criterion. A profound research is therefore recommended to adapt the sustainability calculation formula, in order to be able to integrate several criteria having each the same weight coefficient. Thirdly, we concluded that whatever the criterion used, it is required to report in addition to its value, its conditions of calculation (simulation period, time step, water demand scenario, …). In reality, the results show the change of the criteria values in function of the time step used (daily, monthly or annually), the pluviometry of the simulation period (low, moderate or high) and the scenario of the water demand (normal or high). Thus, every conclusion deducted from the value of a given criterion, for a given management approach, without taking into consideration the calculation condition, is a questionable conclusion. And finally, the use of the PCA to analyze the matrices resulting from the calculation of the values of each criterion, for each approach, and under various combination was very helpful. Indeed, with a matrix of 6*18 for each criterion (excepted the two first), the approaches comparison and the analysis of the effect of the calculation conditions on the performance criteria would be difficult. In addition, the convergence to conclusions like those we have pre-cited before was facilitated by the PCA. Therefore, we recommend to associate the PCA to the performance criteria matrices as a tool to analyze the results. By this way, each criterion will resume all the performance of a given approach under various conditions, and the PCA will resume the matrix of the obtained values of each criterion in one PCA plane facilitating the assessment and the comparison.