Reservoir-system simulation and optimization techniques

Abstract

Reservoir operation is one of the challenging problems for water resources planners and managers. In developing countries the end users are represented by the water sectors in most parts and conflict over water is resolved at the agency level. This paper discusses an overview of simulation and optimization modeling methods utilized in resolving critical issues with regard to reservoir systems. In designing a highly efficient as well as effective dam and reservoir operational system, reservoir simulation constitutes one of the most important steps to be considered. Reservoirs with well-functional and reliable optimization models require very accurate simulations. However, the nonlinearity of natural physical processes causes a major problem in determining the simulation of the reservoir’s parameters (elevation, surface-area, storage). Optimization techniques have shown high efficiency when used with simulation modeling and the combination of the two methods had given the best results in the reservoir management. The principal concern of this review study is to critically evaluate and analyze simulation, optimization and combined simulation–optimization modeling approach and present an overview of their utility in previous studies. Inferences and suggestions which may assist in improving quality of this overview in the future are provided. These will also enable future researchers, system analysts and managers to achieve more precise optimal operational system.

1 Introduction

The operation and management of a reservoir system is a difficult problem for decision makers and reservoir operators. The decision process that identifies the operations of any system is complicated by conflicts existing among the various purposes for the reservoir. Usually, reservoirs are designed and built to serve many purposes, including supplying enough water for agriculture, industrial and municipal demands, generating hydroelectric power, protecting cities and towns from damages associated with floods, improving river transportation, navigation, and recreation. The stochastic nature of reservoir inflows adds greatly to the complexity of the problem (Taghi et al. 2009). In cases where real time operating policies are required, the decision makers face an even greater challenge in handling updated reservoir inflow forecasts. Simulation and optimization are the two powerful tools for reservoir system analysis. A simulation model is a representation of the system used to predict the behavior of the system under a given set of conditions. Alternative executions of a simulation model are made to analyze the performance of the system under varying conditions, such as for alternative operating policies. Optimization refers to a mathematical formulation in which an algorithm is used to compute a set of decision variable values that minimize or maximize an objective function subject to constraints. Whereas simulation models are limited to predicting system performance for a user-specified set of variable values, optimization models automatically search for an “optimal” solution. Although simulation and optimization are two alternative modeling approaches with different characteristics, the distinction is somewhat obscured by the fact that most models, to various degrees, contain elements of both approaches. For the past three decades many scientists and decision makers have been interested in deriving optimal operating policies for reservoir systems. Many studies have been conducted to derive and evaluate these optimal operating policies utilizing a wide variety of optimization techniques for solving different reservoir operation problems. The techniques employed include linear programming (LP) (Houck and Cohon 1978; Marino and Mohammadi 1983; Datta and Burges 1984; Datta and Houck 1984), nonlinear programming (NLP) (Simonovic and Marino 1980; Lall and Miller 1988; Oron et al. 1991), dynamic programming (DP) (Bras et al. 1983; Stedinger et al. 1984; Kuo et al. 1990; Huang et al. 1991), optimal control theory (Grygier and Stedinger 1985; Papageorgiou 1985; Georgakakos and Marks 1987), network analysis (Sabet and Creel 1991) and other artificial intelligence techniques (Wardlaw and Sharif 1999; Neelakantan and Pundarikanthan 2000) for solving reservoir operation problems. Advantages and disadvantages of each optimization technique have been assessed through various case studies of (Gablinger and Loucks 1970; Roefs and Guitron 1975; Karamouz and Houck 1987; Wurbs 1993; Philbrick and Kitanidis 1999; Tilmant et al. 2002a, b). Yakowitz (1982) reviewed exclusively DP models used in water resource systems engineering, while Yeh (1985) provided a broader review of reservoir management and operation models including LP, DP, NLP and their applications in reservoir operation. Yeh (1985) also investigated the role of simulation models, and how they differ from the optimization approaches. Various simulation models and real time operation models used in water resource problems and their applications were presented in this review. These reviews (Yakowitz and Yeh) demonstrated the breadth of applications for these optimization models, as well as the variety of available model formulations which could be used to improve the operation and management of reservoirs. Wurbs and Tibbets (1985) presented an annotated bibliography for various optimization and simulation models and listed several applications made by various researchers in the reservoir operation problems using LP, DP, NLP, and simulation models. Wurbs (1993) reviewed the reservoir system simulation and optimization models. This review was done with the objective to contribute to ongoing efforts throughout the water management community in sorting through the numerous reservoir system analysis models. Simonovic (1992) presented a short review of mathematical models used in reservoir management and operation. This review was intended to present conclusions reached by previous state-of-the-art reviews and to provide ideas for closing the gap between theory and practice. Simonovic (1992) presented a simple simulation optimization model for reservoir sizing and showed how it responded to practical needs of water resource engineers. There are some features that complicate the process of finding solutions to computational problems in operation modes of a reservoir system. Some of these are uncertainty, commensurate or non-commensurate multi objectives and the presence of nonlinear functions, such as hydropower generation, evaporation and other losses. The problems concerning real time reservoir operation models were analysed by Datta (1993). He discussed that improvement in optimization algorithms, large scale simulations, geological information processing and use of artificial intelligence techniques together with the enormous ground-breaking achievements in computer technology will eventually contribute to the elimination of these complexities. In reservoir systems, the principal cause of uncertainty is the inconsistent inflows which show transient (within years or months) and spatial (caused by topographical and geological phenomena) differences. In his study on water resources, Simonovic (2000) critically discussed the uncertainty phenomenon and suggested some meaningful water management system that may be utilized to correct some of the anomalies being experienced presently. Uncertainties in availability and demand for water are some of the possibilities of changes in the climatic conditions of an area; this may end up with a huge adverse effect economically and environmentally. In addition, uncertainty is regarded as an important factor to be considered when designing reservoir system models because of the role it plays in the availability of water for domestic, agricultural, or industrial uses (Ming et al. 2012). Generally, a reservoir system is a multi-purpose tool designed to cater for different operations such as water supply (domestic, industrial and irrigation), flood control, hydropower production etc. All these circumstances and situations must therefore be put into consideration during its design and planning stages (Ko et al. 1992). In the light of the impact of global warming and climate change, the earlier research raised a question of whether climate variations are large enough to be taken into account in management of water systems (Nemec and Schaake 1982). Klemes (1985) performed an assessment of the anticipated sensitivity of water resource systems to climatic variations. His main finding was that the decrease in reliability might occur much faster than any decrease in precipitation or increase in evapotranspiration losses. Kaczmarek (1990) examined the possible implications of an altered hydrologic regime on the operation of the Kariba dam and reservoirs, a large water storage system in southern Africa. Burn and Simonovic (1996) focused on the sensitivity of reservoir operations to changes in the hydrologic regime resulting from different climatic conditions. Two different climate scenarios were used in the analysis of an actual reservoir system that reflects conditions corresponding to a ‘cool’ and a ‘warm’ climatic regime. Yao and Georgakakos (2001) developed an integrated forecast-decision system for Folsom Lake (California) and assessed the reservoir performance using future ensemble forecasts. They demonstrated that adaptive decision systems can benefit reservoir performance and dynamic operational procedures can effectively cope with climate change impacts. Christensen et al. (2004) investigated the potential effects of climate change on the hydrology and water resources system for the Colorado River basin and demonstrated climate change may significantly degrade the performance of the system in water resources management relative to historical conditions. Payne et al. (2004) examined several alternative reservoir operating policies to investigate impacts of climate change for the Columbia River basin. The result of this study was that the combination of earlier reservoir refill with greater storage allocations mitigated some negative impacts on the system. Sahoo et al. (2011) clearly states that climate change is occurring and is almost certain to have anthropogenic origins. In attempting to mitigate and adapt to the effects of climate change, it is therefore necessary to understand the potential effects of climate change on all elements of complex water resources systems and develop appropriate approaches for support of both mitigation and adaptation measures. A common approach to understanding and managing climate change impacts on water resources systems involves the use of mathematical modelling. The impacts of reservoir operations under climate change have been investigated for small reservoirs (Krol et al. 2011) and multi-purpose reservoirs (Raje and Mujumdar 2010) to estimate water availability for current and possible future conditions. Loss of water through evaporation and seepage from reservoirs constitutes a bottleneck in the design stage. This requires additional effort in terms of computation of mathematical models for the problem to be brought under control. Teixeira and Mariño (2002) warned that great attention must be accorded the process of evaporation because of its importance as a foremost factor in the design and planning process as well as the eventual operations of successful reservoir systems. Booker and O’Neill (2006) showed that the size of a reservoir is a determinant of the rate and extent of losses accruable from reservoirs. Reducing this losses to the barest minimum may entail the application of certain operational packages or guidelines, e.g., a typical drought storage rule (Lund 2006); this involves the storage of water in one out of the many available spaces in order to reduce the avenues through which the evaporation takes place. Reservoirs are subject to sedimentation processes which diminish the utilizable reservoir volume. Early work on the impact of sedimentation in reservoirs was conducted by Mahmood (1987), who estimated that a reduction in worldwide reservoir capacity was in the order of 1 % per year. This implies that within about 50 years, the world’s water storage reservoirs will be half of the current storage, which will have large economical and environmental consequences. Oehy and Scheleiss (2007) reported that the annual volume diminution in existing reservoirs is higher than the storage increase achieved through construction of new reservoirs. In this context, sediment management in catchment areas and reservoirs play a key role as previously indicated by Mahmut and Mahmud (2010), Zhu (2012). In order to improve our understanding of the complex phenomena involved in reservoir sedimentation processes, researchers have pursued several theoretical, laboratory, and numerical modeling approaches. Some of these research efforts were focused on reservoir deposits (such as deltas) and turbidity currents. Studies related to delta evolution were performed by Yang et al. (2005). Garcia (1994) studied turbidity currents with poorly sorted sediment and sediment deposition in reservoirs. Sediment bypassing is a method used to manage sediment by preventing it from entering the reservoir. An offstream reservoir can generate benefits in addition to a reduced rate of storage depletion. Empty flushing involves the opening of bottom outlets to completely empty the reservoir and allow stream flow to scour sediment deposits. The scouring of sediment by flood depends on the permeability, consolidation coefficient and the volume fraction of sand related to silt (Jacobs et al. 2007). Therefore, it is of utmost importance to predict sediment yield at the basin scale and understand which factors determine the sedimentation rate of reservoirs. This knowledge will allow to estimate the probable lifespan of a reservoir and moreover to take proper measures against reservoir sedimentation, water shortage, river bank and coastal erosion.

The main goal of this review is to examine the established techniques of simulation, optimization and combined simulation–optimization modeling that are in use in various reservoir system operations or planning studies. Furthermore, apart from the classical optimization system, other methods used in the reservoir system operation studies which emphasize on the application and scope of computational intelligence, such as, evolutionary computations, fuzzy set theory and artificial neural networks are also evaluated. This work also considered some of the challenges faced in the operations system such as: nonlinearity, uncertainty and multi objectivity. The modifications made in some of the existing techniques are also presented. Attempt was also made to review separately pure simulation and combined simulation–optimization methods utilized in reservoir systems modeling. At the end of the whole appraisal, some suggestions were made for the purpose of future works on this aspect of reservoir system operations.

2 Reservoir system optimization problem

2.1 Objective function

The Objective functions used in reservoir system optimization models should incorporate measures such as efficiency (maximizing current and future discounted welfare), survivability (assuring the future welfare exceeds minimum subsistence levels), and sustainability (maximizing cumulative improvement over time). Loucks (2000) states that “sustainability measures provide ways by which we can quantify relative levels of sustainability. One way is to express the relative levels of sustainability as separate weighted combinations of reliability, resilience and vulnerability measures of the various criteria that contribute to human welfare and that vary over time and space. These criteria can be economic, environmental, ecological, and social.” The strategy of shared vision modeling (Palmer 2000) is useful for enhancing communication among impacted stakeholders and attaining consensus on planning and operational goals. A generalized objective function for reservoir system optimization can be expressed as

$$ \hbox{max} \;({\text{or}}\;\hbox{min} )\sum\limits_{{\text{t}} - 1}^{T} {{{\upalpha}}_{\text{t}} {\text{f}}_{\text{t}} ({\text{s}}_{\text{t}} ,{\text{r}}_{\text{t}} ) + {{\uprho}}_{{{\text{T}} + 1}} )({\text{s}}_{{{\text{T}} + 1}} )} $$

(1)

where r_t = a set of control or decision variables (releases from reservoirs) during period t; T = length of the operational time horizon; s_t = is the state vector of storage in each reservoir at the beginning of period t; f_t(s_t, r_t) = objective to be maximized (or minimized); ρ_T+1(s_T+1) = final term representing future estimated benefits (or costs) Beyond time horizon T; and α_t = discount factors for determining present values of future benefits (or costs). The dynamic nature of this problem reflects the need to represent an uncertain future for sustainable water management; i.e., “… a future we cannot know, but which we can surely influence” (Loucks 2000). The time step t used in this formulation may be hourly, daily, weekly, monthly, or even seasonal, depending on the nature and scope of the reservoir system optimization problem. Hierarchical strategies may also be pursued whereby results from long-term monthly or seasonal studies provide input to more detailed short-term operations over hourly or daily time periods (Becker and Yeh 1974).

2.2 Constraints

The system dynamics or state-space equations are written as follows, based on preservation of conservation of mass throughout the system, the mass balance in the reservoir from one time step to another time step is given by the continuity equation. The continuity equation is stated as:

$$ {\text{S}}_{{{\text{t}} + 1 }} = {\text{S}}_{\text{t}} + {\text{Q}}_{\text{t}} - {\text{R}}_{\text{t}} - {\text{E}}_{\text{t}} - {\text{Ot}} $$

(2)

where, S_t+1 = final storage at the time t, Q_t = inflow during the time t, R_t = optimal release from the reservoir during time period t, E_t = evaporation loss in the reservoir during time t, O_t = surplus from the reservoir during time t.

2.2.1 Release constraints

The release from the reservoir should be less than or equal to the estimated target release. Mathematically target release is the sum of irrigation and other demand (municipal, industrial and hydropower generation), and this constraint is given by:

$$ 0 \le {\text{R}}_{\text{t}} \le {\text{TR}}_{\text{t}} $$

(3)

TR_t = target release during time period t.

2.2.2 Storage constraints

The reservoir storage in any month should not be more than the capacity of the reservoir, and should not be less than the desired target storage. Mathematically this constraint is given as:

$$ {\text{S}}_{\hbox{min} } \le {\text{S}}_{\text{t}} \le {\text{S}}_{\hbox{max} } $$

(4)

where S _max = capacity of the reservoir in million cubic meters.

2.2.3 Reservoir evaporation loss constraints

Reservoir evaporation loss during any time period t is given as the product of evaporation rate and the average water spread area at the beginning and at the end of time period (Loucks et al. 1981). Based on this, the evaporation loss during any time period t is given by:

$$ {\text{E}}_\text{t} = {\text{A}}_{\text{d}} {\text{e}}_{\text{t}} + {\text{me}}_{\text{t}} \left( {\frac{{{\text{SA}_\text{t}} + {\text{SA}_\text{t}} + 1}}{2}} \right) $$

(5)

where E_t = evaporation loss during time period t, which is computed as the function of the water spread area corresponding to the active initial and final storage and evaporation rate, e_t = the evaporation rate in mm/month during time period t, A_d = water surface area at the top of the minimum storage level (m²), M = the slope of the straight line with water surface area against reservoir active storage, SA_t = the active initial storage at the beginning of time period t, SA_t+1 = the active final storage at the end of time period t.

2.2.4 Over flow constraints

This constraint takes care of the situation, when the final storage exceeds the capacity of the reservoir. Mathematically this constraint is given by:

$$ {\text{O}}_{\text{t}} = {\text{S}}_{{{\text{t}} + 1 }} - {\text{S}}_{\hbox{max} } $$

(6)

and O_t ≥ 0, O_t = surplus from the reservoir during time t.

3 Implicit stochastic optimization

The implicit stochastic optimization (ISO) methods referred to Monte Carlo optimization, which optimize over a long continuous sequence of historical or synthetically generated unregulated inflow time series, or numerous shorter equally likely sequences (Fig. 1). In this way, most stochastic aspects of the difficulty, including spatial and temporal correlations of unfettered inflows, are implicitly included and deterministic optimization methods can be straight applied. The main disadvantage of this approach is that optimal operational policies are unique to the assumed hydrologic time series. Attempts can be made to apply multiple regression analysis and other methods to the optimization results for developing seasonal operating rules conditioned on observable information such as current storage levels, previous period inflows, and/or forecasted inflows. Unfortunately, regression analysis may result in poor correlations that invalidate the operating rules, and attempting to infer rules from other methods may require extensive trial and error processes with little general applicability.

Fig. 1

Implicit stochastic optimization (ISO) procedure

3.1 Linear programming

This refers to a process where the most viable obtainable value is considered out of the various available linear inequalities with respect to a particular situation. This has been used in various reservoir systems cases of diverse dimensions and goals such as: determining optimal operating policies Crawley and Dandy (1993), sizing of reservoir capacities (Loucks et al. 1981), yield assessment (Dahe and Srivastava 2002), flood control (Needham et al. 2000) and conjunctive use planning (Vedula et al. 2005). The greatest assets of the methods are the dynamic nature of the package towards large scale cases, convergence to global optimal solution, and easily obtainable effective software packages. The major demerits of LP are the inability to use linear and convex objective functions as well as linear constraints. However, the problem of nonlinearity in some reservoir systems (e.g., nonlinear benefit or cost functions) may be resolved by approximating and extending LP to separable LP (Crawley and Dandy 1993) and successive LP (Mousavi and Ramamurthy 2000; Barros et al. 2003). Non convexity cases may equally be resolved by using other LP extensions such as binary, integer and mixed integer LP (Randall et al. 1997; Tu et al. 2003). All the LP and its extensions referred to here may be categorized as deterministic or ISO methods. Explicit stochastic forms of LP methods include stochastic LP (SLP) and chance constrained LP (CCLP). The SLP solution gives optimal steady state probabilities for releases and storages given that there are inflows which can be followed in a single Markov chain. The solution may not necessarily give steady state releases as a function of storages but rather steady state probabilities of releases and storages. These probabilities and their transformation into releases are then adapted for practical usage in operation. For further detailed information on this aspect, interested parties may consult (Loucks et al. 1981). A CCLP approach for a diverse purpose reservoir through which the highest annual hydropower produced while meeting irrigation demand at a defined reliability level is accessible in (Sreenivasan and Vedula 1996; Lee et al. 2006; Ganji and Jowkarshorijeh 2012) a two stage SLP approach to solve the problem of inflow uncertainties is applied. The coordinated multi-reservoir operation model was designed to find optimal monthly target reservoir storage and the LP solver CPLEX was used in solving it. The stochastic approach proved more useful than the deterministic one with average inflows when the two were compared (Li et al. 2009).

3.2 NLP

Many reservoir system optimization problems cannot be realistically modeled using piecewise linearization, and must be attacked directly as NLP problems, particularly with inclusion of hydropower generation in the objective function and/or constraints (Gu et al. 2012). The presence of nonlinearity in many reservoir systems operational cases may be due to the ambiguous relationships that exist between the different physical and hydrological variables or because of identified goals meant for the system. Ideally, problems relating to hydropower issues are nonlinear in nature and their solution could be very difficult to attain at times. However, approximation of nonlinear problem to linear problem or the continuous application of LP as previously discussed can be used in solving such problems. In addition to this, DP may also be used to address problem of nonlinearities. On the contrary, there is a group of cases whose solutions can only be found through the use of NLP methods. These programs include successive or sequential quadratic programming (SQP) and the generalized reduced gradient (GRG) approach. Presently, there exists various general purpose software programming that can be used to solve multi dimension nonlinear optimization problems, for instance; LINGO, LANCELOT (Conn et al. 1992), MINOS (Murtagh and Saunders 1998), FSQP (Zhou et al. 1998), GALAHAD (Gould et al. 2003) and LOQO (Vanderbei 2006). These programmings are used elaborately in finding solutions to various complex problems including reservoir systems operation involving hydropower generation; however, an ideal situation is yet to be accomplished in practical applications of NLP. A mixed integer NLP formulation (Teegavarapu and Simonovic 2000) with binary variables was utilized in evaluating daily hydropower operation of four cascading reservoirs. A NLP model (Barros et al. 2003) designed and used on a multi-purpose Brazilian hydropower system. Besides the use of MINOS by NLP solution, LP and successive LP were also used to linearize and solve the NLP models. The vantage position of the NLP model over LP and successive LP reflected in the operational steps for real time operation. The use of NLP to cover stochastic cases are however not common Labadie (2004) as a result of critical computational prerequisites and a quite a few applications are mentioned in previous studies (Ahmed and Lansey 2001). Meanwhile, it can be seen that of late, the use of heuristic approaches in lieu of NLP has witnessed increased patronage as both can interchangeable be used to handle cases pertaining to both nonlinearity and uncertainty.

3.3 DP

Dynamic programming, a method formulated largely by Bellman in his book entitled “DP” in 1957, is a procedure for optimizing a multistage decision process, making it suitable for reservoir operation. DP is used extensively in the optimization of water resource systems. The popularity and success of this technique can be attributed to the fact that nonlinear and stochastic features which characterize most reservoir systems can be directly incorporated within a DP formulation. In addition, it has the advantage of effectively decomposing highly complex problems with a large number of variables into a series of sub-problems which can be solved recursively as shown in Fig. 2. Water resources have even been proven to be a good field of study for DP theory, as stated by Yakowitz (1982), Nandalal and Bogardi (2007) present applicability and limits of DP methods, specifically in reservoir operation problems. DP models can be classified into a number of categories, depending on how the solution space is discretized and the extent and method used to incorporate the stochastic nature of the reservoir system. One way to classify models proposed for solving reservoir operation problems is by how they characterize the inflow process. One group of models, termed deterministic models, uses a specific sequence of inflows, either historical or synthetically generated, to determine optimal operating rules. Another group of models, termed stochastic models, uses a statistical description of the inflow process instead of specific inflow sequences.

Fig. 2

Illustration of reservoir system optimization as sequential decision process

3.3.1 Deterministic DP

Deterministic DP models incorporate specific inflow sequences directly into DP formulation to determine the optimal operating policy. Published studies of deterministic reservoir applications can be traced as far back as Young (1967), who studied a finite horizon, single-reservoir operation problem. In Young (1967) model, the control variable R_t represents the average release of water during time period t, and the state variable S_t gives the reservoir storage at the beginning of time period t, C denotes the reservoir capacity. The optimal control problem dynamics are specified by a mass balance equation:

$$ {\text{S}}_{{{\text{t}} + {\text{l}}}} = \hbox{min} ({\text{C}}, {\text{S}}_{\text{t}} + {\text{Q}}_{\text{t}} - {\text{R}}_{\text{t}} ) $$

(7)

where Q_t represents the inflow during time period t, which, in the deterministic case, must be presumed known. Under this hypothesis, the inflows are taken to be the expected inflows during the portion of the annual cycle corresponding to time period t. The operating policies generated by this deterministic approach are sub-optimal since they are associated with each inflow sequence. Regression analysis is often chosen to obtain optimal policies that reflect the overall performance of the system. Young (1967) derived operating rules using simple linear regression or multiple lineal-regressions from the deterministic optimization rules. He derived regression equations using inflows and storages to find the optimal release. Bhaskar and Whitlatch (1980) analyzed a single multipurpose reservoir using the backward DP formulation to obtain optimal results. They used the procedure of deriving general operating policies from deterministic optimization as initiated by Young (1967), Bhaskar and Whitlatch (1980) considered a quadratic loss function and derived monthly policies by regressing an optimal set of releases on the input and state variables. Karamouz and Houck (1982) developed reservoir system operating rules by deterministic optimization using the DP-multiple linear regression (DPR) model. The DPR model algorithm had a deterministic dynamic program with a regression analysis. This algorithm also had a simulation model to assess the performance. By considering the recommendations of Bhaskar and Whitlatch (1980), Karamouz and Houck (1982) used a multiple linear regression form in which they regressed the optimal release R_t as a function of initial storage S_t and inflow during period Q_t given as:

$$ {\text{R}}_{\text{t}} = {\text{aQ}}_{\text{t}} + {\text{bS}}_{\text{t}} + {\text{c}} $$

(8)

where a, b, and c are regression coefficients. Karamouz and Houck (1982) analyzed the DPR model performance with case studies, and suggested that the model can be extended easily to more complex and more practical problems. Since specific inflow sequences are used in the deterministic DP formulation and inflow is not included in the deterministic DP formulation as a state variable, deterministic DP has computational advantages over stochastic model formulations. However, ignoring the stochasticity of the system to be considered not only simplifies the model but introduces bias, as described by Loucks et al. (1981). In contrast to the models yielding only “optimal releases” within a fixed deterministic framework, stochastic DP is more attractive (Yakowitz 1982; Yeh 1985).

4 Explicit stochastic optimization

Explicit stochastic optimization (ESO) is planned to operate directly on probabilistic descriptions of arbitrary streamflow processes (as well as other random variables) rather than deterministic hydrologic sequences. This means that optimization is performed without the assumption of perfect foreknowledge of future actions. In addition, optimal policies are determined without the need for inferring operating rules from results of the optimization (Fig. 3). Unhappily, ESO techniques as applied to multi-reservoir systems are more computationally demanding than ISO, as recognized early by Roefs and Bodin (1970). In this case, unfettered inflows are assumed the main source of uncertainty and can be represented by suitable probability distributions. These may be parametric or nonparametric based on frequency analysis. Other arbitrary variables that may be defined include economic parameters in the objective function, demands, and climatological variables impacting net evaporation and other losses. Unfettered inflows may be highly interrelated spatially and/or temporally. For short-term operational problems, inflows may be generated from forecasting models, in which case the main source of uncertainty is the predict error.

Fig. 3

Explicit stochastic optimization (ESO) procedure

4.1 Stochastic DP

Stochastic dynamic programming (SDP) is an extension of DP in which the stochastic nature of the inflows, are considered explicitly in the optimization. This explicit consideration of the stochastic nature of the inflows is achieved by using the returns associated with each of the possible states (normally storage and inflow) in a particular time period, multiplied by the probability of the inflows that give rise to them, to generate the expected value of the objective function. This probability is often termed the transition probability, reflecting the correlation between two successive inflows. The optimal operating policy designated by the SDP formulation is a kind of “look up table” indicating the optimal decision (the ending storage S_t+1 or the release R_t for the time period t) under different initial state status. The stochastic dynamic programming (SDP) model was first used by Little (1955) in an example for a dam on the Columbia River. This particular example was extremely simple and it was recommended that further work should be undertaken. Subsequently, Buras (1966) used the method in a study of the optimal conjunctive operation of dams and aquifers. Loucks and Falkson (1970) compared three different techniques used in solving reservoir operation problems, namely Linear Programming, Stochastic Dynamic Programming and Policy Iteration. Optimal operating policies were derived for a single multi-purpose reservoir where the inflows into the reservoir were assumed to be an order one Markov process. The reservoir storage targets and release targets were assumed predetermined. The policy defined the release as a function of current reservoir storage and inflow. In the linear programming model, only steady state joint probabilities were obtained and steady state policies were derived indirectly from these joint probabilities. In the SDP solution the objective function was to maximize the expected benefits. The DP model defined both transient and steady state operating policies directly, while joint probabilities for those stationary states had to be calculated indirectly. For the policy iteration technique, an arbitrary policy was defined and expected returns associated with this policy were calculated. Starting from that policy, another alternative policy could be chosen and the returns calculated. The iterations proceeded until the policies in two successive iterations were identical and the optimal policy was then defined. The set of equations to be solved by the policy iteration had to be linear simultaneous equations. A numerical example was solved and it was concluded that policies obtained from the three techniques were identical. SDP took less time than policy iteration and both of them yielded directly the transient and steady state policies. The LP did not require writing a computer code or debugging because LP code is readily available, but it took a greater amount of computational time in comparison with SDP. Steady state policies were derived but the transient policies were not determined. Usually, only single reservoir problems can be solved by LP because of limitations in computer capacity and accuracy at that time. Butcher (1971) adopted the same model (including the order one Markov process) as Little (1955). He noted that if the terminal decision time is far enough into the future, then (at least for finite state optimal control problems) one obtains a stationary optimal policy. Butcher (1971) applied discrete SDP to find the optimal stationary policy for operating the Watasheamu Dam on the California-Nevada border. The same problem was formulated in LP by Young (1967), Loucks (1968)) solved it using Monte Carlo simulation with many sets of synthetically generated inflows. Due to its versatility, the SDP technique was also used by Torabi and Mobasheri (1973), Su and Deininger (1974), Loucks et al. (1981), Bras et al. (1983), Stedinger et al. (1984), Buras (1985), Goulter and Tai (1985), Trezos and Yeh (1987), Kelman et al. (1990), Huang et al. (1991), and many other investigators to study various reservoir systems. Fuzzy theory has been used within SDP to solve problems with respects to discretization (Fu et al. 2012), i.e. a situation where the capacity of the reservoir system is divided into various compartments as variables which may assume fuzzy numbers. Thus, there exist smooth passage between the compartments rather than in a single valued point in a unit storage position; this process depicts the real transition between the storage volumes. The SDP approach is referred to as fuzzy-state SDP, which seems to be more compatible with nonfuzzy models dealing with discretization (Mousavi et al. 2004a, b). This approach was used by Mousavi et al. (2004a, b) for optimizing operation of a reservoir. Simulation of optimal release policies derived from fuzzy SDP with coarse partitions and crisp SDP model with finer partitions also did well; though, the lower enthusiasm of fuzzy SDP to the extent of discretization was found to have favored multi reservoir SDP models. Neurodynamic programming (NDP) proposed by Bertsekas and Tsitsiklis (1996), has the merit of solving highly nonlinear cases, while providing templates for dealing with high dimensional SDP by approximating the Bellman function (return or cost-to-go function) through artificial neural networks. Castelletti et al. (2007) elaborated on the NDP approach to solving the problems attached to multipurpose water reservoir network management. The result of their study showed that NDP greatly reduces the computation time and the necessary memory needed to store the Bellman functions, with respect to SDP. The initial findings from the authors through the use of the NDP approach was encouraging, though they still gave room improvement through further studies, especially regarding efficient sampling of the discretized state space where the most efficient approximation of Bellman function can be obtained as well as the reduction of the time needed for learning the artificial neural networks approach.

5 Computational intelligence (CI)

This refers to a set of nature instigated computational mechanisms and means of approaching complicated cases of real life situation of which the conventional approaches such as first principles, probabilistic, black-box, etc. may not have answers for. Basically, this class of methods includes three basic methods: Fuzzy logic systems, Neural Networks and evolutionary computation. Furthermore, such methods as Swarm intelligence and Artificial immune systems seen as subsets of evolutionary computation as well as Chaos theory and Multi-valued logic, also seen as offshoot of Fuzzy Logic System which are all methods that evolved from or revolve around the initial three basic methods can equally be categorized as components of CI.

5.1 Evolutionary computation

The evolutionary computation (EC) mechanism was initially utilized to take care of certain categories of complexities, such as, multi objectives, uncertainty, nonlinearity, discontinuity and discreteness which restrain the usability of analytical optimization methods in reservoir systems optimization. EC methods, otherwise known as heuristic search tools comprises of metaheuristic optimization algorithms such as: evolutionary algorithms (EA), made up of genetic algorithms (GA), evolutionary programming, evolution strategy and genetic programming; swarm intelligence, which consist of ant colony optimization (ACO) and particle swarm optimization (PSO) as well as simulated annealing (SA). In the recent times, research activities which have shifted towards multi-objective optimization had focused on multi-objective optimization methodologies based on EC (Ranjithan 2005) because, these techniques are known for supporting effective search for Pareto optimal solutions to a multi-objective optimization case. Apart from this, a unique feature of the reservoir systems optimization; uncertainty conditions, is easily resolved by the exceptional natural structure of EC based mechanism.

5.1.1 Genetic algorithms

A certain group of stochastic search algorithms whose working mechanism is based on natural evolution principals taken from Darwin theory is referred to as genetic algorithm. It is on the basis of this theory that inference is made of certain population where individuals who have more fitness adapt more into the system and they survive while those with less fitness adapt less and the chances of their survival are very minimal and this may lead to eventual extinction. Consequently, with time, the number of those who can fit into the system grows thereby increasing the population and this ultimately improves the system. The fact that stronger individuals survived and went ahead to form the population, the off springs of such individuals will inherit strong traits and this will lead to a stronger new generation. In addition, these traits will be passed down through generations. Wardlaw and Sharif (1999) examined GA formulation to diverse reservoir operation cases problems in line with a range of sensitivity analysis involving the use of various combinations of parameter settings (binary, grey and real-valued coding), crossover and mutation probability. Also, they used GA to optimize multi-reservoir system (Sharif and Wardlaw 2000) and their findings found to be comparable with DDDP. Jothiprakash and Shanthi (2006) designed a GA model to determine optimal operational mechanisms for a single reservoir, and the result showed that GA can be a replacement for real time operation. The main attraction of GA over other gradient search procedures is that they are more likely to get near global optimal or global optimal solutions because GA searches for optimum with a population of solutions instead of a single solution. Besides this, another attractive feature of the technique is its capability to deal with different problems using the same computer code except for little and easy problem specific modifications. However, a lot of researchers concluded that GA has the ability of replacing SDP. In addition, many authors have argued in favor of real coded (or floating point) GA as having an edge over binary coded GA Michalewicz (1999). In the former, it was pointed out that there is no discretization of decision variable space. Efforts have been made by many researchers to appraise the efficiency of both GA methods with respect to reservoir systems optimization. Chang et al. (2005a, b, c) examined the two methods and declared that real coded GA is better than binary coded GA because the former was more efficient and faster than the latter. Chen and Chang (2007) put forward a real coded hypercubic distributed GA (HDGA). In northern Taiwan where this approach was applied on a multi-reservoir system, it was observed that HDGA is capable of delivering a better performance than the traditional GA. To bring down the computational requirements of GA, efforts were made to use GA in conjunction with other optimization methods. Cai et al. (2001) used a combined GA-LP mechanism to find solution to large nonlinear reservoir systems optimization model. GA was utilized in linearizing the initial problem in each time period which was eventually and gradually resolved through the usage of LP. Good approximate solutions were found for nonlinear models through the hybrid GA-LP method. Due to the computational merits of combined GA-LP methods in solving problems of nonlinearities, Reis et al. (2006) suggested and used a stochastic hybrid GA-LP method to design the operation of reservoir systems, the set-up made use of series of future inflow variability through a treelike structure of synthetically generated inflows.

5.1.2 Simulated annealing

Simulated annealing (SA) is a stochastic search method that operates on the principles of classical physics annealing procedure. This approach was introduced by Kirkpatrick et al. (1983) and it was meant to be an efficient replacement for GA with respect to many optimization problems. Teegavarapu and Simonovic (2002) had the privilege of being the first researchers to use this method in reservoir systems operation problems. By checking with the results of mixed integer NLP, it was observed that SA was better with respect to better objective function value and less computational time requirements. In a separate study, (Georgiou et al. 2006) investigated optimal irrigation reservoir operation where the outcomes of usage of SA were further modified by a stochastic gradient descent algorithm. The researchers suggested that the intended optimization method may be applied in global crop area and optimal irrigation planning. A modified GA–SA algorithm was presented by Chiu et al. (2007). Researchers observed that modified GA–SA carry out other analyses whose functions include increasing the possibility of detecting a most efficient solution and simultaneously reducing the time needed for the computation.

5.1.3 Ant colony optimization

Ant colony optimization (ACO) is a meta heuristic optimization algorithm based on the principle of autocatalysis as learned from ants behavior proposed by Dorigo et al. (1996) for the solution of discrete combinatorial optimization problems such as the traveling salesman problem (TSP) and the quadratic assignment problem (QAP). The method has been shown to outperform other general purpose optimization algorithms including GAs when applied to a number of benchmark combinatorial optimization problems. ACOAs are now being used more frequently to solve optimization problems other than those for which they were originally developed. Application of ACOAs, however, to water resources problems is of recent origin. Abbaspour et al. (2001) used the ACO algorithm for estimating the unsaturated soil hydraulic parameters. Zecchin et al. (2003) compared the performance of original ant system (Dorigo et al. 1996) with that of Max–Min Ant System (MMAS), a modified version of the ant system proposed by Stutzle and Hoos (2001), for optimization of water distribution networks. Simpson et al. (2001) discussed the selection of parameters used in the ACO algorithm for pipe network optimization problems. More recently, (Maier et al. 2003) compared the performance of the ACO algorithm with that of GAs for the optimization of water distribution networks. Afshar (2005) proposed a new transition rule for ACO algorithms using elitist strategies and applied the method to pipe network optimization problems. That method was shown to overcome the premature convergence problem encountered by elitist ACO algorithms while improving the convergence characteristics of the algorithms compared to alternative methods such as MMAS.

5.1.4 PSO

Particle swarm optimization (Kennedy and Eberhart 1995) is a simple model of social learning whose emergent behavior has found popularity in solving difficult optimization problems. The initial metaphor had two cognitive aspects, individual learning and learning from a social group. Within the PSO algorithm, every solution is a bird of the flock and is referred to as a particle: in this framework the birds, besides having individual intelligence, also develop some social behavior and coordinate their movement towards a destination (Shi and Eberhart 1998). Initially, the process starts from a swarm of particles, in which each of them contains a solution to the hydraulic problem that is generated randomly, and then one searches the optimal solution by iteration. The i-th particle is associated with a position in an s-dimensional space, where s is the number of variables involved in the problem; the values of the s variables which determine the position of the particle represent a possible solution of the optimization problem. Each particle i is completely determined by three vectors: its current position Xi, its best position reached in previous cycles Yi, and its velocity Vi:

$$ {\text{Current}}\;{\text{position}}\;{\text{Xi}} = \left( {{\text{x}}_{{{\text{i}}1}} ,{\text{x}}_{{{\text{i}}2}} ,. . .,{\text{x}}_{\text{is}} } \right) $$

(9)

$$ {\text{Best}}\;{\text{previous}}\;{\text{position}}\;{\text{Yi}} = \left( {{\text{y}}_{\text{i1}} ,{\text{y}}_{{{\text{i}}2}} ,. . .,{\text{y}}_{\text{is}} } \right) $$

(10)

$$ \text{Flight} \; \text{velocity}\;\text{Vi} = \left( {\text{v}_{\text{{i1}}} ,\text{v}_{\text{{i2}}} ,. . .,\text{v}_{\text{{is}}} } \right). $$

(11)

This algorithm simulates a flock of birds which communicate during flight. Each bird looks at a specific direction (its best ever attained position Yi), and later, when they communicate among themselves, the bird which is in the best position is identified. With coordination, each bird moves also towards the best bird using a velocity which depends on its present velocity. Thus, each bird examines the search space from its current local position, and this process repeats until the bird possibly reaches the desired position. Note that this process involves as much individual intelligence as social interactivity; the birds learn through their own experience (local search) and the experience of their peers (global search). In each cycle, one identifies the particle which has the best instantaneous solution to the problem; the position of this particle subsequently enters into the computation of the new position for each of the particles in the flock. The most significant advantage of PSO algorithm is its relatively simple coding and hence low computational cost. It is quite similar to a genetic algorithm in aspects of the fitness concept and the random population initialization. However, the evolution of generations of a population of these individuals in such a system is by cooperation and competition among the individuals themselves. The population is responding to the quality factors of the previous best individual values and the previous best group values. The allocation of responses between the individual and group values ensures a diversity of response. The principle of stability is adhered to since the population changes its state if and only if the best group value changes. It is adaptive corresponding to the change of the best group value. The capability of the stochastic PSO algorithm, in determining the global optimum with high probability and fast convergence rate, has been demonstrated in other cases (Kennedy and Eberhart 1997; Clerc and Kennedy 2002). This algorithm can be readily adopted to train the multi-layer perceptrons as an optimization technique.

5.2 Fuzzy set theory

A fuzzy model is a logical-mathematical procedure based on a “IF–THEN” rule system that allows for the reproduction of the human way of thinking in computational form, Zadeh (1965). In general, a fuzzy rule system has four components:

• Fuzzification of the input: process that transforms the “crisp” (traditional) input into a fuzzy input;

• Fuzzy rules: IF–THEN logic system that links the input to the output variables;

• Fuzzy inference: process that elaborates and combines rule outputs;

• Defuzzification of the output: process that transforms the fuzzy output into a crisp output. The idea of fuzzy optimization models was addressed by Bellman and Zadeh (1970). In their context, fuzziness was likened to objective functions, constraints and/or some parameters of the optimization model that are transient or inaccurate in nature. Their membership functions are used to characterize the fuzzy objective and constraints while the intersection of fuzzy constraints and the objective function are the basis for any decision taken within the fuzzy environment. A problem of on non-commensurable objectives and imprecision was resolved through the use of fuzzy DP with implicit stochastic approach by Fontane et al. (1997). An array of decision makers which constitute the best part of the procedure was used in evaluating the fuzzy membership functions which could also be used to estimate the products of a linguistically defined operational objective and constraints. A comparison of the reservoir operational statement of fuzzy SDP (FSDP) and non-fuzzy explicit SDP undertaken by Tilmant et al. (2002a) and presented in their paper showed that though, the performance of the two models exhibited some similarities, however, the basic merit of FSDP over the latter was its explicit ability to capture water user’s and manager’s preferences together with the associated vagueness. A grey fuzzy SDP (GFSDP) model meant to improve the operations of Shiman reservoir in Taiwan was designed by Chang et al. (2002). The system was a representation of data and the fuzzy arithmetic of the rules of reservoir operation. This model i.e. GFSDP, in comparison with the conventional M5 rule curves was found to be much better in terms of performance especially its output and the extent of reduction in water deficits. Another model designed for optimal operation of reservoir systems and presented by Mousavi et al. (2005) was the DP fuzzy rule-based (DPFRB) model. In addition, the designer, (Mousavi et al. 2007); later compared methods such as ordinary least-squares regression (OLSR), fuzzy regression (FR), and adaptive network-based fuzzy inference system (ANFIS) to generate operating rules for a reservoir operation problem. Provision of input–output data set meant for these models was done through the use of DP.

5.3 Artificial neural networks

An ANN is a massively parallel-distributed information processing system that has certain performance characteristics resembling biological neural networks of the human brain (Haykin 1994). ANNs have been developed as a generalization of mathematical models of human cognition or neural biology. A neural network is characterized by its architecture that represents the pattern of connection between nodes, its method of determining the connection weights, and the activation function Fausett (1994). Caudill presented a comprehensive description of neural networks in a series of papers (Caudill 1987, 1988, 1989). A typical ANN consists of a number of nodes that are organized according to a particular arrangement. One way of classifying neural networks is by the number of layers: single (Hopfield nets); bilayer (Carpenter/Grossberg adaptive resonance networks); and multilayer (most backpropagation networks). ANNs can also be categorized based on the direction of information flow and processing. In a feedforward network, the nodes are generally arranged in layers, starting from a first input layer and ending at the final output layer. There can be several hidden layers, with each layer having one or more nodes. Information passes from the input to the output side. The nodes in one layer are connected to those in the next, but not to those in the same layer. Thus, the output of a node in a layer is only a dependent on the inputs it receives from previous layers and the corresponding weights. Over the years, the use of neural network modeling has greatly assisted river flow or inflow forecasting (El-Shafie et al. 2007; Karunanithi et al. 1994), rainfall forecasting (Ramirez et al. 2005), groundwater flow remediation (Ranjithan et al. 1993), flood forecasting (Chang et al. 2007) and the design of operating policies of a reservoir (Raman and Chandramouli 1996; Chandramouli and Raman 2001; Cancelliere et al. 2002; Liu et al. 2006; Li and Huang 2012) in water resources management. Operating guidelines from ISO are generally determined through the use of linear or nonlinear regression analysis (Bhaskar and Whitlatch 1980). In the design of operational guidelines, most of the problems earlier identified with regression analysis method are solved with the use of ANN which serves as a good replacement (Rossi et al. 1999). DP neural network (DPN) model is another mechanism designed for the determination of operating guidelines for a single reservoir. It was proposed by Raman and Chandramouli (1996). In designing this model, initial storages (St), inflows (It) and demands (Dt) were considered as input patterns while the output pattern considered was optimal releases (Rt) as shown in Fig. 4. The resulting input–output patterns from DP were directly trained by back-propagation algorithm based on supervised learning approach. The comparison of DPN model with SDP, standard operating policy (SOP) and DP regression (DPR) showed that the DPN model was superior to the other three operation models. Advanced exploration of the DPN method for multiple-reservoir systems operation was undertaken by Chandramouli and Raman (2001). Mapping of reservoir operation policy and inflow generation was carried out through the use of ANN by Jain et al. (1999). In the same vein, the combination of DPN method and soil water balance model was applied by Cancelliere et al. (2002) to propound operating guidelines for an irrigation reservoir. Liu et al. (2006) presented an improved version of DP neural-network simplex (DPNS) model. In this model, a simplex method was used to modify the optimization of the final output of a DPN model.

Fig. 4

Artificial neural network architectures

6 Multiobjective optimization

In the natural setting, occurrences such as irrigation, flood control, hydropower production and environmental preservation which may not be in the same scheme of planning may constitute some of the reservoir systems operation problems. Generally, two methods are utilized in resolving cases with multi-objective optimization difficulties: some of the objectives can be treated as constraints (otherwise called epsilon-constraint method). The other approach is to use the combination of various objectives by attaching comparative values to each objectives, this is referred to as weighting method. The two approaches may be used in combination with any of the optimization techniques earlier mentioned. For elaborate details about multi-objective programming techniques, interested party may consult appropriate texts (Cohon 1978; Goicoechea et al. 1982). Relative analysis (Ko et al. 1992) of these two methods with respect to reservoir management and operational system shows that the former is most suitable for causing trade-offs among cotemporary objectives. However, the latter may provide non-unique solutions for diverse classes of weights, which oftentimes may be used for a large number of objectives. In general, when there are conflicting objectives where multi-objectives problems cannot be solved directly, LP is useful in determining a unique optimum. The constraint method in combination with LP was used by Mohan and Raipure (1992) to resolve a conflicting objective related large-scale multi-reservoir operation problem. Also, the same method but this time with SDP was used to solve an operational problem of parallel reservoirs by Wang et al. (2005), Ko et al. (1994) presented a mechanism utilizing SLP, epsilon-constraint method and multi-criterion decision analysis (MCDA) for the planning of operations for multi-objective reservoir systems in Han River Basin in Korea. Another multi-objective optimization method that allows dynamic arrangement of policy constraints as goals is referred to as Preemptive goal programming (GP) (Loganathan and Bhattacharya 1990; Can and Houck 1984). This is a method that can be used to solve cases with definite values of comparative weights. The disadvantage of this method (preemptive GP) is that in a high priority setting for a more efficient lower priority section, the method does not allow small degradations. A multi-objective, preemptive linear GP model for reservoir operation which was used in the Tennessee Valley Authority network was developed by Eschenbach et al. (2001). Equally, an application of chance constrained GP (CCGP) to a system of multipurpose reservoirs with conflicting and non-commensurate objectives was presented by Changchit and Terrell (1993), Ouarda and Labadie (2001) a multi objective FSDP method used in deriving steady state multipurpose reservoir operating policies was presented by Tilmant et al. (2002b). The trivial nature of some operating objectives together with the decision-making process was accommodated by the fuzzy set theory (Shiliang et al. 2010). The implementation of FSDP-derived results in real-time operation was accomplished by making use of the combination of Continuous re-optimization models, with discrete FSDP derived membership functions approximated by cubic spline. The use of EAs for solving multi-objective optimization problems due to their ability to simultaneously cater for a set of possible solutions (also referred to as population) have gained the attention or researchers in the recent times. In addition, this method enables them to discover several members of Pareto optimal set in a single run of these algorithms in preference to the performance of series of individual runs which exist in the conventional optimization methods. Although, limited applications of multi-objective GAs (MOGAs) to reservoir systems optimization have been documented, the population based search approach used in GAs makes them suitable for multi-objective problems solutions. An algorithm referred to as Non-dominating Sorting GA-II (NSGA-II), which mitigates some of the problems resulting from MOGAs, such as computational complexity, non-elitism approach and the need for specifying a sharing parameter was developed by Deb et al. (2002). This approach was used for multi-objective reservoir operation problems by Kim et al. (2006), Reddy and Kumar (2006) and it indicated that it is very efficient especially when Pareto-optimal solutions are to be ascertained. Chen and Chang (2007) designed an efficient macro-evolutionary multi-objective GA (MMGA) which is very useful in optimizing the rule curves of a multipurpose reservoir system. Macro-evolution is a new concept of high-profile characters evolution that can alienate impromptu convergence (Liu et al. 2012). Results of some studies have shown that anticipated MMGA gave better-spread solutions and converged nearer the true Pareto frontier than NSGA-II.

7 Simulation–optimization techniques

In spite of the recent development in the usage of optimization techniques, in practical, simulation models are one of the strategic instrument of planning and management of reservoir system studies. The basis for simulation models with bias for reservoir operation still remains the mass balance equation and this portends the hydrological behavior of reservoir systems using inflows and other operating conditions. Nevertheless, some models depict the economic performance of the reservoir system. The use of simulation model to the planning and management of water resources commenced with the simulation of the Missouri River by the U.S. Army Corps of Engineers (USACE). The globally acknowledged Harvard Water Program relied on simulation methods for the economic design of water resources (Maass et al. 1962). The developed model produced the characteristics of the power generation, irrigation and flood control systems. The evaluation of the impacts of salinity on water supply abilities in the basin of the Brazos river was simulated by Wurbs and Karama (1995) while an elaborate simulation of the analysis and design of a large inter-basin water transfer system in India was carried out by Jain et al. (2005). In these simulations, the facilitators brought out all the complications involved in the planning of a large inter-basin water-exchange and processing system and the usefulness of the modeling methods in getting the most optimal results. The first step in the setting up of large-scale system seems to be simulation, however, when there are diverse opinions with respect to configuration, capacity and operating policy, simulation devoid of preliminary screening through optimization would be very cumbersome. It has been confirmed that notwithstanding the various studies of the large scale systems (Chaturvedi and Srivastava 1981; Kuo et al. 1990) the use of simple programming mechanisms like LP has greatly assisted in obtaining valuable results useful in the simplification of simulation problems. Predetermined operating guidelines are used to determine reservoir releases in an unadulterated simulation model. A variety of operating guideline for reservoirs in series and in parallel, useful for real-time, periodical as well as long-term operations of multi-reservoir systems was reviewed by Lund and Guzman (1999). A very challenging task is the identification of effective pre-defined operating guideline for ambiguous multi-reservoir systems with simulation process. The use of optimization models infused with simulation models has been used by researchers to surmount this difficult task (Sigvaldason 1976; Johnson et al. 1991). Most researchers got to know that the latter method was more efficient. Randall et al. (1997) used a simulation-mixed integer LP (MILP) method for long-range water supply planning in the Alameda County Water District (California). The researchers indicated that MILP engine used in long-term simulation model had demonstrable merits compared to network methods. A system involving regional water resources in a complex system was examined by Karamouz et al. (2004) through the use of a combination of optimization and simulation model. In addition, the combination of the constraint technique, decomposition iteration and simulation analysis as a single methodology by Wang et al. (2005) was presented to reduce the problem of dimensionality in providing solution to the stochastic multi-objective optimization problem of reservoirs in parallel. In their study, (Suiadee and Tingsanchali 2007) used the combination of simulation–GA model software to develop a graphical interface capability. The optimal upper and lower rule curves used in simulation was found by GA. The authors discovered that the total annual profit when the combination of simulation and GA model was used was a bit more than when those of HEC-3, SOP and the conventional actual operation were used. The use of a linked GA based on simulation–optimization methodology for optimal control of water quality in the downstream section of a reservoir was proposed by Dhar and Datta (2008). The model is a bridge between an elitist GA and a surface water quality simulation model (CE-QUALW2). It was agreed that the mechanism can be adapted to serve the purpose of multi-reservoirs (Yongyong et al. 2011); nevertheless, the boost in the available reservoirs with a longer time horizon will complicate the computational problem, as the CPU time required for a single reservoir problem was enormous (Majid and Fethi 2008). Authors thus agreed that the design of a separate code or the use of meta models, such as, ANNs could reduce the CPU time for finding solution to large and complicated reservoir systems operation problems. The combination of optimization and generalized simulation model is an established phenomenon recorded by previous researchers. Labadie (1993), Shourian et al. (2008) designed a hybrid PSO-MODSIM model to determine the optimum sizes of the planned water holding capacities and transfer facilities in the upstream Sirvan basin in Iran. In this methodology MODSIM has been infused into PSO algorithm. The design and operation variables are not kept constant and they evolved using PSO. MODSIM is expected to simulate the system performance and to evaluate the appropriateness of each set of these design and operation variables regarding the model’s objective function. The configuration of the simulation–optimization process is shown in Fig. 5.

Fig. 5

General framework of simulation–optimization modeling approach for reservoir operation

8 Suggestion for future work

Reservoir operation and management practices must adapt continuously to changes in water use priorities, physical and land use changes in the river basin, technological developments, and changes in public policy expressed in environment, safety, economic and technical regulations. The uncertainties associated with random property commonly exist in the reservoir operation (e.g., precipitations, reservoir inflows, and water demands); moreover, these uncertainties could become further compounded by ambiguity and vagueness by the relevant hydrologic information obtained. Besides, the complexity may be amplified by their dynamic features such as temporal and spatial variations as well as interactive relationships between water demand and supply. All of these complexities have placed many water resources management and reservoir operation problems beyond the conventional systems analysis methods. The popularity and success of the SDP technique can be attributed to the fact that nonlinear and stochastic features which characterize most reservoir systems can be directly incorporated within a SDP formulation. In addition, it has the advantage of effectively decomposing highly complex problems with a large number of variables into a series of sub-problems which can be solved recursively. The stochastic nature of the inflows, are considered explicitly in the optimization. This explicit consideration of the stochastic nature of the inflows is achieved by using the returns associated with each of the possible states (normally storage and inflow) in a particular time period, multiplied by the probability of the inflows that give rise to them, to generate the expected value of the objective function. This probability is often termed the transition probability, reflecting the correlation between two successive inflows. From the review analysis above it was found that the optimization with SDP can give more robust and reliable model. The earliest reservoir operation it appears to be the work of Little (1955). Who considered operation of a single reservoir. In Little’s model as well as in most of the models for stochastic reservoir operation, the inflows in the dynamic equation are assumed to be observations of a stochastic sequence. An important controversy in the literature of stochastic reservoir operation, as well as that of stream flow modeling, concerns the appropriate statistical assumptions for this inflow sequence. The appropriateness of a stochastic assumption for this inflow sequence assumption will depend on the interval between time steps.

$$ \text{P}_\text{r} (\text{Q}_\text{t} /\text{Q}_{\text{t} - 1} ) $$

(12)

where Q_t is the inflow during the time period t. The order one Markov process assumption was not supported by statistical hydrologists, but Little mentions that the much more convenient independence assumption. In other words the inflow Q_t of the time period t is conditioned solely on the inflow Q_t−1 during the immediate previous time period (t−1). The implication is that the inflow process exhibits weak correlations beyond the lag 1. This assumption has proved adequate when time periods are long (e.g. yearly inflow sequence), or when higher order correlations are insignificant. For models where the time step is short (e.g. daily, weekly and monthly inflows), the assumption that an order one Markov process inadequately captures the precursors of an inflow value may significantly impact the benefits of using a SDP formulation. For most hydrologic regimes, the time-step stream flow (daily, weekly and monthly) sequences show a high order Markov process, and hence the order one Markov process assumption is not supported. If correlations among inflows beyond lag 1 are strong (the shorter the time step, the wider the correlations will span among them, e.g. daily inflows normally show wider span of correlations than weekly or monthly inflows), which implies that the occurrence of current inflow can be more precisely conditioned on more previous in-flows Pr (Q_t+1/Q_t, Q_t−1,……, Q_t−k) (higher order autoregressive models are often used in stochastic hydrological modeling under this situation). Theoretically, these previous inflows can be treated as state variables to be included in the SDP formulation. However, this addition of state variables yields a mathematical formulation that is impractical to solve due to the “curse of dimensionality” inherent in the SDP formulation. Specifically, these difficulties stem from the structural limitations of the SDP, whereby the addition of each new state variable forces exponential increases in the number of system states to be evaluated, quickly making it impractical to solve. In the future study, the previous inflows (Q_t, Q_t−1,…, Q_t−k) should be treat as an inflow pattern to be included in the SDP formulation as a single state variable, and the number of state variables should remain the same as that of the SDP, with one storage state variable and one hydrologic state variable. An innovative approach proposed in this review is to treat previous inflows as inflow patterns. Consider an inflow pattern consisting of three discretized representative inflows in vector form, e.g. P ^m_t  = [${\text{Q}}_{\text{t - 2}}^{\text{k}} ,{\text{Q}}_{\text{t - 1}}^{\text{h}} , {\text{Q}}_{\text{t - 1}}^{\text{g}}$], where P ^m_t is mth inflow pattern of the time period t, consisting of three discretized representative inflows: gth representative inflow of period t, hth representative inflow of period (t−1), and kth representative inflow of period (t−2). The inflow patterns are then incorporated into the SDP formulations as a single state variable. Therefore, the number of state variables in the SDP formulations is kept at two (one storage state variable S_t and one hydrologic state variable P_t), and computational efficiency of the SDP model is maintained. Since inflows tend to be correlated (e.g. a high inflow is normally followed by another high inflow), it is expected that the number of inflow patterns needed is significantly less than the number of total combinations of inflows if multiple inflow states were used. For instance, for the aforementioned inflow pattern P ^m_t  = [${\text{Q}}_{\text{t - 2}}^{\text{k}} ,{\text{Q}}_{\text{t - 1}}^{\text{h}} ,{\text{Q}}_{\text{t - 1}}^{\text{g}}$], if the number of intervals for the three inflows Q_t−2, Q_t−1, and Q_t are K, H, and G respectively, and the number of inflow patterns for period t is M, it is expected that M ≪ K * H * G. The transition from one inflow pattern to the next is illustrated in Fig. 6. In the figure, there are 3 representative inflows for the time period t−3; 5 representative inflows for the time period t−2; 4 representative inflows for the time period t−1; 3 representative inflows for the time period t. Assume that an inflow pattern (dashed line) was observed, this pattern would transition to the next inflow pattern as that in red line, depending on where the inflow for the time period t belongs to. On the other hand, self-organizing method (SOM) is one of the widely applied neural models and has some interesting features over other neural networks. It is a unsupervised learning method and computionally simple. It produces a topology preserving mapping between high dimensional input space and low dimensional map space, which is capable of clustering, classification, estimation, prediction, and data mining (Alhoniemi et al. 1999; Vesanto and Alhoniemi 2000; Kohonen 2001) in a widespread range of disciplines regarding signal recognition, organization of large collections of data, process monitoring and analysis, and modelling as well as water resources problems. Typical for an SOM is that the desired solutions or targets are not given and the network intelligently learns to cluster the data by recognizing different patterns. As a result of this, SOM can be used (as pattern recognition) along with SDP in deriving optimal operation policy.

Fig. 6

Schematic representation of transition from one inflow pattern to the next

The losses from reservoirs may be a significant issue. One of the greatest challenges facing the world in this new century is rethinking the management of freshwater resources. An accurate simulation model is required to determine the relationships among the reservoir parameters (elevation, surface area, storage). The determination of the amount of the losses from reservoirs in the water distribution system is a very critical aspect of the water resources management. The effect of losses is a very important factor that is specially taken into consideration during the design of water distribution network systems. The losses lead to lose the water from the reservoir; as a result of this, the rate of losses must be known in order to evaluate the amount of water that will be lost at any given period of time from the reservoir. This will assist in ensuring that adequate amount of water is maintained for distribution in the system. Losses play a major role in balancing the volume of water available in any reservoir system; one of the techniques used into evaluate the relationship between parameters of a system is regression model. The nonlinearity of natural physical processes causes a major problem in simulating the reservoir systems. The conventional methods use a linear approach in solving such problems thereby obtaining not a very accurate simulation most especially at extreme values, and this greatly influences the efficiency of the model. This technique had been used to determine and evaluate the relationships that exist among the parameters of the reservoir system. However, the models obtained from this analysis were found not to be accurate and hence their simulation cannot be relied upon. This has necessitated the need to find alternative analytical tools that can simulate a better and more accurate result. This study suggests to consider the application of ANN, due to its ability in finding the nonlinearity among different kinds of parameters, Instead of regression model, as an analytical tool in the reservoir simulating system. In order to tackle this, Fig. 7 presents the flow chart of the proposed SDP-ANN model. The model implementation procedure is first, the operating storage volume (level) and stream inflows are discretized. Then, the SDP solution approach starts by initiating the values of the objective function at the last stage (a period in the future) to zero for each combination of state variables. Simulate the reservoir system by artificial neural network (ANN). Afterwards, the process continues backwards along the temporal stages. Each iteration consists T stages completing one annual cycle. The algorithm then finds the end of the period storage levels, for each combination of discretized beginning of a period storage level and the average inflow of the current period. The behavior of policy convergence after several iterations is due to the characteristics of Markov transition probability matrix incorporated into the recursive equation.

Fig. 7

Flow-chart for the solution of the proposed PSDP-ANN model

9 Conclusions

This review of previous works by researchers has shown that efforts have been made recently by researchers to create new optimization techniques, using ISO, ESO, evolutionary algorithms, ANN and fuzzy modeling for optimization of reservoir operation. The nonlinearity of natural physical processes causes a major problem in simulating the reservoir systems. The conventional methods use a linear approach in solving such problems thereby obtaining not accurate simulation most especially at extreme values, and this greatly influences the efficiency of the model. However, their simulation cannot be relied upon. This study suggests to consider the application of ANN due to its ability to mimic the nonlinearity features of the reservoir if compared with the traditional regression model. The popularity and success of the SDP technique can be attributed to the fact that nonlinear and stochastic features which characterize most reservoir systems can be directly incorporated within a SDP formulation. In addition, it has the advantage of effectively decomposing highly complex problems with a large number of variables into a series of sub-problems which can be solved recursively. In the SDP the stochastic nature of inflow is usually described by an order one markov process. This assumption has proved adequate when the time period is long (yearly inflow), for models where the time step is short (daily, weekly and monthly), the assumption inadequately captures the precursors of an inflow value may significantly impact the benefits of using SDP. An innovative approach proposed in this review is to treat previous inflows as inflow patterns. This study suggests to integrate PSDP-ANN model for reservoir operation. Further and wider application of this model especially on reservoir system simulation and optimization is suggested in order to achieve more reliable and robust model.