Best Practice in Communicating Uncertainties in Flood Management in the USA

Abstract

Ensemble forecasting has gained a great deal of popularity for addressing and estimating uncertainty associated with both meteorologic and hydrologic forecasts over the past decade. While ensemble-based hydrologic forecasts have been in routine operations for longer-term forecasts for many years, the notion of short- and medium-term probabilistic forecasts in support of water and flood management efforts is relatively new and is a developing science and service. Approaches to effectively conveying and communicating hydrologic forecast uncertainty are being actively developed and vetted with potential user communities. Important experience and insight will be gained over the next few years as the community of developers, forecasters, and end users work to leverage probabilistic forecasts in a risk-based decision environment. With proper focus and support, these efforts have the potential to significantly improve flood, ecosystems, and water management with benefits to multiple sectors of our society.

1 Introduction

Hydrologic ensemble forecasting procedures have made great strides over the past decade. Progress has been attributable to a growing acceptance that uncertainty is something that can be leveraged to make more informed decisions (National Research Council of the National Academies 2006) and substantial community support as evidenced through the success of the Hydrological Ensemble Prediction Experiment (HEPEX; www.hepex.org).

Among the most vexing challenges of the hydrologic ensemble prediction process is the appropriate conveyance of uncertainty information to decision makers. These decision makers represent many sectors (e.g., emergency services, power generation, recreation, agriculture, navigation, municipal water supply, industry, ecological management). Each has different and specialized needs and each has a different risk tolerance. In addition, their statistical background varies from nearly nothing (i.e., plays the lotto) to very sophisticated.

This chapter describes approaches and examples of how ensemble-based hydrologic forecast information is conveyed to users by the US National Weather Service today. Positive and negative attributes of approaches along with challenges are presented.

2 Conveying Probabilistic Streamflow Information

There are at least four fundamental approaches that can be used to provide uncertainty information associated with forecast streamflow. The oldest approach is to simply accommodate the expected “error” as a function of the user’s substantial experience with forecasts over time and, in particular, events that were memorable. This anecdotal uncertainty is what the hydrologic forecast and user community is attempting to supplant with objective information generated through ensemble techniques. More quantitative vehicles include:

• Generating a collection of ensemble-based products (text and graphics).

• Providing access to an interface that can create custom ensemble-based products (text and graphics).

• Providing ensemble members (data) that can be analyzed by end users in their own decision support architecture. Each approach has benefits and challenges and experience which has shown that all three, together, may represent a more reasonable approach.

2.1 Static Product Generation

Ensemble forecasts can be analyzed to address a seemingly infinite number of information requirements. This flexibility is beneficial, but it also creates challenges for forecast producers. What are the “best” sets of static graphics and text products that meet the greatest need for information? The fundamental questions associated with product generation are:

• What is the time period of interest (e.g., next 3 days, next 2 weeks, month of June)?

• What is the data aggregation period (e.g., hourly, daily, weekly, monthly, seasonal, annual)?

• What aspect of flow is of interest (e.g., summation (volume), mean, peaks, minimums, time to a threshold of interest)?

The more precisely the questions are addressed, the more useful the information for a specific application. Sounds simple enough, but ultimately, choices must be made if the number of routinely generated products is limited.

One of the most vexing issues encountered in generating ensemble-based graphics involves the impact of time aggregation on probability. Customers of hydrologic forecast information really want to see a “hydrograph” with associated uncertainty or “error bars.” What they really get is a series of histograms, at the time-step of analysis, placed side-by-side in sequential order. Interpretation of these sorts of graphics can very easily lead toward the wrong conclusions. Look first at Fig. 1. This graphic includes ten (10) 1-day histograms for flow and stage. It has the look of a hydrograph, but the 1-day time-step defeats that interpretation tendency to some degree. One interpretation of this graphic might be that “the river has a less than a ten percent chance of exceeding 12 feet over the next 5 days.” Now compare this with Fig. 2. This figure shows the distribution of peak flows within the coming 5-day period. This graphic suggests that the “river has a probability of between 25% and 50% of exceeding 12 feet over the next 5 days.” The proper interpretation of Fig. 1 is “the river has less than a ten percent chance of exceeding 12 feet on any of the next 5 individual days”; however, Fig. 2 indicates that collectively (all 5 days considered together), the probability of exceeding 12 feet is much higher. This phenomenon becomes more pronounced as the time-step of the analysis gets shorter. For example, if the time-step is reduced to 1 h, Fig. 1. really begins to look like a hydrograph and the likelihood of any extremes (peaks or minimums) is further reduced by appearance because their likelihood associated with any specific hour is lower than it is for any day or the entire period of consideration (e.g., 5 days). Best practices, therefore, limit the generation of graphics that are easily misinterpreted.

Fig. 1

Streamflow histogram, 1-day duration for Navarro River in California

Fig. 2

Streamflow histogram, 5-day duration for Navarro River in California

It is important for the users of hydrologic ensemble forecasts to continually remember that the generated products are simply an interpretation of the current set of ensemble members. For that reason, it remains good practice to provide a trace plot (spaghetti) among the set of routinely generated graphics. The 10-day trace plot that serves as the basis for the information in Figs. 1 and 2 is shown in Fig. 3. Note that while the analyzed probabilities shown in Figs. 1 and 2 may be well below the region of concern for an emergency/resource manager, a limited number of traces may be very problematic and therefore very much worth being aware of.

Fig. 3

Ensemble streamflow traces for Navarro River in California

Operators of reservoirs are normally more interested in volumetric (often multiday) forecasts of inflow rather than instantaneous or single day inflows. Understanding that the 1-day flows as shown in Fig. 1 cannot to be added together to form a probabilistic multiday volume, graphics such as is shown in Fig. 4 can serve a critical need of the reservoir management community. Again, without the provision of an accumulated volume plot (Fig. 4), a reservoir operator might be led to misinterpret a daily histogram (e.g., Fig. 1).

Fig. 4

10-day accumulated inflow volume for Lake Sonoma in California

For some time, River Forecast Centers in the USA have generated 90-day graphics that depict weekly probabilities of maximum stage (Fig. 5) and the maximum stage probability distribution over the entire 90-day period (Fig. 6). These sorts of graphics are particularly valuable when preparing for spring snowmelt flooding as often occurs in the upper Midwest of the USA. As with all longer-range products, they make heavy reliance on the information content of the model states.

Fig. 5

Web interface for “Create Your Own” ensemble product

Fig. 6

Weekly histogram of maximum stage for the Red River in Minnesota

2.2 Interactive “User” Product Generation

Given the diversity of interests in streamflow projections and the multitude of options for periods, durations, and flow attributes (maximum, minimum, mean, summation, and time to a threshold of interest), providing customers with a tool to analyze a set of ensembles using their specific criteria makes a lot of sense. This sort of feature does come with risk as it assumes that the user is well informed enough to make the required selections and properly interpret the results. This represents a minority of the total number of forecast customers, but to those who use it, it is a very important and powerful service. Figure 7 shows the interface supported by the California-Nevada River Forecast Center. A substantial “help” section is provided to assist users in navigating the options and interpreting the product generated. This sort of interface allows users to “narrow” the scope of their information need and generate products that directly address their requirements.

Fig. 7

90-day Exceedance probability plot for the Red River in Minnesota

2.3 Provision of “Raw” and “Postprocessed” Ensembles

While forecasters and developers struggle with the “best” ways of describing the uncertainty of hydrologic ensemble forecasts with complex and often difficult to interpret graphics, the most effective practice may be to simply provide the data and allow the customer to perform an analysis that is meaningful to them. For sophisticated users with resources, this is clearly the most effective alternative. As an example, a reservoir operator with a model that can simulate operations can easily process each member of an ensemble set to evaluate the benefits/costs of a selected release strategy. Alternatively, that same operator will have to use their imagination to understand how a histogram of daily inflow probabilities will impact their regulation strategy. The difference is profound. Substantial progress is being made along this front. In California alone, the INFORMS project (Georgakakos et al. 2007) as well as the Yuba-Feather Forecast Coodinated Operations (FCO) project have engineered solutions to leverage the full potential of ensemble forecasts in a decision support model. Figure 8 shows the conceptual process schematic for the Yuba-Feather FCO ensemble-based decision support model.

Fig. 8

Schematic of data flow for Yuba-Feather Forecast Coordinated Operations ensemble-based decision support model (by permission of David Ford Consulting Engineers, Sacramento, CA)

Just as ensemble Numerical Weather Prediction (NWP) models exhibit biases and inappropriate spread, so too will the hydrologic forecasts without some sort of postprocessing methodology (Demargne et al. 2014). Sophisticated users, however, have a choice in this process. They may choose to accept the “raw” ensembles without the benefit of postprocessing and instead apply their own error correction processes based on an adequately long history of performance. Such is the case for the ensemble forecast services provided to the New York City Department of Environmental Planning (NYCDEP) by the US National Weather Service. This may be the most efficient way of objectively accounting for ensemble reliability issues, but it also may result in additional workload for operational entities required to issue both raw and postprocessed ensemble information.

2.4 Managing Expectations

It is clear that, given all of the assumptions that must be made to generate an ensemble-based hydrologic forecast, there will be uncertainty in the estimates of uncertainty. The “discrimination” in the system may not be able to reliably differentiate between 85% and 90% probability of exceedance. Further, work is needed to help users understand that some risk must be assumed if one expects to leverage uncertainty in the long run. If you need to be 99% sure before you will take action, you will likely miss a lot of opportunities.

3 Applications to Water Resources Forecasting

Ensemble applications to water resources forecasting are not new but have gained substantial growth and acceptance over the past decade. Early work appeared in the 1970s and the National Weather Service formalized a process within their forecasting system in the mid-1980s (Day 1985). Despite this, the predominate approach for seasonal streamflow forecasting in the Western USA has remained some form of regression modeling driven with data available on a monthly basis (Garen 1992). More recently, forecasters have begun to integrate daily observations, and the US National Weather Service is in the process of shifting toward full reliance upon ensemble processes evaluated every day and year-round.

The attributes of relying on ensemble process for longer-range water resources forecasting include:

• Integration with short-term hydrologic forecasting procedures

• Use and integration of near real-time observations (e.g., precipitation, air temperature, streamflow)

• Integration of current weather and climate forecast information

• Ability to update on a daily basis

As this transition takes place, forecasters are experimenting with graphical products that describe both the uncertainty as well as how the forecasts have changed or “trended” over time. Figures 9, 10, 11, and 12 show potential candidate graphics that describe expected volume over time. The “trend plots” in Figs. 10 and 11 do not show uncertainty but do show how the 50% exceedance probability forecast over seasonal volume has changed over time. Note the precipitous drop in expected water supply volume that took place during the fall and early winter as California received only a very small percentage of normal precipitation during this period. Assuming that the ensemble forecasts leverage the skill in the weather and climate forecasts, one can quickly see that the forecasts were not effective in detecting the coming drought with much or any lead time. This may seem discouraging, but it does highlight the need, value, and process for leveraging improved seasonal weather predictions through a hydrologic ensemble forecasting framework.

Fig. 9

Monthly volume histogram for 1-year for the Feather River inflow to Lake Oroville in California

Fig. 10

Seasonal (April-July) accumulated volume trend plot for the Feather River inflow to Lake Oroville in California

Fig. 11

Water Year (October-September) accumulated volume trend plot for the Feather River inflow to Lake Oroville in California

Fig. 12

Probablistic water year (October-September) accumulated volume trend plot for the Feather River inflow to Lake Oroville in California

In the water resources services domain, the streamflow forecast alone is not adequate to provide customers with the complete picture of the water supply situation. In many areas, reservoirs provide a buffer for interannual variation as well as a means for shifting runoff from the time of occurrence to the time of need (e.g., irrigated agriculture). Information that summarized and combines the expected runoff with the existing reservoir storage is critical for assessment purposes. In addition, water supply customers have historically expressed a need to see information that supports the streamflow forecast itself, such as monthly and seasonal precipitation and snowpack. Comparison of precipitation and snowpack when expressed as a percent of normal provide excellent context for understanding and establishing confidence in a specific volumetric seasonal streamflow forecast.

4 Hindcasting and Validation

The key attribute of ensemble-based probabilistic hydrologic forecasts that makes them useful is reliability. Reliability means that the ensemble members, as a package, are (1) unbiased and (2) have appropriate spread. If the ensembles have not been demonstrated to be adequately reliable, the user incurs a great deal of risk when applying the information contained in the ensembles to their specific decision-making process.

The process of “hindcasting” is well established (Demargne et al. 2014). In essence, the complete hydrologic forecasting system is run in a retrospective process to effectively create the set of forecasts that would have been generated over an adequately long period of time. That period of time is normally constrained by the availability of numerical weather prediction (NWP) models, used to force the hydrologic model set, to the last 25 or 30 years (Hamill et al. 2013). The process of generating the NWP hindcasts requires a great deal of computer resources and is therefore expensive. Once generated, the NWP hindcast serves as a rich dataset that can be used to understand the behavioral climatology of the specific NWP. Everyone accepts that NWPs are not perfect. They are biased to some extent and exhibit uncertainty. The hindcasts allow for measurements of the bias (difference between forecasts and observations) and uncertainty (correlation between forecasts and observations). This hindcast analysis information allows for the proper interpretation “today’s” NWP model run and the effective integration of the NWP forecast information into the hydrologic ensemble forecast process. One approach for doing that is well described by Demargne et al. (2014).

It is important to note that the NWP hindcasts are specific to a model and the parameterization at the time of hindcast generation. If changes are made to the NWP, the hindcasts would simply no longer apply and would need to be rerun, reanalyzed, and reintergrated into the hydrologic ensemble forecast process. Further, customers of the hydrologic ensembles may need to make adjustments to their decision models to accommodate any resulting shifts. For these reasons, it is critically important that “frozen versions” of NWPs are operationally supported when the user community is dependent upon representative hindcasting information. Further, it is important to recognize that both the hydrologic forecast community and the user community need time (months) to integrate new NWP hindcast information before a “frozen version” is operationally discontinued.

While the hindcast process provides a way to understand the behavior of the complete forecasting system and resulting information, it is not perfect. Fully replicating the somewhat interactive hydrologic forecasting process in practice today is not feasible. It is generally accepted that hydrologic forecasters add value (reduce errors) through interaction with the hydrologic forecasting modeling system. This might take the form of small adjustments to forcing data (precipitation or air temperature) or adjustments to model states to better align model simulations with observations of streamflow during the recent observed period. Even if you were able to insert a forecaster into the hindcast process (very labor intensive), it would be extremely difficult to replicate the full forecasting environment that influences human decision-making. As such, the hydrologic ensemble hindcasts are an approximation of what we should expect from the current forecast process, but they may exhibit slightly more uncertainty as they do not benefit from forecaster experience and interaction.

4.1 Validation and Associated Services

With all their conditions and issues, hydrologic ensemble forecast hindcasts offer keen insight into the value of current probabilistic hydrologic forecasts. They provide the body of information that allows for the development of trust. Ensembles allow for a nearly infinite number of questions to be addressed. In turn, the hindcast analysis allows one to assess the reliability of the information used to address those very same questions.

Ensemble verification capability such as those described by Brown et al. (2010) provide the flexibility and rigorous statistical testing needed. Substantial education and training are needed to help consumers of this information to fully understand the implications of the ensemble forecast verification metrics and how they affect their specific decision-making process. Substantial work will be required to create validation information that is general enough to apply to most cases and specific enough to build/demonstrate value and trust.

5 Conclusion

The pace of improvement in hydrologic forecasts is steady, but very slow. Rather than waiting for the perfect forecast, a great deal of value and insight can be gained by understanding and leveraging the uncertainty associated with today’s forecast. Resource managers are pressed harder every day to work “smarter.” Integrating risk into every aspect of decision-making is warranted as long as the information being used is reliable and understood. While hydrologic forecasters and water resource managers have years of experience in using probabilistic seasonal streamflow volume forecasts, the notion and technology of short- and medium-range probabilistic hydrologic forecasts is quite new. Developers, forecasters, and users are challenged to create, provide, and integrate probabilistic information that will yield understanding and improved outcomes for end users.