1 Introduction

Private- and public-sector decision makers are increasingly concerned with the effects that climate change will have on economic activity, public safety, and ecological resources. Effects can arise from changes in average conditions, such as the impact of temperatures on energy demand, rainfall on agriculture, or sea levels on coastal wetlands. Also, there is a concern about possible increases in the frequency and severity of extreme weather events, such as flooding and heat waves (IPCC 2012; Peng et al. 2010; Bakker et al. 2017). Given the inertia of the climate system and continued growth in emissions, the scientific consensus is that anthropogenic impacts on climate will continue to grow (Adger and Barnett 2009). Thus, the need for adaptation is unavoidable and increasingly important (Dessai and van der Sluijs 2007; Berrang-Ford et al. 2011).

Private- and public-sector decision makers are weighing strategies to adapt environmental and human systems to shifts in average climate conditions and the frequency and intensity of extreme weather. Here, we focus on regional and local investment decisions that reduce the vulnerability of utility and transport infrastructure, land development, and natural resources. These actions, such as infrastructure upgrades or ecosystem restoration, often involve a large financial commitment with a long-term planning horizon. The effectiveness of these actions is uncertain in part because a wide range of climate scenarios are plausible (Haer et al. 2013). The challenge we consider is how to structure information about the costs and benefits of alternative adaptation measures under multiple climate scenarios to facilitate decisions about what investments and other commitments to make in the near term and which ones to defer.

Because the field of decision analysis has developed theoretical frameworks and practical tools for evaluating personal, private sector, and government decisions when uncertainties and multiple objectives are involved (Keeney and Raiffa 1993; Belton and Stewart 2002; Clemen and Reilly 2013); these tools could potentially provide useful insights concerning trade-offs associated with climate adaptation decisions. We assume that the complexity resulting from the array of uncertainties, alternatives, and objectives that are relevant to adaptation mean that more comprehensive analyses may be needed in order to avoid missing essential problem features. However, we recognize that tight timelines and analysis budgets or focused problem definitions can often mean that simple analyses will suffice and that more complicated analyses based on poor data might not improve on simpler studies. Three types of decision analyses, which we now summarize, have been widely applied in climate adaptation.

  1. (1).

    Type I: predict-then-act analysis. Analysts compare strategies under a single best-guess future (i.e., expected climate change) (Weaver et al. 2013; Watkiss et al. 2015). For instance, the California Department of Water Resources has used this method in its water planning for over 50 years (Groves and Lempert 2007). This method is straightforward, but it ignores uncertainties and fails to consider extreme situations. The resulting “flaw of averages” is widely recognized; i.e., nonlinear benefits and costs mean that plans based on expected (probability-weighted average) future conditions θ will not accurately quantify expected net benefits NB (i.e., NB(E[θ]) ≠ E[NB(θ)]).

  2. (2).

    Type II: multi-scenario analysis without adaptive options. Type II analysis evaluates adaptation alternatives under multiple plausible futures. This type of analysis can be either probabilistic-based or probability-free. Decision trees are often used for a probabilistic decision analysis (Hobbs et al. 1997); in contrast, robust decision making (RDM) avoids assigning probabilities to scenarios (Lempert et al. 2004).When a multi-scenario analysis considers multiple decision stages while the later decisions do not adapt to later uncertainty development, it is also a type II analysis. For example, Eijgenraam et al. (2016) identify the optimal flood protection policy and its timing through a multi-stage optimization model with multiple scenarios. Type II analysis is more thorough than type I, but the effort it requires can be significantly higher as well. For instance, the relative likelihoods of possible regional (downscaled) scenarios are often based on expert judgment (Thompson et al. 2016). This requires a significant amount of work and time hiring experts in the relevant fields, constructing questionnaires to minimize cognitive biases, and administering and applying consistency checks. According to Hallegatte et al. (2012), a typical RDM analysis takes many months and can cost $100–500K. Computational costs may also be high. For instance, RDM uses many computational experiments to evaluate strategies under many scenarios in order to identify robust near-term decisions.

  3. (3).

    Type III: multi-scenario analysis with adaptive options. Type III analyses are more comprehensive than type II studies because they consider how adaptation strategies can be modified later based on what is learned about climate and the effectiveness of adaptation actions. Adaptation strategies that include options for changing the timing, design, or mix of actions in response to learning will likely be more effective than less flexible strategies in which initial commitments are difficult to modify. However, explicitly accounting for future flexibility will increase the analytical effort required, as the number of possible combinations of near- and longer-term options plus scenarios grows rapidly—the “curse of dimensionality.” Later, we show how traditional decision-tree analysis can be used to choose strategies in type III analyses. Real options analysis (ROA) is another example of a type III solution framework that has been applied to climate adaptation. Some of those ROA applications are more closely aligned with classic financial real-option analysis, such as using Black-Scholes formulas to estimate expected costs of adaptation measures (Sturm et al. 2017) and evaluating option values with the binomial lattice method (Kontogianni et al. 2014). Other ROA applications search for a rule-based optimal strategy in which later decisions depend on whether climate-relevant thresholds are exceeded (Gersonius et al. 2013; Woodward et al. 2014). Dynamic adaptive policy pathways (DAPP) is yet another example of the type III analysis, which searches for robust adaptive strategies rather than optimal ones (Haasnoot et al. 2013). Finally, decision trees can be used for type III analysis with decision nodes following chance nodes, representing possible future adaptions that can be made as uncertainties unfold (Hobbs et al. 1997); multi-stage stochastic programming implements the same framework within a optimization model (e.g., Hung and Hobbs 2019).

Adaptation practitioners may, at first, be unsure about which of the three types of analysis is most suited for their problem, or how their decisions could be affected. When facing a specific adaptation problem and deciding how to analyze it, planners need to trade off the complexity and cost of an analysis versus the usefulness of its insights. The expense of thorough analyses (type II or III) needs to be justified by the benefits in the form of improved decisions. Some adaptation situations might not significantly benefit from these sophisticated analyses, and a simple predict-then-act analysis (type I) might give sufficient insight to justify a near-term decision. Sophisticated analyses are not worthwhile if they are unlikely to change near-term decisions. On the other hand, when managers confront multiple adaptation problems (e.g., cities that are concerned with both flooding and heat waves), managers might want to know which of those problems can benefit most from thorough analysis so that they can deploy their limited analytical budget, personnel, and time most effectively. Thus, this paper addresses two interrelated questions. First, how can adaptation managers identify the most appropriate type of analysis for a particular problem? Second, how can managers screen a large set of adaptation decisions to identify those that are most likely to be improved by comprehensive analysis?

In particular, the goal of this paper is to present two procedures to implement a bidirectional framework to match problems and analysis approaches:

  • From problem to analysis: a screening procedure that selects the most appropriate type of decision analysis for a specific climate adaptation problem; and

  • From analysis to problem: a procedure that identifies which of several adaptation decisions are most likely to benefit from careful decision analyses.

First, we propose a six-step screening procedure in section 2 to identify which of three types of decision analyses is best suited for a particular adaptation problem. A predict-then-act analysis (type I) will be recommended when uncertainty can be disregarded without deterioration in expected performance. Meanwhile, we suggest a more thorough analysis (types II or III) for situations in which explicitly considering uncertainty and adaptation options is essential for comparing near-term strategies. The aim of the procedures is to avoid investing too much effort on analyzing problems whose decisions would be unlikely to significantly change after more in-depth study. We provide a tutorial example to illustrate how this screening procedure can be applied to a real-world adaptation problem. Then in section 3, we introduce three characteristics—“fitness”, “importance”, and “measurable performance”—that contribute to making comprehensive decision analysis valuable for adaptation. Based on those characteristics, we propose a framework in section 4 for evaluating and ranking multiple adaptation problems in terms of which would benefit most from a comprehensive decision analysis. As a practical application, climate adaptation problems in the Chesapeake Bay area are ranked. The logic of our paper is to first help readers understand the principles of analyzing uncertainty in single and multistage decisions with a concrete example and then to introduce a more abstract procedure based upon those concepts. However, in an actual decision-making process, practitioners might first use the ranking framework to select problems that are most likely to benefit from a comprehensive analysis, and then apply the six-step screening procedure to find the most appropriate analytic method for each of those problems.

We do not introduce new decision analysis tools in this paper, but rather new approaches for choosing an appropriately sophisticated tool. To our knowledge, this is the first paper that addresses trade-offs between the costs (e.g., personnel and time) and benefits (e.g., usefulness of insights) of simple deterministic adaptation planning versus more comprehensive analyses incorporating climate scenarios and future options.

2 Procedure 1: identifying an appropriate type of decision analysis for a particular problem

2.1 The screening procedure

Should an organization responsible for managing a particular adaptation problem invest in a sophisticated multi-scenario analysis (e.g., type II or III)? If significantly more cost-effective decisions could be achieved by doing so, the answer can be “yes”. However, if no or slight improvements in decisions would result, a predict-then-act analysis (such as type I) might suffice. For guidance in selecting an appropriate type of analysis, we propose a six-step screening procedure, shown in Fig. 1.

Fig. 1
figure 1

Flowchart of the six-step screening procedure

Full size image

In explaining this procedure, we define an “action” or “alternative” as being a possible choice made at one point in time, given what is known at that time (“information state”). In a decision tree, an alternative would be represented as a particular arc departing a particular decision node. Different nodes can represent different information states, so that the choice can be conditioned on information (e.g., if there are two states at t = 10, there will be two nodes, and one choice would be made for each). A “strategy” is a collection of selected actions over several periods and/or information states (e.g., then choose X at t = 10 if A occurs, but choose Y if instead B occurs). In applying the procedure, we assume the following: (1) an adaptation decision problem has been defined involving choices among multiple alternatives whose future performance is uncertain under a changing climate. Alternatives might involve different levels of investment for reducing vulnerability of development, infrastructure, or ecosystems to anticipated climate impacts. Or alternatives could take the form of rules (e.g., zoning regulations or building codes) that affect private investments that are potentially at risk; (2) scenarios can be defined that characterize a range of plausible future environmental and socioeconomic conditions that determine the vulnerability and resulting costs and benefits of strategies; (3) performance on objectives, such as construction costs or expected damage, can be approximated for each combination of decisions and climate scenario; (4) initial plans can be modified (e.g., delay or accelerate investments) as more information becomes available (type III).

For adaptation problems satisfying these assumptions, the screening procedure is as follows:

  1. (1).

    Base-case analysis with benchmark scenario

    To define a benchmark scenario, users select a single nominal (e.g., expected) set of values for the uncertain climate variables. The most favorable strategy (call it strategy 1) under this scenario is then identified based on the users’ choice of performance metrics.

  2. (2).

    Analysis with extreme scenario

    In this step, users define an extreme climate scenario (e.g., worst case), and then assess whether there is some other strategy (say, strategy 2) that is likely to be preferred to strategy 1 if the worst climate scenario occurs. If the answer is “no”, then the decision is probably insensitive to climate uncertainties and it is unlikely that a more in-depth analysis would significantly affect the decision. The cost of such an analysis might not be justified, and a predict-then-act analysis (type I) is recommended. But if the answer is “yes”, users would proceed to the next step.

  3. (3).

    Regret check

    “Regret” is defined here as the difference between the base case strategy’s performance and that of the optimal strategy in a given scenario (Savage 1951). Here, users need to calculate the regret of strategy 1 in the extreme climate scenario defined in step 2 (i.e., the difference between strategy 1’s performance and strategy 2’s in that case). If users view such regret as minor, the cost of a comprehensive analysis might not be justified even if it identifies a different strategy. Otherwise, users should go to the next step.

  4. (4).

    Multi-scenario analysis without adaptive options

    Considering a small but representative set of climate scenarios (often include the most extreme scenarios (worst- and best-case) and/or a situation in between), users need to undertake a simple probabilistic analysis to calculate the expected performance of all strategies. Since this is a screening process, users can provide rough initial guesses for the probability of each scenario. If the strategy with the best expected performance (strategy 3) is close to or identical to strategy 1 under a range of probability assumptions, a multi-scenario analysis is likely to generate the same result as a type I analysis. Hence, there is no need to invest in a multi-scenario analysis (type II or III). Otherwise, multi-scenario analysis will provide additional insights that are possibly worth the cost. In that case, users also proceed to the next step to decide whether flexibility exists for some alternatives.

  5. (5).

    Identification of flexibility

    Users should further assess if flexibility, in the form of future options to modify system design or operations, are available for some or all of the initial possible alternatives considered in the above steps. Flexibility can, for instance, enable planners to delay, abandon, expand, or otherwise modify the original plan. If such options exist, users should then undertake step 6 to check whether considering flexibility can further improve the performance of the recommended strategy. If improvements are insignificant, users can then apply type II multi-scenario analysis without adaptive options.

  6. (6).

    Multi-scenario analysis with adaptive options

    In the final step, users assess whether considering the ability to adapt the plan at a later stage could significantly enhance its performance (Woodward et al. 2014). For example, they might compare the consequences of acting right now versus delaying a decision until better information is available, or they might consider investing in an initial modest level of an adaptation strategy and then augmenting it later if risks increase. If the option value of waiting or building in flexibility is potentially significant, then a multi-scenario analysis with adaptive options is preferred for this problem. Such an analysis could use either decision trees or real-options analysis, as discussed in “Step 6” of section 2.2. Otherwise, a type II analysis will suffice.

In this procedure, steps 1–3 serve as an initial check on whether the decision is sensitive to the climate conditions and whether the relative performance of the alternatives significantly differs between the base and extreme scenarios. Steps 4–6 are more in-depth, and their purpose is to determine whether consideration of multiple scenarios and adaptation options may lead to a different (and better) decision. To demonstrate how to use this screening procedure, we provide a tutorial adaptation problem here. Users could apply this same general approach for their own problems. With this screening procedure, users can make a quick decision on which type of analysis would likely provide useful insights. If the potential benefit of a sophisticated decision analysis outweighs its cost, we recommend that it be undertaken. Otherwise, a type I analysis may suffice.

2.2 A tutorial example

We present a tutorial example to illustrate how the six-step screening procedure can be used to identify the most appropriate type of decision analysis for a specific adaptation decision problem. We create this example for the purpose of demonstration, progressing through all six steps of the process. Costs and probabilities are meant to be broadly illustrative of what might be encountered. The example is as follows:

A utility owns a set of coastal electric substations that could sustain damage in the event of flooding, resulting in an extended electrical outage for nearby customers. The utility needs to decide whether to build a floodwall to protect the substation from future storm events whose severity and frequency will possibly intensify with climate change. The magnitude of the damage is deeply uncertain since it is affected by such factors as future sea level, storm frequency and intensity, and population growth, especially after 2050 (Kopp et al. 2014). The substation manager’s objective is to minimize the expected present worth of floodwall plus flood damage over the next 60 years.

To simplify our presentation, we make the following assumptions:

  1. (1)

    The utility has two alternatives: (A) build a floodwall or (B) take no action. Building can be done in either year 0 or year 20 with a cost (present worth) of $5000K or $3000K, respectively;

  2. (2)

    Flood damage is divided into two stages: near-term (FD1, years 1–20) and long-term (FD2, years 21–60), respectively. In both stages, damage will range from low to high magnitude depending on both random weather and the effects of climate change;

  3. (3)

    For the benchmark and extreme scenario analysis, the amount of damage for each alternative is assumed to be known and is based on only one climate scenario. For example, the benchmark scenario analysis might assume that the moderate flood damage occurs in that scenario;

  4. (4)

    For the scenario analysis, decision-makers consider the possibilities of climate scenarios that are more or less severe than the base case. We assume the decision-makers have a prior distribution for the resulting long-term flood damage, P (FD2 = low) = 0.7 and P (FD2 = high) = 0.3, respectively. Because of the random nature of severe storm events, it is possible for climate change and long-term damage to be less severe than anticipated, but there may nevertheless be major damage in the near term. We assume that the probability that the near-term damage is consistent with actual long-term damage is 70%, but there is a 30% chance of inconsistency (e.g., P (FD1 = high ∣ FD2 = high) = 0.7 and (FD1 = low ∣ FD2 = high) = 0.3). The unconditional distribution of near-term damages P (FD1) and posterior distributions of long-term damages given what is observed in the short-run P (FD2 ∣ FD1) can be calculated by Bayes Law (Hobbs et al. 1997);

  5. (5)

    Flood damage is a function of the future climate and adaptation strategies.

We create a hypothetical dataset for this example (Table 1), relying in part on information from Balducci et al. (2004). We use decision trees (Clemen and Reilly 2013) to illustrate the calculations made in each of the six steps. In a decision tree, decision nodes are denoted by squares, and uncertainty nodes as circles, with time proceeding from left to right. The performance of a particular sequence of decisions and uncertain outcomes is shown as the cost value at the end of a branch.

Table 1 Dataset for the tutorial example
Full size table

We now apply the six-step procedure to this example. In step 1, we define “moderate flood damage” as the benchmark scenario (i.e., what occurs under expected climate change) and compare the two alternatives (Fig. 2, left). Excluding climate uncertainty means there is no chance node in this step. The sum of the construction cost and the incurred damage of alternative A (build a floodwall) is $6400K (= 5000 + 400 + 1000) and of alternative B (take no action), $5200 K (= 0 + 1700 + 3500). In this case, the optimal strategy is to take no action (“Strategy 1”).

Fig. 2
figure 2

Decision tree for benchmark scenario analysis (left) and extreme scenario analysis (right)

Full size image

Next, step 2 performs a similar analysis using the most severe climate scenario. We compare the two alternatives under the “high” damage scenarios for both stages (Fig. 2, right). Here, the sum of the construction cost and the incurred damage of alternative A (build a floodwall) is $8500K (= 5000 + 1100 + 2400) and for alternative B (take no action), it is $11,200K (= 0 + 3200 + 8000). The optimal strategy, in this case, is to instead build a floodwall (“Strategy 2”). Thus, if we know the extreme climate scenario is going to occur, a decision different from strategy 1 will be made. The decision is therefore sensitive to climate conditions and so this problem passes step 2 and enters step 3.

In step 3, we calculate the ‘regret’ of strategy 1, which is the difference between strategy 1 and strategy 2 under the “high” damage scenario. The regret is $2700K (= 11,200 − 8500), or over 30% of the cost of the optimal decision (build a floodwall). If this regret is judged to be significant, then the user proceeds to step 4. Otherwise, they stop and apply a simple analysis because the cost difference does not justify a thorough analysis. At this point in the example, we assume the regret is significant and proceed to the next steps.

In step 4, we consider two extreme scenarios (low and high flood damage) and conduct a probabilistic analysis. Each scenario can happen in each stage with the probabilities given above (assumption (4)); the probabilities are inserted into the associated branches of each chance node (Fig. 3). The expected total cost of an action is a probability-weighted average across all the four scenarios. For example, when FD1 = FD2 = low, the total cost of building a floodwall is $5000K (= 5000 + 0 + 0). Given the total cost and probability of each scenario, the expected total cost is $6188K (= (5000*0.84 + 7400*0.16)*0.58 + (6100*0.50 + 8500*0.50)*0.42). Solving this tree indicates that the optimal near-term decision under uncertainty (but without later flexibility) is to build a floodwall (“Strategy 3”). The best near-term decision still differs from strategy 1, so we proceed to step 5.

Fig. 3
figure 3

Decision tree for multi-scenario analysis without adaptive options

Full size image

In step 5, we note that the decision to harden the substation can be delayed, and committed to in either year 0 or 20. If the decision is delayed until year 20, the observed impact of FD1 can inform the second-stage decision by enabling us, for instance, to adjust the probability of high damage in the long run (i.e., use the posterior probability P (FD2 = high ∣ observed FD1) rather than the prior probability (P (FD2 = high) = 0.3). In addition, the cost of constructing a floodwall will decrease to $3000K due to interest cost savings and perhaps technological advances.

In step 6, we consider adaptive decision making as represented by multiple decision stages and solve it using the decision tree method (Fig. 4). The results indicate that waiting for more information and postponing the decision is the best near-term strategy (“Strategy 4”). If high damage is observed in the near term ($3200K), then a floodwall should be built in the second stage because high damage early on is an indication that high damage later is more likely. However, if near-term damage turns out to be low, the substations’ owner should just do nothing in year 20. Waiting, in this situation, is worthwhile even if the substations are unprotected in the near term, as the increased confidence in what will happen in the long-term together with the reduced construction cost will decrease the overall expected cost. With this screening procedure, multi-scenario analysis with adaptive options (type III) proves to be the most appropriate type of analysis for this substation hardening decision.

Fig. 4
figure 4

Decision tree for multi-scenario analysis with adaptive options

Full size image

The approach used to solve the multi-stage, multi-scenario problem here is more similar to rule-based ROA applications (Gersonius et al. 2013; Woodward et al. 2014) than applications of classic financial ROA (Kontogianni et al. 2014; Sturm et al. 2017). The decision structures of our approach and rule-based ROA are the same, including immediate commitments, uncertainties unfolding over time, and later wait-and-see options. The methods differ in two aspects. First, our folding-back approach identifies the optimal solution based on discretization of decision variables and uncertain outcomes followed by backwards dynamic programming, while rule-based ROA applies genetic algorithms (Deb et al. 2000) to find optimal thresholds and subsequent choices that define a rule-based strategy. The rules involve thresholds for an uncertain variable such that one alternative is implemented if the variable exceeds the threshold, and another is implemented otherwise. Second, our approach applies Bayes Law to update probabilities to represent learning through time.

In this example, although we only consider three climate scenarios and two decision stages, different near-term strategies were selected in the six steps. An analysis that only considers a single future state (e.g., expected sea level rise) could lead to a poor decision because of the “flaw of averages”. A multi-scenario analysis with future options can better inform decisions by considering more information and flexible strategies.

This screening analysis only requires approximate judgments by the users, which might be highly preliminary but yet still reflective of the relative magnitudes of strategies’ performance (cost, benefits) and the relative likelihood of climate scenarios. This is much less effort than needed in a full analysis that would have to be documented and subjected to public review. Of course, the recommendations of a thorough analysis might differ in some ways from those resulting from approximate six-step screening analysis, but the latter can still provide insights regarding whether the thorough analysis might be worth doing. In a full analysis, the best available information concerning possible climate scenarios and their consequences should be used, based on modeling, historical patterns, or expert judgment. Relative likelihoods of different scenarios can be provided by experts informed, for instance, by regional climate impact analyses (Polsky et al. 2000; Miller et al. 2013) and should be subjected to sensitivity analysis.

This six-step screening procedure can work well if a manager is focusing on one particular adaptation problem. However, managers have limited resources and time, and so might not be able to apply the procedure to several problems at the same time. For example, emergency preparedness agencies may have responsibility for both coastal and inland flooding hazards at several locations. Hence, a practical question is which of several climate adaptation problems might benefit most from a comprehensive analysis. This question can be answered by understanding the characteristics of adaptation problems that make decision analysis useful, even without going through all the above six steps.

3 What characteristics of adaptation problems make decision analyses valuable?

Adaptation problems passing the six-step screening procedure are likely to benefit from a comprehensive decision analysis involving uncertainties and multiple decision stages. Outlining the problem characteristics that increase the value of decision analysis can guide the design of benefit-cost analysis of near-term commitments even when the formal six-step procedure cannot be implemented. We identify three characteristics—fitness, importance, and measurable performance—that can make a comprehensive analysis particularly useful for climate adaptation. Each is broken down further, yielding a total of nine specific criteria, which we use below to rank problems. The detailed definitions and rating schemes for the nine criteria are shown in Table S1.

(1) Fitness. By “fitness,” we mean that an adaptation problem has features that a comprehensive decision analysis can usefully address. First, the optimal decision is sensitive to future climate where the rankings of alternatives vary depending on which climate scenario occurs. Such adaptation problems often consider large and irreversible near-term commitments, span a long planning horizon, and involve alternatives whose long-run performance is unclear due to fundamental climate uncertainties (Hallegatte 2009). In section 2.2’s example, severity of future flooding depends on future climate states, which are difficult to predict when making initial decisions. The optimal decision changes when we consider different climate scenarios (step 1 base-case analysis with benchmark scenario and step 2 analysis with extreme scenario). If only expected climate change is considered, the “flaw of averages” could result in poorer expected performance than an optimal strategy that considers risk (step 4 multi-scenario analysis without adaptive options).

A second feature is that it is possible to add flexibility in the decision system. Flexibility often includes delaying or modifying investments or rules when better climate information is gained (Woodward et al. 2014). In our example, delaying first and deciding later when better information becomes available (step 6 multi-scenario analysis with adaptive options) is the best near-term decision. Near-term risks might increase, but there may compensate savings in investment costs as well as better long-run outcomes. Two mechanisms can explain the value of flexibility (Gersonius et al. 2013). The first is it avoids irreversible investments at the initial stage but reserves the option to expand later if it is necessary. The second is interest savings from delaying investments.

Thus, high sensitivity to climate uncertainty and the possibility of enhancing system flexibility are features that make a climate adaptation problem more likely to benefit from a multi-stage, scenario-based decision framework.

In summary, “fitness” comprises four criteria: (a) multiple climate scenarios; (b) climate relevance; (c) multiple and complex near-term alternatives; and (d) long-term flexibility.

(2) Importance. By “importance,” we mean that an adaptation problem has short-term urgency and its alternatives involve objectives of high concern to stakeholders. In our example, flooding will potentially cause large damages if no adaptation is made, especially in the long-term if severe climate change occurs. Recent hurricanes have raised the public visibility of this threat. Importance can be gauged by the potential magnitude of regret (step 3 regret check). For example, if the owners of the substation decide to take no action but severe damage takes place, the total costs could be $1000K more than the cost of the optimal strategy (build a floodwall). A thorough decision analysis can identify ways to significantly reduce the potential regret.

Multiple objectives considering significant social, environmental, and other co-benefits can increase public concern, and therefore may make a decision even more important. In our example, in addition to repair costs, there may be large social impacts if extensive power outages occur. The aesthetic impacts of floodwalls may also matter to the local community. In addition, some adaptation alternatives may yield significant co-benefits. For example, floodwalls might enhance security against sabotage threats.

To summarize, decision “importance” can be assessed in terms of three criteria: (a) short-term urgency; (b) size of benefits/costs; and (c) significance of co-benefits.

(3) Measurable performance. By “measurable performance,” we mean that how well adaptation strategies perform under various climate scenarios can be meaningfully quantified. In our example, construction costs are relatively easy to estimate, and we might rely on existing studies of the extent of storm surge under different climate scenarios. Availability of such studies or expertise increases the insights and trustworthiness of comparisons of alternatives.

In addition, high measurable performance means that there exist potential collaborators such as researchers, local governments, or non-governmental organizations. Partnerships enlarge the group of experts available for obtaining the data necessary to do a decision analysis.

To conclude, “measurable performance” encompasses two criteria: (a) quantification difficulty and (b) partner availability.

The above discussion explicitly defines the meaning of “fitness”, “importance”, and “measurable performance” and reasons why each contributes to making decision analysis particularly insightful and useful for climate adaptation problems. In the next section, we present a framework that employs the nine specific criteria to quantitatively compare different adaptation problems in terms of the applicability of decision analysis. We then use the framework to rank twelve adaptation problems in the Chesapeake Bay region.

4 Procedure 2: determining which problems might benefit from comprehensive decision analysis

4.1 A general framework

In section 2, we looked at one adaptation problem and provided a quantitative approach to help select the type of decision analysis that could be most useful. This procedure can help planners who are focused on a specific adaptation problem.

However, planners often face several adaptation problems simultaneously. It is neither practical and necessary to apply the six-step screening procedure to each problem, nor can this procedure readily rank adaptation problems in order of net benefit. With limited analytical resources (budget, personnel, time, etc.), a quicker way to set priorities among problems would be useful. Therefore, we now devise a framework to compare multiple adaptation problems in terms of the necessity and value of a thorough decision analysis. We quantify the nine specific criteria outlined in section 3 on a 0–5 scale using expert judgment (Table S1). These ratings can then be used to rank the relative value of comprehensive decision analyses for different adaptation problems; in section 4.2, we illustrate this framework by comparing 12 problems in the Chesapeake Bay region. This comparison was part of a research planning exercise by the Mid-Atlantic Regional Integrated Sciences and Assessments (MARISA) program (www.midatlanticrisa.org).

There are two steps involved in applying this framework.

Step 1: characterize adaptation problems. The first step is to describe candidate adaptation problems, based on the literature and interviews with experts, managers, and stakeholders. The following information is needed for each problem: (1) what concerns and objectives do managers and stakeholders have?; (2) what types of local hazards or other impacts arise from climate change (e.g., sea level rise, changing precipitation, rising temperature)?; (3) what near-term adaptation investment or regulatory commitments might be feasible, and what longer-term options exist to modify near-term plans?; and (4) how might uncertainties concerning climate impacts affect the estimated long-term performance of near-term commitments?

In our case, we defined a representative set of general adaptation problems in the Chesapeake region that encompass the range of issues and decisions faced by resource managers and policy makers. After extensive interviews with managers, researchers, and stakeholders, we identified a large set of specific problems, which we then grouped into coastal and inland flooding, water pollution, and heat impacts. In contrast, for an agency with a specific domain of responsibility, the set of problems might instead include several location-specific instances of one problem type, such as candidate locations for storm surge protection.

Step 2: rank general adaptation problems. When one wants to rank alternatives (here, problems) that differ in many dimensions that are difficult to compare yet all matter, it is widely recognized that, first, multicriteria analysis is a practical and insightful way to compare alternatives and, second, additive value functions are a transparent and relatively simple to apply MCA method. Practice and the literature (Belton and Stewart 2002) indicate that additive functions are widely used and can effectively deal with multiple objectives that are valued differently by different people. Here, users score the problems in terms of section 3’s nine criteria and also weigh criteria in terms of relative importance. Guidelines for defining scoring metrics should be provided for rating the problems. Table S1 shows an example, but users should design their own metrics based on their preferences. Involving a range of experts, such as academics, local planners, environmentalists, and engineers, in these assessments can ensure that different perspectives are taken into consideration (Burgman 2016). Then each of the problems is scored as follows:

$$ {S}_i=\frac{1}{K}\sum \limits_k\sum \limits_j{s}_{i,j,k}\times {w}_{i,j,k} $$
(1)

where Si is the overall score for adaptation problem i, si, j, k, and wi, j, k are the score and weight assigned for criterion j of problem i by expert kFootnote 1. A higher Si means the higher applicability and potential value of decision analysis. Equation (1) is, in essence, an additive multicriteria value function. Attention needs to be paid to ensuring that weights indeed reflect priorities of the participants. A more informative analysis would explore how different perspectives (weights from different people) would change the problem ranks, because the implications of different perspectives might actually be more interesting to managers than some hypothetical average. This is an acknowledged principle of good multicriteria decision making (Keeney and Raiffa 1993).

4.2 Case study: Chesapeake Bay watershed

The Chesapeake Bay watershed (CBW) covers 64,000 mi2 across six states and Washington, DC and is home to diverse natural communities and ~ 17 M people. This region confronts many climate-driven risks whose magnitudes and implications are not fully understood. There are numerous public and private sector decisions in which commitments are being considered today whose net benefits could be dramatically affected by climate change. Typically, such decisions involve large investments or regulatory commitments that will affect system function well into the future. Examples include: investment in infrastructure such as sewerage upgrades in response to CBW nutrient and stormwater mandates; renovation of the Conowingo dam for ecosystem restoration or sediment management; utility and transport infrastructure investments in flood-prone areas, such as the Anacostia area in Washington, DC; and proposals for gray or green coastal infrastructure for reducing shoreline erosion and flooding in areas threatened by sea level rise and increased storm severity. Given time and staffing limits, the MARISA team needs to focus on adaptation decisions that are most likely to benefit from better climate information and careful risk analysis. We therefore applied our nine-criterion scoring framework to compare twelve general CBW adaptation problems.

Step 1: identify general adaptation problems in the CBW. From the existing literature and interviews with 24 adaptation managers and expertsFootnote 2, we identified approximately 40 specific adaptation cases in the CBW. The main climate-related risks are coastal flooding, inland stormwater, and extreme heat. Many studies concern flood risks for infrastructure and development since several states have long coastlines; adaptation to increased risk of coastal flooding can involve large investments in repair and upgrades.

The 40 cases were grouped into 12 general adaptation decision problems (Table 2) for which climate risks are relevant, which were further aggregated into four broad categories of CBW management concerns: natural resources management, infrastructure management, land use protection, and public health and safety. Classifications of the 40 case studies and 12 general problems by risk type and broad decision concern are displayed in Table S2 and S3, respectively.

Table 2 Twelve general adaptation problems in the Chesapeake Bay area and ranks
Full size table

Step 2: ranking adaptation problems. We surveyed six experts from the MARISA team working in academia (Penn State University) or in think tank (RAND Corporation) with relevant experience on climate adaptation and decision analysis. They rated the problems with scoring metrics we provided (Table S1). Figure 5 shows the ratings for the three general characteristics of fitness, importance, and measurable performance (averaged across experts and constituent criteria).

Fig. 5
figure 5

Average scores of three general characteristics for 12 adaptation problems

Full size image

As that figure shows, the problem “coastal protection infrastructure” receives the highest scores for the general characteristics of “fitness” and “importance.” It also scores second highest for “measurable performance,” slightly lower than “coastal land acquisition.” By contrast, “heat resistant pavement” has relatively low scores in all three dimensions, especially in “importance.” Viewed together, these rankings also indicate that trade-offs will need to be made. For instance, “substation hardening” ranks higher in “fitness” than “green infrastructure investment,” but ranks lower in “importance” and “measurable performance.” The overall score Eq. (1) is used to rank our 12 problems for relative value of decision analysis. The average weights of the three characteristics are 0.34, 0.35, and 0.31, respectively. It is clear that experts value each characteristic but place slightly more value on “importance.”

We calculate the total score for each problem, and show the overall rankings in Table 2. Consistent with Fig. 5, two coastal protection problems (involving construction or land acquisition) have the strongest likelihood to benefit from a comprehensive decision analysis. This type of problem is a good fit for decision analysis because it has a long-term planning horizon, and the performance of near-term alternatives is sensitive to climate change. The region has long, densely populated coastlines, so protecting coastal areas from flooding and inundation is highly important. Moreover, numerous organizations such as the Maryland DNR are working on coastal protection projects in the CBW, providing opportunities for collaboration, which addresses the characteristic of measurable performance. Similarly, “green infrastructure investment” possesses those three characteristics as well, but instead involves investments to reduce urban stormwater runoff and nonpoint pollution, both of which could be affected by increased storm severity.

The six experts assigned different weights to the nine characteristics, and so their specific rankings of problems also differ, as indicated in Table S4. However, there is broad agreement on the most and least suitable problems. For instance, four experts identified “coastal protection infrastructure” as the most suitable problem and the other two identified it as second most suitable. In order to further test the sensitivity of rankings to the weights, we randomly changed the weights that each expert assigned. In particular, for each weight, we independently selected a weight from a uniform distribution [0.5X, 2X], where the original weight is X. We repeated the sampling 100 times. We then checked if ranks fluctuated drastically by calculating the standard deviation of each decision’s rank over the 100 repetitions (Table S4). Overall, sensitivities are fairly low, with the standard deviations less than 1.5 in nearly all cases and with only one above 2.0. This suggests that different weights do not drastically change general conclusions about which problems are most suitable.

5 Summary and discussion

Climate adaptation planning is becoming increasingly urgent. Uncertainty is perceived as a major obstacle to assessing and ranking different adaptation strategies, especially when they involve long-lived investments, regulatory reforms, and other difficult to reverse commitments. Considering multiple scenarios can reduce the risk of making—and, later, regretting—a suboptimal decision chosen based on just one single scenario. Further, recognizing future options can improve plans by quantifying the benefit of flexibility to modify plans later after resolving uncertainty.

Decision analysis can help analyze climate uncertainties and future adaptation options, yet the expense of doing such a thorough analysis must be justified. With the six-step screening procedure proposed in section 2, a manager can quickly determine which is the most appropriate type of decision analysis for a particular adaptation problem: (1) predict-then-act analysis (type I), (2) multi-scenario analysis without adaptive options (type II), or (3) multi-scenario analysis with adaptive options (type III). The screening procedure will help reduce the risk of investing resources in a more elaborate analysis when simpler ones will do. However, adaptation managers might also be responsible for multiple adaptation problems at various locations. For that situation, we propose a two-step prioritization procedure in section 4 to identify adaptation decisions that are most likely to benefit from a comprehensive uncertainty-based decision analysis.

With these two complementary procedures, the effectiveness of adaptation management plans can be increased by quantitatively considering whether uncertainties and flexibility can affect decisions concerning near-term commitments to reduce vulnerability.

When rankings of near-term commitments would not be altered by a more sophisticated multi-scenario analysis (type II or III), adaptation managers can address just one or a few climate scenarios, and not consider the complexities resulting from including later adaptation possibilities. However, if rankings depend on which climate scenario is modeled or whether the flexibility to change course later is considered, then a more complex decision analysis is justifiable. Managers should then consult with climate change experts and officials about the relative likelihoods of various climate scenarios and how they might affect the performance (cost, risk, or environmental impact) of alternatives. Although precise probabilities are not required to make informed decisions about adaptation (Groves and Lempert 2007), some sense of the plausibility and relative likelihood of scenarios is useful for obtaining more precise conclusions about which alternatives may have the highest net benefits. An advantage of undertaking a type III analysis of multistage adaptation options under risk is that it will focus a manager’s attention on how flexibility can be built into a plan so that it can be modified in response to climate and social developments, as information improves. For instance, options can be preserved by modular infrastructure designs or delayed decisions. The possibility of acquiring more information can then be valued. This is the basic philosophy of adaptive environmental management (Holling 1978).

The intended practitioners are people and organizations who are responsible for city or regional climate adaptation. Examples include municipal sustainability offices and regional water management organizations. These practitioners often have significant expertise concerning climate change and adaptation. The initial screening/ranking process can be implemented by making judgments regarding probabilities of climate scenarios and costs and benefits associated with each adaptation alternative under each scenario. Such assessments might be approximate, but nonetheless useful for assessing whether comprehensive analyses are likely to be insightful and useful.

However, when applying any of the methods we have described, users should recognize that the quality of the analysis depends on the willingness to consider a wide range of possible actions and scenarios, and by the quality of inputs such as expert advice. Therefore, it is essential to draw upon the expertise of individuals from diverse backgrounds and perspectives. When outside experts are available, such as local universities, environmental groups, or consultancies, practitioners should try to involve them.

In conclusion, decision analysis can provide significant value and new insights for climate adaptation problems. But such analyses take effort that must be justified by the benefits gained. The two complementary procedures we have proposed can assess whether a thorough uncertainty-based decision analysis is needed for a specific adaptation decision and can identify which adaptation decisions can be significantly improved by such an analysis. The two procedures can improve the effectiveness and efficiency of analyses by avoiding wasting time and personnel on problems that are unlikely to benefit from a thorough study. Future research should develop guidance concerning which specific methodology of decision analysis—e.g., traditional decision trees (Hobbs et al. 1997), ROA (Woodward et al. 2014), DAPP (Haasnoot et al. 2013), or RDM (Groves and Lempert 2007)—is most appropriate for particular adaptation decision problems.