1 Setting the context

A critical challenge today is that, if man-made emissions of greenhouse gases are not urgently and drastically reduced, human societies across the globe may be confronted with dangerous tipping points in the climate system. Uncertainties in the precise amplitude of long-term changes in the global climate and their exact potential impacts at the regional and local scales continue to persist. However, there is strong consensus amongst climate scientists that our planet’s climate is shifting rapidly and that recorded climatic changes, particularly since mid-nineteenth century, are very likely to be human-induced through the accelerated release of greenhouse gas (GHG) emissions in the atmosphere (AAAS 2009; IPCC 2013). More extreme weather events such as floods and droughts are already being observed and there is growing evidence and confidence that these are linked to anthropogenic increases in GHG atmospheric concentrations (Panton et al. 2008; IPCC 2012, 2013).

Strong and rapid climate change mitigation and adaptation actions are warranted worldwide. However, international and national policy responses on this front have hitherto been poor, to say the least. Current worldwide climate policy pledges are far from sufficient or able to make a significant dent in curbing the relentless rise in global greenhouse gas emissions and meet climate change targets, such as the 2 °C global warming target by end of 2100. Even if one takes an optimistic view and assumes that all current country pledges to reduce emissions are to be implemented in practice, a significant gap between emission levels consistent with a 2 °C limit and those resulting from the pledges would still be expected in 2020 (UNEP 2013). Furthermore, this gap has been increasing over recent years, meaning that the world would have to rely on more difficult and costlier means after 2020, in order to keep the rise in global average temperature below 2 °C by end of this century (UNEP 2010, 2012, 2013).

Reasons why climate pledges and actions are far from sufficient may be at least partly attributed to intertwining political and epistemological factors. On one hand, there has been a lack of political will and poor international institutional and resource mobilization to this end. On the other hand, a large body of standard climate economics literature has mostly emphasized the burden of structurally changing energy and economic systems towards climate resilient low-carbon development pathways. Mitigation-induced economic welfare losses or the costs of compliance as percentage of national income relative to a baseline are frequently highlighted (e.g. Nordhaus 2010; Edenhofer et al. 2010), whilst important issues of risks and ethics (Dietz and Stern 2008) and the potential benefits (including avoided damages) that might endogenously emerge from such transformational shifts are often side-lined. In other words, the climate policy planning process is complicated by the sheer complexity of the linkages in terms of synergies and trade-offs between climate change-related policy goals and broader developmental policy objectives.

Despite the absence of a global climate deal, both developed and developing countries are nonetheless increasingly taking action at the national and sub-national levels, for a variety of reasons, such as energy security, energy dependency, and local pollution (Climate Policy Initiative 2013). There is also heightened demand for practical assistance to governments, particularly in developing countries, in preparing their climate change mitigation and adaptation strategies and accessing international climate finance. Some developing countries are still at an early stage of developing formal climate change policy plans and identifying specific nationally appropriate mitigation actions (NAMAs) and national adaptation plans and actions (NAPAs).

The aim of this paper is to put forward a novel methodological framework that takes the above challenges into due consideration and offers an alternative practical approach to the formulation of climate policy plans, particularly at the country level. The methodology, which we label MCA4climate (Multi-Criteria Analysis for climate change), draws on a combination of new economics of climate change and a multi-criteria decision analysis approach to identifying, assessing, and prioritizing climate policy options that support national development goals and account for the relevant ethical values at play.

2 A brief review of methodological approaches to climate policy planning

Reducing man-made emissions is conventionally associated with burden sharing and sharing the pain, instead of sharing the benefits and allowing all parties to gain from universal access to clean energy services (Moomaw and Papa 2012). The possibility of policy-induced green economic growth occurring at greater rates than those of business-as-usual or brown growth is typically ruled out by default in standard economics. This typically draws on representative-agent utilitarianism, perfectly rational and self-interested behaviour, competitive general equilibrium theory, and optimization techniques, whilst neglecting strong kinds of uncertainty, such as fundamental uncertainty (Dequech 2008). According to traditional economic theory, the economy has an equilibrium point to which it naturally progresses, which has raised major concerns as to its ability to meaningfully represent socioeconomic realities (Beinhocker 2007).

There are certainly, particularly upfront, investment costs or expenditures involved when shifting efforts and resources away from current practices towards more sustainable societies. However, any investment bears a return, and there are strong reasons to believe that these may substantially outweigh the costs, particularly when costs are defined in a broader social sense. Furthermore, even when costs are defined in strict economic terms, stringent climate stabilization efforts may still result in macroeconomic benefits and increased economic output. This may occur via technological innovation dynamics, improved competitiveness, shifting the tax burden from employment and income to environmental pollution, and market diffusion and spill over effects induced by global trade and technological transfer (Jochem and Madlener 2003; Barker et al. 2012; Bosquet 2000). System-wide effects need to be also considered, as an intervention in a particular sector may reverberate across the entire economy. For example, increasing active travel (walking and cycling) instead of using private cars has been shown to reduce costs to the healthcare services (and improve fiscal sustainability prospects) by reducing prevalence of some chronic diseases (Jarrett et al. 2012). Investment therefore in transport infrastructure to promote active travel and reduce emissions can lead to cost savings and benefits elsewhere, such as the health sector.

No-regret options can also offer substantial incentives for climate action, although these are largely regarded as incompatible with traditional economics since it is assumed that if such options were possible they would have already occurred under optimal equilibrium (Maréchal 2007). Having said this, there are some recent developments in the CGE (Computable General Equilibrium) traditional economics literature that accommodate the suboptimal behaviour of economic systems, such as the DSGE (Dynamic Stochastic General Equilibrium) models. These allow for suboptimal macroeconomic behaviour, such as the existence of involuntary unemployment (Kemfert 2003). Nonetheless, few have been applied to climate policy analysis, and, furthermore, they still assume that economies revert to market equilibrium conditions in the long run, which may not necessarily be the case. Some examples include the QUEST (QUarterly European Simulation Tool) III new-Keynesian DSGE model used by the European Commission (Conte et al. 2010), a DSGE model for the Polish economy (Bukowski and Kowal 2010), and a DSGE model for China (Schenker 2011). In addition, optimal growth and equilibrium models function on the descriptive representative-agent assumption, which has been shown to cause an intrinsic (regressive) distributional bias in favour of the rich and produce questionable optimal emissions recommendations (Skott and Davis 2013). This constitutes a serious issue for social wellbeing should climate action follow in the footsteps of such recommendations.

The stern insistence on the traditional economic (static and inter-temporal) optimization and equilibrium theory has resulted in the dominance of a particular methodology for framing thinking and decision-making in a large range of economic, social and environmental problems, including that of climate change, i.e. the cost-benefit analysis (CBA) approach. CBA provides a strong theoretical framework for the maximization of resource allocation efficiency, with the particular standard variant of market or price-centred valuation being the most commonly used (Sen 2000). This takes a utilitarian perspective by providing monetary valuations to all impacts involved. It compares the marginal costs of a mitigation or adaptation policy with the marginal benefits associated with the climate change that is avoided (including ancillary benefits), in order to identify the most beneficial (economically efficient) policy response (Dessler and Parson 2006). Marginal in this context refers to the additional cost that will be incurred by the current emission to the atmosphere of one unit of greenhouse gas. CBA may be well suited for the pure financial feasibility of investment projects or efficient financial allocation decision-making, where future financial flows may be readily identified and predicted, monetary aggregation justifiable, and price setting clear-cut. Nonetheless, the standard CBA approach has major limitations when applied to the long-term, multi-dimensional, challenging problem of climate change. This is in part because of “the incredible magnitude of the deep structural uncertainties that are involved in the climate change analysis” (Weitzman 2009: 18).

Within the context of dealing with climate change with highly complex features, such as future time, doubt, and irreversibility, standard CBA falters and implementing it is no longer a technical task because many subjective choices are due (Verbruggen 2013). Several non-market impacts, or externalities, are difficult, and we would argue even unethical, to price, and as a result do not figure in the evaluation of costs and benefits. There are also fundamental concerns about intergenerational equity (Broome 2008) and, therefore, the appropriate discount rate to use in CBA analysis. CBA studies are highly sensitive to the choice of discount rates (Egenhofer et al. 2006; Wright and Erickson 2003; Ackerman 2008), reducing the robustness and reliability of their findings and estimations.

Despite the dominance of the CBA method, particularly in its traditional format which does not incorporate issues of deep uncertainty and stakeholder participation (Chambwera et al. 2012), alternative economic approaches and models are nonetheless being developed and increasingly used in the analysis of and decision-support for climate action. Such alternatives include cost-effectiveness analysis (CEA), robust decision-making approaches (RDMA) and multi-criteria decision-analysis (MCDA), which are briefly summarized and compared in Table 1.

Table 1 A summary of different decision-support techniques, their pros and cons, and their suitability for being applied to climate change policy planning
Full size table

CEA is a technique that can be used to identify least-cost options to meet a certain, pre-defined or fixed target or policy objective, for example, in the case of mitigation, the reduction in GHG emissions to particular levels at different periods in time (Haines et al. 2009). As policy intervention costs constitute the key variable of consideration and as it is subjected to finding cost-minimal solutions, CEA does not necessarily require the quantification of benefits, which can be fixed beforehand, such as reducing disaster fatalities and losses. A critical question, however, remains the identification of a threshold (e.g. permissible increase in global average temperature or the willingness to pay per unit gain), which may also be the subject of heavy debate.

RDMA methods have seen limited use in the area of climate change to date, though they are receiving increasing attention. RDMA essentially provides an analytical decision-support framework for situations characterized by high uncertainty. Within this context, uncertainty refers to the lack of agreement among interested parties, lack of analytical approaches to analyze the issue at hand, lack of knowledge about the state and trends of the parameters affecting that issue, or any combination of these. Rather than attempting to make decisions on the basis of predictions of future states in variables of interest, RDMA attempts to identify the full range of plausible future states and, on that basis, make decisions that are robust across as wide a range as possible of those future states. A key aspect is the notion of iteration and repeated analysis with modified assumptions and scenarios. Two main families of RDMA approaches can be distinguished, static and dynamic. The latter take better account of cost-effectiveness considerations, but are much more demanding in terms of human capital and data collection capacities. A recent effort to prepare a water management plan for the city of Ho Chi Minh in Vietnam illustrates the trade-offs between these two approaches (World Bank 2013). Quantitatively, it may mean running many simulations for tracing out uncertainty across key variables. Methods are nevertheless rather complex and often require the use of advanced statistical and mathematical methods (Lempert and Collins 2007; Lempert 2012; Ranger et al. 2010).

In the area of climate change, MCDA studies constitute a relatively narrow body of analysis in comparison to more established evaluation methods when applied to climate change policy analysis. Indeed, despite MCDA being recognized for some time as a valid tool with an important role to play in evaluating trade-offs between climate policy alternatives over multiple, disparate and often conflicting criteria (Bell et al. 2001, 2003), its use in this area remains limited. MCDA has been nonetheless applied extensively to environmental management choices (e.g. Gregory et al. 2012; Khalili and Duecker 2013; Hämäläinen et al. 2010; Kiker et al. 2005; Huang et al. 2011). We would argue though that both the nature of the climate change policy decision problem and the societal responses to this differ significantly from those associated with most environmental issues. Firstly, from a scale perspective, climate change is affecting our entire planet, both human societies and ecological systems, whereas environmental problems occur at a smaller scale, and are typically dealt with locally. Secondly, solutions to climate change entail systemic shifts or deep transformations in our energy and economy, whereas environmental problems can often be remedied without necessarily requiring the fundamental restructuring of our production systems (e.g. end-of-pipe technologies dealing with local pollution versus widespread replacement of oil refineries with low-carbon technologies mitigating GHG emissions).

Having said this, some potential benefits of using MCDA have been demonstrated recently in: the assessment of emissions control options (Solomon and Hughey 2007); carbon capture and storage measures (Gough and Shackley 2006); mitigation policy instruments (Konidari and Mavrakis 2006, 2007; Grafakos et al. 2010); sustainable energy options (Wang et al. 2009; Ehrgott et al. 2010b); and prioritization of water management schemes (Yang et al. 2012; Yilmaz and Harmancioglu 2010). However scarce they may still be, these studies have made some headway in demonstrating the value of using MCDA tools to climate decision-making.

Notwithstanding, the use of alternative frameworks such as RDMA and MCDA continues to be in severe minority in the area of climate change. In addition, the economic thinking underpinning assessments that are used in both CBA and non-CBA studies is seldom questioned, particularly in terms of departing from the traditional optimization equilibrium economics advocating monetization of all impacts and imposing representative and fully rationalistic behaviour on all agents of change, For example, a recent survey of the literature (Scrieciu et al. 2013a) revealed that, out of 30 climate-economy models considered in seven widely-cited model comparison studies in the area of climate mitigation economics, including those used in the reporting of the United Nations Intergovernmental Panel on Climate Change (IPCC), only one model (the E3MG model as in e.g. Barker et al. 2012) adopted a non-optimization and non-equilibrium simulation approach to the issue. Finally, the wider development implications of climate interventions, such as the scope for poverty alleviating mitigation action at the national or local level continue to be insufficiently understood, although important efforts have been made in this respect. For example, the Climate and Development Knowledge Network (CDKN), the Mitigation Action Plans and Scenarios (MAPS) programme, and the Low Emissions Development Strategies Global Partnership (LEDS GP), to name but a few, represent important institutional and research-support efforts in this respect. As is gradually being recognized, benefits essentially cut across many sectors and actors, and strongly interact with non-climate policies, one of the implications being that the standard CBA approach with its narrow focus on monetary outcomes becomes severely challenged (Mechler 2013).

3 Conceptualizing the MCA4climate framework

The MCA4climate methodological framework that this paper advocates is chiefly based on three crucial components, which are captured in Fig. 1. In their totality, these aim to support a systematic assessment and strategic planning of development-compatible climate actions. The proposed framework is chiefly concerned with issues actively operating at the interface between climate policy and wider development issues, and also incorporating, where feasible, mitigation-adaptation interactions. It seeks to help identify which climate policy actions have the most potential in influencing contextualized human development and environmental sustainability outcomes, whilst meeting their primary climate objectives.

Fig. 1
figure 1

The general structure and main components of the MCA4climate methodological framework approach. The top left circles illustrate the scope of the MCA4climate methodological approach, which is to identify, assess, and prioritize climate policy options that take account of the interactions between mitigation, adaptation, and development. The bottom right hand panel of circles portrays the three main components that form the basis of the methodological framework explained and presented in this paper

Full size image

First, there is a set of guiding principles or good practice evaluation standards that is argued to be critical for a meaningful and robust formulation of pro-development climate policies. Second, there is a comprehensive and systematic criteria tree, which provides a generic and structured development impact assessment and climate policy evaluation template, spanning financial, economic, social, environmental, climate and institutional dimensions. Alongside this generic criteria tree, for each of the key areas of adaptation and mitigation, a detailed folio of expert advice could support the interpretation and customization of the generic criteria tree to a specific area of climate policy intervention. And third, there is the process of developing and using MCDA models to support the transparent identification and prioritization of development-compatible climate policy options. This process necessarily involves and thereby facilitates the active engagement of stakeholders over the entire duration of the policy design, evaluation, planning, and implementation phases, through a series of well-defined steps that operationalize the conceptual approach. The first two components are further described in this section, whereas the MCDA process is addressed in the following section.

The MCA4climate framework has been primarily developed with a view to ex ante applications, i.e. to evaluate policies before they are adopted. It is equally valid though to apply the approach to ex post evaluations. In this case, monitoring, reporting and verification issues and ensuring flexibility in policy design play an important role (e.g. focusing on the feedback loop between the third and first components of the MCA4climate framework as illustrated by the dashed arrow line in the right side of Fig. 1).

As such, the added value of our proposed approach to the existing literature on climate policy impact assessment and decision analysis is fourfold. First, as its name suggests, it focuses on the application of multi-criteria decision analysis to the climate problem and its policy responses (discussed in more detail in Section 4), which as outlined above has been less prevalent in the climate policy literature so far. Second, MCA4climate offers a generic framework that may be applied across a wide range of mitigation and adaptation topics and policy options. It is generic, since its fundamental approach and structure can be translated into context-specific variants tailored to a wide array of national or local circumstances. In other words, the framework being advanced in this paper is arguably both widely applicable and flexible enough to cater for the diverse circumstances in which climate decision-making is typically embedded. This is particularly valid within the context of developing countries, for which institutional barriers to climate action are prevalent and for which the climate problem may not figure high on national or regional policy agendas, unless they are explicitly coupled to development objectives. For instance, a major component of our framework is its institutional or governance dimension, against which any climate policy measures need to be evaluated in order to render meaningful situational descriptions. This supports both the universal applicability and contextualization of the approach, and lends a more practical dimension to planning and prioritization of actions.

Third, the economic principles underpinning the MCA4climate approach depart significantly from traditional economic theory, particularly in its neoclassical utilitarian form and its associated value-neutral equilibrium and optimization modelling apparatus. In other words, a different kind of economics (relative to CBA economics) is called for, which embraces an interdisciplinary perspective, pluralism, and combines objective assessments with value judgements. Furthermore, evaluating future socioeconomic impacts, and the benefits and costs of mitigation and adaptation policies, typically involves the use of detailed empirical research and modelling. This inevitably rests upon a number of choices about the methodological approach and underlying assumptions, which have important consequences for socioeconomic projections and, therefore, the ultimate selection of policies to be implemented (Scrieciu et al. 2013b). Chief among these choices are baseline macroeconomic assumptions; technological innovation, learning, dynamics and feedbacks; no-regrets options for mitigation and adaptation; monetary valuation and non-marketed impacts; discounting future costs and impacts; time horizon of the analysis; and risk and uncertainty. Traditional economic approaches to these issues are arguably ill suited to offer good practice evaluation standards in this respect. This is partly because human preferences, ecological properties, and technological possibilities cannot be valuated solely through utilitarian lenses and standard welfare economic theory (Söderholm and Sundqvist 2003), but would require instead a conceptual pluralism approach to the concepts of value, value systems, and valuation (Farber et al. 2002). It is also partly because standard economics techniques tend to neglect important ethical questions (Booth 1994) and overlook the variability in value judgements across population and across time (the so-called value heterogeneity and value endogeneity as defined in Sen 1988). Put differently, the MCA4climate approach being proposed goes beyond the mere application of decision analysis to climate policy making, and attempts to reshape economic thinking underpinning climate policy analysis and the evaluation of related development impacts.

Fourth, the MCA4climate perspective aligns well with recent thinking prevalent in climate science and economics regarding notions of low-regrets, uncertainty, iteration and process focus. As one example, the recent United Nations Intergovernmental Panel on Climate Change (IPCC) report on Managing the Risks of Extreme Events and Disasters to Advance Climate Change Adaptation (IPCC 2012) demonstrates that climate change is shaping risks from climate extremes and there is the need for bolstering resilience to climate impacts. At the same time, it shows that, at least in the short to medium term, many non-climatic factors are fundamentally driving risk (Rogelj et al. 2013). As well, uncertainties about future projections of risk are numerous and unlikely to go away soon. The IPCC suggests that an approach grounded in a low regrets strategy may usefully provide for more robust mitigation and adaptation strategies. Such an approach has high potential for reducing long-term risk and at the same time provide for short-term benefits in terms of, for example, reducing vulnerabilities today in the case of adaptation. As one consequence, the above-mentioned IPCC report suggests to focus more strongly on iteration, process focus, learning and innovative thinking, which are attributes that can be closely aligned with a multi-criteria approach (IPCC 2012).

In short, we argue that the combination of these four factors summarizes the contribution of our MCA4climate approach to the literature situated at the intersection between climate policy analysis, climate economics, development impact assessment, and decision analysis strands. Moreover, the proposed methodological framework rests at the border between research and practice, and, as such, it aims to render methodological thinking in this area practical and with immediate application to real-life climate change related decision problems.

3.1 Guiding principles and good practices for climate policy evaluation

New thinking on the economics of climate change and analytical tools for decision support are emerging in response to the limitations of traditional economic approaches, and their assumptions on economic behaviour, ecological properties, and socio-technical responses. It may be argued that new economics brings under one umbrella a series of common elements spanning disparate schools of economic thought and can draw on both mainstream and heterodox thinking. As argued in Barker (2008), the term new economics may be understood through the prism of Boulding’s work (1992) to include systems thinking, complexity, evolutionary and Post Keynesian theory with an emphasis on institutions, non-linear dynamics, and deep uncertainty. In other words, the new economics literature puts forward analysis that overall departs from the standard practice in contemporary economics of combining optimization, equilibrium, and the aggregation of heterogeneous actors as per the representative agent assumption. With application to climate change, a new economics approach would explicitly account for systemic effects, risk and uncertainty, technological change, multidimensional impacts across space and time, ethical perspectives of multiple stakeholders, and the institutional constraints and drivers for climate policy implementation (Barker 2008; Dietz and Stern 2008; Heal 2009; Stern 2007). As such, an adequate all-embracing framework would need to offer alternative approaches to understanding and incorporating underlying value judgments, and consider the multi-dimensional interactions between the economy, environment and society, which often do not lend themselves to monetization and aggregation. It would also need to recognize the importance of catastrophic risks and irreducible uncertainty, warranting a precautionary approach to climate policy. Finally, a solid understanding of the economics of climate change policies would call upon increased empiricism in understanding socioeconomic behaviour and relations, incorporate policy-induced technological change, and explicitly address the role of institutional drivers and barriers to policy planning, implementation, monitoring and verification (Scrieciu et al. 2013b).

The MCA4climate conceptual framework draws on this growing body of new (climate) economics literature. In this respect, we argue that a meaningful, effective and comprehensive assessment and planning of climate policies should rest on three main principles. The first is that climate change policy has multi-dimensional implications for human societies and the environment, affecting multiple interests and calling for the consideration of a wide range of values and priorities. The second principle asserts that policy responses to climate change may contribute, if adequately formulated, towards meeting country-specific development objectives, and that there may not necessarily be trade-offs between climate action and the economic performance or poverty alleviation targets of a given country. The third principle states that non-monetary values, uncertainty and the long-term dynamics of environmental, socioeconomic, and technological systems should be inherent to the formulation of any responses to the climate change problem.

Projecting future impacts, and calculating and exploring likely benefits and costs for society as a result of climate intervention inevitably involve a number of choices on the methodological approach deployed and its underlying epistemological assumptions. As a result, we also identify and propose a set of good practice evaluation standards with an emphasis on the social economic dimension that could better guide future assessments in the area of climate policy analysis and planning. These constitute the first component of our overall MCA4climate methodological framework approach and are outlined in Table 2. They are to be taken as a wish list, as in practice, actors may lack the capacity to implement these, though they should be regarded as potential aspects to be considered. Preferably and where relevant, both mitigation and adaptation options should be included in the evaluation and planning of development-compatible climate action.

Table 2 Proposed critical issues and good practice evaluation standards to be considered in the performance of robust climate policy analysis and impact assessment
Full size table

3.2 Evaluation criteria tree linking climate policy with development

At the core of the MCA4climate framework, we propose a systematically structured and comprehensive hierarchical criteria tree. This contains a set of generic criteria, against which climate policy planners can evaluate proposed climate-policy actions with regard to their potential contribution to a broad range of climate, environmental and socio-economic development objectives (Fig. 2). An important characteristic of the criteria tree and a requirement of the MCDA approach that we are advocating is that the selected criteria are judgementally independent of each other. That is, the assessment of preference with regard to the consequences of policy options against any one criterion is independent of preference with regard to any other criterion. This ensures that options can be scored on one criterion without knowing what the scores are on any other. However, preference with regard to the policy options themselves may not be independent in the sense that it is important to be aware of potential synergies or negative interactions between options. In other words, preferences for portfolios combining two or more options may not be simply derived from (summing up) preferences defined for the options individually.

Fig. 2
figure 2

The generic criteria tree used for evaluating development-compatible climate policy action according to the MCA4climate methodological framework approach. The generic criteria tree of the MCA4climate methodological framework is structured in three layers. The first level consists of inputs (investments and efforts required) versus outputs (impacts) criteria against which climate policy options are evaluated. The second level comprises seven criteria groups, two on the input side and five on the output side, whereas the third-level criteria refer to nineteen criteria, four associated with inputs and fifteen linked to outputs

Full size image

In public decision-making, it is important to ensure that the set of criteria on which an analysis is based is both value-focused and complete, capturing all relevant concerns; that is, the criteria do not reflect a restricted or partisan perspective on the issue, which might lead to a biased evaluation. In this respect, the generic criteria tree displayed in Fig. 2 has been shaped by extensive, multifarious and systematic consultations amongst leading experts in climate change mitigation and adaptation, and other relevant stakeholders spanning academia, multilateral organizations, and governmental bodies. Put differently, the evaluation template provided by the criteria tree in Fig. 2 was formed through an iterative process of group brainstorming and thought exercises involving around 25 experts, who have participated in the 2-year inception phase of the MCA4climate initiative developed at the United Nations Environment Programme (UNEP 2011). The criteria identified were thought to be important due to their prevalence in both the literature and practical work related to development impact assessment, likely climate change damages, and climate policy planning processes. In addition, the criteria are sufficiently generalized to allow for important flexibility in terms of indicator identification and formulation as a function of the needs and priorities of the country or region under consideration.

The aim of developing the evaluation criteria tree has been to alert decision makers and raise awareness of the complexity and multidimensionality aspects of assessing, planning and implementing climate action in a clear and transparent fashion. As shown in Fig. 2, the generic evaluation criteria tree contains three levels or layers for evaluating development impacts of climate action. On the first level, there are two criteria groups: inputs to (or efforts required for) and outputs (or possible impacts) of proposed policy options. On the second level, inputs and outputs are further split into seven sub-groups of criteria. Two of these relate to the inputs or investment efforts necessary to implement climate actions, i.e. public financing needs and implementation needs, whereas five sub-groups relate to the likely impacts or outputs of climate policy implementation, i.e. the climate-related, economic (including fiscal), environmental, social, and political and institutional dimensions of development. Criteria on the output side can capture either positive or negative impacts. Finally, on the third level of the generic criteria tree, there are a total of 19 generic criteria. Four of these are linked to the input side, which include monetary and non-monetary costs that need to be met for effectively carrying out climate policy interventions, and 15 criteria connect to the output side, which relate to specific impacts on society, the economy and the environment.

Based on these third-level criteria, sets of more specific indicators tailored to the issue under investigation could be identified. The bottom-level generic criteria with examples of quantitative and qualitative indicators are summarized in Table 3. More in-depth practical examples on the interpretation and customization of the generic criteria tree to a specific area of climate policy intervention are provided in another three papers that are also part of this special journal issue: Chalabi and Kovats on health and adaptation, Miller and Belton on water resource management, and de Bremond and Engle on the adaptation of terrestrial ecosystems. These represent complementary theme-specific detailed studies that we argue enhance the contribution of the MCA4climate methodological framework being put forward in this paper.

Table 3 Description of the 19 third-level generic criteria considered in the MCA4climate framework, with examples of possible indicators
Full size table

4 Operationalizing the framework: the use of MCDA in practice

The MCDA term is generic and refers to a collection of formal approaches that take explicit account of multiple criteria in helping individuals or groups explore decisions that matter (Belton and Stewart 2002). Approaches differ in terms of their underpinning philosophy and associated preference model, the nature of the underlying aggregation procedure, processes of value elicitation, and the type of the problems to which they are applicable (see Figueira et al. 2005 and Ehrgott et al. 2010a, for comprehensive overviews of the field). The specific approach embedded in the MCA4climate methodological framework is multi-attribute value analysis (MAVA), which is a well-established method used in the practice of MCDA (Belton and Stewart 2002).

The process of applying MCA4climate involves seven main steps that are aligned with the three key methodological phases of MCDA, namely, problem structuring, model building, and use of the model to inform and challenge thinking about preferred solutions. These steps are summarized in Table 4.

Table 4 Main steps pursued in implementing the MCA4climate methodological framework approach to development-compatible climate policy planning
Full size table

Once the context has been established, including clarifying climate-related and other development policy objectives, the key steps are to: identify policy options or policy portfolios to be evaluated; agree, amongst concerned stakeholders, on the criteria and indicators (starting, we would argue, with those already suggested under this framework); agree on scenarios, the timeline of the analysis and methods of assessment (drawing on the guiding principles and good practice evaluation standards described above); score the different options against the agreed criteria; weight the criteria to reflect different stakeholder perspectives and priorities and use these values, together with the scores, to derive a measure of aggregate performance for each option at higher levels of the criteria tree; and, finally, explore these initial results through appropriate analyses including sensitivity analyses. It needs to be stressed that the process should seek to engage all relevant stakeholders, support a shared understanding of the issues, and identify a commonly-agreed way forward. This would ensure that expertise and values are surfaced and appropriately captured, and that all stakeholders have the opportunity to learn from each other and from the process.

The application of the framework up to the fifth step can provide useful insights into how each policy option performs and may be sufficient to inform decision-making, without attempting to prioritize those options in an explicit way. However, proceeding beyond this step calls for judgment in determining the weights to be assigned to each criterion, reflecting the prioritization of development impacts and the values that concerned stakeholders may attach to the corresponding criteria (step 6). This underpins the calculation of aggregate scores at higher levels of the criteria tree in order to come up with a definitive comparative evaluation of all of the options and enable the exploration of the outcomes through sensitivity analysis (step 7). The analysis up to and including step 5 can be done manually, although an overview of the performance of options is facilitated by formal visual presentations of the scores. The calculation of aggregate scores and associated sensitivity analyses can be done using a spreadsheet, but is greatly assisted by the use of customized decision support tools.

4.1 The use of multi-criteria decision-support methods for consistent scoring and weighting

Unlike CBA, multi-criteria decision analysis has not benefitted from the same political mandate and consequently has been less well known and, to date, less widely applied in support of public policy decision-making. Standard CBA has been, for example, the recognized and required approach to policy appraisal and evaluation in the USA and in the UK for a long time (Shapiro and Morrall 2012; Pearce 1998) and more recently across Europe (e.g. since 2000 for the formulation of the European Commission’s Cohesion Policy; European Commission 2008). However, as observed by Gamper and Turcanu (2007), MCDA is gaining momentum relative to the more established analytical methods of cost-benefit and cost-effectiveness analyses in policy evaluation. Others (e.g. Dietz and Morton 2011) have recognized the potentially complementary nature of CBA and MCDA. In particular, the past decade has seen a significant growth in the use of multi-criteria methods across all areas of environmental decision-making (Huang et al. 2011).

The MCDA approach represents a well-defined technique for identifying multiple impacts and aiding decision-making. It facilitates stakeholder engagement and ensures that the different dimensions of climate policies, including those that cannot be easily measured in monetary terms are taken into consideration. Other non-monetary issues (be they quantitative or qualitative), such as morbidity and mortality, equity, environmental damage, avoiding catastrophic risks, and uncertainty can also be taken into consideration, resulting in a more comprehensive analysis of monetary and non-monetary costs, risks and impacts. An attractive feature of some MCDA methods is that the impacts of policies with regard to these different dimensions are assessed by reference to relevant metrics, which are progressively aggregated to provide an overall evaluation if required, but enable options to be compared at the most appropriate level of (dis)aggregation. Thus, the impact of climate policies is broken down into separate elements, for which data can be compiled and assessments made. These independent assessments provide valuable insights into overall costs and benefits, highlighting the strengths and weaknesses of different policies and enabling the identification of dominating and dominated options (Belton and Stewart 2002; Smith and Hitz 2003). In summary, MCDA does not rely solely on the use of market prices and does not impose limits on the forms of criteria or pre-ordain objectives, allowing for consideration of social objectives and other forms of equity rather than focusing only on resource efficiency (Munasinghe 2007).

The first four steps in the MCA4climate conceptual framework (as outlined in Table 4) can be collectively referred to as problem structuring. The importance of effective problem structuring as a participative process was recognized in the field of OR/MS (Operational Research/Management Science) over 25 years ago (Rosenhead 1989) and was soon acknowledged by MCDA researchers and practitioners (Belton 1990; Bouyssou et al. 1993). Belton and Stewart (2010) provide an overview of current thinking and practice with regard to problem structuring for MCDA. The MCA4climate framework was developed through a process which engaged experts across the spectrum of climate-change impacts and in itself provides substantial support for the process of problem structuring in a specific context, including the identification of options to be evaluated and the set of criteria and associated indicators against which to evaluate these. However, as already noted, it is important that this process is an inclusive one that seeks to involve all relevant stakeholders.

The key components of MCDA are: the set of options/alternatives to be evaluated; the criteria against which these are to be evaluated; the scores which define the performance of the options with respect to the criteria; and the weights which reflect the relative importance that decision makers attribute to the criteria. In the multi-attribute value analysis (MAVA) approach to MCDA that underpins the MCA4climate framework, these weights are clearly defined as the trade-offs decision makers would be willing to accept between levels of performance on different criteria, i.e. the increase in performance on criterion A that would compensate for a unit decrease in performance on criterion B. A discussion of the interpretation and assessment of importance weights in different MCDA methods is discussed for instance in Belton and Stewart (2002). In other words, the MCDA component of the MCA4climate framework is chiefly concerned with how to score, weight and prioritize a multitude of climate policy options against a range of development impacts and investment efforts, when several actors are involved in the evaluation and planning processes of development-compatible climate strategies.

The process of scoring options against a specified criterion aims to capture the added value relative to a defined reference point. There are many different ways of scoring options that vary according to the amount of work involved and the extent to which the outcomes are justifiable to a public audience, explainable and replicable. The MCA4climate approach does not impose any particular method for scoring. As an example, the illustrative case studies, which have been undertaken during the initial development phase of MCA4climate approach while hosted at UNEP, primarily adopted a well-specified approach, widely used in MAVA, of direct rating on a 0 to 100 locally defined preference scale (UNEP 2011). Local preference scales are simply scales anchored at their ends by the most and least preferred options on a specific criterion. For example, the preferred option is assigned a preference score of 100, and the least attractive is given a score of zero. Scores are assigned to the remaining options to reflect their performance relative to the two reference points. The underlying scale is an interval scale, which means that strength of preference is represented by relative differences in the allocated scores. Whilst the process of scoring is one that is easy to use in practice, it is important to remember that it should be underpinned by data, to the extent that this is possible, and informed by sound contextual knowledge. The task is a substantial and challenging one that should not be underestimated and would require high level of commitment from the problem-owner. Furthermore, Belton and Stewart (2002) discuss other approaches to scoring that are appropriate in the context of MAVA, for example modelling could be used for some criteria to construct a partial value function that converts the impacts of climate policy options into scores that are comparable.

As with scoring there are many different approaches to weighting criteria and it is important to ensure that the process used is transparent and robust. However, in some circumstances (e.g. the number of criteria is small, or the evaluation reveals a clear pattern of dominating/dominated options), it may be possible for a decision to be made directly from the scoring information obtained, without the requirement for formal weighting of criteria and the aggregation of values. Where this is not the case, the criteria will need, nonetheless, to be weighted. As mentioned above, in a MAVA, criteria weights reflect the relative worth of value added on different criteria. The meaning of the weights and the associated elicitation process must be well understood by those whose judgments the weights reflect, who should be able to explain and justify the outcomes. As with scoring, this can be achieved by a sound, facilitated, multi-stage elicitation process, involving a number of individuals representing the same stakeholder perspective and forming the basis for discussion that seeks to illuminate and reconcile differences.

Weighting has a significant impact on the aggregated scores for each option. Again using the example of weighting used for the stylized cases contributing to the development of the MCA4climate approach (succinctly summarized below), a sound and commonly used method is swing weighting (Belton and Stewart 2002). Swing weighting is based on comparisons of differences in the same way as for scoring using relative preference scales, and is used to determine the weights across the bottom level of the criteria tree. When all weights have been determined the values are normalized to sum to 1 (simply as a mechanism to keep the aggregate scores at all levels of the tree within the range 0–100). Also, depending on the extent to which involved parties can be expected to have similar priorities, a sharing or comparing approach to determining weights may be more appropriate (Belton and Pictet 1997). A sharing approach seeks to attain an agreed set of weights, possibly starting with the assessment of individual values then seeking to reconcile differences through a process of discussion. A comparing approach accepts that different individuals, or sub-groups of stakeholders, will have different priorities and only seeks agreement on weights within the sub-group, going on to compare the resulting overall evaluations across sub-groups and, if appropriate, to go on to use this as a basis for discussion and negotiation.

The weighting stage of the analysis is necessarily subjective in that there is no value-free or absolute statement of the relative significance of impacts as diverse as those captured in the MCA4climate criteria tree. It is to be expected that different stakeholder groups would prioritize outcomes differently and one of the strengths of MCDA is to enable the exploration of the consequences of those differences. However, it should be recognized that all aspects of decision making and associated methods for support are inherently subjective not only in the importance afforded to different criteria, but also in the selection of what factors should be taken into account and in the measurement of impacts. Even in cases where these are measurable, the value added is not necessarily linearly related to the impact measure. If properly supported, this should be seen as a strength rather than a weakness of MCDA approaches, as discussed below.

The final step in the MCDA process is to explore the performances of policy options at different levels of the value tree through progressive aggregation of the scores and weights, culminating in a comparison of aggregate scores for each of the two main branches of the criteria tree (Fig. 2). A simple plot of each option’s aggregate score for outputs versus its aggregate score for inputs enables the visual identification of those options which provide the highest output at a given level of input, the so called efficient options (it should be noted that the ratio of these values is meaningless, as the level of measurement in use is an interval scale). The aggregation model underlying MAVA is a simple weighted sum as potential interactions between criteria are considered in the building of the value tree and nonlinearities in preference are taken into account in scoring the options. The consideration of aggregate values should be accompanied by sensitivity analyses to gain an understanding of the extent to which the results depend on the specified weights and scores. In particular, such analyses can also be used to explore different stakeholder perspectives.

In many cases, the outcomes are relatively insensitive to changes in the scoring and weighting, which gives confidence that the priorities that have been established are robust. In some cases the outcomes may challenge the intuitive expectations of participants in the process, a situation that might surface factors relevant to the decision which have not been included in the analysis or which may cause some participants to review their priorities. If this happens, the model can be used to reconcile the differences.

The process of comparing options can also be used to encourage creative thinking, potentially leading to the identification of new or slightly modified options, which yield greater benefits in relation to costs. For example, it may be possible to address a weakness of an otherwise strong alternative without significant increase in costs. Or it may also be possible to define a climate policy option, which offers many of the benefits of the strongest option, but at lower cost by reducing the benefits, and thus the cost, on criteria that do not carry much weight. Reducing the cost in this way may more than compensate for the loss of benefit, giving an option that is quite beneficial without being too costly. If new options are generated in this way, they should be added to the set of options and evaluated along with the others in a second run. Several iterations of the scoring and weighting procedures may be necessary to arrive at a final decision. The overall process is one that should both support and challenge thinking, with a view to arriving at a decision that is well founded, transparent and justifiable.

4.2 Methodological challenges to the standard MCDA approach

It is recognized that, as with any analytic approach, there are challenges and limitations that need to be understood when using MCDA. Four key challenges are outlined below: subjectivity, dynamics across time, uncertainty, and extreme events.

MCDA has been criticized because of the subjectivity of the inputs. This criticism applies not only to the scores and criteria weights but also the structure of the criteria tree. However, its proponents recognize that there is subjectivity inherent in all decision making. They claim that a particular strength of MCDA is that it provides a framework in which the nature and degree of that subjectivity is made explicit, in order that it can be appropriately and transparently managed, seeking to minimize it where appropriate and explore its impact when relevant to do so. The effort to accommodate subjectivity should not be taken to imply a lack of rigor. On the contrary, it underpins a sound methodology that provides meaningful and reliable outputs. A rigorous approach to the management of subjectivity will seek to adopt a five-pronged approach. First, an experienced facilitator leads the process, who supports and challenges those responsible for providing inputs and recommending decisions. Second, the processes of eliciting inputs to an analysis are well founded and well documented, seeking to incorporate the knowledge and views of relevant stakeholders and appropriate experts with regard to both content (e.g. health impacts) and local context. Third, all elements of an analysis are explainable and justifiable, with reference to objectively measured impacts where appropriate. Fourth, the consequences of differing views of stakeholder groups can be explored through sensitivity analysis, with a view to finding options that perform well from the perspective of all groups, facilitating compromise if it proves difficult to reach consensus. Finally, fifth, the whole process is subject to a broad critique from a diversity of perspectives. Moreover, in addition to providing an effective framework for the management of subjectivity, effective facilitation by a skilled multi-criteria practitioner should seek to guard against recognized potential weighting biases (Hämäläinen and Alaja 2008; Pöyhönen and Hämäläinen 1998).

MCDA methods have not been explicitly developed to model impacts over time and most applications take static rather than inter-temporal or dynamic perspectives of the evaluation of policy alternatives. However, in assessing climate policy alternatives, addressing the time dimension is important because of the relevance of the temporal distribution of intergenerational impacts and the long-term nature of climate change processes. Although it can be argued that one way of dealing with time is via discount rates, there is strong debate in the literature on what rate to use (Weitzman 1998; Ackerman et al. 2009) to discount future impact and furthermore there could be ethical concerns with discounting some of the impacts such as the health impacts. It is possible to incorporate temporal consideration in a multi-criteria analysis in a number of ways, either by the specification of criteria relevant to different time horizons (which could then be weighted to reflect the relative priorities accorded to these) or by the definition of individual criteria that evaluate the performance of options over a specified time horizon.

Intertwined with the time issue is uncertainty. On the one hand, there is deep uncertainty about what the future will bring, but also, the further in the future the impacts of the climate policy alternatives are compared, the greater the uncertainty about the extent of those impacts in absolute and relative terms. Although there are many ways of dealing with uncertainty in MCDA (Durbach and Stewart 2012; Stewart et al. 2013), the tangled relationship between the time horizon of assessment and uncertainty presents major challenges, whatever methodology is used. Furthermore, the mix of types and levels of uncertainties in the different criteria under consideration (fiscal, economic, environmental, social, and health) also makes a uniform treatment of uncertainty across all criteria challenging.

Fourthly, there is the issue of handling extreme and catastrophic events in the evaluation of climate policies. Historically, multi-criteria approaches that explicitly model uncertainty are based on the classical Von Neumann-Morgentsern axioms of decision-making under uncertainty and were designed to deal with steady-state risk scenarios and not with extreme or catastrophic risks. By their nature, these axioms are insensitive to dealing with low probability high impact events. New axioms of decision-making are required to handle extreme and catastrophic risks under uncertainty (Chichilnisky 2000, 2009). These axioms lead to decision criteria, which are themselves multi-criteria in nature and therefore well aligned with MCDA.

The MCA4climate conceptual framework explicitly acknowledges these challenges that are incorporated into its best practice climate evaluation standards as described in Table 2. However, there is considerable scope for future research in these areas, particularly in terms of improving MCDA techniques to account for and respond to advances in the energy-environment-economy literature in modeling dynamics, uncertainty and extreme events.

Further to the above challenges, which relate specifically to considerations relevant to the formulation of climate policies, we feel that it is important to acknowledge the concern that the use of different MCDA methods, with different underpinning principles, could lead to alternative rankings of policy alternatives (Bell et al. 2001). If used naively, without proper appreciation of the underlying preference model and interpretation of its parameters, then different models may appear to generate conflicting recommendations. If used in an informed and thoughtful manner, the outcomes of different models can potentially enrich understanding but the resource and cognitive burdens of doing so would be excessive for any complex issue. This paper and the rest of this special issue aim to ensure that the MCA4climate approach and its accompanying MCDA methods are methodologically sound.

4.3 Application considerations

It is important to note that the analytical framework being put forward in this paper has not yet been applied widely. For this reason, its impact on actual policy change (to advance pro-development climate policy) remains to be proven. Notwithstanding, three thought experiments have been conducted, which allowed us to draw lessons on the potential value and practicality of the framework. These related to three expert group exercises and scenarios that were developed in order to test the value of the methodological framework as a practical aid to decision-making (UNEP 2011). They included: infrastructure resilience and climate change adaptation in Mumbai, India; water resource management and climate adaptation in the Sana’a Basin, Yemen; and climate mitigation and change in the fuel mix for the electricity sector in South Africa. These exercises drew on existing large-scale studies and incorporated face-to face and virtual workshops involving key stakeholders. Each looked at different types of policy options: single policies in Mumbai, broadly defined energy scenarios in South Africa, and single policies that were combined to form portfolios of options in the Sana’a Basin in Yemen. The latter case is discussed in more detail in the Miller and Belton paper, which is also part of this special issue.

Furthermore, the MCA4climate framework has been recently used, albeit rather schematically, in two developing countries, Mexico and Peru, for aiding decision making in the area of development-compatible climate policy planning. In addition, the approach also inspired the preparation of a comprehensive review of national methodologies exploring baseline scenario calculations in ten major developing country economies (Puig et al. 2013). The Mexican Ministry of Environment applied the framework in its process of prioritizing the country’s policy for adaptation to climate change in the area of irrigation agriculture. To this end, the generic criteria for the agriculture sector were adapted to the Mexican reality and indicators were developed for each criterion. Methods of assessment for the various indicators were developed, distinguishing between the different irrigation districts (crops, climates and climate risks vary markedly from one district to another), types of policy measures (e.g. some focused on building infrastructure, whereas others sought to train farmers, thus requiring different approaches) and types of crops. While the adaptation process retained the main elements of the generic MCA4climate criteria tree, the criteria were changed substantially, to tailor them to the realities of the country. Two sets of stakeholder groups were convened to adapt the generic MCA4climate framework: a policy stakeholders group who identified broad priorities for these areas, and a technical stakeholders group who took further and refined the assessment criteria, whereas an inter-ministerial committee assigned weights. The model was then calibrated by using the scores obtained for a range of pre-existing policy measures that had been developed through an extensive consultative process, with a view to eliminate potential biases. At the time of writing, this work is meant to inform the forthcoming adaptation strategy for the country and the revision of the Mexican special plan for climate change.

In Peru, the MCA4climate approach was experimented in the case of prioritizing climate change adaptation investment projects targeting agriculture. Following discussions at national level, which incorporated agreement on customization of the criteria tree to the specific context, the framework was used (via stakeholder workshops) to support representatives from the Piura region in prioritizing planned agricultural adaptation projects (Rivera et al. 2013). The results produced contributed towards the development and implementation of Peru’s national plan for risk management and adaptation to climate change in the agricultural sector (PLANGRACC) for the period 2012–2021.

Even though these thought experiments and applications are far from representing fully-fledged case studies, several lessons were nonetheless drawn with regard to the potential applicability of the MCA4climate framework and its role in aiding decision-making. First, there may be interactions among different types of mitigation and adaptation options that have important implications for how well the individual options will work in practice. Consequently, policy assessments should consider alternative portfolios of options as well as efficient sequencing of options within a portfolio. Second, it is preferable to identify and agree on criteria and carry out initial scoring of policy options before undertaking detailed analytical modelling in a climate-policy planning process to ensure that the outputs of the modelling are useful in applying the MCA4climate framework or any other multi-criteria decision analysis tool. Third, the methodology could provide a powerful means of enabling a wide range of stakeholders to engage in the complex climate-policy decision-making process. However, for this to occur, there is the need for a supporting environment that would enable and foster stakeholder participation and democratic forms of decision-making. Fourth, the weighting of different criteria can have a profound effect on the relative value of different policy options. This emphasizes the importance of good facilitation to ensure that values are elicited in a sound manner. It also stresses the importance of ensuring that all stakeholder views are represented and appropriate weights are given to different criteria, including those concerned with social and economic development.

5 Conclusions

Any meaningful evaluation and effective implementation of climate policy options would need to account for the multiple values, interests, trade-offs and synergies between climate policy goals, their likely consequences, and the development objectives of the country wishing to put into practice its climate policy plans. Moreover, due to the complexity and multidimensionality of the climate change problem, decision-making in this area would have to involve a range of relevant stakeholders. There is also an urgent need to take stock and reflect on the type of information being used in climate policy evaluation and planning, how the respective knowledge has been generated, what the main assumptions underpinning it are, and how funders and users of climate policy analyses could ensure pluralism in methods and approaches. Identifying and discussing pro-development climate policy options grounded in space and time and which have the potential of being robust across a range of plausible future outcomes, may deliver better suited responses, rather than searching for unique optimal or first-best solutions.

Further efforts are required in order to gather the diverse information available within the research community, so that a range of scientific approaches (particularly involving the economics discipline) is represented. Most importantly, innovative and guiding conceptual frameworks are demanded to foster the mobilization and uptake of interdisciplinary knowledge and catalyse action on the climate policy front. It is hoped that the MCA4climate approach proposed in this paper takes a step forward in this direction and helps catalyse new thinking, debate, and practice in the area. However, an important caveat remains. Any inclusive and effective climate action would require a certain level of organization and co-ordination amongst the private sector, civil society, local governments and other stakeholders at the national or regional level. Where the decision-making process is highly hierarchical or where climate-policy action is the domain of a particular executive power, multiple stakeholder participation may not be possible. Nonetheless, the results of applying the MCA4climate framework can still be of value, especially where they are made available for public consultation.