Keywords

Introduction

The energy and climate policy framework of the European Union (EU) consists of a series of regulations and initiatives that aim at different objectives and affect various actors in the energy and climate field. These policies aim to achieve specific objectives set by the United Nations Framework on Climate Change Convention, which assigns greenhouse gas (GHG) emissions reduction targets for all member states. In December 2008, EU leaders reached agreement over an energy and climate change “package” to deliver the bloc’s ambitious objectives of slashing greenhouse gas emissions by 20%, boosting renewable energies by 20%, and increasing energy efficiency to 20% of the primary energy consumption by 2020. The package has multiple objectives and is designed to increase the EU’s share to combat climate change, reduce the Union’s dependency on imported fuels, promote green technologies, and create new jobs.

Policy instruments addressing such targets are present at EU-wide level and on a national basis. As far as the latter case is concerned, many instruments are currently incorporated into regulations, economic instruments, voluntary agreements, and market-based mechanisms. In the EU-wide context, a unified emissions trading scheme (EU ETS) was established as from 2005, based on an EU Emissions Trading Directive (CEC 2003b), followed up by an additional Directive (CEC 2004) that enables direct links of the EU ETS with the Kyoto Protocol project mechanisms (namely Joint Implementation and Clean Development Mechanism). The climate and energy package lays down certain conditions and requirements for further improvement and amendment of EU ETS specifically for its third phase, which starts in 2013. In addition, EU policy focuses also on the promotion of renewable energy sources by adopting various directives, such as the directive on the promotion of electricity produced from renewable energy sources (CEC 2001), the biofuels directive (CEC 2003a) and the recently agreed energy and climate package, which includes new targets for renewable energy sources for Member States.

Numerous policy instruments are applied simultaneously at an EU, national and regional level, aiming at often contradictory energy, environmental, and economic targets. Given this complex policy environment, it is clear that various objectives are pursued in terms of environmental and energy effectiveness, alongside economic efficiency. As these policies are designed and implemented in an already policy-crowded environment, interactions of their measures are taking place. These interactions can take different forms and shapes and in general can be complementary, overlapping, or indifferent. This raises the issue of compatibility of the different policy schemes, which is of crucial importance for further policy design. In this sense, policy interactions can affect the result of the overall targets of climate policy either in a positive or negative way. In addition, policy interactions could be beneficial towards certain policy objectives, but on the other hand they might affect negatively other objectives, which consequently would undermine the effectiveness of the overall policy. Thus, during the ex ante assessment of policy interactions, a systematic way to highlight and analyse trade-offs and synergies between policy objectives is indispensable. The most common practice in climate policy assessment is the use of quantified tools, models, and neoclassical economic approaches to measure the extent of climate mitigation and economic efficiency simultaneously. Therefore, the majority of researchers and practitioners in climate policy evaluation use approaches such as cost benefit analysis (CBA) and cost effectiveness analysis (CEA), which normally can capture the economic and environmental (in terms of greenhouse gas emissions reduction) dimensions of climate policy. In order to complement these approaches and consider other aspects of climate policy, specific studies are being conducted separately, targeting other dimensions and policy objectives such as competitiveness, employment, energy security of supply, and technological innovation. There is a lack of a unified method that aims to capture the different climate policy objectives in a systematic way and thus reconcile environmental, economic, sociopolitical, and technological aspects.

In order to reconcile the various aspects of climate and energy policy into the evaluation of policy instruments interactions, a Multi Criteria Analysis (MCA) approach is deemed appropriate for the following reasons:

  • Multiple instruments and thus multiple combinations of instruments (policy options) for evaluation can be considered and evaluated by MCA

  • Climate and energy policy have various aspects and objectives that should all be considered while evaluating policy instruments, where MCA is capable to deal with multiple, often conflicting, criteria and objectives

  • Climate policy interaction is a high complex issue, whereas MCA has the ability to deal with complex policy issues by decomposing, analysing, and structuring them in a transparent way

  • MCA can consider and combine objective (facts or likely performances) and subjective type of information (expression of judgments and preferences)

  • MCA can incorporate different stakeholders’ perspectives and preferences by the application of a weight elicitation technique

  • MCA is an aid to decision-making that assists stakeholders to organize the available information, think of the consequences, explore their own objectives and tolerances, and thus provide a widely acceptable policy decision

However, despite the recent interest in participatory and MCA methods, MCA assessments are absent from most of the actual climate policy evaluations for various reasons. Time constraints, data availability problems, lack of guidelines and general tradition in monetized and cost benefit analysis methods, misconceptions, and a large variety of MCA methods comprise some of the main reasons that MCA methods are neglected most of the time in climate policy evaluation (Borges and Villavicencio 2004).

MCA methods should be used as decision aid tools rather than techniques for making decisions. Their outcomes are the result of stakeholders’ evaluations and thus are sensitive to their judgments. Therefore, stakeholders should be informed about the tools they use and comprehend their functions and outcome.

MCA, although appropriate for the evaluation of policy interactions, should have a properly modelled preference system in order to facilitate the decision-making process. In this respect, special attention is paid to distinct stakeholders who tend to weight differently the employed criteria according to their policy objectives and preferences. Therefore, capturing this essential information could be of significant use, especially if it will appropriately feed into the decision-making process.

To this extent, we have developed an integrated assessment tool to evaluate energy and climate policy interactions during the policy design phase, which is able to assess combined policies using multiple criteria and parameters. This decision support tool is qualitative and in an interactive way provides a useful insight into several aspects of policy interactions. It addresses policymakers, policy analysts, and stakeholders, who can use it in order to identify policy interactions and effects of various policies.

Considering the above and following this introduction, we describe in Sect.2 the methodology employed in the tool alongside with its basic characteristics and the parts that focus mainly on the selection of evaluation criteria and the weighting factors determination. In Sect.3, we present an illustrative example of the tool in order to demonstrate its actual function, whereas Sect.4 is dedicated to the presentation and analysis of results obtained from the illustrative case study. Finally, conclusions are drawn and future research areas are identified in Sect.5.

Methodology

The developed multi-criteria decision support tool, the Energy and Climate Policy Interactions (ECPI), provides a qualitative framework for analysing interactions among policy instruments in various policy mixes during the phase of policy design. The key concept is that policymakers and stakeholders are able to examine selected policy instruments for interaction and express their preferences towards certain criteria when assessing options of integrating various instruments. In the ECPI tool, a traditional policy condition is assumed that an optimal policy solution preconditions the relationship one policy instrument for one policy target (Tinbergen’s rule).

ECPI consists of certain features and steps that are described in detail by Oikonomou et al. (2010). In this paper, we focus on the preference system modelling and more specifically on the elicitation of criteria weights from various stakeholders and the investigation of potential trends according to their specific preferences and objectives.

Design Characteristics and Areas of Policy Interaction

Design characteristics refer to parameters that describe several functions of a policy instrument in terms of a measure identification, objectives pursued, scope, market creation, financing, timing, and institutional setup. A detailed explanation of these characteristics is provided in Oikonomou and Jepma (2008). The most important characteristics taken into consideration at this stage are briefly explained in Table38.1.

Table38.1 Design characteristics of policy instruments (Adapted from Oikonomou and Jepma 2008)
Full size table

EUA stands for Emission Unit Allowance (under the EU emissions trading scheme), WhC for White Certificates, TGC for Tradable Green Certificates and CHP for Co-Heat and Power

Design characteristics of standalone policies are combined and provide options for the formation of unified policy instruments with areas of design interaction. In a combined option of policy instruments A and B, a design characteristic X is compared in pairs and an area of policy interaction is extracted.

Design characteristics and areas of policy interaction are practically the same, but we distinguish them in the tool since they belong to different processes. Design characteristics refer to parameters of individual policy instruments, while areas of policy interaction to shared characteristics of combined policy instruments. In the options of combined policy instruments, based on our selection of design characteristics and on formulation of areas of policy interaction, we classify areas of policy interaction as complementary, overlapping, or indifferent. This principle of redundancy of design characteristics is in accordance with our core assumption of Tinbergen’s rule as stated above. Complementary means that a design characteristic of policy A enforces the same characteristic of policy B. Overlapping means that a design characteristic of policy A reduces the value of the same characteristic of policy B. Indifferent means that a design characteristic of A and B do not meet or reinforce each other.

Climate and Energy Policy Objectives and Criteria

Policy and decision-makers implement policies and measures to achieve specific objectives, taking into account different aspects, that they believe will not be achieved in the absence of government intervention, possibly because of the existence of non-internalized externalities and/or public goods supplies. There are various aspects deriving from climate and energy policies that policymakers aim to take into account. The evaluation of climate and energy policies first defines evaluation criteria and second categorizes them into main policy aspect categories. The evaluation criteria are used to measure the extent of the fulfilment of the policy aspects and objectives taken into account. Evaluation criteria are indispensable for both the choice of instruments during the policy design phase and the ex post assessment of implementation of policy instruments. The main EU climate and energy policy objectives which the EU climate and energy package aim to achieve are the following:

  • To combat climate change and reduce GHG emissions

  • To secure energy of supply and diversify the energy fuels

  • To reduce the energy consumption by increasing energy efficiency within the economy

  • To boost technological innovation and competitiveness

  • To create new jobs

In this context, different studies have also identified criteria for the evaluation of climate and energy policy instruments (IPCC 2001; 2007; OECD 1997; 2001; Bondansky 2003; Oikonomou and Jepma 2008; Gaiza-Carmenates et al. 2010) addressing the different dimensions of climate and energy policy evaluation. Following a bottom-up process of selection of criteria and based on a review of these studies, we have selected the most relevant criteria and clustered them in the following five main categories, trying to capture all possible aspects of climate and energy policy interaction evaluation:

1. Environmental category

Environmental effectiveness has been widely emphasized in the environmental and climate change literature as the main criterion able to capture the extent that a policy instrument achieves the environmental goal, such as a GHG emissions reduction target (IPCC 2001; 2007; Bondansky 2003; Oikonomou and Jepma 2008). How reliable is the instrument in achieving that objective? In addition, does the instrument create continual incentives to improve products or processes in ways that reduce GHG emissions? Furthermore, OECD (1997) and Bondansky (2003) identify “soft” effects, which relate to the impact of environmental policy instruments on changes in attitudes and awareness. Thus “environmental awareness” is another environmental criterion which complements the criterion of “reduction of GHG emissions” in environmental category.

2. Sociopolitical category

Considering sociopolitical aspects is often an important issue of climate and energy policies. Blyth and Lefevre (2004) carried out a quantitative study on the interactions between energy security and climate policies, highlighting the significance of “security of energy supply” as an evaluation criterion. Decoupling economic growth and energy use is one of the main EU objectives and thus “reduction of energy intensity” has been added as a criterion in this category.

3. Financial category

The second assessment report (IPCC 2001) identifies cost effectiveness as one of the main criteria for the evaluation of climate policies. Does the policy instrument achieve the environmental objective (e.g. reduction of GHG emissions) at the lowest cost, taking transaction, information, and enforcement costs into account? “Administration” and “compliance” costs have been defined as separate evaluation criteria of climate and energy policy interactions by Oikonomou and Jepma (2008) additional to “transaction” costs. OECD (1997) identifies “governmental revenues” raised in the case of market mechanisms, for instance, may constitute a second source of benefits from their use, over and above their direct environmental impact, depending on if and how the revenues are recycled.

4. Macroeconomic category

Administrative and political feasibility includes considerations such as flexibility in the face of new knowledge, understandability to the general public, impacts on the “competitiveness” of different industries, and other government objectives. “Wider” economic effects include potential effects on variables such as inflation, competitiveness, “employment”, trade, and growth (OECD 1997). One of the priorities of EC energy policy is the enhancement of energy market liberalization (e.g. Directive 2003b) which can be captured by the “market competition” criterion (Oikonomou and Jepma 2008).

5. Technological category

OECD (1997) identifies dynamic effects, which relate to the impact on learning, innovation, technical progress, and dissemination and transfer of technology. Stimulating technological change is stressed also by Bondansky (2003) as one of the main criteria for evaluating climate policies. In the long run, the development and widespread adoption of new technologies can greatly ameliorate what, in the short run, sometimes appear to be overwhelming conflicts between economic wellbeing and environmental quality. Therefore, the effect of public policies on the development and spread of new technologies may be among the most important determinants of success or failure in climate policy.

Criteria must fulfil some qualitative attributes as described by Hajkowicz et al. (2000), and Belton and Stewart (2002), while a few more have been added by Grafakos et al. (2010):

  • Value relevance – Linking the concept of each criterion to the objectives it is meant to represent

  • Operationality – Evaluation criteria should be able to identify how well each option of policy interaction meets the objectives expressed by the criteria

  • Reliability – A malfunctioning criterion should not render the whole set of criteria unworkable

  • Measurability – Degree of measurement of the performance of alternatives against specified criteria

  • Decomposability – Possibility to break down an objective into specific means

  • Non-redundancy – Limiting the number of criteria addressing the same objective, meaning avoidance of duplication of information in criteria

  • Minimum size – The number of criteria employed should be only the absolute necessary to provide representation of policy objectives

  • Preferential independence – Preferences associated with the performances of each option should be independent of each other from one criterion to the next

  • Completeness – The selected criteria should cover all the key elements of the evaluation problem

  • Understandability – The selected criteria should be understandable not only by specialists but by non-technical people too

The selection of evaluation criteria as described is based on a bottom-up approach. By reviewing the relevant literature and assuring that the selected set of criteria meets the above conditions the criteria are categorized according to their association with the climate and energy policy aspects discussed above. At the final stage, stakeholders and experts were asked to approve and refine the set of criteria. Figure38.1 illustrates the main climate and energy policy aspects and criteria categories, whereas Table38.2 provides a brief explanation of each selected criterion employed within the tool.

Fig.38.1
figure 1

Major criteria categories and selected criteria

Full size image
Table38.2 Explanation of selected criteria
Full size table

It is unavoidable that some overlaps between the criteria might exist within a category and between the different categories of criteria. They are not necessarily consistent, but give room for synergies and conflicts. This leads us to the necessity not to see the criteria or groups as separate formulas, but as parts of the overall aim, and to incorporate them whenever possible into the integrated climate and energy policy concept. Coming from the overall aim to the criteria is one step of operationalizing, whereas the next would be to measure the criteria through quantitative – if possible – otherwise qualitative measurement scales.

Assessment of Policy Mixes (Scoring of Policy Options)

The criteria selected in the ECPI tool receive specific values that range from −2 to +2 and reflect the positive or negative effect of each policy instrument on the specific criterion. A zero value reflects that there is no influence on the criterion, which could also illustrate that a policy instrument is not related to targets that the specific criterion represents. As the numbers −2 to 2 are taken as mathematical numbers and as ordinal ones (having a meaning), the distance between the numbers must be the same as between the associated answers (see Table38.3). This overcomes the problem of using an ordinal scale (answer possibilities) for a weighted sum aggregation.

Table38.3 Measurement scale of criteria performance
Full size table

A positive or negative effect does not always mean an increase of a positive or negative value of a criterion. The effect of the value is in accordance with the interpretation of criteria as explained in Table38.2.

The tool provides the user with performance values of policy instruments towards the evaluation criteria, as they are assessed from various literature studies and experts’ judgements. The performance values for the option of integrating the policy instruments result from the design areas of integrated policy instruments and the degree of influence of areas of policy interaction on the criteria. The measurement scale is the same as of the scale of standalone policy instruments (−2/+2). We should stress here that these performance values cannot give more than a preliminary idea about the direction and the probable range of the impact on all criteria.

Weighting of Criteria

Each policymaker and stakeholder may apply different weights to the evaluation criteria according to policy objectives and preferences while evaluating climate and energy policy options. There are numerous methods to determine criteria weights which can be used in various ways for different policy evaluation purposes according to different interpretations of weights (Grafakos et al. 2008). Weights can have different meanings, they can either be perceived as relative importance coefficients stating importance of the criteria, or as scaling factors reflecting impact trade-offs between criteria. The weighting method that has been developed to derive factors of relative importance of criteria is a combination of pair-wise comparisons with an initial ranking technique.

Ranking of Criteria

The methodology combines an initial simple ranking criteria exercise and a pair-wise comparison technique which results in criteria weights determination and a new criteria ranking. The former is a direct ranking whereas the latter one is indirect, determined by the weights derived by the pair-wise comparisons of criteria. The introduction of the initial holistic ranking technique has a twofold meaning and use. It is introduced first to help stakeholders to comprehend the concept of criteria importance and second to provide the means to respondents to resolve any conflicts and discrepancies that may be detected between the two rankings.

Pair-Wise Comparisons of Criteria

Respondents’ weighting judgments regarding the criteria are derived by comparing the criteria in pairs in a structured and constructive manner. We use the abbreviated pair-wise comparison format and thus n−1 pair wise comparisons are performed. Pairs are sequentially assigned (as a-b, b-c, c-d, etc.), where the initial criterion a is the first ranked criterion by the respondent, criterion b is the second ranked criterion, c is the third ranked criterion and sequentially the order of pairs of criteria is according to the initial criteria ranking. This means that first, randomness is assured in the sense that each subsequent pair is selected differently according to respondents’ initial ranking and thus problems with path dependency are being minimized (Saaty 1987) and second, the ranking consistency of stakeholders’ preferences is being maximized.

The criteria weights are derived in a constructive way after completing certain judgmental steps. Firstly, respondents’ preferences between pairs of criteria are expressed verbally. Secondly, the verbal expression of preferences is being assigned with ratios on a 10-point scale between 0 and 1. Thirdly, one criterion is assumed to have relative score of 1, to be used for the calculation of the relative scores of the rest of the criteria. Then, the obtained relative scores of criteria are translated into normalized weighting factors Wi by the following formula:

$$ Wi = \frac{{RSi}}{{\sum\limits_{n - 1} {RS} }} $$
(38.1)

where RSi is the relative score of criterion i compared to criterion j during the pair-wise comparison and Σ(RS) is the sum of relative scores of all criteria (n) after completing the whole set of abbreviated pair wise comparisons (n−1).

It is possible to assign weights both to the criteria (third level), and to the categories (second level), both indicating relative importance. This can be done in different ways. Firstly, we can elicit weights for the criteria one by one (whose sum is 1) and then add the criteria weights according to the category they belong to derive categories’ weights. Secondly, we can assign weighting factors on criteria categories (whose sum is 1) and then further divide each category’s weight across category’s criteria. In both cases, we run the risk of being subject to splitting bias, which leads to the problem that within categories with many criteria, one criterion has less importance than within categories with few criteria or inversely, categories with more criteria have more importance than categories with less criteria (Weber et al. 1988; Pöyhönen et al. 2001). We are aware of this bias and we avoided assigning weights to categories since the focus of the paper is on the particular selected criteria. However, in order to get an idea of interpretable results between the categories, and being aware of the risk of splitting bias, we can just add the weights on the third level in order to derive some indicative weight values for the second, the category level. It should be stressed that the development and application of this weighting methodology proved to overcome the major weighting difficulties, biases, and risks that normally arise during the applications of the weighting process, namely impact range effect, splitting bias, inconsistency, and numerical evaluation scale (Hayashi 2000; Hamalainen and Alaja 2008; Grafakos et al. 2010).

Transitivity and Consistency Test

A complete ranking of criteria is based on the actual choices assuming transitive preferences. For further discussion on transitivity of preferences in pair wise comparisons see Keeney (1982); Peterson and Brown (1998); Strager and Rosenberger (2006); and Grafakos et al. (2008). Despite the assumption of transitivity, a ranking consistency index is introduced, based on Spearman’s Rank Order Correlation Coefficient (SROCC), to explore the degree of consistency between the initial ranking and the ranking based on pair wise comparisons. The formula of the Spearman rank order correlation coefficient (ρ) is:

$$ \rho = \frac{{6*(\sum {{D^2})} }}{{N({N^2} - 1)}} $$
(38.2)

where 6 is a constant that is always used in the formula. D refers to the difference between a criterion’s ranks on the two methods (simple ranking and pair wise) and N is the number of criteria.

Weighted Summation

The main focus of the current study is rather on the underlying stakeholders’ views and preferences towards the different aspects of climate and energy policy instruments than on the final overall ranking of alternatives, and thus a weighted summation is based on the simple, straightforward and transparent aggregation additive rule,

$$ V(p) = \sum\limits_j {{w_j} * } {v_j}\, {p} $$
(38.3)

the value of the overall effect of each policy option, v j , to each criterion is multiplied with its respective criterion weight, w j , whereas the summation of these products determines the overall value of each policy option, in our case of each policy instrument and their combination V(p). This overall value indicates whether two policy instruments should be integrated or not.

Application

Description

An illustrative application of the methodology is presented in this section by comparing the option of implementing two standalone policy instruments to the option of their combined application. The policy options are evaluated by the weighted summation of each option based on their scores and criteria weights that have been assigned by the stakeholders. We compare the option of applying feed-in tariffs for renewable energy (feed-in RE) to energy suppliers in combination with the application of EU ETS to energy producers for CO2 emissions reduction with the option to keep them as standalone policy instruments. The main characteristics of the policy instruments and policy options are illustrated in Table38.4 as presented to stakeholders.

Table38.4 Areas of policy interaction
Full size table

As was described at the methodology section, the policy instruments are compared to their combined application based on the selected evaluation criteria. The performance values (scores) of the policy options have been determined from literature studies and experts’ judgments. The measurement scale of the performance values is common for all criteria and ranges from −2 to 2 as discussed above. Table38.5 depicts the evaluation impact matrix which contains the scores of policy options towards the evaluation criteria.

Table38.5 Evaluation impact matrix
Full size table

It can be noticed from Table38.5 that none of the policy options is superior to others with respect to all evaluation criteria. EU ETS and the interaction policy options have negative scores at the criteria of “administration costs” and “transaction costs”. On the other hand, these two options achieve the best performance (score 2) at the criterion of “business opportunities”. Feed-in tariff for RE policy option also has a negative score at the “governmental revenues” criterion whereas it performs best (score 2) at the criterion of “security of supply”. Therefore, the weighting factors that stakeholders assign to criteria would determine the most desirable policy option with the highest score.

The tool was distributed to various stakeholders to elicit their preferences on criteria weights. The tool includes specific instructions to assist the stakeholders to use it in an easy way and minimize the cognitive burden to users and time to be spent by them. The sample was small and the response rate judged as moderate (50%). The tool was sent to 38 stakeholders, while 19 of them responded. The sample was divided into two main categories: academics (9) and market players (10) (e.g. energy and climate experts, consultants) in the climate and energy policy field. The completion of the tool was performed individually in an interactive way in the sense that each respondent could see and revise the output of his preferences.

Assigning Criteria Weights

First Step: Initial Ranking

The respondent is required to rank criteria according to his preferences, from the most preferred to the least preferred criterion. The initial ranking is also used for a consistency test of the user’s preferences by being compared to the ranking determined by the pair-wise comparisons of criteria.

Second Step: Pair-Wise Comparisons

The respondent is required to express his preferences in three consecutive step-requests: (a) which criterion he prefers at each pair wise comparison, (b) how much he prefers a criterion to the other verbally, and (c) how much preference intensity he assigns arithmetically to the most preferred criterion against the other. Five levels of preferences have been defined in verbal expressions. The five levels of preferences verbally expressed are associated to 10 levels of numerical preference values (Table38.6).

Table38.6 Verbal and ratio numerical intensity of preferences
Full size table

The user is assisted by a developed computer aided Excel tool. A graph automatically reflects his preferences, providing him with the visual representation of the resulted relative importance between the pair of compared criteria. When the respondent completes the whole series of pair-wise comparisons across criteria, then relative scores, weighting factors, and ranking of criteria are determined automatically by the tool.

Final Step: Consistency Test and Revision of Preferences

During the final stage, the respondent can observe the derived weights and ranking of criteria, and revise his preferences if necessary. The obtained ranking of criteria during the pair-wise comparisons is compared with the initial ranking. The consistency indicator, which is calculated automatically by the tool, suggests whether or not the respondent needs to revise his preferences.

Results and Discussion

The analysis of results focuses mainly on how different stakeholders weight various objectives and criteria during the climate policy interactions evaluation. Figure38.2 illustrates the spread of criteria weights for confidence level 95%. It can be clearly noticed from Figs.38.2 and 38.3 that the criterion which has been assigned with the highest average weighting value is the “reduction of GHG emissions”. “Reduction of energy intensity” and “security of supply” follow as second and third most significant criteria respectively. The least significant criteria according to stakeholders are “governmental revenues”, “transaction costs”, and “administration costs”. This could be expected since there were no representatives from governmental institutions that returned the tool within the sample and thus their views are not represented in these results.

Fig.38.2
figure 2

Spread of criteria weights (95% confidence level)

Full size image
Fig.38.3
figure 3

Average and median criteria weights

Full size image

It can be observed from Fig.38.3 that there is significant deviation between average and median value of some criteria weights. In particular, the criteria of “increase of environmental awareness”, “security of supply”, and “innovation cycle” obtain the highest deviation between average and median, which means that few respondents assigned high weights and force the average values upwards. On the contrary, the criteria with less variation of the assigned weights by the stakeholders are those of “reduction of energy intensity”, “business opportunities” and “compliance costs”.

In case we would like to explore how stakeholders value and weight different criteria categories, we can simply add the criteria weights for each specific category (see Fig.38.4). However, we should be aware of the risk of splitting bias that exists, while there is a tendency to weight more the categories with more criteria than the categories with fewer criteria. Therefore, we should interpret this data with care and be cautious about the conclusions that can be drawn. Nevertheless, we can observe that the financial category is being weighted with the second lowest value factor even if it includes four criteria. In addition, environmental and sociopolitical categories have been weighted with high values and much higher than the technological category (which also includes two criteria).

Fig.38.4
figure 4

Criteria categories’ weights

Full size image

Figure38.5 shows the differences of criteria weights that have been assigned between different stakeholder groups. In our application we have distinguished two stakeholder groups: (1) academics and (2) market players (energy experts, consultants, etc.). It can be observed from Fig.38.5 that the group of market players perceives some criteria to be much more significant than the group of academics do.

Fig.38.5
figure 5

Criteria weights of different stakeholder groups

Full size image

In particular, “reduction of GHG emissions”, “reduction of energy intensity”, “compliance costs”, “competitiveness”, and “business opportunities and trade” assigned with much higher weights by market players than by academics. On the contrary, “market competition”, “employment”, and “innovation cycle” have been considered with more significance by the group of academics than by the group of market players. In order to have more robust results or to explore the views and preferences of other type of stakeholders, a bigger sample of respondents would be essential to be involved in the study.

It can be observed from Fig.38.6 that some average and median values of criteria weights had significant differences within the group of academics.

Fig.38.6
figure 6

Average and median weights of group of academics

Full size image

On the other hand, the differences between average and median weights were mainly insignificant within the group of market players, as shown in Fig.38.7.

Fig.38.7
figure 7

Average and median weights of group of market players

Full size image

This probably could be explained by the fact that market players represent more unified preferences than the group of academics. In particular, the median weight of the “reduction of GHG emissions” is estimated higher than the average weight, which means that few academics assign very low values to this criterion and consequently drive the average value downwards. Therefore, this may also explain the major difference between the average weighting values of the two groups for the criterion of “reduction of GHG emissions”.

Regarding the policy options’ final scores and ranking, the policy option of combining the policy instruments (interaction) performed best for all stakeholders irrespective their background and the particular group they belong (see Fig.38.8). This result coincides with many practices in EU countries, where EU ETS has been complemented by the feed-in tariff for renewables.

Fig.38.8
figure 8

Average weighted scores of policy options

Full size image

Figure38.9 demonstrates how different criteria contributed to the final score of the most desirable policy option, taking into account their weighted scores. It can be noticed that the criterion of “reduction of GHG emissions” has the highest contribution to the final average score of the interaction policy option. This figure can also illustrate the main synergies and conflicts between certain criteria concerning the particular examined policy option. For instance, we can clearly observe that this policy option performs high score simultaneously on specific criteria (synergies), such as “reduction of GHG emissions”, “security of supply”, “increase of environmental awareness”, and “business opportunities and trade”. On the contrary, this achievement is being realized at the expense of other criteria (e.g. administration and transaction costs), highlighting conflicts between criteria.

Fig.38.9
figure 9

Contribution of criteria to the final score of the interaction policy option

Full size image

The share of each criterion to the final score can be further analysed and categorized into individual components of criteria weights and scores and thus indicate the main policy drivers of the policy design process and the different perspectives affecting the policy outcome.

The development and application of this weighting methodology contributed also to the improvement of weighting methods for mapping stakeholders’ preferences in climate and energy policy evaluation:

  • It has the capability to consider a high number of criteria

  • It is a user-friendly weighting procedure (structured, simple, transparent), it does not require a lot of time and effort from stakeholders, and it therefore reduces the cognitive burden required by them

  • The weighing method has been applied by the use of an Excel tool which has been developed for this purpose and provides the appropriate automated modules

  • It provides the ability to respondents to interact with the results and revise their initial preferences, while a ranking consistency index gives them the opportunity to check the consistency of their rank order preferences

  • It can be used by many and different individuals simultaneously, either in the form of individual interview or by electronic communication

  • It can be used within groups to identify trends, preference differences, and conflicts, and raise discussion for the evaluation problem at hand

Conclusions

In a policy environment where EU ETS has been introduced since 2005 and renewable energy targets have been set for all EU member states as laid down within the recently agreed climate and energy package, the assessment of interactions between energy and climate policy instruments is essential. ECPI can serve these needs in a quite satisfactory way by considering different aspects and objectives within the analysis and, furthermore, with the ability to embed stakeholders’ preferences and weights towards certain policy objectives. Our analysis of ECPI characteristics and the testing applications are illustrative of the following aspects:

Integration of Different Aspects

ECPI includes specific parts to break down the issue of assessing policy instruments’ interactions into structural elements of the climate policy problem and then to integrate and synthesize them within one unified policy analysis framework. We have distinguished five main aspects as main criteria categories: (1) environmental, (2) sociopolitical, (3) financial, (4) macroeconomic, and (5) technological, which are taken into account and have been further broken down into 18 evaluation criteria. Apart from the integration of various aspects of climate policy, the tool incorporates stakeholders’ preferences as well. In addition, one of its greatest strengths is the ability to integrate normative judgements (e.g. stakeholders’ preferences) and technical expertise (e.g. experts’ judgements).

Transparency

Transparency of the impacts, the preferences, and the conflicts between the criteria is extremely important for every decision maker. The policymakers and stakeholders need insight into the nature of these parameters in order to make the decision. Transparency in the decision process is again important for the acceptance of the decision and the implemented climate and energy policy strategies by the affected people. As MCDA improves this transparency, it can improve the decision process and the design of climate and energy policies. The result of MCDA is usually a ranking or a set of rankings of policy options. This ranking is not unequivocal; it is dependent among others on the preference structure of the stakeholders involved. In our case study, MCDA does not provide only one exact ranking, but it provides the background for the ranking and the information about its formation.

Learning and Awareness Process for Stakeholders

The tool comprises certain interactive elements that keep the respondent aware about the specific characteristics and areas of policy instruments interactions, the likely impacts of interactions towards certain evaluation criteria, his or her own preferences, and how these preferences affect the final outcome.

Identification of Synergies and Conflicts

By the application of the tool, synergies and conflicts between criteria can be identified and therefore areas for further improvement can be highlighted. By categorizing the policy problem into structural elements, we can observe which elements and parts function as potential conflicts and thus try to improve them for optimizing the policy design.

The stakeholders who have tested ECPI and its weighting module have expressed positive opinions about its usefulness, especially with regard to its characteristic of identifying policy instrument interactions that should be further analysed and the improvement of the decision-making process transparency.

Some conclusions can be drawn also based on the application of the tool:

  • Based on the application of the tool, the criteria of “GHG emissions reduction”, “reduction of energy intensity”, “security of energy supply” performed as the most significant, whereas “transaction costs”, “governmental revenues”, and “administration costs” performed as less significant.

  • Different groups of stakeholders, namely academics and market players weight differently the evaluation criteria. Furthermore, market players’ preferences, regarding the criteria, proved to be more unified and less dispersed as was the case with the group of academics where significant variations observed between their responds.

  • The policy option of interaction of the examined policy instruments (EU ETS and feed in tariff) performed best for each one of the respondents.

ECPI tool and its weighting technique should not be considered as static. On the contrary, it is a dynamic instrument that is open to changes, improvements, adaptations. It has an evolutionary character, which lies in the concept of integrated climate and energy policy. Being aware of certain limitations of the current version of the tool, we can draw specific directions for further improvement.

  • In the current version of the tool the selected list of criteria is based on the analysis of climate and energy policy aspects by the research team, whereas stakeholders’ involvement is limited to the final refinement of the set of criteria. The possibility of including stakeholders more actively in the process of selection of criteria could also be explored, where criteria can be discussed, added, changed, or removed. This prospect could also minimize the risk of any personal or institutional bias that might arise during the predefined selection of the criteria.

  • This case study was limited in terms of the number of respondents contacted and who answered. More robust results could be derived by engaging a wider range of stakeholders and forming more groups of stakeholders, and then mapping their perceptions based on the elicitation of criteria weights.

  • Furthermore, the tool can be examined in a group decision-making context and serve stakeholders as a communication and mapping tool. Then, participants can shape and share information in order to reach a reciprocal understanding, highlight differences, identify potential conflicts, and strive towards building upon a communicative consensus. Thus, it could be used as a communication and dialogue tool which should improve the negotiation process through better understanding and more transparent dialogue, which consequently enhances the overall policy design.

Although we have received positive comments about ECPI applicability, it is in our plans to continue working towards its further improvement.