Introduction

The building sector has a large impact on the development of energy plans worldwide (Thomas et al. 2012; Blum et al. 2013), due to its increasing importance in economies (BPIE 2011) and the great potential of the energy retrofitting in improving energy efficiency (GhaffarianHoseini et al. 2013; Vine et al. 2014; Schlomann et al. 2014), which also turns in the reduction of greenhouse gas emissions (IPCC 2007). The importance of this sector is confirmed by the regulatory and technical standard frameworks that push for higher energy efficiency in this sector (European Union 2010) and especially in the residential segment; in Europe, in fact, this segment constitutes a major part of building stocks and accounts for 75 % of the total stock (Eurostat 2010).

In addition, the construction sector has raised concerns about the sustainability of economies because, on the one hand, it is responsible for the depletion of significant amounts of resources and the release of pollutants (Rosenow 2013; Ruá and Guadalajara 2014) and, on the other hand, involves social issues that confer to the building sector rising importance (Assefa et al. 2007). Consequently, new awarding schemes were proposed aimed at recognising the environmental performance of buildings: Ecolabel (http://susproc.jrc.ec.europa.eu/buildings/whatsnew.html), Bream (http://www.breeam.org) and, in general, their holistic qualities (Peri and Rizzo 2012).

The planning process of new master plans, both at the regional and country levels, increasingly reflects this modified context: at present, these plans take into account not only energy but also economic and environmental issues.

The energy master plan for the Sicilian Region, whose development began in 2004, was aimed not only at improving the energy efficiency of the entire region but also at increasing the environmental performances of the region’s productive and economic sectors. A suitable list of priorities for the proposed energy-saving initiatives was originally established with the use of two indicators: the cost of the saved energy and the cost of the saved CO2, which essentially refer to economic issues. These measures are in line with the mission of the plan, which is to optimally allocate the available economic resources of the region in order of promoting sustainable interventions by building owners.

While this energy master plan was under preparation, a new Italian regulation was issued, i.e., Italian law n. 192 and of (2005), as a mandatory part of the pertinent European directive (European Union 2002) that is aimed at improving building energy efficiency in the Member States. Earlier, in 2001, the European directive for the Strategic Environmental Assessment (SEA) of plans and programs (European Commission 2001) had assigned great relevance to the stakeholders and their role in the energy planning process. Nevertheless, in Italy, the stakeholder involvement in the energy planning processes was recognized only starting from 2006, with Italian law n. 152 and of (2006), rather than from the 21 July 2004, when the Member States supposedly had to acknowledge the cited European directive (SEA).

In addition, the importance of the stakeholders’ role in the energy planning processes has recently been confirmed by the Commission Directive 2012/27/EU for the energy efficiency (European Parliament and Council 2012), which, in 2013, led to the release of the Italian Decree Law n. 63 and of (2013). Figure 1 illustrates the continuous evolving regulatory framework to which the plan should have complied.

Fig. 1
figure 1

The changing trend of both the regulatory framework concerning the building energy efficiency and the role of stakeholders, matched with the development process of the regional energy master plan

Full size image

This sequence of laws releasing suggested the University of Palermo that was part of the technical/policy panel, to a posteriori verify the role of stakeholders in the selection of priorities. This operation has been possible thanks to the fact that their preferences were gathered during several consultation meetings, despite not directly utilized in the process of definition of the master plan. In fact, during the formulation of the Sicilian Region energy master plan, particularly during the in itinere and post operam phases (referring to the SEA), groups of stakeholders were called to express their opinions about the plan. Unfortunately, their role was underestimated, and their opinions on the selection of indicators were only partially taken into account by the technical/policy panel. More specifically, although they singled out a large set of indicators, only the economy-based indicators were selected to assess the master plan’s effects.

With the aim of realizing this ex-post verification, a new set of indicators singled out by stakeholders during the cited forums was considered, instead of the economic only ones. In the following sections, the verification process of the effects induced by the different sets of indicators on the allocation of the regional budget to the technical actions is reported. A hybrid application of the analytic hierarchy process (AHP) method was applied on purpose. Important criticisms of the old established priorities have emerged from this application, which further confirm the preeminent role played by the stakeholders in the development of energy and environmental plans.

In 2004, when the development of the Sicilian Region Energy and Environmental Master Plan (EEMP) began, the Sicilian Region’s total energy consumption, by macro sectors, was the following: 136 ktoe in agriculture and fishing, 2693 ktoe in industry, 1615 ktoe in household and 3125 ktoe in transportation. The incidence of the civil sector was 21 %, whereas the agriculture and fish sectors had only an incidence of 2 %. The incidence of transport sector was 41 %, while the industry sector had only an incidence of 36 %. These values are typical for the historical trend of the Sicilian economy.

It is worth noticing that the building sector, due to its relevant final energy consumption, is believed to be very promising for the implementation of appropriate regional energy-saving policies. In fact, despite the stagnant situation of the housing market, the building sector has the potentiality of attracting from 55 to 76 billion euro, according to a recent report of the Polytechnic of Milan (http://www.qualenergia.it) up to the year 2020.

Apart from this, acting in this sector is particularly challenging because comfort conditions (i.e. thermal, acoustic, visual and indoor air quality) must be guaranteed to building occupants and cannot be decreased below certain limits despite that, in the same time, there is a strong call for the energy saving in this sector. Thus, the main challenge consists of providing increasingly high building quality standards with lower energy consumption, which was among the goals of the EEMP.

Materials and methods

After a brief explanation of the manner in which the energy-saving measures considered in the Sicilian Region’s master plan were hierarchized at the time of the plan’s development, an accurate description of the entire set of indicators singled out by stakeholders is reported, along with their calculation. Afterwards, an application of a multi-criteria decision-making method, which is aimed at ranking the energy-related measures, on the basis of stakeholders’ preferences, is presented.

The residential building sector’s EEMP of the Sicilian Region

To properly design effective scenarios for energy consumption reduction along with energy use optimization and limitation of pollutant air emissions (European Parliament 2008; United Nations 1998), the existing state of the residential sector should first be defined. This information is, however, particularly difficult to be achieved because this sector is characterized by a large number of buildings spread across the territory. Therefore, the development process of the Sicilian EEMP has observed a long preliminary phase dedicated to gathering data on the energy consumption of this sector. During this phase, residential building typologies, HVAC systems and climatic conditions of sites were taken into account (Filogamo et al. 2014).

The energy-saving actions, candidate to be financed by the Sicilian Region’s building sector, were concerned with both structural interventions, such as the thermal insulation of the building envelope (external walls, coverings and glazed surfaces), substitution of heat generators (boilers and heat pumps) and non-structural interventions concerning electrical appliances. These interventions were selected among those easily applicable to existing buildings and among those beneficed by Italian national incentives that, at some extent, could be cumulated with those of the regional financing measures. This choice could usefully contribute to enhance the effectiveness of the regional financing tool. The energy and environmental savings achievable in the building sector were calculated using the scheme proposed by the Italian Electric Power and Gas Authority to quantify the primary energy savings related to certain technical interventions (Italian Law 24 2001). It is important to note that only investment costs have been included for each technical action, based on two considerations. Firstly, the budget to be optimally allocated does not include operating and maintenance costs because these are imputed to owners of buildings; moreover, the Italian economic laws for the years 2007 and 2008 expressly financed energy rehabilitation actions referred only to their implementation and not to their management.

To provide a fully representative example of the adopted methodology, we report the algorithm used to determine the energy effects of glazed surfaces, for summer cooling, on the control of the solar radiation in the household sector. The pertinent reduction of the primary energy use was computed using the following expression:

$$ \Delta E = {I}_{\mathrm{glob}}*{k}_{\mathrm{v}}*{k}_{\mathrm{u}}*{k}_{\mathrm{imp}} $$
(1)

where

  • ΔE is the overall reduction of primary energy consumption of the building in the summer season (toe/m2 year);

  • I glob is the total solar radiation beating the window surface in summer season (MWh/m2year);

  • k v is the change in the solar transmission coefficient due to the technical intervention;

  • k u is a “utilization factor” of the solar radiation (range of values 0 ÷ 1);

  • k imp is a conversion coefficient (toe/MWh).

As previously stated, in the original version of the plan, actions were analyzed and hierarchized using only economy-based indicators, i.e. the cost of the saved primary energy (€/toe) and the cost of the saved CO2 (€/t).

Because the main goal of the Sicilian Region’s EEMP was to optimally allocate the available monetary budget among different technical actions, the analysis was carried out by assuming the three following scenarios:

  • Potential scenario, which describes the maximum achievable energy savings. This scenario was derived by assuming that the hypothesized technology is fully introduced to the building residential sector of the region;

  • Actual scenario, which is based on the potential scenario but takes into account the circumstance that a given amount of users will autonomously (that is, without the recourse to economic incentives) replace the old technology with a new one over time; therefore, this scenario represents the actual maximum achievable energy savings;

  • Reachable scenario, which describes what portion of the actual technical scenario might be reasonably implemented during the master plan’s life span.

To provide an example of this modus operandi, Tables 1 and 2 depict the energy-saving trend in the case of installation and substitution of four-star heat generators, fed by natural gas. As it is possible to observe, after having computed the potential energy savings obtainable in each climatic zone (Table 1), the annual energy savings reachable were calculated (Table 2) on the basis of the given percentage of yearly implementation of the new heat generators from 2007 to 2011.

Table 1 Actual energy savings obtainable through the implementation of natural gas-fed four-star heat generators, by climatic zone
Full size table
Table 2 Yearly trends referring to the implementation of four-star heat generators
Full size table

As the Italian territory is subdivided into six climatic zones, based on their heating degree-days (Italian Law 412 and of 1993), Table 1 reports the mean values (weighted at municipal level) of these parameters for each considered climatic zone. As a matter of fact, only four out six climatic zones (i.e. zones B, C, D and E) were reported because only 0.2 and 0.3 % of municipalities in Sicily belong to climatic zones A and F, respectively (http://www.comuni-italiani.it/19/clima.html). On the other hand, in the Sicilian territory, 84 % of dwellings are included in the B (766 DD) and C (1150 DD) climatic zones.

Figure 2 reports the yearly trend of the reachable scenario from 2007 to 2012, along with the energy-saving amounts of the potential (20.8 ktoe) and actual (14.6 ktoe) scenarios.

Fig. 2
figure 2

The trend in saved primary energy (ktoe/year) obtained from the installation and substitution of a four-star natural gas heat generator

Full size image

In Table 3, the most relevant data to all of the EEMP’s planned technical actions are reported. Specifically, table reports the total investment costs, the hypothesized percentage of financial public support, the total amount of the saved energy and the CO2 emissions avoided during the plan’s life span.

Table 3 Data concerning action formats for the residential sector of the Sicilian Region
Full size table

The total investment cost of each considered technical intervention was computed by multiplying the pertinent economic intensive value (i.e. the current cost for implementing the technical action) by the extensive value represented by the number of interventions (based on a census analysis of the interested dwellings).

The cumulated values of the CO2 emissions were calculated by multiplying the energy saved during the intervention’s life span by the CO2 emission factors related to the involved energy source. The emission factors were those officially available (http://www.isprambiente.gov.it/it) at the time of the development of the Sicilian Region energy master plan, which, however, were in good accordance with those indicated in the 1996 IPCC guidelines (Tables 1–2 of vol. 2 workbook) (IPCC 1997). If all of the selected actions were considered, the potential and reachable energy savings in the year 2012 would be approximately 700 ktoe/year and 160 ktoe, respectively.

As observed in the last two columns of Table 3 (i.e. from the columns indicating the amount of money saved per saved energy and per avoided CO2 emissions, respectively), the installation of solar thermal systems (Action J), the replacement of electric boilers with the methane-fed ones (Action H), the installation of solar photovoltaic systems (Action K), the substitution of gas-fired sealed chamber water heaters (Action B) and the installation of vertical wall and roof insulation (Actions D and E) were the most promising interventions, based on the original EEMP analysis. The remaining interventions, i.e. the substitution of household appliances (Action F), replacing old heat generators with four-star appliances (Action A), the installation of high-efficiency air-conditioning systems (Action I) and the replacing of single with double-glazed windows (Action C), appear less useful. In fact, the costs of saved energy and saved CO2 are determined by two concurrent components: the cost of the single technical intervention and the saved energy per unit of capital invested. Therefore, the twelfth and the last columns of Table 3 directly establish a rank of the proposed actions.

Is the EEMP really in agreement with the stakeholders’ preferences?

As remarked earlier, €/toe and €/tCO2 do not consider the actual stakeholders’ preferences, whose full engagement would instead allow setting priorities of actions that are more tailored to the users. In the following sections, the stakeholders’ perspective is fully embodied in the priority-setting process.

As previously clarified, during the development process of the EEMP, an intense release occurred of technical standards (see Fig. 1) concerning the efficiency of the energy resources and the role of stakeholders in the energy planning processes (Berardi 2013). Therefore, it was decided to consult stakeholders representative of the social categories involved in the implementation of the plan, selected following an official document of the Regione Siciliana (2006). These stakeholders were affiliated with the Assessorships of the Sicilian Region (Industry, Agriculture, Culture and Environment), academic institutions (universities of Palermo and Messina, Italian National Research Council), seven out nine Sicilian Provinces, municipalities of the two greatest towns on the island (Palermo and Catania) and the Superintendence of Cultural Goods of three Sicilian provinces. There were also representatives of seven industrial development areas, the scientific and technological park, the most important firm of the renewable energy sources (RES) technologies operating in the island, environmental institutions, the three most important worker syndicates, the most relevant energy suppliers on the island, local groups promoting the sustainable development of the region, the Italian association of municipalities and oil refineries operating in the territory. In total, 67 stakeholders seated at the table.

Environment, energy and economy were proposed by the regional administration as general evaluation criteria, due both to the energy and environmental characterization of the master plan and the obvious economic constraints.

Subsequent to several meeting discussions (de visu or electronically performed), the stakeholders singled out pertinent indicators for each of these three evaluation ambits (see Table 4).

Table 4 Indicators selected by the stakeholders, subdivided by evaluation ambit
Full size table

EN1, the gross reduction of primary energy consumption, was chosen as the most important indicator for assessing the effectiveness of the energy-saving measures included in the regional energy plan (Dall’O’ et al. 2012; Eurostat 2013).

EN2, which is the energy intensity of the residential sector (toe/M€), was designated due to the important relationship existing between energy uses and the gross domestic product (Liddle 2012; McKenna et al. 2013), as a measure of the efficiency with which a country converts the specific GDP into energy commodities.

Indicator EN3 represents the total amount of energy saved throughout the life span of the proposed actions. Some stakeholders explicitly requested this parameter because it was considered a very effective for hierarchizing the different technical actions on the basis not only of their energy performances (Huang et al. 2012) but also of the total energy savings throughout the life span. This indicator was also considered one of the most useful for verifying the suitability of the adopted technological actions in the path towards nearly zero-energy buildings (Berggren et al. 2013; Berry et al. 2014), as required by a recent European Directive (European Union 2010).

The parameter ENV1, which is the CO2 emissions avoided throughout the life span of the proposed actions, was selected as the most representative of the environmental performance (Li et al. 2013) out of the proposed interventions, along with its close link with the consumption of primary energy in the residential sector.

The aptitude of a given system of limiting the greenhouse gas emissions per unit of gross domestic product is described by the indicator ENV2 (tCO2/M€). This parameter represents a sort of elasticity coefficient for the environmental performances of a territory that measures the increase of the global warming-related emissions with the improvement of the economic development (Ibn-Mohammed et al. 2014).

The indicator EC1, which is the cost of saving 1 t of primary energy, was chosen by stakeholders because the administrations were in charge of the management of the energy plan and called for the optimal allocation of economic resources based on different possible technical interventions in the building residential stock (Morrissey et al. 2013).

The environmental cost-benefit ratio resulting from the energy plan’s actions was noted by the majority of stakeholders, although this point has long been debated (Pulselli et al. 2009; Weitzman 1994) due to the controversial relationship between the energy effectiveness and environmental sustainability. Consequently, a further indicator (different from EC1), which specifically refers to the cost of the saved energy, was introduced: indicator EC2, which is the cost for the regional administration of avoided CO2 emissions at the end of the plan.

During the debates among the involved stakeholders, it emerged that regional administration and building owners differently perceive the costs related to the implementation of the energy efficiency measures. In fact, the administration is essentially called to optimally allocate the available resources based on the benefits resulting from the selected actions; conversely, the owners, although benefiting of the energy advantages of the adopted technical actions, are charged the current costs of implementing these actions. For these reasons, the stakeholders requested a specific indicator, EC3, able to evaluate the cost charged to the owners for saving one unit of primary energy in their buildings.

The increase of working hours, resulting from the adoption of the measures selected in the plan, was voted by stakeholders as an indispensable parameter, on the assumption that greening the building sector is supposed to increase the number of new jobs (McGraw Hill Construction 2012). Despite the fact that the construction segment (and employment as a consequence) suffered from the recent economic downturn, it remains a promising sector from the job creation perspective, as recently pointed out by Italian sectorial reports (http://www.energystrategy.it/assets/files/EER_15_def_protetto.pdf). Indicator EC4, that is, the increase of working hours due to the adoption of the selected actions, was selected for this goal. The value of this indicator was obtained by multiplying the extensive value of the technical intervention (i.e. either the total surface to substitute or the total number of new installations) by the number of hours needed to execute the single intervention.

The values of these new nine indicators were calculated for the ten considered technical actions (A to K of Table 3) by using the Sicilian Territorial Official Database (Sistema Informativo Territoriale (SIT), as reported in Table 5.

Table 5 Energy, environment and economy indicators
Full size table

To provide an example of how the values of the indicators (Table 5) were obtained for every action, details for Action A (that is, the substitution of gas boilers with four-star natural gas boilers) are provided in the following sections.

As for the evaluation of the ambit “energy”, the value of the indicator EN1 corresponding to Action A, i.e. 1,310.77 ktoe/year, was the difference between the gross energy consumption (excluding the non-energy uses) of the Sicilian residential sector in 2012 in the “business as usual” scenario (1316.62 ktoe/year) and the reachable energy savings produced by the implementation of Action A up to 2012, that is, 5.85 ktoe (Table 2):

$$ \mathrm{E}\mathrm{N}{1}_{\left(\mathrm{Action}\ \mathrm{A}\right)}=1,316.62\hbox{--} 5.85=1310.77\left[\mathrm{ktoe}\right] $$
(2)

Analogously, for the evaluation of the ambit “environment”, the value of the indicator ENV1, pertinent to the same Action A, was given by the avoided emissions of CO2 in the residential sector in 2012, resulting from the implementation of Action A (Table 3):

$$ \mathrm{E}\mathrm{N}\mathrm{V}{1}_{\left(\mathrm{Action}\ \mathrm{A}\right)}=135,627\left[{\mathrm{ktCO}}_2\right] $$
(3)

Finally, as for the ambit “economy”, the value of the indicator EC4, for the same Action A, that is 192,343 working hours, was given by the number of interventions in 2012, which is 40 % of the total (Table 2), multiplied by the number of hours needed for a single intervention.

Once these nine pertinent indicators of the three main ambits of evaluation were assessed, a multi-criteria decision analysis (MCDA) was applied for properly ranking them.

Application of a multi-criteria decision-making method

During recent years, the analytical methods were applied to urban and environmental planning to prioritize the possible alternatives utilizing a given set of criteria (Nijkamp et al. 1990). Among these tools, the MCDA methods must be mentioned (Ananda and Herath 2009; Grafakos et al. 2010). These methods were developed to support the planning and decision processes where several alternatives have to be considered and evaluated referring to different criteria, and a large number of decision makers with disagreeing preferences are involved also allowing the identification of an overall ranking of preferences from many alternatives.

To prioritize the energy-saving initiatives proposed in the Sicilian Region’s EEMP, among the available MCDA, the analytic hierarchy process (AHP) method (Saaty 1980, 1990) was selected because it uses a quantitative technique based on pairwise comparisons of elements in a hierarchy (Espen Løken 2007) in order of matching the different alternatives with the various criteria and to estimate the values of weights attributed to the criteria. This quantitative approach fits very well with the task of these stakeholders that were requested to choose among various indicators characterized by a quantitative description.

Moreover, this method reproduces the human mind approach to problem solving, by grouping similar elements in the same level of priority; also, it suitably provides a quantitative scale for setting priorities, and finally, instead of pushing towards a consensus, it provides representation of different judgements.

Within the AHP method, a given complex problem is separated into its fundamental constituting elements, while a group of actions and functions leads towards the result using a network structure, a sort of upside down tree, where the roots represent the principal targets to be achieved and the branches the various alternatives to be prioritized.

Figure 3 reports the typical tree-wise arrangement of the AHP method in which a given problem is structured in several levels, according to a so-called dominance hierarchy.

Fig. 3
figure 3

Hierarchical structure of the AHP method

Full size image

In its simplest form, the hierarchy is organized from the highest level (the main goal to be achieved) through the intermediate levels (criteria or attributes upon which a decision is based among different options) down to the lowest level (the list of options). On turn, each level might be divided into other sets of more detailed sub-criteria.

In the present application, a four-level structure was adopted for suitably approaching the decision problem of the residential sector:

  • One-degree level—main goal

    • Energy saving and environmental sustainability of the residential sector in the Sicilian Region, coherent with the optimal allocation of the monetary budget aimed at financing energy rehabilitation actions. This main goal was established by the regional administration.

  • Two-degree level—evaluation ambits

    • Energy (EN)

    • Environment (ENV)

    • Economy (EC)

  • Three-degree level—evaluation ambit indicators

    • Energy efficiency indicators (EN1, EN2 and EN3);

    • Environmental indicators (ENV1 and ENV2);

    • Economic indicators (EC1, EC2, EC3 and EC4);

  • Four-degree level—ten proposed actions within the EEMP

For the fourth level of the hierarchical structure, the energy-saving interventions considered in the AHP method are those reported in the second column of Table 3.

This hierarchy, which will be clarified in the following section, is depicted in Fig. 4 that specifies Fig. 3 for the present case, where the “local” (L) and “global” (G) weights acting in the AHP method are indicated as well.

Fig. 4
figure 4

Structured hierarchy for prioritising the EEMP actions

Full size image

A hybrid AHP scheme

The method usually prescribes pairwise comparisons of the elements at each level of the hierarchy to estimate the importance of each element with respect to those of the upper level, so obtaining an overall weight for each element. In this application of the AHP to the Sicilian Region’s EEMP, a hybrid configuration of this method was applied. Actually, this type of structure is not rare for the AHP method (Chen and Wang 2010; Fahrul Hassan et al. 2012; Kim et al. 2013; Turcksin et al. 2011; Haydt et al. 2015).

In the present application, instead of pairwise comparisons of the second level (evaluation ambits: energy, environment, economy) against the first level (main goal: sustainability and optimal allocation of the regional budget), and the third level (indicators) against the second level (each pertinent evaluation ambit), a different weighing procedure was adopted to derive the relative scales. Specifically, the local weights, L, were assigned to the following:

  • The elements of the evaluation ambits (second level) by taking into account the number of indicators of each ambit, as specifically required by the Sicilian Region Administration, which means that the weights of the evaluation ambits, L 2, are 0.33 for energy (three indicators out of nine: 3/9 = 0.33), 0.22 for environment (two indicators out of nine: 2/9 = 0.22) and 0.44 for economy (four indicators of out nine: 4/9 = 0.44), whose sum is 1

  • The indicators (third level) pertinent to each evaluation ambit by taking into account the stakeholders’ preferences. More specifically, the number of preferences expressed by the stakeholders for the nine above-described indicators was considered as a vote, attributed to the pertinent indicator. Afterwards, these votes were converted into a percentage (that is the number of stakeholders who voted in favour of each indicator divided by the total number of stakeholders), which was considered the local weight, L 3, to be attributed to each indicator in the third level. With specific reference to these local weights, the AHP method seems particularly suitable for the inclusion of stakeholders in the priority-setting processes for a set of alternatives (Lafreniere et al. 2013; Kim et al. 2013; Shen et al. 2013).

Table 6 reports the percentage of stakeholders that expressed a preference for each indicator belonging to the three evaluation ambits, being the total percentage equal to 100 for each ambit. These percentages are the local weight factors to be attributed to level 3.

Table 6 Preferences (expressed by the stakeholders for the proposed indicators) converted into a percentage
Full size table

Instead, the classical pairwise comparison procedure was applied to the elements of the fourth level of the hierarchy, i.e. to the energy-saving actions.

Generally, when two actions of Table 3 are compared by means of an indicator of the evaluation ambits (EN, ENV or EC), a coefficient a ij , which describes the relative importance of each of i elements with respect the other j elements, is generated (Saaty 1980). In the case of quantitative elements (as in the present circumstance), the coefficient a ij might be determined by simply rating the values assumed by the two actions under analysis. For example, referring to the indicator EN1 and the Actions A and B, the pertinent coefficient a ij was calculated as the ratio (Table 5) of 1310.77 ktoe (Action A) to 1296.9 ktoe (Action B), which are the values of the reduced gross energy consumption obtainable when implementing Actions A and B, respectively (a ij  = 1.0107). In the same way, referring to the indicator EC1 and the Actions A and B, the pertinent coefficient a ij was calculated as the ratio (Table 5) of 0.0023 €/ktoe to 0.0063 €/ktoe, which are the values of the average cost at year 2012 of one saved toe, attainable by the implementation of Actions A and B, respectively (a ij   = 0.365).

Totally, the pairwise comparisons led to the determination of nine matrices, one for each indicator, belonging to the ambits EN (three matrices), ENV (two matrices) and EC (four matrices).

It seems worth pointing out that, certainly, the AHP method permits inconsistency, especially when people are asked to pairwise compare qualitative variables, but since, in our case, quantitative indicators were pairwisely compared and the elements in the matrices were obtained simply rating the values that the examined indicator assumes in the options considered, a verification of the logic consistency of the obtained pairwise comparison matrices was not necessary.

As prescribed by the procedure, once the pairwise comparison matrices are obtained for each indicator, the eigenvectors of each of these matrices have to be calculated for determining the local priority order of all the elements in the matrix. Therefore, the eigenvector components, v i , were calculated as follows:

$$ \begin{array}{l}{v}_1=\sqrt[n]{a_{11}\times {a}_{12}\times \cdot \cdot \cdot \times {a}_{1n}}\hfill \\ {}{v}_2=\sqrt[n]{a_{21}\times {a}_{22}\times \cdot \cdot \cdot \times {a}_{2n}}\hfill \\ {}{v}_n=\sqrt[n]{a_{n1}\times {a}_{n2}\times \cdot \cdot \cdot \times {a}_{nn}}\hfill \end{array} $$
(4)

where n is the number of actions under analysis. These elements need to be further normalized, by applying proper normalization factors so that their sum is one. For the indicator EN1, the normalization factors, x i, is computed as follows:

$$ {x}_1=\raisebox{1ex}{${v}_1$}\!\left/ \!\raisebox{-1ex}{$S$}\right.\to \to {x}_2=\raisebox{1ex}{${v}_2$}\!\left/ \!\raisebox{-1ex}{$S$}\right.\to \cdot \cdot \cdot \cdot \cdot \to {x}_n=\raisebox{1ex}{${v}_n$}\!\left/ \!\raisebox{-1ex}{$S$}\right. $$
(5)

where

$$ S={\displaystyle {\sum}_{i=1}^nvi} $$
(6)

The elements x i are the components of the local priority vector, L 4, that sets priorities for the options under analysis with respect to the indicator EN1 (that is L 4,EN1 in the scheme of Fig. 4). In other words, the technical actions (A to K) are listed in order of priority only with respect to the indicator EN1 without accounting for the ranks pertinent to the other indicators (so justifying its denomination of local). Obviously, a local priority vector must be calculated for all the remaining indicators.

At this point, for determining the global priority vector for each indicator (for example, G 4,EN1 for the indicator EN1 in Fig. 4), the principle of hierarchical composition was used. Therefore, the local preference ratings, x i , of the local priority vector were multiplied by the corresponding global weight of the element of the upper level (G 3). To provide an example, in case of the EN1 indicator, the elements of the vector L 4,EN1 were multiplied by G 3,EN1 yielding the vector G 4,EN1.

In general, for an assigned number k of levels of the hierarchy (four in this case), the value of the global weight G k is given by the following:

$$ {G}_{k=}{G}_1\times {\prod}_i^k{L}_i $$
(7)

In particular, when referring to the fourth level, this expression becomes for the case of EN1:

$$ \left|{G}_{4,\ EN1}\right|={G}_1\ *\left|{L}_{4,\ EN1}\right|*{L}_3*{L}_2 $$
(8)

Therefore, the nine local weights associated to Action A (i.e. L A4,EN1, L A4,EN2, L A4,EN3, L A4,ENV1, L A4,ENV2, L A4,ENV2, L A4,ENV3, L A4,ENV4, L A4,EC1, L A4,EC2, L A4,EC3 and L A4,EC4) were converted into nine global weights.

Finally, all these global weights were summed up, in this way obtaining the relative importance of the Action A with respect to the main goal, i.e. its “total” global weight:

$$ {G}_A={\displaystyle \sum_{i=EN1\div EC4}^9{G}_{A4,i}} $$
(9)

Results

With the application of the hybrid scheme of the AHP method, a new arrangement is obtained for the planned technical actions that show how the inclusion of the stakeholders’ preferences2 determines a new order of priorities in the aim of allocating the regional monetary budget into proper financing tools. The comparison of the results attained without (priorities only based on the economic parameter €/toe) and with (priorities based on the stakeholders’ preferences and hierarchized using the AHP procedure) the direct involvement of the interested parties shows some remarkable changes in the rank of the technical actions. As Table 7 illustrates, four interventions, i.e. the building envelope insulation (D), the roof insulation (E), the replacing of single-window glasses with double-window glasses (C) and the installation and/or replacing of high-efficiency air conditioning systems (I), improved their positions when the stakeholders’ preferences were taken into consideration. The remaining six actions achieve worse positions, while only the installation of solar thermal systems (action J) maintains its position at first rank.

Table 7 Changes in the rank of the EEMP’s actions
Full size table

Obviously, these modifications are significant for an effective regional energy policy. Their importance is evident when matching these two rankings with the economic resources of the regional administration, available for financing the implementation of these actions. In Italy, on purpose, the most relevant financial tool is the “Programma Operativo Interregionale” (POI 2007–2013) for the renewable energy and energy saving. This tool applies to the so-called convergence regions (Puglia, Campania, Calabria and Sicilia) according to the European Community decision on strategic guidelines on cohesion (European Commission 2006), devoted at realising the intimate relationship between the economic, social and environmental dimensions.

An estimation of the regional budget allocable to a given plan, and especially an estimation of the percentage of this budget assigned to a specific action for the enhancement of the energy efficiency in the residential building sector, is quite a difficult task. Nonetheless, considering that the development process of the EEMP ended in early 2009, the financial support might be developed over 4 years (2009 ÷ 2012), up to the end of the validity of the plan. This leads to a rough estimation of an available budget of 85 M€. Actually, during the time, budgets entitled for planned purposes are often reallocated by the regional administration to other purposes, due to possible changes of economic targets of the regional policy. Anyway, we assumed here the starting budget of 85 M€ as an estimation of the amount of money that might have been employed over 4 years up to the end of the plan (2012).

The comparison of two different scenarios referring to the same amount of financial support is depicted in Fig. 5. It is possible to observe from the graphs the growth of the cumulated budget, depending on the implementation of the energy-saving actions, along with the corresponding value of the saved energy. In the case where the old ranking is used (with the application of purely economic indicators), five actions could be implemented with the available budget (J, H, K, D and B). In contrast, using the newly prioritized order (with the preferences suggested by the stakeholders), six different actions could be implemented (J, D, E, H, K and C).

Fig. 5
figure 5

Reachable actions with the available budget for the two scenarios

Full size image

This comparison suggests two orders of considerations. First, the number of actions that can be financed with the same budget passes from five to six, by reaching the same amount of energy saved along the duration of the plan. Apart from the mere adding of an energy-saving action, this new group of actions calls for further categories of workers for the implementation of the plan, conferring it a greater social effectiveness. By the way, the number of workers involved in the interventions was one of the new indicators selected by the stakeholders. On the other hand, the new hierarchy depicted by the stakeholders modifies the kind of actions included in the optimization process, by comprising the roof insulation (E) and the double-glazed window substitution (C) and excluding the replacing of gas-fired water heaters (B), in this way introducing a relevant correction in the original plan.

Before proceeding with an examination of these results and with a critical analysis of the importance of the stakeholders’ positions, one would consider which would be the result if different preferences were proposed. This inquiry, being based on different votes of the involved categories of final utilizers of the plan, can be regarded as a sort of a sensitivity analysis of the method; it, in fact, shows the effects produced by changes in the values of the weighted factors in the hierarchizing process. On purpose, we have run a simulation based on another set of votes hypothetically released by stakeholders. The assumed hypothesis is that stakeholders shift their preferences to certain indicators that are usually easily understood by citizens: energy saving in the residential sector (EN3), the effects of the cost of the pollutant emissions (ENV2) and the increasing of number of working hours resulting from the adoption of the selected technical actions (EC4). Such a choice, despite here considered as a mere simulation, could reasonably be made by a given set of users of the plan, being based on well-perceived social issues. Table 8 reports the result of this scenario, compared with the previous one, obtained with the actual choices of stakeholders.

Table 8 Preferences (%) expressed by stakeholders for the ambits’ indicators in the actual and simulated scenarios, along with the corresponding results
Full size table

Despite that the total number of actions implementable with the available budget remains the same of those of the actual scenario (respectively, three and eight actions in the minimum and maximum levels of utilization of the budget), now, the type of the interventions and its rank is different. In fact, in the simulated scenario, the four-star LPG boilers enter the number of the adopted actions. In the same time, methane boilers are now out of the set of the implemented actions with the available budget. Moreover, several other actions change their rank in the list.

This simple simulation shows the importance of the stakeholders’ votes and their pertinent involvement in the optimization process of an energy and environmental plan, besides showing the sensitivity of the AHP method with respect to the stakeholders’ preferences.

Discussions

Due to complexity of the procedure and to the circumstance that human judgements are involved in this optimization process, several points need to be further discussed. They refer to the following points: extension of the present method to other Italian (at least) regions, the criterion for the selection of the considered energy-saving actions, the choice of the utilized indicators, the possible effects produced by the variability of the budget allocable to the financing measures and the choice of the method here applied for the optimization process.

Replicability of the method

Despite that the method here presented is developed as a case study referring to the Sicilian region, actually, it is intended as a tool easily applicable to other Italian (at least) regions. In fact, the so-called Italian “convergence” areas, that is, Sicily, Calabria, Campania and Apulia, are included in the list of the Italian regions that can benefit of particularly profitable financing regime in the field of the energy rehabilitation and saving measures. The positive effects that these regions could get from the application of such an optimization method are those of an effective allocation of the financing resources aimed at energy-saving measures in the residential sector; this sector, in fact, shows similar features in all the convergence Italian regions. Moreover, the method, being only depending on typical data concerning the building park of a given territorial context and on the implementation cost of assigned energy-saving interventions, candidates itself as an effective tool for the European regions for which these data are available. In fact, since in sight of the Cohesion Policy, several European regions are eligible for funding from the structural funds, such an optimization method could be usefully applied up to 84 convergence regions belonging to 17 countries.

About the selection of the energy-saving actions

Stakeholders were called to express their rank of preference among a given set of energy-saving actions for the regional building sector. These actions, candidate to be financed, were selected among those easily applicable to existing buildings and belonging to the most developed and mature technologies. In addition, in order of better fostering the efficacy of the financing tools, these interventions were those beneficed by Italian national incentives that could be usefully cumulated with those of the regional financing ones.

Clearly, the most effective action for a given building would be represented by the contemporary adoption of multiple energy efficiency measures (EEMs). Anyway, this financing tool is not here intended for application to a single building, but for an optimal allocation of a given regional budget into EEMs to which citizens could apply. Nevertheless, a proper combination of effective EEMs can be arranged by owners simply applying for founding to multiple financed actions.

About the selection of stakeholders and indicators

As previously said, stakeholders were selected by the Sicilian Region among the social categories involved in the implementation of the plan. These categories are reported in an official document of the regional administration (Regione Siciliana 2006). Anyway, it must be noted that other categories could have been interested in the plan, despite not included in this official list: for example, traditional manufacturing industries should have been better represented, while the regional administration preferred to involve particularly industries working on the field of RES. Obviously, a different panel of stakeholders could lead to a different definition of indicators and to a different hierarchical order of the proposed actions.

Therefore, a well-established procedure for the selection of stakeholders should be assessed, in order of involving all the interested categories and in the aim of making more comparable plans of different regions.

Moreover, the choice of the indicators utilized for this ex-post analysis of the plan directly depends on the kind of involved stakeholders. Actually, they mainly proposed “technical” indicators, due to their affiliations: almost all, in fact, were representative of companies or factories involved in the production and distribution of energy.

Finally, a regional energy master plan should also properly include non-technical actions. Actually, apart from the “increase in the number of working hours”, other social-related issues were considered not directly in the plan but within its Strategic Environmental Assessment (SEA) that was lately prepared (E.C., 2001).

Possible effects of the budget’s change

Obviously, a territorial policy concerning the utilization of economic resources for the implementation of an energy plan cannot be strictly established a priori because modifications of the available budget can occur during the ongoing process of the plan. Therefore, the feasibility of the planned actions should be better related to this possible variation of the available budget. A change in the budget of, for example, ±25 % would cause a variation in the number of actions that can be implemented, as shown in Fig. 5. The dashed lines indicate the planned budget and its ±25 % variation. It is interesting to note that, even in this case, the intervention of stakeholders would usefully modify the scenario: in fact, for a deviation of the budget of ±25 %, the number of actions that are likely to be implemented would range from 3 (−25 %) to 7 (+25 %) (old ranking) and from 3 (−25 %) to 8 (+25 %) (new ranking).

This simulation shows again the relevance of the stakeholders’ preferences and signals an important criticism of the priorities established in the EEMP, which make a significant difference in the development of a regional energy policy.

About the AHP method here adopted

About the utilization of the AHP method for the optimization process, it is important to note that, here, it is not aimed to optimize the budget allocation based only on economic features. If the optimization should rely only on monetary considerations, maybe other methods should have been applied more properly: probably, the simple adoption of cost-related indicators (cost of the saved energy and cost of the avoided CO2 emissions, for example) would be effective enough for approaching the problem. But, in this case, the allocation of the budget should not be intended as a simple monetary process but as a political allocation of resources. The Sicilian regional administration, in fact, wanted to allocate its budget available for energy-saving and environmental preserving measures in sight of the satisfaction of the final utilizers. Therefore, the choices of the stakeholders are needed to be suitably incorporated in the procedure, in order of mitigating the mere economically based distribution of the available monetary resources. In this perspective, the AHP method has shown to be quite effective, since it enabled the inclusion of the stakeholders’ preferences (that is their votes) by proper weights of the hierarchical procedure.

The application of the “hybrid” AHP model here selected does require some further clarifications, particularly referring to the way with which the quantitative parameters have been included in the method, the scale of preference here adopted and the way with which weights are computed.

Quantitative measures are included in the method in two separate phases. The first one refers to the calculation of values of the indicators for each considered technical action. An example of the calculation of indicators EN1, ENV1 and EC4 is provided in the “Is the EEMP really in agreement with the stakeholders’ preferences?” section. The second phase refers to the evaluation of the a ij parameters. These factors allow evaluating the relative importance of each element i with respect to another element j, that is, the comparison of two actions belonging to a given evaluation ambit (EN, ENV and EC). In the “A hybrid AHP scheme” section, an example has been provided, concerning the relative importance of actions A (replacing gas boilers) and B (replacing gas-fired water heaters), illustrating how the parameter a ij is computed in this case.

Obviously, the computation is replicated for all the combinations of the technical actions.

Concerning Saathy’s scale that belongs to the original formulation of the AHP method, it has not been here adopted because, in this case, we are managing quantitative parameters. In fact, when quantitative factors are in context, the weight of each element is just represented by the value of the parameter in the considered alternative, while only in the case of qualitative elements, the weight is determined by using a semantic scale or by means of rating techniques. Therefore, in the present application, being all elements of the evaluation process represented by quantitative factors, the generic element a ij of the pair comparison has been simply determined by means of the ratio between the values assumed by the element in the different alternatives of the plan, in this way determining the “local weight” in the evaluation process. Clearly, a compensatory method has been here applied, whose main implication is the direct homogenizing of results: this represents a relevant advantage in the complex evaluation of a regional energy master plan. This method also allows a direct hierarchizing of the results, which is exactly the main goal of this plan.

As it is possible to note, no consistency analysis takes place in the present application. It, in fact, particularly applies to qualitative parameters, in order of verifying the robustness of the final autovector. Since, in this case, the weights of the matrices have been singled out by means of quantitative parameters, the pair comparison leads to a “consistent” matrix that for each value of i, j and k is supposed to verify the condition a ij  = a ik a kj . On the other hand, if the weights were determined with Saathy’s semantic scale (or by means of a rating method), the obtained matrix would be generally non-consistent; this is due to the difficulty of the technicians supposed to provide the judgements, to maintain a coherent judge in all the pairwise comparisons.

In our case, the matrices of the pairwise comparison were intrinsically consistent and this avoided us to compute the final principal auto-value λ max that aims to verify the logical consistency of the method in case of qualitative parameters.

Moreover, certain concerns could be raised about the particular scheme of the AHP method here applied that could have affected the results. In fact, a hybrid scheme was here adopted, in which the pairwise comparisons were only done at the level of the alternatives (fourth level), while for the upper levels (second and third levels), the relative importance of the elements was determined on the basis of both the regional administration requirements (that is the selection of the three ambits of evaluation) and the direct suggestions coming from the technical/policy panel (that is the singling out of the nine indicators). However, this hybrid scheme is usually applied in the literature (Chen and Wang 2010; Fahrul Hassan et al. 2012; Kim et al. 2013; Turcksin et al. 2011; Haydt et al. 2015) and allows an easy introduction of the judgments of stakeholders about the selected indicators in the evaluation process.

Conclusions

This work reports an ex-post evaluation of the Energy and Environmental Master Plan of the Sicilian Region assessed using preferences expressed by stakeholders, representing the most important social and productive categories of the island. They, in fact, were not directly involved in the priority-setting process but were only asked to express their opinions about the plan.

The circumstance that the authors were part of the technical panel that developed the plan and participated in the stakeholders’ consultation process constituted an interesting opportunity for verifying, a posteriori, the robustness of the choices initially singled out on only the base of economic considerations (the costs of the saved energy and the saved CO2 emissions).

To prioritize the energy-saving measures planned for the residential building sector, including the stakeholders choices (that refer to different issues and ambits, not only the economic ones), a modified AHP method has been applied. Some interesting differences in the priorities were found, particularly in terms of number and types of actions to be implemented. This discrepancy confirms the criticism about the energy plan developed using only economic indicators.

Based on the results of this work, one consideration certainly rises regarding the level of stakeholder involvement in the development of an energy master plan: it, indeed, should take place in the early phases of the process and should not be limited to the ex-post evaluation of the Strategic Environmental Analysis (SEA), as it, actually, was during the development of the EEMP of the Sicilian Region. Therefore, a stakeholder empowerment, rather than just a simple stakeholder participation, in the design of energy and environmental plans starting from the early phases up to the ex-post stage, is certainly recommendable.

As regard the method proposed, it is, as mentioned previously, a tool easily applicable to other Italian (at least) regions and effective for the European regions for which data concerning the building park of a given territorial context and data on the implementation cost of assigned energy-saving interventions are available. Also, the particular scheme of the AHP method here applied allows an easy introduction of the judgments of stakeholders about the selected indicators in the evaluation process. Nonetheless, since the results obtained by such a scheme strongly depend on the indicators selected which, in turn, are chosen by the involved stakeholders, some issues still need to be carefully addressed. One regards the awareness of stakeholders about their role: stakeholders should intervene in the evaluation process after a proper training in which importance and meaning of their role were suitably explained. The other regards the selection of the stakeholders in itself: a well-established procedure for their selection should be assessed, able to involve all the interested categories in a clear and recognized way.

All these points indicate the need for further research directions concerning the evaluation of energy master plans by means of MCDA methods.