Introduction

Manufacturing industries account for one-third of global energy demand and nearly 40% of carbon dioxide (CO2) emissions (IEA 2009). In EU-27, the sector accounts for 28% of final energy demand and 22% of CO2 emissions due to its fuel use (EC 2010a).Footnote 1 Consequently, it is crucial that industries contribute to targets like 20% primary energy savings of the EU Action Plan on Energy Efficiency and the long-term objective to reduce GHG emissions by 80–95% by 2050. Decision makers will need to engage the industrial sector in constructive ways to meet the challenge. This paper examines the Swedish voluntary agreement (VA) Programme for improving energy efficiency in energy-intensive industries (hereafter referred to as PFE or the programme) to assess whether this can serve as a good practice example among policy initiatives. As in many other VAs, the industrial companies are motivated by a tax rebate to enter into a multi-year legally binding agreement and pursue certain measures for energy efficiency improvement (Price 2005; Krarup and Ramesohl 2000). PFE is thereby guided by the dual ambition of facilitating competitiveness while governing industry towards political goals on energy efficiency improvement.

The development of effective energy efficiency policies as well as practices for monitoring and evaluating their results has become increasingly important with the target setting at different political levels. In National Energy Efficiency Action Plans (NEEAPs) the EU Member States shall list and quantify the impact from those national measures (e.g., policies or market mechanisms) that are planned for reaching the Energy Service Directive (ESD) target of 9% energy savings by 2016 (ESD Article 4 and 14, 2006).Footnote 2 The ESD has also triggered the challenging task of developing EU-harmonised evaluation methods (ESD Annex IV, 2006; EMEEES 2009).

The first 5-year period of PFE was concluded in 2009, and the main purpose of this paper is to evaluate the programme both in terms of its process and impact. Swedish industrial energy use and the PFE policy design section provides a background on Swedish industrial energy use and describes the main elements of PFE. Features of the inherent policy theory and how programme activities have progressed in relation to these are discussed in Process evaluation section. In Impact evaluation section, the impact evaluation brings forth programme results in terms of quantified energy savings and cost-effectiveness; common criteria for judging the success of policy instruments. The combination of perspectives aims at contributing to the deeper understanding of PFE which is found necessary for interpreting its results. In Discussion and remarks on policy implications section, we discuss the results and its implications for energy efficiency policy.

Swedish industrial energy use and the PFE policy design

Industrial energy use in Sweden

Since 1970 the Swedish energy system has made a notable shift away from oil as the dominating primary energy source. Nuclear capacity has been scaled up to the extent that hydro and nuclear power provide almost equal shares, and together some 90% of total electricity production (i.e., 146 TW h in 2008). CHP, foremost biomass-fuelled, provide most of the remaining generation capacity and a substantial heat supply via the extensive district heating grid. The industrial sector, including mining and quarrying and the manufacturing industries, has contributed to the development by shifting its energy end-use away from oil products towards more electricity, as shown in Table 1. Biomass has become increasingly important in the energy demanding pulp and paper industry (PPI). In 2007, 78% of its fuel consumption was covered by internal biomass sources, primarily black liquor and bark (Wiberg 2007).

Table 1 Swedish industrial energy use 1970–2008
Full size table

Induced by a scheme of tradable renewable electricity certificates the PPI auto-produced 5.9 TW h electricity in 2008, which represents a 40% increase since the scheme was launched in 2003 (SFIF 2011). This corresponds to 25% of the electricity demand of the entire PPI (i.e., 22.6 TW h in 2008). The industrial use of natural gas and district heating has increased steadily since the 1980s when these energy carriers were introduced in the sector. Consumption of coal and coke has been more or less constant due to its function in reducing iron oxides in the blast-furnace process of iron manufacturing.

The Swedish industrial sector has a record of decreasing energy intensity. Industrial final energy demand has been around the same level since 1970, while the total value added has increased by a factor of about 2.5 (SEA 2009b). When considering the primary energy demand of electricity production the decoupling effect appears somewhat less pronounced. Assuming 40% generation efficiency the industrial primary energy use has increased by 30.5 TW h, or 15%, over the same period due to increasing electricity demand. The trend of decreasing electricity intensity starts first in the early 1990s. Over the past decades, the less energy-intensive types of manufacturing industries have become increasingly important to the Swedish economy. In 2005, almost 50% of the industry’s value added was generated in the engineering industry, including for instance manufacturing of machinery, electronic and optical components, and transport equipment (Johansson et al. 2007). The same engineering industry accounts for less than 10% of industrial final energy use. Indeed, structural change is an important factor behind the decrease in specific energy consumption but decomposition analysis has also identified that Sweden has had an industrial energy efficiency improvement of 14% between 1990 and 2005 (Odyssee 2009).

Policy making for industrial electricity efficiency

In terms of policy making for industrial energy efficiency, there are certain reasons for focusing on electricity. Over the years, it has become the dominating energy carrier in the sector (see Table 1). In previous decades, Swedish energy-intensive industries have had a competitive advantage from low electricity prices but after the deregulation of the electricity market in 1996, an increased integration with continental Europe and the introduction of EU-ETS, the situation has been altered. For these companies, being export-oriented and subject to international competition, the increase in wholesale electricity prices from the low level of year 2000 until today has become a serious concern (Nord Pool Spot AS 2011). Cost-cutting by improving electricity efficiency can reduce the exposure to increasing and volatile electricity prices.

Industrial competitiveness is a prioritized political goal. Hence, a policy that obligates the sector to act will typically be combined with an economic incentive (e.g., a tax reduction). This is complicated by the fact that Swedish energy-intensive industries are largely exempted from energy related taxes. The general energy tax on fuels is set at zero for manufacturing processes and the carbon tax on fossil fuel use is significantly reduced for energy-intensive industries. Since fossil CO2 emissions from many of these facilities are controlled under the EU-ETS cap and trade, further policy interference would be futile in terms of short-term emission reductions (Henriksson and Söderholm 2009). Moreover, industrial electricity consumption was untaxed for many years but in 2004 this received criticism from the European Commission for being incompatible with the common market (EC 2004). Hence, the Swedish government was forced to promptly remove the illicit state aid of zero taxation by introducing the minimum tax of 0.5 Euro/MW h on industrial electricity use.Footnote 3 The Energy Tax Directive (ETD), however, can provide the opportunity of reduced taxation for energy-intensive businesses if these enter into agreement on energy efficiency improvement (ETD Article 17 2003). Thus, to enable the tax exemption for energy-intensive companies and to stimulate their energy and in particular electricity efficiency improvement, Sweden launched PFE in 2005.Footnote 4

The PFE outline

PFE is intended for energy-intensive companies as defined by the criteria: (1) purchases of energy products and electricity amount to at least 3% of the production value and/or (2) the energy-, carbon dioxide-and sulphur tax on energy products and electricity used by the company amount to at least 0.5% of the added value (§ 4 of SFS 2004:1196; ETD Article 17 2003). Companies from the sectors pulp and paper, mining, iron and steel, non-metal minerals and industrial chemicals are typical candidates. In some cases also food processing industries, saw mills and engineering industries can qualify as energy-intensive. From January 2005 and onwards the programme period starts when the company is accepted for participation and lasts for the 5 subsequent years.Footnote 5

During the first 2 years, the company has to introduce and obtain certification for a standardized energy management system (EnMS)Footnote 6 and carry out an energy audit and analysis. The audit report describes the plant’s energy use and proposes energy saving measures based on an analysis of energy demand in short- and long-term perspective (SEA 2004). This work is concluded with a list of identified electricity saving measures. Those listed measures with payback periods of less than 3 years have to be implemented while measures with lower rate-of-returns are pursued on a voluntary basis. The list is submitted to and approved by the Swedish Energy Agency (SEA), which is the administrating agency. During the first 2 years, the company also has to introduce routines for procurement of high-consumption electrical equipment as well as routines for energy efficient project planning. The core purpose of these routines is that the company shall acknowledge the life cycle cost in its procurement and investment decisions and thus give preference to energy efficient equipment (SEA 2006).

After the first 2 years the participating company must submit its first report to SEA demonstrating how the requirements have been met and what level of electricity savings to be expected from the listed measures. During the following 3 years, the company shall implement the measures and continuously apply the EnMS as well as the routines for procurement and project planning. At the end of the programme period the company must submit its final report in which the impact of all electricity efficiency improvement measures has to be assessed. Each company must achieve an improvement in electricity efficiency which broadly speaking is equivalent to the improvement that would have been achieved if the tax (i.e., 0.5 Euro/MW h)Footnote 7 had been imposed instead of PFE (§ 11 of SFS 2004:1196). As a consequence, the programme builds on the theory that the attention-raising effect of its components (e.g., the EnMS and the routines) will offset the impacts of the removed tax on electricity.

PFE can be classified as a medium-term VA in which companies are incentivised by the tax rebate to enter the programme, fulfil its binding obligations and thereby improve energy efficiency.Footnote 8 Hence, in the taxonomy of VAs suggested by Price (2005), PFE fits into the category of programmes that are implemented in conjunction with existing energy/GHG emissions tax policy or with strict regulations. The agreement is signed and entered by individual companies comprising one or several production sites. The programme is regulated by a law (SFS 2004:1196) that defines the binding commitments of all parties and leaves little room for negotiation. Divergence from regulations needs to be reported and tested. The SEA reviews the companies’ reports and occasionally conducts site visits to monitor compliance. Companies are liable to the regulations and the threat in case of non-compliance is that participation is terminated and the minimum tax is repaid for the entire period. In its structure and details of procedures PFE is rather advanced and goes beyond being a gentlemen’s agreement. It is also rich on programme components. Rather than being one policy instrument, PFE provides a packaged mix of instruments, which also complicates the task of evaluation.

Process evaluation

Methodological remarks

This analysis of PFE takes inspiration from the process-oriented approach of Theory-Based Evaluation (TBE) (Weiss 1972; Chen and Rossi 1983; and others). Theory should be understood as “the set of beliefs and assumptions that undergird program activities” (Weiss 1997, p. 503). Thus, the policy theory constitutes the basis for how programme activities are expected to bring about the desired changes. Advocates of TBE claim it is superior to a conventional impact evaluation in that it can answer to not only if, but also why, targeted impacts are achieved. For energy efficiency policies the impact will typically be defined as the quantified energy efficiency improvement and/or energy savings. In case this impact is found uncertain or insufficient the TBE should obtain answers about where in the chain of activities that the policy programme failed to function as expected. If, on the other hand, prior expectations are met or even exceeded a TBE should pinpoint the activities that explains the success. Impact can arise also by other paths than those presumed by the policy theory and to estimate additionality is an important, yet difficult, task of evaluation.Footnote 9 TBE can contribute in solving this issue through its system analytical procedure of assessing the programme by: separating its components; examine these; and communicate the interpretations. Eventually, the evaluation may fulfil its virtue of supporting programme administrators in determining what, if any, modifications that are needed for a forthcoming effective operation.

The research method applied to understand PFE and its policy theory includes in-depth studies of official policy documents and previous evaluations. These have been conducted over a longer period of time, almost 2 years, and have evolved with the programme period as new records about its results have become available. Also, at several occasions, to make details clearer, conversations have been held with staff at the SEA. PFE is pushing for rather multifaceted changes (e.g., technology, actor and market related) but delimitations are necessary to avoid the myth of an all-purpose evaluation (Weiss 1972). The process perspective here given can only give attention to a few aspects and it serves foremost as a supplement to the impact evaluation. In the following we will analyze two areas of essential importance for understanding—and evaluating—the effectiveness and cost-effectiveness of PFE: (1) the eligibility and coverage of PFE and (2) the programme goals and achievement.

Eligibility and programme coverage

All energy-intensive industrial companies are eligible for PFE (§ 4 of SFS 2004:1196), which is about 1,250 companies (SEA 2005). In March 2007, 95 companies, comprising some 250 industrial sites, had submitted their second-year reports (SEA 2007a). Since then, another ten to 20 companies have entered, while a few have left the programme (SEA personal communication 2009a, b, c). Hence, less than 10% of eligible companies are participating in PFE. In Fig. 1, the numbers of eligible and actually participating companies are grouped by the size of their electricity use. Notably, the participation rate decline sharply for companies with an electricity use below 100 GW h/year. Since the size of the tax cut is proportional to the electricity use large consumers (>100 GW h/year) are highly motivated to attend.Footnote 10

Fig. 1
figure 1

Eligible and participating companies categorized by electricity consumption. Source: SEA (2005, 2007a)

Full size image

A result of the self-selection mechanism is that only 3% of eligible companies with lower electricity demand (<100 GW h/year) have joined PFE. This shows that a tax relief below 50,000 Euro is often considered insufficient as a motivator and to cover for the compliance cost related to, for example, EnMS implementation and certification (Sjögren et al. 2007). The low overall participation rate may appear disappointing and staff at the SEA has stated they would like to see as many companies as possible in the programme (SEA personal communication 2008). In the early planning phase of PFE, the intended target group comprised 60 companies from the conventional energy-intensive sectors: pulp and paper; mining; iron and steel; non-metal minerals; and industrial chemicals (Ds 2001:65 2001). However, the previously mentioned criteria of energy-intensive business (ETD Article 17 2003) implicated that the much greater number of 1250 companies became eligible (including e.g., saw mills, food industry, engineering companies). With this distinction being made, it is clear that while participation rate is low compared to eligible companies, it actually exceeds the intended target group of the policy theory.

In terms of electricity consumption, the 10% participating companies account for 85% of the eligible electricity demand (i.e., 30 out of 35 TW h). Hence, in absolute numbers, PFE comprise most of the saving potential. It is possible though, as pointed out by Henriksson and Söderholm (2009), that PFE components like EnMS could do a better job in detecting cost-effective energy efficiency improvement measures among the 1150 non-participating companies, since these can be assumed to lack prior experience in energy management compared to the really energy-intensive companies. In practice, the administrative cost of having a full scale EnMS constitutes a barrier for these companies often being SMEs with lower energy cost shares (Sjögren et al. 2007). From a programme administration standpoint, the burden would increase multi-fold if all 1,250 eligible companies were to join. In this sense, the tax incentive has successfully attracted the lion’s share of eligible electricity use, and thereby potential savings, while the total administrative burden has been kept at moderate level.

Goals and achievement

The evaluation of programme effectiveness can be carried out from the two perspectives of goal achievement and additionality (Krarup and Ramesohl 2000). While the former is discussed in the following, the latter is being analyzed in relation to the impact evaluation of Impact evaluation section. The existence of goals is essential for assessing goal achievement. It is problematic in this regard that policy instruments often lack quantitative targets and are governed by multiple but unclear objectives (AID-EE 2007). Evaluators may find it necessary to make independent interpretations and formulations of goals on the basis of the policy theory (Weiss 1972).

PFE does contain many requirements: the companies have to perform energy auditing and analysis, implement and certify their EnMS and adopt routines for energy efficient procurement and project planning. PFE has also proven successful considering that nearly 100% of the companies have complied with these obligations. Regarding the electricity savings impact no quantified target has been formulated.Footnote 11 There is, however, the requirement mentioned in The PFE outline section that: companies must submit a list of measures and later implement these so to reach electricity savings of the same level that would have been achieved if the minimum tax were to be applied over the same period. This counterfactual reference situation stems from the paragraph of the ETD stating what level of savings that an agreement in substitution for the tax must achieve (ETD Article 17(4) 2003). The potential saving impact from a fictive tax has not been estimated by the SEA or any other authority. The course taken by this paper is therefore to quantify the counterfactual situation, i.e., to estimate the impact in terms of electricity savings due to a fictive tax and frame this as the level of savings that PFE ought to achieve (i.e., the programme’s impact goal).

In the second- and fifth-year reports, the companies were asked whether their electricity savings could match the counterfactual situation. Different justifications were made and indeed there is no clear-cut answer to this question. A majority of companies, however, made the interpretation that cost reductions from electricity saving measures should at least equal and thus eliminate the cost-raising effect of a fictive tax (i.e., had it existed). Theoretically, this implies that the companies must report and implement electricity saving measures as if they were facing a unitary (−1.00) own price elasticity of demand for electricity. To exemplify: a tax of 0.05 Euro cent on an electricity price of 4 Euro cent per kW h results in a 1.25% cost-raising effect and thus a PFE company with this electricity price should in substitution for the tax achieve electricity savings and cost reductions of 1.25%.

On the basis of official policy documents (ETD 2003; SFS 2004:1196) and our examination of the companies’ goal interpretations, the PFE impact goal is here being formulated as a target-level. Figure 2a and b plots, for 36 of the PFE companies, reported electricity savings against the cost-raising effect of the fictive tax.Footnote 12 The latter is decided by each company’s electricity price, varying with power contract and exposure to the Nord Pool spot market. The target-level is decided by the simple relationship that a 1% cost-raising effect must be eliminated by a 1% electricity savings and cost reduction. While Fig. 2a contains the ex-ante deemed savings from measures that were reported in the second year, Fig. 2b contains more reliable ex-post data where the planned and additional measures have been implemented and estimated by the means of measurement or engineering calculations. In both cases, the percentage of annual electricity savings is compared to a base year, represented by the companies’ electricity demand in 2004.

Fig. 2
figure 2

a Ex-ante deemed electricity savings from listed measures of the second-year report, as compared to the cost-raising effect of a fictive minimum tax. Source: SEA (2007a). b Ex-post measured or engineering estimated electricity savings from listed and additional measures of the fifth-year report, as compared to the cost-raising effect of a fictive minimum tax. Source: SEA unpublished data (2010)

Full size image

In the second-year report the planned measures for ten of the companies were inferior to reach the target-level. In the final report, however, many companies showed a significantly improved performance. Average electricity savings of the 36 companies increased from 3% to 5.1%, and some companies were even reporting savings in the high range of 15–20% (see Fig. 2b). In the next section, which evaluates the PFE impact, explanations are provided for this increased performance after the second-year report.

The target-level can also be expressed as a quantitative electricity saving target for the entire group of PFE companies. Given their yearly electricity demand of about 30 TW h, the tax exemption for all companies is 15 MEuro per year. To equal out the cost-raising effect, given the tax had existed, the programme should achieve 375 GW h of annual electricity savings (equal to 1.25%) when assuming an average electricity price of 4 Eurocent per kW h.Footnote 13 It is made evident by the impact evaluation in the next section that the reported electricity savings are well above this level. The PFE collective of companies has thus successfully fulfilled and surpassed the impact goal, as it is defined here. A few individual companies, however, did not the reach the targeted saving level (as illustrated in Fig. 2b) and stand the risk of being excluded from the programme. The lists of planned measures in the second-year report and implemented measures in the fifth-year report should reflect their level of ambition. These can be examined to determine if companies are reluctant of listing measures, although saving opportunities with short payback periods (<3 years) can be assumed to exist. Can such behaviour be justified within the framework of PFE? These are relevant issues in supervision.

Impact evaluation

Methodological remarks

The impact of an energy efficiency policy is the quantified energy efficiency improvement and/or energy savings expressed in whatever metric is appropriate. Several extensive guidelines for energy efficiency policy evaluation have been developed to support evaluators in quantifying impact (EMEEES 2009; IEA DSM 2005; SRCI 2001). It is foremost the guidelines for bottom-up calculations presented by the EMEEES project that have shaped the impact evaluation in this section. A distinction is made between gross and net impact. Gross impact refers to the quantification of all energy savings or efficiency improvements that are documented under the policy framework, without considering that other driving forces could have caused part of the impact. With a gross-to-net impact conversion, the evaluator seeks to solve the additionality issue by raising the question: How large energy savings would not have been achieved if the policy programme had not existed? Consequently, the net impact or additional savings refers to the quantified impact induced by the programme per se, not resulting from autonomous changes that would have taken place also without the programme. This has a clear relevance in the striving for effective and cost-effective energy efficiency policies.

Gross impact

This presentation of gross impact is aligned with the PFE reporting procedure that gathers data both from the second-year interim report and the fifth-year final report. While the former data set contains the ex-ante deemed savings from the reported lists of planned measures, the latter report concludes PFE with its ex-post estimates from all programme components (e.g., the adoption of routines and changes in operation and maintenance (O&M) as stipulated by the EnMS). The fifth-year report thus captures the total gross annual electricity savings that the companies have reported to the SEA under the PFE scheme. Table 2 compiles the technical and O&M measures from the two checkpoints. In the beginning of 2007, 95 companies had submitted their second-year reports comprising 860 measures equalling ex-ante deemed annual savings of 726 GW h. In 2010, when 101 companies had submitted their fifth-year final reports the number of measures had increased to 1,254 and the ex-post estimate of annual savings was 917 GW h; an increase by 191 GW h (or 26%) compared to the second-year data.Footnote 14 The measures have been subdivided into types of end-use technologies. A large part is motor-related, e.g., variable speed drive (VSD) installations. Measures that relate to pumping systems are also common due to the large participation of the pulp and paper industry that uses pumping equipment throughout the mills.

Table 2 Reported annual electricity savings from technical and O&M measures
Full size table

In addition to all the technical and O&M measures presented in Table 2, a few more categories of electricity savings measures have been reported by the companies. These are (SEA 2011a):

  • Electricity savings due to the energy efficient routines: 174 GW h/year from project planning and 36 GW h/year from procurement practices

  • Supplementary electricity saving measures: 323 GW h/year from a number of measures with large impact that for various reasons have been categorised separately

By summing up all these measures, SEA concludes that the total gross annual impact of PFE is 1,450 GW h (SEA 2011b). The result is quite remarkable considering the target-level of 375 GW h/year that was formulated in the previous section. It even exceeds the previous high expectations of 1,000 GW h annual electricity savings based on the second-year data (SEA 2007b). By comparing data from the two checkpoints, we can better understand the PFE process and how it corresponds to policy theory. The measures reported in the second-year report can be seen as a response to the legal requirement of conducting energy audit and analysis to identify profitable measures for implementation. The elevated, actually doubled, impact thereafter indicates that the companies did not stagnate in their efforts to implement measures. It also shows their willingness, with a few exceptions, to report more measures than the law requires. There are, at least, three complementing explanations for their behaviour.

The energy management system

The policy theory suggests that PFE components like the EnMS, including the routines, create an attention-raising effect that will offset the impacts of the removed tax on electricity. Since these programme components were fully introduced first after the second year, it is reasonable that the elevated impact of about 700 GW h is observed in the later years of the programme period. The companies have estimated the impact from the two categories of routines to be 210 GW h/year. The remaining 500 GW h of reported annual electricity savings are partly from technical and O&M measures implemented under the guidance of certified EnMSs. It is difficult to verify a direct causality between the EnMS and reported savings for each company and measure. In general, though, companies claim that the EnMS has helped establishing an organisational structure with a strong focus on energy efficiency. As many as 80% of the companies claim that the EnMS has introduced new methods for monitoring energy use that have been valuable for their energy efficiency improvements (Hörnsten and Selberg 2007).

A legally binding agreement

In a review of VAs worldwide Price (2005) concludes that the most effective ones are legally binding. The PFE legislation (SFS 2004:1196) should therefore be seen as a strong motive for compliance. As an example, the companies have taken it serious that the listed electricity saving measures must be implemented over the programme period. This requirement has facilitated the allocation of investment capital for PFE measures. It has also led some companies to be careful not to list measures that they were not sure about in the second year (Hörnsten and Selberg 2007). Thereafter, as measures have been analysed in detail and investment funds have been secured, companies have taken decision to implement additional measures.

Electricity price development

Variable but on the whole increasing electricity prices and the perceptions held by industry that future electricity prices will remain at high levels are fundamental reasons for energy-intensive companies to improve electricity efficiency. Thollander and Ottosson (2008) show that “cost reductions resulting from lower energy use” and “the threat of rising energy prices” is ranked as the first and fourth most important among 23 driving forces for energy efficiency improvement in the Swedish PPI.

Acknowledging the strong underlying motive of energy cost reductions in the energy-intensive firms the total gross impact cannot be attributed to PFE exclusively. The programme does, however, emphasize electricity efficiency rather than other factors of production as a prerequisite for industrial competitiveness. It also make requirements and provide tools for companies to overcome commonly cited barriers like “the lack of access to capital” and “the lack of time or other priorities” (Thollander and Ottosson 2008). It is therefore likely that PFE has realised energy savings that have been overlooked before. The following section will examine the importance of PFE by estimating its attributable share of reported savings, i.e., the net impact of PFE.

Net impact

In a bottom-up evaluation method the gross-to-net impact conversion is done by adjusting total gross annual impact with a number of correction factors as expressed in Fig. 3 and further explained below.

Fig. 3
figure 3

Bottom-up calculation for a gross-to-net impact conversion. Source: EMEEES (2009)

Full size image

Owing to the PFE documentation and reporting procedures, it has been possible for the SEA to estimate a total gross annual impact of 1,450 GW h. To complete the equation the correction factors needs to be determined:

  • Free-rider coefficient: expresses the share of savings, ranging between 0 and 1, that would have been implemented also without the support from the policy programme.

  • Multiplier coefficient (also called spill-over): expresses the savings that are indirectly caused by the programme in addition to what was targeted. Both participant and non-participants can implement measures without involvement (e.g., financial, technical or informative support) from the programme administration. The possible range is from 0 to in principle very large numbers.

  • Double counting coefficient: expresses the potential effect from overlap and whether savings have to be shared between different policies and/or saving measures. The range is between 0 and 1, where 1 represents a situation without shared savings.

In an intermediate evaluation of PFE, an attempt is made to estimate the coefficients (Stenqvist and Nilsson 2009). Some factors of influence are discussed in a qualitative manner but the authors surrender the issue of quantifying all coefficients. Though there is still a lack of surveys, some new knowledge is added in the following which allows for a revision of previous estimates.

Free-rider

In the fifth-year report, the companies have answered for each measure belonging to the category of technical and O&M measures how it was identified. Of the 917 GW h/year of electricity savings it is claimed that: 43% was identified during the energy audit; 32% was known from before; and the remaining 25% was identified by other means. Hence, without the PFE requirement on energy auditing and analysis the identification and implementation of 43% of these savings would not come about, or at least have been deferred. It is also assumed that the increased awareness and energy efficiency focus resulting from a certified EnMS can explain most of the savings that were identified by other means. Likewise, the adoption of energy efficient routines is considered to be a direct result of the PFE participation. On the other hand there are technical and O&M measures that were known already from before and these would probably have been implemented also without PFE. The last category of supplementary electricity saving measures involves rather large-scale upgrading of production processes that to a large extent is taken to be autonomous improvements.

With this reasoning, it is found that somewhere around 40% of the electricity savings can be free-rider savings and to express the uncertainty involved here the free-rider coefficient is estimated to be in the range [0.3,0.5].

Multiplier

Energy efficient choices made by PFE companies may have a transforming effect on the market if suppliers change their offers to stimulate non-participants in favouring energy efficient solutions. Market transformation studies (e.g., sales data analysis) are often recommended to identify such a multiplier effect (EMEEES 2009; IEA DSM 2005; SRCI 2001). In one survey, equipment manufacturers and retailers were asked about how PFE influenced their business in terms of awareness, demand, offers, sales, etc. (SEA unpublished report 2007a, b). The results give interesting, but ambiguous, insight to the business. A market structure with a lot of middlemen is revealed. Motor manufacturers, for example, could observe an increasing demand on energy efficient motors but since the end-users are unknown to them so are the reasons for the change in demand. Motor-related measures are common but PFE is technology-neutral and it is evident in Table 2 that the 1,254 measures are diverse. This complicates the focus of a market transformation study compared to a policy targeting a specific consumer product like cold appliances. A survey has to be well planned to target the relevant market actors and, more than impressions, collect actual sales data on the most important technologies. High efficiency motors and variable speed drive installations are measures standing out in terms of increased demand over the last years (SEA unpublished report 2007a, b). A survey would be needed to confirm the role of PFE and estimate the size of a possible multiplier effect behind a market transformation.

During the 5-year programme period, PFE companies, mostly in the pulp and paper industry, have increased their levels of auto-produced electricity with 15% (SEA 2009c). This development has been driven by the scheme of tradable renewable electricity certificates along with the price levels of electricity. It has little to do with PFE, however, and since this involves supply side measures it is not a case for multiplier effects. It is possible, anyhow, that EnMS practices have supported project planning for reaching optimised solution with regards to the back pressure turbines and the demand for steam. In such cases, whenever electricity savings has been an outcome, this can be reported as a result from routines for energy efficient project planning. There are some examples of such company reporting (SEA 2011a).

A multiplier effect can be derived from PFE participants that apart from electricity savings have made heat and fuels savings. Due to the tax incentive PFE only account electricity savings as programme impact. It can be argued, however, that the EnMS stimulates a general energy efficiency improvement concerning all energy carriers (SEA 2008). Case in point: in the fifth-year report 75% of the companies voluntarily reported measures related to other energy carriers than electricity (SEA 2011a). It is not possible, due to the diversity of measures and complexities involved (e.g., some measures are fuel shift rather than energy saving measures), to derive an impact for the complete data set. It is possible, however, to separate a subset of measures that are reported as implemented and for which the energy savings are clearly quantified (SEA 2011c). These measures represent heat and fuel savings in the size of 950 GW h/year. Taking this to be a potential result from having a certified EnMS, the multiplier coefficient can be somewhere in the range [0,0.65].

Double counting

There is no other policy instrument with influence on the PFE companies that, like PFE, specifically requires electricity saving measures to be identified, implemented, monitored and reported. The documentation and reporting procedure includes information about the companies, their sites and their implemented measures. This information can be cross-checked to avoid double-counting in case attempts are made to evaluate electricity savings from partly overlapping policies. For example, the promotion of energy efficient products through the EU Ecodesign Directive represents a support for industrial energy efficiency improvement. This policy interaction will probably be enhanced in the future as the revised Ecodesign and its respective requirements are being implemented. In a bottom-up accounting of national energy savings, the concerned agencies will have to deal with the double counting issue by allocating savings only to one preferred policy. In the case of the Swedish Environmental Code that can be used to mandate energy efficiency improvement (Johansson et al. 2007), the potential problem of overlapping regulation has been solved by a decree. This implies that PFE companies are considered to fulfill any requirement on energy conservation stipulated by the Code (SEA 2008). Thereby, the risk for double counting is reduced since the PFE companies only need to report their measures to one agency (i.e., the SEA).

The documentation and reporting procedures applied in PFE also helps to avoid double counting between categories of measures. The SEA explains that the implementation of technical and O&M measures should be an outcome of energy auditing and continuous energy management, that the procurement routines concern equipment that is acquired repeatedly, and that the routines for project planning involves larger scale restructuring (SEA 2006). Some companies have claimed it difficult to make a separation, but in the reporting they are required to allocate each measure to one or another category and state the method for verification. Adequate actions have been taken to reduce the risk of double counting, but because of the different measures involved (i.e., routines as well as technical measures) the occurrence cannot be fully excluded. We estimate that the double counting coefficient lies in the interval [0.95,1].

When the estimated correction factors are used in the formula of Fig. 3, the result is a net annual electricity saving impact between 689 and 1,015 GW h. In addition to this, PFE has caused a potential multiplier effect of up to 950 GW h/year of heat and fuel savings.Footnote 15

Clarifications on gross and net impact

The preceding sections have estimated the PFE electricity savings in terms of gross and net annual impact, yet some clarifications can help to avoid misunderstandings about the meaning of these results. The gross annual impact of 1,450 GW h does not, by necessity, imply that the participants have decreased their absolute electricity use by that amount. On the contrary, the actual electricity demand of the PFE companies was almost 3% higher in 2008 compared to 2004 (SEA 2007a, 2011a). This is because plant throughput and other conditions typically fluctuate from year to year. Due to the latest economic downturn, for example, overall production level and industrial electricity use was much lower in 2009 than average over the last couple of years. Therefore it is more accurate to interpret impact as avoided demand rather than absolute savings.

To be precise, gross annual impact should only be understood as the estimated savings from the specific measures compared to the base situation of the prior operation. Since measures have been implemented over the whole 5-year period and measurement and verifications are done by each company, the total gross annual impact relies on a range of different base situations. To claim an aggregated PFE gross impact, a practical approximation is to consider the year before the programme started (i.e., 2004) as the common base year for all measures. Figure 4 shows the estimated impact from 101 companies as the percentage of avoided electricity use compared to the baseline representing the companies’ electricity demand in 2004, which was 30 TW h.

Fig. 4
figure 4

Avoided electricity consumption from PFE measures compared to the 2004 baseline. Source: SEA (2007a, 2011a)

Full size image

While the gross annual impact of 1,450 GW h equals almost 4.8% savings, the net annual impact interval 689–1,015 GW h correspond to 2.3–3.4% savings. The target-level of 1.25% is clearly exceeded by the collective PFE savings. Another ten companies can be expected to submit their final report for the first 5-year period. Since these are companies with lower electricity consumption they can only make a marginal contribution to the final programme impact.

The cost-effectiveness of PFE

The term cost-effectiveness in relation to energy efficiency policies can have dual meanings: the ratio of the cost per saved amount of energy (e.g., Euro/MW h) or; whether the energy efficiency improvement measures take place where they are the least expensive. Here the cost-effectiveness ratio for the PFE electricity savings is assessed from the perspective of society which includes programme costs for government and for end-users/companies. The main costs involved are:

  • Companies’ investments into electricity saving measures (2005–2009): 70.8 MEuro (SEA 2011a)

  • Companies’ administrative costs to comply with PFE regulations (2005–2009): 13 MEuro (NUTEK 2008)Footnote 16

  • SEA’s costs for administrating PFE (2004–2009): 4.2 MEuro (SEA personal communication 2008)

From the perspective of society, the annual tax rebate of 15 MEuro is not considered a cost but a relocation of capital between government and end-users. The total cost for society is thus 88 MEuro between 2004 and 2009. The discount rate is set at 4%. A depreciation period of 12 years is selected based on simplifying assumptions about the persistence of energy savings. According to recommendations on measurement and verification methods in the framework of ESD many industrial energy efficiency improvement measures are assigned default saving lifetimes of 15 years.Footnote 17 The category efficient electric motors and variable speed drives, however, has a default saving lifetime of 12 years. Moreover, savings that arise from good energy management and monitoring have default saving lifetimes of 5 years (EC 2010b). Table 2 shows that the PFE impact is resulting from a variety of measures but that motor-related and VSD installations have been especially important. Also, the EnMS-related O&M measures and the routines applied are essential to PFE. It is a reasonable approximation to use a uniform 12-year saving lifetime and depreciation period to all measures. Thereby, the annualised cost for society is 9.4 MEuro. The gross annual impact implies a unit cost of 6.5 Euro/MW h. Based on the net annual impact interval (i.e., 689–1,015 GW h) the unit cost is 9.3–13.6 Euro per MW h of saved electricity, depending foremost on the presence of free-riders. This cost calculation compare very favourable to yearly average wholesale electricity prices which have been between 29 and 51 Euro/MW h in the period 2005–2009 (Nord Pool Spot AS 2011). Comparison can also be made with the cost for electricity production from new generation capacity which depends largely on the power technology and its related fees and subsidies. A study based on Swedish conditions, using a 6% discount rate and a 20-year depreciation period for commercial energy technologies, derives results between 16 and 110 Euro per unit of produced MW h (Hansson et al. 2007).

The second perspective on cost-effectiveness also deserves some attention. The companies have made significant investments and still the average payback period of all measures is less than 1.5 years (SEA 2011b). In interviews, companies have declared that PFE activities have raised their competence level in energy management for energy efficiency and that the EnMS has made them question their former routines (SEA 2009c). With a macroeconomic perspective this could indicate an occurrence of firm-specific information asymmetries prior to PFE. Based on this perspective it can be discussed whether addressing the measures has been overall economically efficient in the sense that the measures have been implemented where they are least expensive (see Henriksson and Söderholm 2009). Theory on information asymmetries implies that electricity taxes could do a better job in companies with a high electricity cost share while EnMSs and other attention-raising activities could be more effective in companies with a lower electricity cost share, since the latter are relatively less experienced with energy efficiency improvement (Ibid.). Given that there is a positive correlation between electricity demand and electricity cost share, Eligibility and programme coverage section has shown that PFE induces a reverse situation. The largest electricity consumers are eager to join and substitute the tax with the EnMS (and the other programme components) while companies using less electricity are typically facing the tax. The programme results partly support the theory. Some 40 companies with lower electricity demand (<100 GW h/year) did join PFE and their average gross annual impact is 9%. For those, about 60, companies with higher consumption (>100 GW h/year) the corresponding figure is 4%. A review of specific sectors shows that manufacturers of food products and beverages (NACE 15) have found the energy auditing and analysis to be most useful (Hörnsten and Selberg 2007). The sector as a whole reported gross electricity savings of 5.3%. Another less energy-intensive sector, the manufacturers of wood and wood products (NACE 20), reported gross electricity savings of 6.7%. This can be compared with the more energy-intensive sectors (i.e., NACE 21, 24, 25, 27 and 28) each of which reported gross electricity savings close to 3%. This indicates that the most energy-intensive companies, though they provide the large bulk of total savings, are less responsive to PFE and EnMS in terms of their reported percentage savings. Still, as demonstrated in Goals and achievement section, the 3% electricity savings exceeds the estimated impact of a minimum tax at 0.5 Euro/MW h. Consequently, PFE can do a good job in promoting electricity savings in all the concerned sectors.

Discussion and remarks on policy implications

This ex-post evaluation shows that the PFE gross annual impact (i.e., 1,450 GW h) as well as the interval of net annual impact (i.e., 689–1,015 GW h) greatly exceeds the estimated annual impact of a minimum tax (375 GW h) which is interpreted here as the programme’s target-level. The evaluation also shows a cost-effectiveness ratio with a relatively low cost per saved amount of energy. Moreover, PFE has caused a multiplier effect of heat and fuel savings that can be as large as 950 GW h/year. On these merits PFE can be judged successful against its objectives to improve industrial electricity efficiency while safeguarding industrial competitiveness. EnMS procedures have been a key to the successful outcome, as indicated by the elevated programme impact after the second year. This result is promising and well-timed given the opportunity for worldwide EnMS implementation according to the international ISO 50001 standard being published in 2011. Stimulating industrial EnMS practices as a main ingredient of a VA can be a viable and cost-effective policy solution. All in all the continuing 5-year period of PFE is justified. It can be assumed, however, given the short average payback period of less than 1.5 years that many “low hanging fruit” measures were harvested during the first period. If significant energy savings are to be realised also in the future, the companies need to make continuous improvements as prescribed by the EnMS standard.

There is no officially declared PFE impact target. Hence, in order to evaluate the programme effectiveness the authors had to interpret and formulate a target-level. This assessment is complicated by the difficulty to envisage the impact on electricity efficiency improvement from a non-existing counterfactual situation. Even among non-participating companies, that are facing the minimum tax, it would be difficult to determine its influence on electricity consumption. The cost-raising effect is negligible in view of the increase in wholesale electricity price observed over the last 5–10 years. On the other hand, the tax rebate has been an important carrot incentive for companies to join PFE and undertake the agreed activities. This is evident among companies with annual electricity consumption above 100 GW h for which the programme coverage has been close to total.

Below, the 100 GW h/year threshold PFE is gradually becoming less attractive. As non-participating firms only account for 15% of eligible industrial electricity consumption the collective ineffectiveness from the missed-out savings should not be too large. Nevertheless, it is especially companies with lower electricity consumption (<100 GW h/year) that have reported large percentage savings. This shows that less energy-intensive industries have a lot to gain from the attention-raising activities that are typical for PFE and some other VAs (e.g., energy auditing and analysis, EnMS and routines). Hence, there should be a large potential also for non-participating companies to engage in the kind of energy management that has proved successful within all industrial sectors of PFE. It remains a major challenge for policy makers as well as commercial players (e.g., ESCOs) how to best stimulate these companies, with relatively high saving potentials, to make energy efficiency improvements. As an alternative policy instrument an industrial energy audit programme for SMEs was launched in 2010. It offers subsidized energy audits, requires companies to set up an energy plan and finally to report their implementation of measures (STEMFS 2010:2). Ex-ante estimates are expectant (Thollander and Dotzauer 2010) but since the scheme will remain for 5 years it is too early to evaluate its effectiveness.

Supported by the PFE documentation and reporting procedures, this paper used a bottom-up approach to evaluate gross and net impact. Another option would be a top-down approach to examine how the industrial electricity intensity has developed prior to and during the PFE period, then single out the actual electricity efficiency improvement and attribute an appropriate share to PFE. Given the heterogeneity between and within industrial sectors this methodology might not be feasible. At least it would require additional reporting by the companies and further data analysis by the administrating agency, which would increase administrative burden. Currently, the cost carried by the SEA for administration, which only partly goes to monitoring and evaluation (M&E), has equalled 5% of the total programme cost (see The cost-effectiveness of PFE section). Given that cost for M&E should be kept at reasonable level, the feasibility of such an evaluation effort would need to be more closely examined.

In addition, a bottom-up methodology needs to compromise between accuracy and evaluation cost. It is expressed by the net impact interval that the gross-to-net impact conversion suffers from uncertainties. Additional surveys could serve to improve the accuracy of the free-rider coefficient and provide better evidence for estimating a multiplier effect. Ideally, an evaluation plan is developed in the early phase of planning and formulating a policy instrument. In PFE such a plan could have served to integrate all necessary data into the overall documentation and reporting procedure, still with respect to the administrative burden shouldered by companies. In this way, relevant correction factors and other details would be given systematic attention and the need for supplementary surveys or making guesstimates could be avoided. An evaluation plan can also identify the forthcoming energy efficiency targets that the policy instrument should contribute to. The work of the EMEEES project on EU harmonised evaluation methodologies deserves some attention in this regards. To better pinpoint additionality is not only an issue about accountability against set targets. It also has a practical importance in strive for cost-effective policy implementation. The ESD does not explicitly require that only additional energy savings are counted against the 9% target of 2016. It does mention, though, that evaluation methods should be cost-effective and minimise administrative burden while reaching an acceptable level of accuracy. These considerations, i.e., reaching a cost-effective policy impact that can be monitored and evaluated by practical methods, is highly relevant also in view of the EU target of saving 20% of the projected primary energy demand in 2020.

With its process perspective, this evaluation has strived to go beyond the focus of quantifiable impact. Activities like the energy audit and analysis, the certified EnMS, the routines for energy efficient procurement and project planning, would unlikely come about in the absence of PFE. Neither would companies have been incentivised to document and report savings and thereby improve their practices for monitoring and verification. These are programme components with capacity to alter organisational structures around energy issues in the companies and at their industrial sites. PFE has been an impetus for such organisational changes that, in turn, can have long-lasting effect in terms of energy efficiency improvement.