Social Cost Efficient Service Quality: Integrating Customer Valuation in Incentive Regulation—Evidence from the Case of Norway

Abstract

In order to overcome the perverse incentives of excessive maintenance reductions and insufficient network investments arising with incentive regulation of electricity distribution companies, regulators throughout Europe have started regulating service quality. In this paper, we explore the impact of incorporating customers’ willingness-to-pay for service quality in benchmarking models on cost efficiency of distribution networks. Therefore, we examine the case of Norway, which features this approach to service quality regulation. We use the data envelopment analysis technique to analyse the effectiveness of such regulatory instruments. Moreover, we discuss the extent to which this indirect regulatory instrument motivates a socially desired service quality level. The results indicate that internalising external or social cost of service quality does not seem to have played an important role in improving cost efficiency in Norwegian distribution utilities.

4.1 Introduction

The transition from cost-plus to incentive regulation of natural monopoly energy networks entails numerous new challenges for regulators and network operators. In principle, the objective of incentive regulation is to encourage network operators to improve their cost efficiency towards a given target and to reward them for over-performance and penalise them for under-performance. The underlying parameter is a regulatory formula that caps the allowed prices (price cap regulation) or the allowed revenues (revenue cap regulation) of a network operator. This stimulus may, however, create perverse incentives with regard to the level of supplied service quality. The network operator may focus solely on efficiency targets to the detriment of maintaining an adequate level of quality. Therefore, service quality regulation is being introduced in a growing number of countries.

Quality in the electricity distribution and retail sector is a multi-faceted output that comprises technical and non-technical dimensions. The aspects that are usually regulated span three main areas: commercial quality, voltage quality, and continuity of supply and/or reliability (CEER 2008). The generic terms for these three dimensions are “service quality ” and “service quality regulation”, respectively. In this paper we focus on the aspect of continuity of supply in distribution networks, which is arguably the most important and widely targeted dimension of service quality by regulators (see e.g., CEER 2008). The aspect of commercial quality in the retail sector as well as the technical issue of voltage quality is not part of our study.

From a regulatory point of view, continuity of supply appears in two dimensions: the first dimension is its availability to energy to customers (or inversely the absence of interruptions). Basically, this dimension can be measured by different (groups of) indicators,¹ either the customer minutes lost (e.g., in form of the SAIDI²), the number of interruptions (e.g., in form of the SAIFI³) or the energy not supplied (ENS) , which gives the total amount of energy that would have been supplied to a customer if there would not have been any interruption . The second regulatory dimension is the customers’ preference for continuity of supply. One form to measure customer preferences is to reveal their willingness-to-pay (WTP) for a certain service quality level, or for its inverse, i.e., the interruptions cost (IC) customers incur due to poor quality (Fumagalli et al. 2007).

With regard to the latter dimension, incentive based penalty and reward schemes for continuity of supply performance prove to be a sophisticated regulatory instrument to excite the regulated company to deliver a desired service quality level to its customers.⁴ More specifically, these schemes adjust the companies’ revenues according to their performance against a predefined service quality indicator, e.g., the ENS⁵ or the SAIDI combined with the customer’s WTP.⁶ For the company, higher quality levels are associated with higher revenues and vice-versa.

Whilst the UK, the Scandinavian countries, the Netherlands, Italy and Portugal can be considered as pioneers in this field in Europe, more and more other European regulators follow this practise and enhance or introduce regulatory approaches⁷ for incentive based service quality regulation. When introducing such a scheme, the objective usually is to neutralize potential quality deterioration due to incentive regulation. Mostly, a baseline is defined to move towards the desired level of service quality. As a matter of fact, the service quality level in Europe substantially differs among the different countries and regulators employ diverse approaches to measure it. Overall, there are rather decreasing trends for customer minutes lost and the trends indicate a rapprochement of the continuity level in Europe (CEER 2008).

From a national perspective, however, the main regulatory challenge is to define the accurate reference for the country specific, socially desired level of continuity of supply rather than targeting regulatory measures towards a maximum quality level . Therefore, the regulator needs to know the companies cost of providing service quality as well as the customers’ preferences. Provided with that, the pivotal regulatory objective is to harmonize the utilities’ profit incentive with economic efficiency and customers’ preferences in terms of continuity of supply. In other words, the idea is to internalize the costs of (poor) quality from a customers’ perspective into the profit optimisation calculus of the network operator. With it, the service quality incentive reflects the costs incurred by customers affected by a poor quality level. Thus, the network operator will aim at providing quality up to the level where the marginal cost of quality equals the reward offered and therefore aims at a socially desirable quality level (Growitsch et al. 2009). This economically efficient approach raises the question how network operators actually respond to the introduction of such regulatory instruments in practice. We are particularly interested in the case of Norway since the Norwegian regulator NVE was the first to incorporate customer valuation of service quality in the regulatory scheme and nowadays features a state-of-the-art approach in this context.

Overall, empirical research on the effectiveness of service quality in distribution networks is rather scarce and findings are heterogeneous. Ter-Martirosyan (2003) analyses the impact of incentive regulation on the duration and frequency for electric outages for a panel of 78 US utilities. She finds that incentive regulation is associated with an increase of outages. Moreover, the study detects that the number of outages decreases with the introduction of explicit quality benchmarks. Korhonen and Syrjänen (2003) find an improvement in technical efficiency after introducing a continuity of supply indicator (interruption time per customer) in their Data Envelopment Analysis (DEA) of 106 Finish distribution companies. A report by CEPA (2003) applies a two-step DEA model to cross-sectional data for the UK. They find no significant correlation between technical efficiency measures and continuity of supply in terms of customer minutes lost. Ajodhia et al. (2004) apply a DEA and a Corrected Ordinary Least Square (COLS) model to a cross-sectional sample of 44 electric utilities from the UK, the Netherlands, Hungary, and Malaysia, reporting a significant efficiency increase when quality is taken into account, especially for smaller network operators. Giannakis et al. (2005) carried out a DEA-based quality incorporated efficiency study on UK electricity distribution companies. They show that technical efficiency does not necessarily also involve high service quality. Moreover, they find that quality incorporated regulatory benchmarking is superior to cost-only approaches. Jamasb and Söderberg (2009) analyse the effects of the application of norm models within an ex-post incentive regulation of electricity distribution networks in Sweden. In the examination of the companies cost and service quality performance they find that service quality regulation has not affected the relative performance of utilities.

Some recent studies performed by Edvardsen et al. (2006) and Burger and von Geymueller (2007a, b) specifically examine the efficiency of Norwegian distribution networks. Edvardsen et al. focus on the general productivity of the networks. They find a productivity improvement albeit flattening out as from 2000. They generally explain this decrease with the introduction of new regulatory requirements and a potential retention in efficiency awaiting changing regulatory parameters. Burger and von Geymueller (2007a) find that quality regulation induced Norwegian network operators to optimise their quality strategy from a social point of view based on a DEA analysis and Malmquist indices for the period 1999–2005. However, their sample covers a rather limited number of observations (31 distribution companies), which might involve an uncontrolled sample bias. Indeed, in another paper, the authors find that ENS was reduced more significantly prior to the introduction of quality regulation than afterwards (Burger and von Geymueller 2007b).

Albeit previous empirical research addresses service quality and/or productivity and welfare related issues there is—to our best knowledge—no empirical case study that clearly focuses on the impact of WTP-based continuity of supply regulation on the efficiency of distribution networks.

Thus, the aim of this paper is to shed some empirical light on this issue by assessing whether WTP-based⁸ service quality regulation has a noticeable effect on the social cost efficiency of distribution networks in Norway and to what extent this regulatory instrument motivates a socially desired quality level. We give empirical evidence by means of a Data Envelopment Analysis and associated tests based on a complete dataset of Norwegian utilities, which was prepared by the Norwegian regulator NVE for the purpose of regulatory benchmarking analysis. The results enable us to discuss the effectiveness of service quality regulation based on customers’ WTP and the impact on the quality level in Norway.

The remainder of this paper is as follows: In Sect. 2 we explain the economics of WTP-based service quality regulation. Section 3 describes the Norwegian regulatory approach in terms of service quality regulation and gives empirical evidence. Section 4 concludes and highlights policy implications.

4.2 WTP Based Service Quality Incentives and the Optimal Quality Level

Regulating service quality regulation challenges regulators both methodologically and practically. More specifically, the regulator has difficulties to define and measure precisely the desired service quality level delivered to consumers. One consequence of this lack of verifiability is the difficulty for the regulator to adequately reward or penalise the regulated utility for providing service quality. Therefore, indirect regulatory instruments have to be employed to motivate the desired service quality level (Sappington 2005). Among these instruments arise incentive based continuity of supply schemes based on reliability indicators. The latter are an innovative means to induce regulated companies to deliver the socially desired service quality level.

From a distribution network operator’s perspective, providing quality has a certain cost, both in form of capital expenditures (CAPEX) and operation and maintenance expenditures (OPEX) . These network cost (together: total expenditures, TOTEX ) increase with higher quality levels, marginal improvements being more costly when quality is already high. Likewise, the customers’ benefit increase (or cost associated with poor quality decrease) with a higher quality level. From a social perspective, reflecting the optimisation calculus of both customers and companies, the optimal quality level is where the marginal customer benefit of additional quality equals the marginal cost of supplying it. In other words, the optimal quality level corresponds to the minimum of a total cost function incorporating companies’ cost in providing quality and the costs experienced by customers due to a poor quality level (Fumagalli et al. 2007).

Taking this theoretical setting for a socially optimal quality level into account, the regulator’s task is to include the benefit from service quality (from a customer’s point of view) into the firm’s decision-making process. In general terms, the customers’ preferences for a certain quality level represent their WTP for it. As the quantification of this critical parameter is rather difficult, WTP is usually approximated with its inverse, this is the customers’ interruption cost (IC) due to a poor service quality level (Ajodhia 2006). Service quality incentives for the regulated company are thus calculated by multiplying the respective ENS⁹ with the IC for a certain customer group. More specifically, this constitutes the external cost of energy not supplied (CENS) which are then internalised in the network operators profit maximisation by regulation.¹⁰ Within such an indirect incentive scheme the regulated firm will aim to optimise its trade-off between CENS and TOTEX. These together form the total social cost (SOTEX) of network provision. The more the network operators invest in network reliability to reduce CENS, the higher TOTEX becomes. At some point, the companies will reach the theoretical optimal quality level where the marginal customer and, by regulation, operator’s benefit of additional quality equals the marginal cost of supplying it, in other works where the sum of CENS and TOTEX is lowest (as illustrated in Fig. 4.1). This implies that network operators will only increase quality as long as this leads to a net reduction in SOTEX, or until the marginal costs to provide more quality equal the reduction in CENS incurred by customers (Ajodhia 2006).

Fig. 4.1

Trade-off between CENS and TOTEX

Now, these economic considerations imply the necessity to carefully distinguish between a maximum quality level , which constitutes only a unilateral optimisation from either the companies’ or the customers’ perspective and a socially optimal level for both players. The social optimum, however, is only obtained when SOTEX are taken into account.

This section briefly presented the theory of WTP-based service quality incentives applied for indirect regulatory measures to motivate a socially desirable service quality level. In the following section we practically assess the effectiveness of such regulatory instruments by means of a concrete case study.

4.3 The Norwegian Example

4.3.1 Overview

This section explores and assesses the development path of quality regulation in Norway, one of the pioneering countries in this field. The objective is to further scrutinise the issue of implementing quality incentives based on customer WTP for continuity of supply, and to analyse the impact of such regulatory measures on the efficiency of the Norwegian network operators by means of a concrete case study. After a brief description of Norwegian quality regulation, we analyse the adaptation of the network operators in terms of their improvement in social cost efficiency. Comparing this with the development of private cost (TOTEX) efficiency provides evidence of the effectiveness of quality regulation in Norway.

4.3.2 Quality Regulation in Norway: Development and Status Quo

The first features of quality regulation were introduced after regulatory reform in 1991 by the Norwegian regulator (NVE). In 1995, NVE implemented a standardized reporting system for interruptions and outages called Fault and Supply Interruption and Information Tool (FASIT). As a result, network operators were obliged to report all interruptions and outages longer than 3 min (Brekke 2007). From 1997 on, network operators at 33–420 kV were required to report any incidents, disturbances and system failures. Simultaneously, a revenue cap was introduced yet without any incentive for quality management , thus leading to a tendency towards underinvestment. Likewise standardised methods to compute the ENS per customer category were set up and a reporting system was made mandatory. Eventually in 2001, a quality term based on the CENS was incorporated into the regulatory formula to determine the revenue cap for the second regulatory period (2001–2006).

The main objective of the CENS-arrangement is to give the network owners incentives to plan, operate and maintain their networks in a socio-economic optimal way and thereby provide a socio-economic optimal level of continuity of supply. In more general terms, the objective of service quality regulation in Norway is to obtain a service quality level that is beneficial for the society as a whole. This does not necessarily include the need for a general improvement in the current service quality level, but rather the objective to create a social awareness for service quality defining what the socially desirable actually looks like (Brekke 2007).

As a consequence, the utilities revenue cap was adjusted in accordance with the customers’ interruption cost during the regulatory period 2001–2006. In pursuing this approach all planned and unplanned interruptions longer than 3 min in networks over 1 kV were considered. Based on estimates of expected ENS and average outage costs per customer group, the underlying model annually computes the expected outage costs per network operator. The latter particularly depends on two determinants: the customer group and the type of interruption (planned or unplanned) as illustrated by (4.1):

$$ IC={\displaystyle \sum_{n,m}EN{S}_{n,m}\cdot {c}_{n,m}} $$

(4.1)

where

• IC = Cost of energy not supplied/Outage cost (€)

• ENS = Energy not supplied (kWh)

• c = average specific outage costs

• n = customer group

• m = planned, unplanned interruption¹¹

ENS is defined as the amount of energy that would have been supplied to the customer if there had been no interruption. This amount can be estimated by means of FASIT, which provides a uniform standardised methodology. The average specific outage cost (c) can however be appraised based on customer surveys that have been conducted since 1991 (Langset et al. 2001). Table 4.1 illustrates the respective values per customer group resulting from a nationwide survey conducted in 2002.¹²

Table 4.1 Specific outage costs in the Norwegian CENS arrangement 2003–2006 (€/kWh)

The outage cost in the Norwegian CENS-arrangement comprises both notified and non-notified interruptions, featuring different outage costs for those different types of interruptions .

Network operators are also set individual quality targets . In other words, the outage costs for all customers that are connected to the distribution networks are capped at a specific sum. To this end the expected value for ENS for each network is estimated by means of regression analysis (4.2). This analysis uses parameters such as network structure, number of transformers, geographic and climatic factors. Panel data from previous years provide the historical values for ENS. Consequently, quality targets can be derived from the expected value of outage costs.

$$ E(IC)={\displaystyle {\sum}_{n,m}E{(ENS)}_{n,m}\cdot {c}_{n,m}} $$

(4.2)

where

• E (IC) = Expected outage costs [NOK]

• E (ENS) = Expected ENS [kWh]

C_n,m as above.

At the end of the year the difference between expected and actual outage costs is calculated. In the case of a positive difference, i.e., the reliability is higher than expected, the difference is added to the revenue cap. In the case of a negative difference, the amount is subtracted from the revenue cap. This mechanism is illustrated by (4.3) and Fig. 4.2.

Fig. 4.2

Outage costs and revenues. Source: Brekke (2007)

$$ dR=E(IC)-IC $$

(4.3)

where

• dR = change in Revenue Cap

The calculations described above are carried out 1 year after the determination of network charges by the network operator. Therefore a gap usually occurs between the expected (allowed) revenues and the actual revenues as already illustrated in Fig. 4.2. If the difference is to the benefit of the network operator, the firm is obliged to pay back the windfall profit through lower network charges to its customers in the following years. Conversely the firm is allowed to be compensated for a potential loss through higher network charges. Thus an increase in reliability (i.e., a decrease in outage costs IC) leads to higher revenues whilst a decrease in quality leads to lower revenues. Given this mechanism (4.4) applies:

$$ \mathrm{R}' = \mathrm{I}\mathrm{C}' $$

(4.4)

where

• R’ = marginal revenue

• IC’ = marginal outage costs for a specific customer group

Moreover, the economic costs for network operation can be considered as the result of company specific capital expenditures (CAPEX) and operational expenditures (OPEX) as well as the outage cost of the customers as shown under (4.5).

$$ \mathrm{C}=\mathrm{OPEX}+\mathrm{CAPEX}+\mathrm{I}\mathrm{C} $$

(4.5)

The economic optimum for marginal outage costs results from a minimisation of (4.6), given that

$$ \mathrm{OPEX}'+\mathrm{CAPEX}' = \mathrm{I}\mathrm{C}' $$

(4.6)

Consequently the profit of a network operator can be expressed as

$$ \Pi =\mathrm{R}\hbox{--} \mathrm{OPEX}\hbox{--} \mathrm{CAPEX} $$

(4.7)

Therefore a profit-maximising network operator would act on the assumption that

$$ \mathrm{OPEX}'+\mathrm{CAPEX}'=\mathrm{R}' $$

(4.8)

Taking these assumptions into account as per (4.4), (4.6) and (4.8) we deduce that a profit-maximising network operator under the Norwegian regulatory regime would strive towards a social optimum by minimising overall economic (social) costs.

Brekke (2007) concludes that the implementation of the quality regulation system has sensitised the network operators to outage costs incurred by their customers. This motivated a change in the operation and management of their assets. Moreover, the regulatory regime allows for a clear definition of responsibilities in the network and therefore, facilitates performance improvements. Brekke detected, however, some shortcomings in the system such as the unsatisfactory recovery time following an interruption for those clients for whom the CENS-regulation does not set strong enough incentives. Moreover, short interruptions are not taken into account, which may lead to higher costs to the affected customers.

The shortcomings detected by Brekke (2007) have partly been addressed by amendments to the regulatory regime with the start of the new regulatory period in 2007. Since then outage costs have been directly integrated into the calculation of the revenue cap (Fig. 4.3). Thus, the costs incurred for the provision of a certain quality level are considered as part of OPEX and feed into the DEA-based benchmarking; and the revenue caps are adjusted on an annual basis with respect to the actual level of interruptions.

Fig. 4.3

Revenue Cap and outage costs since 2007. Source: Brekke (2007)

In parallel, another component of quality regulation has been introduced, namely direct compensation payments. As a result, network operators are obliged to pay direct compensation to those customers affected by interruptions longer than 12 h (Brekke 2007).

These payment obligations follow the schedule below:

• For 12–24 h: 600 NOK (approx. 70 €).

• For more than 24 till 48 h: 1.400 NOK (approx. 160 €).

• For more than 48 till 72 h: 2.700 NOK (approx. 310 €).

Additionally, 1.300 NOK (approx. 150 €) applies to each subsequent 24-h period. However, the payments should not exceed the annual tariff payments . Moreover, short interruptions lasting from 1 to 3 min as well as the time dependency are integrated into the CENS-system as from 2009.¹³

The previous sub-section provided an overview of the evolution of quality of supply regulation in Norway. In summary we conclude that Norway has a mature system for determining the external costs of quality and for incorporating them into the regulatory formula.

It is also worthwhile to look behind the scenes of the Norwegian system in order to gain empirical evidence of the actual impact of service quality regulation on the efficiency situation of Norwegian network operators and the service quality level as a whole. This review is carried out in the following section.

4.3.3 Data and Methodology

In order to examine the performance of the Norwegian approach to service quality regulation , we use a panel dataset for 131 Norwegian distribution network operators (DNO) from the period 2001–2004.¹⁴ In Norway, the DNOs are mainly publicly owned. The Norwegian Energy Act stipulates full ownership unbundling between generation and supply functions on the one hand and transmission and distribution on the other.¹⁵

The productivity analysis method used is Data Envelopment Analysis (DEA) technique .¹⁶ DEA measures the relative efficiency of a company relative to the best performing companies (peers) by means of a non-parametric, linear frontier over the sample. This piece-wise approach aims at fitting a linear “hull” around the data assuming that this hull adequately forms the frontier of the most productive firms by means of a deterministic approach with multiple inputs and outputs. The resulting efficiency score reflects the amount by which a given company could improve its productivity relative to its peers. The most efficient company is assigned an efficiency score of one given that it scores best by minimising its inputs for a given level of output. This approach has been first proposed by Farrell in 1957. In the following two decades options for mathematical programming have been suggested by Boles (1966), Shephard (1970) and Afriat (1972). Nevertheless, only in the late seventies this method eventually attracted global interest following a paper published by Charnes et al. (1978) which first introduced the term “DEA”. In this paper the authors argued that the main task of DEA was to compute an efficiency score for each company based on a linear program formulation. Moreover Charnes et al. (1978) advocated an input-oriented model assuming constant returns to scale (CRS) to estimate the production frontier. The CRS approach assumes that changes in output result from proportionate changes in input. Firms having altered their sizes and thus diverging from the optimal operating scale will therefore be detected as inefficient by the model. Alternatively the assumption of variable returns to scale (VRS) provides for a limited control of the firms on their scale and therefore implies a correction of the efficiency score for such differences. Consequently, only firms operating on a similar scale will be compared to each other (Ajodhia 2006). In the following example, we assume constant returns to scale (CRS) since the networks operators may, in general, be able to optimize their size and scale. By that, we consider inefficiencies due to a deviation from optimal scale.¹⁷ A CRS input-oriented frontier is calculated by solving the linear optimization program in (4.9) for each of N companies. Moreover, it is assumed that the companies use K inputs and M outputs (Shephard 1970):

$$ \begin{array}{l} \max \kern1em \theta, \\ {}\mathrm{s}.\mathrm{t}.\kern1em -{y}_i+Y\lambda \ge 0,\\ {}{x}_i/\theta -X\lambda \ge 0,\\ {}\lambda \ge 0,\end{array} $$

(4.9)

where X is the K*N matrix of inputs and Y is the M*N matrix of outputs. The i-th company’s input and output vectors are represented by x_i and y_i respectively. λ is a N 1 vector of constants and θ is the input distance measure.

We chose to use the DEA technique as it is particularly suitable for multiple input and multiple output efficiency analysis and for this reason it is often the method of choice by most regulators that practice benchmarking including the Norwegian regulator (see Jamasb and Pollitt 2001).

A in its original form, however, is unable to provide unbiased efficiency estimates and confidence limits for the efficiency scores. The theoretical bias is evident since the observed input–output combination is just a fraction of any possible one: $\left(\mathrm{x},\mathrm{y}\right)\ \subseteq \left(X,Y\right)$. This implies that the estimated production set $\widehat{\psi}$ is a just a subset of Ψ, $\widehat{\psi}\subseteq \psi$. Efficiency is estimated and compared within a restricted sample and the estimator is upward biased as a result. We apply a bootstrap procedure suggested by Simar and Wilson (1998) to overcome this problem. It provides an estimate for DEA’s upward efficiency bias and confidence intervals by drawing random samples from the efficiency scores’ truncated probability density functions.

As highlighted above, DEA determines the efficiency score of a firm compared to its peers and therefore, indicates the catch-up potential within a given sample. For the purpose of this paper the cost of service quality is incorporated into the benchmarking. Therefore, it is crucial to provide for the ambivalent relationship between productive efficiency and quality. In general one may assume that higher quality levels lead to higher costs. In a cost-based DEA, companies operating at higher quality levels would therefore likely score worse than their efficiency-oriented counterparts albeit running their business to the benefit of quality. This potential trade-off can be reduced by incorporating SOTEX into the DEA and thus accounting for the provision of quality (Ajodhia 2006).

The model specification incorporates total expenditures TOTEX and SOTEX, respectively. These are considered separately as a single input in monetary terms. Hence, we use two models, one with TOTEX and the other one with SOTEX as input variable. In Model one TOTEX describes the sum of OPEX and CAPEX, both influencing the productivity of the network operator without explicitly considering quality aspects. By contrast, Model two incorporates SOTEX as the input variable in order to reflect the impact of quality incentives. SOTEX is the sum of TOTEX (corporate production costs) and the external costs of low quality, i.e., the CENS incurred by customers. Thus, the resulting efficiency scores of SOTEX reflect the ability of the network operator to balance the trade-off between efficient costs and quality (Ajodhia 2006).

We use a simple model with one input and two outputs. Inputs are either TOTEX or SOTEX. The outputs consist of energy supplied and the number of customers, following the most frequently used output variables in international regulatory benchmarking (Jamasb and Pollitt 2001).¹⁸ Although the two cost drivers form one joint service in electricity distribution they are considered separately since they drive different cost categories, namely fixed and variable costs (Growitsch et al. 2009). The model assumes input-orientation, i.e., the efficiency score depends on the ability of the network operator to minimise its inputs given a fixed vector of outputs. Table 4.2 shows the descriptive statistics of our sample aggregated for the considered period and individually for the respective years. Table 4.3 exhibits the mean for the years 2001–2004.¹⁹

Table 4.2 Descriptive statistics of the sample (aggregated)

Table 4.3 Means for the period 2001–2004

With regard to SOTEX we find that costs slightly increase in 2002 followed by a decline in the following years. A similar development can be observed for TOTEX. Accordingly the cost of quality decreases in 2002 followed by a significant increase in 2003. Simultaneously the standard deviation and the maximum more than double compared to 2002. This development suggests that a significant event took place in 2003 featuring increased prices. Looking at the output variables, the final customers slightly increase after an initial stagnation, whilst the energy supply declines over the period. Overall we show in Table 4.3 that there is only a rather marginal gap between TOTEX and SOTEX . Moreover, homogenous trends can be reported for SOTEX and TOTEX.

Based on this first impression, we hypothesize that the external costs of quality have a small effect on the cost and, as a result, the incentives of the Norwegian network operators. In the following section we test this hypothesis by analysing the results of the DEA regarding the efficiency of the sample of Norwegian network operators.

4.3.4 Estimation and Results

Table 4.4 shows the bootstrap results of the DEA for Model one (input: TOTEX) and Model two (input: SOTEX). As expected, the unbiased estimates obtained from bootstrapping are slightly lower than the original scores. In order to test whether the annual average unbiased efficiency scores for TOTEX and SOTEX differ significantly from each other, we use the non-parametric Wilcoxon ranksum test.²⁰ Comparing the results for TOTEX and SOTEX for each year, we find no systematic differences between the models.²¹

Table 4.4 Efficiency for model 1 (TOTEX)

Analysing the development of TOTEX estimates only, the efficiency decreases significantly after the first year and remains statistically constant from 2002 to 2004.²²

Concerning the SOTEX cross-sections, the efficiency scores do not vary from year to year. Comparing average efficiencies from 2001 to 2004, however, indicates marginally but statistically significantly lower social cost efficiency 4 years after the introduction of the CENS regulation.

Overall we find that TOTEX and SOTEX almost develop in similar manners, corroborating the initial hypothesis we made. Moreover, the Wilcoxon ranksum test showed that there is no significant difference in the unbiased efficiency score between the years 2002 and 2004, neither for TOTEX nor SOTEX. The reduction in TOTEX and SOTEX efficiency in 2004 relative to 2001 coincides with the development of the average efficiency score shown in the descriptive statistics.

A closer examination of efficiency scores on a per company basis, however, shows that the efficiency scores for individual firms can change significantly from year to year. At the same time, the TOTEX and SOTEX scores, for a given year, are rather similar. Figures 4.4 and 4.5 show the utilities’ efficiency scores (Y-Axis) for 2001 in increasing order relative to those of 2002–2004 (Company ID, X-Axis). Moreover, the figures show that the scores of more efficient utilities in 2001 (i.e., right hand side of the figures) also tend to be higher than in subsequent years. However, the peers change over time.

Fig. 4.4

TOTEX efficiency scores by company and year

Fig. 4.5

SOTEX efficiency scores by company and year

The analysis of the efficiency development shows that the introduction of quality regulation did not significantly change the efficiency scores of the companies. Moreover, it appears that the external costs for quality are quite low, which is proven by the fact that the difference between TOTEX and SOTEX is nearly zero. These findings are substantiated by the fact that the costs of energy not supplied in Norway are rather low compared to other European countries (Ajodhia 2006 and Fumagalli et al. 2007).

The results suggest that the introduction of quality regulation in Norway does not have a strong negative impact nor does it economically conflict with cost efficiency of the networks—i.e., the external quality costs play a relatively minor role. Moreover, the level of quality appears to be close to the social optimum, which explains the limited impact of CENS-regulation on the efficiency scores. However, benchmarking results in general and the empirical findings for the Norwegian example in particular have to be treated prudently since they only provide a first quantitative approximation of the implications of service quality regulation.

4.4 Conclusions

The objective of this paper was to scrutinise the issue of quality-related incentives based on customers’ WTP for continuity of supply regulation and to analyse the impacts of such indirect regulatory measures by means of a concrete case-study.

After a theoretical introduction on WTP-based service quality regulation and the optimal quality level we described how Norway, a pioneer in this field, has put this regulatory approach into practice. In summary we conclude that Norway has a mature system for determining the external costs of quality and for incorporating them into the regulatory formula. In the following, we empirically examined how the distribution network operators adapted to the Norwegian CENS-arrangement. In order to do this, we analysed whether the distribution network operators changed their quality-related optimisation strategies reflected by their cost efficiency developments. The results indicate that incorporating the external cost of quality in incentive regulation benchmarking models has not played a major role during the period 2001–2004.

A first intuitive explanation for this may be that the service quality level was already close to the social optimum prior to the implementation of quality regulation. This confirms the statement by Brekke (2007) that the actual level of service quality was generally perceived as “satisfactory” during our observation time. This rationale should however, be pulled together with the current enhancement to the CENS-regulation in Norway. The current amendments (incorporation of short interruptions and time dependency in the CENS-arrangement) in Norway are motivated with the need to find the “most optimal” level for the society as a whole (Kjølle et al. 2009). This implies that politics, regulators and society still call for an optimisation potential with regard to the Norwegian service quality level from am social point of view.

As regards the empirical visibility of such endeavours, our quantitative results should, however, be treated with caution since our data panel only covers the period from 2001 to 2004. Moreover, we only focused on TOTEX and SOTEX efficiencies and did not further elaborate on productivity developments and welfare implications due to limited data availability. This caveat indicates that data availability (especially for a longer time horizon) and robustness are limiting factors for this kind of analysis. Moreover, there is a time lag between the introduction of quality regulation and its impact on the investment decisions of network operators. Thus, the full impact of quality and asset management related strategies of network operators might not yet be reflected in the efficiency scores within the time horizon considered in the this study. Nevertheless, our findings generally confirm that the service quality level has already reached an economically reasonable level. The verification whether the optimisation potential towards the most optimal quality level for the Norwegian society as a whole is actually exhausted still requires empirical evidence.

Therefore, future research should in particular address the issue of delayed reactions in terms of continuity of supply improvement. Moreover a parallel analysis of productivity developments based on data reflecting the latest amendments to CENS-regulation should be carried out in order to empirically disclose the step from a satisfactory towards a most optimal service quality level.

Appendix

See Tables 4.5, 4.6, 4.7 and 4.8.

Table 4.5 Descriptive statistics year 2001

Table 4.6 Descriptive statistics year 2002

Table 4.7 Descriptive statistics year 2003

Table 4.8 Descriptive statistics year 2004