Strategic behavior in the German balancing energy mechanism: incentives, evidence, costs and solutions

Abstract

This paper investigates the incentives market participants have in the German electricity balancing mechanism. Strategic over and undersupply positions are the result of existing stochastic arbitrage opportunities between the spot market and the balancing mechanism. Clear indications for strategic behavior can be observed in aggregate market data. These structural imbalances increase the need for reserve capacity, raise system security concerns, and therefore place significant costs on consumers. The underlying problem is the disconnect between spot market, reserve capacity market and balancing mechanism. Alternative market design options discussed in this paper suggest better alignment between these markets/mechanisms.

1 Introduction

The liberalization and the resulting deconstruction of the integrated electricity value chain brings new coordination requirements for the technical balancing of demand and supply in the electricity system. In Germany and in many other liberalized electricity markets, the crucial balancing task has been separated into two stages: decentralized planning via balancing groups before, and centrally coordinated actual balancing by the transmission system operators (TSOs) via the use of highly flexible reserve capacity after gate closure.

The balancing mechanism effectively connects the two stages in the German electricity system. Its core is the pricing of actual imbalances caused by each balancing group. This shall provide the incentive for effective decentralized planning and thus minimize the remaining balancing requirements that are carried out by centrally coordinated reserve capacity in real-time. All market participants have to belong to a balancing group. They need to forecast their load commitments and procure the corresponding electricity volumes. The according load schedules have to be submitted to the TSO before gate closure. Every real-time deviation from this planned schedule, the imbalance, is settled at the balancing price. In Germany, the specific nature of the balancing prices provides predictable arbitrage opportunities between spot and balancing energy. Since balancing energy and reserve capacity is limited, the exploitation of such arbitrage on purpose is prohibited. It would jeopardize the effective coordination at the planning stage and may result in severe system instabilities.

Suspected strategic behavior in the German balancing mechanism has recently sparked public attention (see FAZ 2012). During a cold spell with critical system conditions in February 2012, the German electricity system was structurally undersupplied, which nearly caused a blackout. Market participants were suspected to intentionally rely on balancing energy to avoid procuring more expensive spot electricity. Alternatively, unexpected weather conditions may have caused an underestimation of the actual load (see Energate 2012; BNetzA 2012a). For specific moments, it is difficult to determine whether undersupply behavior was intended or merely the result of unavoidable forecasting uncertainty.

This paper investigates the incentives in the German balancing mechanism and the empirical evidence for potential strategic behavior, using longer data time series. In Germany, the prices for balancing energy and spot electricity are largely disconnected due to the specific market design. This provides structural opportunities for strategic behavior at one of the most crucial links in the electricity system at times when the system is most vulnerable. As we will show, market participants have an incentive to over and undersupply their expected load commitments depending on the expected spot price.

The closely-related German market for reserve capacity has attracted increasing attention in the economic literature, in which the market design and results are debated.¹ However, the balancing mechanism and the link to spot markets have only been studied selectively. Wawer (2007) shows that balancing groups have an incentive to oversupply in off-peak and undersupply in peak periods. Moeller et al. (2011) apply time series analysis to detect strategic positions taken in the German balancing mechanism. They conclude that the strategies, and the market design that fosters them, contribute to the overall effectiveness of the electricity market.

Outside of the specific German context, Boogert and Dupont (2005) study gaming-behavior in the Dutch balancing mechanism, which differs from the market design in Germany. They find that, in the Netherlands, strategic behavior is rarely profitable and comes with large risks. Vandezande et al. (2010) discuss different balancing designs by the use of simplified examples and conclude that a mechanism designed as a one-price system like in Germany without penalties or link to spot prices should be preferred.

The aim of this paper is to extend the existing academic literature on strategic behavior in the German market through explicitly studying the incentives created by the specific market design. We show that a broader spectrum of incentives for over and undersupply exists than described by Wawer (2007). This gaming behavior can be observed in aggregated market data. It creates external costs of likely more than €200m that are borne ultimately by the consumer. Solutions to prevent strategic behavior exist and their implications are evaluated. Our analysis leads to different conclusions than Vandezande et al. (2010) and Moeller et al. (2011). The differences are discussed in the conclusions of this paper.

The paper is organized as follows: Sect. 2 describes the German electricity market design. The data and analysis horizon are described in Sect. 3. The formation and predictability of balancing prices are studied in Sect. 4. In Sect. 5, the incentives in the balancing mechanism are explored and analyzed with a simplified model. Empirical evidence of strategic behavior is investigated in Sect. 6. The implied costs of this behavior are estimated in Sect. 7 and possible solutions to prevent it are discussed in Sect. 8. Finally, conclusions are drawn.

2 Market design of electricity markets in Germany

The German electricity market—like many other European electricity markets—consists of a sequence of separate bilateral markets. This philosophy is in contrast to centrally coordinated pool markets in many liberalized markets in the US. Three types of interrelated markets or mechanisms can be distinguished in Germany (see Fig. 1).

Fig. 1

Sequence of electricity markets in Germany

Firstly, in the scheduled energy market, the actual output of power plants is traded, either in forward/future or spot markets. The physical day-ahead spot auction with the crucial price-setting and market coordination function is held at 12 a.m. with hourly products for the following day. The European Energy Exchange (EEX) day-ahead spot trading volume reached $>$40 % of consumption in 2012 and can be considered as highly liquid. The German market is administrated as an energy-only market and as a single price zone. Physical transmission constraints—when they exist—are exclusively handled by the TSOs by out-of-market re-dispatch. The costs are socialized via grid fees.

After the day-ahead spot market closes, all market participants (technically all balancing groups) have to submit their 1/4-hourly energy schedules to the TSOs for their respective control zone.² The schedules can be adjusted until gate closure, 45 min ahead of delivery. A continuous intraday spot market facilitates energy transactions necessary for re-scheduling. At the EEX, hourly intraday products (with traded volumes of $\sim$3 % of consumption) and over-the-counter (OTC) 1/4-hourly increments are traded. These are the last market-based transactions before the submitted energy schedules become binding. After gate closure, the TSOs take over the responsibility for any further action.

Secondly, in parallel with the scheduled energy market, the TSOs jointly procure system reserve capacity, which is required for continuously balancing all deviations between planned and actual load schedules after gate closure or, in other words, for balancing the deviations between consumed and produced electricity in real-time. Therefore, the necessary reserve energy is also called balancing energy; they are actually two sides of the same coin.³

Three reserve qualities with different response times and activations are distinguished. Primary reserve or frequency control activates immediately and has to reach full response within 30 s. The activation of secondary reserves also starts immediately and needs to be fully available within 5 min. The objective of tertiary reserves is to release and supplement secondary reserves starting 15 min after activation.

The option-like character of reserves is reflected by two-part pricing.⁴ A reservation price (€/MW) is paid for reserving the capacity⁵ and a reserve energy price (€/MWh) is paid for exercising the reserve option to generate the required real-time energy. Reserve energy is called according to the respective merit order of the reserve energy price bids and is paid as bid.

Auctions for secondary reserve capacity were held monthly⁶ during the analysis period; separately for peak and off-peak periods. Tertiary reserve is procured daily, ahead of the day-ahead spot market in six separate 4-h time segments. For both secondary and tertiary reserve, separate incremental and decremental reserve products are defined. Incremental reserve is required when the consumed electricity is larger than the produced electricity (the control zone has a deficit) and vice versa. In case decremental energy is called, the supplier pays the reserve energy price, whereas for incremental reserve energy the supplier is paid.

Thirdly, the balancing mechanism distributes the costs of the reserve energy among the originators of the imbalance. Its main objective is to provide the incentive to minimize the actual imbalances and thus the need for balancing energy (see ERGEG 2006). Therefore, the mechanism determines ex-post the individual payments for addressing the respective individual imbalances (difference between planned and actual load schedules) of every balancing group. Partly, the imbalances of the balancing groups offset each other within a control zone. The remaining imbalance of the control zone is offset by the TSOs by employing reserve energy as described above. The reserve energy costs set the price for all individual imbalances within the control zone for the respective 1/4-h⁷—either the decremental balancing price when the control zone is long or the incremental balancing price when the control zone is short. All balancing groups with a positive balance (an oversupply of energy) receive this balancing energy price and all undersupplied balancing groups have to pay for the missing energy. Hence, the balancing mechanism is a cost-based one-price system and a zero-sum activity for the TSOs as all reserve energy related costs are passed through to the balancing groups.⁸

Given its physical and commercial characteristics, balancing energy is a substitute for spot electricity traded in the scheduled energy markets.

3 Data basis and time horizon

The behavior of market participants within the German balancing mechanism should ideally be investigated at the individual balancing group level. Unfortunately, the corresponding planned and actual energy schedules are not publicly available. Therefore, the aggregated imbalance data for each control zone are analyzed instead. The four German TSOs publish 1/4-hourly balancing price and imbalance data for their control zones (Amprion 2012a; TennetTSO 2012; 50Hertz 2012; TransnetBW 2012). Furthermore, the published 1/4-hourly load data are used to determine the relative imbalances. This allows normalizing the imbalance data during daily, weekly and seasonal load cycles as well as for the different relative sizes of the control zones. The day-ahead and intraday spot price data from the EEX (2012) serve as principal reference.

The analyzed data cover a period of more than 5 years starting from January 2006 till April 2011. The market data are investigated in 5 separate periods: 2006, 2007, 2008–April 2009, May 2009–April 2010 and May 2010–April 2011. These specific data samples have been selected for two reasons. Firstly, periods with at least 12 months have a sufficient sample size to obtain statistically significant results. Secondly, they reflect two relevant changes in the market design and thus in market data. By order of the German regulator (see BNetzA 2010), the TSOs have to manage their separate control zones as one zone in order to realize portfolio effects starting from May 2009. As a result of this order, they are required to balance any imbalance among the control zones in order to reduce the need for reserve energy, effectively leading to only one balancing price for the unified control zone. TennetTSO, 50Hertz and TransnetBW started with this new procedure. This joint management is called Grid Control Coordination (GCC). Amprion joined 1 year later in May 2010.

The following analysis focuses on the two recent 1-year periods May 2009–April 2010 and May 2010–April 2011. The analysis of results obtained for the preceding periods is shown in the Appendix.

4 Predictability and formation of balancing prices

Despite the fact that the actual balancing prices are only published with a 2 months delay, the known reserve energy bid curves allow for a good ex-ante estimation of the expected balancing prices. Both auctions for secondary and tertiary reserve are held before the spot markets. Furthermore, adding to the predictability, balancing prices are mainly determined by the secondary reserve energy prices, as tertiary reserve is only called occasionally.⁹

The secondary reserve energy bids and the resulting bid merit order apply throughout the full period for which reserve capacity is auctioned (e.g. for a month during the analysis periods in focus). This leads to a high degree of predictability of the balancing prices. This is illustrated in Fig. 2, in which the secondary reserve energy bid curves and the actual 1/4-hourly balancing prices are depicted for the peak period in January 2010. The depicted actual balancing prices are distinguished for the 1/4-h in which only secondary reserve, and in which tertiary reserve energy is additionally called.

Fig. 2

Reserve energy/balancing prices for peak period in January 2010

When only secondary reserve is called, the actual balancing prices do not fully match the secondary reserve bid curves for two reasons. Firstly, pay-as-bid remuneration is applied in the German reserve capacity market, resulting in lower average costs of all called bids. Secondly, the balancing prices are determined as the sum of all costs for reserve energy divided by the average imbalance of the control zone during the 1/4-h. This results in larger deviations from the bid curve in all those 1/4-h intervals in which incremental as well as decremental reserve energy is called. These are typically 1/4-h intervals with low average imbalances.

When tertiary reserve is additionally called, balancing prices tend to deviate more from the bid curves. However, the impact on the ex-ante expectation is small due to the low probability of tertiary reserve being called.

Figure 2 also shows that the general price formation is very distinctive with a large gap between incremental and decremental balancing prices.¹⁰ Incremental balancing prices tend to be close to, or above, 100 €/MWh with an average of about 120 €/MWh and the decremental balancing prices tend to be below 20 €/MWh with an average of about $-$20 €/MWh.

As balancing prices result from the deployment of reserve energy according to the merit order of bid prices, they are generally increasing with higher balancing requirements and vice versa. However, this tendency is not very pronounced and the price patterns tend to be relatively flat. As a result, the balancing prices are highly dependent on the direction of the imbalance of the control zone, but largely independent of the actual size of the imbalance.

This distinct pattern of the balancing prices is recurrent over time and during peak as well as off-peak periods. As an example, Fig. 3 shows the balancing prices for peak and off-peak in August 2010.

Fig. 3

Reserve energy/balancing prices in August 2010

In the long-run, (incremental) balancing prices are expected to be correlated with spot prices as both are impacted by fuel price changes. However, they are largely independent of spot prices in the short-run, at least under the prevailing market design. The secondary reserve energy prices, which predominately determine the balancing prices, are set in the monthly auction. They are, therefore, not significantly impacted by hourly load variations and supply fluctuations e.g. from intermittent renewables as spot prices are. The quarter-hourly correlation coefficients between balancing and spot prices are low within a range of mainly 0.1–0.2 as shown in Table 1. The day-ahead spot price tends to be slightly less correlated with the balancing prices than the intraday spot prices.

Table 1 Correlation between balancing and spot prices

As a result, balancing prices in Germany are highly predictable once the secondary reserve auction results are known. This property can be exploited by market participants and is discussed in the following section.

5 Incentives in the balancing mechanism

Balancing groups are expected to provide unbiased energy schedules that reflect their best estimate of the actual energy flow. They are expected to forecast their load commitments and procure the necessary electricity to match their best forecast.

However, the relatively high predictability of balancing prices and their very low correlation with spot prices could be considered when deciding how much electricity to procure before gate closure. The resulting strategy space for over and undersupply is summarized in Fig. 4.

An undersupply strategy is straightforward when spot prices are expected to be very high and above the expected incremental balancing prices (see RHS of Fig. 4). This is a fairly simple market arbitrage. Independently of the imbalance of the control zone, balancing energy is very likely to be less expensive than spot electricity. Since it is a dominant strategy for all market participants, the risk of large imbalances and thus unstable system conditions is high. This would put a further burden on the electricity system in situations when it is already under stress, as reflected by high spot prices, and might even cause blackouts. Imbalances exceeding the incremental reserve capacity can only be resolved by load-shedding.

Fig. 4

Strategies to game the German balancing mechanism

On the other hand, when the expected spot price is below the expected decremental balancing price, the dominant strategy is to oversupply the expected load commitments (see LHS of Fig. 4). Independently of the imbalance of the control zone, excess energy of a balancing group will be remunerated at a balancing price that is very likely higher than the spot price. Such opportunities might appear in off-peak periods with a high level of renewables production and must-run generation capacity serves the marginal load, resulting in low or negative spot prices. Imbalances that exceed the decremental reserve capacity can only be resolved by forced shut down of must-run capacity. In such situations the electricity system is also under stress, even before any further oversupply imbalances are added to the system.

For all scenarios with spot prices between the expected decremental and incremental balancing prices (central part of Fig. 4), the strategy depends on the expected imbalance of the control zone and the relative amount one can gain and lose. Under the current balancing mechanism in Germany, it is advantageous to be contrary to the control zone requirements and thereby reducing the overall imbalance. Thereby, one either receives a relatively high price for excess energy or pays a relatively low price (or even receives a payment in case of negative decremental balancing prices) for a shortage of energy. As the direction of the imbalance of the control zone is largely random, it is a stochastic arbitrage game with expected pay-offs depending on the relative differences between balancing and spot prices. There can be even a positive pay-off if the probability of being contrary to the control zone is significantly below 50 % when the expected arbitrage gains outweigh the expected losses. The success of such a strategy is, however, curbed and limited as any over or undersupply position moves the imbalance of the control zone in the unfavorable direction. Nevertheless, the incentive to act strategically exists.

To explore this stochastic arbitrage incentive further, a simplified model of the strategic over and undersupply decision is developed. Suppose all supply companies active in a control zone are able to forecast the demand of their customers on average correctly and the forecast error is normally distributed. Thus, the actual demand $D^{Total}$ in the control zone is normally distributed around the expected demand $E(D^{Total})$ with a standard deviation $\sigma = e{\times }E(D ^{Total})$ and follows the cumulative distribution function $F(D^{Total})$.¹¹

Without loss of generality, there are $n$ supply companies that act all strategically in their procurement decision. They serve each a share $k_{i}$ of the demand within the control zone. Thus, the expected demand of company $i$ is $E(D_{i})=k_{i}{\times }E(D^{Total})$. It procures an amount $d_{i}$ in the spot market with:

$$\begin{aligned} d_{i}=(1+a_{i}) E(D_{i}) \end{aligned}$$

(1)

The factor $a_{i}$ denotes the relative over-/undersupply of its expected load commitments. The total amount procured is:

$$\begin{aligned} d^{Total} = \sum \limits _{i=1}^{n} {d_{i}} = \sum \limits _{i=1}^{n} {\left( {1+a_{i}} \right) k_{i}} E(D^{Total})\qquad with \sum \limits _{i=1}^{n} {k_{i}} =1. \end{aligned}$$

(2)

The probability that the control zone is long or oversupplied is $F(d^{Total})$, and $1- F(d^{Total})$    that it is undersupplied.

The supply company $i$ is assumed to be risk neutral and to maximize its expected profit $E(\varPi _{i})$ while choosing the parameter $a_{i}$:

$$\begin{aligned} \mathop {\max }\limits _{a_{i}} E\left( {\varPi _{i}} \right) = P^{\mathrm{Re}tail} E(D_{i}) - E(P^{Spot}) d_{i}- E(P^{Balancing}) \left[ {E(D_{i}) - d_{i}} \right] \end{aligned}$$

(3)

with $P^{Retail}$ the retail price, $E(P^{Spot})$ the expected spot price and $E(P^{Balancing})$ the expected balancing price. The expected spot price is assumed to be independent of the over and undersupply behavior $a_{i}$.¹² Using the definition (1), it can be reformulated as a maximization problem with the expected profit margin:

$$\begin{aligned} \mathop {\max }\limits _{a_{i}} E\left( {\pi _{i}} \right) = P^{\mathrm{Re}tail} - E(P^{Spot}) + a_{i}\left[ {E(P^{Balancing}) - E(P^{Spot})} \right] \end{aligned}$$

(4)

The expected balancing price is determined by:

$$\begin{aligned} E(P^{Balancing}) = F(d^{Total}) E(P^{Dec}) + (1-F(d^{Total})) E(P^{Inc}) \end{aligned}$$

(5)

with $E(P^{Dec})$ and $E(P^{Inc})$ the expected decremental and incremental balancing prices. For simplicity, both are assumed to be independent of the size of the imbalance of the control zone. This assumption is in accordance with actual prices as indicated by Figs. 2 and 3.

The optimal strategy $a_{i}$ can be derived from the first-order condition of the profit maximization problem:

$$\begin{aligned} \frac{E(P^{Inc})-E(P^{Spot})}{E(P^{Inc})-E(P^{Dec})}&= F\left( {d^{Total}} \right) + a_{i}k_{i}E(D^{Total}) f\left( {d^{Total}} \right) \forall i \nonumber \\&= \varPhi \left( {\frac{\sum \limits _{j=1}^{n} {a_{j}} k_{j}}{e}} \right) + \frac{a_{i}k_{i}}{e} \varphi \left( {\frac{\sum \limits _{j=1}^{n} {a_{j}} k_{j}}{e}} \right) \forall i \end{aligned}$$

(6)

with $\varphi (\cdot )$ and $\varPhi (\cdot )$ the probability density and cumulative distribution function of the standardized normal distribution.

Since the optimality condition is analogous for all $i$ and only depends on the product $a_{i}\cdot k_{i}$, the strategic Nash equilibrium must be symmetric and the relationship between the optimal strategies may be derived as:

$$\begin{aligned} a_{j}= a_{i}\frac{k_{i}}{k_{j}} \quad \forall i,j \end{aligned}$$

(7)

Thus, the optimal strategy $a_{i}$ in a Nash equilibrium can be restated as:

$$\begin{aligned} \frac{E(P^{Inc})-E(P^{Spot})}{E(P^{Inc})-E(P^{Dec})} = \varPhi \left( {\frac{n a_{i}k_{i}}{e}} \right) + \frac{a_{i}k_{i}}{e} \varphi \left( {\frac{n a_{i}k_{i}}{e}} \right) \end{aligned}$$

(8)

The optimal relative over and undersupply $a_{i}$ is hence inversely proportional to the market share $k_{i}$ of company $i$. Or stated in other words: the absolute over and undersupply quantity $a_{i}\cdot k_{i}$, is in equilibrium independent of the size of the market participant.

Given the non-linear characteristics of the equilibrium condition (8), analytical solutions and approximations are difficult. Therefore, the optimal strategies are determined by numerical simulation.¹³ Choosing the following parameters for illustration purposes: average forecast error $e$ = 2 %, expected incremental and decremental balancing price $E(P^{Inc} )=120$ €/MWh and $E(P^{Dec})= -20$ €/MWh, the optimal strategy $a_{i}$ is determined depending on the relative share $k_{i}$ and the expected spot price $E(P^{Spot})$. The results are shown in Fig. 5 for the case $n=10$. For a better visualization, the strategic over and undersupply factor $a_{i}$ is depicted on a logarithmic scale.

Fig. 5

Over-/undersupply incentive depending on spot price and size of a supply company

The results clearly show that over and undersupply incentives exist when the expected spot price is between the decremental and incremental balancing price. The closer the spot price is to the incremental balancing price, the stronger is the incentive to undersupply. With an increasing spot price, the relative benefit from paying only the low decremental balancing price in case of an oversupplied control zone increases and the relative penalty from being charged the incremental balancing price in case of an undersupplied control zone decreases. The opposite holds for low spot prices. The incentive to deviate from the forecast only fades away when the expected spot price equals the average of the decremental and incremental balancing prices, i.e. when the balancing prices are symmetrical to the spot price. This further implies that structural asymmetries between balancing and spot prices should result in structural imbalances of a control zone.

Furthermore and following from Eq. (7), smaller supply companies have a larger relative incentive to act strategically. Their own impact to move the control zone imbalance in the unfavorable direction is smaller. This additionally implies that the more companies play the over and undersupply game, the less profitable it is and the lower is the strategic incentive for the individual company. This self-limiting effect should prevent extensive over and undersupply in the situations when the expected spot price is between the expected decremental and incremental balancing prices.

The individual strategic behavior results in the expected over and undersupply pattern of the control zone as depicted in Fig. 6. Its magnitude increases with the number of companies acting strategically. This theoretically expected over and undersupply pattern, depending on the spot price, will be later used as a primary reference when analyzing the empirical imbalance data in the next section.

Fig. 6

Expected over-/undersupply pattern of the control zone due to strategic behavior

The distinct pattern has been derived assuming risk neutral players, taking into account that decision makers submitting energy schedules are generally professional traders. Any degree of risk-adversity would make the over and undersupply pattern less pronounced. A slight dampening effect on strategic behavior is also expected when balancing prices are not flat, but rather increase with a higher imbalance. A dampening effect also is to be expected when the cumulated load forecast error is more leptokurtic as the probability to move the control zone in the unfavorable direction increases. Nevertheless, the distinct over and undersupply pattern, depending on the expected spot price, remains structurally unchanged.

This analysis shows that there are ample opportunities for strategic behavior at one of the most crucial links in the electricity system, particularly in periods with very high and very low spot prices. The TSOs are afraid of critical situations that could arise from the abuse of these opportunities and the resulting structural system imbalances. For that reason, they included anti-abuse clauses in the contracts with the balancing groups, for example:

Control energy is exclusively reserved for maintaining security and reliability of the transmission system and is therefore not available for any energy disposition. A remuneration of a surplus of energy takes only place if there are no clues about abusive additional feed-in.

The balancing group contract can be terminated when a short supply was intended or tacitly approved. (Amprion 2012b)

Abusive behavior is assumed when systematic deviations exist, “so that the arithmetic average value of all negative and positive differences is positive [or negative] to a greater extent” (Amprion 2012b).

In reality, it might be difficult to draw a clear line between forecast errors and intended over and undersupply behavior—especially if the balancing groups (and supply companies) do not act naively. In the next section, we will explore whether the anti-abuse clauses are sufficient to prevent strategic behavior in practice.

Note, it is not only supply companies that have an incentive for strategic behavior, but also generators, large industrial customers and the TSOs themselves. The same incentives apply for them as well. However, it is generally easier to investigate the reasons why conventional generators deviate from their submitted production schedule. If the TSOs were to game their own mechanism to increase their profits, it would happen largely without independent control.

6 Empirical evidence of strategic behavior

Unfortunately, historical “best” load forecasts of the balancing groups are private. Similarly, actual imbalance data for individual balancing groups are not publicly available. However, the TSOs publish the control zone imbalance data for every 1/4-h interval. Figure 7 shows the actual imbalances relative to the intraday spot price for the period May 2009–April 2010 in the Amprion and GCC control zones. The intraday spot price is chosen, since the intraday market is the last opportunity for adjustments.¹⁴

Both control zones show the clear tendency that market participants used the arbitrage opportunities to fulfill their load commitments partly with relatively cheap balancing energy when spot prices were relatively high, and vice versa. In periods with intraday spot prices above 120 €/MWh, the control zones were predominantly undersupplied. The Amprion control zone was undersupplied in 75 %, and the GCC control zone in 80 %, of those 64 1/4-h intervals, with an average imbalance of $-$267 and $-$488 MW respectively. The opposite was true for spot prices below $-$20 €/MWh. Amprion was oversupplied in 77 % of those 196 1/4-h intervals with an average of +501 MW, and GCC in 74 % with +819 MW on average.

It should be noted that imbalances did not exceed the contracted reserve capacity and that periods with “extreme spot prices” were not as overly exploited as the straightforward arbitrage incentives suggest (cf. LHS and RHS of Fig. 4). The risk of raising suspicion of abusive behavior increases significantly when the TSOs find themselves in situations with insufficient reserve capacity. Out-of-market actions, such as load shedding, would likely trigger the attention of the regulator. As a result, the TSOs might be inclined to investigate the causes more carefully and market participants would anticipate that.

Figure 8 takes a closer look at the same data for the periods with “non-extreme” spot prices ($-$20 to 120 €/MWh) only. The imbalances are depicted in relative terms using the actual 1/4-hourly load in the control zones as a denominator. In this way, they are comparable with the results derived from the theoretical model above. The empirical average imbalance is calculated for individual spot price clusters.

Fig. 7

Actual imbalances in control zones May 2009–April 2010

The strategic behavior with structural oversupply in periods with low intraday spot prices and structural undersupply in periods with high spot prices is clearly visible. The empirical average imbalance does very closely match the expected theoretical average imbalance for the case $n=10$. It deviates from the theoretical expectations only somewhat in case of high intraday spot prices. This might be partly due to the reasons explained in the paragraph above.

Fig. 8

Actual imbalances versus expected imbalance in control zones May 2009–April 2010

The Amprion control zone was on average structurally oversupplied with +113 MW. As the theoretical model suggests, this is mainly driven by the asymmetry of the incremental and decremental balancing price relative to the spot price. The average incremental balancing price of 117 €/MWh deviated stronger from the average spot price of 38 €/MWh than the average decremental balancing price of $-$15 €/MWh during the 2009/2010 period. Therefore, the balancing groups in the Amprion zone had on average a stronger incentive to oversupply and thereby avoiding the asymmetrically high incremental balancing prices. The balancing prices were more symmetrical in the GCC zone (108 and $-$15 €/MWh), resulting in a significantly lower structural oversupply behavior on average of $+$2 MW. This is the reason why the actual as well as the theoretical clustered average imbalance curves intersect the abscissa at higher spot prices for the Amprion control zone.

The identified pattern is not specific to the considered time horizon, but can be traced in the preceding years¹⁵ (see Appendix) and in the more recent period May 2010–April 2011 (see Fig. 9) as well. The strategic over and undersupply behavior in 2010/2011 was less pronounced compared to the prediction of the theoretical model than in the 12 months before. This might be partially explained by external factors such as the ownership change in two of the German TSOs during the first half of 2010. The former E.ON Netz was sold to the Dutch TSO Tennet in Jan 2010 and the former Vattenfall Europe Transmission, now 50Hertz, was sold to a consortium of the Belgian TSO Elia and the Australian infrastructure fund IFM in May 2010. The higher degree of independence and the uncertainty regarding the behavior of the new owners might have led market participants to reduce their over and undersupply strategies.

Fig. 9

Actual imbalances versus expected imbalance in control zones May 2010–April 2011

The imbalance pattern has been analyzed in large data samples. Each of the analyzed periods comprises at least 35,000 1/4-hourly data points. Furthermore, the obtained results have been tested for statistical significance. The average empirical imbalances for nearly all price clusters are statistically different from the overall average imbalance of the data sample at a 1 % confidence level (see Fig. 11 in the Appendix for the results).

The analysis clearly indicates that strategic behavior exists and that stochastic arbitrage opportunities between spot markets and balancing mechanism are exploited. The anti-abuse clauses applied by the TSOs appear to be not sufficiently effective. The biased schedules increase the need for reserve energy and capacity. They also increase the risk of severe system instabilities if the contracted reserve capacity proves insufficient.¹⁶

7 Estimation of the financial impact of strategic behavior

This section analyzes the associated costs caused by strategic behavior. Instead of using comprehensive modeling of all effects and detailed stochastic analysis to determine the resulting impact on reserve capacity requirements, rough estimations are used to gain a basic understanding of the financial impact. It is estimated for the May 2009–April 2010 period.

There are three essential drivers of the associated costs: (i) the increased need for reserve capacity, (ii) arbitrage gains between spot and balancing energy, and (iii) the increased risk of system instabilities.

Firstly, the increased need for reserve capacity is considered. Strategic behavior effectively moves the conditional mean of the imbalance probability distribution for individual spot price clusters. During periods of low spot prices, the imbalance probability distribution is shifted towards more positive imbalances, and vice versa. Hence, strategic behavior increases the overall variance of the imbalances and the confidence intervals of the imbalance distribution, which is the major driver of the required amount of reserve capacity.

Figure 10 shows that the conditional standard deviation of the imbalances is largely constant and independent of the spot price and thus of strategic behavior. Assuming that the imbalances are normally distributed implies that the confidence intervals are shifted in parallel with the conditional mean of the imbalance distribution. With this observation in mind, the increased need for reserve capacity can be directly estimated from the impact that strategic behavior has on the average imbalance.

Fig. 10

Impact of strategic behavior on imbalance distribution

Therefore a reference case with rather unbiased imbalances (intraday spot prices 20–60 €/MWh) is chosen and compared to periods with distinct over and undersupply behavior (spot prices $<$20 and $>$60 €/MWh). This indicates that strategic behavior led to an increased need of $\sim$443 MW incremental and $\sim$575 MW decremental reserve capacity (cf. Fig. 10). Assuming that the increased need is provided by 50 % secondary and 50 % tertiary reserve capacity valued at the respective average reservation prices for May 2009–April 2010, this results in $\sim$€67m higher capacity payments.¹⁷

The increased incremental secondary reserve capacity demand has an increasing impact on spot prices as more (partly infra-marginal) generation capacity is effectively withheld from the spot markets. This spot merit order effect is discussed in detail in Just and Weber (2008) and Just (2011). Applying the secondary reserve market equilibrium model with parameters calibrated to the German market used in Just (2011), the resulting increase of the average spot price is estimated at $\sim$0.20 €/MWh. It can be assumed that the price effect applies to the whole market of $\sim$530 TWh annual electricity demand in Germany (BMWi 2012). The spot market effect would be $\sim$€106m, valuing the total effect $\sim$€173m for the additionally required reserve capacity due to strategic behavior.

Secondly, arbitrage gains between balancing and spot energy are considered. The direct effect can be estimated by comparing the costs for the actual imbalances with the imbalances corrected by the average clustered imbalance (as if strategic arbitrage would not exist), valued at the respective balancing and spot prices. For every negative imbalance, the spot price is avoided and the incremental balancing price is incurred, and vice versa. For the GCC zone, the avoided spot cost are €19m and the additional balancing costs €11m, resulting in a net arbitrage gain of €8m. Due to the significant structural oversupply of the Amprion zone, the arbitrage is inverted with €36m avoided balancing cost and €26m additional spot costs, yielding a net effect of €10m. Thus, the effective arbitrage gains for 2009/10 were $\sim$€18m.

The market arbitrage has secondary effects on prices. Undersupply behavior implies lower demand in the spot markets and higher demand for reserve energy at times of high spot prices; whereas the opposite applies for oversupply. The resulting overall effect depends on the shape of the spot supply curve, which is estimated via a regression of actual hourly prices against residual hourly load (net of the day-ahead wind generation forecast), similarly as described in Just (2011). The average 2009/10 spot supply curve is best approximated by the cubic function $1.73{\times }10^{-12 }\, \hbox {x}^{3}$ – $2 .99{\times }10^{-7 }\,\hbox {x}^{2} + 0.0175\,\hbox {x}$ – 312.66. The fit is decent with $\hbox {R}^{2}=0.48$, implying that about half of the spot price variance is explained by the variation in the demand.

The secondary effect on spot prices is estimated as the relative impact the average clustered imbalance has on spot demand distribution and therefore supply-demand balance. The estimation yields a spot price increase of 0.06 €/MWh for the GCC zone and 0.19 €/MWh for the Amprion zone due to strategic behavior. The larger effect for the Amprion zone is essentially driven by the structural oversupply behavior. The additionally procured spot volumes have an increasing effect on the spot price. The total effect of 0.25 €/MWh results in $\sim$€130m additional costs for the overall market of 530 TWh.

The secondary effect on balancing prices is expected to be negligible as the balancing supply curves are relatively flat and the impacted volume is very small.

Overall, the benefits from the market arbitrage of $\sim$€18m are significantly overcompensated by the expected secondary price effects in the spot market $\sim$€130m, yielding a net effect of $\sim$€112m of additional costs due market arbitrage.

Thirdly, the probability of unavoidable load shedding and blackouts increases particularly in periods with extreme spot prices, in which over and undersupply is a dominant strategy. The macroeconomic costs can be immense. The economic impact of the August 2003 blackout in the Northeastern US/Canada was estimated at $4–10 billion (see ELCON 2004). The value of lost load is often assumed around 10,000 €/MWh (see Stoft 2002). One hour of blackout in Germany with 70 GW of lost load would have a macroeconomic cost of $\sim$€700m. Even if such extreme scenarios are not very likely, their probability increases due to strategic behavior. A probability-weighted quantification of this effect is difficult.

The implied net costs of the imperfect market design and the associated strategic behavior are significant and estimated to be in a range of €200–300m per year. Possible options for correcting this market design inefficiency are discussed in the next section.

8 Discussion of solutions of the incentive problem

The root cause of the incentive problem or strategic behavior is the disconnect between the spot markets and the balancing mechanism. Under the current market design in Germany, the balancing mechanism is directly linked with the reserve capacity market, which in turn is largely disconnected from the spot markets. As a result, balancing prices do not move in line with spot prices. In fact, the correlation between them is very low (cf. Table 1). This creates largely predictable opportunities that can be exploited.

Basically, there are two options to fix this problem: Either (i) establishing a direct link between balancing and spot prices while cutting the link to the reserve capacity market or (ii) improving the link between the reserve and spot markets while keeping the direct relation between reserve energy and balancing prices.

Firstly, it could be possible to directly link the spot and balancing prices by remunerating any imbalance of a balancing group at the spot price plus/minus a penalty. This eliminates the incentive to deviate from the best forecast as required incremental balancing energy would always cost more than the spot price, and vice versa. The payment does not depend on the control zone imbalance and applies specifically to the imbalance of the balancing group.¹⁸ The analysis in Sect. 4 shows that the penalty needs to be symmetrical. Otherwise, a steady bias is to be expected in the direction with the smaller penalty. The higher the penalty, the higher is the incentive to invest in better forecasting of load commitments and therefore additionally reducing the reserve capacity need.

If the link to the reserve market were to be cut, the reserve energy cost/receipts would not necessarily equal the balancing energy receipts/costs. Thus, the TSOs would generate a financial deficit or surplus due to the change in the balancing mechanism. The grid tariff regulation could pass on a surplus or allow for recovery of a deficit. This does not have an efficiency but a distribution effect with relatively minor overall relevance.

The change from a one-price to a two-price system implies that portfolio effects would arise. Larger balancing groups would have an advantage as netting imbalances pays off under a two-price system. However, the possibility of re-adjusting load schedules until 4 p.m. the following day by trading in the day-after market might allow even small players to reap portfolio effects. Under a two-price system, the possibility of trading imbalances allows win-win situations for balancing groups with opposite imbalances. The agreed prices would be equivalent to a “fair” sharing of the portfolio effect, as if the two were just one combined balancing group.

Secondly, the link between reserve and spot market could be improved, while keeping the identity of reserve energy and balancing prices. The incentive for strategic behavior only breaks down when the incremental and decremental reserve energy prices are symmetrical relative to the expected spot price. The incentive is significantly reduced if the spread between incremental and decremental reserve price is smaller and/or if the correlation between spot and reserve energy prices is higher.

To achieve this, the reserve capacity market design would need to be amended. Secondary reserve is currently procured weekly. Reducing the contract duration (e.g. to daily or hourly auctions) would certainly lead to a stronger alignment between the reserve capacity market and the spot market, but might not eliminate strategic behavior fully.¹⁹

Alternatively, called reserve energy could be remunerated at the spot price. By adding/ reducing a symmetrical premium/discount for incremental/decremental reserve energy, the strategic over and undersupply problem would disappear. As consequences, on the one hand, the reserve capacity auction would simplify to a one-price auction with a reservation price only. All difficulties of an appropriate scoring rule for two-price auctions would disappear.²⁰ On the other hand, productive efficiency of calling the reserve energy by their increasing marginal costs could not be achieved as the generation costs are not revealed. The productive efficiency and distortion problem could be corrected by bidding mark-ups/mark-downs on the spot price as a second part of the bid. However, this would not guarantee that the resulting balancing prices are symmetrical.

As a result, the first option would directly eliminate the incentives for strategic behavior and should therefore be preferred. The proposed changes, under option 2, in the reserve capacity market would reduce, but not fully eliminate the incentives.

The German regulator recently stipulated two modifications of the balancing mechanism in December 2012 to curb strategic behavior (see BNetzA 2012b). The first modification aims to eliminate dominant under-/oversupply behavior by using the higher of the intraday spot price or the balancing price as the balancing payment in case of undersupply, and vice versa. Thus, the balancing price cannot be lower than the intraday spot price when the control zone is short, and the balancing price cannot be higher than the intraday spot price when the control zone is oversupplied. This modification eliminates the dominant arbitrage opportunities, but does not reduce the incentives for stochastic arbitrage strategies investigated in this paper (cf. Fig. 4). The second modification targets to curb strategic arbitrage in case of large imbalances. If the imbalance is larger than 80 % of the contracted reserve capacity, the balancing price is increased by 50 % (at least by 100 €/MWh) in undersupply situations, or reduced by 50 % (at least by 100 €/MWh) in oversupply situations. This reduces the incentives for stochastic arbitrage strategies especially in the most severe system situations when remaining reserve capacity is short. However, this modification would have been applied only in $\sim$0.2 % of the 1/4-h in 2009/2010 and 2010/2011 and therefore would have not impacted the arbitrage strategies significantly.

The German regulator, through implementing these modifications, has acknowledged the ineffective market design. The stipulated modifications partly establish a direct link between spot and balancing prices, but do not cut the link between reserve energy and balancing prices completely as our first option suggests in order to eliminate strategic behavior. Incentives for strategic behavior still largely exist. It would be worthwhile analyzing the impact of the recent modifications in further research.

Any effective solution needs to bring the ex-ante expected balancing/reserve energy prices in line with spot prices to avoid systematic over and undersupply. The main challenge is that spot and reserve energy are not homogeneous. Technical requirements differ and the economic decision which plants are more economical to provide reserve capacity influences the reserve energy bids. The resulting, more far-reaching conclusion that real-time markets with similar design as intraday spot markets (i.e. no reservation price) would be the best alternative is only valid if the system is sufficiently flexible in the short run and information on bids and demands is distributed extremely rapidly in the market place. Given the existing trading structures in Germany and Europe with a separation of trading platforms (power exchanges) and grid operation, the latter condition is difficult to achieve. In a pool market, coordinated by an ISO as in the U.S. with centralized information, such a real-time market becomes possible. As pool markets with a security constraint central dispatch approach do not separate between decentralized and centralized coordination stages, the issue of inadequate linkage and related incentive problems do not arise. However, a fundamental switch from a bilateral market approach to a pool market should be motivated by more than correcting issues in the balancing mechanism.

9 Conclusion

This paper studies the incentives for balancing groups to deliberately under and oversupply their expected load commitments. The market design of the balancing mechanism in Germany provides opportunities for strategic behavior depending on the spot price. Empirical market data mirror the expected pattern very well. This suggests that market participants use stochastic arbitrage opportunities between the spot markets and the real-time one-price balancing mechanism. The considerations depend on the expected difference between spot and balancing prices as well as on the expected direction of the imbalance of the control zone. The outcomes are relatively simple. The higher the spot price, the higher the expected pay-off from an undersupply strategy and the stronger is the expected undersupply, and vice versa. This leads to increased and even structural imbalances of the electricity system.

The anti-abuse clauses currently used by the TSOs seem to be not sufficiently effective. The strategic misuse of the balancing energy artificially inflates the need for reserve capacity and increases the probability of system instabilities and required load shedding. The implied costs are significant and estimated to be in a range of €200–300m per year. These costs, placed on consumers, are too high and the security concerns are too severe to be ignored.

Various options for amending the market design and reducing the incentives for the strategic behavior have been discussed. They focus on better alignment between the balancing mechanism, the reserve capacity and spot markets. The existing disconnect between these three markets/mechanisms is not only a problem in Germany but also generally in all jurisdictions with a market design based on decentralized bilateral markets (as opposed to a centralized pool market design).

At first sight, our conclusions differ significantly from those drawn by Moeller et al. (2011) and Vandezande et al. (2010). Yet, both recognize that balancing mechanisms may induce incentives for market arbitrage in a very similar way as this paper does. The differences may be explained by looking in detail at the setup of the different analyses.

Studying the German balancing mechanism with time-series analysis, Moeller et al. (2011) indirectly find strategic arbitrage positions, the same ones we identified when studying the incentives. Their conclusions that the current balancing mechanism allows market arbitrage, reduces price peaks in the spot market, diminishes the ability to exploit market power, and thus effectively contribute to a functioning electricity market is undisputable in a general context. However, the balancing mechanism cannot supply unlimited energy. The potential supply of balancing energy comes with reservation costs that add external costs to the arbitrage mechanism. This changes the conclusion as these external costs likely outweigh the arbitrage gains (cf. Sect. 7). The increased need for incremental reserve capacity to keep the same security level means that inframarginal generation capacity is not fully utilized. This underutilization is the main source of market inefficiencies caused by the current market design and strategic arbitrage behavior.

Vandezande et al. (2010) discuss different balancing designs by the use of simplified examples. They compare a one-price system with symmetric balancing prices to a two-price system based on spot prices with asymmetric penalties. Given that the ex-ante expected imbalance costs correspond to the day-ahead price, there is no incentive for strategic behavior in their setting, whereas the asymmetric two-price system does induce strategic behavior. At first glance, their conclusion that a two-price system should be avoided seems essentially opposite to some of our findings, but essentially the arguments are similar, only the settings investigated differ. The common conclusion is that ex-ante expected balancing prices have to be in line with spot prices in order to avoid systematic over- and undersupply.

By investigating the specific incentive structures in the German balancing market, this paper complements the analyses by Moeller et al. (2011) and Vandezande et al. (2010) and provides important contributions and insights on the subject of effective balancing mechanisms.

Appendix

See Fig. 11.

Fig. 11

Actual imbalances in the respective control zones 2006–April 2011