1 Introduction

In liberalized power systems, power producers and users (suppliers and retailers) exchange electricity at the wholesale level in markets that close before real-time delivery, the so-called Day-Ahead (DA) markets. However, in real time, it occurs that the amount of electricity effectively injected (or withdrawn) differs form the scheduled amount. Hence, an imbalance occurs. In this case, the Transmission System Operator (TSO)Footnote 1 must intervene with balancing servicesFootnote 2 to maintain the stability of the system. The TSO does so by calling additional electricity generators to supply the electricity that was lacking or curtailing the excess supply, and charging the cost of the imbalance to the subject that created it.

Balancing services can be exchanged in explicit markets between balancing services providersFootnote 3 and TSOs. This is what happens in the real-time German balancing power markets (Ocker and Ehrhart [39]) and in the Italian balancing services markets (called Dispatching Services Market—MSD). In the latter, balancing services (in particular FCR—Frequency Controlled Reserves, FRR–Frequency Restoration Reserves and RR—Replacement Reserves) are exchanged between qualified sellers and the TSO through a set of explicit auctions, and priced with a double-pricing system. In the last-mentioned, the price of the DA market sets the floor for the prices of the negative imbalances (due to excess load or lack of supply in real time) and the price cap for the prices of positive imbalances (the excess supply of electricity or the reduction in the real-time load). From a theoretical point of view, there are both arguments in favour and against the hypothesis of price convergence of DA and balancing prices in markets with a double-pricing system. For the former, Cretì and Fontini [13, Ch.11] show that under perfect competition DA and balancing prices converge to the marginal cost of production. However, prices can diverge for two reasons. On the one hand, the existence of market power by some power producers can drive away prices from marginal cost. Therefore, if in the balancing market power producers have more market power than in the DA one (for instance because of local transmission constraints or due to the existence of some technical or regulatory constraints that reduce market participation), the prices of the two markets might not converge. On the other hand, it is also possible that power production costs are non-convex due to technical constraints (such as start-up or rump-up costs, see for instance Cretì and Fontini [13, Ch.2]), and because of them, the provision of balancing services can be more costly than power production.

The empirical literature has provided mixed evidence about price convergence of DA and balancing prices. Some authors have shown price convergence focusing on US markets (Borenstein et al. [7], Arciniegas et al. [1], Longstaff and Wang [31], Jha and Wolak [28]). However, there are considerably fewer studies for European markets. Boogert and Dupont [6] study the profitability of trading strategies across the DA and balancing market in the Netherlands. Asan and Tasaltin [2] explicitly measure the impact of the introduction of dual pricing rule on the price convergence in Turkey. A related stream of literature focuses on balancing prices and the role that factors such as strategic behavior of agents, balancing market design, and market structure, can have on them. The price convergence can be influenced by the strategic behavior of agents acting in the balancing market, that can exploit their market power by strategically withholding capacity (Heim and Goetz [25]). Market power can be enhanced by the design of the balancing market, such as auction formats, settlement rules, limited participation (Ocker et al. [40, 41], Muesgens et al. [33]). The market structure can also influence price convergence, and in particular the role played by Renewable Energy Sources (RES). The impact of RES on DA prices has received a vast attention (see, among others, Gelabert et al. [20], Mauritzen [32], Mulder and Scholtens [34], Sapio [43], and Woo et al. [49]). A more recent stream of literature has focused on the institutional design of balancing under increasing RES penetration (Hirth and Ziegenhagen [26], Ocker and Ehrhart [39], and Brijs et al. [8]) and on the condition for RES to participate to balancing markets (Sorknæs et al. [46], Fernandes et al. [19], and Müsgens et al. [35]). Gianfreda et al. [23] study the impact of RES generation in the Italian DA, intraday and balancing prices. Gianfreda et al. [21], evaluates the impact that RES penetration had on the balancing costs for the Italian TSO.

In this paper, we empirically assess the existence of a long-run relationship between DA and balancing prices in the continental Italian electricity market, which is divided in four zones: North (NO, in Italian Nord), Central-North (CN, in Italian Centro-nord), Central-South (CS, in Italian Centro-sud), South (SO, in Italian Sud).Footnote 4 To carry out our study, we need to take into account the seasonal nature of power prices. Electricity prices are subject to a complex seasonal structure, at the daily, weekly and annual frequency. There is a large stream of literature focusing on the seasonality of wholesale electricity prices (see Weron [48], Caporin et al. [9], Janczura et al. [27], Nowotarski and Weron [38], Uniejewski et al. [47], among many others). We take seasonality into account in the empirical analysis evaluating the characteristics of the deterministic patterns of both DA and balancing prices. We first compare, with a descriptive view, the periodic patterns in the two markets in each zone, pointing out similarities and differences. Then, we apply a filtering methodology that allows to remove the periodic components from the data and later focus on the analysis of seasonally adjusted prices, to verify if they converge to a common long-run trend.

From an econometric perspective, price convergence calls for the presence of cointegration, i.e. the presence of common stochastic trends or, equivalently, the existence of long-run relationships between price series. We proceed in steps and first discuss the dynamic properties of the seasonally adjusted DA and balancing services prices in all zones. Our analysis shows that these prices are not characterized by unit roots, thus excluding the possible presence of cointegration in the classic sense, that is associated with the long-run equilibrium between non-stationary stochastic processes characterized by unit roots. However, all the price series (filtered from the periodic patterns) show evidence of long-range dependence (or long memory), that is an high and significant correlation between observations distant in time. Hence, we cannot exclude the possible presence of fractional cointegration (see Robinson and Yajima [42] and Johansen [29] among others). The latter feature allows for the presence of a long-run link among price series that have long memory. Therefore, we first estimate the memory properties of the price series, and then determine if the latter are fractionally cointegrated.

We show that the wholesale and the balancing markets are linked in the long-run; however, each zone has its specific behavior. The evidence of price convergence is stronger for the NO zone, less so for CN and SO, while there is evidence of divergence between the series in CS. To further investigate the dynamics of convergence over time, if any, we study how the price difference between the series evolves in each zone throughout yearly rolling windows. We show that in NO and CN zones the average price difference converges to zero, even though in an unstable way. In the SO zone it quickly converges to zero in the latest period, while in zone CS it tends to diverge over time. Overall, the zone that shows the strongest price convergence is the NO zone, followed by CN and, more recently, SO. However, in CS there is evidence of price divergence, which calls for further investigations of the price dynamics, and suggests a careful assessment of the behavior of power producers in this zone. Overall, our analysis shows that even in markets that share the same regulation and common institutional factors, local specific factors that can be related to the structure of the grid are the key elements that affect price convergence.

The paper is structured as follows. In Sect. 2, we present the main features of the Italian DA and balancing services markets. In Sect. 3 data is discussed and analyzed. Section 4 introduces the methodological approach followed. Results are presented in Sect. 5. Policy implications are discussed in Sect. 6. References follow. Furthermore, a supplementary document contains additional empirical results.

2 The Italian DA and balancing service markets

2.1 The market design of the DA and balancing services markets

The Italian Power Exchange (IPEX), managed by the Gestore del Mercato Elettrico (GME), is organized in several markets, depending on products delivered and on the time horizon of the delivery. For the purpose of this analysis the relevant markets are the following:

  1. a)

    The DA Market (Italian acronym MGP, Mercato del Giorno Prima), where producers, wholesalers, and eligible final customers may sell/purchase electricity for each hour of the subsequent day;

  2. b)

    The dispatching services market (Italian acronym MSD, Mercato del Servizio di Dispacciamento—Dispatching Services Market),Footnote 5, Footnote 6

2.2 Price determination in the MGP

In the MGP generators participate making offers at plants level. With the exception of plants with production larger than 10 MW, the offers of RES-sources are grouped by GSE (the Italian public company managing all activities related to RES) and are submitted at zero prices to the market. These have priority dispatch. Only a subset of plants that participate to the MGP are allowed to participate to the MSD namely, the large thermal and hydro power production plants with size above 10 MW.Footnote 7 In the MGP there are uniform auctions that fix the system marginal price at each hour. The winning bidders receive the system marginal price of the zones in which they are located. The load pays a weighted average, namely, the average of the (possibly) different zonal prices weighted by the volume of effective exchanges. This is called Single National Price (Italian acronym PUN Prezzo Unico Nazionale).

2.3 Price determination in the MSD

A relevant difference between the MGP and the MSD is related to the equilibrium pricing rule in the auction. The equilibrium pricing rule of the MSD is a pay-as-you-bid-rule, also termed discriminatory auction. Companies receive the price they have offered/demanded, if their offer to sale/purchase balancing services to/from the TSO has been accepted. A double-settlement system is in place.Footnote 8 When power plants do not provide the amount of power that was scheduled in the MSD (and eventually adjusted in the intraday markets) an imbalance arises. Power producers can therefore make offers to increase or reduce power to cope with imbalances. The plants can sell energy to the TSO in the MSD. These are called sales offers, or up-regulation offers. Since producing more energy typically implies extra costs, the power producers can require an extra remuneration on top of the one that arises from MGP. This is the price of the up-regulation, which is paid by the TSO and received by the producer if the bid is accepted in the MSD. The double-settlement rule specifies that this energy cannot be priced less (but can be priced more) than the system marginal price of the MGP. Power plants can also sell to the TSO offers to reduce production, called purchase offers or down-regulation offers. The TSO will accept them if it faces, for instance, an imbalance due to an excess supply of power. Power producers pay the TSO in order to reduce the production from the levels that were offered and accepted in the MGP. By doing so, they receive the full payment for the energy that was scheduled in the MGP, regardless of whether it is fully produced or not, save the cost of the energy that they do not supply in real time and pay the price of the down-regulation offer accepted in the MSD for the amount of energy they do not produce. The double-settlement system implies that the price of the down-regulation cannot be higher (but can be smaller) than the system marginal price of the MGP. The competition in the balancing market across power suppliers is expected to bring the prices of the up and down offers to converge to the system marginal price. The TSO receives the payments for the accepted down-regulation offers, while it pays for the accepted up-regulation ones. Therefore, the price difference between the up-regulation and down-regulation offers measures the net payment that the TSO makes to the producers (if positive) or that receives from them (if negative) for the energy that is exchanged in the MSD. In a given hour and zone it can happen that the payments for all the accepted up-regulation offers do not exceed the ones for the down-regulation ones; however, normally, over a sufficiently large period of time, the up-regulation exceeds the down-regulation.

The prices at MSD are settled for every hour of the day and for every zone of the Italian electricity market. Each offer that is accepted in the MSD is priced at its own price. Therefore, no single price arises at the MSD level. However, the market operator provides data of the weighted average price of accepted up-regulation offers and of accepted down-regulation offers constructed by weighting the price of each accepted bid by the corresponding amount of power. Hence, to compute the net social cost of balancing in a given hour and zone, we calculate the weighted average of all the up- and down-regulation offers, for every hour including both the MSD ex-ante and the MB. The up-regulation has a positive sign, while the down-regulation a negative one. The algebraic sum of the (weighted average of all accepted) offers represents the effective cost for the electricity system of the provision of balancing services that are needed because of aggregated imbalances in a given hour and zone. This net imbalance price corresponds to the imbalance cost due to the differences between the scheduled quantities and the quantities needed by the TSO to maintain the system balanced. In other words, it represents the social costs (for the electricity system users) of having the electricity system balanced by the TSO.

2.4 The Italian market zones

Both the MGP and the MSD have a zonal configuration. There are six market zones: four continental (NO, CN, CS, SO) plus the two islands of Sicily (SI, in Italian Sicilia) and Sardinia (SA, in Italian Sardegna). The islands have a limited interconnection with the continent and a weak impact on the overall power exchanged. Furthermore, institutional features make the quality of the data on the balancing market very poor, thus challenging the validity of the statistical analysis.Footnote 9 We have therefore excluded Sicily and Sardinia from the present analysis.

On top of the market zones, there exist limited production poles. They are defined as production units with no load that are interconnected to the grid and whose production cannot be exported due to limited interconnection capacity. Although the units in these poles are physically located in a region belonging to a given market zone, they are not included in that. Hence, we do not include limited poles in the analysis. Note that when the transmission constraints are removed the limited poles cease to exist and their thermal capacity is merged into their corresponding market zone. This was the case of the limited production poles of Foggia and Brindisi, which are located in the region of Puglia in SO, but that were not included in SO until the end of 2018.

Italian market zones share a common institutional framework, i.e., common pricing rules and market design. Moreover, in these zones power producers have access to the same markets for primary energy sources, in particular natural gas and coal.Footnote 10 However, there are different structures in terms of load and power supply across zones. Previous results in the literature Bigerna et al. [4] show that a high RES penetration can enhance market power, which can impact on DA and balancing prices. Italian zones exhibit different degree of RES penetration. Table 1 below displays the scheduled energy at the MGP level in year 2018 per type of generation (different RES and thermal) in each of the four continental zones of Italy. The distribution of energy generated from RES differs between zones NO, CN and CS on the one hand, and zone SO on the other hand. Most of thermal capacity physically located in the regions of SO was excluded from it and grouped in the limited production poles of Brindisi and Foggia up until the beginning of year 2019. For the other zones, it is worth noting the different share of wind energy, which is high in CS, as well as the relevant share of the small-scale RES. The latter category includes plants of size smaller than 10 MW, which receives incentives and that make offer through a purposely-built public company. It is mostly composed of small-scale hydro in NO, of both small scale hydro and small photovoltaic (PV) power plants in CN, and mostly of small PV in CS and SO.

Table 1 Percentage of energy generated at the MGP level in each zone.
Full size table

The four continental zones of the Italian electricity market can provide an interesting case study to assess the differences in the degree of convergence between MGP and balancing prices. In particular, it is of interest to investigate if the power supply structure and the distribution of RES across zones in Italy is reflected in the degree of convergence between MGP and balancing prices in each zone.

3 Data analysis

3.1 Data description

Fig. 1
figure 1

Time series of MGP. The figure reports the time series of the MGP prices for the four zones, North (NO), Central-North (CN), Central-South (CS), South (SO). The unit on the y-axis is expressed in Euro/MWh

Full size image
Fig. 2
figure 2

Time series of MSD. The figure reports the time series of the MSD prices for the four zones, North (NO), Central-North (CN), Central-South (CS), South (SO). The time series have been winsorized (at the 0.1% and 99.9% quantiles) to reduce the effect on the y-scale of outliers and increase readability. The unit on the y-axis is expressed in Euro/MWh

Full size image

We use publicly available data provided by GME on its website. The prices are hourly, zonal, ranging from 1st January 2010 to 31st August 2019, for a total of 84,720 observations for each zone in each market. Figures 1 and 2 report the time series of MGP and MSD for the four areas. From a visual inspection of Fig. 1, it appears that MGP and MSD have a mean-reverting and stationary behavior, with MSD displaying larger dispersion around the mean. Furthermore, by looking at the same price series in the four different areas, we notice common dynamic patterns, which will be studied in Sect. 4 in terms of fractional cointegration. Table 2 reports descriptive statistics for MGP and MSD prices by zone. There are clear differences between MGP and MSD. Zones are quite different in terms of price values, as well as with respect to the presence of zeros or negative values. For what concerns the MGP prices, the median values are around 55 Euro/MWh in all cases. Instead, we observe larger differences between zones for the MSD. First, the median values highly differ across zones and also in relation with the median MGP value. In the NO zone, the median is 50% higher than the corresponding MGP value, while in the CN zone the MSD price is only slightly higher than the MGP price. On the contrary, the MSD price is almost twice as much as the MGP price in CS. This is due to the frequent need of costly up-regulation. For the SO zone, the median MSD price is zero, which is associated with the large fraction of zeros included in the data on SO, except for very recent periods. Note that negative figures arise whenever the overall amount of payments for up-regulation are outweighed by the ones for the down-regulation. This occurs if the zone is long on power.

Table 2 Descriptive analysis of MGP and MSD prices
Full size table

Negative prices can be observed in a limited number of cases, less than 2% for NO, while for CN and CS, the percentage of negative prices reaches much larger frequencies, about 21% and about 12.5%, respectively. This signals the fact that, in the observed period, these zones went long more frequently than the others, and power producers are less willing to reduce their scheduled programs. The fraction of zero prices is also a relevant quantity, as the distribution of zeros across zones shows in which zones dispatching services were less used in the sample period. NO is the only zone without zero prices in the sample. Recalling that a zero price signals that balancing services are not needed in that hour and zone (and therefore have null value), it follows that NO needs a continuous balancing of power. Differently, zeros are a relevant fraction for CN (about 19%), and a more limited fraction of the sample for CS (about 6%). The table also shows that the MGP prices are very rarely equal to zero (negative prices are not allowed in the Italian day ahead market).

3.2 Seasonality

The seasonality in the MGP and MSD prices derives from the superposition of several cyclical patterns: the diurnal ones, due to the differences in electricity demand between day and night; the weekly pattern, with different demands during workdays and week-ends (with holidays usually behaving as Sunday); the yearly one, due to the alternation of seasons and summer breaks in the industrial activities. To study the level of temporal dependence in the time series of MGP and MSD, we look at the sample auto-correlation function (ACF), which measures the degree of correlation between observations at different time lag between them, e.g. 1 h, 2 h, 24 h (1 day). Figure 3 displays the ACFs of the MGP and MSD prices for the four zones and highlights their strong seasonal patterns.

Fig. 3
figure 3

ACF of MGP and MSD. The figure reports the ACF of the MGP and MSD prices for the four zones, North (NO), Central-North (CN), Central-South (CS), South (SO)

Full size image

To deal with the complex cyclical pattern we follow, among the various methods proposed in the literature, the approach by Bernardi and Petrella [3] that introduce a flexible exponential smoothing method to capture seasonal cycles in time series. Their model allows to deal with monthly, weekly and intra-daily patterns. Note that by adopting the method of Bernardi and Petrella [3] and given the existence of a yearly cyclical pattern in the series, the filtering procedure leads to a reduction of the length of the series by one year (the year 2010 in our case). We follow Bernardi and Petrella [3] and estimate the following model on the zonal prices. Let \(y_t\) be the series of interest (like MGP or MSD prices for a given zone), observed from \(t=1,2,\ldots T\) at a hourly frequency, then

$$\begin{aligned} y_t= \mu _{t-1} + \sum _{j=1}^{J} \lambda _j d_{j,t} + \sum _{i=1}^I x_{i,t} s_{i,t-24} + \varepsilon _t \end{aligned}$$
(1)
$$\begin{aligned} \mu _t= \mu _{t-1} + \alpha \varepsilon _t \end{aligned}$$
(2)
$$\begin{aligned} s_{i,t}= & {} s_{i,t-24} + \left( \sum _{l=1}^I \gamma _{i,l} x_{l,t} \right) \varepsilon _t, \quad i=1,2,\ldots I \end{aligned}$$
(3)
$$\begin{aligned} \varepsilon _t= & {} \sum _{i=1}^p \phi _i \varepsilon _{t-i} + \sum _{i=1}^q \theta _i\zeta _{t-i} + \zeta _t. \end{aligned}$$
(4)

The model includes several components. First, \(\mu _t\) is the long-run evolution of the series, the trend component, following a random walk plus noise specification. The variables \(d_{j,t}\) with \(j=1,2,\ldots J\) are monthly dummies taking value 1 if a given day belongs to month j, but note that we might set the monthly dummies such that we have \(J \le 12\) dummies, thus J different monthly intercepts. The collection of \(s_{i,t}\), \(i=1,2,\ldots I\) represents the latent cyclical component of the model. It captures the differences in the daily patterns across days of the week, with \(1 \le I \le 7\) different patterns. Note that each \(s_{i,t}\) follows a daily seasonally integrated process with a multiplicative error term. In the latter, the variables \(x_{l,t}\) for \(l=1,2,\ldots I\) are dummies taking value 1 if the observation at time t falls within one of the I intra-weekly cycles. The error term \(\varepsilon _t\) follows an Auto Regressive Moving Average (ARMA) process whose innovations are assumed to be distributed as a Normal with mean zero and unit variance. Finally, \(\lambda _j\), \(j=1,2,\ldots J\), \(\alpha\), \(\gamma _{i,l}\), \(i,l=1,2,\ldots I\), \(\phi _i\), \(i=1,2,\ldots p\), and \(\theta _i\), \(i=1,2,\ldots q\) are coefficients to be estimated.

For details on the implementation and estimation of the model we refer to Bernardi and Petrella [3]. In our analysis, we set \(I=5\) different day types, setting Tuesday, Wednesday and Thursday to share the common intra-daily periodic cycle. In terms of monthly dummies, we adopt the same approach of Bernardi and Petrella [3] that consider the electricity demand in Italy from 2004 to 2014, and consider five monthly patterns, \(J=5\), where the first group of months includes January, March, June, September and October, the second group comprises November and December, April and May constitutes the third group while February and July the fourth. Finally, August is separately considered since it is the summer holiday period in Italy and several working activities are suspended. Similarly to Bernardi and Petrella [3], we also separately consider irregular days (holidays). For the innovation term, we specify a simple autoregressive process of order 1. Once the parameters and the components of the model are estimated (hats identify estimated quantities), the seasonally adjusted (filtered) series are computed as

$$\begin{aligned} {\tilde{y}}_{t}=y_t - \sum _{j=1}^{J} {\hat{\lambda }}_j d_{j,t} - \sum _{i=1}^I x_{i,t} {\hat{s}}_{i,t-24}, \end{aligned}$$
(5)

where we remove the cyclical behaviors only, while maintaining the long-term component and the irregular component.

The empirical ACFs of the seasonally adjusted series, reported in Fig. 4, show evidence of two phenomena. First of all, the filtered prices of MSD (and to a lesser extent also of MGP) display some residual seasonal behaviour associated with the daily frequency, as highlighted by the mild periodic pattern of the ACFs, with an oscillation with a period of 24 observations (1 day). This suggests that some residual stochastic periodic component is still present in the filtered series. Secondly, all series display long-range dependence. Indeed, in all cases the ACF slowly decreases toward zero and it is still highly significant after 100 lags. This indicates that the adjusted price series might follow a stationary and predictable process with long memory and not, as usually expected for prices in financial markets, a random walk process; see, among many others (Fama [18]).

Fig. 4
figure 4

ACF of the filtered MGP and MSD. The figure reports the ACF of the filtered MGP and MSD prices for the four zones, North (NO), Central-North (CN), Central-South (CS), South (SO).The filtering has been performed following the method of Bernardi and Petrella [3]

Full size image

3.3 Long memory

The existence of common trends in prices points at the existence of a long-run relationship. In particular, the classic way to determine whether two or more series are linked in the long-run and to verify if there is an equilibrium relation between them (with non persistent deviations from it) is by means of the well known concept of cointegration, which is a key property of multivariate time-series data. In particular, consider a collection of time series that are individually non-stationary, that is, they evolve over time like random walks, i.e., I(1) processes. If there exists a linear combination of these series that evolves as a mean reverting process (i.e. that is an I(0) process or stationary), then the non-stationary series are said to be cointegrated (see the seminal contributions of Granger [24] and Engle and Granger [16]). The stationary process resulting from the linear combination (often called the error correction (EC) term) reflects the deviations from the long-run equilibrium between the I(1) series.

Unfortunately, the concept of cointegration has been originally limited to I(1) time series only. Thus, we first carry out the augmented Dickey–Fuller and Philips–Perron tests to verify if the filtered zonal MGP and MSD prices are unit root processes. The test statistics reported in Table 3 strongly reject the null hypothesis of unit root in all cases, thus excluding that the dynamics of the two series are likely to be generated by I(1) processes. Consequently, the prerequisite for the classic definition of cointegration is missing, i.e. the series are not I(1). However, such a finding does not completely exclude the possible presence of long-run links among the variables of interest. In fact, all series share a relevant feature; they are all characterized by strong persistence. This suggests that a specific form of long-run relation might exist, the one associated with the concept of fractional cointegration, which arises between series that are not I(1) but are nevertheless characterized by long-range dependence. The latter thus becomes a prerequisite for fractional cointegration.

Table 3 Augmented Dickey Fuller (ADF) test, Phillips–Perron (PP) test, and estimates of the memory parameters on the seasonally adjusted series
Full size table

As a first step, we proceed to the estimation of the long-memory parameter (d), representing the degree of persistence of the series. We follow the semiparametric approach of Shimotsu and Phillips [45] and Shimotsu [44], which is robust to deterministic terms. Table 3 reports the estimated memory coefficients, d. A significantly positive coefficient indicates the presence of long memory (or long-range dependence). In particular, if \(d<0.5\), the series is long memory but stationary. The semiparametric estimator of Shimotsu and Phillips [45] and Shimotsu [44] is defined in the frequency domain so that its asymptotic properties (bias and variance) depend on the number of frequencies used in the estimation, namely the bandwidth (\(m_d\)). Table 3 reports the estimates for two different bandwidth: in all cases the memory coefficient is positive, and in most of them, the memory coefficient is lower than 0.5 when \(m_d=T^{0.6}\), and slightly above 0.5 when \(m_d=T^{0.5}\). In general, the long memory parameters of MGP and MSD are very close, thus suggesting that the two series share the same level of long memory. Consequently, we state that all the zonal prices, filtered from the periodic patterns, display significant long memory and are stationary. Given that the estimated long-memory parameters are very close to each other we proceed with the estimation of a dynamic model coherent with both the presence of long-memory and the possible converge between markets.

4 The model

On the basis of the statistical evidence outlined above, we consider a fully parametric model coherent with the presence of a common stochastic trend with long memory, namely fractional cointegration. The goal is to shed further light on the long-run dependence between MGP and MSD in each zone. We adopt a fractional vector error correction model specification to study if the series of de-seasonalized hourly MGP and MSD prices are characterized by common trends in each continental Italian zone. The properties of a fully parametric specification for the analysis of fractionally cointegrated series have been studied by Johansen [29] and Johansen and Nielsen [30]. In particular, the asymptotic theory of the maximum likelihood estimator for the model parameters has been fully derived in Johansen and Nielsen [30], thus allowing proper inference on the estimated parameters. The model specification of Johansen and Nielsen [30] has been adopted by Caporin et al. [10] in the context of high-frequency financial data, by [5] to characterize the dynamics of the financial risk premia and by Dolatabadi et al. [15] in the context of commodity prices. More recently, Carlini and Santucci de Magistris [11] have illustrated a potential pitfall in the specification of Johansen [29] and Johansen and Nielsen [30], associated with the choice of the number of lags in the short run dynamics. Therefore, Carlini and Santucci de Magistris [12] proposed a slightly different version of the fractionally cointegrated model, namely the \(\hbox {FVECM}_{d,b}\), which is identified for any choice of number of lags and coitegration rank. The \(\hbox {FVECM}_{d,b}\) model is

$$\begin{aligned} \Delta ^d X_t =\xi +\alpha \beta ^\prime \Delta ^{d-b}L_b X_t+\sum _{i=1}^k \Gamma _i \Delta ^d L^i X_t +\varepsilon _t \quad \varepsilon _t \sim iid (0,\Omega ), \end{aligned}$$
(6)

where \(X_t\) is a p-dimensional vector,Footnote 11\(\alpha\) and \(\beta\) are \(p\times r\) matrices, where r defines the cointegration rank, while \(\xi\) denotes the unrestricted intercept. \(\Omega\) is the positive definite covariance matrix of the errors, and \(\Gamma _j\), \(j=1,\ldots ,k\), are \(p\times p\) matrices loading the short-run dynamics. \(\varepsilon _t\) is the i.i.d. error term with finite eight moment, see Johansen and Nielsen [30]. The operator \(L_b:=1-(1-L)^b=1-\Delta ^b\) is the so called fractional lag operator, which, as noted by Johansen [29], is necessary for characterizing the solutions of the system. The model in (6) has k lags and \(\theta =vec( d,b,\xi ,\alpha ,\beta ,\Gamma _1,\ldots ,\Gamma _k,\Omega )\) is the parameter vector. The parameter space of the model is \(\Theta =\{\xi \in {\mathbb {R}}^{p}, \alpha \in {\mathbb {R}}^{p \times r}, \beta \in {\mathbb {R}}^{p \times r}, \xi \in {\mathbb {R}}^p,\Gamma _j\in {\mathbb {R}}^{p \times p},j=1,\ldots ,k, d\in {\mathbb {R}}^+, b\in {\mathbb {R}}^+,d\ge b>0, \Omega >0 \}\), where r is the cointegration rank, such that \(p-r\) determines the number of common stochastic trends between the series. We apply the model in (6) to zones NO, CN, CS, SO. We then consider several model specifications designed to verify convergence between markets at the single zone level. The existence of convergence is associated with the existence of a unique common trend, which requires the existence of one cointegrating relation. In other words, under cointegration, there is a unique long-run equilibrium (attractor) towards which the two series converge to.

5 Estimation results

5.1 Full sample analysis

We consider the \(\hbox {FVECM}_{d,b}\) model for each pair of MGP and MSD (seasonally-adjusted) price series in each of the four zones. We use the full-sample of 75,960 hourly prices from 1st January 2011 to 31st August 2019, and we estimate the following \(\hbox {FVECM}_{d,b}\) model

$$\begin{aligned} \begin{bmatrix} \Delta ^d MGP_t^{i}\\ \Delta ^d MSD_t^{i}\\ \end{bmatrix}= \begin{bmatrix} \xi _{1} \\ \xi _{2} \\ \end{bmatrix} + \begin{bmatrix} \alpha _{1} \\ \alpha _{2} \\ \end{bmatrix} L_b EC_{t} +\sum _{j=1}^{k^{*}} \Gamma _j \Delta ^{d} L^j Y_t + \begin{bmatrix} \varepsilon _t^{MGP,i}\\ \varepsilon _t^{MSD,i}\\ \end{bmatrix} \qquad \end{aligned}$$
(7)

where \(Y_t=[MGP_t^{i},MSD_t^{i}]'\) for \(i=NO,CN,CS, SO\), and the error correction term is \(EC_t=\Delta ^{d-b} MGP_t^{i}+\beta _{2} \Delta ^{d-b}MSD_t^{i}\).

Table 4 reports the estimation results for fractional cointegration between MGP and MSD, in each of the four zones. The estimates of the \(\hbox {FVECM}_{d,b}\) signal that the strength of the cointegration relation in terms of memory gap is maximal, as \(d=b\) in all cases. This means that the Error Correction (EC) term is short memory. In addition, the Likelihood Ratio (LR) test for fractional cointegration identifies the presence of cointegration in three of the four zones. The only exception is the CS zone, for which we reject the hypothesis of fractional cointegration. The estimated models are similar in terms of lag length (\(k^*\)), with NO, CN and CS zones characterized by two lags and SO by four lags. We attribute this difference to the larger presence of zeros in the SO time series. The intercepts, \(\xi _1\) and \(\xi _2\), are statistically significant for all zones. The parameter \(\beta _2\) of the NO zone is the closest to − 1, while for SO and CN zones it takes slightly lower values. Finally, it is positive and larger than 1 for CS. Thus, the result shows that for the NO zone, a rise of one Euro per MWh in the MSD in the long-run is coupled with a rise of 0.83 Euro per MWh in the MGP. In other words, there is a price difference between MSD and MGP of almost 20 cent per MWh whenever price rises in both markets. This differential, which signals the difference in the cost of electricity exchanged in the MSD vis-a-vis the one in MGP, is slightly higher in CS and SO. In zone CS, data do not show evidence of fractional cointegration and the \(\beta _2\) coefficient cannot be meaningfully interpreted. However, the absence of cointegration might signal the existence of divergent behaviors in the MGP and MSD prices (for the CS zone), in the sense that a rise of one euro in the MGP implies a more than proportional fall of MSD.

Table 4 \(\hbox {FVECM}_{d,b}\) estimates (Est.) for the pairs of MGP and MSD of the four main regions (NO,CN,CS,SO)
Full size table
Fig. 5
figure 5

EC term of the four zones. The figure reports the time series for the EC term of the MGP and MGP prices for the four zones, North (NO), Central-North (CN), Central-South (CS), South (SO)

Full size image

We also look at the estimates of the speed-of-adjustment parameters, \(\alpha\). Despite all parameters \(\alpha\) are statistically significant or marginally significant, only the MSD prices significantly move to restore equilibrium. The adjustment is larger for NO and CN while it is much weaker for SO. For CS, the absence of cointegration does not allow interpreting the speed of adjustment parameters. Overall, the evidence suggests that MGP and MSD have common dynamics within NO, CN and SO zones. This result is in favor of price convergence, although for CN and CS the evidence is weaker than for NO. Finally, Fig. 5 reports the error correction terms \(EC_t^i\) of Eq. (7) for \(i=NO,CN,CS,SO\).Footnote 12 We find evidence of a reduction in the persistence over the EC terms compared to what observed among the seasonally adjusted series. This is coherent with the model feature, the presence of fractional cointegration and the associated convergence.

5.2 Dynamic analysis

We study how the average difference of the two price series, namely MGP and MSD, changes over time. In particular we investigate if the price difference between MGP and MSD is likely to shrink (or to widen) over time. This allows us to shed further light on the behavior of the price converge. We perform this analysis by means of a rolling estimation of the average price difference based on the following linear time-series regression

$$\begin{aligned} D_{t}^i=\alpha _j^i+u_t^i, \quad t=1,2,\ldots ,8760 \end{aligned}$$
(8)

where \(D_{t}^i=MGP_t^i-MSD_t^i\) for \(i=NO,CN,CS,SO\) and \(\alpha _j^i\) represents the average price difference in the j-th subperiod of 1-year length \((24\times 365=8760)\) for the i-th zone. The estimation of \(\alpha _j\) is carried out by rolling Ordinary Least Squares (OLS) regression with step equal to 1 day (24 h), leading to \(J=2800\) estimates, which are plotted in Fig. 6 together with the 95% confidence interval.

Fig. 6
figure 6

Average difference MGP-MSD for the four zones. The figure reports the parameter estimates obtained from the rolling OLS regression of the difference between MGP and MSD prices on a constant. The length of the estimation window is 1 year (8760 observations). The black-solid line is the point estimate, and red-dotted lines denote the 95% confidence interval obtained with Newey and West [36] robust standard errors. The unit on the y-axis is expressed in Euro/MWh

Full size image

We note that in the NO zone the average price difference tends to zero, i.e., the prices converge over time. This trend is clearer from 2017 onward. A similar consideration applies for the CN zone, even though the converging trend has been more unstable, with periods of converging trends followed with diverging ones; yet, over time the price difference tends to zero. Zone SO shows a clear indication of convergence over the last part of the sample (from the beginning of 2019), that followed a first phase in which the price difference was rather constant and high. This can be explained considering that in the SO zone, the limited production poles of Brindisi and Foggia ceased to exists and the power units were included into the SO zone from beginning of year 2019, after the elimination of relevant bottlenecks. Before this period, the capacity that was located in the area of Brindisi and Foggia (which was the largest share of thermal capacity of the Regions that pertain to the SO zone) was kept separate from the SO zone. Thus, the SO zone has started exhibiting a relevant activity in MSD after the incorporation of these limited production poles. The only diverging trend is for CS, where the price difference tends to widen over time. The dynamic analysis confirms the finding of the fractional cointegration analysis on the full sample displayed before. The gap between the MSD and the MGP prices in the CS zone has been widening over time.

6 Policy implications and conclusion

In this paper, we have been focusing on the convergence between DA and balancing prices in the four continental price zones of Italy. To shed light on this aspect, we first construct a price index for balancing services which measures the net cost of those services for the TSO. Then, in order to assess the possible long-run correlation hypotheses, we investigate the statistical properties of the time-series and seasonally adjusted them focusing on the statistical properties of the structural component of the series. Afterward, we test the existence of common long-memory of DA prices and balancing prices, and show that DA and balancing price series have been characterized by converging dynamics within each zone, except for the CS zone, which has exhibited a price diverging path.

It is a general concept in economics that markets are efficient if there are no arbitrage opportunities. Our results of convergence indicate sufficient market efficiency, since prices in the two markets converge in the long-run and the average price difference tends to reduce over time. However, the findings differ across zones. Recall that the share of installed RES are almost the same in CN and CS. Also for NO, even though it is the zone with the highest share of hydro, while the most different one in terms of power supply structure is SO. Yet, NO, CN and SO exhibit convergence (even though with a different pace) while CS exhibits a diverging pattern. There is a possible impact of types of RES in explaining price convergence. Wind power is higher in CS than CN and our result could signal that wind power, being less predictable than PV, brings higher need for balancing energy.Footnote 13 However, we conjecture that RES distribution is not the key explaining factor. Zone SO is the one that has the highest share of wind power, yet MGP and MSD prices are converging over time in it, while this is not the case for CS. There are other peculiarities that affect market convergence, that most likely have to do with the very definition of the zones. Each zone is defined on the basis of permanent congestion on transmission lines which limit transit across zones. However, there are also relevant congestion within zones that are reflected in the cost of balancing services but that are not apparent since these congestion do not give rise to a separate zone. The existence of local congestion within zones is a well-known characteristic of the Italian system; an example is the area of Naples, which is located in the CS zone, and that sees a limited number of producers that are deemed necessary by the TSO to maintain system stability. This situation clearly increases market power of local producers. In the design of the Italian market zones, some relevant local congestion are made apparent by means of limited production poles. This was the case of the limited poles of Brindisi and Foggia. The power units located in those places were incorporated into SO zone at the beginning of year 2019 upon the resolution of the local congestion, and from that period onward the SO zone has shown a quick tendency toward price convergence. On the contrary, local congestion in the CS zone could not be solved with a different market design since the area where this occurs is too big to give rise to a limited production pole yet too small to be considered as an independent market zones (since there are too few producers). Therefore, it seems that local congestion within the CS zone and the increased market power induced by them is what causes the market inefficiency of the CS zone.

Throughout the paper we have shown that there is a tendency towards price convergence in each of the continental zone of Italy but the CS zone. We have also evaluated the relative price difference of those zones where price are converging: we measured the difference in balancing versus DA electricity price, and shown that this average price difference is converging over time. Despite our study referred to the Italian market, we believe that our approach, far from being just an analysis of a given market, can be of interest for other markets as well. It shows a robust methodology that can be applied to evaluate market efficiency in terms of price convergence between DA and balancing markets. It also enables us to measure the inefficiency due to the average difference between cost of provision of electricity in real time and forecasted DA figures. Finally, it shows that even under a common institutional framework, the definition of the zone and the existence of relevant congestion within a zone is the crucial parameter that can explain market inefficiency better than the different structural composition of power supply. This latter point can be of relevant importance for policy makers, and in particular for market regulators and for the market surveillance activity. Regulators and policy makers should focus their activity on tackling grids’ bottlenecks as this seems to be the crucial parameter affecting competitiveness and price convergence. Monitoring agencies could use the methodology we propose here to have an indication about market price convergence (if any), possible local market power abuse and be aware of which balancing markets they should focus on in order to enhance market efficiency.