1 Introduction

Quantitative studies and mathematical models are necessary to understand the economic and strategic issues that define energy markets in the world. In that vein, the study of natural gas markets is particularly interesting because most of them, particularly in Europe, show a high dependence on a small number of producers exports. According to Mathiesen et al. (1987), this market structure can be analyzed with strategic interactions and market power. This market power can be exerted at the different stages of the gas chain: by the producers in the upstream market or the local intermediate traders in the downstream market. The European markets are also characterized by long-term contracts established between the producers and the intermediate local independent traders. These long-term contracts were initially designed as a risk-sharing measure between producers and local traders. They are usually analyzed, in particular, as a tool to mitigate the producers’ market power. The combination of strategic interactions and long-term contracts makes the study of the natural gas markets evolution particularly subtle and rich.

The economic literature provides an important panel of numerical models whose objective is to describe the natural gas trade structure. As an example, we can cite the “World Gas Trade Model” (Baker Institute) (Rice University 2004), the “EUGAS” model (Cologne University) (Perner and Seeliger 2004), the “GASTALE” model (Energy Research Centre of the Netherlands) (Lise and Hobbs 2008) or the “World Gas Model” (University of Maryland) [Egging et al. (2010), an extension of the work developed in Gabriel et al. (2005a, b)]. Other works include Gabriel et al. (2003), Aune et al. (2009), Boots et al. (2004), Egging and Gabriel (2006), Holz et al. (2008) and Brito and Rosellón (2010). However, most of these models present some necessary simplifying assumptions concerning either the description of the market economic structure or the demand function. For instance, the “EUGAS” model assumes pure and perfect competition between the players and thus neglects market power to allow a detailed description of the infrastructure. The “GASTALE” and “World Gas Model” depict strategic interactions between the players via a Nash–Cournot competition and the latter model also uses exogenous long-term contracts. However, the former model does not include investments in production or in pipeline and storage infrastructure. Besides, the demand representation for all these previous models does not explicitly take into account the possible substitution between different types of fuels (natural gas, oil, and coal, for instance). All these drawbacks have been analyzed in detail in Smeers (2008).

The model we develop, named GaMMES, Gas Market Modeling with Energy Substitution, tries to address some of the limitations proposed in Smeers (2008). It is also based on an oligopolistic approach of the natural gas markets. The interaction between all the players is a Generalized Nash–Cournot competition and we explicitly take into consideration, in an endogenous way, the long-term contractual aspects (prices and volumes) of the markets. Our representation of the demand is new and rich because it includes the possible substitution, within the overall primary energy consumption, between different types of fuels. Hence, in our work, we mitigate market power exerted by the strategic players: they cannot force the natural gas price up freely because some consumers would switch to other fuels.

We study both the upstream and downstream stages of the gas chain, while modeling the possible strategic interactions between all the players, through all the stages. The production side is detailed at the production node level and we choose a functional form derived from Golombek et al. (1995) for the production costs. We assume, in our representation that the producers sell their gas through long-term contracts to a set of independent traders who sell it back to end-users, where the Nash–Cournot competition is exerted. Storage and transportation aspects are taken care of by global regulated storage and transportation operators. Producers also have the possibility to directly target end-users for their sales. Both producers and independent traders share market power. The long-term contracts are endogenous to our model and this property (among others) makes our formulation a Generalized Nash–Cournot game. The introduction of non-symmetric independent traders that can exert market power in the spot markets and contract in the long-term with the producers, and are in an oligopolistic competition with them in the downstream induces a rich, double layer economic structure. This is a new feature of the description of the natural gas trade. It allows us to represent long-term contracts and mitigate the producers’ market power.

The demand side is also detailed. We use a system dynamics approach (Abada et al. 2011) in order to model possible fuel substitutions within the fossil primary energy demand of a consuming country, between the consumption of coal, oil, and natural gas. This approach allows us to derive a new and interesting mathematical functional form for the demand function that includes naturally the competition between these. This particular new feature of the gas markets description that we have introduced in our model induces a flexibility in the gas demand representation. It allows us, for example, to study the sensitivity of gas consumption and prices over the oil and coal prices.

We include all the possible investments in the gas chain (production, infrastructure, etc.) and make the long-term contracts’ prices and quantities endogenous to the model using an MCP (mixed complementarity problem) formulation.

The remaining parts of the paper are as follows: the first part is a general description of the chosen economic structure representation. All the players are presented and are divided into two categories: the strategic and the non-strategic ones. The strategic interaction is also detailed in this part. The second part presents the notation used and a brief description of a system dynamics approach to model the consumers’ behavior investment in coal, oil or natural gas so that their utility is optimized. The third part is dedicated to the mathematical representation of the markets: the optimization programs associated with all the strategic and non-strategic players are presented and discussed. We also explain in this part how we make the long-term contracts’ prices and volumes endogenous to our model. The next part is an application of our model to the European natural gas trade where the calibration process and the results are discussed. A comparison between our model, a more standard one where the demand does not take into consideration fuel substitution and the European Commission natural gas forecast is carried out in order to compare between the results. The last part summarizes the work.

2 The model

2.1 Economic description

Our description of the natural gas markets divides them into two stages.

The upstream market is represented by gas producers, each with a dedicated trader (export division) to sell gas to other traders or directly to end-users. An example would be Gazexport for Gazprom. The set of producers and dedicated traders is denoted as P.

Besides the market players just mentioned, there are a number of independent traders whose activity is to buy gas from the big producers (or their traders) and to sell it to the final users in the downstream market. This type of traders includes all the firms whose production is small, compared to their sales (e.g., EDF and GDF-SUEZFootnote 1). The associated index for these players is I.

The different target markets (the consumers) are divided into three sectors: power generation, industrial, and residential, represented respectively as D 1, D 2 and D 3. However, it is easy to demonstrate that if the sectors do not interact with each other (i.e., the different demand curves are independent), the study of only one sector can easily be generalized to the three. We will make the assumption that the different demand curves do not interact (as an example, the gas price in the industrial sector does not depend a priori on the residential price), which may not be realistic for some situations. Hence, to simplify our notation and modeling, we will consider only one consumption set D to represent each country’s gross natural gas consumption.

We assume that each dedicated trader can either establish long-term contracts with independent traders or sell his gas to the spot markets.

The first situation corresponds to a gas trade under a fixed, contracted price, not dependent on the quantities sold (in a first approximation). These quantities are also fixed by the contract. The second situation is characterized by the fact that the spot price is a consequence of the competition between all the traders in the downstream markets, via a specified inverse demand function.

The long-term contracts we consider are modeled as follows: each pair of producer-independent trader have to contract, if needed, on a fixed volume that must be exchanged each year, at a fixed price. We allow for seasonal flexibility within a year, for the low-consumption regimes. This description takes into account the basis of the long-term contracts’ Take-Or Pay-clauses (Hubbard and Weiner 1986). For computational reasons and to keep the model’s formulation simple, we do not allow for annual flexibility of the long-term contract volumes.

All the traders compete via a Nash–Cournot interaction, during a finite number of years Num. Time will be indexed by t ∈ T (five-year time steps) and we will take into account seasonality by distinguishing, for each year t, between the off-peak and peak seasons. The seasons will be indexed by M. They basically correspond to different demand regimes.

More precisely, the strategic interaction between the players is modeled as the following: the producers can sell their gas directly to the end-users in the spot markets, or to the independent traders via long-term contracts. The independent traders buy gas from the producers only via these long-term contracts and they can sell gas to all the possible spot markets. All the producers and the independent traders are strategic players. They are in competition in the spot markets where they exert market power. This situation is modeled using a Nash–Cournot competition. All the strategic players (producers and independent traders) see the same inverse demand function. All the markets are liberalized. Therefore, each producer can make contracts with all the possible independent traders and sell gas to all the possible spot markets. Similarly, an independent trader can make contracts with all the possible producers and sell gas to all the possible spot markets. Each trader can also store gas in all the possible storage nodes, if the storage capacity is sufficient.

The competition in the upstream is not represented as an oligopoly (unlike some models like Lise and Hobbs 2008). Indeed, we do not model the possible traders’ demand functions that can be considered, a priori, by the producers in their optimization programs. The upstream activity, which is dominated by long-term contracts, is modeled with a supply/demand equilibrium in the long-term between the producers and the independent traders. The corresponding long-term contract price is issued from the supply/demand equality constraints’ dual variables.

Since the model is dynamic, we need to take care of possible capacity investment. For infrastructure-related capacity, this corresponds to additional installed capacities. Regarding the production, we do not explicitly model exploration activities, because of a lack of geological data. Therefore, we assume that investments only increase the extraction capacity. We also make the model conservative as we do not endogenously consider possible additional reserves due to exploration activities. Therefore, a gas-producing firm may want to increase its production capacity by investing if this would lead to an increase of its revenue.

We take into consideration the depreciation of the production capacity in the upstream side of the market by introducing a depreciation factor per time unit at each production node: dep f . To simplify the model (and because of a lack of data concerns), we decided not to take into account the transport or storage capacity depreciations.

The main advantage of the GaMMES model is that it takes into account, in an endogenous way, long-term contracts between the independent traders and the producers. Obviously, this representation is quite realistic for the natural gas trade since the latter is still dominated by long-term selling/purchase prices and volumes. In 2004 the long-term contracts’ imports represented more than 46% of the European natural gas consumption and 80% of the total European imports (European Commission 2007; International Energy Agency 2004). Another advantage inherent to our description is that the inverse demand function explicitly takes into consideration the possible substitution between consumption for natural gas and the competing fuels.

Considering the energy substitutions in the natural gas demand mitigates the market power that can be exerted by all the strategic players in the end-use markets. Indeed, this is due to the fact that the consumers have the ability to reduce the natural gas share in their energy mixes if the market price for natural gas is much higher than the substitution fuel’s (such as oil and coal) price. Therefore, the producers may not have much incentive to reduce their natural gas production in order to force the price up. This model property allows us to take into account the natural gas price dependence on oil and coal prices. Indeed, the Nash–Cournot interaction will link the natural gas price to the coal and oil prices because of the demand function dependence on these parameters.

In order to take into consideration the intra and extra-European physical network of the transport and distribution networks, we need to introduce a pipeline operator whose role is to minimize the transmission costs over all the arcs of the topology. We denote by N the set of all the nodes including the production nodes, the consuming markets, and the storage nodes. Added to the transport cost minimization objective, the pipeline operator also has the possibility to make investments in order to increase the arc capacities, if necessary.

All the arc transport costs are exogenous to the model. The congestion prices are taken into consideration endogenously: they can be obtained by computing the dual variables corresponding to the infrastructure capacity constraint. The set of all these arcs is A. An arc can either be a pipeline or an LNG route.

In order to be able to meet high levels of consumption, we assume that the independent traders have access to a set of storage nodes to store natural gas in the off-peak season, and withdraw it in the peak one. Obviously, they have to support a capacity reservation, storage, withdrawal, and transport costs. All the storage nodes, indexed by the set S, are managed by a global storage operator player. This player can invest in order to increase the storage capacity of each storage node.

Both the pipeline and the storage operators are assumed not to have market power. The storage and transport costs are hence exogenous to the model. The strategic players are therefore the producers/dedicated traders and the independent traders. Obviously, this assumption is an important simplification of reality, where market power can also be exerted by the storage and pipeline operators. However, it is consistent with what can be found in the literature: Egging et al. (2010) and Lise and Hobbs (2008).

The storage cost, which is assumed to be supported by the independent traders, is represented using capacity reservation and storage/withdrawal costs. We consider that the average time for the storage investments to be realized is delay s years (five years). The situation is similar for the infrastructure (delay i ) and production capacity investments (delay p ) costs supported by the pipeline operator and the producers.

2.2 Notation

The units chosen for the model are the following: quantities in toe (i.e., Ton Oil Equivalent) or Bcm and unit prices in $/toe or $/cm. The following table summarizes the notation chosen for the exogenous parameters and the endogenous variables.

Exogenous factors

P :

Set of producers-dedicated traders

I :

Set of independent traders

D :

Set of gas consuming countries in the downstream market (no distinction between the sectors) D ⊂ N

T :

time T = {0,1,2, ..., Num}

M :

Set of seasons. Off-peak (low-consumption) and peak (high-consumption) regimes

F :

Set of all the gas production nodes. F ⊂ N

N :

Set of the nodes

S :

Set of the storage nodes S ⊂ N

A :

Set of the arcs (topology)

Rf f :

Production node f’s total gas resources (endowment)

Kf f :

Production node f’s initial capacity of production, year 0

Lf f :

Production node f’s maximum increase of the production capacity (in %)

Ic s :

Injection marginal cost at storage node s (constant)

Wc s :

Withdrawal marginal cost at storage node s (constant)

Rc s :

Reservation marginal cost at storage node s (constant)

Ls s :

Storage node s’s maximum increase of the storage capacity (in %)

Pc f :

Production cost function, production node f

Tc a :

Transport marginal cost through arc a (constant)

Tk a :

Pipeline initial capacity through arc a, year 0

Ks s :

Initial storage capacity at node s, year 0

Is s :

Investment marginal costs in storage (constant)

Ip f :

Investment marginal costs in production (constant)

Ik a :

Investment marginal costs in pipeline capacity through arc a (constant)

La a :

Arc a’s maximum increase of the transport capacity (in %)

O :

Incidence matrix \(\in \mbox{M}_{F\times P}\). O fp  = 1 if and only if producer p owns production node f

B :

Incidence matrix \(\in \mbox{M}_{I\times D}\). B id  = 1 if and only if trader i is located at the consumption node d

M1:

Incidence matrix \(\in \mbox{M}_{F \times N}\). M1 fn  = 1 if and only if node n has production node f

M2:

Incidence matrix \(\in \mbox{M}_{I \times N}\). M2 in  = 1 if and only if trader i is located at node n

M3:

Incidence matrix \(\in \mbox{M}_{D \times N}\). M3 dn  = 1 if and only if node n has market d

M4:

Incidence matrix \(\in \mbox{M}_{S \times N}\). M4 sn  = 1 if and only if node n has storage node s

M5:

Incidence matrix \(\in \mbox{M}_{A \times N}\). M5 an  = 1 if and only if arc a starts at node n

M6:

Incidence matrix \(\in \mbox{M}_{A \times N}\). M6 an  = 1 if and only if arc a ends at node n

H :

Maximum value for the quantities produced and consumed

We could have used different upper bounds for the different variables. However, to simplify the notation, we will use the same value H.

fl f :

Production node f’s flexibility: the maximum modulation between production during off-peak and peak seasons

min pi :

Percentage of the minimum quantity that has to be exchanged on the long-term contract trade between i and p

δ :

Discount factor

delay s,i,p :

Period of time necessary to undertake the technical investments

loss a :

Loss factor through arc a

dep f :

Depreciation factor of the production capacity at production node f

Endogenous variables

\(x^t_{mfpd}\) :

Quantity of gas produced by p from production node f for the end-use market d, year t, season m in Bcm

\(zp^t_{mfpi}\) :

Quantity of gas produced by p from production node f dedicated to a long-term contract with trader i, year t, season m in Bcm

\(zi^t_{mpi}\) :

Quantity of gas bought by trader i from producer p with a long-term contract year t, season m in Bcm

up pi :

Quantity of gas sold by producer p to trader i with a long-term contract, each year in Bcm

ui pi :

Quantity of gas bought by trader i from producer p on a long-term contract, each year in Bcm

\(y^t_{mid}\) :

Quantity of gas sold by i to the market d, year t, season m in Bcm

\(ip^t_{fp}\) :

Producer p’s increase of production node f’s production capacity, due to investments in production year t, in Bcm/time unit

\(q^t_{mfp}\) :

Production of producer p from production node f, year t, season m in Bcm

\(p^t_{md}\) :

Market d’s gas price, result of the Cournot competition between all the traders, year t, season m, in $/cm

η pi :

Long-term contract price contracted between producer p and trader i in $/cm

\(r^t_{is}\) :

Amount of storage capacity reserved by trader i at node s, year t in Bcm

\(in^t_{is}\) :

Volume injected by trader i at storage node s, year t in Bcm

\(is^t_s\) :

Increase of storage capacity at node s, year t due to the storage operator investments in Bcm/time unit

\(ik^t_{a}\) :

Increase of the pipeline capacity through arc a, year t, due to the TSO investments in Bcm/time unit

\(fp^t_{mpa}\) :

Gas quantity that flows through arc a from producer p year t, season m in Bcm

\(f\/i^t_{mia}\) :

Gas quantity that flows through arc a from trader i year t, season m in Bcm

\(\tau^t_{ma}\) :

The dual variable associated with arc a capacity constraint year t, season m, in $/cm. It represents the congestion transportation cost over arc a.

The table is divided into two parts. The upper half represents the exogenous parameters or functions whereas the lower half represents the different decision variables and the inherent retail prices.

The indices p, d, i, f, n, s, a, m and t are such that p ∈ P, d ∈ D, i ∈ I f ∈ F, n ∈ N, s ∈ S, a ∈ A, m ∈ M and t ∈ T.

The long-term contract between producer p and trader i fixes both a unit selling price and an amount to be purchased by the independent trader i each year from producer p. Both price and quantity will be specified endogenously by the model.

Matrix O is such that O fp  = 1 if producer p owns production node f and O fp  = 0 otherwise.

Figure 1 represents a schematic overview of GaMMES.

Fig. 1
figure 1

The market representation in GaMMES

2.3 The inverse demand function

We need to specify a functional form for the inverse demand function, which links the price p d at market d to the quantity brought to the market. Most of the natural gas models (Rice University 2004; Perner and Seeliger 2004; Lise and Hobbs 2008; Egging et al. 2010) do not take into account fuel substitution. Let h be the specific inverse demand function. We assume that the long-term contract quantities do not directly influence the market competition price, which is to say that \(p^t_{md}=h\big(\sum_i y^t_{mid}+\sum_f \sum_p x^t_{mfpd}\big)\). (Actually, this assumption is necessary to guarantee the concavity of the objective functions of each strategic player’s maximization problem, regardless of the quantities decided by the other competitors. Otherwise, this assumption can be dropped if linear functions are used).

As mentioned in the introduction, we want to capture the inter-fuel substitution in the fossil primary energy consumption. To be able to do so, we used a system dynamics approach that models the behavior of the consumers who have to decide whether to invest in new technologies that use either oil, coal or natural gas. Our model, based on the work presented in Moxnes (1987), is fully developed in Abada et al. (2011). Other theoretical works related to natural gas consumption include Abrell and Weigt (2011).

The System Dynamics approach aims at predicting the consumption of coal, oil, and natural gas observed at time t using both the historical and current values of fuel prices, and the history and current value of the overall demand for hydrocarbon fuels. In this model, the dynamics of interfuel substitution involves a distinction between the flow of freshly installed equipment, and the stocks of existing equipment that is represented by two vintages of capital. The model is based on a putty-clay framework and assumes that the choice of fuels can be freely adjusted ex ante, whereas no substitution is possible ex post. Using this decomposition, the model captures the irreversibility associated with the decision to install and operate a durable burning equipment.

The fuel options are indexed by an integer i and the fuel option coal (respectively oil, and natural gas) is labeled 1 (respectively 2 and 3). The fuel shares in the new burning equipment installed at time t are assumed to be determined by the relative cost of the three fuel options. The total cost C i of fuel option i is related to its market price, the associated payback time, capital and operating costs (related to its use), the price of CO2 and its emission factor.

The share s i of fuel option i in the new burning equipment is determined by the relative cost of the three fuel options. The following multinomial logit model is used:

$$ s_i=\frac{e^{-\alpha C_i}}{\sum_i e^{-\alpha C_i}} \text{ ,} $$
(1)

where α is a (non-negative) parameter, and C i are the total fuels costs. s i is a decreasing function of the fuel price P i . The validity of this logit model conceptually presupposes a “macroscopic” perspective, meaning that the energy system under scrutiny must contain a large enough number of individual decision-makers.

In this model, capital is measured in units of capacity to burn fuels (that is, in energy unit per unit of time). Thus, the total investment I represents the overall capacity of new burning equipment. The total investment in new equipment associated with the fuel option i is denoted I i and satisfies:

$$ I_i=s_i I \text{.} $$
(2)

Now we detail the dynamics of fuel substitution. As mentioned above, a vintaging structure is used to portray the aging process of installed equipment. Here, two vintages of capital are kept track of. Accordingly, two stock variables are defined for each fuel option i: the capacity of recently installed equipment, the “new” ones KN i , and those of the older ones KO i . Investment in new burners I i increases the capacity of the new equipment. New equipment becomes old after a use of half the lifetime: \(\frac{T_i}{2}\). Similarly, old equipment is scrapped after a use of \(\frac{T_i}{2}\) and the flow of scrapped old equipment DO i is assumed to be equal to \(\frac{KO_i}{T_i/2}\). With these assumptions, the dynamics can be formulated as follows:

$$ \frac{dKN_i}{dt} = I_i-\frac{KN_i}{\frac{T_i}{2}} \text{ ,} $$
(3)
$$ \frac{dKO_i}{dt} = \frac{KN_i}{\frac{T_i}{2}}-\frac{KO_i}{\frac{T_i}{2}} \text{ .} $$
(4)

For each fuel i at time t, the change in the overall stock of new equipment with respect to time is given by the inflow of new equipment associated with investment I i , and the outflow caused by aging. Similarly, the temporal variation of the stock of old burners results from the inflow of these previously new equipment, and the outflow corresponding to the scrapping of old equipment.

The next step is to model the dependence between the flow of total investment I and the overall stock of existing equipment. We can first define K i  = (KN i  + KO i ) the total capacity of installed burning equipment with fuel option i, and K the total capacity of installed burning equipment: K = ∑  i K i .

At time t, the overall capacity of scrapped equipment is:

$$ DO=\sum\limits_i DO_i = \sum\limits_i\frac{KO_i}{\frac{T_i}{2}} \text{ .} $$
(5)

Let’s call ED the overall demand for the three fuels at time t, which is an exogenous parameter in this model. The total investment has to be modeled as an increasing function of \(\frac{ED-K}{TI}\), where TI is the time to adjust new investments. In addition, investment has to be connected to the total scrapping of old equipment DO to allow a regeneration of the stock of equipment. To model these interactions, we use the following formula that defines the total investment:

$$ I=DO.f\left(\frac{ED-K}{TI.DO}\right) \text{ ,} $$
(6)

where f is an increasing continuous function.

One then has to determine the capacity utilization to allow the model to track exogenous energy demand in case of large downward variations (compared to total scrapping DO). Capacity utilization U is simply defined as:

$$ U=\frac{ED}{K} \text{ .} $$
(7)

Here, capacity utilization is assumed not to be fuel specific as the same capacity utilization figure is posited for the three fuels:

$$ \forall i,\ \ \ U_i=U \text{ .} $$
(8)

As a result, the simulated demand for fuel i, denoted \(\hat{D}_i\), is:

$$ \hat{D}_i=U_i K_i= ED \frac{K_i}{K} \text{ .} $$
(9)

To summarize, the model’s role is the following: given the dynamics of the total fossil demand and the fuels’ market price, the model captures the interfuel substitution in order to find how the total demand is shared between the coal, oil, and natural gas demands. The model’s equations correspond to a system of non-linear differential equations. Because of its complexity, this system has to be simulated with numerical techniques (Euler’s method) and solved on MATLAB. Once this model is calibrated to the consuming countries, it is used to derive the inverse demand function.

If we denote by \(Q^t_{md}\) the quantity brought to the spot market d at season m of year t, the system dynamics approach provides the following inverse demand function:

$$ \begin{array}{rll} p^t_{md}= {}& pc^t_{md} + \frac{1}{\gamma^t_{md}} \mbox{atanh} \left(\frac{\alpha^t_{md}+\beta^t_{md}-Q^t_{md}}{\alpha^t_{md}} \right) \ \ \ \mbox{if $Q^t_{md}$} \geq \beta^t_{md}+\frac{\alpha^t_{md}\beta^t_{md}}{\alpha^t_{md}+\beta^t_{md}} \\ & p'c^t_{md} + \frac{1}{\gamma'^t_{md}} \mbox{atanh} \left(\frac{\alpha'^t_{md}+\beta'^t_{md}-Q^t_{md}}{\alpha'^t_{md}} \right) \ \ \ \mbox{if $Q^t_{md}$} \leq \beta^t_{md}+\frac{\alpha^t_{md}\beta^t_{md}}{\alpha^t_{md}+\beta^t_{md}}\\[4pt] \end{array} $$
(10)

where the parameters α, β, γ and pc, which are time and season-dependent must be calibrated. \(Q^t_{md}\) is the total gas volume consumed in market d at year t and season m and \(p^t_{md}\) is the corresponding gas market price. Note that this function links the gas prices and volumes in the spot markets.

The distinction between the domains \(Q^t_{md} \geq \beta^t_{md}+\frac{\alpha^t_{md}\beta^t_{md}}{\alpha^t_{md}+\beta^t_{md}}\) and \(Q^t_{md} \leq \beta^t_{md}+\frac{\alpha^t_{md}\beta^t_{md}}{\alpha^t_{md}+\beta^t_{md}}\) is needed to take into account the anticipated scrapping of burnersFootnote 2 and avoids absurd situations where the price rises towards + ∞ (and also to guarantee the concavity of the objective functions). The splitting of the domains is not restrictive for practical applications. The parameters α′, β′, γ′ and pc are calculated to ensure the continuity of h and its derivative h′.

The function atanh is such that:

$$ \forall x \in (-1,1) \ \mbox{atanh}(x)=\frac{1}{2}\ \mbox{ln}\left(\frac{1+x}{1-x}\right) $$

The following table gives the values of the inverse demand function parameters, for the primary natural gas consumption in year 2003 in France, Germany, Italy, the UK, Belgium, and the Netherlands. The natural gas volumes in 2002 are exogenous.

Table 1

Figure 2 gives the demand function shape (i.e., the variation of the quantity Q d over the price p d in a given market). Note that we preferred showing the demand function rather than the inverse demand function for more clarity.

Fig. 2
figure 2

The demand function

We take account of the anticipated scrapping of burners to avoid situations where the quantity does not converge towards 0 when the price is very high. Obviously, such situations provide demand functions that cannot be used in Nash–Cournot competition modeling. Hence, we distinguish between two domains of the demand function, regarding whether we are in a standard scrapping regime or the anticipated scrapping one. This distinction is clearly shown in Eq. (10). Also, Fig. 2 shows the difference between the domains: \(Q^t_{md} \geq \beta^t_{md}+\frac{\alpha^t_{md}\beta^t_{md}}{\alpha^t_{md}+\beta^t_{md}}\) (standard scrapping of burners) and \(Q^t_{md} \leq \beta^t_{md}+\frac{\alpha^t_{md}\beta^t_{md}}{\alpha^t_{md}+\beta^t_{md}}\) (anticipated scrapping of burners). The inflection point of the demand function, which is shown in Fig. 2, is the parameter \(pc^t_{md}\). It represents a competitive price, regarding the consumption of natural gas. It is an aggregation of the oil and coal prices and can be seen as a threshold for the gas price that determines whether natural gas is a competitive fuel or not. This feature captures the possible fuel substitution in the natural gas inverse demand function. Besides, Fig. 2 shows that the domains distinction and the calibration of the (inverse) demand function ensures its continuity and differentiability.

As mentioned in the economic description of the markets, we need to distinguish between the off-peak/peak season parameters of the inverse demand function. Besides, to calibrate the demand function for the future, we need to specify a scenario for the fossil primary energy demand and the oil and coal market prices, that are considered as exogenous by GaMMES. Our system dynamics approach (Abada et al. 2011) will allow us to understand how the fossil demand is going to be shared between the consumption of the three fuels.

2.4 The mathematical description

This section details the mathematical description of our model. It presents the optimization problems of all the supply chain players. Note that the dual variables are written in parentheses by their associated constraints.

Producer p’s maximization program is given below. The corresponding decision variables are \(zp^t_{mfpi}\), \(x^t_{mfpd}\), \(ip^t_{fp}\), \(q^t_{mfp}\) and up pi . A producer can extract natural gas from all the possible production nodes he owns. He can sell gas to the independent traders via long-term contracts or directly target the spot markets, where a Nash–Cournot competition is exerted, between him, the other producers, and the independent traders. He pays the transportation costs necessary to bring gas to the independent traders’ location (for the LTCs sales) or the spot markets (for the spot markets sales). Production investments are also considered.

$$ \begin{array}{rll} \mbox{Max} \\[-3pt] & \sum\limits_{t,m,f,i} \delta^t \eta_{pi}\big(zp^t_{mfpi}\big) \\[-3pt] & {\kern11pt} +\sum\limits_{t,m,f,d} \delta^t \left(p^t_{md}\big(x^t_{mfpd}+\overline{x^t_{mfpd}}\,\big)\right)x^t_{mfpd} \\[-3pt] & {\kern11pt} -\sum\limits_{t,f} \delta^t Pc_f \left(\,\sum\limits_{t'\leq t} \sum\limits_m q^{t'}_{mfp} ,Rf_f \right) \\[-3pt] & {\kern11pt} +\sum\limits_{t,f} \delta^t Pc_f \left(\,\sum\limits_{t'< t} \sum\limits_m q^{t'}_{mfp} ,Rf_f \right) \\[-3pt] & {\kern11pt} -\sum\limits_{t,f} \delta^{t} Ip_f ip^t_{fp} \\[-3pt] & {\kern11pt} -\sum\limits_{t,m,p,a} \delta^t \big(\big(Tc_{a}+\tau^t_{ma}\big)fp^t_{mpa}\big) \end{array} $$
$$ \begin{array}{rll} \mbox{such that:} \\ \forall t,\ f,\ \ \ \ \ \ \ \sum\limits_p \sum\limits_{t'\leq t}\sum\limits_m q^{t'}_{mfp} -Rf_f \leq 0 \ \ \ \ \ (\phi^t_{f}) \ \ \ \end{array} $$
(11a)
$$ \begin{array}{rll} \forall t,\ f,\ m,\ \ \ \ \ \ \ & \sum\limits_p q^t_{mfp}-Kf_f(1-dep_f)^{t} \\[4pt] & -\sum\limits_p \sum\limits_{t'\leq t-delay_p} ip^{t'}_{fp}(1-dep_f)^{t-t'} \leq 0 \ \ \ \ \ \big(\chi^t_{mf}\big) \ \ \ \end{array} $$
(11b)
$$ \forall t,\ m,\ f,\ \ \ \ \ \ \ -q^t_{mfp}+ \left(\sum\limits_i \ zp^t_{mfpi}+ \sum\limits_d x^t_{mfpd}\right) \leq 0 \ \ \ \ \ \big(\gamma^t_{mfp}\big) \ \ \ $$
(11c)
$$ \forall t,\ f\ \ \ \ \ \ \ \sum\limits_m \sum\limits_p \big((-1)^m q^t_{mfp}\big)-f\/l_f \leq 0 \ \ \ \ \ \big(\vartheta1^t_{f}\big) \ \ \ \\ $$
(11d)
$$ \forall t,\ f,\ \ \ \ \ \ \ - \sum\limits_m \sum\limits_p \big((-1)^m q^t_{mfp}\big)-f\/l_f \leq 0 \ \ \ \ \ \big(\vartheta2^t_{f}\big) \ \ \ $$
(11e)
$$ \forall t,\ f,\ d,\ m,\ \ \ \ \ \ x^t_{mfpd}-O_{fp} H \ \ \ \ \leq 0 \ \ \ \ \big(\epsilon1^t_{mfpd}\big) $$
(12a)
$$ \forall t,\ f,\ i,\ m,\ \ \ \ \ \ zp^t_{mfpi}-O_{fp} H \ \ \ \ \leq 0 \ \ \ \ \big(\epsilon2^t_{mfpi}\big) $$
(12b)
$$ \forall t,\ f,\ m,\ \ \ \ \ \ q^t_{mfp}-O_{fp} H \ \ \ \ \leq 0 \ \ \ \ \big(\epsilon3^t_{mfp}\big) $$
(12c)
$$ \forall t,\ f,\ \ \ \ \ \ ip^t_{fp}-O_{fp} H \ \ \ \ \leq 0 \ \ \ \ \big(\epsilon4^t_{fp}\big) $$
(12d)
$$ \begin{array}{rll} \forall t,\ f,\ \ \ \ \ \ & \sum\limits_p ip^t_{fp}-Lf_f Kf_f(1-dep_f)^{t} \\ & -Lf_f \sum\limits_p \sum\limits_{t'\leq t-delay_p} ip^{t'}_{fp}(1-dep_f)^{t-t'} \ \ \ \ \leq 0 \ \ \ \ \big(\iota p^t_{f}\big) \\ \end{array} $$
(12e)
$$ \begin{array}{rll} \forall t,\ m,\ n,\ \ \ \ \ \ & \sum\limits_{a} M6_{an}fp^t_{mpa}(1-loss_a) \\ & -\sum\limits_{a} M5_{an}fp^t_{mpa} + \sum\limits_f M1_{fn} q^t_{mpf} \\ & -\sum\limits_d \sum\limits_f M3_{dn} x^t_{mfpd} \\ & -\sum\limits_i \sum\limits_f M2_{in} zp^t_{mfpi} \ \ \ \ =0 \ \ \ \ \big(\alpha p^t_{mpn}\big) \end{array} $$
(12f)
$$ \forall t,\ i,\ \ \ \ \ \ up_{pi}-\sum\limits_{f,m} zp^t_{mfpi} \ \ \ \ =0 \ \ \ \ \big(\eta p^t_{pi}\big) $$
(12g)
$$ \forall \ i,\ \ \ \ \ \ ui_{pi}-up_{pi} \ \ \ \ = 0 \ \ \ \ (\eta_{pi}) $$
(12h)
$$ \forall t,\ m,\ d,\ i,\ f, \ \ \ \ \ \ zp^t_{mfpi}, \ x^t_{mfpd}, \ ip^t_{fp}, \ q^t_{mfp}, \ up_{pi} \geq 0 $$

We denote by \(\overline{x^t_{mfpd}}\) the total amount of gas brought in year t, season m to the market d by all the players different from producer p. Hence, the total quantity brought to the market \( Q^t_{dm}= \sum_i y^t_{mid}+\sum_f \sum_p x^t_{mfpd} \) will be denoted \(Q^t_{dm}= x^t_{mfpd}+\overline{x^t_{mfpd}}\) in order to clearly show the strategic interaction and the dependence of \(Q^t_{dm}\) over \(x^t_{mfpd}\) (producer p’s decision variable). Using this notation, the KKT conditions will be written more easily.

The term

$$ \sum\limits_{t,m,f,i} \delta^t \eta_{pi}\big(zp^t_{mfpi}\big) + \sum\limits_{t,m,f,d} \delta^t \left(p^t_{md}\big(x^t_{mfpd}+\overline{x^t_{mfpd}}\big)\right)x^t_{mfpd} $$

is the revenue, which is obtained from the sales on the long-term contracts to the independent traders or directly from the retail markets.

The term

$$ \sum\limits_{t,m,p,a} \delta^t \big(\big(Tc_{a}+\tau^t_{m,a}\big)fp^t_{mpa}\big) $$

is the transport and congestion costs charged by the pipeline operator to producer p. The dual variable \(\tau^t_{ma}\) is associated with the pipeline capacity constraint through the arc a. It represents the congestion price on the corresponding pipeline (see the transport operator optimization problem for a more explanation).

The term

$$ \sum\limits_{t,f}\delta^{t} Ip_f ip^t_{fp} $$

is the investment cost in production at the different production nodes.

The term

$$ \sum\limits_{t,f}\delta^t \left(Pc_f \left(\sum\limits_{t'\leq t} \sum\limits_m q^{t'}_{mfp} ,Rf_f \right)-Pc_f \left(\sum\limits_{t'< t} \sum\limits_m q^{t'}_{mfp} ,Rf_f \right)\right) $$

is the actualized production cost. This term’s explanation is as follows:

The production cost (at production node f) Pc f depends on two variables, the total quantity produced, which will be denoted q and the natural gas resources Rf f . The Golombek production cost function we used is as follows:

$$ \begin{array}{rll} \forall q \in [0,Rf_f), \ Pc_f(q,Rf_f)&=&a_f q+b_f \frac{q^2}{2} \\ &&-\ Rf_f c_f\left(\frac{Rf_f-q}{Rf_f}\mbox{ln}\left(\frac{Rf_f-q}{Rf_f}\right)+ \frac{q}{Rf_f}\right)\\ \end{array} $$
(13)

or if written for the marginal production cost

$$ \forall q \in [0,Rf_f),\ \frac{\mbox{d}Pc_f}{\mbox{d}q}=a_f+b_f q\ +\ c_f \mbox{ln}\left(\frac{Rf_f-q}{Rf_f}\right) $$
(14)

In our model, the production cost function is dynamic. The gas volume available to be extracted is dynamically reduced at each period, taking into account the exhaustivity of the resource.

If at year 1, the production is q1 and at year 2 q2, the total cost is thus:

$$ cost=Pc_f(q1,RES_f)+\delta (Pc_f(q1+q2,RES_f)-Pc_f(q1,RES_f)) $$

Hence, to estimate that cost at year t, we need to calculate the production cost of the sum over all the extracted volumes until year t and subtract the cost we have at year t − 1.

The explanation of the constraints is straightforward:

The constraint (11a) bounds each production node’s production by its reserves.

The constraint (12b) bounds the seasonal quantities produced by each production node’s production capacity, explicitly taking into account the different dynamic investments. The total installed production capacity decreases with time because of the production depreciation factor dep f .

The constraint (11c) states that the total production must be greater than the sales (to the long-term and spot markets). The constraints (11d) and (11e) can be rewritten as follows:

$$ \forall t,\ f\ \left|\sum\limits_m \left((-1)^m \sum\limits_p q^t_{mfp}\right)\right| \leq f\/l_f. $$

This fixes a maximum spread between the off-peak/peak production at each production node. ( − 1)m is equal to 1 in the off-peak season and −1 in the peak season.

The constraint (12f) is a market-clearing condition at each node, regarding the flows from producer p depending on whether this node is a production node, an independent trader location or a demand market.

The constraint (12e) bounds the capacity expansion of each production node f: each year, the investment decided to increase the production capacity is less than 100 ×Lf f percent the installed capacity at that year. A historical study of the capacity expansion of some production nodes allowed us to calibrate the value of Lf f : Lf f  = 0.20.

The constraint (12g) equates the sales of producer p for the long-term contracts to the contracted volume up pi , each year.

The constraint (12h) describes the following: For each pair of producer/ independent trader (p, i), the gas quantity sold by p in the long-term contract market must be equal to the gas quantity purchased by i. Therefore, this is a supply/demand equation in the long-term contracts market. The associated dual variable η pi is the corresponding contract unit selling/purchase price, because we do not assume the existence of market power in the long-term contract trade. Using this technique, it is possible to make the long-term contracts prices and volumes endogenous to the description so that they become an output of the model.

The constraint (and the similar other ones) (12a) allows producer p to use only the production nodes he owns (for production, investments, sales, etc.). We recall that the incidence matrix O is such as O fp  = 1 if and only if producer p owns production node f.

Independent trader i’s maximization program is given below. The corresponding decision variables are \(zi^t_{mpi}\), \(y^t_{mid}\), \(r^t_{is}\), \(in^t_{is}\) and ui pi . The independent trader buys gas only from the producers via long-term contracts. The sales are dedicated to all the spot markets, where trader i is in an oligopolistic competition with the other independent traders and the producers. He can store his gas in all the different storage nodes while supporting capacity reservation, storage and withdrawal costs. He also has to support the transportation costs to bring gas to the spot markets or to store/withdraw it.

$$ \begin{array}{rll} \mbox{Max} \\[-2pt] & \sum\limits_{t,m,d} \delta^t \left(p^t_{md}(y^t_{mid}+\overline{y^t_{mid}})y^t_{mid}\right) \\[-2pt] & -\sum\limits_{t,p,m} \delta^t \left(\eta_{pi} zi^t_{mpi} \right) \\[-2pt] & -\sum\limits_{t,s} \delta^t \left(Rc_s r^t_{is}\right) \\[-2pt] & -\sum\limits_{t,s} \delta^t \left((Ic_{s}+Wc_{s}) in^t_{is}\right) \\[-2pt] & -\sum\limits_{t,m,i,a} \delta^t \left(Tc_{a}+\tau^t_{ma}\right)f\/i^t_{mia} \end{array} $$
$$ \begin{array}{rll} \mbox{such that:} \\ \forall t,\ m,\ \ \ \ \ \sum\limits_p zi^t_{mfpi}-\left(\sum\limits_d y^t_{mid} +(-1)^m \sum_s in^t_{is}\right) \ \ \ \ = 0 \ \ \ \ \ \ \big(\psi^t_{mi}\big) \\[-2pt] \end{array} $$
(15a)
$$ \forall t,\ s,\ \ \ \ \ in^t_{is}-r^t_{is} \ \ \ \ \leq 0 \ \ \ \ \big(\mu^t_{is}\big) $$
(15b)
$$ \begin{array}{rll} \forall t,\ m,\ n,\ \ \ \ & \sum\limits_{a} M6_{an}f\/i^t_{mia}(1-loss_a)-\sum\limits_{a} M5_{an}f\/i^t_{mia} \\[-2pt] & -\sum\limits_d M3_{dn} y^t_{mid} +\sum\limits_p M2_{in} zi^t_{mpi} \\[-2pt] & -(-1)^m \sum\limits_s M4_{sn} in^t_{is} \ \ \ \ =0 \ \ \ \ {\kern-5pt} \big(\alpha i^t_{min}\big) {\kern-10pt} \end{array} $$
(15c)
$$ \forall t,\ p,\ \ \ \ \ ui_{pi}-\sum\limits_m zi^t_{mpi} \ \ \ \ = 0 \ \ \ \ \big(\eta i^t_{pi}\big) $$
(15d)
$$ \forall \ p,\ \ \ \ \ ui_{pi}-up_{pi} \ \ \ \ = 0 \ \ \ \ (\eta_{pi}) $$
(15e)
$$ \forall t,\ m,\ p,\ \ \ \ \ -zi^t_{mpi} + min_{pi} \sum\limits_m zi^t_{mpi} \ \ \ \ \leq 0 \ \ \ \ \big(\upsilon^t_{mpi}\big) $$
(15f)
$$ \forall t,\ s,\ \ \ \ \sum\limits_i r^t_{is}- Ks_s- \sum\limits_{t'\leq t-delay_s} is^{t'}_s \ \ \ \ \leq 0 \ \ \ \big(\beta s^t_s\big) $$
(15g)
$$ \forall t,\ m,\ s,\ d, \ \ \ \ \ zi^t_{mpi},\ y^t_{mid},\ r^t_{is},\ in^t_{is},\ ui_{pi} \ \ \ \ \geq 0 $$

We denote by \(\overline{y^t_{mid}}\) the total amount of gas brought in year t, season m to the market d by all the players different from trader i. Hence, the total quantity brought to the market \( Q^t_{dm}= \sum_i y^t_{mid}+\sum_f \sum_p x^t_{mfpd} \) will be denoted \(Q^t_{dm}= y^t_{mid}+\overline{y^t_{mid}}\) in order to clearly show the strategic interaction and the dependence of \(Q^t_{dm}\) over \(y^t_{mid}\) (trader i’s decision variable). Using this notation, the KKT conditions will be written more easily. Note that the producers and independent traders see the same inverse demand function in the spot markets. The notation we have chosen implies that:

$$ \forall p,\ i,\ d,\ t,\ m,\ \ Q^t_{dm}= \sum\limits_i y^t_{mid}+\sum\limits_f \sum\limits_p x^t_{mfpd} = y^t_{mid}+\overline{y^t_{mid}}= x^t_{mfpd}+\overline{x^t_{mfpd}}` $$
(16)

The term

$$ \sum\limits_{t,m,d} \delta^t \left(p^t_{md}\big(y^t_{mid}+\overline{y^t_{mid}}\big)y^t_{mid}\right)-\sum_{t,p,m} \delta^t \big(\eta_{pi} zi^t_{mpi}\big) $$

is the net profit.

The term

$$ \sum\limits_{t,s} \delta^t \left(Rc_s r^t_{is}\right) $$

is the storage capacity reservation cost.

The term

$$ \sum\limits_{t,s} \delta^t \left((Ic_{s}+Wc_{s}) in^t_{is}\right) $$

is the storage/withdrawal costs.Footnote 3

The term

$$ \sum\limits_{t,m,i,a} \delta^t \left(Tc_{a}+\tau^t_{ma}\right)f\/i^t_{mia} $$

is the transport and congestion costs charged by the pipeline operator from the independent trader i.

As for the feasibility set, it is also easy to specify:

The constraint (15a) is a gas quantity balance for each trader. The term ( − 1)m is equal to 1 in the off-peak season and − 1 otherwise. An implicit assumption we use in our description is that all the storage nodes must be “empty” (regardless of the working gas quantities) at the end of each year.

The Eq. (15b) implies that each independent trader has to pay for a storage reservation quantity, each year and at each storage node s, to be able to store his gas.

The constraint (15d) forces each trader to purchase the same quantity, in long-term contracts from each producer and year.

The constraint (15e) is similar to the constraint (15h) of the producers’ optimization program. For each pair of producer/independent trader (p,i), the gas quantity sold by p in the long-term contract market must be equal to the gas quantity purchased by i. Therefore, this is a supply/demand equation in the long-term contracts market. The associated dual variable η pi is the corresponding contract unit selling/purchase price, because we do not assume the existence of market power in the long-term contract trade. Using this technique, it is possible to make the long-term contracts prices and volumes endogenous to the description so that they become an output of the model.

The constraint (15f) fixes a minimum percentage of the annual contracted volume \(\min\nolimits_{pi}\) that has to be exchanged between p and i each season of each year.

The constraint (15g) is a storage constraint expressed at each storage node, taking into account the investments decided by the storage operator.

On the transportation side of our model, we will assume that the producers pay the transport costs to bring natural gas from the production nodes to the independent traders’ locations and the end-use markets. The traders support the transport costs to store/withdraw gas or bring it to the end-users for their sales.

The pipeline operator optimization (cost minimization) program is given below. The corresponding decision variables are \(fp^t_{mpa},\ f\/i^t_{mia}\) and \(ik^t_{a}\). The pipeline operator minimizes the total transportation, congestion, and capacity investments costs.

$$ \begin{array}{rll} \mbox{Min} \\[5pt] & \sum\limits_{t,m,a} \delta^t \left(Tc_{a}+\tau^t_{ma}\right) \sum\limits_p fp^t_{mpa} \\[5pt] &+ \sum\limits_{t,m,a} \delta^t \left(Tc_{a}+\tau^t_{ma}\right) \sum\limits_i f\/i^t_{mia} \\[5pt] &+ \sum\limits_{t,a} \delta^{t} Ik_{a} ik^t_{a} \end{array} $$
$$ \begin{array}{rll} \mbox{such that:} \\ \forall t,\ m,\ a,\ \ \ \ \ \sum\limits_{p} fp^t_{mpa}+\sum\limits_{i} f\/i^t_{mia} -\left({\kern-3pt} Tk_{a}+ \sum\limits_{t'\leq t-delay_i} ik^{t'}_{a}\right) \ \ \ \ \leq 0 \ \ \ \ \big(\tau^t_{ma}\big) \end{array} $$
(17a)
$$ \forall t,\ a,\ \ \ \ \ ik^t_a -La_a \left({\kern-3pt} Tk_{a}+La_a \sum\limits_{t'\leq t-delay_i} ik^{t'}_{a}\right) \ \ \ \ \leq 0 \ \ \ \ \big(\iota a^t_{a}\big) $$
(17b)
$$ \begin{array}{rll} \forall t,\ m,\ p,\ n,\ \ \ \ \ & \sum\limits_{a} M6_{an}fp^t_{mpa}(1-loss_a) \\[-2pt] & -\sum\limits_{a} M5_{an}fp^t_{mpa} + \sum\limits_f M1_{fn} q^t_{mpf} \\[-2pt] &-\sum\limits_d \sum\limits_f M3_{dn} x^t_{mfpd} \\[-2pt] & -\sum\limits_i \sum\limits_f M2_{in} zp^t_{mfpi} =0 \ \ \ \ \big(\alpha p^t_{mpn}\big) \end{array} $$
(17c)
$$ \begin{array}{rll} \forall t,\ m,\ i,\ n,\ \ \ \ \ & \sum\limits_{a} M6_{an}f\/i^t_{mia}(1-loss_a) \\[-2pt] & -\sum\limits_{a} M5_{an}f\/i^t_{mia} -\sum\limits_d M3_{dn} y^t_{mid} \\[-2pt] &+\sum\limits_p M2_{in} zi^t_{mpi} \\[-2pt] & -(-1)^m \sum\limits_s M4_{sn} in^t_{is} =0 \ \ \ \ \big(\alpha i^t_{min}\big) \end{array} $$
(17d)
$$ \forall t,\ m,\ a,\ p,\ i,\ \ \ \ \ fp^t_{mpa},\ f\/i^t_{mia},\ ik^t_{a} \geq 0 $$

The objective function contains both the transport/congestion and investment costs.

The congestion cost through arc a, \(\tau^t_{ma}\), is the dual variable associated with the constraint (17a). This constraint concerns the physical seasonal capacity of arc a, including the possible time-dependent investments.

The constraint (17b) bounds the capacity expansion of each arc a: each year, the investment decided to increase the transport capacity is less than 100 ×La a percent the installed capacity at that year. In GaMMES, we used the value La a  = 0.2.

The other constraints are market-clearing conditions at each node, depending on whether this node is a production node, an independent trader location, a demand market or a storage node and depending on whether the transportation costs are supported by the producers or the independent traders.

We consider both pipeline and LNG routes for transport. The liquefaction and regasification costs are included in the transportation cost on the LNG arcs. We assume, in our representation that the physical losses occur at the end nodes of the arcs.

The storage operator optimization (cost minimization) program is given below. The corresponding decision variable is \(is^t_s\). The storage operator minimizes the total operational and capacity investments costs.

$$ \begin{array}{rll} \mbox{Min} \\[-4pt] & \sum\limits_{t,s} \delta^{t} Is_s is^t_s + \sum\limits_{t,i,s} \delta^{t} (Ic_{s}+Wc_{s}) in^t_{is} + \sum\limits_{t,i,s} \delta^{t} Rc_s r^t_{is} \end{array} $$
$$ \begin{array}{rll} \mbox{such that:} \\ \forall t,\ s,\ \ \ \ \ \sum\limits_i r^t_{is}- Ks_s- \sum\limits_{t'\leq t-delay_s} is^{t'}_s \leq 0 \ \ \ \ \big(\beta s^t_s\big) \ \ \end{array} $$
(18a)
$$ \forall t,\ s,\ \ \ \ \ is^t_s- Ls_s Ks_s- Ls_s \sum\limits_{t'\leq t-delay_s} is^{t'}_s \leq 0 \ \ \ \ \big(\iota s^t_s\big) \ \ $$
(18b)
$$ \forall t,\ s,\ \ \ \ \ \ is^t_s \geq 0 \ \ \ \ $$

The storage operator minimizes the total operation cost that includes investment, storage, withdrawal and storage capacity reservation costs. His decision variable is \(is^t_s\), which means that he only controls the different investments that dynamically increase the storage capacity of each storage node. The incentive this player has to invest is due to the constraint he must satisfy: the capacity available at each storage node must be sufficient to meet the volumes the independent traders have to store each year in the off-peak season. Capacity expansion is bounded and we used the value Ls s  = 0.2.

If we take a closer look at the optimization program of a producer, we will notice that his feasibility set depends on the decision variables of the independent traders. Also, the feasibility set of any independent trader’s optimization program depends on the producers’ decision variables. The situation is similar for the pipeline and storage operators. This particularity makes our formulation (the KKT conditions) a Generalized Nash–Cournot problem. Similarly, the Generalized Nash–Cournot problem can also be formulated as a Quasi Variational Inequality problem (QVI). In order to solve our problem, we look for the particular solution that makes our problem a VI formulation (Harker 1991; Harker and Pang 1998). More details about the VI solution search are given in Section 2.5.

The concavity of all the players’ objective functions has been demonstrated.

When the KKT conditions are written, we obtain the Mixed Complementarity Problem given in Appendix A.

2.5 The (quasi)-variational inequality and generalized Nash–Cournot games

In this section, we recall Harker’s result (Harker 1991) in order to understand how to theoretically solve a Generalized Nash–Cournot problem.

A standard Nash–Cournot problem is a set of optimization programs where some of the players can influence other players’ payoff via the objective functions. In a Generalized Nash–Cournot formulation, some players can also change the feasibility sets of other players, via their decision variables. In our particular model, if we consider an independent trader i, the constraint

$$ \forall \ p,\ i,\ ui_{pi}=up_{pi} $$

contains the producers’ decision variables up pi . These decision variables influence trader i’s feasibility set. The situation is symmetric for the producers. More generally, our double-layer economic structure makes the producers and independent traders influence each-other’s feasibility sets. This is principally due to the formulation of the long-term contracts that are issued from a supply/demand equilibrium constraint.

A VI (Variational Inequality) problem can be formulated as follows: given a set K ∈ ℝn and a mapping \(F: K \longrightarrow \mathbb{R}^n\), find x * ∈ K s.t.

$$ \forall y \in K,\ F(x^*)^t(y-x^*) \geq 0 $$

It is straightforward that a standard Nash–Cournot problem can be expressed as a VI formulation if the objective functions are differentiable (is suffices to write the necessary and sufficient conditions on the gradient of the objective functions that characterize the optimum).

A QVI (Quasi-Variational Inequality) problem adds mixed constraints (Facchinei et al. 2003). Given n point-to-set mappings K i : ℝand F : ℝ, find x * ∈ ℝn s.t. \(\forall i \in \{1,2...n\}\ x_i^* \in K_i(x^*) \) and

$$ \forall y \in \mathbb{R}^n\ s.t.\ \forall i \in \{1,2...n\}\ y_i \in K_i(x^*),\ \ F(x^*)^t(y-x^*) \geq 0 $$

A generalized Nash–Cournot problem can be expressed as a QVI formulation. Unlike VI problems, a QVI formulation often has an infinite set of equilibria. In some particular cases, a QVI problem can be slightly changed into a VI formulation. This is possible, in particular if the QVI is issued from a Generalized Nash–Cournot problem, which is our case. The idea is quite simple: we want to make the mappings K i independent of the variables x i . To do so, we make all the constraints that mix different players’ decision variables common to all these players. From the KKT conditions point of view, Harker (1991) demonstrated that the “VI solution” is obtained by giving the same dual variables to the common constraints.

If we apply the previous results to our model, this leads to the fact that the producers and independent traders see the same dual variables η pi and must consider the common constraints (12h) and (15e) in their optimization program. Economically speaking, this means that they have the same value for the long-term contract prices.

Using this technique, we make sure we end up with a VI solution (Harker 1991).

The use of VI techniques in energy markets modeling has already been exploited, such as in Smeers (2003). Besides, Smeers et al. (2011) also develops a Generalized Nash–Cournot model to describe market coupling in the European power system.

3 The European natural gas markets model

This section puts the model at work and presents our numerical results.

3.1 The representation

The model we presented in Section 2.4 has been used in order to study the northwestern European natural gas trade. The following array summarizes the representation we have studied.

Table 2
Table 3

The model is run up through 2045 but only the results through 2035 are used to avoid end-of-horizon effects (depletion of all the production nodes, etc.).

We aggregate all the production nodes of each producer into one production node. We assume that each consuming market is associated with one independent local trader (indexed by tr). As an example, France tr would be GDF-SUEZ and Germany tr would be E-On Ruhrgas. All the storage nodes are also aggregated so that there is one storage node per consuming country. As for the transport, the different gas routes given in Fig. 3 were considered.

Fig. 3
figure 3

The northwestern European natural gas routes, production and storage nodes

The local production in the different consuming countries is also taken into consideration (the imports from non-represented producers, which are small, are also considered). We assume that these locally consumed volumes are exogenous to the model.

We consider Algeria as an LNG producer who can exert market power. The other LNG exchanges between producers “outside” the scope of the model (such as the UAE) and the represented consumers are considered exogenously in the model. Therefore, we assume that the LNG demand, except for Algeria, is inelastic to the gas price. This approach is an assumption that overestimates the market power allowed to standard (not LNG) natural gas producers. However, the missing LNG volumes are very small according to International Energy Agency (2009a) (less than 1%).

3.2 The calibration

The calibration process has been carried out in order to best meet:

  • the primary natural gas consumption,

  • the industrial sector gas price and

  • the volumes produced by each gas producer,

between 2000 and 2004 (the first time period).

The model has been solved using the solver PATH (Ferris and Munson 1987) from GAMS. In order to shorten the running time, we used a five-year time-step resolution. We chose five years because it is the typical length of time needed to construct investments in production, infrastructure or storage. Also, the demand function has been linearized.

The data for the market prices, consumed volumes, and imports is the publicly available set from International Energy Agency (2009a). We define a new variable \(exch^t_{mpd}\) that represents the exported volume from producer p to market d. More precisely:

$$ \forall t,\ m,\ p, \ d,\ exch^t_{mpd}=\sum\limits_i B_{id}\ zp^t_{mpi}+x^t_{mpd} $$

The matrix B is such that B id  = 1 if the independent trader i is located in market d (e.g., GDF-SUEZ in France, E-On Ruhrgas in Germany) and B id  = 0 otherwise. Hence, one can notice that the exchanged volumes include both the spot and long-term contract trades.

The calibration elements we used are the inverse demand function parameters \(\alpha^t_{md}\), \(\gamma^t_{md}\), \(pc^t_{md}\) and \(\beta^t_{md}\). The idea is that the system dynamics (Abada et al. 2011) model is run in order to calculate all the inverse demand function parameters, for all the markets and at each year and season of our study. The calibration technique slightly adjusts these values to make the model correctly describe the historical data (between 2000 and 2004).

In order to calibrate the produced volumes properly, we introduced security of supply parameters that link each pair of producer/consuming countries (p,d). A security of supply measure forces each country not to import from any producer, more than a fixed percentage (denoted by SSP) of the overall imports. This property can be rewritten as follows:

$$ \forall t,\ m,\ p, \ d,\ exch^t_{mpd} \leq SSP_{pd} \sum\limits_p exch^t_{mpd} $$

The security of supply parameters are also an output of the calibration process. As mentioned before, the calibration concerned only the first time period.

The calibration tolerates a maximum error of 5% for the prices and consumed quantities and 10% for the imported/exported volumes. The tolerated error is higher for the exchanged volumes because they depend on the exports decided by the producers for all the targeted consumers, even those that are not in the scope of the model. As an example, the exported volumes from Russia to CIS (CEI) countries are exogenous to our model.

3.3 Numerical results

In order to estimate the demand function parameters, our model requests exogenous inputs: the fossil primary energy demand and the evolution of the oil and coal prices. For that purpose, we used a scenario provided by the European Commission (2008). The annual fossil primary consumption and prices growth per year that we used are given in the following chart (starting from 2000):

Table 4

Figure 4 gives the evolution of the natural gas consumption between 2000 and 2030 provided by our model for the countries represented. The consumption is given in Bcm/year. The figure also shows the evolution of the natural gas prices ($/cm), in the industrial sector, for the represented countries. We recall that the industrial sector prices are taken as a proxy for natural gas prices. The figure also gives the evolution of the producing countries’ sales between 2000 and 2030, in Bcm/year.

Fig. 4
figure 4

The natural gas consumption, prices, and sales

The average annual growth between 2000 and 2030 is given in the following chart:

Table 5

According to our simulation, France shows the highest annual consumption growth, averaging 0.61%, between 2005 and 2030. Both the UK and the Netherlands experience a significant decrease in their natural gas consumption, as their domestic supplies are replaced by more expensive foreign imports. This effect is magnified in our model by the fact that only existing reserves are taken into account, which are depleted relatively quickly due to high installed capacities.

The consumption of all the countries shown flattens out or decreases in 2030, compared to 2000, despite the increase of the fossil primary demand. This is mainly due to the fact that competition in the upstream market becomes less and less important with time. Indeed, in 2025, the continental Europe gas production (the UK and the Netherlands) is expected to be around 25 Bcm. This will increase the exercise of market power and the consumption growth will therefore be reduced.

The price average annual growth between 2000 and 2030 is given in the following chart:

Table 6

As expected, the natural gas prices increase continuously in all the countries. The prices values are driven, as a resut of the Nash–Cournot interaction by the combination of two effects: the fossil primary energy demand and the competition between fuels (see Eq. (10)). Since the fossil primary energy demand and the coal and oil prices increase with time, they force the gas price up. This combination explains why the natural gas price annual growth in all the countries is less important than the growth in both oil and coal. Indeed, this is due to the fact that the fossil primary energy consumption does not increase with time as quickly as the coal and oil prices.

The production in continental Europe is expected to greatly decrease in the forthcoming decades. The Norwegian production is expected to increase until 2012 before starting to decrease. The Dutch decrease is smooth (−4.5% per year between 2000 and 2020) whereas the UK one is very sharp. The model indicates that the United Kingdom will use up more than 75% of its natural gas reserves (starting from 2000) until 2015. This may seem surprising but can be understood by the fact that we take into account only the proven reserves in 2000: 900 Bcm (BP Statistical Review of World Energy 2009). Thus, we do not consider the reserves discoveries that may occur till 2045.

On the other hand, the Russian and Algerian shares in the European natural gas consumption is expected to grow in the coming decades: in 2020, the foreign imports will represent 47% of the northwestern European consumption.

In order to test the strength of the model, we compare its output versus historical values. For that purpose, we consider the consumption and prices in the European countries between 2005 and 2010 (second time-step) and compare them to what actually happened in that period. Let us recall that the second time-step has not been used in the calibration. Figure 5 gives the natural gas consumption between 2005 and 2010 in Bcm/year and prices in $/cm in the countries represented. The left bars represent the model’s output whereas the right bars represent the real historical data.

Fig. 5
figure 5

Comparison between the model’s output and historical data

The average model estimation errors are 2.2% for the consumption and 3.5% for the prices. They are in the same range as the ones tolerated when calibrating the model (period 2000–2005).

Figure 6 gives the evolution of the northwestern European natural gas dependence on foreign imports (those considered in the model). The dependence is the ratio between the foreign exports to northwestern Europe and the domestic consumption.Footnote 4

Fig. 6
figure 6

The northwestern European natural gas dependence over time

The natural gas dependence is expected to reach 70% around 2030, which will bring about important security of supply concerns (Abada and Massol 2011). However, these conclusions should be cautiously considered because they are based on strong assumptions. Indeed, in our study, we assume that no more natural gas reserves will be found in the future and no shale gas will be produced in Europe.Footnote 5

$$ \emph{dependence}=\frac{\emph{foreign\ exports}}{\emph{total\ consumption}} $$
(19)

Now we present the results related to the long-term contracts (LTC) provided by GaMMES. The following tables give the LTC volumes and prices between the different producers and the independent traders:

Table 7
Table 8

One can notice that if a pair of producer-independent trader contract on the long-term, the corresponding LTC price is nonnegative, which is not straightforward since the correponding LTC price is a free dual variable. Also, the spot prices in the consuming countries reported in Fig. 4 are in general higher than the LTC prices. The explanation is as follows: since long-term contracts are the only means for the independent traders to obtain natural gas, LTC prices can be considered as marginal supply costs. Similarly, the spot prices are directly related to the independent traders’ revenue. Therefore, if an independent trader has an incentive to contract in the long-term, it implies that his revenues, over the time horizon, are greater than his costs. In a similar fashion, spot prices are greater than LTC prices.

The Belgian trader is the one that diversifies his gas supplies the most (four sources). This is due to its geographical location, which is close to three producing countries: Norway, The Netherlands and Algeria (recall that the Algerian production node is directly linked to Belgium via an LNG route). For a particular trader, the LTC price is the same with respect to all the possible supply sources (same price within the column). This suggests that the LTC prices are correlated to the spot prices: an independent trader may tolerate high supply marginal costs if his marginal revenue in his spot market is high enough. Besides, we assumed in our model that the producers do not exert market power when contracting in the long-term.

The UK does not contract in the long-term with the independent traders. This is due to its limited gas reserves that do not create an incentive to invest in production. Therefore, the producer does not have an investment-related risk hedging strategy and prefers directly targeting the spot markets without creating long-term contracts. This situation has been observed in recent years.

Regarding the LTC prices, the GaMMES results are close to reality. As for the LTC volumes, the results suggest that they represent, on average, 28% of the total (contract+spot) trade. This value is relatively low, compared to what is currently observed in Europe (70%) (International Gas Union 2011). This can be explained by the fact that in GaMMES, we only consider contracts endogenously determined after 2000 without taking into consideration the pre-existing ones signed before that time as part of the calibration process. Furthermore, from the point of view of the model, given installed production capacity as of 2000, the producers may not have a strong incentive to contract with the traders after this time because related investments have already been made.

The following array gives the size of the transport infrastructure in 2035, between the producers and the consumers. For the sake of brevity, we only present the results related to the biggest gas routes (pipelines and LNG).

Table 9

As expected, Russia and Algeria are the producers who invest the most in the transport network towards Europe, which matches the increase of their market shares (Russia–Germany, Algeria–France and Algeria–Belgium). On the contrary, Norway, the UK and the Netherlands do not invest a lot in the transport capacity because their reserves of natural gas are relatively low.

The purpose of the next comparison is to show the effects of the fuel substitution-based demand function. To that end, we consider an alternative linear demand function of the following form:

$$ q^t_{md}=a^t_{md}-b_{\!d}p^t_{md} $$
(20)

where the slope b should remain constant over time and the interceipt \(a^t_{md}\) changes as a function of the fossil primary energy demand. In our study, we made \(a^t_{md}\) evolve with the fossil primary energy demand annual growth. The slope b d is a result of the calibration process. This description of the markets will be referred to as the standard model whereas the model we proposed in this article will be refered to as the GaMMES model. Note that the standard model is rather simplistic and does not correctly capture the demand behavior, because the inverse demand function’s slope b d is kept constant. However, the main purpose of the comparison is not to present a new model but rather to remove one feature of the GaMMES model (energy substitution) and see how this would alter the results.

Figure 7 provides the consumption and price levels for both models considered.

Fig. 7
figure 7

Comparison between the standard and the GaMMES model: consumption and prices

We notice that the standard model provides a lower consumption than the GaMMES results. The average difference in consumption is 13%. The standard model provides lower prices than the GaMMES results. The average difference between the two models is 23% which is quite large.

Now, let’s compare between the results provided by the GaMMES model, the standard model and some official forecast. For that purpose, we choose the forecast of the European Commission (2008).

Figure 8 shows the evolution of the global European energy consumption between 2000 and 2030 and the average European price, forecasted in three scenarios. Ths first one is issued from the European Commission report (baseline scenario) (European Commission 2008). The second one is our model forecast and the third one is the standard model forecast.

Fig. 8
figure 8

The European Commission, the GaMMES model and the standard model forecasts

Comparing the results of both the GaMMES model and the standard model with the 2007 European Commission forecasts (European Commission 2008) gives strong support to the need to take into account fuel substitution, especially in the long run. The standard model output shows a very fast decrease of natural gas consumption in the long-run. This seems at odds with the perspective of the market, since as fossil primary energy consumption is exogenous, the remaining energy consumption has to be met with oil and coal. This view clearly contradicts the global evolution of the different energy shares in the recent past as well as the strong support for cleaner fuels given by the European policy framework. On the contrary, the GaMMES model output gives a better outcome. The demand for gas slowly increases in the medium term, due to both higher fossil primary domestic consumption and a higher share for natural gas in the energy mix (International Energy Agency 2009b). The trend is compensated in the long run by the increased exercise of market power. The 2010 kink is mostly explained by the quick depletion of domestic reserves.

These previous results and those of Fig. 5 show that consumed quantities provided by the model are in line with the European Commission forecasts. This gives confidence in the GaMMES results, for the European Commission forecasts are subject to countries’ review and acceptance. Regarding the prices, GaMMES is closer to the European Commission scenario than the standard model, even if both of these scenarios underestimate the prices.

In conclusion, compared to a standard description, the GaMMES model gives a better representation of the evolution of the natural gas prices and consumption. It is necessary to take into consideration the fuel substitution in the natural gas markets’ modeling because they allow a better understanding of the consumers’ behavior.

To test the effects of the systems dynamics approach, starting from time-step three (2010–2014), six sets of exogenous coal and oil price patterns over time were input varying only in time-step three. Then the different endogenous gas prices that resulted were analyzed. Hence, we are able to draw, in the third time-step, the dependence of the gas price on the oil and coal prices. Figure 9 gives the evolution of the (average) European natural gas price in the third time-step vs. the oil and coal prices. For the sake of clarity, we showed the evolution of the natural gas price over the competitive price p c .

Fig. 9
figure 9

Evolution of the natural gas price over the competitive price in 2015

Obviously, this evolution is an increasing function of the substitution fuels’ prices. The higher the oil and coal prices are, the greater the natural gas demand will be and, therefore, the higher the natural gas price will be. This property also concerns the long-term contracts’ prices between the producers and the independent traders η pi . Hence, our model allows us to capture part of the indexation (on coal and oil prices) effects via the substitution in the inverse demand function.

4 Conclusions

This paper presents a Generalized Nash–Cournot model in order to describe the natural gas market evolution. The demand representation is rich because it takes into account the possible energy substitution that can be made between oil, coal, and natural gas. This appears in the introduction of a competitive price, in the demand function. The exhaustibility of the resource is taken care of by the use of Golombek production cost functions.

The long-term contract prices and volumes are endogenously computed as dual variables to long-term contracts constraints. This aspect makes our formulation a Generalized Nash–Cournot model, more generally a QVI formulation. In order to solve it, we derived the corresponding VI formulation.

The model is dynamic (2000–2045) and has been solved using the PATH solver with GAMS. After the calibration process, the model was applied to the European natural gas trade between 2000 and 2035 to understand consumption, prices, production, and natural gas dependence. The consumption and price forecast are consistent with those found in the literature. A study of the evolution of the natural gas dependence on foreign supplies has been carried out. It shows that northwestern Europe will become more and more dependent on foreign supplies in the future. Long-term contract prices and volumes have been presented, analyzed, and compared with current data in order to understand the producers/traders’ interaction.

Our results have been compared with other forecasts: one provided by the European Commission and another one issued from a standard model where the energy substitution is not present. The results show that it is important to capture, while studying the natural gas demand function, the possible energy substitution regarding other possible usable fuels market’ prices.

In order to illustrate the possible use of fuel substitution, we studied the evolution of the natural gas price over the coal and oil prices. The coal-oil prices indexation of the natural gas price in the spot markets or in the long-term contracts can be understood using these studies.

Future work can include addressing gas supply scenarios in Europe focused on various market aspects such as LNG and shale gas development. Also, stochasticity can be introduced when representing the impact of market risks. The demand can also be made random by modeling the fluctuations of the oil price to understand its influence on gas price/consumption.