Abstract
In many game situations, the evolutionary process is in discrete time rather than in continuous time. An extension of the analysis to a discrete-time dynamic framework is provided in this chapter. In particular, it presents an analysis on subgame consistent solutions which entail group optimality and individual rationality for cooperative (deterministic and stochastic) dynamic games. It integrates the works of Yeung and Petrosyan (2010) and Chapters 12 and 13 of Yeung and Petrosyan (2012a). We first present in Sect. 7.1 a general formulation of cooperative dynamic games in discrete time with the noncooperative outcomes, and the notions of group optimality and individual rationality. Subgame consistent cooperative solutions with corresponding payoff distribution procedures are derived in Sect. 7.2. An illustration of cooperative resource extraction in discrete time is given in Sect. 7.3. A general formulation of coopeartive stochastic dynamic games in discrete time is given in Sect. 7.4. Subgame consistent cooperative solutions with corresponding payoff distribution procedures are derived in Sect. 7.5. An illustration of cooperative resource extraction under uncertainty in discrete time is given in Sect. 7.6. A heuristic approach to obtaining subgame consistent solutions for cooperative dynamic games is provided in Sect. 7.7. Section 7.8 contains Appendices of the Chapter. Chapter Notes are given in Sect. 7.9 and problems in Sect. 7.10. In addition, to make the discrete-time analysis in this Chapter fully in line with the continuous-time analyses presented in earlier chapters a terminal condition is added to each player’s payoff in Yeung and Petrosyan (2010, 2012a).
Access provided by Autonomous University of Puebla. Download chapter PDF
Similar content being viewed by others
Keywords
- Subgame Consistent Solution
- Payoff Distribution Procedure (PDP)
- Dynamic Cooperative Games
- Noncooperative Outcome
- imputationImputation
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
In many game situations, the evolutionary process is in discrete time rather than in continuous time. An extension of the analysis to a discrete-time dynamic framework is provided in this chapter. In particular, it presents an analysis on subgame consistent solutions which entail group optimality and individual rationality for cooperative (deterministic and stochastic) dynamic games . It integrates the works of Yeung and Petrosyan (2010) and Chapters 12 and 13 of Yeung and Petrosyan (2012a). We first present in Sect. 7.1 a general formulation of cooperative dynamic games in discrete time with the noncooperative outcomes, and the notions of group optimality and individual rationality. Subgame consistent cooperative solutions with corresponding payoff distribution procedures are derived in Sect. 7.2. An illustration of cooperative resource extraction in discrete time is given in Sect. 7.3. A general formulation of coopeartive stochastic dynamic games in discrete time is given in Sect. 7.4. Subgame consistent cooperative solutions with corresponding payoff distribution procedures are derived in Sect. 7.5. An illustration of cooperative resource extraction under uncertainty in discrete time is given in Sect. 7.6. A heuristic approach to obtaining subgame consistent solutions for cooperative dynamic games is provided in Sect. 7.7. Section 7.8 contains Appendices of the Chapter. Chapter Notes are given in Sect. 7.9 and problems in Sect. 7.10. In addition, to make the discrete-time analysis in this Chapter fully in line with the continuous-time analyses presented in earlier chapters a terminal condition is added to each player’s payoff in Yeung and Petrosyan (2010, 2012a).
1 Cooperative Dynamic Games
In this Section we present the basic framework of discrete-time cooperative dynamic games.
1.1 Game Formulation
Consider the general \( T- \) stage \( n- \) person nonzero-sum discrete-time cooperative dynamic game with initial state x 0. The state space of the game is \( X\in {R}^m \) and the state dynamics of the game is characterized by the difference equation:
-
for \( k\in \left\{1,2,\cdots, T\right\}\equiv \kappa \) and \( {x}_1={x}^0 \),
where \( {u}_k^i\in {U}^i\subset {R}^{m_i} \) is the control vector of player i at stage k, \( {x}_k\in X\subset {R}^m \) is the state of the game.
The payoff of player i is
-
for \( i\in \left\{1,2,\cdots, n\right\}\equiv N \),
where r is the discount rate, and \( {q}_{T+1}^i\left({x}_{T+1}\right) \) is the terminal benefit that player i received at stage \( T+1 \).
The payoffs of the players are transferable.
1.2 Noncooperative Outcome
In this subsection, we characterize the noncooperative outcome of the discrete-time economic game (1.1 and 1.2). Let \( \left\{{\phi}_k^i(x),\kern0.49em \mathrm{f}\mathrm{o}\mathrm{r}\ k\in \kappa \kern0.24em \mathrm{and}\kern0.24em i\in N\right\} \) denote a set of strategies that provides a feedback Nash equilibrium solution to the game (1.1 and 1.2), and
where \( {x}_k=x \), for \( k\in K \) and \( i\in N \), denote the value function indicating the game equilibrium payoff to player i over the stages from k to \( T+1 \). A frequently used way to characterize and derive a feedback Nash equilibrium of the game is provided in the following theorem.
Theorem 1.1
A set of strategies \( \left\{{\phi}_k^i(x),\kern0.24em \mathrm{f}\mathrm{o}\mathrm{r}\;k\in \kappa \kern0.24em \mathrm{and}\kern0.24em i\in N\right\} \) provides a feedback Nash equilibrium solution to the game (1.1 and 1.2) if there exist functions V i(k, x), for \( k\in K \) and \( i\in N \), such that the following recursive relations are satisfied:
for \( i\in N \) and \( k\in \kappa \).
Proof
Invoking the discrete-time dynamic programming technique in Theorem A.5 of the Technical Appendices, V i(k, x) is the maximized payoff of player i for given strategies \( \left\{\;{\phi}_k^i(x)\right.,\kern0.24em \mathrm{f}\mathrm{o}\mathrm{r}\;j\in N\;\mathrm{and}\;j\ne i\left.\right\} \) of the other \( n-1 \) players. Hence a Nash equilibrium appears. ■
For the sake of exposition, we sidestep the issue of multiple equilibria and focus on solvable games in which a particular noncooperative Nash equilibrium is chosen by the players in the entire subgame.
1.3 Dynamic Cooperation
Now consider the case when the players agree to cooperate and distribute the payoff among themselves according to an optimality principle . Two essential properties that a cooperative scheme has to satisfy are group optimality and individual rationality . An agreed upon optimality principle entails group optimality and an imputation to distribute the total cooperative payoff among the players.
We first examine the group optimal solution and then the condition under which individual rationality will be maintained.
1.3.1 Group Optimality
Maximizing the players’ joint payoff guarantees group optimality in a game where payoffs are transferable. To maximize their joint payoff the players have to solve the discrete-time dynamic programming problem of maximizing
subject to (1.1).
Invoking the discrete-time dynamic programming technique an optimal solution to the control problem (1.1) and (1.5) can be characterized by the theorem below.
Theorem 1.2
A set of strategies \( \left\{{\psi}_k^i(x),\kern0.24em \mathrm{f}\mathrm{o}\mathrm{r}\;k\in \kappa \kern0.24em \mathrm{and}\kern0.24em i\in N\right\} \) provides an optimal solution to the problem (1.1) and (1.5) if there exist functions W(k, x), for \( k\in K \), such that the following recursive relations are satisfied:
Proof
Follow the proof of discrete-time dynamic programming technique in Theorem A.5 of the Technical Appendices. ■
Substituting the optimal control \( \left\{{\psi}_k^i(x),\kern0.24em \mathrm{f}\mathrm{o}\mathrm{r}\kern0.24em k\in \kappa \kern0.24em \mathrm{and}\kern0.24em i\in N\right\} \) into the state dynamics (1.1), one can obtain the dynamics of the cooperative trajectory as:
for \( k\in \kappa \) and \( {x}_1={x}^0 \).
Let \( {\left\{{x}_k^{*}\;\right\}}_{k=1}^T \) denote the solution to (1.8) and hence the optimal cooperative path. The total cooperative payoff over the stages from k to \( T+1 \) can be expressed as:
We then proceed to consider individual rationality .
1.3.2 Individual Rationality
The players have to agree on an optimality principle in distributing the total cooperative payoff among themselves. For individual rationality to be upheld the payoffs an player receives under cooperation have to be no less than his noncooperative payoff along the cooperative state trajectory. For instance, (i) the players may share the excess of the total cooperative payoff over the sum of individual noncooperative payoffs equally, or (ii) they may share the total cooperative payoff proportionally to their noncooperative payoffs.
Let \( \xi \left(\cdot, \cdot \right) \) denote the imputation vector guiding the distribution of the total cooperative payoff under the agreed-upon optimality principle along the cooperative trajectory \( {\left\{{x}_k^{*}\;\right\}}_{k=1}^T \). At stage k, the imputation vector according to \( \xi \left(\cdot, \cdot \right) \) is \( \xi \left(k,{x}_k^{*}\right)=\left[{\xi}^1\left(k,{x}_k^{*}\right),{\xi}^2\left(k,{x}_k^{*}\right),\dots, {\xi}^n\left(k,{x}_k^{*}\right)\right] \), for \( k\in \kappa \).
If for example, the optimality principle specifies that the players share the excess of the total cooperative payoff over the sum of individual noncooperative payoffs equally, then the imputation to player i becomes:
for \( i\in N \) and \( k\in \kappa \).
If the optimality principle specifies that the players share the total cooperative proportional to their noncooperative payoffs, then the imputation to player i becomes:
for \( i\in N \) and \( k\in \kappa \).
For individual rationality to be maintained throughout all the stages \( k\in \kappa \), it is required that:
In particular, the above condition guaranties that the payoff allocated to a player under cooperation will be no less than its noncooperative payoff.
To satisfy group optimality , the imputation vector has to satisfy
This condition guarantees the highest joint payoffs for the participating players.
2 Subgame Consistent Solutions and Payment Mechanism
To guarantee dynamical stability in a dynamic cooperation scheme, the solution has to satisfy the property of subgame consistency . In particular, the specific agreed-upon optimality principle must remain effective at any stage of the game along the optimal state trajectory. Since at any stage of the game the players are guided by the same optimality principles and hence do not have any ground for deviation from the previously adopted optimal behavior throughout the game. Therefore for subgame consistency to be satisfied, the imputation \( \xi \left(\cdot, \cdot \right) \) according to the original optimality principle has to be maintained at all the T stages along the cooperative trajectory \( {\left\{{x}_k^{*}\;\right\}}_{k=1}^T \). In other words, the imputation
for k ∈ κ
has to be upheld.
Crucial to the analysis is the formulation of a payment mechanism so that the imputation in (2.1) can be realized.
2.1 Payoff Distribution Procedure
Similar to the analysis of cooperative differential games , we first formulate a Payoff Distribution Procedure (PDP) so that the agreed imputations (2.1) can be realized. Let B i k (x * k ) denote the payment that player i will receive at stage k under the cooperative agreement along the cooperative trajectory \( {\left\{{x}_k^{*}\;\right\}}_{k=1}^T \).
The payment scheme involving B i k (x * k ) constitutes a PDP in the sense that the imputation to player i over the stages from k to T can be expressed as:
for \( i\in N \) and \( k\in \kappa \).
Using (2.2) one can obtain
Upon substituting (2.3) into (2.2) yields
for \( i\in N \) and \( k\in \kappa \).
A theorem characterizing a formula for B i k (x * k ), for \( k\in \kappa \) and \( i\in N \), which yields (2.2) is provided below.
Theorem 2.1
A payment equaling
for \( i\in N \),
given to player i at stage \( k\in \left\{1,2,\cdots, T\right\} \) along the cooperative trajectory \( {\left\{{x}_k^{*}\;\right\}}_{k=1}^T \) would lead to the realization of the imputation {ξ(k, x * k ), for \( k\in \kappa \)}.
Proof
From (2.4), one can readily obtain (2.5). Theorem 2.1 can also be verified alternatively by showing that from (2.2)
and \( {\xi}^i\left(T+1,{x}_{T+1}^{*}\right)={q}_{T+1}^i\left({x}_{T+1}\right){\left(\frac{1}{1+r}\right)}^T \).
Hence Theorem 2.1 follows. ■
The payment scheme in Theorem 2.1 gives rise to the realization of the imputation guided by the agreed-upon optimal principle and will be used to derive time (optimal-trajectory-subgame) consistent solutions in the next subsection.
2.2 Subgame Consistent Solution
We denote the discrete-time cooperative game with dynamics (1.1) and payoff (1.2) by Γ c (1, x 0). We then denote the game with dynamics (1.1) and payoff (1.2) which starts at stage υ with initial state x * υ by Γ c (υ, x * υ ). Moreover, we let \( P\left(1,{x}_0\right)=\left\{{u}_h^i\ \mathrm{and}\ {B}_h^i\ \mathrm{f}\mathrm{o}\mathrm{r}\ h\in \kappa\ \mathrm{and}\ i\in N,\ \xi \left(1,{x}_0\right)\right\} \) denote the agreed-upon optimality principle for the cooperative game Γ c (1, x 0). Let \( \mathrm{P}\left({x}_{\upsilon}^{*}, \upsilon \right)=\left\{{u}_h^i\ \mathrm{and}\ {B}_h^i\ \mathrm{f}\mathrm{o}\mathrm{r}\ h\in \left\{\upsilon, \upsilon +1,\dots, T\right\}\ \mathrm{and}\ i\in N,\ \xi \left(\upsilon, {x}_{\upsilon}^{*}\right)\right\} \) denote the optimality principle of the cooperative game Γ c (υ, x * υ ) according to the original agreement.
A theorem characterizing a subgame consistent solution for the discrete-time cooperative game Γ c (1, x 0) is presented below.
Theorem 2.2
For the cooperative game Γ c (1, x 0) with optimality principle \( \mathrm{P}\left(1,{x}_0\right)=\left\{{u}_h^i\ \mathrm{and}\ {B}_h^i\ \mathrm{f}\mathrm{o}\mathrm{r}\ h\in \kappa\ \mathrm{and}\ i\in N,\ \xi \left(1,{x}_0\right)\right\} \) in which
-
(i)
\( {u}_h^i={\psi}_h^i\left({x}_h^{*}\right) \), for \( h\in \kappa \) and \( i\in N \), is the set of group optimal strategies for the game Γ c (1, x 0), and
-
(ii)
\( {B}_h^i={B}_h^i\left({x}_h^{*}\right) \), for \( h\in \kappa \) and \( i\in N \), where
and [ξ 1(h, x * h ), ξ 2(h, x * h ), …, ξ i(h, x * h )], is the imputation according to the optimality principle P(h, x * h );
is subgame consistent.
Proof
Follow the proof of the continuous-time analog in Theorem 2.2 of Chap. 3. ■
When all players are using the cooperative strategies, the payoff that player i will directly receive at stage k given that along the cooperative trajectory \( {\left\{{x}_k^{*}\;\right\}}_{k=1}^T \) is
However, according to the agreed upon imputation , player i will receive B i k (x * k ) at stage k. Therefore a side-payment
-
for \( k\in \kappa \) and \( i\in N \),
will be given to player i to yield the cooperative imputation ξ i(k, x * k ).
3 An Illustration in Cooperative Resource Extraction
Consider an economy endowed with a renewable resource and with two resource extractors (firms). The lease for resource extraction begins at stage 1 and ends at stage 3 for these two firms. Let u i k denote the amount of resource extracted by firm i at stage k, for \( i\in \left\{1,2\right\} \). Let U i be the set of admissible extraction rates, and\( {x}_k\in X\subset {R}^{+} \) the size of the resource stock at stage k. The extraction cost for firm \( i\in \left\{1,2\right\} \) depends on the quantity of resource extracted u i k , the resource stock size x k , and cost parameters c 1 and c 2. The extraction cost for firm i at stage k is specified as c i (u i k )2/x k . The price of the resource is P.
The profits that firm 1 and firm 2 will obtain at stage k are respectively:
In stage 4, the firms will receive a salvage value equaling qx 4.
The growth dynamics of the resource is governed by the difference equation:
for \( k\in \left\{1,2,3\right\} \) and \( {x}_1={x}^0 \).
There exists an extraction constraint that human harvesting can at most exploit Y proportion of the existing biomass, hence \( {u}_k^1+{u}_k^2\le Y{x}_k \). Moreover \( b<1-Y \). The payoff of extractor \( i\in \left\{1,2\right\} \) is to maximize the present value of the stream of future profits:
subject to (3.2).
Invoking Theorem 1.1, one can characterize the noncooperative equilibrium strategies in a feedback solution for game (3.2 and 3.3). In particular, a set of strategies \( \left\{{\phi}_k^i(x),\ \mathrm{f}\mathrm{o}\mathrm{r}\ k\in \left\{1,2,3\right\}\ \mathrm{and}\ i\in \left\{1,2\right\}\right\} \) provides a Nash equilibrium solution to the game (3.2 and 3.3) if there exist functions V i(k, x), for \( i\in \left\{1,2\right\} \) and \( k\in \left\{1,2,3\right\} \), such that the following recursive relations are satisfied:
Performing the indicated maximization in (3.4) yields:
-
for \( i\in \left\{1,2\right\} \) and \( k\in \left\{1,2,3\right\} \).
From (3.5), the game equilibrium strategies can be expressed as:
for \( i\in \left\{1,2\right\} \) and \( k\in \left\{1,2,3\right\} \).
The game equilibrium profits of the firms can be obtained as:
Proposition 3.1
The value function indicating the game equilibrium profit of firm i is:
where A i k and C i k , for \( i\in \left\{1,2\right\} \) and \( k\in \left\{1,2,3\right\} \), are constants in terms of the parameters of the game (3.2 and 3.3).
Proof
See Appendix A of this Chapter. ■
Substituting the relevant derivatives of the value functions in Proposition 3.1 into the game equilibrium strategies (3.6) yields a noncooperative feedback equilibrium solution of the game (3.2 and 3.3).
Now consider the case when the extractors agree to maximize their joint profit and share the excess of cooperative gains over their noncooperative payoffs equally. To maximize their joint payoff, they solve the problem of maximizing
subject to (3.2).
Invoking Theorem 1.2, one can characterize the optimal controls in the dynamic programming problem (3.2) and (3.8). In particular, a set of control strategies \( \left\{{\psi}_k^i(x),\ \mathrm{f}\mathrm{o}\mathrm{r}\ k\in \left\{1,2,3\right\}\ \mathrm{and}\ i\in \left\{1,2\right\}\right\} \) provides an optimal solution to the problem (3.2) and (3.8) if there exist functions W(k, x): \( R\to R \), for \( k\in \left\{1,2,3\right\} \), such that the following recursive relations are satisfied:
Performing the indicated maximization in (3.9) yields:
for \( i\in \left\{1,2\right\} \) and \( k\in \left\{1,2,3\right\} \).
In particular, the optimal cooperative strategies can be obtained from (3.10) as:
for \( i\in \left\{1,2\right\} \) and \( k\in \left\{1,2,3\right\} \).
The firms’ joint profit under cooperation can be obtained as:
Proposition 3.2
The value function indicating the maximized joint payoff is
where A k and C k , for \( k\in \left\{1,2,3\right\} \), are constants in terms of the parameters of the problem (3.8) and (3.2).
Proof
See Appendix B of this Chapter. ■
Using (3.11) and Proposition 3.2, the optimal cooperative strategies of the players can be expressed as:
Substituting ψ i k (x) from (3.13) into (3.2) yields the optimal cooperative state trajectory:
for \( k\in \left\{1,2,3\right\} \) and \( {x}_1={x}^0 \).
Dynamics (3.14) is a linear difference equation readily solvable by standard techniques. Let \( \left\{{x}_k^{*},\ \mathrm{f}\mathrm{o}\mathrm{r}\ k\in \left\{1,2,3\right\}\right\} \) denote the solution to (3.14).
Since the extractors agree to share the excess of cooperative gains over their noncooperative payoffs equally, an imputation
for \( k\in \left\{1,2,3\right\} \) and \( i\in \left\{1,2\right\} \) has to be maintained.
Invoking Theorem 2.1, if \( {x}_k^{*}\in X \) is realized at stage k a payment equaling
for i ∈ {1,2};
given to player i at stage \( k\in \kappa \) would lead to the realization of the imputation (3.15).
A subgame consistent solution can be readily obtained from (3.13), (3.15) and (3.16).
4 Cooperative Stochastic Dynamic Games
In this Section we present the basic framework of discrete-time cooperative stochastic dynamic games.
4.1 Game Formulation
Consider the general \( T- \) stage \( n- \) person nonzero-sum discrete-time cooperative stochastic dynamic game with initial state x 0. The state space of the game is \( X\in {R}^m \) and the state dynamics of the game is characterized by the stochastic difference equation:
for \( k\in \left\{1,2,\cdots, T\right\}\equiv \kappa \) and \( {x}_1={x}^0, \)
where \( {u}_k^i\in {R}^{m_i} \) is the control vector of player i at stage k, \( {x}_k\in X \) is the state, and θ k is a set of statistically independent random variables.
The objective of player i is
where r is the discount rate and \( {E}_{\theta_1,{\theta}_2,\cdots, {\theta}_T} \) is the expectation operation with respect to the statistics of θ 1, θ 2, \( \cdots, {\theta}_T \).
The payoffs of the players are transferable.
We then characterize the noncooperative outcome of the discrete-time stochastic economic game (4.1 and 4.2). Let \( \left\{{\phi}_k^i(x),\ \mathrm{f}\mathrm{o}\mathrm{r}\ k\in \kappa\ \mathrm{and}\ i\in N\right\} \) denote a set of strategies that provides a feedback Nash equilibrium solution (if it exists) to the game (4.1 and 4.2), and
where \( {x}_k=x \), for \( k\in K \) and \( i\in N \), denote the value function indicating the expected game equilibrium payoff to player i over the stages from k to \( T+1 \).
A frequently used way to characterize and derive a feedback Nash equilibrium of the game is provided in the theorem below.
Theorem 4.1
A set of strategies \( \left\{{\phi}_k^i(x),\ \mathrm{f}\mathrm{o}\mathrm{r}\ k\in \kappa\ \mathrm{and}\ i\in N\right\} \) provides a feedback Nash equilibrium solution to the game (4.1 and 4.2) if there exist functions V i(k, x), for \( k\in K \) and \( i\in N \), such that the following recursive relations are satisfied:
for \( i\in N \) and \( k\in \kappa \),
where \( {\tilde{f}}_k^i\left(x,{u}_k^i\right)={f}_k\left[x,{\phi}_k^1(x),{\phi}_k^2(x),\dots, {\phi}_k^{i-1}(x),{u}_k^i,{\phi}_k^{i+1}(x),\dots, {\phi}_k^n(x)\right] \) and \( {E}_{\theta_k} \) is the expectation operation with respect to the statistics of θ k .
Proof
Invoking the discrete-time stochastic dynamic programming technique in Theorem A.6 of the Technical Appendices, V i(k, x) is the maximized payoff of player i for given strategies \( \left\{\;{\phi}_k^i(x)\right.,\ \mathrm{f}\mathrm{o}\mathrm{r}\ j\in N\ \mathrm{and}\ j\ne i\left.\right\} \) of the other \( n-1 \) players. Hence a Nash equilibrium appears. ■
Again, for the sake of exposition, we sidestep the issue of multiple equilibria and focus on solvable games in which a particular noncooperative Nash equilibrium is chosen by the players in the entire subgame.
4.2 Dynamic Cooperation under Uncertainty
Now consider the case when the players agree to cooperate and distribute the payoff among themselves according to an optimality principle . Once again, the essential properties of group optimality and individual rationality have to be satisfied. An agreed upon optimality principle entails group optimality and an imputation to distribute the total cooperative payoff among the players.
We first examine the group optimal solution and then the condition under which individual rationality will be maintained.
4.2.1 Group Optimality
Maximizing the players’ expected joint payoff guarantees group optimality in a game where payoffs are transferable. To maximize their expected joint payoff the players have to solve the discrete-time stochastic dynamic programming problem of maximizing
subject to (4.1).
Invoking the discrete-time stochastic dynamic programming technique an optimal solution to the problem (4.1) and (4.5) can be characterized in the following theorem.
Theorem 4.2
A set of strategies \( \left\{{\psi}_k^i(x),\ \mathrm{f}\mathrm{o}\mathrm{r}\ k\in \kappa\ \mathrm{and}\ i\in N\right\} \), provides an optimal solution to the problem (4.1) and (4.5) if there exist functions W(k, x), for \( k\in K \), such that the following recursive relations are satisfied:
Proof
Follow the proof of the discrete-time stochastic dynamic programming technique in Theorem A.6 of the Technical Appendices. ■
Substituting the optimal control \( \left\{{\psi}_k^i(x),\ \mathrm{f}\mathrm{o}\mathrm{r}\ k\in \kappa\ \mathrm{and}\ i\in N\right\} \) into the state dynamics (4.1), one can obtain the dynamics of the cooperative trajectory as:
for \( k\in \kappa \) and \( {x}_1={x}^0 \).
We use X * k to denote the set of realizable values of x k at stage k generated by (4.8). The term \( {x}_k^{*}\in {X}_k^{*} \) is used to denote an element in X * k .
The term W(k, x * k ) gives the expected total cooperative payoff over the stages from k to \( T+1 \) if \( {x}_k^{*}\in {X}_k^{*} \) is realized at stage \( k\in \kappa \).We then proceed to consider individual rationality .
4.2.2 Individual Rationality
The players have to agree to an optimality principle in distributing the total cooperative payoff among themselves. For individual rationality to be upheld the expected payoffs an player receives under cooperation have to be no less than his expected noncooperative payoff along the cooperative state trajectory. Let \( \xi \left(\cdot, \cdot \right) \) denote the imputation vector guiding the distribution of the total cooperative payoff under the agreed-upon optimality principle along the cooperative trajectory \( {\left\{{x}_k^{*}\;\right\}}_{k=1}^T \). At stage k, the imputation vector according to \( \xi \left(\cdot, \cdot \right) \) is \( \xi \left(k,{x}_k^{*}\right)=\left[{\xi}^1\left(k,{x}_k^{*}\right),{\xi}^2\left(k,{x}_k^{*}\right),\dots, {\xi}^n\left(k,{x}_k^{*}\right)\right] \), for \( k\in \kappa \).
For individual rationality to be maintained throughout all the stages \( k\in \kappa \), it is required that:
In particular, the above condition guaranties that the expected payoff allocated to any player under cooperation will be no less than its expected noncooperative payoff.
To satisfy group optimality , the imputation vector has to satisfy
This condition guarantees the highest expected joint payoffs for the participating players.
If the optimality principle specifies that the players share the excess of the expected total cooperative payoff over the sum of expected individual noncooperative payoffs equally, then the imputation to player i becomes:
for \( i\in N \) and \( k\in \kappa \).
If the optimality principle specifies that the players share the expected total cooperative proportional to their expected noncooperative payoffs, then the imputation to player i becomes:
for \( i\in N \) and \( k\in \kappa \).
5 Subgame Consistent Solutions and Payment Mechanism
Now, we proceed to consider dynamically stable solutions in cooperative stochastic dynamic games . To guarantee dynamical stability in a stochastic dynamic cooperation scheme, the solution has to satisfy the property of subgame consistency . A cooperative solution is subgame-consistent if an extension of the solution policy to a subgame starting at a later time with any realizable state brought about by prior optimal behavior would remain optimal under the agreed upon optimality principle . In particular, subgame consistency ensures that as the game proceeds players are guided by the same optimality principle at each stage of the game, and hence do not possess incentives to deviate from the previously adopted optimal behavior. Yeung and Petrosyan (2010) developed conditions leading to subgame consistent solutions in stochastic differential games .
For subgame consistency to be satisfied, the imputation \( \xi \left(\cdot, \cdot \right) \) according to the original optimality principle has to be maintained at all the T stages along the cooperative trajectory \( {\left\{{x}_k^{*}\;\right\}}_{k=1}^T \). In other words, the imputation
has to be upheld.
Crucial to the analysis is the formulation of a payment mechanism so that the imputation in (5.1) can be realized.
5.1 Payoff Distribution Procedure
Following the analysis of Yeung and Petrosyan (2010), we formulate a discrete-time Payoff Distribution Procedure (PDP) so that the agreed imputations (5.1) can be realized. Let B i k (x * k ) denote the payment that player i will receive at stage k under the cooperative agreement if \( {x}_k^{*}\in {X}_k^{*} \) is realized at stage \( k\in \kappa \).
The payment scheme involving B i k (x * k ) constitutes a PDP in the sense that if\( {x}_k^{*}\in {X}_k^{*} \) is realized at stage k the imputation to player i over the stages from k to T can be expressed as:
for \( i\in N \) and \( k\in \kappa \).
A theorem characterizing a formula for B i k (x * k ), for \( k\in \kappa \) and \( i\in N \), which yields (5.2) is provided below.
Theorem 5.1
A payment equaling
for \( i\in N \),
given to player i at stage \( k\in \kappa \), if \( {x}_k^{*}\in {X}_k^{*} \) would lead to the realization of the imputation \( \left\{\xi \left(k,{x}_k^{*}\right),\ \mathrm{f}\mathrm{o}\mathrm{r}\ k\in \kappa \right\} \).
Proof
Using (5.2) one can obtain
Upon substituting (5.4) into (5.2) yields
for \( i\in N \) and \( k\in \kappa \).
Hence Theorem 5.1 follows. ■
The payment scheme in Theorem 5.1 gives rise to the realization of the imputation guided by the agreed-upon optimal principle and will be used to derive subgame consistent solutions in the next subsection.
5.2 Subgame Consistent Solution
We denote the discrete-time cooperative game with dynamics (4.1) and payoff (4.2) by Γ c (1, x 0). Then we denote the game with dynamics (4.1) and payoff (4.2) which starts at stage \( \upsilon \ge 1 \) with initial state \( {x}_{\upsilon}^{*}\in {X}_{\upsilon}^{*} \) by Γ c (υ, x * υ ). Moreover, we let \( P\left(1,{x}_0\right)=\left\{{u}_h^i\ \mathrm{and}\ {B}_h^i\ \mathrm{f}\mathrm{o}\mathrm{r}\ h\in \kappa\ \mathrm{and}\ i\in N,\xi \left(1,{x}_0\right)\right\} \) denote the agreed-upon optimality principle for the cooperative game Γ c (1, x 0). Let \( \mathrm{P}\left({x}_{\upsilon}^{*}, \upsilon \right)=\left\{{u}_h^i\ \mathrm{and}\ {B}_h^i\ \mathrm{f}\mathrm{o}\mathrm{r}\ h\in \left\{\upsilon, \upsilon +1,\dots, T\right\}\ \mathrm{and}\ i\in N,\xi \left(\upsilon, {x}_{\upsilon}^{*}\right)\right\} \) denote the optimality principle of the cooperative game Γ c (υ, x * υ ) according to the original agreement.
A theorem characterizing a subgame consistent solution for the discrete-time cooperative game Γ c (1, x 0) is presented below.
Theorem 5.2
For the cooperative game Γ c (1, x 0) with optimality principle \( P\left(1,{x}_0\right)=\left\{{u}_h^i\left({x}_h^{*}\right)\ \mathrm{and}\ {B}_h^i\left({x}_h^{*}\right)\ \mathrm{f}\mathrm{o}\mathrm{r}\ h\in \kappa\ \mathrm{and}\ i\in N\ \mathrm{and}\ {x}_h^{*}\in {X}_h^{*},\xi \left(1,{x}_0\right)\right\} \) in which
-
(i)
\( {u}_h^i\left({x}_h^{*}\right)={\psi}_h^i\left({x}_h^{*}\right) \), for \( h\in \kappa \) and \( i\in N \) and \( {x}_h^{*}\in {X}_h^{*} \), is the set of group optimal strategies for the game Γ c (1, x 0), and
-
(ii)
\( {B}_h^i\left({x}_h^{*}\right)={B}_h^i\left({x}_h^{*}\right) \), for \( h\in \kappa \) and \( i\in N \) and \( {x}_h^{*}\in {X}_h^{*} \), where
and \( \left[{\xi}^1\left(h,{x}_h^{*}\right),{\xi}^2\left(h,{x}_h^{*}\right),\dots, {\xi}^i\left(h,{x}_h^{*}\right)\right]\in P\left(h,{x}_h^{*}\right) \) is the imputation according to optimality principle P(h, x * h );
is subgame consistent.
Proof
Follow the proof of the continuous-time analog in Theorem 5.2 of Chap. 7. ■
When all players are using the cooperative strategies, the payoff that player i will directly receive at stage k given that \( {x}_k^{*}\in {X}_k^{*} \) is
However, according to the agreed upon imputation , player i will receive B i k (x * k ) at stage k. Therefore a side-payment
-
for \( k\in \kappa \) and \( i\in N \),
will be given to player i to yield the cooperative imputation ξ i(k, x * k ).
6 Cooperative Resource Extraction under Uncertainty
Consider an economy endowed with a renewable resource and with two resource extractors (firms). The lease for resource extraction begins at stage 1 and ends at stage 3 for these two firms. Let u i k denote the rate of resource extraction of firm i at stage k, for \( i\in \left\{1,2\right\} \). Let U i be the set of admissible extraction rates, and \( {x}_k\in X\subset {R}^{+} \) the size of the resource stock at stage k. The extraction cost for firm \( i\in \left\{1,2\right\} \) depends on the quantity of resource extracted u i k , the resource stock size x k , and cost parameters c 1 and c 2. In particular, extraction cost for firm i at stage k is specified as c i (u i k )2/x k . The price of the resource is P.
The profits that firm 1 and firm 2 will obtain at stage k are respectively:
In stage 4, the firms will receive a salvage value equaling qx 4. The growth dynamics of the resource is governed by the stochastic difference equation:
-
for \( k\in \left\{1,2,3\right\} \) and \( {x}_1={x}^0 \),
where θ k is a random variable with non-negative range {θ 1 k , θ 2 k , θ 3 k } and corresponding probabilities {λ 1 k , λ 2 k , λ 3 k }.
With no human harvesting, the natural growth of the resource stock is \( {x}_{k+1}-{x}_k=a-{\theta}_k{x}_k \). The natural growth of the resource is while the death rate exhibits stochasticity. There exists an extraction constraint that human harvesting can at most exploit b proportion of the existing biomass, hence \( {u}_k^1+{u}_k^2\le b{x}_k \). In addition, the highest value of \( {\theta}_k^y<\left(1-b\right) \) for \( k\in \left\{1,2,3\right\} \) and \( y\in \left\{1,2,3\right\} \).
The objective of extractor \( i\in \left\{1,2\right\} \) is to maximize the present value of the expected stream of future profits:
subject to (6.2).
Invoking Theorem 4.2, one can characterize the noncooperative equilibrium strategies in a feedback solution for game (6.2 and 6.3). In particular, a set of strategies \( \left\{{\phi}_k^i(x),\ \mathrm{f}\mathrm{o}\mathrm{r}\ k\in \left\{1,2,3\right\}\ \mathrm{and}\ i\in \left\{1,2\right\}\right\} \) provides a Nash equilibrium solution to the game (6.2 and 6.3) if there exist functions V i(k, x), for \( i\in \left\{1,2\right\} \) and \( k\in \left\{1,2,3\right\} \), such that the following recursive relations are satisfied:
Performing the indicated maximization in (6.4) yields:
for \( i\in \left\{1,2\right\} \) and \( k\in \left\{1,2,3\right\} \).
From (6.5), the game equilibrium strategies can be expressed as:
for \( i\in \left\{1,2\right\} \) and \( k\in \left\{1,2,3\right\} \).
The expected game equilibrium profits of the firms can be obtained as:
Proposition 6.1
The value function indicating the expected game equilibrium profit of firm i is
where A i k and C i k , for \( i\in \left\{1,2\right\} \) and \( k\in \left\{1,2,3\right\} \), are constants in terms of the parameters of the game (6.2 and 6.3).
Proof
See Appendix C of this Chapter. ■
Substituting the relevant derivatives of the value functions in Proposition 6.1 into the game equilibrium strategies (6.6) yields a noncooperative feedback equilibrium solution of the game (6.2 and 6.3).
Now consider the case when the extractors agree to maximize their expected joint profit and share the excess of cooperative gains over their expected noncooperative payoffs equally. To maximize their expected joint payoff, they solve the problem of maximizing
subject to (6.2).
Invoking Theorem 4.2, one can characterize the optimal controls in the stochastic dynamic programming problem (6.2) and (6.8). In particular, a set of control strategies \( \left\{{\psi}_k^i(x),\ \mathrm{f}\mathrm{o}\mathrm{r}\ k\in \left\{1,2,3\right\}\ \mathrm{and}\ i\in \left\{1,2\right\}\right\} \) provides an optimal solution to the problem (6.2) and (6.8) if there exist functions \( W\left(k,x\right):R\to R \), for \( k\in \left\{1,2,3\right\} \), such that the following recursive relations are satisfied:
Performing the indicated maximization in (6.9) yields:
for \( i\in \left\{1,2\right\} \) and \( k\in \left\{1,2,3\right\} \).
In particular, the optimal cooperative strategies can be obtained from (6.10) as:
for \( i\in \left\{1,2\right\} \) and \( k\in \left\{1,2,3\right\} \).
The expected joint profit under cooperation is given below.
Proposition 6.2
The value function indicating the maximized expected joint payoff is
where A k and C k , for \( k\in \left\{1,2,3\right\} \), are constants in terms of the parameters of the problem (6.8) and (6.2).
Proof
See Appendix D of this Chapter. ■
Using (6.11) and Proposition 6.2, the optimal cooperative strategies of the extracting firms can be expressed as:
Substituting ψ i k (x) from (6.13) into (6.2) yields the optimal cooperative state trajectory:
for \( k\in \left\{1,2,3\right\} \) and \( {x}_1={x}^0 \).
Dynamics (6.14) is a linear stochastic difference equation readily solvable by standard techniques. Let \( \left\{{x}_k^{*},\ \mathrm{f}\mathrm{o}\mathrm{r}\ k\in \left\{1,2,3\right\}\right\} \) denote the solution to (6.14).
Since the extractors agree to share the excess of cooperative gains over their expected noncooperative payoffs equally, an imputation
for \( k\in \left\{1,2,3\right\} \) and \( i\in \left\{1,2\right\} \) has to be maintained.
Invoking Theorem 4.1, if \( {x}_k^{*}\in X \) is realized at stage k a payment equaling
-
where \( {x}_{k+1}^{*\left({\theta}_k^y\right)}={x}_k^{*}+a-{\theta}_k^y{x}_k^{*}-{\displaystyle \sum_{j=1}^2}\left[P-{A}_{k+1}{\left(1+r\right)}^{k-1}\right]\frac{x_k^{*}}{2{c}_j} \), for \( y\in \left\{1,2,3\right\} \),
given to firm i at stage \( k\in \kappa \) would lead to the realization of the imputation (6.15).
A subgame consistent solution can be readily obtained from (6.13), (6.15) and (6.16).
7 A Heuristic Approach
In some game situations it may not be possible or practical to obtain all the information needed in this Chapter. Therefore a heuristic method may have to be considered to resolve the problem. To solve the problem in concern a heuristic method employs a practical methodology not guaranteed to be optimal or perfect, but sufficient for the immediate goals. Where finding an optimal solution is impossible or impractical, heuristic methods often prove to be able to speed up the process of finding a satisfactory solution. In particular, heuristic methods use strategies and information that are readily accessible (though not a 100% exact and accurate) to obtain a solution.
Consider the case of a heuristic approach to solving a subgame consistent solution in a situation where the differentiable functions
G k (x k )θ k , and
\( {g}_k^i\left[{x}_k,{u}_k^1,{u}_k^2,\cdots, {u}_k^n\right] \), for \( i\in \left\{1,2,\cdots, n\right\}\equiv N \) and \( k\in \left\{1,2,\cdots, T\right\}\equiv \kappa \), in (4.1 and 4.2)
are not available.
However, the players concur with the adoption of a set of cooperative strategies \( \Big\{{\widehat{\psi}}_k^i\left({x}_k\right) \), for \( k\in \kappa \) and \( i\in N\Big\} \). Though the cooperative strategies may not be the set of theoretically optimal controls they are perceived to be certainly beneficial to the joint well-being of all players.
In addition, with expert knowledge and statistical techniques the expected value of cooperative payment \( {\displaystyle \sum_{j=1}^n}{\widehat{g}}_{\tau}^j\left[{\widehat{x}}_{\tau },{\widehat{\psi}}_{\tau}^1\left({\widehat{x}}_{\tau}\right),{\widehat{\psi}}_{\tau}^2\left({\widehat{x}}_{\tau}\right),\cdots, {\widehat{\psi}}_{\tau}^n\left({\widehat{x}}_{\tau}\right)\right] \) received in each stage \( \tau \in \left\{k,k+1,k+2,\cdots, T\right\} \) can be estimated with acceptable degrees of accuracy. The value \( \widehat{W}\left(k,{\widehat{x}}_k\right) \) can be obtained by summing the cooperative payments \( {\displaystyle \sum_{j=1}^n}{\widehat{g}}_{\tau}^j\left[{\widehat{x}}_{\tau },{\widehat{\psi}}_{\tau}^1\left({\widehat{x}}_{\tau}\right),{\widehat{\psi}}_{\tau}^2\left({\widehat{x}}_{\tau}\right),\cdots, {\widehat{\psi}}_{\tau}^n\left({\widehat{x}}_{\tau}\right)\right] \) expected to be received in each stage from stage k to stage T for \( k\in \kappa \) along the cooperation path \( {\left\{{\widehat{x}}_{\tau}\right\}}_{\;\tau =k}^{\kern0.5em T} \), that is:
Again, with expert knowledge and statistical techniques the expected value of non-cooperative payment \( {\overline{g}}_{\tau}^i\left[{\overline{x}}_{\tau },{\overline{\phi}}_{\tau}^1\left({\overline{x}}_{\tau}\right),{\overline{\phi}}_{\tau}^1\left({\overline{x}}_{\tau}\right),\cdots, {\overline{\phi}}_{\tau}^1\left({\overline{x}}_{\tau}\right)\right] \) of player \( i\in N \) received in each stage \( \tau \in \left\{k,k+1,k+2,\cdots, T\right\} \) if the players revert to non-cooperation from stage k to stage T for \( k\in \kappa \) can be estimated with acceptable degrees of accuracy. The value \( {\overline{V}}^i\left(k,{\widehat{x}}_k\right) \) can be obtained by summing of the expected payments to be received by player i in each stage from stage k to stage T for \( k\in \kappa \) along the non-cooperation path \( {\left\{{\overline{x}}_{\tau}\right\}}_{\;\tau =k}^{\kern0.5em T} \) where \( {\overline{x}}_k={\widehat{x}}_k \), that is
If the agreed upon optimality principle specifies that the players share the expected total cooperative proportional to their expected noncooperative payoffs, then the imputation to player i becomes:
for \( i\in N \) and \( k\in \kappa \).
Invoking Theorem 5.1 a theoretically subgame consistent payment distribution procedure can be obtained with:
for i ∈ N,
given to player i at stage \( k\in \kappa \), if \( {x}_k^{*}\in {X}_k^{*} \).
Using (7.1, 7.2, 7.3 and 7.4) a subgame consistent PDP under a heuristic scheme can be obtained with:
given to player \( i\in N \) at stage \( k\in \kappa \), along the cooperation path \( {\left\{{\widehat{x}}_k\right\}}_{\;k=1}^{\kern0.5em T} \).
The heuristic approach allows the application of subgame consistent solution in dynamic game situations if estimates of the expected cooperative payoffs and individual non-cooperative payoffs with acceptable degrees of accuracy are available. This approach would be helpful to resolving the unstable elements in cooperative schemes for a wide range of game theoretic real-world problems.
8 Chapter Appendices
Appendix A. Proof of Proposition 3.1
Consider first the last stage, that is stage 3. Invoking that \( {V}^i\left(3,x\right)=\left[{A}_3^ix+{C}_3^i\right] \) from Proposition 3.1 and \( {V}^i\left(4,x\right)={\left(\frac{1}{1+r}\right)}^3qx \), the conditions in Eq. (3.4) become
Performing the indicated maximization in (8.1) yields the game equilibrium strategies in stage 3 as:
Substituting (8.2) into (8.1) yields:
Using (8.3), we can obtain A i3 and C i3 , for \( i\in \left\{1,2\right\} \).
Now we proceed to stage 2, the conditions in Eq. (3.4) become
Performing the indicated maximization in (8.4) yields the game equilibrium strategies in stage 2 as:
Substituting (8.5) into (8.4) yields
Substituting A i3 for \( i\in \left\{1,2\right\} \) into (8.6), A i2 and C i2 for \( i\in \left\{1,2\right\} \) are obtained in explicit terms.
Finally, we proceed to the first stage, the conditions in Eq. (3.4) become
Performing the indicated maximization in (8.7) yields the game equilibrium strategies in stage 1 as:
Substituting (8.8) into (8.7) yields:
Substituting the explicit terms for A i2 , A j2 , C i2 and C j2 from (8.6) into (8.9), A i1 and C i1 for \( i\in \left\{1,2\right\} \) are obtained in explicit terms. ■
Appendix B. Proof of Proposition 3.2
Consider first the last stage, that is stage 3. Invoking that \( W\left(3,x\right)=\left[{A}_3x+{C}_3\right] \) from Proposition 3.2 and \( W\left(4,x\right)=2{\left(\frac{1}{1+r}\right)}^3qx \), the conditions in Eq. (3.9) become
Performing the indicated maximization in (8.10) yields the optimal cooperative strategies in stage 3 as:
Substituting (8.11) into (8.10) yields:
Using (8.12), we obtain A 3 and C 3.
Now we proceed to stage 2, the conditions in Eq. (3.9) become
Performing the indicated maximization in (8.13) yields the optimal cooperative strategies in stage 2 as:
Substituting (8.14) into (8.13) yields:
Substituting A 3 into (8.15), A 2 and C 2 are obtained in explicit terms.
Finally, we proceed to the first stage, the conditions in Eq. (3.9) become
Performing the indicated maximization in (8.16) yields the optimal cooperative strategies in stage 1 as:
Substituting (8.17) into (8.16) yields:
Substituting the explicit terms for A 2 and C 2 from (8.15) into (8.18), A 1 and C 1 are obtained in explicit terms. ■
Appendix C. Proof of Proposition 6.1
Consider first the last operating stage, that is stage 3. Invoking that \( {V}^i\left(3,x\right)=\left[{A}_3^ix+{C}_3^i\right] \) from Proposition 6.1 and \( {V}^i\left(4,x\right)={\left(\frac{1}{1+r}\right)}^3q{x}_4, \) the conditions in Eq. (6.4) become
Performing the indicated maximization in (8.19) yields the game equilibrium strategies in stage 3 as:
Substituting (8.20) into (8.19) yields:
Using (8.21), we can obtain A i3 and C i3 , for \( i\in \left\{1,2\right\} \).
Now we proceed to stage 2, the conditions in Eq. (6.4) become
Performing the indicated maximization in (8.22) yields the game equilibrium strategies in stage 2 as:
Substituting (8.23) into (8.22) yields:
Substituting A i3 for \( i\in \left\{1,2\right\} \) into (8.24), A i2 and C i2 for \( i\in \left\{1,2\right\} \) are obtained in explicit terms.
Finally, we proceed to the first stage, the conditions in Eq. (6.4) become
Performing the indicated maximization in (8.25) yields the game equilibrium strategies in stage 1 as:
Substituting (8.26) into (8.25) yields:
Substituting the explicit terms for A i2 , A j2 , C i2 and C j2 from (8.24) into (8.27), A i1 and C i1 for \( i\in \left\{1,2\right\} \) are obtained in explicit terms.
Appendix D. Proof of Proposition 6.2
Consider first the last stage, that is stage 3. Invoking that \( W\left(3,x\right)=\left[{A}_3x+{C}_3\right] \) from Proposition 6.2 and \( W\left(4,x\right)=2{\left(\frac{1}{1+r}\right)}^3q{x}_4 \), the conditions in Eq. (6.9) become
Performing the indicated maximization in (8.28) yields the optimal cooperative strategies in stage 3 as:
Substituting (8.29) into (8.28) yields:
Using (8.30), we obtain A 3 and C 3.
Now we proceed to stage 2, the conditions in Eq. (6.9) become
Performing the indicated maximization in (8.31) yields the optimal cooperative strategies in stage 2 as:
Substituting (8.32) into (8.31) yields:
Substituting A 3 into (8.33), A 2 and C 2 are obtained in explicit terms.
Finally, we proceed to the first stage, the conditions in Eq. (6.9) become
Performing the indicated maximization in (8.34) yields the optimal cooperative strategies in stage 1 as:
Substituting (8.35) into (8.34) yields:
Substituting the explicit terms for A 2 and C 2 from (8.33) into (8.36), A 1 and C 1 are obtained in explicit terms.
9 Chapter Notes
Discrete-time dynamic games often are more suitable for real-life applications and operations research analyses. Properties of Nash equilibria in dynamic games are examined in Basar (1974, 1976). Solution algorithm for solving dynamic games can be found in Basar (1977a, b). Petrosyan and Zenkevich (1996) presented an analysis on cooperative dynamic games in discrete time framework. The SIAM Classics on Dynamic Noncoperative Game Theory by Basar and Olsder (1995) gave a comprehensive treatment of discrete-time noncooperative dynamic games. Bylka et al. (2000) analyzed oligopolistic price competition in a dynamic game model. Wie and Choi (2000) examined discrete-time traffic network. Beard and McDonald (2007) investigated water sharing agreements, and Amir and Nannerup (2006) considered resource extraction problems in a discrete-time dynamic framework. Krawczyk and Tidball (2006) considered a dynamic game of water allocation. Nie et al. (2006) considered dynamic programming approach to discrete time dynamic Stackelberg games. Dockner and Nishimura (1999) and Rubio and Ulph (2007) presented discrete-time dynamic game for pollution management. Dutta and Radner (2006) presented a discrete-time dynamic game to study global warming. Ehtamo and Hamalainen (1993) examined cooperative incentive equilibrium for a dynamic resource game. Yeung (2014) examined dynamically consistent collaborative environmental management with technology selection in a discrete-time dynamic game framework. Lehrer and Scarsini (2013) considered the core of dynamic cooperative games .
Discrete-time stochastic differential game analyses are less frequent than its continuous-time counterpart. Basar and Ho (1974) examined informational properties of the Nash solutions of stochastic nonzero-sum games. Elimination of informational nonuiqueness in Nash equilibrium through a stochastic formulation was first discussed in Basar (1976) and further examined in Basar (1975, 1979, 1989). Basar and Mintz (1972, 1973) and Basar (1978) developed equilibrium solution of linear-quadratic stochastic dynamic games with noisy observation. Bauso and Timmer (2009) considered robust dynamic cooperative games where at each point in time the coalitional values are unknown but bounded by a polyhedron. Smith and Zenou (2003) considered a discrete-time stochastic job searching model. Esteban-Bravo and Nogales (2008) analyzed mathematical programming for stochastic discrete-time dynamics arising in economic systems including examples in a stochastic national growth model and international growth model with uncertainty. Basar and Olsder (1995) gave a comprehensive treatment of noncooperative stochastic dynamic games . Yeung and Petrosyan (2010) provided the techniques in characterizing subgame consistent solutions for stochastic dynamic games subgame consistent solutions for stochastic dynamic games . Finally, a heuristic approach of obtaining subgame consistent solutions is provided in Sect. 7.7 to widen the application to a wide range of cooperative game problems in which only estimates of the expected cooperative payoffs and individual non-cooperative payoffs with acceptable degrees of accuracy are available.
10 Problems
-
(1)
Consider an economy endowed with a renewable resource and with 2 resource extractors (firms). The lease for resource extraction begins at stage 1 and ends at stage 3 for these two firms. Let u i k denote the rate of resource extraction of firm i at stage k, for \( i\in \left\{1,2\right\} \). Let U i be the set of admissible extraction rates, and \( {x}_k\in X\subset {R}^{+} \) the size of the resource stock at stage k. In particular, we have U i ∈ R+and \( {u}_k^1+{u}_k^2\le {x}_k \). The extraction cost for firms 1 and 2 are respectively
(u 1 k )2/x k and 1.5(u i k )2/x k .
The profits that firm 1 and firm 2 will obtain at stage k are respectively:
$$ \left[10{u}_k^1-\frac{4}{x_k}{\left({u}_k^1\right)}^2\right]\kern0.75em \mathrm{and}\ \left[4{u}_k^2-\frac{2}{x_k}{\left({u}_k^2\right)}^2\right]\kern0.5em . $$A terminal payment of 4x 4 will be given to each firm after stage 3.
The growth dynamics of the resource is governed by the difference equation:
$$ \begin{array}{cc}\hfill {x}_{k+1}={x}_k+20-0.1{x}_k-{\displaystyle \sum_{j=1}^2{u}_k^j},\hfill & \hfill \mathrm{f}\mathrm{o}\mathrm{r}\ k\in \left\{1,2,3\right\}\ \mathrm{and}\ {x}_1=24.\hfill \end{array} $$Characterize the feedback Nash equilibrium solution for the above resource economy.
-
(2)
If the extractors agree to cooperate and maximize their joint payoff, derive the optimal cooperative strategies and the optimal resource trajectory.
-
(3)
Consider the case when the extractors agree to share the excess of cooperative gains over their noncooperative payoffs equally. Derive a subgame consistent solution.
-
(4)
Consider an economy endowed with a renewable resource and with two resource extractors (firms). The lease for resource extraction begins at stage 1 and ends at stage 4 for these two firms. Let u i k denote the rate of resource extraction of firm i at stage k, for \( i\in \left\{1,2\right\} \). Let U i be the set of admissible extraction rates, and \( {x}_k\in X\subset {R}^{+} \) the size of the resource stock at stage k. In particular, we have U i ∈ R+ and \( {u}_k^1+{u}_k^2\le {x}_k \).
The profits that firm 1 and firm 2 will obtain at stage k are respectively:
$$ \left[5{u}_k^1-\frac{2}{x_k}{\left({u}_k^1\right)}^2\right]\kern0.75em \mathrm{and}\ \left[3{u}_k^2-\frac{1}{x_k}{\left({u}_k^2\right)}^2\right]\kern0.5em . $$A terminal payment of 3x 4 will be given to each firm after stage 4.
The growth dynamics of the resource is governed by the stochastic difference equation:
$$ {x}_{k+1}={x}_k+15-0.1{x}_k-{\displaystyle \sum_{j=1}^2{u}_k^j}+{\theta}_k{x}_k, $$for \( k\in \left\{1,2,3,4\right\} \) and \( {x}_1=55 \),
where θ k is a random variable with range {0, 0.1, 0.2} and corresponding probabilities {0.3, 0.5, 0.2}
Characterize a Nash equilibrium solution for the above discrete-time stochastic market game.
-
(5)
If the extractors agree to cooperate and maximize their expected joint payoff, derive the group optimal cooperative strategies .
-
(6)
Consider the case when the extractors agree to share the excess of expected cooperative gains proportional to their expected noncooperative payoffs. Derive a subgame consistent solution.
References
Amir, R., Nannerup, N.: Information structure and the tragedy of the commons in resource extraction. J. Bioecon. 8, 147–165 (2006)
Basar, T.: A counter example in linear-quadratic games: existence of non-linear Nash solutions. J. Optim. Theory Appl. 14, 425–430 (1974)
Basar, T.: Nash strategies for M-person differential games with mixed information structures. Automatica 11, 547–551 (1975)
Basar, T.: On the uniqueness of the Nash solution in linear-quadratic differential games. Int. J. Game Theory 5, 65–90 (1976)
Basar, T.: Two general properties of the saddle-point solutions of dynamic games. IEEE Trans. Autom. Control. AC-22, 124–126 (1977a)
Basar, T.: Existence of unique equilibrium solutions in nonzero-sum stochastic differential games. In: Roxin, E.O., Liu, P.T., Sternberg, R. (eds.) Differential Games and Control Theory II, pp. 201–228. Marcel Dekker, Inc., New York (1977b)
Basar, T.: Decentralized multicriteria optimization of linear stochastic systems. IEEE Trans. Autom. Control AC-23, 233–243 (1978)
Basar, T.: Information structures and equilibria in dynamic games. In: Aoki, M., Marzollo, A. (eds.) New Trends in Dynamic System Theory and Economics, pp. 3–55. Academic Press, New York/London (1979)
Basar, T.: Time consistency and robustness of equilibria in noncooperative dynamic games. In: Van der Ploeg, F., de Zeeuw, A. (eds.) Dynamic Policy Games in Economics, pp. 9–54. Elsevier Science Pub Co, Amsterdam/North-Holland (1989)
Basar, T., Ho, Y.C.: Informational properties of the nash solutions of two stochastic nonzero-sum games. J. Econ. Theory 7, 370–387 (1974)
Basar, T., Mintz, M.: On the existence of linear saddle-point strategies for a two-person zero-sum stochastic game. In: Proceedings of the IEEE 11th Conference on Decision and Control, New Orleans, LA, 1972, pp. 188–192, IEEE Computer Society Press, Los Alamitos, CA (1972)
Basar, T., Mintz, M.: A multistage pursuit-evasion game that admits a Gaussian random process as a maximum control policy. Stochastics 1, 25–69 (1973)
Basar, T., Olsder, G.J.: Dynamic Noncooperative Game Theory, 2nd edn. Academic Press, London (1995)
Bauso, D., Timmer, J.: Robust dynamic cooperative games. Int. J. Game Theory 38, 23–36 (2009)
Beard, R., McDonald, S.: Time-consistent fair water sharing agreements. Ann. Int. Soc. Dyn. Games 9, 393–410 (2007)
Bylka, S., Ambroszkiewicz, S., Komar, J.: Discrete time dynamic game model for price competition in an oligopoly. Ann. Oper. Res. 97, 69–89 (2000)
Dockner, E.J., Nishimura, K.: Transboundary pollution in a dynamic game model. Jpn. Econ. Rev. 50(4), 443–456 (1999)
Dutta, P.-K., Radner, R.: Population growth and technological change in a global warming model. Economic Theory 29(2), 251–270 (2006)
Ehtamo, H., Hamalainen, R.: A cooperative incentive equilibrium for a resource management. J. Econ. Dyn. Control 17(4), 659–678 (1993)
Esteban-Bravo, M., Nogales, F.J.: Solving dynamic stochastic economic models by mathematical programming decomposition methods. Comput. Oper. Res. 35, 226–240 (2008)
Krawczyk, J.B., Tidball, M.: A discrete-time dynamic game of seasonal water allocation. J. Optim. Theory Appl. 128(2), 411–429 (2006)
Lehrer, E., Scarsini, M.: On the core of dynamic cooperative games. Dyn. Games Appl. 3, 359–373 (2013)
Nie, P., Chen, L., Fukushima, M.: Dynamic programming approach to discrete time dynamic feedback stackelberg games with independent and dependent followers. Eur. J. Oper. Res. 169, 310–328 (2006)
Petrosyan, L.A., Zenkevich, N.A.: Game Theory. World Scientific Publishing Co. Pte. Ltd., Republic of Singapore (1996)
Rubio, S.J., Ulph, A.: An infinite-horizon model of dynamic membership of international environmental agreements. J. Environ. Econ. Manag. 54(3), 296–310 (2007)
Smith, A.E., Zenou, Y.: A discrete-time stochastic model of job matching. Rev. Econ. Dyn. 6, 54–79 (2003)
Wie, B.W., Choi, K.: The computation of dynamic Cournot-Nask traffic network equilibria in discrete time. KSCE J. Civ. Eng. 4(4), 239–248 (2000)
Yeung, D.W.K.: Dynamically consistent collaborative environmental management with production technique choices. Ann. Oper. Res. 220(1), 181–204 (2014)
Yeung, D.W.K., Petrosyan, L.A.: Subgame consistent solutions for cooperative stochastic dynamic games. J. Optim. Theory Appl. 145(3), 579–596 (2010)
Yeung, D.W.K., Petrosyan, L.A.: Subgame Consistent Economic Optimization: An Advanced Cooperative Dynamic Game Analysis. Birkhäuser, Boston, 395 pp (2012a)
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer Science+Business Media Singapore
About this chapter
Cite this chapter
Yeung, D.W.K., Petrosyan, L.A. (2016). Subgame Consistent Cooperative Solution in Dynamic Games. In: Subgame Consistent Cooperation. Theory and Decision Library C, vol 47. Springer, Singapore. https://doi.org/10.1007/978-981-10-1545-8_7
Download citation
DOI: https://doi.org/10.1007/978-981-10-1545-8_7
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-10-1544-1
Online ISBN: 978-981-10-1545-8
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)