Abstract
We study stochastic games of resource extraction, in which the players have identical preferences. The transition probability is either non-atomic or a convex combination of transition probabilities depending on the investment with coefficients also dependent on the investment. Our approach covers the unbounded utility case, which was not examined in this class of games beforehand. We prove the existence of a stationary Markov perfect equilibrium in a non-randomised class of strategies.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
1 Introduction
This paper deals with a strategic version of the discrete-time one-sector optimal growth model (see Bhattacharya and Majumdar 2007; Stachurski 2009), which plays a crucial role in both economic dynamics and resource economics. In principle, it can be described as follows. Two agents own a common natural resource and consume certain amount of the available stock in each time period. Their objective is to maximise their individual expected discounted utilities. The next state is usually given either by a deterministic production function or by some stochastic transition probability. The seminal paper of Levhari and Mirman (1980) studies such a strategic optimal growth model assuming that the players have the same logarithmic one-period utilities. Moreover, the next state \(s_{t+1}\) evolves according to the Cobb–Douglas production function, i.e., \(s_{t+1}=y_t^{\kappa }\), where \(\kappa \in (0,1)\) and \(y_t\) denotes a joint investment in period t.
Their model has been extended by Sundaram (1989), who considered general utility and production functions. Assuming that the players have identical preferences he proved the existence of a stationary Markov perfect equilibrium in the class of non-randomised strategies. Later Majumdar and Sundaram (1988) and Dutta and Sundaram (1992) reported the existence of a stationary Markov perfect equilibrium in the same class of strategies as Sundaram (1989), but for stochastic resource extraction games with symmetric payoffs. The former work analyses atomless transition probabilities, whereas the latter one embraces the conditions used in Sundaram (1989) and Majumdar and Sundaram (1988). Both papers deal with weakly continuous transition probabilities.
In this paper, we also prove the existence of a stationary Markov perfect equilibrium in the class of non-randomised strategies assuming identical preferences for the players and weak continuity of the transition probabilities. However, there are some essential features which not only distinguish our work from the aforementioned ones but also extend the previous results. Namely, in contrast to Sundaram (1989), Majumdar and Sundaram (1988), Dutta and Sundaram (1992), we deal with unbounded utility functions and we allow the state space to be the \([0,\infty )\) interval. The latter case requires additional work, since we have to show the compactness of certain function spaces. Furthermore, we propose two alternative sets of assumptions. The first set allows to study only games with non-atomic transition probabilities, whereas the second set embraces deterministic transitions and a class of transition probabilities not covered by Dutta and Sundaram (1992). Namely, we assume that the transition probability is a convex combination of transitions depending on the investment with coefficients also depending continuously on the investment. Particularly, our transition probabilities need not satisfy a stochastic dominance condition, whose stronger version plays a fundamental role in the proof of the equilibrium theorem in Dutta and Sundaram (1992).
The paper is organised as follows. The next section is devoted to a description of the game model. In Sect. 3, we provide the assumptions imposed on the transition probability and one-period utility function. Then, we formulate the main theorem and compare in detail our conditions with the ones used by Dutta and Sundaram (1992) and Majumdar and Sundaram (1988). The next section presents examples of transition probabilities that satisfy our assumptions, but not necessarily the ones in Dutta and Sundaram (1992). Section 4 contains compactness results, which are essential in the equilibrium proof. The proof of equilibrium result is given in Sect. 6 and is preceded by a sequence of auxiliary lemmas. Final comments and concluding remarks are put in Sect. 7.
2 The model
Let \(\mathbb {N}\) be the set of positive integers and \(\mathbb {R}\) be the set of all real numbers. Let \(S=[0,\infty )\) and \(S_+=(0,\infty ).\) We consider a discounted stochastic game for which:
-
(i)
S is the state space, i.e., the set of available resource stocks,
-
(ii)
\(A_i(s)=[0, s]\) is the set of actions available to player \(i\in P:=\{1,2\}\) in state \(s\in S,\)
-
(iii)
\(u_i:S \rightarrow \mathbb {R}\) is a non-negative stage utility function for player \(i\in P,\)
-
(iv)
for any \(s\in S\) and any feasible pair of actions \((a,b)\in A_1(s)\times A_2(s),\) that is, \(a+b\le s, q(\cdot |s-a-b)\) is a probability measure on S,
-
(v)
\(\beta \in (0,1)\) is a discount coefficient.
We assume that \(u_1=u_2 =:u\). The function u may be unbounded, but it satisfies the following condition.
-
(W1)
There exists a continuous increasing function \(w:S\rightarrow [1,\infty )\) such that \(0\le u (c)\le w( c)\) for all \(c\in S.\)
-
(W2)
There exists a constant \(\alpha >0\) such that \( \alpha \beta <1 \) and
$$\begin{aligned} \int _S w(z)q (\mathrm{d}z|y)\le \alpha w(y)\quad \text{ for } \text{ all } \quad y\in S \end{aligned}$$
In each period the players observe the state \(s\in S\) and simultaneously choose their actions \((a,b)\in A_1(s)\times A_2(s)\) that yield the utility vector (u(a), u(b)) provided that the actions are feasible, i.e., \(a+b\le s.\) A new state \(s'\) is realised from the probability distribution \(q(\cdot | s-a-b)\) and the next period begins. The stage utilities are discounted by \(\beta .\) Note that the actions available at any state to one player depend on those chosen by the other player. This model is known as a generalised game in the terminology of Debreu (1954), or a coupled constraint game as in Haurie et al. (2012). If the pair of actions (a, b) is infeasible in state s, then one can follow Dutta and Sundaram (1992) and assume that every player receives utility u(s / 2). We restrict attention to strategies generating feasible action pairs during the play. An equilibrium in the symmetric case (where the stage utility functions are identical) will consist of feasible strategies of the players.
A strategy for player \(i\in P\) is a sequence of Borel-measurable mappings from the history space to the space of actions available to her/him.Footnote 1 The set of strategies for player i is denoted by \(\varPi _i\) and its generic element by \(\pi _i.\) Let \(F_i\) be the set of all Borel-measurable functions \(\phi _i: S\rightarrow S\) such that \(\phi _i(s) \in A_i(s)=[0,s]\) for each \(s\in S.\) A stationary Markov strategy for player \(i\in P\) is a constant sequence \((\pi _{it})\) where \(\pi _{it}=\phi _i\) for some \(\phi _i\in F_i\) and for all \(t\in \mathbb {N}.\) Hence, a stationary Markov strategy for player i can be identified with the Borel-measurable mapping \(\phi _i\in F_i.\) For any feasible pair \((\pi _1,\pi _2)\in \varPi _1\times \varPi _2,\) an initial state \(s\in S\) and \(t\in \mathbb {N},\) by \(u_i^{(t)}(\pi _1,\pi _2)(s)\) we denote the expected utility for player i in the tth period of the game. The expected discounted utility for player \(i\in P\) is
A feasible profile of strategies \((\pi ^*_1,\pi ^*_2)\in \varPi _1\times \varPi _2\) is called a Nash equilibrium if
for each \(s\in S\) and \(\pi _1\in \varPi _1 \) such that \((\pi _1,\pi _2^*)\) is feasible, and
for each \(s\in S\) and \(\pi _2\in \varPi _2 \) such that \((\pi ^*_1,\pi _2)\) is feasible.
Definition 1
A Stationary Markov Perfect Equilibrium (SMPE) is a Nash equilibrium \((\phi ^*_1,\phi ^*_2)\) that belongs to the class of strategy pairs \(F_1\times F_2.\) An \(SMPE (\phi ^*_1,\phi ^*_2)\) is symmetric if \(\phi ^*_1=\phi ^*_2.\)
Remark 1
Conditions (W1)–(W2) are used to guarantee the convergence of the discounted utilities for the players. These assumptions have been already used in dynamic programming and in zero-sum stochastic games, see for instance, Wessels (1977), Hernández-Lerma and Lasserre (1999), Jaśkiewicz and Nowak (2011) and references cited therein. One can observe that under conditions (W1)-(W2), for any feasible pair of strategies \((\pi _1,\pi _2)\) and \(t\ge 2\), we have \(u^{(t)}_i(\pi _1,\pi _2)(s)\le \alpha ^{t-1}w(s),\) and consequently
Hence, the discounted expected utility is well-defined for any pair of feasible strategies for the players.
3 Main result
Let \(\Pr (S)\) be the set of all probability measures on the state space S. We recall that a sequence \((\mu _n)\) of probability measures on S converges weakly to some \(\mu _0\in \Pr (S)\) (\(\mu _n \Rightarrow \mu _0\) for short) if, for any bounded continuous function \(v:S\rightarrow \mathbb {R},\) we have \(\lim _{n\rightarrow \infty }\int _Sv(s)\mu _n(\mathrm{d}s)= \int _Sv(s)\mu _0(\mathrm{d}s)\); see Billingsley (1968).
We now formulate further assumptions which will be needed in our proofs.
(U) The function \(u :S\rightarrow \mathbb {R}\) is non-negative, increasing, strictly concave and continuous at \(s=0.\)
For the transition probabilities, we accept two alternative sets of conditions (A) or (B1)–(B3).
(A) The transition probability q is weakly continuous on S, that is, if \(y_m\rightarrow y_0\) in S, then \(q(\cdot |y_m)\Rightarrow q(\cdot |y_0)\) as \(m\rightarrow \infty .\) Moreover, for each \(y\in S_+,\) the probability measure \(q(\cdot |y)\) is non-atomic and \(q(\cdot |0)\) has no atoms in \(S_+.\)
-
(B1)
Assume that \(\lambda _j:S\rightarrow [0,1], j\in J:=\{1,\ldots ,l\},\) are continuous functions such that \(\sum _{j=1}^l \lambda _j(y)=1\) for all \(y\in S.\) In addition, suppose that there exist transition probabilities \(q_j\) from S to \(S, j\in J\), such that for each \(y\in S\), we have
$$\begin{aligned} q(\cdot |y)=\sum _{j=1}^l\lambda _j(y)q_j(\cdot |y). \end{aligned}$$(1)Moreover, for every \(j\in J,\) the transition probability \(q_j(\cdot |y)\) is weakly continuous on S and \(q_j(\{0\}|0)=1.\)
-
(B2)
Every transition probability \(q_j(\cdot |y)\) in (1) is either non-atomic for \(y\in S_+\) or it satisfies the stochastic dominance condition.Footnote 2
-
(B3)
For every \(s\in S\) the set \(Z^s =\{y\in S:\; q(\{s\}|y)>0\}\) is countable.
We also impose the following assumption.
(C) The function \(y\rightarrow \int _Sw(z)q(dz|y)\) is continuous on S.
We now define a special class of stationary strategies of the players. By \(F_i^0\), we denote the set of mappings \(\phi \in F_i\) such that the function \(\varphi (s):= s-\phi (s)\) is non-decreasing, upper semicontinuous and \(0\le \phi (s)\le s/2\) for all \(s\in S.\) Note that \(\varphi \) and thus \(\phi \) are continuous from the right. Clearly, \(\phi \in F^0_i\) is lower semicontinuous.
We can now state our main result.
Equilibrium Theorem Let either (A) or (B1)–(B3) hold. Assume that (U), (W1)–(W2) and (C) are also satisfied. Then, the game has a symmetric \(SMPE (\phi ^*,\phi ^*)\in F^0_1\times F^0_2.\)
Remark 2
The result of Dutta and Sundaram (1992) is based on the following assumption.
-
(DS)
For any \(s\in S_+\) and \(y_1<y_2, \) we have
$$\begin{aligned} \lim _{z\nearrow s}Q(z|y_1):=Q(s^-|y_1)\ge Q(s|y_2), \end{aligned}$$where \(Q(z|y):=q([0,z]|y).\)
This assumption is called strong stochastic dominance. Clearly, condition (DS) implies the stochastic dominance property called by Dutta and Sundaram (1992) the weak stochastic dominance. It is not difficult to construct \(q(\cdot |y)\) that has the stochastic dominance property and does not satisfy condition (DS). In Examples 4 and 5 we provide transition probabilities q for which conditions (B1)–(B3) hold and (DS) is not satisfied.
Remark 3
The predecessors of our work on symmetric dynamic games of resource extraction are Dutta and Sundaram (1992), Majumdar and Sundaram (1988) and Sundaram (1989). The first two papers deal with stochastic transition probabilities, whereas the last one studies the purely deterministic case. As noted by Dutta and Sundaram (1992), their assumptions cover the ones studied not only by Sundaram (1989) but also by Majumdar and Sundaram (1988), who examined games with atomless transition probabilities. We propose two alternative set of assumptions: (A) or (B1)–(B3). Conditions (A) require \(q(\cdot |y)\) to be non-atomic measure on S for any \(y\in S_+.\) However, in contrast to the aforementioned papers our novelty is twofold. First, we allow u to be unbounded. As pointed out by Bhattacharya and Majumdar (2007) or Stachurski (2009), such unbounded utilities are commonly used in the theory of economic growth. Second, we get rid of compactness of the resource space S.
More importantly, the alternative set of conditions (B1)–(B3) embraces transition probabilities which were not covered by Dutta and Sundaram (1992). Specifically, our assumptions allow to consider transition probabilities that do not satisfy stochastic dominance property, even its weak version. As mentioned in Remark 2, the strong stochastic dominance was the crux in their proof on existence of a symmetric equilibrium. In particular, conditions (B1)–(B3) enable to analyse transition probabilities which are convex combinations of deterministic transitions with coefficients depending on the investment. Such transition probabilities, as observed in Examples 4–5 in Sect. 4, do not meet the stochastic dominance property. Hence, they do not satisfy the strong version of this condition either.
Our proofs owe much both the techniques developed in Majumdar and Sundaram (1988) and Dutta and Sundaram (1992), and the methods used in the study of multigenerational games; see Balbus et al. (2015b) and references cited therein. In the main body of our proof we exploit assumption (B3) instead of (DS). Assumption (B3) was applied by Balbus et al. (2015b), who studied a different class of non-cooperative games with countably many players called multigenerational games. Here, we only mention that under the (weak) stochastic dominance property condition (B3) implies (DS). However, in general assumptions accepted by Dutta and Sundaram (1992) and in this paper do not coincide. Finally, we wish to mention that our proof, in contrast to Dutta and Sundaram (1992) and Majumdar and Sundaram (1988), does not require an analysis of the generalised game in the sense of Debreu (1954).
Remark 4
Dutta and Sundaram (1992) and Majumdar and Sundaram (1988) also impose other conditions except either strong stochastic dominance or atomless of transition probabilities, respectively. For instance, they assume that for a positive, sufficiently small level of investment, the stock tomorrow is no less than investment today with probability one. This requirement is usually referred to as the Inada condition. Moreover, they also assume that the utility function is differentiable and its right-hand side derivative at zero is \(+\infty .\) These two assumptions allow them to obtain an interior symmetric equilibrium. However, their analysis exclude the following utility functions: \(u(c)=1-e^{-c}\) or \(u(c)=r\ln (1+c),\) where r is arbitrary positive constant. Here, we do not impose the Inada conditions, but then in equilibrium \((\phi ^*,\phi ^*)\) it may happen that \(\phi ^*(s)\) is zero or s / 2 for some states.
We do not assume we do not assume other assumptions made by Dutta and Sundaram (1992) either. Specifically, they assume that there is no free production, that strictly positive investment today results in strictly positive stock tomorrow and that there is a maximum sustainable stock.
4 Examples
In this section, we provide some examples of transition probabilities and utility functions for which our assumptions hold. We start with condition (A). A natural transition probability satisfying this assumption is given by the recurrence equation
where \(y_t=s_t - a_t\) is the investment in state \(s_t\), \((\xi _t)\) is a sequence of i.i.d. random “shocks” having a probability distribution \(\pi \in \Pr (S).\) The function \(\bar{f}: S\times S\rightarrow S\) is continuous and for any Borel set D in S, investment \(y\in S,\) the transition probability is
Here, \(1_D\) denotes the indicator function of the set D. We now point out three special cases for the above recurrence equation.
Example 1
Let \(\bar{f}(y_t,\xi _t)= \xi _tf_1(y_t) +(1-\xi _t)f_2(y_t)\), where \(f_i:S \rightarrow S\) is continuous, increasing for \(i=1,2\) and \(f_1(y)>f_2(y)\) for \(y\in S_+.\) Moreover, assume that \(f_1(0)=f_2(0)=0.\) In addition, \(\pi \) is a non-atomic probability measure on [0, 1].
Example 2
A model with additive shocks. Let \(\bar{f} (y_t,\xi _t)= f (y_t) +\xi _t,\) where \(f :S\rightarrow S\) is a continuous increasing function. The probability measure \(\pi \) is non-atomic with support included in \([0, +\infty ).\)
Example 3
A model with multiplicative shocks. Assume that \(\bar{f}(y_t,y_t)= f (y_t)\xi _t,\) where f is as in Example 2 and the probability measure \(\pi \) is non-atomic with support included in \([0,+\infty ).\)
Next we give three examples of transition probabilities satisfying conditions (B1)–(B3) with \(l=2.\)
Example 4
Let the transition probability be as follows:
and let \(f_1, f_2\) be as in Example 1. The symbol \(\delta _{f_i(y)}(\cdot )\) denotes the Dirac measure concentrated at the point \(f_i(y).\) The functions \(\lambda _1,\lambda _2: S\rightarrow [0,1]\) are continuous and \(\lambda _1(y)+\lambda _2(y)=1\) for all \(y\in S.\) Obviously, this transition probability satisfies conditions (B1) and (B2). In order to see that (B3) holds as well, choose any \(s\in S\) and note that \(Z^s= \{y\in S: f_i(y)=s \}\) consists of at most two elements. In the pure deterministic case (where \(q(\cdot |y) =\delta _{f_1(y)}(\cdot )\)) \(Z^s\) has at most one point. Furthermore, let \(E_q(y)\) we denote the expected value of the resource stock following \(y\in S,\) that is,
Assume that \(f_1(y)= y+\sqrt{y}\), \(f_2(y)=y,\) for \(y\in S.\) Moreover, let \(\lambda _1(y)= y\) for \(y\in [0,1], \lambda _1(y)= 1/y\) for \(y\ge 1\) and \(\lambda _2(y)=1-\lambda _1(y)\) for each \(y\in S.\) Let \(Q(z|y):= q([0,z]|y).\) Note that \(Q(z|4)= 3/4\) for \(z\in [4,6)\) and \(Q(z|5)= 4/5\) for \(z\in [5,5+\sqrt{5}).\) Hence, \(Q(z|4) <Q(z|5)\) for \(z\in [5,6),\) i.e., \(q(\cdot |y)\) does not satisfy the stochastic dominance condition. Observe that in this case
Hence, the function \(y\rightarrow E_q(y)\) is continuous and increasing, although the stochastic dominance condition fails.
Example 5
Consider now the transition probability from Example 4 with the following functions: \(f_1(y)= y\), \(f_2(y)=\sqrt{y},\) for \(y\in S\) and \(\lambda _1(y)=1-\lambda _2(y),\) for \(y\in S,\) where \(\lambda _2(y)= y\) for \(y\in [0,1], \lambda _2(y)= 1-1/y\) for \(y\ge 1.\) Observe that \(Q(z|9)= 8/9\) for \(z\in [3,9)\) and \(Q(z|16)= 15/16\) for \(z\in [4,16).\) Hence \(Q(z|9) <Q(z|16)\) for \(z\in [4,9),\) i.e., \(q(\cdot |y)\) does not satisfy the stochastic dominance condition. Observe that
Hence, the expected value \(E_q(y)\) is again an increasing and continuous function with respect to \(y\in S.\)
Example 6
Assume now that
where the functions \(\lambda _i\) are as in (B1) and the function \(f: S\rightarrow S\) is continuous and increasing with \(f(0)=0.\) If \(q_1(\cdot |y)\) is non-atomic for \(y>0 \) and \(q(\{0\}|0)=1,\) then this transition probability satisfies assumptions (B1)–(B3). Note that the stochastic dominance condition may fail in this case either.
Finally, we give two examples satisfying conditions (W1)–(W2) and either (A) or (B1)–(B3).
Example 7
A model with Cobb–Douglas production function with constant depreciation. Suppose that the resource stock evolves due to the following recursive equation
where \(\theta \in (0,1)\) and \(\delta \in (0,1]\) is a depreciation rate. Suppose that \(\bar{z}:=\int _{S}\xi \pi (d\xi )\) is finite and \(\pi \) is a non-atomic measure with support included in \([0,+\infty ).\) In addition, let the utility function be given by \(u(c)=c^\sigma \) with \(\sigma \in (0,1).\) Clearly, assumptions (U) and (A) are satisfied. We claim that (W2) also holds for \(w(y)=(r+y)^\sigma \) with arbitrary \(r\ge 1.\) Then,
Analogous calculations to the ones on p. 263 in Jaśkiewicz and Nowak (2011) show that (W2) is met with
where \(r\ge 1\) must be sufficiently large so that \(\alpha \beta <1.\) Obviously, (C) also holds true.
Example 8
Let \(u(c)=\sqrt{c}\) for each \(c\in S\) and
where each function \(f_j: S\rightarrow S\) is continuous, increasing and \(f_j(0)=0\) for every \(j=1\in J.\) The coefficient \(\lambda _j(y)\) describes a chance that the resource stock will be \(f_j(y).\) Moreover, let the functions \(\lambda _j, j\in J,\) be as in (B1). Assume that there exist two constants \(\rho >0\) and \(\theta >1\) such that \(\theta \beta <1\) and \(f_j(y)\le \rho +\theta y\) for all \(j\in J\) and \(y\in S.\) For some \(r\ge 1\) define \(w(y)= r+y, y\in S\). Then,
Hence, it follows that
If we set \(\alpha := \frac{\rho }{r} +\theta \), then (W2) holds for sufficiently large values of r. Note that (W1), (U), (C) hold as well. Moreover, this transition probability satisfies (B1)–(B3).
Other transition probabilities and utility functions satisfying our conditions can be obtained by an adaptation of examples from Sect. 4 in Jaśkiewicz and Nowak (2011) or from Sect. 6 in Jaśkiewicz and Nowak (2018).
5 Basic compactness lemmas
Let X be the vector space of all continuous from the right functions \(\phi :S\rightarrow \mathbb {R}\) with bounded variation on every interval \([0,n], n\in \mathbb {N}.\) We assume that X is endowed with the topology of weak convergence. Recall that a sequence \((\varphi _m)\) converges weakly to \(\varphi \in X\) iff \(\varphi _m(s)\rightarrow \varphi (s)\) as \(m\rightarrow \infty \) at any continuity point \(s\in S\) of \(\varphi .\) The weak convergence of \((\varphi _m)\) to \(\varphi \) is denoted by \(\varphi _m{\mathop {\rightarrow }\limits ^{\omega }} \varphi \).
Let \(\eta :S\rightarrow S\) be a continuous increasing function. We define \(X^\eta \) as the set of all non-decreasing functions \(\varphi \in X\) such that \(0\le \varphi (s)\le \eta (s)\) for all \(s\in S.\) Note that each \(\varphi \in X^\eta \) is upper semicontinuous. Observe that 0 is a continuity point of every function \(\varphi \in X^\eta .\)
Proposition 1
The set \(X^\eta \) is convex and sequentially compact in X.
Proof
It is obvious that \(X^\eta \) is convex. For any \(f\in X^\eta \) and \(m\in \mathbb { N}\), we define the function \(f^m\) as follows: \(f^m(s)=f(s)\) for all \(s\in [0,m)\) and \(f_m(s) = \eta (m)\) for al \(s\ge m.\) Then \(f^m \in X^\eta \) and can be viewed as a continuous from the right “distribution function” of some non-negative countably additive measure \(\nu _m\) such that \(\nu _m(S)=\eta (m).\)
Consider now an arbitrary sequence \((\varphi _k)\) of functions in \(X^\eta .\) We now apply the standard “diagonal method”. By Helly’s selection theorem [see p. 227 in Billingsley (1968)], there exists a subsequence \((n_1(k))\) of (n) such that \((\varphi ^1_{n_{1}(k)})\) converges weakly (as \(k\rightarrow \infty \)) to some \(\gamma ^1\in X^\eta .\) Next, there exists a subsequence \((n_2(k))\) of \((n_1(k))\) such that \((\varphi ^2_{n_{2}(k)})\) converges weakly to some \(\gamma ^2\in X^\eta \) and \(\gamma ^2(s)=\gamma ^1(s)\) for each \(s\in [0,1).\) By induction we infer that for any \(r\ge 2\), there exists a subsequence \((n_r(k))\) of \((n_{r-1}(k))\) such that \((\phi ^r_{n_{r}(k)})\) converges weakly to some \(\gamma ^r\in X^\eta \) and \(\gamma ^r(s)=\gamma ^{r-1}(s)\) for each \(s\in [0,r-1).\) Define \(\gamma (s):= \gamma ^m(s)\) if \(s\in [0,m), m\in \mathbb {N}.\) Then, \(\gamma \in X^\eta .\) Consider the “diagonal sequence” defined by \(d(k):= n_k(k)\), \(k\in \mathbb {N}.\) Then, \((\varphi _{d(k)}){\mathop {\rightarrow }\limits ^{\omega }} \gamma \) as \(k\rightarrow \infty .\) Thus \(X^\eta \) is sequentially compact. \(\square \)
Let \(Y=X^\eta \) where \(\eta (s)=s\) for all \(s\in S\) and let \(Y^0\) be the subset of all \(\varphi \in Y\) such that \(\varphi (s)\ge s/2.\) Observe that \(F_i^0=K(Y^0)\) where K is the continuous mapping defined by \(K(\varphi )(s)= s-\varphi (s), s\in S.\) From Proposition 1, we obtain the following conclusion.
Proposition 2
Y and \(F_i^0\) are convex and sequentially compact spaces when endowed with the topology of weak convergence.
Observe that every \(\phi \in F^0_i\) is lower semicontinuous and continuous from the right, but it need not be non-decreasing. The function \(s\rightarrow s-\phi (s)\) that belongs to \(Y^0\) is non-decreasing and upper semicontinuous. This fact will be used frequently in our considerations.
6 Proofs
In this section, \(X^\eta \) is considered with \(\eta (s)=w(s)/(1-\alpha \beta ) \) for all \(s\in S.\)
Lemma 1
Assume that \(f_n {\mathop {\rightarrow }\limits ^{\omega }} f\) in \(X^\eta \) and \(y_n\rightarrow y_0\) in S as \(n\rightarrow \infty .\) Then, \(f(y_0)\ge \limsup _{n\rightarrow \infty } f_n(y_n).\)
Proof
Let \(y>y_0\) be a continuity point of f. Then, there is some \(N\in \mathbb {N}\) such that \(y_n<y\) for all \(n>N.\) Hence, \(f_n(y_n)\le f_n(y)\) for \(n>N\) and consequently,
Since y can be chosen arbitrarily close to \(y_0\) and f is continuous from the right, we deduce that \(\limsup _{n\rightarrow \infty }f_n(y_n)\le f(y_0).\) \(\square \)
Lemma 2
Let either (A), (W1), (C) or (B1), (W1), (C) hold. Assume that \(f_n {\mathop {\rightarrow }\limits ^{\omega }} f\) in \(X^\eta \) and \(y_n\rightarrow y\) in S as \(n\rightarrow \infty .\) Then, it follows that
Proof
Let \(\tilde{f}_n(z):=f_n(z)-w(z)/(1-\alpha \beta )\) for \(z\in S.\) Define
and observe that by the continuity of w
By Lemma 3.2 in Serfozo (1982) and Lemma 1, we have
On the other hand, by (C) we obtain
Thus, the result follows. \(\square \)
Lemma 3
Let (A), (W1) and (C) hold. Assume that \( f \in X^\eta \) and \(y_n \rightarrow y\) in S as \(n\rightarrow \infty .\) Then, we have
Proof
For any \(z\in S\) define
The function \(f_*\) is lower semicontinuous and \(f_*(z)=f(z)\) for any continuity point \(z\in S\) of f (hence, \(f_*(0)=f(0)\) as well). Therefore, by (A) we infer
By Lemma 3.2 in Serfozo (1982), it follows that
These facts and Lemma 2 yield
Thus, the result follows. \(\square \)
Lemma 4
Let (B1), (B2), (W1) and (C) hold. Assume that \(y_n\searrow y\) in S as \(n\rightarrow \infty \) and \(f\in X^\eta .\) Then, it follows that
Proof
By (B1), we have
where the functions \(\lambda _j\) are continuous. If \(q_j\) is non-atomic, then by Lemma 3 (with \(q:=q_j\)), we obtain
Suppose that \(q_j\) admits some atoms and satisfies the stochastic dominance condition. Since f is non-decreasing, by Corollary 3.9.1(a) in Topkis (1998) , we have
for all \(n\in \mathbb {N}.\) Using (2) or (3) and Lemma 2, we obtain that
These inequalities finish the proof. \(\square \)
Let \(\phi \in F_2^0\) and \(\varPi _1(\phi )\) be the set of all strategies \(\pi _1\) for player 1 such that the pair \((\pi _1,\phi )\) is feasible. Define
In below lemmas we assume that (C), (U), (W1)–(W2) hold. In addition, we assume that (A) or (B1), (B2) are satisfied.
Lemma 5
Let \(\varPhi (s)=[0,s-\phi (s)]\) for each \(s\in S.\) The function \(V_\phi \) is non-decreasing and upper semicontinuous and belongs to the space \( X^\eta .\) Moreover, \(V_\phi \) is the unique solution to the Bellman equation, i.e.,
for all \(s\in S.\)
Proof
For any function \(V\in X^\eta \) define the dynamic programming operator T as follows:
By Lemma 2, the fact that u is continuous and increasing and the function \(s\rightarrow s-\phi (s)\) upper semicontinuous, it follows that the function \((s,y) \rightarrow u(s-\phi (s)-y)+\beta \int _SV(z)q(\mathrm{d}z|y)\) is upper semicontinuous. Since the correspondence \(s\rightarrow \varPhi (s)\) has a closed graph, it follows by Theorem 2 in Berge (1963) that TV is upper semicontinuous.
Since \(s\rightarrow s-\phi (s)\) is non-decreasing, we observe that \(u(s-\phi (s)-y)\le u(s'-\phi (s')-y)\) for \(s<s'\) and all \(y\in \varPhi (s).\) Hence, \(TV(s)\le TV(s'),\) because \(\varPhi (s)\subset \varPhi (s').\) Thus, the above considerations and (W1)–(W2) imply that \(TV\in X^\eta \). We claim that T is contractive. Indeed, by (W2) for any \(V_1, V_2\in X^\eta \) we obtain that
Defining the w-norm of any function \(V\in X^\eta \) as follows \(\Vert V\Vert _w:=\sup _{s\in S} \frac{|V(s)|}{w(s)},\) it is easily seen that the last display implies the inequality
By the Banach fixed point theorem, there exists a unique function \(V_\phi \in X^\eta \) such that \(TV_\phi =V_\phi .\) Note that \(X^\eta \) is a closed subset of the Banach space of all Borel-measurable functions with the finite w-norm. By standard programming arguments (see Hernández-Lerma and Lasserre 1999), it follows that \(V_{\phi }(s)=\sup _{\pi _1\in \varPi _1(\phi )} U_1(s,\pi _1,\phi )\) for all \(s\in S,\) which completes the proof. \(\square \)
Define
Obviously, the set \(A_{\phi }(s)\) is non-empty and compact. For any \(s\in S\) we set
Lemma 6
The mapping \(s\rightarrow A_\phi (s)\) is ascending, i.e., if \(s_1<s_2\) and \(y_1\in A_\phi (s_1), y_2\in A_\phi (s_2),\) then \(y_1\le y_2.\)
Proof
Suppose that \(s\rightarrow A_\phi (s)\) is not ascending. Then there exist \(s_1 < s_2\) and \(y_1 \in A_\phi (s_1), y_2 \in A_\phi (s_2)\) such that \(y_1 > y_2.\) Clearly, the set \(\mathcal{L}:=\{(s,y):s\in S, \ y\in \varPhi (s)\}\) is a lattice with the usual component-wise order on the plane \(\mathbb {R}^2.\) Thus \((s_2,y_1)\) and \((s_1,y_2)\) belong to \(\mathcal{L}\). Since u is strictly concave, from the proof of Lemma 2 in Nowak (2006) and the fact that \(s_2-\phi (s_2)>s_1-\phi (s_1),\) we conclude the following
Adding \(\beta \int _SV_\phi (z)q(\mathrm{d}z|y_1)-\beta \int _SV_\phi (z)q(\mathrm{d}z|y_2)\) to both sides of (5) and remembering that \(y_1 \in A_\phi (s_1)\) and \(y_2 \in A_\phi (s_2)\), we obtain
This contradiction implies that the correspondence \(s\rightarrow A_\phi (s)\) is ascending. \(\square \)
Lemma 7
Let \(\psi \) be any selector of the correspondence \(s\rightarrow A_\phi (s)\), i.e., \(\psi (s)\in A_\phi (s)\) for all \(s\in S.\) If \(\psi \) is continuous at \(s_0\), then \(A_\phi (s_0)\) is a singleton.
Proof
Clearly, \(\psi (0)=0.\) Hence, it is enough to consider \(s_0>0.\) Suppose that \(y_1\) and \(y_2\) belong to \(A_\phi (s_0)\) and \(y_1 < y_2\). Since \(s\rightarrow A_\phi (s)\) is ascending, we conclude that \(\psi \) is non-decreasing. Therefore, we have \(\lim _{s\rightarrow s_0^-}\psi (s) \le y_1 <y_2\le \lim _{s\rightarrow s_0^+}\psi (s).\) This contradicts our assumption that \(\psi \) is continuous at \(s_0\in S_+.\) \(\square \)
Lemma 8
The function \(g(\phi )\) is the unique non-decreasing and continuous from the right selector of the correspondence \(s\rightarrow A_\phi (s).\)
Proof
Clearly, by Lemma 6 the function \(g(\phi )\) is non-decreasing. Note that the graph of the correspondence \(s\rightarrow A_\phi (s)\) is closed from the right, i.e., for any \(s_n\searrow s\) and \(y_n\in A_\phi (s_n)\) such that \(y_n\) converges to some y, it follows that \(y\in A_\phi (s).\) Therefore, we infer that \(g(\phi )\) is continuous from the right. Consequently, \(g(\phi )\) is upper semicontinuous. The uniqueness is a consequence of Lemma 7. \(\square \)
Proof of the equilibrium theorem
Define the operator \(L:F^0_2\rightarrow F^0_1=F^0_2\) by
By Lemma 8, \(L\phi \in F^0_1.\) We must show that L is continuous. Suppose that \(\phi _n{\mathop {\rightarrow }\limits ^{\omega }} \phi \) as \(n\rightarrow \infty .\) By Proposition 1, we can assume without loss of generality that \(V_n:=V_{\phi _n} {\mathop {\rightarrow }\limits ^{\omega }} V\) in \(X^\eta \) (if necessary take a subsequence). By Proposition 2, we also assume that \(\psi _n:=g(\phi _n) {\mathop {\rightarrow }\limits ^{\omega }} \psi \) in Y. Thus, for each \(n\in \mathbb {N},\) we have
Let \(S_1\subset S\) be the set of all continuity points of the functions V, \(\phi \) and \(\psi .\) For any \(s\in S_1\), \(V_n(s)\rightarrow V(s), \phi _n(s)\rightarrow \phi (s)\) and \(\psi _n(s)\rightarrow \psi (s)\) as \(n\rightarrow \infty .\) By (6), Lemma 2 and assumption (U), we obtain that
Let \(s\not \in S_1.\) Since \(S_1\) is dense in S and the functions V, \(\psi \) and \(\phi \) are continuous from the right, we may choose a sequence \((s_m)\) in \(S_1\) such that \(s_m\searrow s\) as \(m\rightarrow \infty .\) Thus, we have
Letting \(m\rightarrow \infty \) in the above inequality, we conclude from Lemma 2, the continuity from the right of V and \(s\rightarrow s-\phi (s)\) that (7) holds for all \(s\in S.\)
On the other hand, for any \(n\in \mathbb {N}, s\in S\) and \(y\in [0,s-\phi _n(s)],\) we have the following inequality
Let \(S_d\) be the countable set of discontinuity points of the function V. Note that \(0\not \in S_d.\) By \(S_2\) we denote the set of all continuity points of the functions V and \(\phi .\) Further define \(S_3\) as the set of all \(y\in S\) such that \(q(S_d|y)=0.\) The set \(S_2\) is dense in S and the set \(S_3\) is also dense in S either by (A) or (B3). Clearly, by (A) or (B1), the state \(0\in S_3.\) Choose any \(s\in S_2\cap S_+\) and \(y\in S_3\cap [0,s-\phi (s)).\) Then, there exists some \(N\in \mathbb {N}\) such that, \(y\in [0,s-\phi _n(s)]\) for all \(n>N.\) Hence, we have
By the dominated convergence theorem and the fact that \(y\in S_3\), we have
Therefore, we can conclude that
for \(y\in [0,s-\phi (s))\cap S_3\) and \(s\in S_2\cap S_+.\) Now let us consider \(s_0\in S\) and \(y_0\in [0,s_0-\phi (s_0)].\) There exist sequences \((s_m)\) and \((y_m)\) such that \(s_m\searrow s_0, y_m\searrow y_0\) as \(m\rightarrow \infty \) and \(s_m\in S_2\cap S_+, y_m\in S_3\cap [0,s_m-\phi (s_m))\) for all \(m\in \mathbb {N}.\) Observe that \(s_m-\phi (s_m)\ge s_0-\phi (s_0).\) Hence, by (9) we obtain
Letting m tend to infinity and making use of Lemma 3 in case of assumption (A) or of Lemma 4 in case of assumptions (B), the continuity of u and the continuity from the right of the functions \(V, s\rightarrow s-\phi (s)\), we deduce that inequality (9) holds for \(s_0\in S\) and \(y_0\in [0,s_0-\phi (s_0)].\)
By (7) and (9), we obtain for any \(s\in S\) that
Since \(\psi \) is non-decreasing and upper semicontinuous, from Lemma 8, it follows that \(\psi = g(\phi ).\) Thus, the operator L is continuous. By the Schauder–Tychonoff fixed point theorem, there exists \(\phi ^*\in F^0_2\) such that \(L\phi ^*=\phi ^*.\) This implies that \(\phi ^*\) is the best response of player 1 to the strategy \(\phi ^*\) chosen by player 2. Since the game is symmetric, it follows that \((\phi ^*,\phi ^*)\) is a SMPE. \(\square \)
7 Concluding remarks
This section contains a list of conclusions and open problems:
-
1.
Our result on the equilibrium existence is also valid for bounded state space \(S=[0,\bar{s}],\) with some \(\bar{s}>0.\) Then, it is enough to put \(w\equiv 1\) in (W1) and (W2). However, in order to avoid upper-endpoint problems in S, we need to apply the trick used, for instance, by Dutta and Sundaram (1992). They expand the state space to \(S^*=[0,s^*]\) with some \(s^*>\bar{s}\) and study a set of investment functions defined on \(S^*.\) Particularly, Dutta and Sundaram (1992) deal with upper semicontinuous non-decreasing functions on \(S^*\) whose values at s do not exceed s and such that the value of such a function at \(s^*\) is \(s^*.\)
-
2.
The problem of proving the existence of a Nash equilibrium in a stochastic game of resource extraction with different utility functions for the players seem to be difficult. Partial results were reported by Amir (1996), Balbus and Nowak (2008), Jaśkiewicz and Nowak (2015), where specific structures of transition probabilities were accepted. For example, Amir (1996) analysed so-called “convex transitions”. More precisely, he assumed that the conditional cumulative distribution function induced by the transition probability is convex with respect to investments. He proved the existence of pure stationary Markov perfect equilibria in the class of Lipschitz continuous strategies. The convexity assumption imposed on the transition functions made by Amir (1996) is very restrictive. It holds, for example, if the transition probability is a convex combination of some probability measures on the state space with coefficients depending on joint investments. This class was also examined Balbus and Nowak (2008). However, as argued by Jaśkiewicz and Nowak (2015), this type of transition probabilities makes sense only in the bounded state space case. The most general class of non-symmetric resource extraction games was studied by Jaśkiewicz and Nowak (2015), who considered transition probabilities being a convex combinations of transition probabilities depending on the state space and coefficients depending on a joint investment. Under these conditions they proved the existence of pure stationary Markov perfect equilibria. For further comments and references the reader is referred to Jaśkiewicz and Nowak (2018).
-
3.
The problem of an equilibrium existence in non-symmetric stochastic games with weakly continuous transition probabilities is an open non-trivial problem. There are two main reasons. First, the state space is uncountable. Second, the payoff functions for either player i in the auxiliary one-shot game with a continuation vector function \(v=(v_1,v_2)\) are neither convex nor concave. More precisely, the function
$$\begin{aligned} y\rightarrow u_i(c_i)+\beta \int _S v_i(z)q(\mathrm{d}z|y), \quad c=(c_1,c_2) \end{aligned}$$is neither convex nor concave for some non-decreasing function \(v_i\) on S. Hence, we cannot apply the Nash theorem (see Nash 1951). Furthermore, if we allow to study a general form of the transition probability \(q(\cdot |s,c),\) then the auxiliary games are not supermodular in the sense of Milgrom and Roberts (1990) either. Therefore, we cannot apply the techniques from lattice programming, see Topkis (1998). Further examples and a detailed discussion can be found in Jaśkiewicz and Nowak (2015) and Jaśkiewicz and Nowak (2018).
-
4.
Finally, we would like to pay attention of researchers to the paper of Amir (1989). It contains some errors that cannot be fixed. Specifically, the limit argument given on p. 1349 is incorrect. Therefore, the problem of equilibrium existence in non-symmetric deterministic dynamic games of resource extraction is still open.
Notes
It is natural to restrict attention to deterministic consumption strategies.
If \(z\rightarrow Q_j(z|y):=q_j([0,z]|y)\) is the cumulative distribution function for \(q_j(\cdot |y),\) then for any \(y_1<y_2\) and \(z\in S\), we have that \(Q_j(z|y_1)\ge Q_j(z|y_2).\)
References
Amir R (1989) A lattice–theoretic approach to a class of dynamic games. Comput Math Appl 17:1345–1349
Amir R (1996) Continuous stochastic games of capital accumulation with convex transitions. Games Econ Behav 15:111–131
Balbus Ł, Nowak AS (2008) Existence of perfect equilibria in a class of multigenerational stochastic games of capital accumulation. Automatica 44:1471–1479
Balbus Ł, Jaśkiewicz A, Nowak AS (2015b) Stochastic bequest games. Games Econ Behav 90:247–256
Berge C (1963) Topological spaces. Oliver and Boyd Ltd, Edinburgh
Bhattacharya R, Majumdar M (2007) Random dynamical systems: theory and applications. Cambridge University Press, Cambridge
Billingsley P (1968) Convergence of probability measures. Wiley, New York
Debreu G (1954) A social equilibrium existence theorem. Proc Natl Acad Sci USA 38:931–938
Dutta PK, Sundaram R (1992) Markovian equilibrium in a class of stochastic games: existence theorems for discounted and undiscounted models. Econ Theory 2:197–214
Haurie A, Krawczyk JB, Zaccour G (2012) Games and dynamic games. World Scientific, Singapore
Hernández-Lerma O, Lasserre JB (1999) Further topics on discrete-time Markov control processes. Springer, New York
Jaśkiewicz A, Nowak AS (2011) Stochastic games with unbounded payoffs: applications to robust control in economics. Dyn Games Appl 1:253–279
Jaśkiewicz A, Nowak AS (2015) Stochastic games of resource extraction. Automatica 54:310–316
Jaśkiewicz A, Nowak AS (2018) Non-zero-sum stochastic games. In: Başar T, Zaccour G (eds) Handbook of dynamic game theory. Birkhäuser, Basel
Levhari D, Mirman LJ (1980) The great fish war: an example using a dynamic Cournot–Nash solution. ACA Trans 11:322–344
Majumdar MK, Sundaram RK (1988) Symmetric stochastic games of resource extraction. The existence of non-randomized stationary equilibrium. In: Raghavan TES et al (eds) Stochastic games and related topics. Kluwer Academic Publishers, Dordrecht, pp 175–190
Milgrom P, Roberts J (1990) Rationalizability, learning, and equilibrium in games with strategic complementarities. Econometrica 58:1255–1277
Nash JF (1951) Non-cooperative games. Ann Math 54:286–295
Nowak AS (2006) On perfect equilibria in stochastic models of growth with intergenerational altruism. Econ Theory 28:73–83
Serfozo R (1982) Convergence of Lebesgue integrals with varying measures. Sankhya Indian J Stat (Ser A) 44:380–402
Stachurski J (2009) Economic dynamics: theory and computation. MIT Press, Cambridge
Sundaram RK (1989) Perfect equilibrium in a class of symmetric dynamic games. J Econ Theory 47:153–177 Corrigendum: J Econ Theory 49:385–187
Topkis D (1998) Supermodularity and complementarity. Princeton University Press, Princeton
Wessels J (1977) Markov programming by successive approximations with respect to weighted supremum norms. J Math Anal Appl 58:326–335
Acknowledgements
We thank anonymous referees for helpful comments. The authors acknowledge the financial support from the National Science Centre, Poland: Grant 2016/23/B/ST1/00425.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
About this article
Cite this article
Jaśkiewicz, A., Nowak, A.S. On symmetric stochastic games of resource extraction with weakly continuous transitions. TOP 26, 239–256 (2018). https://doi.org/10.1007/s11750-017-0465-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11750-017-0465-0
Keywords
- Stochastic game
- Resource extraction game
- Stationary Markov perfect equilibrium
- Weakly continuous transition probability