Indeterminacy in stochastic overlapping generations models: real effects in the long run

Abstract

Indeterminate equilibria are known to exist for overlapping generations models, though recent research has been limited to deterministic settings in which all equilibria converge to a steady state in the long run. This paper analyzes stochastic overlapping generations models with three-period-lived representative consumers and adopts a novel computational algorithm to numerically approximate the entire set of competitive equilibria. In a stochastic setting with incomplete markets, indeterminacy has real effects in the long run. Our numerical simulations reveal that indeterminacy is an order of magnitude more important than endowment shocks in explaining long-run consumption and asset price volatility.

1 Introduction

Stochastic versions of the overlapping generations model of Allais (1947) and Samuelson (1958) can be useful models for macroeconomic policy making, but not before the theoretical and quantitative implications of indeterminacy are properly accounted for. In deterministic versions of the model, indeterminacy may exist (Gale 1973; Balasko and Shell 1980; Geanakoplos and Polemarchakis 1984; Kehoe and Levine 1984; Kehoe et al. 1991), but it has no effects in the long run as all equilibria converge to one of the steady states (Kehoe and Levine 1990; Spear et al. 1990; Wang 1993; Feng 2013). This paper considers stochastic versions of the model and decomposes indeterminacy into two types: one characterized by the initial conditions and one characterized by incomplete financial markets. We introduce a numerical algorithm to compute the entire set of competitive equilibrium. We use simulations to approximate the volatility of consumption across cohorts and the volatility of asset prices. Our findings show that indeterminacy has long-run effects and is an order of magnitude more important than endowment shocks in explaining long-run consumption and asset price volatility.

In deterministic overlapping generations (OLG) models, a sufficient condition for a determinate equilibrium is the property of gross substitution in consumption (Gale 1973; Balasko and Shell 1980; Geanakoplos and Polemarchakis 1984; Kehoe and Levine 1984; Kehoe et al. 1991).¹ This property is implausible given the empirical findings of Mankiw et al. (1985). In the same setting, Spear et al. (1990) and Wang (1993) have conjectured that even if the equilibrium set is indeterminate, all equilibria in that set converge in the long run to one of the steady states. This conjecture has been numerically verified for a handful of canonical economies in Kehoe and Levine (1990) and Feng (2013), where the former log-linearized the equilibrium system of equations and the latter numerically approximated the entire equilibrium set.²

In stochastic overlapping generations (SOLG) models, the properties of existence and Pareto inefficiency of recursive equilibria have been analyzed in Citanna and Siconolfi (2010) and Henriksen and Spear (2012), respectively.³ However, very little is known about indeterminacy in SOLG models. Several papers have provided examples showing the existence of a continuum of recursive (stationary Markov) equilibria (Spear et al. 1990; Farmer and Woodford 1997; Benhabib and Farmer 1999), but conditions for either the existence or nonexistence of indeterminacy are unavailable. The present paper focuses on the indeterminacy of competitive equilibria and studies its impact on the aggregate economy, specifically on the consumption and asset price volatility.⁴

In deterministic OLG models, equilibrium indeterminacy arises whenever a continuum of endogenous initial period variables is consistent with equilibrium. Each equilibrium is indexed by the vector of initial period variables which are typically asset prices or portfolio choices, though this paper looks at shadow prices of investment. Feng (2013) characterizes the entire set of initial period variables consistent with equilibrium and demonstrates that each numerically approximated equilibrium converges to one of the steady states. Although the equilibrium set is indeterminate, there are no effects in the long run.

This paper considers a SOLG model and characterizes two types of indeterminacy: initial condition indeterminacy and incomplete markets indeterminacy. Initial condition indeterminacy is identical to the indeterminacy described in deterministic settings and is indexed by the endogenous variables in the initial period. Incomplete markets indeterminacy is a new type of indeterminacy that arises in stochastic settings. As the name suggests, incomplete markets is a necessary condition for incomplete markets indeterminacy. Long-run effects only occur in the presence of incomplete markets indeterminacy.

Consider a simple incomplete markets setting with a 3-period-lived representative consumer born every period, a risk-free bond (one asset), and two states of uncertainty each period. The three periods of a consumer’s life are denoted young, middle-aged, and old. The equilibrium variables consist of the vector of asset prices and portfolio choices. In any node, only two consumers participate in the bond market: the young and the middle-aged. By market clearing, we only consider the portfolio choice of the middle-aged. An equilibrium is characterized by Euler equations for the young and middle-aged (recall we already internalized the market clearing condition). We use the Euler equations for the middle-aged to determine the portfolio choice for the middle-aged, reducing the system to only Euler equations for the young and the asset prices.

In the initial period $t=0$, the initial middle-aged and the initial old are endowed with a wealth vector. If we fix these parameters and the initial period shock, the young agents affected by the initial conditions are only those born in periods $t=0$ and $t=1$. There exists one equilibrium equation for the young born in period $t=0$ and two for the young born in period $t=1$ (1 for each node in period $t=1$). Denoting $q\left( s_{0},\ldots ,s_{t}\right)$ as the asset price in period t for the history of shocks $\left( s_{0},\ldots ,s_{t}\right)$, the equilibrium equations are written in the form (where the subscript “y” denotes the Euler equation for the young):

$$\begin{aligned} ee_{y}\left( q\left( s_{0}\right) ,\left( q\left( s_{0},s_{1}\right) \right) _{s_{1}\in \{1,2\}}\right)&=0,\\ ee_{y}\left( q\left( s_{0},s_{1}\right) ,\left( q\left( s_{0},s_{1} ,s_{2}\right) \right) _{s_{2}\in \{1,2\}}\right)&=0\text { for }s_{1} \in \left\{ 1,2\right\} . \end{aligned}$$

These equations are in terms of seven asset price variables, one for each node in periods $t=0$ through $t=2$.

As we can see, the number of variables exceeds the number of equations. This captures the dimension of the initial condition indeterminacy. If we denote S as the number of states of uncertainty each period and J as the number of assets, there are $J(1+S)$ equations and $J(1+S+S^{2})$ asset price variables, implying $JS^{2}$ degrees of freedom. In the deterministic case, $S=J=1$, there exists one degree of freedom, which is consistent with what (Kehoe and Levine 1990) would find for a real asset economy.⁵

As stressed above, the presence of this form of indeterminacy in a stochastic setting is not novel, but merely an extension of the indeterminacy analyzed by Kehoe and Levine (1990) for the case of real assets. Magill and Quinzii (2003) analyze indeterminacy in stochastic OLG models with 2-period-lived representative consumers. In such a model, one needs to introduce an infinitely lived asset that pays no dividends (usually called fiat money) to facilitate intergenerational trade. Hence, this model only captures nominal indeterminacy. Kehoe and Levine (1990) consider a deterministic setting with 3-period-lived consumers. They show that indeterminacy can still exist without fiat money. We restrict ourselves to an analysis of real indeterminacy by only including real assets in a stochastic OLG model with 3-period-lived consumers.⁶ Including nominal assets in a stochastic setting with 3-period-lived consumers would lead to even greater indeterminacy than what we find in the present paper.

Continuing with our example, the progression to a new node in period $t\ge 2$ introduces three Euler equations for the newborn young, but two of these are period $t+1$ middle-aged Euler equations, which are used to determine the two middle-aged bond choices in period $t+1$. There is only one new equation, the Euler equation in period t for the newborn young, together with two new asset price variables, one for each of the two nodes that can arise in period $t+1$:

$$\begin{aligned} ee_{y}\left( q\left( s_{0},\ldots ,s_{t}\right) ,\left( q\left( s_{0},\ldots ,s_{t},s_{t+1}\right) \right) _{s_{t+1}\in \{1,2\}}\right) =0. \end{aligned}$$

With two new variables and only one new equation, there exists one degree of freedom. Using the previous notation, each new node introduces J new Euler equations and SJ new asset price variables, implying $J(S-1)$ degrees of freedom. In the deterministic case, $S=J=1$, there are zero degrees of freedom, consisting with the findings in Kehoe and Levine (1990) and Feng (2013). Under sequentially complete markets ($S=J$), there exist $S\left( S-1\right)$ degrees of freedom, which is consistent with the findings from Henriksen and Spear (2012) for the $S=J=2$ case.⁷

Our decomposition of indeterminacy into two types is stark as the indeterminacy that we refer to as incomplete markets indeterminacy can only arise in economies with uncertainty. In that sense, it is quite different than initial condition indeterminacy, which captures the entirety of the indeterminacy that is found in deterministic models. Furthermore, only the former can lead to long-run effects. Our objective in this paper is not only to characterize the conditions under which incomplete markets indeterminacy arises, but also to estimate the effects of such indeterminacy on simulated time paths of equilibrium variables.

Formally, incomplete markets indeterminacy is present when the dimension of the image of the equilibrium transition correspondence is strictly positive. With such an equilibrium correspondence, there exists a continuum of next-period endogenous state variables that are consistent with equilibrium. This form of indeterminacy is closely related to the concept of initial condition indeterminacy that occurs in the deterministic setting. If we consider the vector of variables in any period t, some of the variables are state variables chosen in period $t-1$ and others are determined by the policy correspondence (in terms of the state variables). When you analogize period t state variables to the initial conditions (period $t=0$) and the period t policy variables to the initial period variables (period $t=0$), then, by definition, indeterminacy in period t is only possible if it was possible in the initial period. We have defined initial condition indeterminacy to capture this latter effect, making it a necessary condition for incomplete markets indeterminacy.

Our main theoretical result shows that initial condition indeterminacy and incomplete markets are sufficient conditions for incomplete markets indeterminacy (and hence long-run effects).

The fact that incomplete markets indeterminacy exists is of limited importance unless it is combined with an estimation of the effects of this indeterminacy. We apply the numerical method developed by Feng (2013) to numerically approximate the entire set of competitive equilibrium. In economies with incomplete markets indeterminacy, by definition, there exists a continuum of continuation values consistent with equilibrium. In our numerical simulations, we adopt a consistent means to select continuation values from this continuum. We consider a variety of different selection rules, where we run each simulation using a consistent selection rule throughout. The choice of the selection rule has real effects, so we are thorough in considering a broad range of different selection rules.

In each simulation, we generate a simulated vector of equilibrium variables over time. We are particularly interested in two simulated moments: consumption volatility and asset price volatility. The consumption volatility is the standard deviation of consumption across cohorts (holding fixed the age of consumption). The asset price volatility is the standard deviation of asset prices across time. Our initial findings reveal that both of these simulated volatility measures are an order of magnitude larger than what is predicted from the endowment volatility alone. Further, when we compute the simulated consumption volatilities after conditioning on the shock realization, we find that the conditional consumption volatilities are on average more than 90 % as large as the unconditional volatilities.

Next, we numerically approximate the equilibrium set in a sunspot economy. The sunspot economy is identical to our original economy, except that the states of uncertainty are now states of extrinsic uncertainty, meaning that the endowments are independent of the shock realization. As before, we generate simulated vectors of equilibrium variables and compute the simulated consumption and asset price volatilities. For both variables, the simulated volatilities for the sunspot economy are on average more than 90 % as large as the simulated volatilities in the original economy with endowment risk.

Our interpretation is to attribute any volatility in equilibrium variables that cannot be explained by fundamentals to the effects of indeterminacy. Our numerical results suggest that indeterminacy is an order of magnitude more important in explaining consumption and asset price volatility than endowment risk.

The remainder of the paper is organized as follows. Section 2 introduces the model and defines the competitive equilibrium concept. Section 3 introduces an equivalent recursive formulation called Markov equilibrium, which is important for subsequent computation and simulation. Section 4 applies the computational algorithm and presents the simulation results. Section 5 concludes, and the Appendix contains the proofs of our main results and further details on the algorithm.

2 The economic model

In this section, we first introduce the economic environment and then provide the competitive equilibrium definition.

2.1 Economic environment

Time is discrete $t=0,1,2,\ldots$ At every date t, a new cohort of consumers enters the economy. Each cohort consists of a representative consumer that remains in the economy for 3 periods.

At every date t, the economy is hit by a shock s. The shock follows a Markov chain over a finite set $\mathbf {S}=\left\{ 1,\ldots ,S\right\}$ as described by the Markov transition matrix $\varPi$ with elements $\pi \left( s,\sigma \right)$ for all $s,\sigma \in \mathbf {S}$. The observed shock in period t is $s_{t}$. The initial shock $s_{0}$ is known to all consumers in the economy. The history of shocks up to and including period t is $s^{t}=(s_{0},s_{1},\ldots ,s_{t})$. The history of shocks uniquely characterizes the location of the economy in the space of time and uncertainty and is often called a date-event or node. We use the notation $(s^{t},\sigma )_{\sigma \in \mathbf {S}}$ to refer to the set of nodes that immediately succeed the node $s^{t}$ and the notation $(s^{t},\sigma ,\sigma ^{\prime })_{\sigma ,\sigma ^{\prime }\in \mathbf {S}^{2}}$ to refer to the set of nodes that follow 2 periods after the node $s^{t}$.

At each node, a single consumption good is traded.

The consumers are identified by the node at birth and the age $a\in \{0,1,2\}$ in the current node. The parameter $e_{a}(s^{t+a})$ is the endowment of a consumer of age a in node $s^{t+a}$. This means that the consumer was born in node $s^{t}$. A consumer’s individual endowments follow a Markov process governed by the stationary function $\mathbf {e}:\left\{ 0,1,2\right\} \times {\mathbf {S}\rightarrow \mathbb {R}}_{++}$, such that for all $a\in \{0,1,2\}$ and all nodes $s^{t}$, $e_{a}(s^{t+a})=\mathbf {e}_{a}(s_{t+a})$.

Similarly, the variable $c_{a}(s^{t+a})$ is the consumption of a consumer of age a in node $s^{t+a}$.

In the initial node $s_{0}$, there exists an age $a=0$ consumer, an age $a=1$ consumer, and an age $a=2$ consumer. The age $a=2$ consumer has the consumption $c_{2}\left( s_{0}\right)$ and utility function $u(c_{2}\left( s_{0}\right) )$. The age $a=1$ consumer has the consumption vector $\left( c_{1}\left( s_{0}\right) ,\left( c_{2}\left( s_{0},\sigma \right) \right) _{\sigma \in \mathbf {S}}\right)$ and utility function

$$\begin{aligned} u(c_{1}\left( s_{0}\right) )+\beta \sum \limits _{\sigma \in \mathbf {S}} \pi \left( s_{0},\sigma \right) u(c_{2}\left( s_{0},\sigma \right) ). \end{aligned}$$

For a consumer born in node $s^{t}$, define the lifetime contingent consumption vector as $c(s^{t})=\left( c_{0}(s^{t}),\left( c_{1} (s^{t},\sigma )\right) _{\sigma \in \mathbf {S}},\left( c_{2}(s^{t} ,\sigma ,\sigma ^{\prime })\right) _{\left( \sigma ,\sigma ^{\prime }\right) \in \mathbf {S}^{2}}\right) \in \mathbb {R}_{+}^{1+S+S^{2}}$. The consumer preferences are assumed to be identical and are represented by the time-separable utility function $U:\mathbb {R}_{+}^{1+S+S^{2}}\rightarrow \mathbb {R}\cup \{-\infty \}$ defined as

$$\begin{aligned} U(c(s^{t}))&=u(c_{0}(s^{t}))+\beta \sum \limits _{\sigma \in \mathbf {S}} \pi (s_{t},\sigma )u(c_{1}(s^{t},\sigma ))\\&\quad +\,\beta ^{2}\sum \limits _{\sigma ,\sigma ^{\prime }\in \mathbf {S}^{2}}\pi (s_{t},\sigma )\pi (\sigma ,\sigma ^{\prime })u(c_{2}(s^{t},\sigma ,\sigma ^{\prime })). \end{aligned}$$

The one-period utility u satisfies the following conditions:

Assumption 1

The one-period utility function $u:\mathbb {R}_{+} \rightarrow \mathbb {R}\cup \{-\infty \}$ is $C^{2}$, differentiably strictly increasing (i.e., $u_{c}\left( c\right) >0$ $\forall c>0$), differentiably strictly concave (i.e., $u_{cc}\left( c\right) <0$ $\forall c>0$), and satisfies the Inada condition (i.e., $\underset{c\rightarrow 0}{\lim } u_{c}\left( c\right) =+\infty$).

For each node $s^{t}$, there exist J short-lived numeraire assets with fixed payouts in terms of the consumption good. The J assets are indexed by a superscript $j\in \mathbf {J}=\left\{ 1,\ldots ,J\right\}$. The equilibrium price of asset j in node $s^{t}$ is denoted $q^{j}(s^{t})$. The prices for all assets traded in node $s^{t}$ are collected in the row vector $q(s^{t} )=\left( q^{j}(s^{t})\right) _{j\in \mathbf {J}}$.

The asset payouts follow a Markov chain such that the payouts in the nodes $\left( s^{t},\sigma \right) _{\sigma \in \mathbf {S}}$ for the asset j traded in node $s^{t}$ are given by the column vector $r^{j}=\left( r^{j} (\sigma )\right) _{\sigma \in \mathbf {S}}$. Additionally, define $r\left( \sigma \right) =\left( r^{j}(\sigma )\right) _{j\in \mathbf {J}}$ as the row vector of portfolio payouts for the current shock $\sigma$. The asset payouts can be collected into the $S\times J$ payout matrix

$$\begin{aligned} R=\left( r^{1},\ldots ,r^{J}\right) =\left( r\left( s\right) \right) _{s\in \mathbf {S}}. \end{aligned}$$

Assumption 2

The payout matrix is a nonnegative and full rank matrix.

Let $\theta _{a}^{j}(s^{t})$ denote the amount of asset j purchased by a consumer of age a in node $s^{t}$. The assets pay out in the following period, specifically in the nodes $\left( s^{t},\sigma \right) _{\sigma \in \mathbf {S}}$. The column vector $\theta _{a}(s^{t})=\left( \theta _{a} ^{j}(s^{t})\right) _{j\in \mathbf {J}}$ contains the entire portfolio of all assets positions of the consumer of age a in node $s^{t}$. The payout of the portfolio in node $\left( s^{t},\sigma \right)$ is $r\left( \sigma \right) \theta _{a}(s^{t})$.

In the initial node $s_{0}$, both the age $a=1$ consumer and the age $a=2$ consumer enter the period with a portfolio of assets from a previous (unmodeled) period. We can refer to this previous (unmodeled) period as period $t=-1$. The portfolios carried into the initial node $s_{0}$ are parameters of the model. The portfolio for the age $a=1$ consumer in node $s_{0}$ is denoted $\theta _{0}(-1)=\left( \theta _{0}^{j}(-1)\right) _{j\in \mathbf {J}}$ as an age $a=1$ consumer in node $s_{0}$ would have age $a=0$ in the previous (unmodeled) period $t=-1$. Likewise, the portfolio for the age $a=2$ consumer in node $s_{0}$ is denoted $\theta _{1}(-1)=\left( \theta _{1}^{j}(-1)\right) _{j\in \mathbf {J}}$ as an age $a=2$ consumer in node $s_{0}$ would have age $a=1$ in the previous (unmodeled) period $t=-1$.

Market clearing for assets traded in node $s^{t}$ is given by:

$$\begin{aligned} \sum \limits _{a=0}^{2}\theta _{a}^{j}(s^{t})=0 \quad \forall j\in \mathbf {J}. \end{aligned}$$

For any given $\theta _{1}(-1)$ and $q(s_{0})$, the household problem for the age $a=2$ consumer in the initial node $s_{0}$ is given by:

$$\begin{aligned} \begin{array}{ll} \underset{c_{2}\left( s_{0}\right) }{\max } &{} u\left( c_{2}\left( s_{0}\right) \right) \\ subj.\,to &{} c_{2}\left( s_{0}\right) +q(s_{0})\theta _{2}(s_{0} )\le \mathbf {e}_{2}(s_{0})+r\left( s_{0}\right) \theta _{1}(-1). \end{array} \end{aligned}$$

For any given $\theta _{0}(-1)$ and $\left( q\left( s_{0}\right) ,\left( q(s_{0},\sigma )\right) _{\sigma \in \mathbf {S}}\right)$, the household problem for the age $a=1$ consumer in the initial node $s_{0}$ is given by:

$$\begin{aligned} \begin{array}{ll} \underset{c_{1}\left( s_{0}\right) ,\theta _{1}(s_{0}),\left( c_{2}\left( s_{0},\sigma \right) \right) _{\sigma \in \mathbf {S}}}{\max } &{} u(c_{1}\left( s_{0}\right) )+\beta \sum \limits _{\sigma \in \mathbf {S}}\pi \left( s_{0} ,\sigma \right) u(c_{2}\left( s_{0},\sigma \right) )\\ subj.\,to &{} c_{1}\left( s_{0}\right) +q(s_{0})\theta _{1}(s_{0} )\le \mathbf {e}_{1}(s_{0})+r\left( s_{0}\right) \theta _{0}(-1)\\ &{} c_{2}\left( s_{0},\sigma \right) +q(s_{0},\sigma )\theta _{2}(s_{0} ,\sigma )\le \mathbf {e}_{2}(\sigma )+r\left( \sigma \right) \theta _{1} (s_{0})\quad \forall \sigma \in \mathbf {S}. \end{array} \end{aligned}$$

For simplicity, define $\theta (s^{t})=\left( \theta _{0}(s^{t}),\left( \theta _{1}(s^{t},\sigma )\right) _{\sigma \in \mathbf {S}}\right) \in \mathbb {R}^{J(1+S)}$ as the entire vector of lifetime contingent portfolios for a consumer born in node $s^{t}$. Given asset prices $\left( q(s^{t}),\left( q(s^{t},\sigma )\right) _{\sigma \in \mathbf {S}},\left( q(s^{t},\sigma ,\sigma ^{\prime })\right) _{\sigma ,\sigma ^{\prime }\in \mathbf {S} }\right)$, the household problem for a consumer born in node $s^{t}$ is given by:

(1)

2.2 Equilibrium

We define a sequential competitive equilibrium (SCE) as follows.

Definition 1

A SCE is a collection of prices and choices of consumers $\big \{ q(s^{t}),\theta (s^{t}),c(s^{t})\big \}$ such that: (i) For each $s^{t}$, taking as given the prices $\big ( q(s^{t}),\left( q(s^{t},\sigma )\right) _{\sigma \in \mathbf {S}},\big ( q(s^{t},\sigma ,\sigma ^{\prime })\big ) _{\sigma ,\sigma ^{\prime }\in \mathbf {S}}\big )$, a consumer born in $s^{t}$ solves (1). (ii) Commodity market clearing for each $s^{t}$:

$$\begin{aligned} \sum \limits _{a=0}^{2}c_{a}(s^{t})=\sum \limits _{a=0}^{2}\mathbf {e} _{a}(s_{t}). \end{aligned}$$

(2)

(iii) Asset market clearing for each $s^{t}$:

$$\begin{aligned} \sum \limits _{a=0}^{2}\theta _{a}(s^{t})=0. \end{aligned}$$

(3)

The existence of a SCE can be verified using standard methods (Balasko and Shell 1980; Schmachtenberg 1988). Moreover, Balasko and Shell (1980) and Schmachtenberg (1988) prove that every sequence of equilibrium asset prices $\left\{ q\left( s^{t}\right) \right\}$ is bounded.

Under Assumption 1, the equilibrium asset holdings $\theta _{2}(s^{t})=0$ in all date-events. In all date-events, the old-age consumers will not carry asset holdings into the future. In the (unmodeled) period $t=-1$, young-age and middle-age consumers receive the portfolios that they carry into the initial period $t=0$. The young-age consumers in period $t=-1$ are middle-age consumers in period $t=0$, and the middle-age consumers in period $t=-1$ are old-age consumers in period $t=0$. Market clearing must hold for the (unmodeled) period $t=-1$, meaning that the parameters $\theta _{0}(-1)$ and $\theta _{1}(-1)$ must satisfy:

$$\begin{aligned} \theta _{0}^{j}(-1)+\theta _{1}^{j}(-1)=0\quad \forall j\in \mathbf {J}. \end{aligned}$$

3 Markov equilibrium and indeterminacy

In this paper, we characterize the entire set of recursive (Markov) equilibrium in SOLG by adopting the methodology of Feng (2013). We then identify the existence of indeterminacy by examining the set of recursive equilibrium. In the next section, we also study the impact of indeterminacy on long-run economy by simulating the models.

3.1 Markov equilibrium

First, we will economize on notation. Recall that market clearing in any node $s^{t}$ is such that $\theta _{0}(s^{t})=-\theta _{1}(s^{t})$. Define the portfolio payout for the age $a=2$ consumer in node $s^{t}$ as $\omega \left( s^{t}\right) =r\left( s_{t}\right) \theta _{1}(s^{t-1})\in \mathbb {R}$. This implies that the portfolio payout for the consumer $a=1$ consumer in node $s^{t}$ is $-\omega \left( s^{t}\right)$. Using these facts, we can rewrite the budget constraints faced by all consumers alive in node $s^{t}$:

$$\begin{aligned}&c_{0}(s^{t})-q(s^{t})\theta _{1}(s^{t})\le \mathbf {e}_{0}(s_{t} ), \end{aligned}$$

(4)

$$\begin{aligned}&c_{1}(s^{t})+q(s^{t})\theta _{1}(s^{t})\le \mathbf {e}_{1}(s_{t} )-\omega \left( s^{t}\right) ,\end{aligned}$$

(5)

$$\begin{aligned}&c_{2}(s^{t})\le \mathbf {e}_{2}(s_{t})+\omega \left( s^{t}\right) . \end{aligned}$$

(6)

One can define the recursive equilibrium on the natural state space consisting of the current shock $s_{t}$, and the portfolio payout $\omega \left( s^{t}\right)$. However, as shown in Kubler and Polemarchakis (2004), such equilibrium may not exist. To restore the recursive formulation of SCE, we enlarge the state space to include the shadow values of investment as an additional state variable. Using the framework of Duffie et al. (1994), Kubler and Schmedders (2003), and Feng et al. (2014), we develop an iterative procedure to characterize the recursive equilibrium on this enlarged state space.

In line with Feng et al. (2014), the state variables we consider include the current shock $s_{t}$, the portfolio payout $\omega \left( s^{t}\right)$, and the shadow values of investment $m\left( s^{t}\right) =\left( m^{j}\left( s^{t}\right) \right) _{j\in \mathbf {J}}$ for the age $a=1$ consumer in the current node:

$$\begin{aligned} m^{j}\left( s^{t}\right) =q^{j}(s^{t})u_{c}(c_{1}(s^{t})). \end{aligned}$$

(7)

Denote the state space as ${\mathbf {S}\times \mathbb {R}\times \mathbb {R}}_{+}^{J}$ with typical element $\left( s,\omega ,m\right) \in {\mathbf {S}\times \mathbb {R}\times \mathbb {R}}_{+}^{J}$. The policy function is defined as $\mathbf {f}:{\mathbf {S}\times \mathbb {R}\times \mathbb {R}}_{+}^{J}\rightarrow {\mathbb {R}}_{+}^{J}\times {\mathbb {R}}^{J}$ such that $\left( q,\theta _{1}\right) =\mathbf {f}\left( s,\omega ,m\right)$ satisfies the following equations:

$$\begin{aligned} m^{j}&=q^{j}u_{c}\left[ \mathbf {e}_{1}(s)-q\theta _{1}-\omega \right] \quad \forall j\in \mathbf {J}, \end{aligned}$$

(8)

$$\begin{aligned} m^{j}&=\beta \sum _{\sigma \in \mathbf {S}}\pi (s,\sigma )u_{c}\left[ \mathbf {e}_{2}(\sigma )+r\left( \sigma \right) \theta _{1}\right] r^{j}\left( \sigma \right) \quad \forall j\in \mathbf {J,} \end{aligned}$$

(9)

where Eq. (8) is the definition of the shadow value of investment and (9) represents the Euler equation for the consumer of age $a=1$.⁸

The expectations correspondence $\mathbf {g}:{\mathbf {S}\times \mathbb {R} \times \mathbb {R}}_{+}^{J}\rightrightarrows \left( {\mathbb {R} \times \mathbb {R}}_{+}^{J}\right) ^{S}$ is a mapping from the current-period state variables $\left( s,\omega ,m\right)$ to the next-period state variables $\left( \omega ^{\prime }(\sigma ),m^{\prime }\left( \sigma \right) \right) _{\sigma \in \mathbf {S}}$, where $\left( \omega ^{\prime } (\sigma ),m^{\prime }\left( \sigma \right) \right) \in {\mathbb {R} \times \mathbb {R}}_{+}^{J}$ $\forall \sigma \in \mathbf {S}$. By definition,

$$\begin{aligned} \left( \omega ^{\prime }(\sigma ),m^{\prime }\left( \sigma \right) \right) _{\sigma \in \mathbf {S}}\in \mathbf {g}\left( s,\omega ,m\right) \end{aligned}$$

iff for $\left( q,\theta _{1}\right) =\mathbf {f}\left( s,\omega ,m\right)$ and $\left( q^{\prime }\left( \sigma \right) ,\theta _{1}^{\prime }\left( \sigma \right) \right) =\mathbf {f}\left( \sigma ,\omega ^{\prime } (\sigma ),m^{\prime }\left( \sigma \right) \right)$ $\forall \sigma \in \mathbf {S}$ the following conditions are satisfied:

$$\begin{aligned} \omega ^{\prime }(\sigma )&=r\left( \sigma \right) \theta _{1}\quad \forall \sigma \in \mathbf {S}, \end{aligned}$$

(10)

$$\begin{aligned} q^{j}u_{c}\left[ \mathbf {e}_{0}(s)+q\theta _{1}\right]&=\beta \sum \limits _{\sigma \in \mathbf {S}}\pi (s,\sigma )\frac{m^{\prime j}\left( \sigma \right) }{q^{\prime j}\left( \sigma \right) }r^{j}\left( \sigma \right) \quad \forall j\in \mathbf {J,} \end{aligned}$$

(11)

where Eq. (10) is the definition of the portfolio payout and (11) represents the Euler equation for the consumer of age $a=0$.

Definition 2

Markov equilibrium is defined by the policy correspondence $\mathbf {V}^{*}:{\mathbf {S}\times \mathbb {R}}\rightrightarrows {\mathbb {R}}_{+}^{J}$ and the transition correspondence $\mathbf {F}:graph(\mathbf {V}^{*})\rightrightarrows \left( {\mathbb {R}\times \mathbb {R}}_{+}^{J}\right) ^{S}$ satisfying the following two properties:

• 1. For all $\left( s,\omega ,m\right) \in graph(\mathbf {V}^{*})$, $\mathbf {F}\left( s,\omega ,m\right) \subseteq \mathbf {g}\left( s,\omega ,m\right)$.

• 2. For all $\left( s,\omega ,m\right) \in graph(\mathbf {V}^{*})$ and all $\sigma \in \mathbf {S},\left( \sigma ,\mathbf {F}_{\sigma }\left( s,\omega ,m\right) \right) \subseteq graph(\mathbf {V}^{*})$, where $\mathbf {F}_{\sigma }:graph(\mathbf {V}^{*})\rightrightarrows {\mathbb {R}\times \mathbb {R}}_{+}^{J}$ is the projection onto the shock $\sigma$ state variables.

We refer to $\mathbf {V}^{*}$ as the Markov equilibrium policy correspondence and $\mathbf {F}$ as the Markov equilibrium transition correspondence.

Theorem 1

A Markov equilibrium is a SCE.

Proof

See Sect. 1. $\square$

Theorem 2

A Markov equilibrium exists.

Proof

See Sect. 1. $\square$

3.2 Indeterminacy

Given the Markov equilibrium policy correspondence $\mathbf {V}^{*}$ and transition correspondence $\mathbf {F}$, the SCE for a given vector of initial conditions can be determined. The initial conditions are $s_{0}\in \mathbf {S,}$ $\theta _{1}(-1)\in {{\mathbb {R}}}^{J}$, and $m\left( s_{0}\right) \in {{\mathbb {R}}}_{+}^{J}$. While both $s_{0}$ and $\theta _{1}(-1)$ are parameters of the model, the shadow prices $m\left( s_{0}\right) \in \mathbf {V}^{*}\left( s_{0},r\left( s_{0}\right) \theta _{1}(-1)\right)$ are endogenous state variables.

• In period $t=0$, given $\omega \left( s_{0}\right) =r\left( s_{0}\right) \theta _{1}(-1)$, the vector

  $$\begin{aligned} \left( q\left( s_{0}\right) ,\theta _{1}\left( s_{0}\right) \right) =\mathbf {f}\left( s_{0},\omega \left( s_{0}\right) ,m\left( s_{0}\right) \right) \end{aligned}$$

  is determined as the unique solution to Eqs. (8) and (9). The variables $\left( \omega (s_{0},\sigma ),m\left( s_{0},\sigma \right) \right) _{\sigma \in \mathbf {S}}\in \mathbf {F}\left( s_{0},\omega \left( s_{0}\right) ,m\left( s_{0}\right) \right)$ must be consistent with the transition correspondence.

• In period $t>0$, given $\left( s_{t},\omega \left( s^{t}\right) ,m\left( s^{t}\right) \right)$, the vector

  $$\begin{aligned} \left( q\left( s^{t}\right) ,\theta _{1}\left( s^{t}\right) \right) =\mathbf {f}\left( s_{t},\omega \left( s^{t}\right) ,m\left( s^{t}\right) \right) \end{aligned}$$

  is determined as the unique solution to Eqs. (8) and (9). The variables $\left( \omega (s^{t},\sigma ),m\left( s^{t},\sigma \right) \right) _{\sigma \in \mathbf {S}}\in \mathbf {F}\left( s_{t},\omega \left( s^{t}\right) ,m\left( s^{t}\right) \right)$ must be consistent with the transition correspondence.

From the above discussion, we find that there are two types of indeterminacy: (i) initial condition indeterminacy and (ii) incomplete markets indeterminacy.

Definition 3

Initial condition indeterminacy occurs if $\dim \left( \mathbf {V}^{*}\left( s,\omega \right) \right) >0$ for some $\left( s,\omega \right) \in {\mathbf {S}\times \mathbb {R}}$.

Initial condition indeterminacy is indexed by $m\left( s_{0}\right) \in \mathbf {V}^{*}\left( s_{0},\omega \left( s_{0}\right) \right)$, meaning that if the image of $\mathbf {V}^{*}$ is determinate, initial condition indeterminacy does not arise.

Definition 4

Incomplete markets indeterminacy occurs if $\dim \left( \mathbf {F}\left( s,\omega ,m\right) \right) >0$ for some $\left( s,\omega ,m\right) \in {\mathbf {S}\times \mathbb {R\times R}}_{+}^{J}$.

Theorem 3

Initial condition indeterminacy is a necessary condition for incomplete markets indeterminacy.

Proof

This follows by definition. $\square$

The relation between the two types of indeterminacy depends upon properties of the asset structure.

Definition 5

Markets are sequentially complete if $J=S$ and sequentially incomplete if $J<S$.

As seen in Demange (2002) and Henriksen and Spear (2012), sequential completeness does not suffice for (interim) Pareto efficiency. Recall from Demange (2002) that a feasible allocation is $\left\{ c(s^{t})\right\} \ge 0$ such that the resource constraints (2) are satisfied in all date-events. An allocation $\left\{ c(s^{t})\right\}$ is (interim) Pareto efficient if there does not exist another feasible allocation $\left\{ \tilde{c}(s^{t})\right\}$ such that for all possible histories $s^{t}$ and all periods t:

$$\begin{aligned} U(\tilde{c}(s^{t}))\ge U(c(s^{t})) \end{aligned}$$

with strict inequality for at least one history $s^{t}$.⁹

Definition 6

The asset markets are complete if for each initial shock $s_{0}$, there exists initial wealth $\omega \left( s_{0}\right)$ such that the resulting SCE allocation is (interim) Pareto efficient.

As seen in Henriksen and Spear (2012), complete markets is a stronger condition than sequentially complete markets. While sequentially complete markets is not necessary for incomplete markets indeterminacy, we find that the stronger condition of complete markets is necessary.

Theorem 4

Incomplete markets is a necessary condition for both initial condition indeterminacy and incomplete markets indeterminacy.

Proof

See Sect. 1. $\square$

Our main theoretical result provides a partial converse, namely that incomplete markets and a stronger notion of initial condition indeterminacy are sufficient for incomplete markets indeterminacy.

Definition 7

Strong initial condition indeterminacy occurs if $\dim \left( \mathbf {V} ^{*}\left( s,\omega \right) \right) >0$ for all equilibrium $\left( s,\omega \right) \in {\mathbf {S}\times \mathbb {R}}$.

Theorem 5

An economy with strong initial condition indeterminacy and incomplete markets will also have incomplete markets indeterminacy.

Proof

See Sect. 1. $\square$

4 Computation and simulation

Knowing that an economy exhibits incomplete markets indeterminacy is of limited use for welfare analysis. In this section, we consider a stochastic economy with incomplete markets and conduct numerical analysis to approximate the long-run effects of incomplete markets indeterminacy. To approximate the long-run effects, we use the methodological contribution in Feng (2013) and Feng et al. (2014) to compute a numerical approximation of the Markov equilibrium correspondences.

4.1 Numerical specifications

We consider an economy with one asset ($J=1$) and two states of uncertainty ($S=2$). There is an exogenous shock that affects the endowments of the household. Given the shock realization, the endowment of the age $a=2$ consumer changes, while the other endowments remain unchanged. Specifically, we assume that

$$\begin{aligned} \begin{array}{ll} \mathbf {e}_{0}(s)=3, &{} \forall s\in \{1,2\}\\ \mathbf {e}_{1}(s)=12, &{} \forall s\in \{1,2\}\\ \mathbf {e}_{2}(1)=1+\epsilon , &{} \epsilon =0.05\\ \mathbf {e}_{2}(2)=1-\epsilon . &{} \end{array} \end{aligned}$$

The transition matrix that governs the Markov chain is given by

$$\begin{aligned} \varPi =\left[ \begin{array}{cc} 0.95 &{} 0.05\\ 0.05 &{} 0.95 \end{array} \right] \text {.} \end{aligned}$$

The utility function is given by: $u(c)=\frac{c^{1-b}-1}{1-b}$, where the coefficient of relative risk aversion is $b=4$. We set the discount factor $\beta =0.5$.

We borrow these parameter values from Kehoe and Levine (1990), which provides greater details on the justification of the parameter values chosen for the economy. To summarize their justification, let one period represent 20 years, meaning that the discount factor of $\beta =0.5$ corresponds to an annual discount factor of $0.966=0.5^{\frac{1}{20}}$. The risk-aversion parameter b implies an intertemporal elasticity of substitution of 0.25. This is similar to the value chosen by Auerbach and Kotlikoff (1987). The life-cycle earnings profile of the household is hump-shaped as in Gourinchas and Parker (2002).

The asset is a real bond with payouts equal to 1 for both states $s\in \mathbf {S}$. The initial conditions of the economy are the initial period shock $s_{0}$, the initial period bond payout $\omega \left( s_{0}\right) =\theta _{1}(-1)$, and the initial shadow value of investment $m\left( s_{0}\right)$.

4.2 Convergence results

We apply the numerical algorithm detailed in Feng (2013) to approximate the Markov equilibrium policy correspondence $\mathbf {V^{*}}$. The numerical approximation will be termed the Markov policy correspondence $\mathbf {V} :{\mathbf {S}\times \Theta }\rightrightarrows {\mathbb {R}}_{+}$, where $\Theta \subseteq {\mathbb {R}}$ is the compact set of portfolio payouts for consumers of age $a=1$. Such a set is known to exist since the set of SCE variables is contained in a compact set.

We iterate the algorithm until the Euler equation residuals are bounded above by some small error bound $\epsilon >0$. Specifically, the Markov policy correspondence $\mathbf {V}:{\mathbf {S}\times \Theta }\rightrightarrows {\mathbb {R}}_{+}$ is defined such that for any $m\in \mathbf {V} (s,\omega )$, there exists a vector of variables $\left( q,\theta _{1},\left( \omega ^{\prime }(\sigma ),m^{\prime }\left( \sigma \right) \right) _{\sigma \in \mathbf {S}}\right)$ satisfying (8) and (10) with the Euler equation residuals from (9) and (11) bounded above by $\epsilon$. For the numerical examples we consider, we are able to compute the Markov policy correspondence $\mathbf {V}$ for any arbitrarily small value $\epsilon >0$. Citing Theorems 2 and 3 from Feng et al. (2014), the operator $\mathbf {B}$ is such that the fixed point $\mathbf {V}$ of our numerical approximation converges uniformly to the Markov equilibrium policy correspondence $\mathbf {V^{*}}$ as a function of the discrete partition of the state space. Further details about the discrete version of the operator $\mathbf {B}$ and the numerical algorithm are contained in the “Appendix.”

4.3 Discussion of incomplete markets indeterminacy

For this numerical verification, we first compute the Markov policy correspondence $\mathbf {V}$, which is the numerical approximation to the Markov equilibrium policy correspondence $\mathbf {V}^{*}$. Given the Markov policy correspondence $\mathbf {V,}$ the Markov transition correspondence $\mathbf {F}:graph(\mathbf {V})\rightrightarrows \left( {\mathbb {R} \times \mathbb {R}}_{+}^{J}\right) ^{S}$ is approximated such that Eqs. (10) and (11) are satisfied (the latter with residuals bounded above by $\epsilon$).

Figure 1 in the Appendix contains the graph of the Markov transition correspondence $\mathbf {F}$. Specifically, it contains the variables $\left( s,\omega ,m,\left( m^{\prime }\left( \sigma \right) \right) _{\sigma \in \mathbf {S}}\right)$ (for both possible shocks $s\in \{1,2\}$, using the fact that $\omega$ is independent of the shock realization s) such that

$$\begin{aligned} \left( \omega ^{\prime }\left( \sigma \right) ,m^{\prime }\left( \sigma \right) \right) _{\sigma \in \mathbf {S}}\in \mathbf {F}\left( s,\omega ,m\right) , \end{aligned}$$

where $\omega ^{\prime }\left( \sigma \right) =\mathbf {f}_{\theta }\left( s,\omega ,m\right)$. This numerical approximation includes Euler equation residuals bounded above by $\epsilon >0$ for any arbitrarily small $\epsilon$. If the observed incomplete markets indeterminacy is simply a result of numerical error, then we should observe that the graphs in Fig. 1 are affected by changes in the errors bound $\epsilon$ and the mesh size of the discretization. Our numerical experiments show that the graphs in Fig. 1 do not change once we reach a certain level of precision, namely an error bound $\epsilon =10^{-10}$ and mesh size equal to $10^{-6}$. Applying Proposition 2 from Feng (2013), we are able to numerically confirm that the economy exhibits initial condition indeterminacy.

Consider the right panel of Fig. 1 in the Appendix, which displays the cross section of the image of the Markov transition correspondence for both possible shocks $s\in \{1,2\}$. For both shocks $s\in \{1,2\}$, the possible values for the next-period shadow values of investment $\left( m^{\prime }\left( \sigma \right) \right) _{\sigma \in \{1,2\}}$ belong to a continuum. The dimension of the image of the Markov transition correspondence equals 1. This finding is consistent with Theorem 5 which implies that the economy has incomplete markets indeterminacy.

4.4 Simulation

Once we solve for the Markov equilibrium, we can generate (simulate) a sequential competitive equilibrium with the following procedure. First, we pick initial condition $\left( s_{0},\omega \left( s_{0}\right) ,m\left( s_{0}\right) \right) \in graph\left( \mathbf {V}^{*}\right)$. We solve for $\left( q\left( s_{0}\right) ,\theta _{1}\left( s_{0}\right) \right) =\mathbf {f}\left( s_{0},\omega \left( s_{0}\right) ,m\left( s_{0}\right) \right)$. Second, we use a random number generator to determine the value of $s_{1}$.¹⁰ By definition, $\omega \left( s_{0},s_{1}\right) =r(s_{1})\theta _{1}\left( s_{0}\right)$. Third, we pick $m\left( s_{0},s_{1}\right)$ such that (i) we satisfy our selection rule (described below) and (ii) $\left( \omega \left( s_{0},\sigma \right) ,m\left( s_{0},\sigma \right) \right) _{\sigma \in \mathbf {S}}\in \mathbf {F}\left( s_{0},\omega \left( s_{0}\right) ,m\left( s_{0}\right) \right)$. The final step is the iterative step in which $\big ( s_{1},\omega \left( s_{0},s_{1}\right) ,m\left( s_{0},s_{1}\right) \big )$ allows us to solve for $\left( q\left( s_{0},s_{1}\right) ,\theta _{1}\left( s_{0} ,s_{1}\right) \right) =\mathbf {f}\left( s_{1},\omega \left( s_{0} ,s_{1}\right) ,m\left( s_{0},s_{1}\right) \right)$. We continue this process for 5000 periods.

There is a continuum of choices of initial conditions in the image of $\mathbf {V}^{*}\left( s_{0},\omega \left( s_{0}\right) \right)$, and this represents the initial condition indeterminacy. Holding fixed $\left( \omega \left( s^{t}\right) ,m\left( s^{t}\right) \right)$, the right panel of Fig. 1 graphs the projection of the transition correspondence $\mathbf {F}\left( s_{t},\omega \left( s^{t}\right) ,m\left( s^{t}\right) \right)$ onto $\left( m\left( s^{t},\sigma \right) \right) _{\sigma \in \mathbf {S}}$ as a function of $s_{t}$. The dashed line refers to $s_{t}=1$ and the solid one to $s_{t}=2$. From Fig. 1, there is a continuum of $\left( m\left( s^{t},\sigma \right) \right) _{\sigma \in \mathbf {S}}$ consistent with the equilibrium, and this represents the incomplete markets indeterminacy.

Each of the selection rules specifies a certain property that the continuation variables $\left( m\left( s^{t},\sigma \right) \right) _{\sigma \in \{1,2\}}$ must satisfy, and these properties are held constant for the entire length of that simulation.¹¹ We consider eight different selection rules:

• 1. Maximize difference in asset prices

  Given the current-period state variables $\left( s,\omega ,m\right)$, the selection rule chooses $\left( m^{\prime }\left( \sigma \right) \right) _{\sigma \in \mathbf {S}}$ such that (i) $\left( \omega ^{\prime }(\sigma ),m^{\prime }\left( \sigma \right) \right) _{\sigma \in \mathbf {S}} \in \mathbf {F}\left( s,\omega ,m\right)$ and (ii) the difference $\left| q^{\prime }\left( \hat{\sigma }\right) -q\right|$ is maximized for the realized shock $\hat{\sigma }$ (as determined by the random number generator), where $(q,\theta _{1})=\mathbf {f}\left( s,\omega ,m\right)$, $\omega ^{\prime }(\hat{\sigma })=r(\hat{\sigma })\theta _{1}$, and $\left( q^{\prime }\left( \hat{\sigma }\right) ,\theta _{1}^{\prime }\left( \hat{\sigma }\right) \right) =\mathbf {f}\left( \hat{\sigma },\omega ^{\prime }(\hat{\sigma }),m^{\prime }\left( \hat{\sigma }\right) \right)$.

• 2. Minimize difference in asset prices.

• 3. Maximize difference in bond holdings, i.e., the difference $\left| \theta _{1}^{\prime }\left( \hat{\sigma }\right) -\theta _{1}\right|$ is maximized.

• 4. Minimize difference in bond holdings.

• 5. Maximize difference in young consumption, i.e., the difference

  $$\begin{aligned} \left| \mathbf {e}_{0}(s)+q\theta _{1}-\left( \mathbf {e}_{0}(\hat{\sigma })+q^{\prime }\left( \sigma \right) \theta _{1}^{\prime }\left( \hat{\sigma }\right) \right) \right| \end{aligned}$$

  is maximized.

• 6. Minimize difference in young consumption.

• 7. Maximize difference in middle-age consumption, i.e., the difference

  $$\begin{aligned} \left| \mathbf {e}_{1}(s)-q\theta _{1}-\omega -\left( \mathbf {e}_{1} (\hat{\sigma })-q^{\prime }\left( \hat{\sigma }\right) \theta _{1}^{\prime }\left( \hat{\sigma }\right) -\omega ^{\prime }(\hat{\sigma })\right) \right| \end{aligned}$$

  is maximized.

• 8. Minimize difference in middle-age consumption.

Notice that we do not consider selections with respect to the old-age consumption variable. Old-age consumption is $\mathbf {e}_{2}(\sigma )+\theta _{1}$. Indeterminacy does not play a role in this value, as the only element that depends upon $\sigma$ is the endowment parameter $\mathbf {e} _{2}(\sigma )$.

4.5 Simulation results

We choose the initial conditions so that $\theta _{1}(-1)=3.0$ and the initial shock is $s_{0}=2$, meaning $e_{2}\left( s_{0}\right) =1-\epsilon$. The initial shadow value of investment is $m_{0}=5.50$, where $m_{0}\in \mathbf {V}\left( s_{0},\theta _{1}(-1)\right)$.¹² Simulations last for 5000 periods, where the first 1000 periods are ignored when computing simulated moments and simulated conditional moments.

4.5.1 Effects of the selection rules

We run simulations under each of the eight selection rules introduced previously. The unconditional moments are reported in Tables 1 and 2 below. The first observation from the data is that the choice of selection rule matters and has real effects. Among all eight selection rules, the young consumption mean is smallest ($mean\left( c_{0}\right) =5.862$) and the middle consumption mean is largest ($mean\left( c_{1}\right) =5.308$) for the selection rule that maximizes the difference in bond holdings. Diametrically, among all 8 selection rules, the young consumption mean is largest ($mean\left( c_{0}\right) =6.037$) and the middle consumption mean is smallest ($mean\left( c_{1}\right) =5.293$) for the selection rule that minimizes the difference in bond holdings. The means for the young consumption can differ by as much as 3 %, and the means for middle consumption can differ by as much as 0.3 %.

Comparing the simulation in which the asset price difference is maximized and the simulation in which the asset price difference is minimized, the average lifetime utility for the consumers is 3.96 % higher under the latter, which corresponds to a consumption equivalent gain of 1.3 %.

Table 1 Simulated means

Table 2 Simulated standard deviations

4.5.2 Consumption volatility

Table 3 reports the simulated standard deviations conditional on either shock $s=1$ or shock $s=2$ being realized.

To assess the volatility of the young-age consumption (for age $a=0$ consumers), consider the six simulations that did not include young-age consumption $c_{0}(\hat{\sigma })=\mathbf {e}_{0}(\hat{\sigma })+q^{\prime }\left( \hat{\sigma }\right) \theta _{1}^{\prime }\left( \hat{\sigma }\right)$ in the objective function. Computing the averages across these six simulations and both potential shocks $s\in \left\{ 1,2\right\}$, the conditional standard deviations for $c_{0}$ are 92 % as large as their respective unconditional standard deviations.

To assess the volatility of the middle-age consumption (for age $a=1$ consumers), consider the six simulations that did not include middle-age consumption $c_{1}(\hat{\sigma })=\mathbf {e}_{1}(\hat{\sigma })-q^{\prime }\left( \hat{\sigma }\right) \theta _{1}^{\prime }\left( \hat{\sigma }\right) -\omega ^{\prime }(\hat{\sigma })$ in the objective function. Computing the averages across these 6 simulations and both potential shocks $s\in \left\{ 1,2\right\}$, the conditional standard deviations for $c_{1}$ are 94 % as large as their respective unconditional standard deviations.

Table 3 Simulated standard deviations (conditional)

The simulation results (Tables 1, 2, 3) reveal three facts: (i) the unconditional standard deviations for consumption volatility are an order of magnitude larger than the endowment standard deviation, (ii) the conditional standard deviations are strictly positive, and (iii) the conditional standard deviations are on average more than 90 % as large as the unconditional standard deviations.

These findings suggest that initial condition indeterminacy is present and that endowment volatility is not of first-order importance for explaining consumption volatility.

4.5.3 Robustness check on initial conditions

We also analyze the effects of the initial conditions on the behavior of the economy. For each of the following experiments, we remain consistent by applying the same selection rule (chosen from one of the eight possibilities previously introduced) for both the benchmark economy and economies with different initial conditions. Recall that the benchmark economy specifies $\{\theta _{1}(-1),s_{0},m_{0}\}=\{3.0,2,5.50\}$. The first experiment specifies $m_{0}=5.10$, the second specifies $s_{0}=1$ such that $\mathbf {e}_{2}\left( s_{0}\right) =1+\epsilon$, while the third specifies $\theta _{1}(-1)=4.3128$. After we drop the first 1000 periods, the simulated moments and simulated conditional moments are identical to those for the benchmark economy.

4.6 Sunspot equilibria

To decompose the effects of incomplete markets indeterminacy and endowment volatility on consumption and asset price volatility, we construct a sunspot equilibrium based on our benchmark economy. We maintain the same Markov chain, but the shocks are now states of extrinsic uncertainty, meaning that the endowments remain unchanged. The endowment process is given by

$$\begin{aligned} \begin{array}{ll} \mathbf {e}_{0}(s)=3, &{} \forall s\in \{1,2\}\\ \mathbf {e}_{1}(s)=12, &{} \forall s\in \{1,2\}\\ \mathbf {e}_{2}(1)=1+\epsilon , &{} \epsilon =0\\ \mathbf {e}_{2}(2)=1-\epsilon . &{} \end{array} \end{aligned}$$

There still remain $S=2$ states of uncertainty, and consumers need not have the same price expectations for both states. If the price expectations differ, then any consumption volatility is owing only to the incomplete markets indeterminacy, since the fundamentals of the economy remain unchanged.

For each of the eight consistent selection rules, we run 5000 simulations as before (where each simulation lasts for 5000 periods and the first 1000 periods are ignored when computing simulated moments).

The results are presented in Tables 4 and 5.

Table 4 Simulated standard deviations (sunspots)

Table 5 Simulated standard deviations (conditional, sunspots)

Broken down by variable, the following subsections show that the volatility for any of the variables (consumption, asset price, asset choice) is driven by the effects of incomplete markets indeterminacy and not by the endowment shocks.

4.6.1 Consumption volatility

To assess the volatility of young-age consumption, consider the six simulations that did not include young-age consumption in the objective function. Averaged across these six simulations, the standard deviations for young-age consumption in the sunspot model (no endowment risk) are 97 % as large as the standard deviations in the original model with endowment risk. Similar patterns hold for middle-age consumption. If the sunspot model accounts for 97 % of the consumption volatility, then the volatility is decomposed as 3 % due to endowment shocks and 97 % due to indeterminacy.

4.6.2 Asset price volatility

To assess the volatility of the asset prices, consider the six simulations that did not include the asset price $q^{\prime }\left( \hat{\sigma }\right)$ in the objective function. Averaged across these six simulations, the unconditional standard deviations for q in the sunspot model (no endowment risk) are 93 % as large as their respective unconditional standard deviations with endowment risk. The range for this ratio across all six simulations runs from 87 to 97 %.

4.6.3 Asset size volatility

In the sunspot model, since $c_{2}\left( \hat{\sigma }\right) =\mathbf {e} _{2}(\hat{\sigma })+\theta _{1}$ and the endowment value is equal across states, then the old-age consumption volatility is identical to the asset size volatility.

To assess the volatility of the asset holdings themselves, consider the six simulations that did not include the asset choice $\theta _{1}^{\prime }\left( \hat{\sigma }\right)$ in the objective function. Averaged across these 6 simulations, the unconditional standard deviations for $\theta _{1}$ in the sunspot model (no endowment risk) are 89 % as large as their respective unconditional standard deviations with endowment risk. The range for this ratio across all six simulations runs from 81 to 100 %.

4.7 Economies with initial condition determinacy

A sufficient condition for initial condition determinacy is the property of gross substitution in consumption. While sufficient, this property is not necessary. In a deterministic setting, Kehoe and Levine (1990) and Feng (2013) find economies that do not satisfy this sufficient condition and yet exhibit initial condition determinacy.

In a stochastic setting, finding the set of economies that lead to initial condition determinacy remains just as relevant. We consider two experiments in which the economy parameters are changed. In the first experiment, the parameter for consumer risk aversion is reduced from $b=4$ to $b=3.2$, with all other parameters held constant. In the second experiment, the endowment process is changed from $e=\{3,12,1 \pm 5\,\%\}$ to $e=\{3,8,2 \pm 5\,\%\}$, with all other parameters held constant. The two endowment processes are given by:

 Endowment process

$e=\{3,12,1\pm 5\,\%\}$	$e=\{3,8,2\pm 5\,\%\}$
$\begin{array}{ll} \mathbf {e}_{0}(s)=3 &{} \forall s\in \{1,2\}\\ \mathbf {e}_{1}(s)=12 &{} \forall s\in \{1,2\}\\ \mathbf {e}_{2}(1)=1+\epsilon &{} \epsilon =0.05\\ \mathbf {e}_{2}(2)=1-\epsilon &{} \end{array}$	$\begin{array}{ll} \mathbf {e}_{0}(s)=3 &{} \forall s\in \{1,2\}\\ \mathbf {e}_{1}(s)=8 &{} \forall s\in \{1,2\}\\ \mathbf {e}_{2}(1)=2+\epsilon &{} \epsilon =0.10\\ \mathbf {e}_{2}(2)=2-\epsilon &{} \end{array}$

For each of the two experiments, we compute simulated moments as in the original economy. We numerically confirm that the two economies exhibit initial condition and incomplete markets determinacy. We do not need to implement selection rules, since there exists a unique vector of state variables each period: $\left( \omega ^{\prime }(\sigma ),m^{\prime }\left( \sigma \right) \right) _{\sigma \in \mathbf {S}}=\mathbf {F}\left( s,\omega ,m\right)$.

The simulated moments are given in Table 6. For the first experiment (with b changed from $b=4$ to $b=3.2$), the ratio $\frac{std\left( \theta _{1}\right) }{mean\left( \theta _{1}\right) }=0.0085$, which is $10\,\%$ as large as the average ratio across all simulations in the model with $b=4$ (average across all simulations is$\frac{std\left( \theta _{1}\right) }{mean\left( \theta _{1}\right) }=0.088$). In terms of prices, the ratio $\frac{std\left( q\right) }{mean\left( q\right) }=0.0399$, which is $20\,\%$ as large as the average ratio across all simulations in the model with $b=4$ (average across all simulations is $\frac{std\left( q\right) }{mean\left( q\right) }=0.192$).

For the second experiment (with e changed from $e=\{3,12,1\pm 5\,\%\}$ to $e=\{3,8,2\pm 5\,\%\}$), the ratio $\frac{std\left( \theta _{1}\right) }{mean\left( \theta _{1}\right) }=0.0161$, which is $20\,\%$ as large as the average across all 8 simulations in the model with $e=\{3,12,1\pm 5\,\%\}$. In terms of prices, the ratio $\frac{std\left( q\right) }{mean\left( q\right) }=0.0896$, which is $50\,\%$ as large as the average across all 8 simulations in the model with $e=\{3,12,1\pm 5\,\%\}$.

Table 6 Simulated moments (determinacy)

Relative to the baseline economy, a reduction in the risk-aversion parameter or a reduction in the volatility of the endowment process can lead to initial condition determinacy. Initial condition determinacy implies incomplete markets determinacy (Theorem 3), meaning that determinacy does not have long-run effects. The simulation results in Table 6 reveal that economies with initial condition determinacy have asset price and asset holding volatilities on the same order of magnitude as the endowment volatility. Moreover, the asset price and asset holding volatilities for the determinate economies are 10–50 % as large as the corresponding volatilities for nearby economies with incomplete markets indeterminacy.

4.8 Discussion of the example economy

Well-known empirical puzzles concerning asset price and consumption volatility document that the observed volatility of both variables is higher than what is predicted from classical theory.¹³ This paper does not attempt to solve these empirical puzzles, but rather to understand what role, if any, incomplete markets indeterminacy plays in supporting asset price and consumption volatility. We study simple economies in which the effects of indeterminacy on asset price and consumption volatility are easily elicited, and thus only utilize a stylized calibration as in Kehoe and Levine (1990).

The cohorts in our economies consist of a unit mass of homogeneous households that each live for three periods. This is the simplest setting in which asset trade is nontrivial. The mechanism under which indeterminacy has real effects requires nontrivial asset trade: households form beliefs about the asset prices in future periods, trade assets based upon these beliefs, and use the asset payouts to smooth consumption. We view each period as lasting for 20 years and impose a life-cycle earnings profile consistent with Auerbach and Kotlikoff (1987) and Gourinchas and Parker (2002).

Without a more realistic calibration, our numerical results provide an incomplete answer to the question of whether incomplete markets indeterminacy in SOLG models provides a theoretical foundation for the asset price and consumption volatility observed in the data. What we have learned is that incomplete markets indeterminacy does matter; it has real effects, and these persist in the long run. The next step is to evaluate the degree to which our numerical results extend to a more realistic setting. Does the scale of the model matter? Both our theoretical results and computational methodology are immediately applicable to a large-scale model, but in a large-scale model, the relation between indeterminacy and volatility becomes blurred and the computation becomes untractable.

In the current model, households live for 60 years (the expected life span for adults), but only receive three realizations of uncertainty during their lifetime. In a large-scale model, households would continue to live for 60 years, but would instead receive realizations of uncertainty every year (or every quarter). The partition of uncertainty will be finer, and households will be able to trade on this uncertainty with higher frequency. We hypothesize that the effects of indeterminacy will be amplified with higher frequency trading, as households have more opportunities to form self-fulfilling beliefs about asset prices in future periods.

5 Conclusion

In this paper, we analyze the effects of indeterminacy on consumption and asset price volatility in SOLG models. We introduce the concept of incomplete markets indeterminacy and compute its effects by (i) approximating the entire set of competitive equilibria and (ii) running simulations over a variety of selection rules. Our simulations indicate that the choice of selection rule has welfare effects. Even for the selection rules with the most conservative predictions, we find that indeterminacy is an order of magnitude more important than endowment risk in explaining consumption and asset price volatility.

These findings suggest that for economies in which indeterminacy is present, consumers’ expectations of prices play an important role in the allocation of resources. It is only in understanding how these expectations affect resource allocation that we can implement welfare-improving policies. Analysis of specific welfare-improving policies in this class of models is left for future research.

Appendix

1.1 Proof of Theorem 1

To show that a Markov equilibrium satisfies the SCE definition, the Euler Eqs. (9) and (11) must be necessary and sufficient for household optimality. Necessity is immediate. Sufficiency follows as households are finite-lived.

1.2 Proof of Theorem 2

From the standard arguments used to show the existence of a SCE, the set of shadow prices of investment $m\left( s^{t}\right)$ belong to a compact set $\Delta \subseteq \mathbb {R}^{J}$ in all nodes. The iterative construction begins with the correspondence $\mathbf {V}_{0}:{\mathbf {S}\times \mathbb {R} }\rightrightarrows {\mathbb {R}}_{+}^{J}$ such that $\mathbf {V} _{0}\left( s,\omega \right) =\Delta$ $\forall \left( s,\omega \right) \in {\mathbf {S}\times \mathbb {R}.}$

Given a correspondence $\mathbf {V}_{n}:{\mathbf {S}\times \mathbb {R} }\rightrightarrows {\mathbb {R}}_{+}^{J}$ for any $n\ge 0$, define an operator $\mathbf {B}$ that maps the correspondence $\mathbf {V}_{n} :{\mathbf {S}\times \mathbb {R}}\rightrightarrows {\mathbb {R}}_{+}^{J}$ to a new correspondence $\mathbf {V}_{n+1}:{\mathbf {S}\times \mathbb {R}} \rightrightarrows {\mathbb {R}}_{+}^{J}$ defined as follows:

$$\begin{aligned}&\mathbf {V}_{n+1}\left( s,\omega \right) \\&\quad =\left\{ m\in \Delta : \begin{array}{l} \hbox {for }\left( q,\theta _{1}\right) =\mathbf {f}\left( s,\omega ,m\right) ,\\ \hbox {there exists }m^{\prime }\left( \sigma \right) \in \mathbf {V}_{n}\left( \sigma ,r(\sigma )\theta _{1}\right) \qquad \forall \sigma \in \mathbf {S}\quad \hbox {and}\\ \left( q^{\prime }\left( \sigma \right) ,\theta _{1}^{\prime }\left( \sigma \right) \right) =\mathbf {f}\left( \sigma ,r(\sigma )\theta _{1},m^{\prime }\left( \sigma \right) \right) \qquad \forall \sigma \in \mathbf {S} \quad \hbox {such that}\\ q^{j}u_{c}\left[ \mathbf {e}_{0}(s)+q\theta _{1}\right] =\beta \sum \limits _{\sigma \in \mathbf {S}}\pi (s,\sigma )\frac{m^{\prime j}\left( \sigma \right) }{q^{\prime j}\left( \sigma \right) }r^{j}\left( \sigma \right) \quad \forall j\in \mathbf {J} \end{array} \right\} . \end{aligned}$$

The correspondences are defined recursively using this operator $\mathbf {B}:$

$$\begin{aligned} \mathbf {V}_{n+1}=\mathbf {B}\left( \mathbf {V}_{n}\right) . \end{aligned}$$

The Markov equilibrium policy correspondence is defined as follows:

$$\begin{aligned} \mathbf {V}^{*}\left( s,\omega \right) =\lim _{n\rightarrow \infty } \mathbf {B}\left( \mathbf {V}_{n}\left( s,\omega \right) \right) \quad \forall \left( s,\omega \right) \in {\mathbf {S}\times \mathbb {R}.} \end{aligned}$$

Theorem 1 from Feng et al. (2014), reproduced below, guarantees the existence of a Markov equilibrium policy correspondence.

Theorem 6

Let $\mathbf {V}_{0}$ be a compact-valued correspondence such that $\mathbf {V}_{0}\supset \mathbf {V}^{*}$. Let $\mathbf {V}_{n+1}=\mathbf {B} \left( \mathbf {V}_{n}\right) ,n\ge 0$. Then, $\mathbf {V}_{n}\rightarrow \mathbf {V}^{*}$ as $n\rightarrow \infty$. Moreover, $\mathbf {V} ^{*}$ is the largest fixed point of the operator $\mathbf {B}$, i.e., if $\mathbf {V}=\mathbf {B(V)}$, then ${\mathbf {V}\subset \mathbf {V}}^{*}$.

1.3 Proof of Theorem 4

Suppose, in order to obtain a contradiction, that the markets are complete. This implies that the allocation is (interim) Pareto efficient. The equilibrium allocation is stationary (Henriksen and Spear 2012). This implies that both $\mathbf {V}^{*}$ and $\mathbf {F}$ are single-valued correspondences.

1.4 Proof of Theorem 5

The transition correspondence $\mathbf {F}:graph(\mathbf {V}^{*})\rightrightarrows \left( {\mathbb {R}\times \mathbb {R}}_{+}^{J}\right) ^{S}$ defined by

$$\begin{aligned}&\mathbf {F}\left( s,\omega ,m\right) \\&\quad \! =\left\{ \left( \omega ^{\prime } (\sigma ),m^{\prime }\left( \sigma \right) \right) _{\sigma \in \mathbf {S}}: \begin{array}{c} \omega ^{\prime }(\sigma )=r(\sigma )\mathbf {f}_{\theta }\left( s,\omega ,m\right) \\ m^{\prime }\left( \sigma \right) \in \mathbf {V}^{*}\left( \sigma ,\omega ^{\prime }(\sigma )\right) \\ q^{\prime }\left( \sigma \right) =\mathbf {f}_{q}\left( \sigma ,\omega ^{\prime }(\sigma ),m^{\prime }\left( \sigma \right) \right) \\ q^{j}u_{c}\left[ \mathbf {e}_{0}(s)+q\theta _{1}\right] =\beta \sum \limits _{\sigma \in \mathbf {S}}\pi (s,\sigma )\frac{m^{\prime k}\left( \sigma \right) }{q^{\prime k}\left( \sigma \right) }r^{j}\left( \sigma \right) \quad \forall j,k\in \mathbf {J} \end{array} \right\} . \end{aligned}$$

By definition of $\mathbf {f}_{q}\left( \sigma ,\omega ^{\prime }(\sigma ),m^{\prime }\left( \sigma \right) \right) :$

$$\begin{aligned} \frac{m^{\prime k}\left( \sigma \right) }{q^{\prime k}\left( \sigma \right) }=\frac{m^{\prime 1}\left( \sigma \right) }{q^{\prime 1}\left( \sigma \right) }\quad \forall k. \end{aligned}$$

This means that the J Euler equations given by:

$$\begin{aligned} q^{j}u_{c}\left[ \mathbf {e}_{0}(s)+q\theta _{1}\right] =\beta \sum \limits _{\sigma \in \mathbf {S}}\pi (s,\sigma )\frac{m^{\prime 1}\left( \sigma \right) }{q^{\prime 1}\left( \sigma \right) }r^{j}\left( \sigma \right) \quad \forall j\in \mathbf {J.} \end{aligned}$$

Suppose, in order to obtain a contradiction, that incomplete markets indeterminacy does not hold. This implies that $\mathbf {F}\left( s,\omega ,m\right)$ is determinate (0-dimensional image) for all state variables $\left( s,\omega ,m\right)$. Under initial condition indeterminacy, $\mathbf {V}^{*}\left( \sigma ,\omega ^{\prime }(\sigma )\right)$ is indeterminate (strictly positive dimension) for all state variables $\left( s,\omega ,m\right)$ (with corresponding state variable $\left( \sigma ,\omega ^{\prime }(\sigma )=r(\sigma )\mathbf {f}_{\theta }\left( s,\omega ,m\right) \right)$). This implies that for all state variables $\left( s,\omega ,m\right)$, $\exists !\left( \omega ^{\prime }(\sigma ),m^{\prime }\left( \sigma \right) \right) _{\sigma \in \mathbf {S}}$ such that the Euler equations are satisfied.¹⁴ If $\left( \omega ^{\prime }(\sigma ),m^{\prime }\left( \sigma \right) \right) _{\sigma \in \mathbf {S}}$ is uniquely determined, then with $q^{\prime }\left( \sigma \right) =\mathbf {f}_{q}\left( \sigma ,\omega ^{\prime }(\sigma ),m^{\prime }\left( \sigma \right) \right)$, the vector $\left( \frac{m^{\prime 1}\left( \sigma \right) }{q^{\prime 1}\left( \sigma \right) }\right) _{\sigma \in \mathbf {S}}$ is uniquely determined.

If $\left( \frac{m^{\prime 1}\left( \sigma \right) }{q^{\prime 1}\left( \sigma \right) }\right) _{\sigma \in \mathbf {S}}$ is uniquely determined, the Euler equations for both the young and middle-aged consumers imply:

$$\begin{aligned} \frac{\left( u_{c}\left[ \mathbf {e}_{1}(\sigma )-\omega ^{\prime }\left( \sigma \right) -q^{\prime }\left( \sigma \right) \theta _{1}^{\prime }\left( \sigma \right) \right] \right) _{\sigma \in \mathbf {S}}}{u_{c}\left[ \mathbf {e}_{0}(s)+q\theta _{1}\right] }=\frac{\left( u_{c}\left[ \mathbf {e}_{2}(\sigma )+\omega ^{\prime }\left( \sigma \right) \right] \right) _{\sigma \in \mathbf {S}}}{u_{c}\left[ \mathbf {e}_{1}(s)-\omega -q\theta _{1}\right] }. \end{aligned}$$

For all state variables $\left( s,\omega ,m\right)$, it is not possible to find a Pareto-improving reallocation. This means that the allocation is (interim) Pareto efficient, meaning that markets are complete. This completes the argument.

1.5 Numerical algorithm

The vector of possible values for bond-holding and shocks are given by $\hat{\varvec{\Theta }}=\left\{ \theta _{0}^{i_{1}}\right\} _{i_{1} =1}^{N_{\theta }}$, $\hat{\mathbf {S}}=\left\{ s_{0}^{i_{2}}\right\} _{i_{2}=1}^{N_{s}}$. For each pair of the bond-holding and shock grids, $\left( \theta _{0}^{i_{1}},s_{0}^{i_{2}}\right)$, we also define a finite vector of possible values for the image of the correspondence: $\hat{\mathbf {V}}_{0}^{\mu ,\varepsilon }\left( \theta _{0}^{i_{1}},s_{0}^{i_{2}}\right) =\left\{ m_{0}^{i_{1},i_{2},j}\right\} _{j=1}^{Nv}$.¹⁵ Notice, $\lim _{N_{\theta }\rightarrow \infty }\hat{\varvec{\Theta }}=\varvec{\Theta }$, $\lim _{Nv\rightarrow \infty }\hat{\mathbf {V}}_{0}^{\mu ,\varepsilon }\left( \theta _{0}^{i_{1}},s_{0}^{i_{2}}\right) =\tilde{\mathbf {V}}_{0} ^{\mu ,\varepsilon }\left( \theta _{0}^{i_{1}},s_{0}^{i_{2}}\right)$. Finally, we construct the discrete version of operator $\mathbf {B}^{h,\mu ,N}$ by eliminating points (in the Euler equation, for a predetermined tolerance $\epsilon >0$) as follows:

• 1. Given $\left( \theta _{0}^{i_{1}},s_{0}^{i_{2}}\right)$, pick a point $m_{0}^{i_{1},i_{2},j}$ in the vector $\hat{\mathbf {V}}_{0}^{\mu ,\varepsilon }\left( \theta _{0}^{i_{1}},s_{0}^{i_{2}}\right)$. From $m_{0}^{i_{1} ,i_{2},j}$, we can determine the values of $\left( \theta ^{^{\prime } i_{1},i_{2},j},q^{i_{1},i_{2},j}\right)$ by solving for

  $$\begin{aligned} m_{0}^{i_{1},i_{2},j}-q^{i_{1},i_{2},j}\cdot u_{c}\left( e_{1}(s_{0}^{i_{2} })+\theta _{0}^{i_{1}}-q^{i_{1},i_{2},j}\theta ^{^{\prime }i_{1},i_{2},j}\right)&=0, \end{aligned}$$

  (12)
  $$\begin{aligned} m_{0}^{i_{1},i_{2},j}-\beta \sum _{s^{\prime }}\pi (s^{\prime }|s_{0})u_{c}\left( e_{2}(s^{\prime ^{\prime }i_{1},i_{2},j}\right)&=0. \end{aligned}$$

  (13)

  Thus, if for all $m^{\prime }\in \hat{\mathbf {V}}_{0}^{\mu ,\varepsilon } (\theta ^{^{\prime }i_{1},i_{2},j},s^{\prime })=\left\{ m^{^{\prime }l} (\theta ^{^{\prime }i_{1},i_{2},j},s^{\prime })\right\} _{l=1}^{N_{V}}$ we have

  $$\begin{aligned} \min _{m^{\prime }\in \left\{ m^{\prime l}\right\} _{l=1}^{N_{V}}}\left\| q^{i_{1},i_{2},j}\cdot u_{c}\left( e_{0}(s_{0}^{i_{2}})-q^{i_{1},i_{2} ,j}\theta ^{^{\prime }i_{1},i_{2},j}\right) \!-\!\beta \sum \pi (s^{\prime } |s_{0}^{i_{2}})\left( \frac{m^{\prime }}{q^{\prime }}\right) \right\| >\epsilon \nonumber \\ \end{aligned}$$

  (14)

  where the value of $q^{\prime }$ is determined by the same procedure in finding $\big ( \theta ^{^{\prime }i_{1},i_{2},j},q^{i_{1},i_{2},j}\big )$, then $\hat{\mathbf {V}}_{1}^{\mu ,\varepsilon }\left( \theta _{0}^{i_{1}},s_{0} ^{i_{2}}\right) =\hat{\mathbf {V}}_{0}^{\mu ,\varepsilon }\left( \theta _{0}^{i_{1}},s_{0}^{i_{2}}\right) -m_{0}^{i_{1},i_{2},j}$.

• 2. Iterate over all possible values $m_{0}^{i_{1},i_{2},j}\in \hat{\mathbf {V}}_{0}^{\mu ,\varepsilon }\left( \theta _{0}^{i_{1}},s_{0}^{i_{2} }\right)$, and all possible $\left( \theta _{0}^{i_{1}},s_{0}^{i_{2} }\right) \in \hat{\varvec{\Theta }}\times \hat{\mathbf {S}}$.

• 3. Iterate until convergence is achieved $\sup \left\| \hat{\mathbf {V} }_{n}^{\mu ,\varepsilon }-\hat{\mathbf {V}}_{n-1}^{\mu ,\varepsilon }\right\| =0$.

At the limit of the above algorithm, we have $\lim _{n\rightarrow \infty }\hat{\mathbf {V}}_{n}^{\mu ,\varepsilon }=\hat{\mathbf {V}}^{\mu ,\varepsilon ^{*}}$ (Fig. 1).

Fig. 1

Equilibrium set of $\left\{ \left\{ m^{\prime }(\sigma )\right\} _{\sigma \in \{1,2\}},m\right\}$ at given $\{s,\omega \}$