Abstract
This paper analyzes problems in which a large benevolent player, controlling a set of policy variables, maximizes aggregate welfare in a continuous-time economy populated by atomistic agents subject to idiosyncratic shocks. We first provide as a benchmark the social optimum solution, in which a planner directly determines the individual controls. Then we analyze the optimal design of social policies depending on whether the large player may credibly commit to the future path of policies. On the one hand, we analyze the open-loop Stackelberg solution, in which the optimal policy path is set at time zero and the problem is time-inconsistent. On the other hand we analyze the time-consistent feedback Stackelberg solution.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
1 Introduction
Many problems of interest in economics involve a major player, typically the Government or the Central Bank, choosing some aggregate policy instrument such as a tax or an interest rate in order to maximize some aggregate welfare criterion. Most of the existing models analyzing optimal policies drastically simplify the economy by assuming a “representative agent,” that is, they summarize the behavior of heterogeneous firms or households in a single individual that accounts for the mean of the distribution.Footnote 1 The few exceptions typically rely either on “ brute force” numerical methods, that is, parameterizing the time-path of the optimal policies and then running a numerical search to find the optimal nodes, or on some particular set of assumptions such that a closed-form analytical solution can be obtained.Footnote 2
In this paper we analyze problems in which a large benevolent player, controlling a set of policy variables, maximizes an aggregate welfare criterion in a continuous-time economy populated by atomistic agents subject to idiosyncratic shocks. This can be seen as a particular case of the theory of mean field games (MFGs), introduced by [32, 33] and [27].Footnote 3 The economy is described as an infinite-horizon mean field game with state constraints in which the aggregate distribution affects individual agents through the dynamics of some aggregate variables. This framework encompasses the standard notion of a dynamic competitive equilibrium in macroeconomics, in which individual agents choose their control variables to maximize their value functions given the path of some aggregate variables (typically prices) and simultaneously the value of these variables is set such that aggregate supply equals aggregate demand (i.e., markets clear).Footnote 4 In continuous time, the system is composed by a Hamilton-Jacobi-Bellman (HJB) equation, which characterizes the individual problem in terms of the value function, a Kolmogorov forward (KF) or Fokker-Planck equation, which describes the dynamics of the cross-sectional distribution, and a number of market-clearing conditions based on the aggregation of individual variables. The individual agents may also face state constraints, so that the accessible state space is restricted to a subset of \( \mathbb {R} ^{n}.\) This model is typically denoted as the “ incomplete-market model with idiosyncratic shocks,” as there is no aggregate uncertainty.
Before analyzing the optimal policies, we set as a benchmark the social optimum, defined as the allocation produced by a planner that maximizes aggregate welfare by directly determining the individual controls of each agent, under full information. The welfare criterion is summarized by a social welfare function, which aggregates the individual utility flows across time and states. We assume that the planner discounts future utility flows using the same discount factor as individual agents.Footnote 5 This problem can be seen as a particular case of the mean field control problem analyzed in [7] or the control of McKean-Vlasov dynamics studied by [11, 13] and [14]. The problem can be solved using calculus techniques in infinite-dimensional Hilbert spaces.Footnote 6 The necessary conditions can be characterized, as in the competitive equilibrium, by a forward-backward system of partial differential equations. The difference is that the individual value function is now replaced by the social value function, which describes the value that the planner assigns to each agent depending on her state. This social value function can be obtained from the planner’s HJB equation, which includes some Lagrange multipliers capturing the “ shadow price” of the market clearing conditions.
In order to analyze the optimal social policies we extend the competitive equilibrium model to include some aggregate policy variables controlled by a large benevolent agent, that we denote as ‘the leader,’ who maximizes the social welfare function. In contrast to the social optimum above, this is not a mean field control problem but a mean field game with a large (non-atomistic) player. In order to characterize this kind of games it is essential to understand whether the leader is able to make credible commitments about the future path of the policy variables. We consider two polar cases. On the one hand, we consider what economists typically define as the “ Ramsey problem,” which corresponds to the open-loop Stackelberg solution of the game.Footnote 7 In this case the leader solves at time zero, given the initial state distribution, a maximization problem in which it takes into account the impact of its decisions on the individual agents’ value and control functions. The necessary conditions for optimality include a social value function similar to the one in the social optimum and a distribution of costates that keep track of the value of breaking the “ promises” made at time zero about the future path of aggregate policies. As originally discussed by [31] this problem is time-inconsistent. On the other hand, we analyze the feedback (Markov) Stackelberg solution, in which the leader cannot make credible commitments.Footnote 8 This problem is time-consistent and can be seen as a setting in which the leader has only instantaneous advantage. The solution in this case is similar to the solution under commitment with the Lagrange multiplier associated to the individual HJB equation equal to zero. The intuition for this result is that in the feedback Stackelberg solution no credible promises can be made by the leader and thus the value of breaking them is zero.
Related Literature Since the original contribution of [34] a growing literature has emerged analyzing mean field control problems. In addition to the papers commented above, we should mention recent contributions by [29, 44,45,46, 49] and [25], among others. In economics, the problem has been analyzed in [17] in discrete time and in [37] and [42] in continuous time. The present paper reproduces the results in [42] analyzing the optimal allocation in a mean field game with state constraints in which the aggregate distribution affects individual agents only through some aggregate variables.
The literature analyzing mean field games with a non-atomistic (‘major’) player is less extensive. [28] and [39] introduced a linear-quadratic model with a major player whose influence does not fade away as the number of players tends to infinity. [41] generalized the model to the nonlinear case. In these early contributions the major player does not directly affect the dynamics of the atomistic players, only their cost functionals, and hence they are of little interest in most economic applications. [40] consider the more general case in which the major player directly affects the individual dynamics, but only in the context of linear-quadratic models. [8] analyze the general nonlinear case assuming a closed-loop Stackelberg game strategy in which the major player chooses her own control to minimize its expected cost taking into account the impact of this decision on the controls selected by the minor players. The solution is characterized by a set of stochastic partial differential equations. Carmona and Zhu [16] and Carmona and Wang [15], instead, consider a Nash game strategy using the probabilistic approach developed by [12]. Carmona and Wang [15], in particular, characterize the solution under open-loop, closed-loop and feedback controls. Our paper contributes to this literature in three main aspects. First, up to our knowledge this is the first paper to analyze both the open-loop and the feedback Stackelberg solutions in a model without aggregate uncertainty, characterizing these solutions as forward-backward systems of partial differential equations.Footnote 9 Second, we consider a case in which the major player (‘the leader’) maximizes the aggregate welfare of the atomistic agents—instead of its own individual welfare—in a model with state constraints and aggregate variables. This provides a useful tool for the future analysis of optimal policies in economic problems. Third, by presenting together the results under competitive equilibrium, social optimum and optimal social policies under commitment and discretion this paper aims at providing a unified framework to compare the properties of the resulting forward-backward systems.
The structure of the paper is as follows. Section 2 introduces the competitive equilibrium in a MFG form. Section 3 analyzes the social optimum, following [42]. Section 4 builds on [43] to analyze the optimal policies under commitment and discretion, including necessary conditions for the open-loop and feedback Stackelberg solutions. Finally, Sect. 5 concludes. All the proofs are presented in the Appendix.
It is important to remark that the proofs in this paper should be considered as “ informal” or as “ sketches of a proof” at best, and that many important issues have been overlooked. We hope that this paper will open new avenues for future research in mean field game theory with important applications in economics.
2 Competitive Equilibrium
First we provide a general model of a “ competitive equilibrium,” as it is typically understood in economics. We consider a continuous-time infinite-horizon economy. Let \(\left( \Sigma ,\mathcal {F} ,\left\{ \mathcal {F}_{t}\right\} ,\mathbb {P}\right) \) be a filtered probability space. There is a continuum of unit mass of ex-ante identical agents indexed by \(i\in [0,1].\)
2.1 Individual Problem
State First we analyze the problem of an individual agent. Let \( W^{i}\left( t\right) \) be a n-dimensional \(\mathcal {F} _{t}\)-adapted Brownian motion and \(X^{i}\left( t\right) \in \mathbb {R} ^{n}\) denote the state of the agent i at time \(t\in [0,\infty ).\) The individual state evolves according to a multidimensional Itô process of the form
where \(u\in U\subset \mathbb {R} ^{m}\) is a m-dimensional vector of control variables and \(Z\left( t\right) \in \mathbb {R} ^{p}\) is a deterministic p-dimensional vector of aggregate variables. The functional coefficients are defined as follows
The measurable functions b and \(\sigma \) satisfy a uniform Lipschitz condition in U : \(\exists K\ge 0,\) such that \(\forall x,x^{\prime }\in \mathbb {R} ^{n},\) \(\forall u,u^{\prime }\in U,\) \(\forall Z,Z^{\prime }\in \mathbb {R} ^{p}\)
We assume that U is a closed subset of \( \mathbb {R} ^{m}.\) Let \(\mathcal {U}\) be the set of measurable controls taking values in U. We allow for state constraints in which the state \(X\left( t\right) \) cannot leave the compact region \(\Omega \subset \mathbb {R} ^{n}\), that is, control \(u\left( \cdot \right) \) at a point \(X\left( t\right) =x\) is an admissible control if \(u\left( \cdot \right) \in \mathcal {U}\left( t,x\right) ,\) whereFootnote 10
We also assume that \(\sigma _{n}\left( x\right) =0\) if \(x\in \partial \Omega _{n}\) that is, that the volatility in the nth dimension is zero if the \( n- \)th dimensional boundary is reached. From now on, we drop the superindex i as there is no possibility of confusion.
Utility Functional Each agent maximizes her utility functional
where the discount factor \(\rho \) is a positive constant. The instantaneous utility function
satisfies a polynomial growth condition: \(\exists K,c>0,\) such that \(\forall x\in \mathbb {R} ^{n},\) \(\forall u\in U,\)
The optimal value function V(t, x) is defined as
subject to (1). The transversality condition is
Hamilton–Jacobi–Bellman (HJB) Equation The solution to this problem is given by a value function V(t, x) and a control strategy u(t, x) that satisfy the HJB equation
where \(\mathcal {A}_{u,Z}\) is given by:
and \(U_{t,x}\) is the subset of controls such that the corresponding vector field \(b\left( \cdot \right) \) points inside the constraint, i.e.
with \(\nu \left( x\right) \) being the outward normal vector at \(x\in \partial \Omega .\) Footnote 11
2.2 Aggregate Distribution and Aggregate Variables
Kolmogorov Forward (KF) Equation Assume that the transition measure of \(X\left( t\right) \) with initial value \(x_{0}\) has a density \(\mu (t,x;0,x_{0}),\) such that \(\forall F\in L^{2}( \mathbb {R} ^{n}):\)
The initial distribution of X at time \(t=0\) is \(\mu (0,x)=\mu _{0}(x).\) The dynamics of the distribution of agents
are given by the Kolmogorov Forward (KF) or Fokker-Planck equation
where \(\mathcal {A}_{u,Z}^{*}\) is the adjoint operator of \( \mathcal {A}_{u,Z}:\)
Market Clearing Conditions The vector of aggregate variables is determined by a system of p equations:
where
These equations are typically the market clearing conditions of the economy.
We may define the competitive equilibrium of this economy.
Definition 1
(Competitive equilibrium) The competitive equilibrium is composed by the vector of aggregate variables \(Z\left( t\right) \), the value function V(t, x), the control u(t, x) and the distribution \(\mu (t,x)\) such that
-
1.
Given \(Z\left( t\right) \) and \(\mu (t,x)\), V(t, x) is the solution of the HJB equation (4) and the optimal control is u(t, x).
-
2.
Given u(t, x) and \(Z\left( t\right) \), \(\mu (t,x)\) is the solution of the KF equation (6, 7).
-
3.
Given u(t, x) and \(\mu (t,x),\) the aggregate variables \(Z\left( t\right) \) satisfy the market clearing conditions (8).
Remark 1
It should be clear from this definition that a competitive equilibrium is just a particular instance of mean field game theory in which the aggregate distribution affects each individual agent only through the dynamics of the aggregate variables \(Z\left( t\right) .\)
3 The Social Optimum
Social Welfare Functional We study as a benchmark the allocation produced by a benevolent social planner who maximizes an aggregate welfare criterion, that is, instead of a decentralized problem with multiple decision makers we consider the case of a single decision-maker who controls each individual agent. This is a mean field control problem instead of a mean field game. The planner chooses the vector of control variables u(t, x) to be applied to every agent. The social welfare functional is
where \(\omega (t,x)\) are state-dependent Pareto weights. If \(\omega (t,x)=1,\) for all t and x, then we have a purely utilitarian social welfare function which gives the same weight to every agent.
The planner’s optimal value functional is
subject to the law of motion of the distribution (6, 7) and to the market clearing conditions (8).
Remark 2
Notice that the state variable at time t in this case is the infinite-dimensional density \(\mu \left( t\right) .\)
Remark 3
In the utilitarian case, the planner’s social welfare functional under a given control \(\tilde{u}\left( t,x\right) \in \mathcal {U}\left( t,x\right) \) is equivalent to aggregating the individual value function under the same control across all agents at time zero:
where \(V^{\tilde{u}}(t,x)\ \)is the individual value function under control \( \tilde{u}\), characterized by the HJB
and \(\mu (t,\tilde{x};0,x)\) is the transition probability from \(X\left( 0\right) =x\) to \(X\left( t\right) =\tilde{x}\) and
is the Chapman–Kolmogorov equation.
We provide necessary conditions to the problem (10).
Proposition 1
(Necessary conditions - social optimum) If a solution to problem (10) exists with \( e^{-\rho t}u\), \(e^{-\rho t}\mu \in L^{2}\left( [0,\infty )\times \mathbb {R} ^{n}\right) \) and \(e^{-\rho t}Z\in L^{2}[0,\infty )\), then the optimal value functional \(V^{opt}\left( \mu \left( 0,\cdot \right) \right) \) can be expressed as
where \(\phi \left( t,x\right) \) is the marginal social value function, which represents the social value of an agent at time t and state x. The social value function satisfies the planner’s HJB
where the Lagrange multipliers \(\lambda (t):=\left[ \lambda _{1}(t),\ldots ,\lambda _{k}(t),\ldots ,\lambda _{p}(t)\right] ^{\top },\) are given by
The social optimum of this economy is defined in a similar fashion as in the case of a competitive equilibrium above.
Remark 4
The social optimum is composed by the vector of aggregate variables \(Z\left( t\right) \), the social value function \(\phi (t,x)\), the control u(t, x), the Lagrange multipliers \(\lambda (t)\) and the distribution \(\mu (t,x)\) such that
-
1.
Given \(Z\left( t\right) ,\) \(\lambda (t)\) and \(\mu (t,x)\), \(\phi (t,x)\) is the solution of the planner’s HJB equation (12) and the optimal control is u(t, x).
-
2.
Given u(t, x) and \(Z\left( t\right) \), \(\mu (t,x)\) is the solution of the KF equation (6, 7).
-
3.
Given u(t, x) and \(\mu (t,x),\) the aggregate variables \(Z\left( t\right) \) satisfy the market clearing conditions (8).
-
4.
Given u(t, x), \(Z\left( t\right) \) and \(\mu (t,x),\) the Lagrange multipliers \(\lambda (t)\) satisfy (14).
Remark 5
The Lagrange multipliers \(\lambda (t)\) reflect the ‘shadow prices’ of the market clearing condition (8). They price, in utility terms, the deviation of an agent from the value of the aggregate variable: \( g_{k}(x,u)-Z_{k}.\)
Corollary 1
If the competitive equilibrium allocation satisfies
then the competitive equilibrium and the utilitarian optimal allocation \( \left( \omega =1\right) \) coincide:
and
4 Optimal Social Policies
4.1 General Setting
Aggregate Policy Variables Consider again the decentralized competitive equilibrium and assume that the state of each individual agent is now given by
where \(Y\left( t\right) \in \mathbb {R} ^{q}\) is a q-dimensional vector of aggregate policy variables:
and b satisfy a uniform Lipschitz condition.Footnote 12 These policy variables are chosen by a large agent, which we denote as ‘the leader.’ The leader maximizes a social welfare function
similar to the one in the previous section.
Remark 6
The difference between this problem and the social optimum is that, instead of a mean field control case, here we are analyzing a mean field game including a large non-atomistic agent (the leader).
Equilibrium Concepts We consider two alternative equilibrium concepts, which depend on the ability of the leader to make credible commitments about future policies.
-
1.
Commitment In the first case, we assume that at time zero the leader is able to credibly commit to the complete future path of policies \(\left\{ Y\left( t\right) \right\} _{t=0}^{\infty }.\) This corresponds to the open-loop Stackelberg equilibrium of the game, with
$$\begin{aligned} Y\left( t\right) =\Upsilon ^{C}\left( t,\text { }\mu (0,\cdot )\right) , \end{aligned}$$where \(\Upsilon ^{C}\) is a deterministic measurable function of calendar time and the initial distribution. This is equivalent to say that, given the initial distribution \(\mu \left( 0,\cdot \right) ,\) the leader announces at time \(t=0\) the complete future evolution of the aggregate policy variables \( \left\{ Y\left( t\right) \right\} _{t=0}^{\infty }\) and commits not to reevaluate this initial plan. When formulating the optimal plan, the leader takes into account the impact of its aggregate policies on each atomistic agent’s optimal controls. Given the leader’s policy path, individual agents maximize their individual value functions (2). The result is a vector optimal individual controls \(u\left( t,x;\left\{ Y\left( s\right) \right\} _{s=0}^{\infty }\right) \) which depends on the complete path of the leader policy variables.
-
2.
Discretion In the second case, no commitment device is available. This corresponds to the feedback Stackelberg equilibrium of the game, with
$$\begin{aligned} Y\left( t\right) =\Upsilon ^{D}\left( t,\text { }\mu (t,\cdot )\right) , \end{aligned}$$where \(\Upsilon ^{D}\) is a deterministic progressively measurable function of the current state distribution. In this case the aggregate policies are time-consistent. This problem can be seen as the limit as \(\Delta \rightarrow 0\) of a sequence of open-loop Stackelberg problems of length \( \Delta \) in which the initial state at each stage n is given by the distribution at the beginning of the stage \(\mu (t_{n},\cdot ).\)
4.2 Commitment
First we consider the solution under commitment, which in economics is typically denoted as the ‘Ramsey problem’ and which corresponds to the open-loop Stackelberg solution of this game.
Definition 2
(Commitment) The problem of the leader under commitment is to choose the complete path of policies \(\left\{ Y\left( t\right) \right\} _{t=0}^{\infty }\) at time zero in order to maximize the aggregate welfare (17) when the aggregate distribution \(\mu (t,x),\) aggregate variables \(Z\left( t\right) \) and individual value function \(V(t,x)\ \)and controls u(t, x) constitute a competitive equilibrium given \(\left\{ Y\left( t\right) \right\} _{t=0}^{\infty }\). Formally, this amounts to
subject to law of motion of the distribution (6, 7), to the market clearing conditions (8) and to the individual HJB equation (4).
The solution is given by the following proposition.
Proposition 2
(Necessary conditions—Commitment) If a solution to problem (18) exists with \(e^{-\rho t}u\), \( e^{-\rho t}\mu ,\) \(e^{-\rho t}V\in L^{2}\left( [0,\infty )\times \mathbb {R} ^{n}\right) \) and \(e^{-\rho t}Z,e^{-\rho t}Y\in L^{2}[0,\infty )\), it should satisfy the system of equations
where \(\phi (t,x)\) is the marginal social value function, given by
The Lagrange multipliers associated to the market clearing condition (8)
satisfy, \(k=1,\ldots ,p:\)
The distribution of Lagrange multipliers \(\theta \left( t,x\right) \) associated to the individual HJB equation follows
and the Lagrange multipliers associated to the individual first-order conditions
satisfy, \(j=1,\ldots ,m:\)
Remark 7
The equilibrium under commitment is composed by the competitive equilibrium equations described in Definition 1 plus the necessary conditions of the leader (19)–(24).
Remark 8
Notice that the problem in the case with \(m=1,\) \(\omega \left( \cdot \right) =1,\ f\) strictly concave and \(\frac{\partial ^{2}b_{i}}{\partial u_{j}\partial u_{k}}=0\) for \(j=1,\ldots ,m,\) \(k=1,\ldots ,p,\ \)if the solution is such that \(\lambda _{k}\left( \cdot \right) =0,\) \(k=1,\ldots ,p,\) then the other Lagrange multipliers are zero: \(\theta \left( \cdot \right) =\eta \left( \cdot \right) =0\) and the social value function coincides with the individual one, \(\phi (t,x)=V(t,x).\) The optimal aggregate policy \(Y\left( t\right) \) is such that
4.3 Discretion
Next we consider the case without commitment or feedback Stackelberg equilibrium of the game. We first define a finite-horizon commitment problem, in the same lines as Definition 2.
Definition 3
(Commitment - finite horizon) Given an initial density \(\mu (t,x),\) the problem of the leader under commitment in an interval \([t,t+\Delta ]\) with a terminal value functional \( W\left( \cdot \right) ,\) is to choose the complete path of policies \( \left\{ Y^{\Delta }\left( s\right) \right\} _{s\in [t,t+\Delta ]}\) at time t in order to maximize the aggregate welfare (17) when the aggregate distribution \(\mu (s,x),\) aggregate variables \(Z\left( s\right) \) and individual value function \(V(s,x)\ \)and controls u(s, x) constitute a competitive equilibrium given \(\left\{ Y^{\Delta }\left( s\right) \right\} _{s\in [t,t+\Delta ]}\). Formally, this amounts to
subject to law of motion of the distribution (6, 7), to the market clearing conditions (8) and to the individual HJB equation (4).The terminal indvidual value function \( v\left( t+\Delta ,\cdot \right) \) is also taken as given.
Given \(T>0,\) we assume that the interval [0, T] is divided in N intervals of length \(\Delta :=T/N.\)
Definition 4
(Discretion) An equilibrium under discretion in a finite interval [0, T] with a terminal value functional \(W^{T}\left( \cdot \right) \) is defined as the limit as \(N\rightarrow \infty ,\) or equivalently \(\Delta \rightarrow 0,\) of a sequence of functions\(\ Y^{\Delta }\left( t\right) \) given by the finite-horizon commitment problem introduced in Definition 3 over the intervals \([t,t+\Delta ]\) where \(t=n\Delta ,\) \(n=0,\ldots ,N-1\) and the terminal value of an interval n is defined as the value functional of the next interval:
with \(W^{N}\left( \cdot \right) =W^{T}\left( \cdot \right) .\) The infinite-horizon case is defined as the limit as \(T\rightarrow \infty \) with a transversality condition
The solution is given by the following proposition.
Proposition 3
(Necessary conditions—Discretion) If a solution to problem under discretion exists, it should satisfy the system of equations
\(r=1,\ldots ,q,\) where \(\phi (t,x)\) is the marginal social value function, given by
the Lagrange multipliers associated to the market clearing condition (8), \(\lambda _{k}(t),\) \(k=1,\ldots ,p,\) satisfy
and the Lagrange multipliers associated to the individual first-order conditions
satisfy, \(j=1,\ldots ,m:\)
Remark 9
The equilibrium under discretion is composed by the competitive equilibrium equations described in Definition 1 plus the necessary conditions of the leader (28)–(32).
Remark 10
Equations (28)–(32) coincide with the equivalent equations in the case of commitment with the Lagrange multipliers \(\theta \left( \cdot \right) =0.\) Lagrange multipliers \(\theta \) can be interpreted as the value to the leader of breaking the “promises” that the leader is making to individual agents. Under discretion, no promises can be made and thus these multipliers are zero.
5 Conclusions
This paper has analyzed the design of optimal social policies in an economy composed by a continuum of atomistic players subject to idiosyncratic shocks. The optimality of the policies is defined according to a social welfare function that aggregates, given some state-dependent Pareto weights, the individual utilities across agents. First, we consider two alternative benchmarks without social policies. On the one hand, the decentralized competitive equilibrium is defined as mean field game with aggregate variables and state constraints. On the other hand, the social optimum is a mean field control problem in which a planner chooses the individual policies in order to maximize aggregate welfare. Next we assume that a (non-atomistic) leader controls a vector of aggregate policies. This is a mean field game with a large player. We analyze two different equilibrium concepts. In the open-loop Stackelberg solution of the game the large player is able to make a credible commitment about the future path of the aggregate policy variables. In the feedback Stackelberg solution no such a commitment is possible and the policies are time-consistent. We characterize the necessary conditions, but we do not analyze important issues such as the existence or uniqueness of the solutions, which we leave for future research.
The main analytical tool employed in this paper is the Lagrange multiplier method in infinite-dimensional Hilbert spaces. An interesting question would be to analyze to what extent these results can also be obtained by means of the Pontryagin principle.
Finally, neither have we discussed the numerical implementation of the solution in the cases in which no analytical results are available. Nuño and Moll [42] and Nuño and Thomas [43] provide some insights on this respect extending previous work by [1, 2] and [3]. Due to the relevance of the potential applications, we are sure that this will be a fruitful field of research in the coming years.
Notes
For example, see Woodford [48] for a textbook treatment of monetary policy following a representative-agent approach.
For a textbook introduction to dynamic general equilibrium models in macroeconomics, see for instance [36].
A particular case of interest is the utilitarian one, in which the planner equaly weighs the utility of every agent. In this case we show how the welfare criterion is equivalent to aggregate the initial value function of the agents, given the initial state distribution.
See, e. g., Basar and Olsder [6, p. 413].
The process (16) is now characterized by an operator \(\mathcal {A} _{u,Z,Y}.\)
Notice that we are working now in \(\tilde{L}^{2}\left( \Phi \right) .\)
The limit is taken in an “ informal” way. Investigating the limit properly should require a careful analysis that we leave for future research.
References
Achdou, Y., Camilli, F., Capuzzo-Dolcetta, I.: Mean field games: numerical methods for the planning problem. SIAM J. Control Optim. 50, 77–109 (2012)
Achdou, Y., Capuzzo-Dolcetta, I.: Mean field games: numerical methods. SIAM J. Numer. Anal. 48, 1136–1162 (2010)
Achdou, Y., Han, J., Lasry, J.-M., Lions, P.-L., Moll, B.: Heterogeneous Agent Models in Continuous Time. Mimeo, New York (2015)
Aiyagari, R.: Uninsured idiosyncratic risk and aggregate saving. Q. J. Econ. 109(3), 659–684 (1994)
Bardi, M., Capuzzo-Dolcetta, I.: Optimal Control and Viscosity Solutions of Hamilton-Jacobi-Bellman Equations. Birkhä user, Boston (1997)
Basar, T., Olsder, G.J.: Dynamic Noncooperative Game Theory, 2nd edn. Society for Industrial and Applied Mathematics, Philadelphia (1999)
Bensoussan, A., Frehse, J., Yam, P.: Mean Field Games and Mean Field Type Control Theory. Springer, Berlin (2013)
Bensoussan, A., Chau, M.H.M., Yam, P.: Mean field games with a dominating player. Appl. Math. Optim. (2015, forthcoming)
Bewley, T.: Stationary monetary equilibrium with a continuum of independently fluctuating consumers. In: Hildenbrand, W., Mas-Collel, A. (eds.) Contributions to Mathematical economics in Honor of Gerard Debreu. North Holland, Amsterdam (1986)
Brezis, H.: Functional Analysis, Sobolev Spaces and Partial Differential Equations. Springer, Berlin (2011)
Carmona, R., Delarue, F.: Probabilistic analysis of mean-field games. SIAM J. Control Optim. 51(2013), 2705–2734 (2013)
Carmona, R., Delarue, F.: Mean field forward-backward stochastic differential equations. Electron. Commun. Probab. 18(68), 1–15 (2013b)
Carmona, R., Delarue, F.: Forward-backward stochastic differential equations and controlled McKean-Vlasov dynamics. Ann. Probab. 43(5), 2647–2700 (2015)
Carmona, R., Delarue, F., Lachapelle, A.: Control of McKean-Vlasov dynamics versus mean field games. Math. Financ. Econ. 7, 131–166 (2013)
Carmona, R., Wang, P.: A probabilistic approach to mean field games with major and minor players (2016). arXiv:1610.05404
Carmona, R., Zhu, X.: A probabilistic approach to mean field games with major and minor players. Ann. Appl. Probab. 26, 1535–1580 (2014)
Dávila, J., Hong, J.H., Krusell, P., Ríos-Rull, J.V.: Constrained efficiency in the neoclassical growth model with uninsurable idiosyncratic shocks. Econometrica 80(6), 2431–2467 (2012)
Dockner, E.J., Jorgensen, S., Van Long, N., Sorger, G.: Differential Games in Economics and Management Science. Cambridge University Press, Cambridge (2001)
Dyrda, S., Pedroni, M.: Optimal Fiscal Policy in a Model with Uninsurable Idiosyncratic Shocks. Mimeo, University of Minnesota, Minneapolis (2014)
Fabbri, G., Gozzi, F., Swiech, A.: Stochastic Optimal Control in Infinite Dimensions: Dynamic Programming and HJB Equations, with Chapter 6 by M. Fuhrman and G. Tessitore. Mimeo, New York (2016)
Falcone, M., Ferretti, R.: Semi-Lagrangian Approximation Schemes for Linear and Hamilton-Jacobi Equations. Society for Industrial and Applied Mathematics, Philadelphia (2014)
Fleming, W.H., Soner, H.M.: Controlled Markov Processes and Viscosity Solutions. Springer, Berlin (2006)
Gelfand, I.M., Fomin, S.V.: Calculus of Variations. Dover Publications, Mineola, NY (1991)
Gottardi, P., Kajii, A., Nakajima, T.: Optimal taxation and constrained inefficiency in an infinite-horizon economy with incomplete markets. Economics Working Papers ECO2011/18, European University Institute (2011)
Graber, P.J.: Linear quadratic mean field type control and mean field games with common noise, with application to production of an exhaustible resource. Appl. Math. Optim. 74(3), 459–486 (2016)
Heathcote, J., Storesletten, K., Violante, G.L.: Quantitative macroeconomics with heterogeneous households. Annu. Rev. Econ. 1, 319–354 (2009)
Huang, M., Caines, P.E., Malhamé, R.P.: Individual and mass behaviour in large population stochastic wireless power control problems: centralized and nash equilibrium solutions. Proceedings of the 42nd IEEE Conference on Decision and Control, Maui, Hawaii, pp. 98–103 (2003)
Huang, M.: Large-population LQG games involving a major player: the nash certainty equivalence principle. SIAM J. Control Optim. 48(5), 3318–3353 (2010)
Huang, M., Caines, P.E., Malhamé, R.P.: Social optima in mean field LQG control: centralized and decentralized strategies. IEEE Trans. Autom. Control 57(7), 1736–1751 (2012)
Itskhoki, O., Moll, B.: Optimal Development Policies with Financial Frictions. Mimeo, New York (2015)
Kydland, F., Prescott, E.: Rules rather than discretion: the inconsistency of optimal plans. J. Polit. Econ. 85, 473–490 (1977)
Lasry, J.M., Lions, P.L.: Jeux à champ moyen I—Le cas stationnaire. C. R. Acad. Sci. Ser I 343, 619–625 (2006a)
Lasry, J.M., Lions, P.L.: Jeux à champ moyen II. Horizon fini et contrôle optimal. C. R. Acad. Sci. Ser. I 343, 679–684 (2006b)
Lasry, J.M., Lions, P.L.: Mean field games. Jpn. J. Math. 2, 229–260 (2007)
Lippi, F., Ragni, S., Trachter, N.: Optimal monetary policy with heterogeneous money holdings. J. Econ. Theory 159, 339–368 (2015)
Ljungqvist, L., Sargent, T.: Recursive Macroeconomic Theory, 3rd edn. The MIT Press, Cambridge, MA (2012)
Lucas, R., Moll, B.: Knowledge growth and the allocation of time. J. Polit. Econ. 122(1), 1–51 (2014)
Luenberger, D.: Optimization by Vector Space Methods. Wiley-Interscience, Hoboken (1969)
Nguyen, S.L., Huang, M.: Linear-quadratic-Gaussian mixed games with continuum-parametrized minor players. SIAM J. Control Optim. 50(5), 2907–2937 (2012a)
Nguyen, S.L., Huang, M.: Mean field LQG games with mass behavior responsive to a major player. In: 51st IEEE Conference on Decision and Control (2012)
Nourian, M., Caines, P.E.: \(\epsilon \)-Nash mean field game theory for nonlinear stochastic dynamical systems with major and minor agents. SIAM J. Control Optim. 51(4), 3302–3331 (2013)
Nuño, G., Moll, B.: Social Optima in Economies with Heterogeneous Agents. Mimeo, New York (2017)
Nuño, G., Thomas, C.: Optimal Monetary Policy with Heterogeneous Agents. Mimeo, New York (2017)
Pham, H.: Linear quadratic optimal control of conditional McKean-Vlasov equation with random coefficients and applications (2016). arXiv:1604.06609
Pham, H., Wei, X.: Bellman equation and viscosity solutions for mean-field stochastic control problem (2015). arXiv:1512.07866
Pham, H., Wei, X.: Dynamic programming for optimal control of stochastic McKean-Vlasov dynamics (2016). arXiv:1604.04057
Sagan, H.: Introduction to the Calculus of Variations. Dover Publications, Mineola, NY (1992)
Woodford, M.: Interest and Prices: Foundations of a Theory of Monetary Policy. Princeton University Press (2003)
Yong, J.: Linear-quadratic optimal control problems for mean-field stochastic differential equations. SIAM J. Control Optim. 51, 2809–2838 (2013)
Yong, J.: Differential Games : A Concise Introduction. World Scientific Publishing Company, Singapore (2015)
Acknowledgements
The author is very grateful to Carlos Thomas and to an anonymous referee for helpful comments and suggestions. All remaining errors are mine.
Author information
Authors and Affiliations
Corresponding author
Additional information
The views expressed in this manuscript are those of the author and do not necessarily represent the views of Banco de España or the Eurosystem.
Appendix
Appendix
1.1 Proof of Proposition 1: Necessary Conditions in the Social Optimum
The problem of the planner is to maximize \(J^{opt}\left( u\left( \cdot \right) \right) \) subject to the KF equation (6) and the market clearing conditions (8). The latter can be expressed as
We define the domain \(\Phi :=[0,\infty )\times \mathbb {R} ^{n}.\) The problem of the planner can be expressed as an optimization problem in a suitable functional space such as
Nuño and Moll [42] show how \(\tilde{L}^{2}\left( \Phi \right) \) is a Hilbert space with the inner product
where \(\left\langle \cdot ,\cdot \right\rangle _{\Phi }\) is the standard inner product in \(L^{2}\left( \Phi \right) :\)
The idea is to construct a Lagragian including the KF equation (6) and the market clearing conditions (8) and to optimize with respect to the individual control \(u\left( \cdot \right) \) and the aggregate variables \(Z\left( \cdot \right) .\)
The Lagrangian functional results in
where \(e^{-\rho t}\phi (t,x)\in L^{2}\left( \Phi \right) \) and \(e^{-\rho t}\lambda _{k}(t)\in L^{2}[0,\infty ),\) \(k=1,\ldots ,p\) are the Lagrange multipliers associated to the KF equation (6) and market clearing conditions (8), respectively.
If \(\mathcal {L}\) has continuous Fréchet derivatives, a necessary condition for \(\left( \mu ,u_{1},\ldots ,u_{m},Z_{1},\ldots ,Z_{p}\right) \) to be a maximum of (34) is that the Gateaux derivatives with respect to each of these functions equals zero.Footnote 13
It will prove useful to modify the second term in the Lagrangian
where we have integrated by parts with respect to time in the term \(\frac{ \partial \mu }{\partial t}\) and applied the fact that \(\mathcal {A} _{u,Z}^{*}\) is the adjoint operator of \(\mathcal {A}_{u,Z}\) in \( L^{2}\left( \mathbb {R} ^{n}\right) \subset \tilde{L}^{2}\left( \Phi \right) .\)
The Gateaux derivative with respect to \(\mu \) is
and it equals zero in the maximum for any function \(h(t,x)\in \tilde{L} ^{2}\left( \Phi \right) \). The term \(\int \phi \left( 0,x\right) \mu \left( 0,x\right) dx\) can be ignored in the optimization as \(\mu \left( 0,x\right) =\mu _{0}(x)\), that is, the initial distribution is given and thus \(h\left( 0,x\right) =0\) for all \(x\in \mathbb {R} ^{n}\). We obtain
which is the HJB equation of the planner (12).
The Gateaux derivative with respect to the control \(u_{j}\) is
where \(\mathcal {A}_{u_{j}+\alpha h,Z}:=\mathcal {A}_{u_{1},\ldots ,u_{j}+\alpha h,\ldots ,u_{m},Z}\). Given the state constraint \(u\in \mathcal {U}(t,x)\) and the optimality condition that (38) equals zero in the maximum for any \( h(t,x)\in \tilde{L}^{2}\left( [0,\infty )\times \Omega \right) \) then
The Gateaux derivative with respect to the aggregate variable \(Z_{k}\) is
and it equals zero in the maximum for any \(e^{-\rho t}h(t)\in L^{2}[0,\infty ).\) Here \(\mathcal {A}_{u,Z_{k}+\alpha h}^{*}:=\mathcal {A} _{u,Z_{1},\ldots ,Z_{k}+\alpha h,..Z_{p}}^{*}.\) This can be expressed as
and hence
As this is satisfied for any h(t), we obtain that
where we have integrated by parts.
Finally, if we multiply by \(e^{-\rho t}\mu \left( t,x\right) \) and integrate at both sides of the planner’s HJB equation (36)
where in the second line we have applied the market clearing condition (8) and in the third line the fact that \(\mathcal {A} _{u,Z}^{*}\) is the adjoint operator of \(\mathcal {A}_{u,Z}.\) If we integrate by parts the first term
as \(\lim _{T\rightarrow \infty }e^{-\rho T}\phi \left( T,x\right) =0.\) Therefore, we have
where we have applied the fact that \(\mu \) satisfies the KF equation (6): \(-\frac{\partial \mu }{\partial t}+\mathcal {A}_{u,Z}^{*}\mu =0.\) The social value functional is thus
1.2 Proof of Proposition 2: Necessary Conditions in the Problem with Commitment
The problem of the leader is to maximize (17) subject to the KF equation (6), the market clearing conditions (8) and to the individual HJB equations (4), where the optimal individual controls are given by the first-order conditions
The Lagragian in this case is the one in Proposition 1 extended to include two extra terms that capture the value function and control dynamics:
where \(\theta \left( t,x\right) ,\eta _{j}\left( t,x\right) \in \tilde{L} ^{2}\left( \Phi \right) ,\) \(j=1,\ldots ,m,\) are the Lagrange multipliers associated to the HJB equation (4) and to the first-order conditions (41), respectively.
The Gateaux derivative with respect to \(\mu \) is again
and therefore \(\phi \left( t,x\right) \) should satisfy the leader’s HJB
The Gateaux derivative with respect to the aggregate variable \(Z_{k}\) is
for any \(e^{-\rho t}h(t)\in L^{2}[0,\infty ).\) Here \(\mathcal {A} _{u,Z_{k}+\alpha h,Y}^{*}:=\mathcal {A}_{u,Z_{1},\ldots ,Z_{k}+\alpha h,\ldots ,Z_{p},Y}^{*}\) and
The Gateaux derivative should be equal to zero in the maximum:
As this is satisfied for any h(t), we obtain that
where we have integrated by parts in the last equality.
In order to compute the Gateaux derivative with respect to the individual value function V, we first expressed the fourth term in the Lagragian as
where we have integrated by parts with respect to time in the term \(\frac{ \partial V}{\partial t}\) and applied the fact that \(\mathcal {A} _{u,Z,Y}^{*}\) is the adjoint operator of \(\mathcal {A}_{u,Z,Y}.\) The Gateaux derivative simplifies to
The last term in the derivative can be expressed as
where we have integrated by parts. Due to the transversality condition of the individual problem, \(\lim _{T\rightarrow \infty }e^{-\rho T}V\left( T,x\right) =0,\) we have \(\lim _{T\rightarrow \infty }h\left( T,x\right) =0\) \( \forall x\in \mathbb {R} ^{n}.\) For \(t<\infty ,\) the Gateaux derivative should be zero for any \( h\left( t,x\right) \in \tilde{L}^{2}\left( \Phi \right) \) and therefore we obtain:
The Gateaux derivative with respect to the individual control \(u_{j}\) is
and thus the maximum should satisfy
Notice that \(\frac{\partial f}{\partial u_{j}}+\sum _{i=1}^{n}\frac{\partial b_{i}}{\partial u_{j}}\frac{\partial V}{\partial x_{i}}=0\) due to the first-order conditions (41).
Finally, the Gateaux derivative with respect to the aggregate policy \(Y_{r}\)
equals zero in the maximum for any \(h(t)\in e^{-\rho t}L^{2}[0,\infty ).\) Here \(\mathcal {A}_{u,Z,Y_{r}+\alpha h}^{*}:=\mathcal {A} _{u,Z,Y_{1},\ldots ,Y_{r}+\alpha h,\ldots ,Y_{q}}^{*}.\) This can be expressed as
As this is satisfied for any h(t), we obtain that
where we have integrated by parts to obtain the last expression.
1.3 Proof of Proposition 3: Necessary Conditions in the Problem with Discretion
The proof proceeds in two steps. First we solve a commitment problem over a fixed period of length \(\Delta \) taking as given the next period value functional \(W\left( \mu \left( t+\Delta ,\cdot \right) \right) .\) Then we take the limit as \(\Delta \rightarrow 0.\)
Step 1: Solution Given a Fixed Time Step \(\Delta \)
We have assumed that, given \(T>0,\) the interval [0, T] is divided in N intervals of length \(\Delta :=T/N.\) First we solve the open-loop Stackelberg problem (26) over a fixed time interval \(s\in [t,t+\Delta ],\) where t is a multiple of \(\Delta ,\) subject to the KF equation (6), the market clearing conditions (8) and to the individual HJB equations (4) with optimal individual controls (41). The solution mimics the proof of Proposition 2 above with two major differences. The first one is the finite-horizon nature of the problem. The second is the presence of the terminal value \(W\left( \mu \left( t+\Delta ,\cdot \right) \right) .\)
The Lagragian is similar as the one in (42) with the inclusion of the terminal value functional \(W\left( \mu \left( t+\Delta ,\cdot \right) \right) \):
where time is denoted as \(s\in [t,t+\Delta ]\) and \(\Phi _{t}:=[t,t+\Delta ]\times \mathbb {R} ^{n}\).
The Gateaux derivative with respect to \(\mu \) isFootnote 14
If W is Frechet differentiable then the Gateaux derivative of W can be expressed as
where \(\frac{\delta W}{\delta \mu }(\mu \left( t+\Delta ,\cdot \right) )\in L^{2}\left( \mathbb {R} ^{n}\right) .\) The optimality condition then implies that
The optimality conditions with respect to aggregate variables \(Z_{k}\), individual controls \(u_{j}\) and aggregate policies \(Y_{r}^{\Delta }\) are the same as in Proposition 2:
Finally, the Gateaux derivative with respect to the individual value function V is
and the optimality condition then results in
where we have taken into account the fact that \(h\left( t+\Delta ,\cdot \right) =0\) as the terminal individual value function \(v\left( t+\Delta ,\cdot \right) \) is given.
Step 2: Taking the Limit \(\Delta \rightarrow 0\)
We take the limit as \(N\rightarrow \infty ,\) or equivalently, \(\Delta \rightarrow 0.\) Footnote 15 In this case, the value of the Lagrange multiplier \(\theta \) in equation (47) is zero:\(\ \theta \left( t,x\right) =0,\) \(\forall x\in \mathbb {R} ^{n}.\) The HJB equation (45) then results in
If we take the limit as \(T\rightarrow \infty ,\) then \(\lim _{T\rightarrow \infty }e^{-\rho T}\frac{\delta W}{\delta \mu }(\mu \left( T,x\right) )=\lim _{T\rightarrow \infty }e^{-\rho T}\phi \left( T,x\right) =0,\) which is the transversality condition of the infinite-horizon problem.
Taking into account the values of \(\theta \left( \cdot \right) =0\) and \(\phi \left( \cdot \right) =w\left( \cdot \right) ,\) the rest of optimality conditions simplify to
Rights and permissions
About this article
Cite this article
Nuño, G. Optimal Social Policies in Mean Field Games. Appl Math Optim 76, 29–57 (2017). https://doi.org/10.1007/s00245-017-9433-1
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00245-017-9433-1