Optimal Social Policies in Mean Field Games

Abstract

This paper analyzes problems in which a large benevolent player, controlling a set of policy variables, maximizes aggregate welfare in a continuous-time economy populated by atomistic agents subject to idiosyncratic shocks. We first provide as a benchmark the social optimum solution, in which a planner directly determines the individual controls. Then we analyze the optimal design of social policies depending on whether the large player may credibly commit to the future path of policies. On the one hand, we analyze the open-loop Stackelberg solution, in which the optimal policy path is set at time zero and the problem is time-inconsistent. On the other hand we analyze the time-consistent feedback Stackelberg solution.

1 Introduction

Many problems of interest in economics involve a major player, typically the Government or the Central Bank, choosing some aggregate policy instrument such as a tax or an interest rate in order to maximize some aggregate welfare criterion. Most of the existing models analyzing optimal policies drastically simplify the economy by assuming a “representative agent,” that is, they summarize the behavior of heterogeneous firms or households in a single individual that accounts for the mean of the distribution.¹ The few exceptions typically rely either on “ brute force” numerical methods, that is, parameterizing the time-path of the optimal policies and then running a numerical search to find the optimal nodes, or on some particular set of assumptions such that a closed-form analytical solution can be obtained.²

In this paper we analyze problems in which a large benevolent player, controlling a set of policy variables, maximizes an aggregate welfare criterion in a continuous-time economy populated by atomistic agents subject to idiosyncratic shocks. This can be seen as a particular case of the theory of mean field games (MFGs), introduced by [32, 33] and [27].³ The economy is described as an infinite-horizon mean field game with state constraints in which the aggregate distribution affects individual agents through the dynamics of some aggregate variables. This framework encompasses the standard notion of a dynamic competitive equilibrium in macroeconomics, in which individual agents choose their control variables to maximize their value functions given the path of some aggregate variables (typically prices) and simultaneously the value of these variables is set such that aggregate supply equals aggregate demand (i.e., markets clear).⁴ In continuous time, the system is composed by a Hamilton-Jacobi-Bellman (HJB) equation, which characterizes the individual problem in terms of the value function, a Kolmogorov forward (KF) or Fokker-Planck equation, which describes the dynamics of the cross-sectional distribution, and a number of market-clearing conditions based on the aggregation of individual variables. The individual agents may also face state constraints, so that the accessible state space is restricted to a subset of $\mathbb {R} ^{n}.$ This model is typically denoted as the “ incomplete-market model with idiosyncratic shocks,” as there is no aggregate uncertainty.

Before analyzing the optimal policies, we set as a benchmark the social optimum, defined as the allocation produced by a planner that maximizes aggregate welfare by directly determining the individual controls of each agent, under full information. The welfare criterion is summarized by a social welfare function, which aggregates the individual utility flows across time and states. We assume that the planner discounts future utility flows using the same discount factor as individual agents.⁵ This problem can be seen as a particular case of the mean field control problem analyzed in [7] or the control of McKean-Vlasov dynamics studied by [11, 13] and [14]. The problem can be solved using calculus techniques in infinite-dimensional Hilbert spaces.⁶ The necessary conditions can be characterized, as in the competitive equilibrium, by a forward-backward system of partial differential equations. The difference is that the individual value function is now replaced by the social value function, which describes the value that the planner assigns to each agent depending on her state. This social value function can be obtained from the planner’s HJB equation, which includes some Lagrange multipliers capturing the “ shadow price” of the market clearing conditions.

In order to analyze the optimal social policies we extend the competitive equilibrium model to include some aggregate policy variables controlled by a large benevolent agent, that we denote as ‘the leader,’ who maximizes the social welfare function. In contrast to the social optimum above, this is not a mean field control problem but a mean field game with a large (non-atomistic) player. In order to characterize this kind of games it is essential to understand whether the leader is able to make credible commitments about the future path of the policy variables. We consider two polar cases. On the one hand, we consider what economists typically define as the “ Ramsey problem,” which corresponds to the open-loop Stackelberg solution of the game.⁷ In this case the leader solves at time zero, given the initial state distribution, a maximization problem in which it takes into account the impact of its decisions on the individual agents’ value and control functions. The necessary conditions for optimality include a social value function similar to the one in the social optimum and a distribution of costates that keep track of the value of breaking the “ promises” made at time zero about the future path of aggregate policies. As originally discussed by [31] this problem is time-inconsistent. On the other hand, we analyze the feedback (Markov) Stackelberg solution, in which the leader cannot make credible commitments.⁸ This problem is time-consistent and can be seen as a setting in which the leader has only instantaneous advantage. The solution in this case is similar to the solution under commitment with the Lagrange multiplier associated to the individual HJB equation equal to zero. The intuition for this result is that in the feedback Stackelberg solution no credible promises can be made by the leader and thus the value of breaking them is zero.

Related Literature Since the original contribution of [34] a growing literature has emerged analyzing mean field control problems. In addition to the papers commented above, we should mention recent contributions by [29, 44,45,46, 49] and [25], among others. In economics, the problem has been analyzed in [17] in discrete time and in [37] and [42] in continuous time. The present paper reproduces the results in [42] analyzing the optimal allocation in a mean field game with state constraints in which the aggregate distribution affects individual agents only through some aggregate variables.

The literature analyzing mean field games with a non-atomistic (‘major’) player is less extensive. [28] and [39] introduced a linear-quadratic model with a major player whose influence does not fade away as the number of players tends to infinity. [41] generalized the model to the nonlinear case. In these early contributions the major player does not directly affect the dynamics of the atomistic players, only their cost functionals, and hence they are of little interest in most economic applications. [40] consider the more general case in which the major player directly affects the individual dynamics, but only in the context of linear-quadratic models. [8] analyze the general nonlinear case assuming a closed-loop Stackelberg game strategy in which the major player chooses her own control to minimize its expected cost taking into account the impact of this decision on the controls selected by the minor players. The solution is characterized by a set of stochastic partial differential equations. Carmona and Zhu [16] and Carmona and Wang [15], instead, consider a Nash game strategy using the probabilistic approach developed by [12]. Carmona and Wang [15], in particular, characterize the solution under open-loop, closed-loop and feedback controls. Our paper contributes to this literature in three main aspects. First, up to our knowledge this is the first paper to analyze both the open-loop and the feedback Stackelberg solutions in a model without aggregate uncertainty, characterizing these solutions as forward-backward systems of partial differential equations.⁹ Second, we consider a case in which the major player (‘the leader’) maximizes the aggregate welfare of the atomistic agents—instead of its own individual welfare—in a model with state constraints and aggregate variables. This provides a useful tool for the future analysis of optimal policies in economic problems. Third, by presenting together the results under competitive equilibrium, social optimum and optimal social policies under commitment and discretion this paper aims at providing a unified framework to compare the properties of the resulting forward-backward systems.

The structure of the paper is as follows. Section 2 introduces the competitive equilibrium in a MFG form. Section 3 analyzes the social optimum, following [42]. Section 4 builds on [43] to analyze the optimal policies under commitment and discretion, including necessary conditions for the open-loop and feedback Stackelberg solutions. Finally, Sect. 5 concludes. All the proofs are presented in the Appendix.

It is important to remark that the proofs in this paper should be considered as “ informal” or as “ sketches of a proof” at best, and that many important issues have been overlooked. We hope that this paper will open new avenues for future research in mean field game theory with important applications in economics.

2 Competitive Equilibrium

First we provide a general model of a “ competitive equilibrium,” as it is typically understood in economics. We consider a continuous-time infinite-horizon economy. Let $\left( \Sigma ,\mathcal {F} ,\left\{ \mathcal {F}_{t}\right\} ,\mathbb {P}\right)$ be a filtered probability space. There is a continuum of unit mass of ex-ante identical agents indexed by $i\in [0,1].$

2.1 Individual Problem

State First we analyze the problem of an individual agent. Let $W^{i}\left( t\right)$ be a n-dimensional $\mathcal {F} _{t}$-adapted Brownian motion and $X^{i}\left( t\right) \in \mathbb {R} ^{n}$ denote the state of the agent i at time $t\in [0,\infty ).$ The individual state evolves according to a multidimensional Itô process of the form

$$\begin{aligned} dX^{i}\left( t\right)= & {} b\left( X^{i}\left( t\right) ,u^{i}(t),Z\left( t\right) \right) dt+\sigma \left( X^{i}\left( t\right) \right) dW^{i}\left( t\right) , \\ X^{i}\left( 0\right)= & {} x_{0}^{i}, \nonumber \end{aligned}$$

(1)

where $u\in U\subset \mathbb {R} ^{m}$ is a m-dimensional vector of control variables and $Z\left( t\right) \in \mathbb {R} ^{p}$ is a deterministic p-dimensional vector of aggregate variables. The functional coefficients are defined as follows

$$\begin{aligned}&b : \mathbb {R} ^{n}\times U\times \mathbb {R} ^{p}\rightarrow \mathbb {R} ^{n}, \\&\sigma : \mathbb {R} ^{n}\rightarrow \mathbb {R} ^{n}, \\&Z :[0,\infty )\rightarrow \mathbb {R} ^{p}, \\&u :[0,\infty )\times \mathbb {R} ^{n}\rightarrow U. \end{aligned}$$

The measurable functions b and $\sigma$ satisfy a uniform Lipschitz condition in U :  $\exists K\ge 0,$ such that $\forall x,x^{\prime }\in \mathbb {R} ^{n},$ $\forall u,u^{\prime }\in U,$ $\forall Z,Z^{\prime }\in \mathbb {R} ^{p}$

$$\begin{aligned} \left| b\left( x,u,Z\right) -b\left( x^{\prime },u^{\prime },Z^{\prime }\right) \right|\le & {} K\left( \left| x-x^{\prime }\right| +\left| u-u^{\prime }\right| +\left| Z-Z^{\prime }\right| \right) , \\ \left| \sigma \left( x\right) -\sigma \left( x^{\prime }\right) \right|\le & {} K\left| x-x^{\prime }\right| . \end{aligned}$$

We assume that U is a closed subset of $\mathbb {R} ^{m}.$ Let $\mathcal {U}$ be the set of measurable controls taking values in U. We allow for state constraints in which the state $X\left( t\right)$ cannot leave the compact region $\Omega \subset \mathbb {R} ^{n}$, that is, control $u\left( \cdot \right)$ at a point $X\left( t\right) =x$ is an admissible control if $u\left( \cdot \right) \in \mathcal {U}\left( t,x\right) ,$ where¹⁰

$$\begin{aligned} \mathcal {U}\left( t,x\right) :=\left\{ u\left( \cdot \right) \in \mathcal {U} \text { such that }X\left( s\right) \in \Omega ,\text { }\forall s\ge t\text { with }X\left( t\right) =x\right\} . \end{aligned}$$

We also assume that $\sigma _{n}\left( x\right) =0$ if $x\in \partial \Omega _{n}$ that is, that the volatility in the nth dimension is zero if the $n-$th dimensional boundary is reached. From now on, we drop the superindex i as there is no possibility of confusion.

Utility Functional Each agent maximizes her utility functional

$$\begin{aligned} J\left( t,x,u\left( \cdot \right) \right) =\mathbb {E}\left[ \int _{t}^{\infty }e^{-\rho \left( s-t\right) }f(X\left( s\right) ,u\left( s\right) )ds\text { } |\text { }X\left( t\right) =x\right] , \end{aligned}$$

where the discount factor $\rho$ is a positive constant. The instantaneous utility function

$$\begin{aligned} f: \mathbb {R} ^{n}\times \mathbb {R} ^{m}\rightarrow \mathbb {R} , \end{aligned}$$

satisfies a polynomial growth condition: $\exists K,c>0,$ such that $\forall x\in \mathbb {R} ^{n},$ $\forall u\in U,$

$$\begin{aligned} \left| f\left( x,u\right) \right| \le K\left( 1+\left| x\right| ^{c}+\left| u\right| ^{c}\right) . \end{aligned}$$

The optimal value function V(t, x) is defined as

$$\begin{aligned} V(t,x)=\max _{u\left( \cdot \right) \in \mathcal {U}\left( t,x\right) }J\left( t,x,u\left( \cdot \right) \right) , \end{aligned}$$

(2)

subject to (1). The transversality condition is

$$\begin{aligned} \lim _{t\rightarrow \infty }e^{-\rho t}V(t,x)=0. \end{aligned}$$

(3)

Hamilton–Jacobi–Bellman (HJB) Equation The solution to this problem is given by a value function V(t, x) and a control strategy u(t, x) that satisfy the HJB equation

$$\begin{aligned} \rho V\left( t,x\right) =\frac{\partial V}{\partial t}+\max _{u\in U_{t,x}}\left\{ f(x,u)+\mathcal {A}_{u,Z}V\right\} , \end{aligned}$$

(4)

where $\mathcal {A}_{u,Z}$ is given by:

$$\begin{aligned} \mathcal {A}_{u,Z}V=\sum _{i=1}^{n}b_{i}\left( x,u,Z\right) \frac{\partial V}{ \partial x_{i}}+\sum _{i=1}^{n}\sum _{k=1}^{n}\frac{\left( \sigma (x)\sigma (x)^{\top }\right) _{i,k}}{2}\frac{\partial ^{2}V}{\partial x_{i}\partial x_{k}}. \end{aligned}$$

(5)

and $U_{t,x}$ is the subset of controls such that the corresponding vector field $b\left( \cdot \right)$ points inside the constraint, i.e.

$$\begin{aligned} U_{t,x}=\left\{ \begin{array}{ll} U, &{} \text {if }x\in \text {int}\left( \Omega \right) , \\ \left\{ u\in \mathcal {U}:b\left( x,u,Z\left( t\right) \right) \cdot \nu \left( x\right) <0\right\} &{} \text {if }x\in \partial \Omega , \end{array} \right. \end{aligned}$$

with $\nu \left( x\right)$ being the outward normal vector at $x\in \partial \Omega .$ ¹¹

2.2 Aggregate Distribution and Aggregate Variables

Kolmogorov Forward (KF) Equation Assume that the transition measure of $X\left( t\right)$ with initial value $x_{0}$ has a density $\mu (t,x;0,x_{0}),$ such that $\forall F\in L^{2}( \mathbb {R} ^{n}):$

$$\begin{aligned} \mathbb {E}_{0}\left[ F(X\left( t\right) )|X\left( 0\right) =x_{0}\right] =\int F(x)\mu (t,x;0,x_{0})dx. \end{aligned}$$

The initial distribution of X at time $t=0$ is $\mu (0,x)=\mu _{0}(x).$ The dynamics of the distribution of agents

$$\begin{aligned} \mu (t,x)=\int \mu (t,x;0,x_{0})\mu _{0}(x_{0})dx_{0} \end{aligned}$$

are given by the Kolmogorov Forward (KF) or Fokker-Planck equation

$$\begin{aligned} \frac{\partial \mu }{\partial t}= & {} \mathcal {A}_{u,Z}^{*}\mu , \end{aligned}$$

(6)

$$\begin{aligned} \int \mu (t,x)dx= & {} 1, \end{aligned}$$

(7)

where $\mathcal {A}_{u,Z}^{*}$ is the adjoint operator of $\mathcal {A}_{u,Z}:$

$$\begin{aligned} \mathcal {A}_{u,Z}^{*}\mu&=-\sum _{i=1}^{n}\frac{\partial }{\partial x_{i}} \left[ b_{i}\left( x,u,Z\right) \mu \left( t,x\right) \right] \nonumber \\&\quad +\frac{1}{2} \sum _{i=1}^{n}\sum _{k=1}^{n}\frac{\partial ^{2}}{\partial x_{i}\partial x_{k} }\left[ \left( \sigma (x)\sigma (x)^{\top }\right) _{i,k}\mu \left( t,x\right) \right] . \end{aligned}$$

Market Clearing Conditions The vector of aggregate variables is determined by a system of p equations:

$$\begin{aligned} Z_{k}(t)=\int g_{k}(x,u\left( t,x\right) )\mu (t,x)dx, \quad k=1,\ldots ,p, \end{aligned}$$

(8)

where

$$\begin{aligned} g_{k}: \mathbb {R} ^{n}\times U\rightarrow \mathbb {R} . \end{aligned}$$

These equations are typically the market clearing conditions of the economy.

We may define the competitive equilibrium of this economy.

Definition 1

(Competitive equilibrium) The competitive equilibrium is composed by the vector of aggregate variables $Z\left( t\right)$, the value function V(t, x), the control u(t, x) and the distribution $\mu (t,x)$ such that

• 1. Given $Z\left( t\right)$ and $\mu (t,x)$, V(t, x) is the solution of the HJB equation (4) and the optimal control is u(t, x).

• 2. Given u(t, x) and $Z\left( t\right)$, $\mu (t,x)$ is the solution of the KF equation (6, 7).

• 3. Given u(t, x) and $\mu (t,x),$ the aggregate variables $Z\left( t\right)$ satisfy the market clearing conditions (8).

Remark 1

It should be clear from this definition that a competitive equilibrium is just a particular instance of mean field game theory in which the aggregate distribution affects each individual agent only through the dynamics of the aggregate variables $Z\left( t\right) .$

3 The Social Optimum

Social Welfare Functional We study as a benchmark the allocation produced by a benevolent social planner who maximizes an aggregate welfare criterion, that is, instead of a decentralized problem with multiple decision makers we consider the case of a single decision-maker who controls each individual agent. This is a mean field control problem instead of a mean field game. The planner chooses the vector of control variables u(t, x) to be applied to every agent. The social welfare functional is

$$\begin{aligned} J^{opt}\left( \mu \left( 0,\cdot \right) ,u(\cdot )\right) =\int _{0}^{\infty }e^{-\rho t}\left[ \int \omega (t,x)f(x,u)\mu (t,x)dx\right] dt, \end{aligned}$$

(9)

where $\omega (t,x)$ are state-dependent Pareto weights. If $\omega (t,x)=1,$ for all t and x,  then we have a purely utilitarian social welfare function which gives the same weight to every agent.

The planner’s optimal value functional is

$$\begin{aligned} V^{opt}\left( \mu \left( 0,\cdot \right) \right) =\max _{u\left( \cdot \right) \in \mathcal {U}\left( t,x\right) }J^{opt}\left( \mu \left( 0,\cdot \right) ,u(\cdot )\right) , \end{aligned}$$

(10)

subject to the law of motion of the distribution (6, 7) and to the market clearing conditions (8).

Remark 2

Notice that the state variable at time t in this case is the infinite-dimensional density $\mu \left( t\right) .$

Remark 3

In the utilitarian case, the planner’s social welfare functional under a given control $\tilde{u}\left( t,x\right) \in \mathcal {U}\left( t,x\right)$ is equivalent to aggregating the individual value function under the same control across all agents at time zero:

$$\begin{aligned} \int V^{\tilde{u}}(0,x)\mu (0,x)dx= & {} \int \mathbb {E}\left[ \int _{0}^{\infty }e^{-\rho t}f(X\left( t\right) ,\tilde{u}\left( t\right) )dt|X\left( 0\right) =x\right] \mu (0,x)dx \\= & {} \int \left[ \int \int _{0}^{\infty }e^{-\rho t}f(\tilde{x},\tilde{u})\mu (t,\tilde{x};0,x)d\tilde{x}dt\right] \mu (0,x)dx \\= & {} \int _{0}^{\infty }e^{-\rho t}\int f(\tilde{x},\tilde{u})\left[ \int \mu (t,\tilde{x};0,x)\mu (0,x)dx\right] d\tilde{x}ds \\= & {} \int _{0}^{\infty }e^{-\rho t}\int f(\tilde{x},\tilde{u})\mu (t,\tilde{x})d \tilde{x}ds=J^{opt}\left( \mu \left( 0,\cdot \right) ,\tilde{u}(\cdot )\right) , \end{aligned}$$

where $V^{\tilde{u}}(t,x)\$is the individual value function under control $\tilde{u}$, characterized by the HJB

$$\begin{aligned} \rho V^{\tilde{u}}\left( t,x\right) =\frac{\partial V^{\tilde{u}}}{\partial t }+\left\{ f(x,\tilde{u})+\mathcal {A}_{\tilde{u},Z}V^{\tilde{u}}\right\} , \text { } \end{aligned}$$

and $\mu (t,\tilde{x};0,x)$ is the transition probability from $X\left( 0\right) =x$ to $X\left( t\right) =\tilde{x}$ and

$$\begin{aligned} \int \mu (t,\tilde{x};0,x)\mu (0,x)dx=\mu (t,\tilde{x}), \end{aligned}$$

is the Chapman–Kolmogorov equation.

We provide necessary conditions to the problem (10).

Proposition 1

(Necessary conditions - social optimum) If a solution to problem (10) exists with $e^{-\rho t}u$, $e^{-\rho t}\mu \in L^{2}\left( [0,\infty )\times \mathbb {R} ^{n}\right)$ and $e^{-\rho t}Z\in L^{2}[0,\infty )$, then the optimal value functional $V^{opt}\left( \mu \left( 0,\cdot \right) \right)$ can be expressed as

$$\begin{aligned} V^{opt}\left( \mu \left( 0,\cdot \right) \right) =\int \phi (0,x)\mu (0,x)dx, \end{aligned}$$

(11)

where $\phi \left( t,x\right)$ is the marginal social value function, which represents the social value of an agent at time t and state x. The social value function satisfies the planner’s HJB

$$\begin{aligned}&\displaystyle \rho \phi (t,x) =\frac{\partial \phi }{\partial t}+\max _{u\in U_{t,x}}\left\{ \omega (t,x)f(x,u)+\sum _{k=1}^{p}\lambda _{k}(t)\left[ g_{k}(x,u)-Z_{k}\left( t\right) \right] +\mathcal {A}_{u,Z}\phi \right\} \! ,\nonumber \\ \end{aligned}$$

(12)

$$\begin{aligned}&\displaystyle \lim _{T\rightarrow \infty }e^{-\rho T}\phi (T,x) =0 \end{aligned}$$

(13)

where the Lagrange multipliers $\lambda (t):=\left[ \lambda _{1}(t),\ldots ,\lambda _{k}(t),\ldots ,\lambda _{p}(t)\right] ^{\top },$ are given by

$$\begin{aligned} \lambda _{k}(t)=\int \sum _{i=1}^{n}\frac{\partial \phi }{\partial x_{i}} \frac{\partial b_{i}}{\partial Z_{k}}\mu (t,x)dx. \end{aligned}$$

(14)

The social optimum of this economy is defined in a similar fashion as in the case of a competitive equilibrium above.

Remark 4

The social optimum is composed by the vector of aggregate variables $Z\left( t\right)$, the social value function $\phi (t,x)$, the control u(t, x),  the Lagrange multipliers $\lambda (t)$ and the distribution $\mu (t,x)$ such that

• 1. Given $Z\left( t\right) ,$ $\lambda (t)$ and $\mu (t,x)$, $\phi (t,x)$ is the solution of the planner’s HJB equation (12) and the optimal control is u(t, x).

• 2. Given u(t, x) and $Z\left( t\right)$, $\mu (t,x)$ is the solution of the KF equation (6, 7).

• 3. Given u(t, x) and $\mu (t,x),$ the aggregate variables $Z\left( t\right)$ satisfy the market clearing conditions (8).

• 4. Given u(t, x), $Z\left( t\right)$ and $\mu (t,x),$ the Lagrange multipliers $\lambda (t)$ satisfy (14).

Remark 5

The Lagrange multipliers $\lambda (t)$ reflect the ‘shadow prices’ of the market clearing condition (8). They price, in utility terms, the deviation of an agent from the value of the aggregate variable: $g_{k}(x,u)-Z_{k}.$

Corollary 1

If the competitive equilibrium allocation satisfies

$$\begin{aligned} \int \sum _{i=1}^{n}\frac{\partial V}{\partial x_{i}}\frac{\partial b_{i}}{ \partial Z_{k}}\mu (t,x)dx=0, \end{aligned}$$

(15)

then the competitive equilibrium and the utilitarian optimal allocation $\left( \omega =1\right)$ coincide:

$$\begin{aligned} \lambda _{k}(t)=0, \quad k=1,\ldots ,p \end{aligned}$$

and

$$\begin{aligned} \phi (t,x)=V(t,x). \end{aligned}$$

4 Optimal Social Policies

4.1 General Setting

Aggregate Policy Variables Consider again the decentralized competitive equilibrium and assume that the state of each individual agent is now given by

$$\begin{aligned} dX\left( t\right) =b\left( X\left( t\right) ,u(t),Z\left( t\right) ,Y\left( t\right) \right) dt+\sigma \left( X\left( t\right) \right) dW\left( t\right) , \end{aligned}$$

(16)

where $Y\left( t\right) \in \mathbb {R} ^{q}$ is a q-dimensional vector of aggregate policy variables:

$$\begin{aligned} Y:[0,\infty )\rightarrow \mathbb {R} ^{q}, \end{aligned}$$

and b satisfy a uniform Lipschitz condition.¹² These policy variables are chosen by a large agent, which we denote as ‘the leader.’ The leader maximizes a social welfare function

$$\begin{aligned} J^{lead}\left( t,\mu (t,\cdot ),Y\left( \cdot \right) \right) =\int _{t}^{\infty }e^{-\rho \left( s-t\right) }\left[ \int \omega (s,x)f(x,u)\mu (s,x)dx\right] ds, \end{aligned}$$

(17)

similar to the one in the previous section.

Remark 6

The difference between this problem and the social optimum is that, instead of a mean field control case, here we are analyzing a mean field game including a large non-atomistic agent (the leader).

Equilibrium Concepts We consider two alternative equilibrium concepts, which depend on the ability of the leader to make credible commitments about future policies.

• 1. Commitment In the first case, we assume that at time zero the leader is able to credibly commit to the complete future path of policies $\left\{ Y\left( t\right) \right\} _{t=0}^{\infty }.$ This corresponds to the open-loop Stackelberg equilibrium of the game, with

  $$\begin{aligned} Y\left( t\right) =\Upsilon ^{C}\left( t,\text { }\mu (0,\cdot )\right) , \end{aligned}$$

  where $\Upsilon ^{C}$ is a deterministic measurable function of calendar time and the initial distribution. This is equivalent to say that, given the initial distribution $\mu \left( 0,\cdot \right) ,$ the leader announces at time $t=0$ the complete future evolution of the aggregate policy variables $\left\{ Y\left( t\right) \right\} _{t=0}^{\infty }$ and commits not to reevaluate this initial plan. When formulating the optimal plan, the leader takes into account the impact of its aggregate policies on each atomistic agent’s optimal controls. Given the leader’s policy path, individual agents maximize their individual value functions (2). The result is a vector optimal individual controls $u\left( t,x;\left\{ Y\left( s\right) \right\} _{s=0}^{\infty }\right)$ which depends on the complete path of the leader policy variables.

• 2. Discretion In the second case, no commitment device is available. This corresponds to the feedback Stackelberg equilibrium of the game, with

  $$\begin{aligned} Y\left( t\right) =\Upsilon ^{D}\left( t,\text { }\mu (t,\cdot )\right) , \end{aligned}$$

  where $\Upsilon ^{D}$ is a deterministic progressively measurable function of the current state distribution. In this case the aggregate policies are time-consistent. This problem can be seen as the limit as $\Delta \rightarrow 0$ of a sequence of open-loop Stackelberg problems of length $\Delta$ in which the initial state at each stage n is given by the distribution at the beginning of the stage $\mu (t_{n},\cdot ).$

4.2 Commitment

First we consider the solution under commitment, which in economics is typically denoted as the ‘Ramsey problem’ and which corresponds to the open-loop Stackelberg solution of this game.

Definition 2

(Commitment) The problem of the leader under commitment is to choose the complete path of policies $\left\{ Y\left( t\right) \right\} _{t=0}^{\infty }$ at time zero in order to maximize the aggregate welfare (17) when the aggregate distribution $\mu (t,x),$ aggregate variables $Z\left( t\right)$ and individual value function $V(t,x)\$and controls u(t, x) constitute a competitive equilibrium given $\left\{ Y\left( t\right) \right\} _{t=0}^{\infty }$. Formally, this amounts to

$$\begin{aligned} \max _{\left\{ Y\left( t\right) \right\} _{t\in [0,\infty )}}J^{lead}\left( 0,\mu \left( 0,\cdot \right) ,Y\left( \cdot \right) \right) , \end{aligned}$$

(18)

subject to law of motion of the distribution (6, 7), to the market clearing conditions (8) and to the individual HJB equation (4).

The solution is given by the following proposition.

Proposition 2

(Necessary conditions—Commitment) If a solution to problem (18) exists with $e^{-\rho t}u$, $e^{-\rho t}\mu ,$ $e^{-\rho t}V\in L^{2}\left( [0,\infty )\times \mathbb {R} ^{n}\right)$ and $e^{-\rho t}Z,e^{-\rho t}Y\in L^{2}[0,\infty )$, it should satisfy the system of equations

$$\begin{aligned}&\int \left\{ \theta (t,x)\sum _{i=1}^{n}\frac{\partial b_{i}}{\partial Y_{r}} \frac{\partial V}{\partial x_{i}}+\sum _{j=1}^{m}\sum _{i=1}^{n}\eta _{j}(t,x) \frac{\partial ^{2}b_{i}}{\partial Y_{r}\partial u_{j}}\frac{\partial V}{ \partial x_{i}}+\mu (t,x)\sum _{i=1}^{n}\frac{\partial b_{i}}{\partial Y_{r}} \frac{\partial \phi }{\partial x_{i}}\right\} dx=0,\nonumber \\&\quad \text { }r=1,\ldots ,q, \end{aligned}$$

(19)

where $\phi (t,x)$ is the marginal social value function, given by

$$\begin{aligned}&\displaystyle \rho \phi (t,x) =\frac{\partial \phi }{\partial t}+\omega (t,x)f(x,u)+\sum _{k=1}^{p}\lambda _{k}(t)\left( g_{k}(x,u)-Z_{k}\left( t\right) \right) +\mathcal {A}_{u,Z,Y}\phi , \nonumber \\\end{aligned}$$

(20)

$$\begin{aligned}&\displaystyle \lim _{T\rightarrow \infty }e^{-\rho T}\phi (T,x) =0. \end{aligned}$$

(21)

The Lagrange multipliers associated to the market clearing condition (8)

$$\begin{aligned} \lambda (t):=\left[ \lambda _{1}(t),\ldots ,\lambda _{k}(t),\ldots ,\lambda _{p}(t) \right] ^{\top } \end{aligned}$$

satisfy, $k=1,\ldots ,p:$

$$\begin{aligned} \lambda _{k}(t)= & {} \int \left\{ \theta (t,x)\sum _{i=1}^{n}\frac{\partial b_{i}}{ \partial Z_{k}}\frac{\partial V}{\partial x_{i}}+\sum _{j=1}^{m} \sum _{i=1}^{n}\eta _{j}(t,x)\frac{\partial ^{2}b_{i}}{\partial Z_{k}\partial u_{j}}\frac{\partial V}{\partial x_{i}}\right. \nonumber \\&\quad \quad \quad \quad +\left. \mu (t,x)\sum _{i=1}^{n}\frac{ \partial b_{i}}{\partial Z_{k}}\frac{\partial \phi }{\partial x_{i}}\right\} dx. \end{aligned}$$

(22)

The distribution of Lagrange multipliers $\theta \left( t,x\right)$ associated to the individual HJB equation follows

$$\begin{aligned}&\displaystyle \frac{\partial \theta }{\partial t} =\mathcal {A}_{u,Z,Y}^{*}\theta -\sum _{i=1}^{n}\sum _{j=1}^{m}\frac{\partial }{\partial x_{i}}\left( \eta _{j}(t,x)\frac{\partial b_{i}}{\partial u_{j}}\right) ,\nonumber \\&\displaystyle \theta \left( 0,\cdot \right) =0, \end{aligned}$$

(23)

and the Lagrange multipliers associated to the individual first-order conditions

$$\begin{aligned} \eta \left( t,x\right) :=\left[ \eta _{1}\left( t,x\right) ,\ldots ,\eta _{k}\left( t,x\right) ,\ldots ,\eta _{m}\left( t,x\right) \right] ^{\top } \end{aligned}$$

satisfy, $j=1,\ldots ,m:$

$$\begin{aligned}&\left( \omega (t,x)\frac{\partial f}{\partial u_{j}}+\sum _{i=1}^{n}\frac{ \partial b_{i}}{\partial u_{j}}\frac{\partial \phi }{\partial x_{i}} +\sum _{k=1}^{p}\lambda _{k}\frac{\partial g_{k}}{\partial u_{j}}\right) \mu \left( t,x\right) \nonumber \\&\quad +\sum _{k=1}^{m}\eta _{k}\left( t,x\right) \left( \frac{ \partial ^{2}f}{\partial u_{j}\partial u_{k}}+\sum _{i=1}^{n}\frac{\partial ^{2}b_{i}}{\partial u_{j}\partial u_{k}}\frac{\partial V}{\partial x_{i}} \right) =0. \end{aligned}$$

(24)

Remark 7

The equilibrium under commitment is composed by the competitive equilibrium equations described in Definition 1 plus the necessary conditions of the leader (19)–(24).

Remark 8

Notice that the problem in the case with $m=1,$ $\omega \left( \cdot \right) =1,\ f$ strictly concave and $\frac{\partial ^{2}b_{i}}{\partial u_{j}\partial u_{k}}=0$ for $j=1,\ldots ,m,$ $k=1,\ldots ,p,\$if the solution is such that $\lambda _{k}\left( \cdot \right) =0,$ $k=1,\ldots ,p,$ then the other Lagrange multipliers are zero: $\theta \left( \cdot \right) =\eta \left( \cdot \right) =0$ and the social value function coincides with the individual one, $\phi (t,x)=V(t,x).$ The optimal aggregate policy $Y\left( t\right)$ is such that

$$\begin{aligned} \int \sum _{i=1}^{n}\mu (t,x)\frac{\partial b_{i}}{\partial Y_{r}}\frac{ \partial \phi }{\partial x_{i}}dx=0,\text { }r=1,\ldots ,q. \end{aligned}$$

4.3 Discretion

Next we consider the case without commitment or feedback Stackelberg equilibrium of the game. We first define a finite-horizon commitment problem, in the same lines as Definition 2.

Definition 3

(Commitment - finite horizon) Given an initial density $\mu (t,x),$ the problem of the leader under commitment in an interval $[t,t+\Delta ]$ with a terminal value functional $W\left( \cdot \right) ,$ is to choose the complete path of policies $\left\{ Y^{\Delta }\left( s\right) \right\} _{s\in [t,t+\Delta ]}$ at time t in order to maximize the aggregate welfare (17) when the aggregate distribution $\mu (s,x),$ aggregate variables $Z\left( s\right)$ and individual value function $V(s,x)\$and controls u(s, x) constitute a competitive equilibrium given $\left\{ Y^{\Delta }\left( s\right) \right\} _{s\in [t,t+\Delta ]}$. Formally, this amounts to

$$\begin{aligned} \max _{\left\{ Y^{\Delta }\left( s\right) \right\} _{s\in [t,t+\Delta ]}}\int _{t}^{t+\Delta }e^{-\rho \left( s-t\right) }\left[ \int \omega (s,x)f(x,u)\mu (s,x)dx\right] ds+e^{-\rho \Delta }W\left( \mu \left( t+\Delta ,\cdot \right) \right) \end{aligned}$$

(25)

subject to law of motion of the distribution (6, 7), to the market clearing conditions (8) and to the individual HJB equation (4).The terminal indvidual value function $v\left( t+\Delta ,\cdot \right)$ is also taken as given.

Given $T>0,$ we assume that the interval [0, T] is divided in N intervals of length $\Delta :=T/N.$

Definition 4

(Discretion) An equilibrium under discretion in a finite interval [0, T] with a terminal value functional $W^{T}\left( \cdot \right)$ is defined as the limit as $N\rightarrow \infty ,$ or equivalently $\Delta \rightarrow 0,$ of a sequence of functions$\ Y^{\Delta }\left( t\right)$ given by the finite-horizon commitment problem introduced in Definition 3 over the intervals $[t,t+\Delta ]$ where $t=n\Delta ,$ $n=0,\ldots ,N-1$ and the terminal value of an interval n is defined as the value functional of the next interval:

$$\begin{aligned}&W^{n}\left( \mu \left( n\Delta ,\cdot \right) \right) \nonumber \\&\quad =\max _{\left\{ Y^{\Delta }\left( s\right) \right\} _{s\in [n\Delta ,(n+1)\Delta ]}}\int _{n\Delta }^{\left( n+1\right) \Delta }e^{-\rho \left( s-t\right) } \left[ \int \omega (s,x)f(x,u)\mu (s,x)dx\right] ds \quad \quad \quad \end{aligned}$$

(26)

$$\begin{aligned}&\qquad +\,e^{-\rho \Delta }W^{n+1}\left( \mu \left( (n+1)\Delta ,\cdot \right) \right) , \end{aligned}$$

(27)

with $W^{N}\left( \cdot \right) =W^{T}\left( \cdot \right) .$ The infinite-horizon case is defined as the limit as $T\rightarrow \infty$ with a transversality condition

$$\begin{aligned} \lim _{T\rightarrow \infty }e^{-\rho T}W^{T}\left( \cdot \right) =0. \end{aligned}$$

The solution is given by the following proposition.

Proposition 3

(Necessary conditions—Discretion) If a solution to problem under discretion exists, it should satisfy the system of equations

$$\begin{aligned} \int \left\{ \sum _{j=1}^{m}\sum _{i=1}^{n}\eta _{j}(t,x)\frac{\partial ^{2}b_{i}}{\partial Y_{r}\partial u_{j}}\frac{\partial V}{\partial x_{i}} +\sum _{i=1}^{n}\frac{\partial \phi }{\partial x_{i}}\frac{\partial b_{i}}{ \partial Y_{r}}\mu (t,x)\right\} dx=0, \end{aligned}$$

(28)

$r=1,\ldots ,q,$ where $\phi (t,x)$ is the marginal social value function, given by

$$\begin{aligned}&\displaystyle \rho \phi (t,x) =\frac{\partial \phi }{\partial t}+\omega (t,x)f(x,u)+\sum _{k=1}^{p}\lambda _{k}(t)\left( g_{k}(x,u)-Z_{k}\left( t\right) \right) +\mathcal {A}_{u,Z,Y}\phi , \quad \quad \quad \! \end{aligned}$$

(29)

$$\begin{aligned}&\displaystyle \lim _{T\rightarrow \infty }e^{-\rho T}\phi (T,x) =0, \end{aligned}$$

(30)

the Lagrange multipliers associated to the market clearing condition (8), $\lambda _{k}(t),$ $k=1,\ldots ,p,$ satisfy

$$\begin{aligned} \lambda _{k}(t)=\int \left\{ \sum _{j=1}^{m}\sum _{i=1}^{n}\eta _{j}(t,x)\frac{ \partial ^{2}b_{i}}{\partial Z_{k}\partial u_{j}}\frac{\partial V}{\partial x_{i}}+\sum _{i=1}^{n}\frac{\partial \phi }{\partial x_{i}}\frac{\partial b_{i}}{\partial Z_{k}}\mu (t,x)\right\} dx, \end{aligned}$$

(31)

and the Lagrange multipliers associated to the individual first-order conditions

$$\begin{aligned} \eta \left( t,x\right) :=\left[ \eta _{1}\left( t,x\right) ,\ldots ,\eta _{k}\left( t,x\right) ,\ldots ,\eta _{m}\left( t,x\right) \right] ^{\top } \end{aligned}$$

satisfy, $j=1,\ldots ,m:$

$$\begin{aligned}&\left( \omega (t,x)\frac{\partial f}{\partial u_{j}}+\sum _{i=1}^{n}\frac{ \partial b_{i}}{\partial u_{j}}\frac{\partial \phi }{\partial x_{i}} +\sum _{k=1}^{p}\lambda _{k}\frac{\partial g_{k}}{\partial u_{j}}\right) \mu \left( t,x\right) \nonumber \\&\quad \quad +\sum _{k=1}^{m}\eta _{k}\left( t,x\right) \left( \frac{ \partial ^{2}f}{\partial u_{j}\partial u_{k}}+\sum _{i=1}^{n}\frac{\partial ^{2}b_{i}}{\partial u_{j}\partial u_{k}}\frac{\partial V}{\partial x_{i}} \right) =0. \end{aligned}$$

(32)

Remark 9

The equilibrium under discretion is composed by the competitive equilibrium equations described in Definition 1 plus the necessary conditions of the leader (28)–(32).

Remark 10

Equations (28)–(32) coincide with the equivalent equations in the case of commitment with the Lagrange multipliers $\theta \left( \cdot \right) =0.$ Lagrange multipliers $\theta$ can be interpreted as the value to the leader of breaking the “promises” that the leader is making to individual agents. Under discretion, no promises can be made and thus these multipliers are zero.

5 Conclusions

This paper has analyzed the design of optimal social policies in an economy composed by a continuum of atomistic players subject to idiosyncratic shocks. The optimality of the policies is defined according to a social welfare function that aggregates, given some state-dependent Pareto weights, the individual utilities across agents. First, we consider two alternative benchmarks without social policies. On the one hand, the decentralized competitive equilibrium is defined as mean field game with aggregate variables and state constraints. On the other hand, the social optimum is a mean field control problem in which a planner chooses the individual policies in order to maximize aggregate welfare. Next we assume that a (non-atomistic) leader controls a vector of aggregate policies. This is a mean field game with a large player. We analyze two different equilibrium concepts. In the open-loop Stackelberg solution of the game the large player is able to make a credible commitment about the future path of the aggregate policy variables. In the feedback Stackelberg solution no such a commitment is possible and the policies are time-consistent. We characterize the necessary conditions, but we do not analyze important issues such as the existence or uniqueness of the solutions, which we leave for future research.

The main analytical tool employed in this paper is the Lagrange multiplier method in infinite-dimensional Hilbert spaces. An interesting question would be to analyze to what extent these results can also be obtained by means of the Pontryagin principle.

Finally, neither have we discussed the numerical implementation of the solution in the cases in which no analytical results are available. Nuño and Moll [42] and Nuño and Thomas [43] provide some insights on this respect extending previous work by [1, 2] and [3]. Due to the relevance of the potential applications, we are sure that this will be a fruitful field of research in the coming years.

Appendix

1.1 Proof of Proposition 1: Necessary Conditions in the Social Optimum

The problem of the planner is to maximize $J^{opt}\left( u\left( \cdot \right) \right)$ subject to the KF equation (6) and the market clearing conditions (8). The latter can be expressed as

$$\begin{aligned} \int \left( g_{k}(x,u)-Z_{k}(t)\right) \mu (t,x)dx, \quad k=1,\ldots ,p,\text { }\forall t\in [0,\infty ). \end{aligned}$$

(33)

We define the domain $\Phi :=[0,\infty )\times \mathbb {R} ^{n}.$ The problem of the planner can be expressed as an optimization problem in a suitable functional space such as

$$\begin{aligned} \tilde{L}^{2}\left( \Phi \right) :=\left\{ f:\Phi \rightarrow \mathbb {R} \text { such that }\left\| e^{-\rho t}f\right\| _{L^{2}\left( \Phi \right) }<\infty \right\} . \end{aligned}$$

Nuño and Moll [42] show how $\tilde{L}^{2}\left( \Phi \right)$ is a Hilbert space with the inner product

$$\begin{aligned} \left( f,g\right) _{\Phi }:=\left\langle e^{-\rho t}f,g\right\rangle _{\Phi },\text { for all }f,g\in \tilde{L}^{2}\left( \Phi \right) , \end{aligned}$$

where $\left\langle \cdot ,\cdot \right\rangle _{\Phi }$ is the standard inner product in $L^{2}\left( \Phi \right) :$

$$\begin{aligned} \left\langle f,\mu \right\rangle _{\Phi }=\int _{\Phi }f\mu dx, \quad \forall f,\mu \in L^{2}\left( \Phi \right) . \end{aligned}$$

The idea is to construct a Lagragian including the KF equation (6) and the market clearing conditions (8) and to optimize with respect to the individual control $u\left( \cdot \right)$ and the aggregate variables $Z\left( \cdot \right) .$

The Lagrangian functional results in

$$\begin{aligned} \mathcal {L}\left( \mu ,u_{1},\ldots ,u_{m},Z_{1},\ldots ,Z_{p}\right)= & {} \left\langle e^{-\rho t}\omega f,\mu \right\rangle _{\Phi }+\left\langle e^{-\rho t}\phi ,-\frac{\partial \mu }{\partial t}+\mathcal {A}_{u,Z}^{*}\mu \right\rangle _{\Phi } \nonumber \\&+\sum _{k=1}^{p}\left\langle e^{-\rho t}\lambda _{k},\left( g_{k}-Z_{k}\right) \mu \right\rangle _{\Phi }, \end{aligned}$$

(34)

where $e^{-\rho t}\phi (t,x)\in L^{2}\left( \Phi \right)$ and $e^{-\rho t}\lambda _{k}(t)\in L^{2}[0,\infty ),$ $k=1,\ldots ,p$ are the Lagrange multipliers associated to the KF equation (6) and market clearing conditions (8), respectively.

If $\mathcal {L}$ has continuous Fréchet derivatives, a necessary condition for $\left( \mu ,u_{1},\ldots ,u_{m},Z_{1},\ldots ,Z_{p}\right)$ to be a maximum of (34) is that the Gateaux derivatives with respect to each of these functions equals zero.¹³

It will prove useful to modify the second term in the Lagrangian

$$\begin{aligned}&\left\langle e^{-\rho t}\phi ,-\frac{\partial \mu }{\partial t}+\mathcal {A} _{u,Z}^{*}\mu \right\rangle _{\Phi }\nonumber \\&\quad =-\int _{_{0}}^{\infty }\int e^{-\rho t}\phi \left( t,x\right) \frac{\partial \mu }{\partial t} dxdt+\left\langle e^{-\rho t}\phi ,\mathcal {A}_{u,Z}^{*}\mu \right\rangle _{\Phi } \nonumber \\&\quad =-\int \left. e^{-\rho t}\phi \left( t,x\right) \mu \left( t,x\right) \right| _{0}^{\infty }dx+\int _{_{0}}^{\infty }\int e^{-\rho t}\left( \frac{\partial \phi }{\partial t}-\rho \phi \left( t,x\right) \right) \mu dtdx \nonumber \\&\quad \quad +\left\langle e^{-\rho t}\mathcal {A}_{u,Z}\phi ,\mu \right\rangle _{\Phi } \nonumber \\&\quad =-\lim _{T\rightarrow \infty }\int e^{-\rho T}\phi \left( T,x\right) \mu \left( T,x\right) dx+\int \phi \left( 0,x\right) \mu \left( 0,x\right) dx \nonumber \\&\quad \quad +\left\langle e^{-\rho t}\left( \frac{\partial \phi }{\partial t}-\rho \phi +\mathcal {A}_{u,Z}\phi \right) ,\mu \right\rangle _{\Phi }, \end{aligned}$$

(35)

where we have integrated by parts with respect to time in the term $\frac{ \partial \mu }{\partial t}$ and applied the fact that $\mathcal {A} _{u,Z}^{*}$ is the adjoint operator of $\mathcal {A}_{u,Z}$ in $L^{2}\left( \mathbb {R} ^{n}\right) \subset \tilde{L}^{2}\left( \Phi \right) .$

The Gateaux derivative with respect to $\mu$ is

$$\begin{aligned}&\left. \frac{d}{d\alpha }\mathcal {L}\left( \mu +\alpha h,u_{1},\ldots ,u_{m},Z_{1},\ldots ,Z_{p}\right) \right| _{\alpha =0}\\= & {} \left. \frac{d}{d\alpha }\left\langle e^{-\rho t}\omega f,\mu +\alpha h\right\rangle _{\Phi }\right| _{\alpha =0} \\&+\left. \frac{d}{d\alpha }\left\langle e^{-\rho t}\left( \frac{\partial \phi }{\partial t}-\rho \phi +\mathcal {A}_{u,Z}\phi \right) ,\mu +\alpha h\right\rangle _{\Phi }\right| _{\alpha =0} \\&+\left. \frac{d}{d\alpha }\sum _{k=1}^{p}\left\langle e^{-\rho t}\lambda _{k},\left( g_{k}-Z_{k}\right) \left( \mu +\alpha h\right) \right\rangle _{\Phi }\right| _{\alpha =0} \\&-\left. \frac{d}{d\alpha }\lim _{T\rightarrow \infty }\int e^{-\rho T}\phi \left( T,x\right) \left( \mu \left( T,x\right) +\alpha h\left( T,x\right) \right) dx\right| _{\alpha =0} \\= & {} \left\langle e^{-\rho t}\omega f,h\right\rangle _{\Phi }+\left\langle e^{-\rho t}\left( \frac{\partial \phi }{\partial t}-\rho \phi +\mathcal {A} _{u,Z}\phi \right) ,h\right\rangle _{\Phi } \\&+\sum _{k=1}^{p}\left\langle e^{-\rho t}\lambda _{k},\left( g_{k}-Z_{k}\right) h\right\rangle _{\Phi }-\lim _{T\rightarrow \infty }\int e^{-\rho T}\phi \left( T,x\right) h\left( T,x\right) dx, \end{aligned}$$

and it equals zero in the maximum for any function $h(t,x)\in \tilde{L} ^{2}\left( \Phi \right)$. The term $\int \phi \left( 0,x\right) \mu \left( 0,x\right) dx$ can be ignored in the optimization as $\mu \left( 0,x\right) =\mu _{0}(x)$, that is, the initial distribution is given and thus $h\left( 0,x\right) =0$ for all $x\in \mathbb {R} ^{n}$. We obtain

$$\begin{aligned}&\displaystyle \frac{\partial \phi }{\partial t}+\omega f+\sum _{k=1}^{p}\lambda _{k}\left( g_{k}-Z_{k}\right) +\mathcal {A}_{u,Z}\phi =\rho \phi ,\text { }\forall \left( t,x\right) \in \Phi , \end{aligned}$$

(36)

$$\begin{aligned}&\displaystyle \lim _{T\rightarrow \infty }e^{-\rho T}\phi \left( T,x\right) =0,\text { } \forall x\in \mathbb {R} ^{n}, \end{aligned}$$

(37)

which is the HJB equation of the planner (12).

The Gateaux derivative with respect to the control $u_{j}$ is

$$\begin{aligned}&\left. \frac{d}{d\alpha }\mathcal {L}\left( \mu ,u_{1},\ldots ,u_{j}+\alpha h,..u_{m},Z_{1},\ldots ,Z_{p}\right) \right| _{\alpha =0}\nonumber \\= & {} \left. \frac{d}{ d\alpha }\left\langle e^{-\rho t}\omega f\left( x,u_{j}+\alpha h\right) ,\mu \right\rangle _{\Phi }\right| _{\alpha =0} \nonumber \\&+\left. \frac{d}{d\alpha }\left\langle e^{-\rho t}\left( \frac{\partial \phi }{\partial t}-\rho \phi +\mathcal {A}_{u_{j}+\alpha h,Z}\phi \right) ,\mu \right\rangle _{\Phi }\right| _{\alpha =0} \nonumber \\&+\left. \frac{d}{d\alpha }\sum _{k=1}^{p}\left\langle e^{-\rho t}\lambda _{k},\left( g_{k}\left( x,u_{j}+\alpha h\right) -Z_{k}\right) \mu \right\rangle _{\Phi }\right| _{\alpha =0}, \end{aligned}$$

(38)

where $\mathcal {A}_{u_{j}+\alpha h,Z}:=\mathcal {A}_{u_{1},\ldots ,u_{j}+\alpha h,\ldots ,u_{m},Z}$. Given the state constraint $u\in \mathcal {U}(t,x)$ and the optimality condition that (38) equals zero in the maximum for any $h(t,x)\in \tilde{L}^{2}\left( [0,\infty )\times \Omega \right)$ then

$$\begin{aligned} u=\arg \max _{\tilde{u}\in U_{t,x}}\left\{ \omega f\left( x,\tilde{u}\right) +\sum _{k=1}^{p}\lambda _{k}g_{k}\left( x,\tilde{u}\right) +\mathcal {A}_{ \tilde{u},Z}\phi \right\} . \end{aligned}$$

(39)

The Gateaux derivative with respect to the aggregate variable $Z_{k}$ is

$$\begin{aligned}&\left. \frac{d}{d\alpha }\mathcal {L}\left( \mu ,u_{1},\ldots ,u_{m},Z_{1},..Z_{k}+\alpha h,..Z_{p}\right) \right| _{\alpha =0} \\= & {} \left. \frac{d}{d\alpha }\left\langle e^{-\rho t}\phi ,\left( -\frac{ \partial \mu }{\partial t}+\mathcal {A}_{u,Z_{k}+\alpha h}^{*}\mu \right) \right\rangle _{\Phi }\right| _{\alpha =0} \\&+\left. \frac{d}{d\alpha }\sum _{k=1}^{p}\left\langle e^{-\rho t}\lambda _{k},\left( g_{k}-\left( Z_{k}+\alpha h\right) \right) \mu \right\rangle _{\Phi }\right| _{\alpha =0}, \end{aligned}$$

and it equals zero in the maximum for any $e^{-\rho t}h(t)\in L^{2}[0,\infty ).$ Here $\mathcal {A}_{u,Z_{k}+\alpha h}^{*}:=\mathcal {A} _{u,Z_{1},\ldots ,Z_{k}+\alpha h,..Z_{p}}^{*}.$ This can be expressed as

$$\begin{aligned}&\lim _{\alpha \rightarrow 0}\int _{_{0}}^{\infty }\int e^{-\rho t}\phi (t,x) \frac{d}{d\alpha }\left\{ -\sum _{i=1}^{n}\frac{\partial }{\partial x_{i}} \left[ b_{i}\left( x,u,Z_{1},\ldots ,Z_{k}+\alpha h,\ldots ,Z_{p}\right) \mu \left( t,x\right) \right] \right. \nonumber \\&\quad -\left. \sum _{k=1}^{p}\lambda _{k}\left( Z_{k}+\alpha h\right) \mu \right\} dxdt, \end{aligned}$$

and hence

$$\begin{aligned}&\int _{_{0}}^{\infty }e^{-\rho t}h(t)\left\{ \int \phi (t,x)\left( \sum _{i=1}^{n}\left[ \frac{\partial ^{2}b_{i}}{\partial Z_{k}\partial x_{i}} \mu (t,x)\right. \right. \right. \nonumber \\&\quad \left. \left. +\left. \sum _{j=1}^{m}\frac{\partial ^{2}b_{i}}{\partial Z_{k}\partial u_{j}}\frac{\partial u_{j}}{\partial x_{i}}\mu +\frac{\partial b_{i}}{ \partial Z_{k}}\frac{\partial \mu }{\partial x_{i}}\right] \right) dx+\lambda _{k}(t)\right\} dt=0. \end{aligned}$$

As this is satisfied for any h(t),  we obtain that

$$\begin{aligned} \lambda _{k}(t)= & {} -\int \phi (t,x)\left\{ \sum _{i=1}^{n}\left[ \frac{ \partial ^{2}b_{i}}{\partial Z_{k}\partial x_{i}}\mu (t,x)+\sum _{j=1}^{m} \frac{\partial ^{2}b_{i}}{\partial Z_{k}\partial u_{j}}\frac{\partial u_{j}}{ \partial x_{i}}\mu (t,x)+\frac{\partial b_{i}}{\partial Z_{k}}\frac{\partial \mu }{\partial x_{i}}\right] \right\} dx \nonumber \\= & {} \int \sum _{i=1}^{n}\frac{\partial \phi }{\partial x_{i}}\frac{\partial b_{i}}{\partial Z_{k}}\mu (t,x)dx, \end{aligned}$$

(40)

where we have integrated by parts.

Finally, if we multiply by $e^{-\rho t}\mu \left( t,x\right)$ and integrate at both sides of the planner’s HJB equation (36)

$$\begin{aligned}&\int _{_{0}}^{\infty }\int e^{-\rho t}\left( \frac{\partial \phi }{\partial t} +\omega f+\sum _{k=1}^{p}\lambda _{k}\left( g_{k}-Z_{k}\right) +\mathcal {A} _{u,Z}\phi \right) \mu dxdt \\&\quad =\int _{_{0}}^{\infty }\int e^{-\rho t}\rho \phi \mu dxdt, \nonumber \\&\int _{_{0}}^{\infty }\int e^{-\rho t}\left( \frac{\partial \phi }{\partial t} -\rho \phi +\omega f+\mathcal {A}_{u,Z}\phi \right) \mu dxdt =0, \nonumber \\&\int _{_{0}}^{\infty }\int e^{-\rho t}\left( \frac{\partial \phi }{\partial t} \mu -\rho \phi \mu +\omega f\mu +\phi \mathcal {A}_{u,Z}^{*}\mu \right) dxdt =0, \end{aligned}$$

where in the second line we have applied the market clearing condition (8) and in the third line the fact that $\mathcal {A} _{u,Z}^{*}$ is the adjoint operator of $\mathcal {A}_{u,Z}.$ If we integrate by parts the first term

$$\begin{aligned}&\int _{_{0}}^{\infty }\int e^{-\rho t}\left( \frac{\partial \phi }{\partial t} \mu -\rho \phi \mu \right) dxdt\\= & {} \int \left. e^{-\rho t}\phi \left( t,x\right) \mu \left( t,x\right) \right| _{0}^{\infty }dx \\&+\int _{_{0}}^{\infty }\int e^{-\rho t}\left( -\frac{\partial \mu }{ \partial t}\phi +\rho \phi \mu -\rho \phi \mu \right) dxdt \\= & {} -\int \phi \left( 0,x\right) \mu \left( 0,x\right) dx-\int _{_{0}}^{\infty }\int e^{-\rho t}\phi \frac{\partial \mu }{\partial t}dxdt \end{aligned}$$

as $\lim _{T\rightarrow \infty }e^{-\rho T}\phi \left( T,x\right) =0.$ Therefore, we have

$$\begin{aligned} \int _{_{0}}^{\infty }\int e^{-\rho t}\left[ \omega f\mu +\phi \overset{0}{ \overbrace{\left( -\frac{\partial \mu }{\partial t}+\mathcal {A}_{u,Z}^{*}\mu \right) }}\right] dxdt= & {} \int \phi \left( 0,x\right) \mu \left( 0,x\right) dx, \\ \int _{_{0}}^{\infty }\int e^{-\rho t}\omega f\mu dxdt= & {} \int \phi \left( 0,x\right) \mu \left( 0,x\right) dx, \end{aligned}$$

where we have applied the fact that $\mu$ satisfies the KF equation (6): $-\frac{\partial \mu }{\partial t}+\mathcal {A}_{u,Z}^{*}\mu =0.$ The social value functional is thus

$$\begin{aligned} V^{opt}\left( \mu \left( 0,\cdot \right) \right)= & {} \int _{_{0}}^{\infty }\int e^{-\rho t}\omega \left( t,x\right) f\left( x,u\right) \mu \left( t,x\right) dxdt\\= & {} \int \phi \left( 0,x\right) \mu \left( 0,x\right) dx. \end{aligned}$$

1.2 Proof of Proposition 2: Necessary Conditions in the Problem with Commitment

The problem of the leader is to maximize (17) subject to the KF equation (6), the market clearing conditions (8) and to the individual HJB equations (4), where the optimal individual controls are given by the first-order conditions

$$\begin{aligned} \frac{\partial f}{\partial u_{j}}+\sum _{i=1}^{n}\frac{\partial b_{i}}{ \partial u_{j}}\frac{\partial V}{\partial x_{i}}=0, \quad j=1,\ldots ,m,\forall \left( t,x\right) \in \Phi . \end{aligned}$$

(41)

The Lagragian in this case is the one in Proposition 1 extended to include two extra terms that capture the value function and control dynamics:

$$\begin{aligned}&\mathcal {L}\left( \mu ,V,u_{1},\ldots ,u_{m},Z_{1},\ldots ,Z_{p},Y_{1},\ldots ,Y_{q}\right) \nonumber \\&\quad =\left\langle e^{-\rho t}\omega f,\mu \right\rangle _{\Phi }+\left\langle e^{-\rho t}\phi ,-\frac{\partial \mu }{\partial t}+\mathcal {A}_{u,Z,Y}^{*}\mu \right\rangle _{\Phi } +\sum _{k=1}^{p}\left\langle e^{-\rho t}\lambda _{k},\left( g_{k}-Z_{k}\right) \mu \right\rangle _{\Phi } \nonumber \\&\quad \quad +\,\left\langle e^{-\rho t}\theta ,-\rho V+\frac{\partial V}{\partial t}+f+\mathcal {A}_{u,Z,Y}V\right\rangle _{\Phi } +\sum _{j=1}^{m}\left\langle e^{-\rho t}\eta _{j},\frac{\partial f}{\partial u_{j}}+\sum _{i=1}^{n}\frac{\partial b_{i}}{\partial u_{j}}\frac{ \partial V}{\partial x_{i}}\right\rangle _{\Phi },\nonumber \\ \end{aligned}$$

(42)

where $\theta \left( t,x\right) ,\eta _{j}\left( t,x\right) \in \tilde{L} ^{2}\left( \Phi \right) ,$ $j=1,\ldots ,m,$ are the Lagrange multipliers associated to the HJB equation (4) and to the first-order conditions (41), respectively.

The Gateaux derivative with respect to $\mu$ is again

$$\begin{aligned}&\left. \frac{d}{d\alpha }\mathcal {L}\left( \mu +\alpha h,V,u,Z,Y\right) \right| _{\alpha =0}\\= & {} \left\langle e^{-\rho t}\omega f,h\right\rangle _{\Phi }+\left\langle e^{-\rho t}\left( \frac{\partial \phi }{\partial t} -\rho \phi +\mathcal {A}_{u,Z,Y}\phi \right) ,h\right\rangle _{\Phi } \\&+\sum _{k=1}^{p}\left\langle e^{-\rho t}\lambda _{k},\left( g_{k}-Z_{k}\right) h\right\rangle _{\Phi } -\lim _{T\rightarrow \infty }\int e^{-\rho T}\phi \left( T,x\right) h\left( T,x\right) dx, \end{aligned}$$

and therefore $\phi \left( t,x\right)$ should satisfy the leader’s HJB

$$\begin{aligned} \frac{\partial \phi }{\partial t}+\omega f+\sum _{k=1}^{p}\lambda _{k}\left( g_{k}-Z_{k}\right) +\mathcal {A}_{u,Z,Y}\phi= & {} \rho \phi ,\text { }\forall \left( t,x\right) \in \Phi , \\ \lim _{T\rightarrow \infty }e^{-\rho T}\phi \left( T,x\right)= & {} 0,\text { } \forall x\in \mathbb {R} ^{n}. \end{aligned}$$

The Gateaux derivative with respect to the aggregate variable $Z_{k}$ is

$$\begin{aligned}&\left. \frac{d}{d\alpha }\mathcal {L}\left( \mu ,V,u,Z_{1},..Z_{k}+\alpha h,..Z_{p},Y\right) \right| _{\alpha =0} \\= & {} \left. \frac{d}{d\alpha } \left\langle e^{-\rho t}\phi ,\left( -\frac{\partial \mu }{\partial t}+ \mathcal {A}_{u,Z_{k}+\alpha h,Y}^{*}\mu \right) \right\rangle _{\Phi }\right| _{\alpha =0} \\&+\left. \frac{d}{d\alpha }\sum _{k=1}^{p}\left\langle e^{-\rho t}\lambda _{k},\left( g_{k}-\left( Z_{k}+\alpha h\right) \right) \mu \right\rangle _{\Phi }\right| _{\alpha =0} \\&+\left. \frac{d}{d\alpha }\left\langle e^{-\rho t}\theta ,-\rho V+\frac{ \partial V}{\partial t}+f+\mathcal {A}_{u,Z_{k}+\alpha h,Y}V\right\rangle _{\Phi }\right| _{\alpha =0} \\&+\left. \frac{d}{d\alpha }\sum _{j=1}^{m}\left\langle e^{-\rho t}\eta _{j}, \frac{\partial f}{\partial u_{j}}+\sum _{i=1}^{n}\frac{\partial b_{i}\left( Z_{k}+\alpha h\right) }{\partial u_{j}}\frac{\partial V}{\partial x_{i}} \right\rangle _{\Phi }\right| _{\alpha =0}, \end{aligned}$$

for any $e^{-\rho t}h(t)\in L^{2}[0,\infty ).$ Here $\mathcal {A} _{u,Z_{k}+\alpha h,Y}^{*}:=\mathcal {A}_{u,Z_{1},\ldots ,Z_{k}+\alpha h,\ldots ,Z_{p},Y}^{*}$ and

$$\begin{aligned} b_{i}\left( Z_{k}+\alpha h\right) :=b_{i}\left( x,u,Z_{1},\ldots ,Z_{k}+\alpha h,\ldots ,Z_{p},Y\right) . \end{aligned}$$

The Gateaux derivative should be equal to zero in the maximum:

$$\begin{aligned} 0= & {} -\int _{_{0}}^{\infty }e^{-\rho t}h(t)\left\{ \int \phi (t,x)\left( \sum _{i=1}^{n}\left[ \frac{\partial ^{2}b_{i}}{\partial Z_{k}\partial x_{i}} \mu (t,x)\right. \right. \right. \\&\left. \left. \left. +\sum _{j=1}^{m}\frac{\partial ^{2}b_{i}}{\partial Z_{k}\partial u_{j}}\frac{\partial u_{j}}{\partial x_{i}}\mu +\frac{\partial b_{i}}{ \partial Z_{k}}\frac{\partial \mu }{\partial x_{i}}\right] \right) dx+\lambda _{k}(t)\right\} dt \\&+\int _{_{0}}^{\infty }e^{-\rho t}h(t)\left\{ \int \theta (t,x)\left( \sum _{i=1}^{n}\frac{\partial b_{i}}{\partial Z_{k}}\frac{\partial V}{ \partial x_{i}}\right) dx\right. \\&\left. +\sum _{j=1}^{m}\int \eta _{j}(t,x)\left( \sum _{i=1}^{n}\frac{\partial ^{2}b_{i}}{\partial Z_{k}\partial u_{j}}\frac{ \partial V}{\partial x_{i}}\right) dx\right\} dt. \end{aligned}$$

As this is satisfied for any h(t),  we obtain that

$$\begin{aligned} \lambda _{k}(t)= & {} \int \left\{ \theta \sum _{i=1}^{n}\frac{\partial b_{i}}{ \partial Z_{k}}\frac{\partial V}{\partial x_{i}}+\sum _{j=1}^{m}\eta _{j}\sum _{i=1}^{n}\frac{\partial ^{2}b_{i}}{\partial Z_{k}\partial u_{j}} \frac{\partial V}{\partial x_{i}}\right. \\&\left. -\phi \sum _{i=1}^{n}\left[ \frac{\partial ^{2}b_{i}}{\partial Z_{k}\partial x_{i}}\mu +\sum _{j=1}^{m}\frac{\partial ^{2}b_{i}}{\partial Z_{k}\partial u_{j}}\frac{\partial u_{j}}{\partial x_{i}} \mu +\frac{\partial b_{i}}{\partial Z_{k}}\frac{\partial \mu }{\partial x_{i} }\right] \right\} dx \\= & {} \int \left\{ \theta \sum _{i=1}^{n}\frac{\partial b_{i}}{\partial Z_{k}} \frac{\partial V}{\partial x_{i}}+\sum _{j=1}^{m}\eta _{j}\sum _{i=1}^{n}\frac{ \partial ^{2}b_{i}}{\partial Z_{k}\partial u_{j}}\frac{\partial V}{\partial x_{i}}+\mu \sum _{i=1}^{n}\frac{\partial b_{i}}{\partial Z_{k}}\frac{\partial \phi }{\partial x_{i}}\right\} dx, \end{aligned}$$

where we have integrated by parts in the last equality.

In order to compute the Gateaux derivative with respect to the individual value function V, we first expressed the fourth term in the Lagragian as

$$\begin{aligned}&\left\langle e^{-\rho t}\theta ,-\rho V+\frac{\partial V}{\partial t}+\omega f+\mathcal {A}_{u,Z,Y}V\right\rangle _{\Phi }\\= & {} \int _{_{0}}^{\infty }\int e^{-\rho t}\theta \left( t,x\right) \left( -\rho V+\frac{\partial V}{ \partial t}\right) dxdt \\&+\left\langle e^{-\rho t}\theta ,\omega f+\mathcal {A}_{u,Z,Y}V\right\rangle _{\Phi } \\= & {} \int \left. e^{-\rho t}\theta \left( t,x\right) V\left( t,x\right) \right| _{0}^{\infty }dx-\int _{_{0}}^{\infty }\int e^{-\rho t}\frac{ \partial \theta }{\partial t}Vdtdx \\&+\left\langle e^{-\rho t}\mathcal {A}_{u,Z,Y}^{*}\theta ,V\right\rangle _{\Phi }+\left\langle e^{-\rho t}\theta ,\omega f\right\rangle _{\Phi } \\= & {} \lim _{T\rightarrow \infty }\int e^{-\rho T}\theta \left( T,x\right) V\left( T,x\right) dx-\int \theta \left( 0,x\right) V\left( 0,x\right) dx \\&+\left\langle e^{-\rho t}\left( -\frac{\partial \theta }{\partial t}+ \mathcal {A}_{u,Z,Y}^{*}\theta \right) ,V\right\rangle _{\Phi }+\left\langle e^{-\rho t}\theta ,\omega f\right\rangle _{\Phi }, \end{aligned}$$

where we have integrated by parts with respect to time in the term $\frac{ \partial V}{\partial t}$ and applied the fact that $\mathcal {A} _{u,Z,Y}^{*}$ is the adjoint operator of $\mathcal {A}_{u,Z,Y}.$ The Gateaux derivative simplifies to

$$\begin{aligned}&\left. \frac{d}{d\alpha }\mathcal {L}\left( \mu ,V+\alpha h,u,Z,Y\right) \right| _{\alpha =0}\\= & {} \lim _{T\rightarrow \infty }\int e^{-\rho T}\theta \left( T,x\right) \left. \frac{d}{d\alpha }\left( V\left( T,x\right) +\alpha h\left( T,x\right) \right) \right| _{\alpha =0}dx \\&-\int \theta \left( 0,x\right) \left. \frac{d}{d\alpha }\left( V\left( 0,x\right) +\alpha h\left( 0,x\right) \right) \right| _{\alpha =0}dx \\&+\left. \frac{d}{d\alpha }\left\langle e^{-\rho t}\left( -\frac{\partial \theta }{\partial t}+\mathcal {A}_{u,Z,Y}^{*}\theta \right) ,V+\alpha h\right\rangle _{\Phi }\right| _{\alpha =0} \\&+\left. \frac{d}{d\alpha }\sum _{j=1}^{m}\left\langle e^{-\rho t}\eta _{j}, \frac{\partial f}{\partial u_{j}}+\sum _{i=1}^{n}\frac{\partial b_{i}}{ \partial u_{j}}\frac{\partial \left( V+\alpha h\right) }{\partial x_{i}} \right\rangle _{\Phi }\right| _{\alpha =0} \\= & {} \lim _{T\rightarrow \infty }\int e^{-\rho T}\theta \left( T,x\right) h\left( T,x\right) dx-\int \theta \left( 0,x\right) h\left( 0,x\right) dx \\&+\,\left\langle e^{-\rho t}\left( -\frac{\partial \theta }{\partial t}+ \mathcal {A}_{u,Z,Y}^{*}\theta \right) ,h\right\rangle _{\Phi } +\sum _{j=1}^{m}\left\langle e^{-\rho t}\eta _{j},\sum _{i=1}^{n}\frac{ \partial b_{i}}{\partial u_{j}}\frac{\partial h}{\partial x_{i}} \right\rangle _{\Phi }. \end{aligned}$$

The last term in the derivative can be expressed as

$$\begin{aligned} \sum _{j=1}^{m}\left\langle e^{-\rho t}\eta _{j},\sum _{i=1}^{n}\frac{\partial b_{i}}{\partial u_{j}}\frac{\partial h}{\partial x_{i}}\right\rangle _{\Phi }= & {} \sum _{i=1}^{n}\sum _{j=1}^{m}\int _{_{0}}^{\infty }\int e^{-\rho t}\eta _{j}\left( t,x\right) \frac{\partial b_{i}}{\partial u_{j}}\frac{\partial h}{ \partial x_{i}}dxdt \\= & {} -\sum _{i=1}^{n}\sum _{j=1}^{m}\int _{_{0}}^{\infty }\int e^{-\rho t}\frac{ \partial }{\partial x_{i}}\left( \eta _{j}\frac{\partial b_{i}}{\partial u_{j}}\right) hdxdt, \end{aligned}$$

where we have integrated by parts. Due to the transversality condition of the individual problem, $\lim _{T\rightarrow \infty }e^{-\rho T}V\left( T,x\right) =0,$ we have $\lim _{T\rightarrow \infty }h\left( T,x\right) =0$ $\forall x\in \mathbb {R} ^{n}.$ For $t<\infty ,$ the Gateaux derivative should be zero for any $h\left( t,x\right) \in \tilde{L}^{2}\left( \Phi \right)$ and therefore we obtain:

$$\begin{aligned} \frac{\partial \theta }{\partial t}= & {} \mathcal {A}_{u,Z,Y}^{*}\theta -\sum _{i=1}^{n}\sum _{j=1}^{m}\frac{\partial }{\partial x_{i}}\left( \eta _{j} \frac{\partial b_{i}}{\partial u_{j}}\right) , \\ \theta \left( 0,x\right)= & {} 0, \forall x\in \mathbb {R} ^{n}. \end{aligned}$$

The Gateaux derivative with respect to the individual control $u_{j}$ is

$$\begin{aligned}&\left. \frac{d}{d\alpha }\mathcal {L}\left( \mu ,u_{1},\ldots ,u_{j}+\alpha h,..u_{m},Z,Y\right) \right| _{\alpha =0}\\= & {} \left. \frac{d}{d\alpha } \left\langle e^{-\rho t}\omega f\left( x,u_{j}+\alpha h\right) ,\mu \right\rangle _{\Phi }\right| _{\alpha =0} \\&+\left. \frac{d}{d\alpha }\left\langle e^{-\rho t}\left( \frac{\partial \phi }{\partial t}-\rho \phi +\mathcal {A}_{u_{j}+\alpha h,Z,Y}\phi \right) ,\mu \right\rangle _{\Phi }\right| _{\alpha =0} \\&+\left. \frac{d}{d\alpha }\sum _{k=1}^{p}\left\langle e^{-\rho t}\lambda _{k},\left( g_{k}\left( x,u_{j}+\alpha h\right) -Z_{k}\right) \mu \right\rangle _{\Phi }\right| _{\alpha =0} \\&+\left. \frac{d}{d\alpha }\left\langle e^{-\rho t}\theta ,-\rho V+\frac{ \partial V}{\partial t}+f+\mathcal {A}_{u_{j}+\alpha h,Z,Y}V\right\rangle _{\Phi }\right| _{\alpha =0} \\&+\left. \frac{d}{d\alpha }\sum _{k=1}^{m}\left\langle e^{-\rho t}\eta _{k}, \frac{\partial f\left( x,u_{j}+\alpha h\right) }{\partial u_{k}} +\sum _{i=1}^{n}\frac{\partial b_{i}\left( u_{j}+\alpha h\right) }{\partial u_{k}}\frac{\partial V}{\partial x_{i}}\right\rangle _{\Phi }\right| _{\alpha =0}, \end{aligned}$$

and thus the maximum should satisfy

$$\begin{aligned}&\left( \omega \frac{\partial f}{\partial u_{j}}+\sum _{i=1}^{n}\frac{\partial b_{i}}{\partial u_{j}}\frac{\partial \phi }{\partial x_{i}} +\sum _{k=1}^{p}\lambda _{k}\frac{\partial g_{k}}{\partial u_{j}}\right) \mu +\theta \overset{0}{\overbrace{\left( \frac{\partial f}{\partial u_{j}}+\,\sum _{i=1}^{n}\frac{\partial b_{i}}{\partial u_{j}}\frac{\partial V}{ \partial x_{i}}\right) }}\nonumber \\&\quad +\sum _{k=1}^{m}\eta _{k}\left( \frac{\partial ^{2}f }{\partial u_{j}\partial u_{k}}+\sum _{i=1}^{n}\frac{\partial ^{2}b_{i}}{ \partial u_{j}\partial u_{k}}\frac{\partial V}{\partial x_{i}}\right) =0. \end{aligned}$$

(43)

Notice that $\frac{\partial f}{\partial u_{j}}+\sum _{i=1}^{n}\frac{\partial b_{i}}{\partial u_{j}}\frac{\partial V}{\partial x_{i}}=0$ due to the first-order conditions (41).

Finally, the Gateaux derivative with respect to the aggregate policy $Y_{r}$

$$\begin{aligned}&\lim _{\alpha \rightarrow 0}\frac{d}{d\alpha }\mathcal {L}\left( \mu ,V,u,Z,Y_{1},..Y_{r}+\alpha h,..Y_{q}\right) \\= & {} \lim _{\alpha \rightarrow 0}\left\{ \frac{d}{d\alpha }\left\langle e^{-\rho t}\phi ,\left( -\frac{\partial \mu }{ \partial t}+\mathcal {A}_{u,Z,Y_{r}+\alpha h}^{*}\mu \right) \right\rangle _{\Phi } \right. \\&+\frac{d}{d\alpha }\left\langle e^{-\rho t}\theta ,-\rho V+\frac{\partial V }{\partial t}+f+\mathcal {A}_{u,Z,Y_{r}+\alpha h}V\right\rangle _{\Phi } \\&+\left. \frac{d}{d\alpha }\sum _{j=1}^{m}\left\langle e^{-\rho t}\eta _{j},\frac{ \partial f}{\partial u_{j}}+\sum _{i=1}^{n}\frac{\partial b_{i}\left( Y_{r}+\alpha h\right) }{\partial u_{j}}\frac{\partial V}{\partial x_{i}} \right\rangle _{\Phi }\right\} , \end{aligned}$$

equals zero in the maximum for any $h(t)\in e^{-\rho t}L^{2}[0,\infty ).$ Here $\mathcal {A}_{u,Z,Y_{r}+\alpha h}^{*}:=\mathcal {A} _{u,Z,Y_{1},\ldots ,Y_{r}+\alpha h,\ldots ,Y_{q}}^{*}.$ This can be expressed as

$$\begin{aligned} 0= & {} -\int _{_{0}}^{\infty }e^{-\rho t}h(t)\int \phi (t,x)\sum _{i=1}^{n}\left[ \frac{\partial ^{2}b_{i}}{\partial Y_{r}\partial x_{i}}\mu (t,x)\right. \\&\quad +\left. \sum _{j=1}^{m}\frac{\partial ^{2}b_{i}}{\partial Y_{r}\partial u_{j}} \frac{\partial u_{j}}{\partial x_{i}}\mu +\frac{\partial b_{i}}{\partial Y_{r}}\frac{\partial \mu }{\partial x_{i}}\right] dxdt \\&+\int _{_{0}}^{\infty }e^{-\rho t}h(t)\left\{ \int \theta (t,x)\left( \sum _{i=1}^{n}\frac{\partial b_{i}}{\partial Y_{r}}\frac{\partial V}{ \partial x_{i}}\right) dx\right. \\&\left. +\sum _{j=1}^{m}\int \eta _{j}(t,x)\left( \sum _{i=1}^{n}\frac{\partial ^{2}b_{i}}{\partial Y_{r}\partial u_{j}}\frac{ \partial V}{\partial x_{i}}\right) dx\right\} dt \end{aligned}$$

As this is satisfied for any h(t),  we obtain that

$$\begin{aligned}&\int \left\{ \theta \sum _{i=1}^{n}\frac{\partial b_{i}}{\partial Y_{r}}\frac{ \partial V}{\partial x_{i}}+\sum _{j=1}^{m}\eta _{j}\sum _{i=1}^{n}\frac{ \partial ^{2}b_{i}}{\partial Y_{r}\partial u_{j}}\frac{\partial V}{\partial x_{i}}\right. \\&\quad \left. -\phi \sum _{i=1}^{n}\left[ \frac{\partial ^{2}b_{i}}{\partial Y_{r}\partial x_{i}}\mu +\sum _{j=1}^{m}\frac{\partial ^{2}b_{i}}{\partial Y_{r}\partial u_{j}}\frac{\partial u_{j}}{\partial x_{i}}\mu +\frac{\partial b_{i}}{\partial Y_{r}}\frac{\partial \mu }{\partial x_{i}}\right] \right\} dx =0, \\&\int \left\{ \theta \sum _{i=1}^{n}\frac{\partial b_{i}}{\partial Y_{r}}\frac{ \partial V}{\partial x_{i}}+\sum _{j=1}^{m}\eta _{j}\sum _{i=1}^{n}\frac{ \partial ^{2}b_{i}}{\partial Y_{r}\partial u_{j}}\frac{\partial V}{\partial x_{i}}+\sum _{i=1}^{n}\frac{\partial \phi }{\partial x_{i}}\frac{\partial b_{i}}{\partial Y_{r}}\mu \right\} dx =0, \end{aligned}$$

where we have integrated by parts to obtain the last expression.

1.3 Proof of Proposition 3: Necessary Conditions in the Problem with Discretion

The proof proceeds in two steps. First we solve a commitment problem over a fixed period of length $\Delta$ taking as given the next period value functional $W\left( \mu \left( t+\Delta ,\cdot \right) \right) .$ Then we take the limit as $\Delta \rightarrow 0.$

Step 1: Solution Given a Fixed Time Step $\Delta$

We have assumed that, given $T>0,$ the interval [0, T] is divided in N intervals of length $\Delta :=T/N.$ First we solve the open-loop Stackelberg problem (26) over a fixed time interval $s\in [t,t+\Delta ],$ where t is a multiple of $\Delta ,$ subject to the KF equation (6), the market clearing conditions (8) and to the individual HJB equations (4) with optimal individual controls (41). The solution mimics the proof of Proposition 2 above with two major differences. The first one is the finite-horizon nature of the problem. The second is the presence of the terminal value $W\left( \mu \left( t+\Delta ,\cdot \right) \right) .$

The Lagragian is similar as the one in (42) with the inclusion of the terminal value functional $W\left( \mu \left( t+\Delta ,\cdot \right) \right)$:

$$\begin{aligned}&\left\langle e^{-\rho t}\omega f,\mu \right\rangle _{\Phi _{t}}+e^{-\rho \left( t+\Delta \right) }W\left( \mu \left( t+\Delta ,\cdot \right) \right) +\left\langle e^{-\rho t}\phi ,-\frac{\partial \mu }{\partial s}+\mathcal {A }_{u,Z,Y^{\Delta }}^{*}\mu \right\rangle _{\Phi _{t}} \nonumber \\&\quad +\sum _{k=1}^{p}\left\langle e^{-\rho t}\lambda _{k},\left( g_{k}-Z_{k}\right) \mu \right\rangle _{\Phi _{t}} +\left\langle e^{-\rho t}\theta ,-\rho V+\frac{\partial V}{\partial s}+f+ \mathcal {A}_{u,Z,Y^{\Delta }}V\right\rangle _{\Phi _{t}}\nonumber \\&\quad +\sum _{j=1}^{m}\left\langle e^{-\rho t}\eta _{j},\frac{\partial f}{ \partial u_{j}}+\sum _{i=1}^{n}\frac{\partial b_{i}}{\partial u_{j}}\frac{ \partial V}{\partial x_{i}}\right\rangle _{\Phi _{t}}, \end{aligned}$$

(44)

where time is denoted as $s\in [t,t+\Delta ]$ and $\Phi _{t}:=[t,t+\Delta ]\times \mathbb {R} ^{n}$.

The Gateaux derivative with respect to $\mu$ is¹⁴

$$\begin{aligned}&\left\langle e^{-\rho t}\omega f,h\right\rangle _{\Phi _{t}}+\left\langle e^{-\rho t}\left( \frac{\partial \phi }{\partial s}-\rho \phi +\mathcal {A} _{u,Z,Y^{\Delta }}\phi \right) ,h\right\rangle _{\Phi _{t}}\nonumber \\&\quad +\sum _{k=1}^{p}\left\langle e^{-\rho t}\lambda _{k},\left( g_{k}\!-\!Z_{k}\right) h\right\rangle _{\Phi _{t}} -\int e^{-\rho \Delta }\phi \left( t+\Delta ,x\right) h\left( t\!+\!\Delta ,x\right) dx+e^{-\rho \left( t+\Delta \right) }\\&\quad \left. \frac{d}{d\alpha } W\left( \mu \left( t+\Delta ,\cdot \right) +\alpha h\left( t+\Delta ,\cdot \right) \right) \right| _{\alpha =0}. \end{aligned}$$

If W is Frechet differentiable then the Gateaux derivative of W can be expressed as

$$\begin{aligned} \left. \frac{d}{d\alpha }W\left( \mu \left( t+\Delta ,\cdot \right) +\alpha h\left( t+\Delta ,\cdot \right) \right) \right| _{\alpha =0}=\int \frac{ \delta W}{\delta \mu }(\mu \left( t+\Delta ,\cdot \right) )h\left( t+\Delta ,x\right) dx, \end{aligned}$$

where $\frac{\delta W}{\delta \mu }(\mu \left( t+\Delta ,\cdot \right) )\in L^{2}\left( \mathbb {R} ^{n}\right) .$ The optimality condition then implies that

$$\begin{aligned} \frac{\partial \phi }{\partial s}+\omega f+\sum _{k=1}^{p}\lambda _{k}\left( g_{k}-Z_{k}\right) +\mathcal {A}_{u,Z,Y^{\Delta }}\phi= & {} \rho \phi ,\text { } \forall s\in [t,t+\Delta ),x\in \mathbb {R} ^{n}, \nonumber \\ \phi \left( t+\Delta ,x\right)= & {} \frac{\delta W}{\delta \mu }(t+\Delta ,x), \text { }\forall x\in \mathbb {R} ^{n}. \end{aligned}$$

(45)

The optimality conditions with respect to aggregate variables $Z_{k}$, individual controls $u_{j}$ and aggregate policies $Y_{r}^{\Delta }$ are the same as in Proposition 2:

$$\begin{aligned} \lambda _{k}(t)= & {} \int \left\{ \theta \sum _{i=1}^{n}\frac{\partial b_{i}}{ \partial Z_{k}}\frac{\partial V}{\partial x_{i}}+\sum _{j=1}^{m}\eta _{j}\sum _{i=1}^{n}\frac{\partial ^{2}b_{i}}{\partial Z_{k}\partial u_{j}} \frac{\partial V}{\partial x_{i}}+\mu \sum _{i=1}^{n}\frac{\partial b_{i}}{ \partial Z_{k}}\frac{\partial \phi }{\partial x_{i}}\right\} dx, \\ 0= & {} \left( \omega \frac{\partial f}{\partial u_{j}}+\sum _{i=1}^{n}\frac{ \partial b_{i}}{\partial u_{j}}\frac{\partial \phi }{\partial x_{i}}+\sum _{k=1}^{p}\lambda _{k}\frac{\partial g_{k}}{\partial u_{j}}\right) \mu \\&+\sum _{k=1}^{m}\eta _{k}\left( \frac{\partial ^{2}f}{\partial u_{j}\partial u_{k}}+\sum _{i=1}^{n}\frac{\partial ^{2}b_{i}}{\partial u_{j}\partial u_{k}} \frac{\partial V}{\partial x_{i}}\right) , \\ 0= & {} \int \left\{ \theta \sum _{i=1}^{n}\frac{\partial b_{i}}{\partial Y_{r}} \frac{\partial V}{\partial x_{i}}+\sum _{j=1}^{m}\eta _{j}\sum _{i=1}^{n}\frac{ \partial ^{2}b_{i}}{\partial Y_{r}\partial u_{j}}\frac{\partial V}{\partial x_{i}}+\sum _{i=1}^{n}\frac{\partial \phi }{\partial x_{i}}\frac{\partial b_{i}}{\partial Y_{r}}\mu \right\} dx. \end{aligned}$$

Finally, the Gateaux derivative with respect to the individual value function V is

$$\begin{aligned}&\int e^{-\rho \left( t+\Delta \right) }\theta \left( t+\Delta ,x\right) h\left( t+\Delta ,x\right) dx-\int \theta \left( t,x\right) h\left( t,x\right) dx \\&+\left\langle e^{-\rho t}\left( -\frac{\partial \theta }{\partial s}+ \mathcal {A}_{u,Z,Y^{\Delta }}^{*}\theta \right) ,h\right\rangle _{\Phi _{t}}+\sum _{j=1}^{m}\left\langle e^{-\rho t}\eta _{j},\sum _{i=1}^{n}\frac{ \partial b_{i}}{\partial u_{j}}\frac{\partial h}{\partial x_{i}} \right\rangle _{\Phi _{t}}, \end{aligned}$$

and the optimality condition then results in

$$\begin{aligned} \frac{\partial \theta }{\partial s}= & {} \mathcal {A}_{u,Z,Y^{\Delta }}^{*}\theta -\sum _{i=1}^{n}\sum _{j=1}^{m}\frac{\partial }{\partial x_{i}}\left( \eta _{j}\frac{\partial b_{i}}{\partial u_{j}}\right) , \forall s\in [t,t+\Delta ),x\in \mathbb {R} ^{n}, \end{aligned}$$

(46)

$$\begin{aligned} \theta \left( t,x\right)= & {} 0,\text { }\forall x\in \mathbb {R} ^{n}, \end{aligned}$$

(47)

where we have taken into account the fact that $h\left( t+\Delta ,\cdot \right) =0$ as the terminal individual value function $v\left( t+\Delta ,\cdot \right)$ is given.

Step 2: Taking the Limit $\Delta \rightarrow 0$

We take the limit as $N\rightarrow \infty ,$ or equivalently, $\Delta \rightarrow 0.$ ¹⁵ In this case, the value of the Lagrange multiplier $\theta$ in equation (47) is zero:$\ \theta \left( t,x\right) =0,$ $\forall x\in \mathbb {R} ^{n}.$ The HJB equation (45) then results in

$$\begin{aligned} \frac{\partial \phi }{\partial t}+\omega f+\sum _{k=1}^{p}\lambda _{k}\left( g_{k}-Z_{k}\right) +\mathcal {A}_{u,Z,Y}\phi= & {} \rho \phi ,\text { }\forall t\in [0,T),x\in \mathbb {R}^{n}, \\ \phi \left( T,x\right)= & {} \frac{\delta W}{\delta \mu }(\mu \left( T,x\right) ),\text { }\forall x\in \mathbb {R} ^{n}. \nonumber \end{aligned}$$

(48)

If we take the limit as $T\rightarrow \infty ,$ then $\lim _{T\rightarrow \infty }e^{-\rho T}\frac{\delta W}{\delta \mu }(\mu \left( T,x\right) )=\lim _{T\rightarrow \infty }e^{-\rho T}\phi \left( T,x\right) =0,$ which is the transversality condition of the infinite-horizon problem.

Taking into account the values of $\theta \left( \cdot \right) =0$ and $\phi \left( \cdot \right) =w\left( \cdot \right) ,$ the rest of optimality conditions simplify to

$$\begin{aligned} \lambda _{k}(t)= & {} \int \left\{ \sum _{j=1}^{m}\eta _{j}\sum _{i=1}^{n}\frac{ \partial ^{2}b_{i}}{\partial Z_{k}\partial u_{j}}\frac{\partial V}{\partial x_{i}}+\mu \sum _{i=1}^{n}\frac{\partial b_{i}}{\partial Z_{k}}\frac{\partial \phi }{\partial x_{i}}\right\} dx, \\ 0= & {} \left( \omega \frac{\partial f}{\partial u_{j}}+\sum _{i=1}^{n}\frac{ \partial b_{i}}{\partial u_{j}}\frac{\partial \phi }{\partial x_{i}} +\sum _{k=1}^{p}\lambda _{k}\frac{\partial g_{k}}{\partial u_{j}}\right) \mu \\&+\sum _{k=1}^{m}\eta _{k}\left( \frac{\partial ^{2}f}{\partial u_{j}\partial u_{k}}+\sum _{i=1}^{n}\frac{\partial ^{2}b_{i}}{\partial u_{j}\partial u_{k}} \frac{\partial V}{\partial x_{i}}\right) , \\ 0= & {} \int \left\{ \sum _{j=1}^{m}\eta _{j}\sum _{i=1}^{n}\frac{\partial ^{2}b_{i}}{\partial Y_{r}\partial u_{j}}\frac{\partial V}{\partial x_{i}} +\sum _{i=1}^{n}\frac{\partial \phi }{\partial x_{i}}\frac{\partial b_{i}}{ \partial Y_{r}}\mu \right\} dx. \end{aligned}$$