The Derivation of Ergodic Mean Field Game Equations for Several Populations of Players

Abstract

This note contains a detailed derivation of the equations of the recent mean field games theory (abbr. MFG), developed by M. Huang, P.E. Caines, and R.P. Malhamé on one hand and by J.-M. Lasry and P.-L. Lions on the other, associated with a class of stochastic differential games, where the players belong to several populations, each of which consisting of a large number of similar and indistinguishable individuals, in the context of periodic diffusions and long-time-average (or ergodic) costs. After introducing a system of N Hamilton–Jacobi–Bellman (abbr. HJB) and N Kolmogorov–Fokker–Planck (abbr. KFP) equations for an N-player game belonging to such a class of games, the system of MFG equations (consisting of as many HJB equations, and of as many KFP equations as is the number of populations) is derived by letting the number of the members of each population go to infinity. For the sake of clarity and for reader’s convenience, the case of a single population of players, as formulated in the work of J.-M. Lasry and P.-L. Lions, is presented first. The note slightly improves the results in this case too, by dealing with more general dynamics and costs.

1 Introduction

The objective of this note is to provide a detailed explanation of the derivation of the stationary equations of the mean field games theory—which is due to M. Huang, P.E. Caines, and R.P. Malhamé on one hand and to J.-M. Lasry and P.-L. Lions on the other—associated with certain ergodic problems for single and several populations of similar and indistinguishable players.

Let us recall briefly that mean field game models were introduced by M. Huang, P.E. Caines, and R.P. Malhamé [5–7] and J.-M. Lasry and P.-L. Lions [8–10] in order to describe games with a large number of small, similar, and indistinguishable players and have applications in economics, finance, and possibly, in sociology, urban planning, engineering, etc. (see [3] for a broad panorama of the possible applications). Roughly speaking, these models are obtained as a limit of games with a finite number of players, letting the number of the players go to infinity, a procedure that reminds somewhat the mean field approaches in particle physics (e.g., the derivation of Boltzmann or Vlasov equations in the kinetic theory of gasses) and thus also serves as a justification for the denomination of these models.

The focus of this note is on a class of stochastic differential games with a large number of players N, with periodic dynamics and long-time-average (or ergodic) costs, in which the dynamics of the players evolve independently of each other and players influence each other only through the costs. Following [10], we introduce a system of N Hamilton–Jacobi–Bellman (abbr. HJB) equations and N Kolmogorov–Fokker–Planck (abbr. KFP) equations in order to build certain Nash equilibria in feedback form for these games. We are interested in a class of games with large N that are symmetrical as far as the players are concerned. Carrying out an appropriate limit procedure as N→∞, whose detailed description is the aim of this note, we arrive at a system of only just one HJB equation and one KFP equation (if players belong all to a single homogeneous group of similar and indistinguishable individuals), which, however, captures the main features of this class of N-player games; the larger is N, the “better” this limit system of mean field game equations describes the N-player games. For example, the limit system may be used in order to calculate approximate Nash equilibria for the N-player games, see [6].

More precisely, in this note we show that limit points of sequences of solutions to the systems of N HJB and N KPP partial differential equations associated with the N-player games are solutions for the limit mean field game equations. In addition, we show that all Nash equilibria of these N-player games, obtained in feedback form, by solving the relative systems of N HJB and N KFP equations, become asymptotically symmetric as N→∞. Only in some situations it has been possible to give a rigorous proof of these facts, one of them being the aforesaid case of a class of ergodic stochastic differential games coupled only through the costs. Thus, in Sect. 2 of this note we provide a detailed proof of the derivation of the ergodic mean field game equations as stated by J.-M. Lasry and P.-L. Lions. More explicitly, we prove [8, Theorem 3.1] or [10], but J.-M. Lasry and P.-L. Lions do not give a detailed proof of their results; only some hints are given in [8]. Moreover, we are generalizing somewhat here by dealing with more general cost criteria and dynamics. Another reason for presenting this proof here is to make the sequel of this note (Sect. 3) more easily readable and understandable. In Sect. 2 we also fix some notation and assumptions that are going to be valid throughout the rest of this note.

The second goal of this note, and probably its main novelty, is to explain how to derive the mean field game equations for ergodic games with players belonging to several (say two, for simplicity) large homogeneous groups or populations of players. Each population consists of a large number of similar players, and each player cannot distinguish the other members of its own population or the members of the other populations from each other. Moreover, the information each player has in order to optimize its strategy is of statistical nature. (Of course, players belonging to different populations are different and hence distinguishable from each other in general.) Any player can collect population based statistics: e.g., any player knows the average of any numerical characteristic, which is a function of the members states, for any population of players. All of this is made mathematically precise in Sect. 3. Formally, in deriving the said equations, we have to let the number of the players constituting each population go to infinity. The resulting limit system of partial differential equations consists of as many HJB equations and as many KFP equations as is the number of the populations of players. More precisely, we prove the asymptotic symmetrization effect for Nash strategies corresponding to players belonging to any of the populations as the cardinality of the population itself tends to infinity. Moreover, we show that limit points of sequences whose elements are parts of solutions to the system of HJB and KFP equations for the games with a finite number of players, corresponding to each of the population of players, solve the limit mean field game equations

Although the results and techniques of this note may be used in order to build approximate Nash equilibria for the above-mentioned “prelimit” games with a finite number of players, we do not pursue this topic here, and, instead, for that purpose, we refer the reader to the paper of M. Huang, P.E. Caines, and R.P. Malhamé [6], of which the present note may be seen, in some sense, as a complement. However, we would like to inform the reader that the assumptions of this note are somewhat different from those of [6], the main difference being that here we allow the control set of each player to be noncompact and that Hamiltonians may grow arbitrarily, provided that they satisfy some technical condition, which means, at least formally, that Hamiltonians are not allowed to oscillate too much in the space variable.

2 The Case of a Single Homogeneous Population of Players

Now we describe in detail the “mean-field” approach following J.-M. Lasry and P.-L. Lions [8, 10]. Consider a control system driven by the stochastic differential equations

$$ dX^i_t=f^i \bigl(X^i_t, \alpha_t^i \bigr) \,dt + \sigma^i \bigl(X^i_t \bigr) \,dW^i_t ,\qquad X^i_0=x^i\in\mathbb{R}^d , \quad i=1, \dots,N, $$

(1)

where: $\{W^{i}_{t}\}$ are N independent Brownian motions in $\mathbb{R}^{d}$, d≥1; A ⁱ, the control set of the ith player, is a metric space;

$$f^i: \mathbb{R}^d \times A^i \to\mathbb{R}^d\quad\mbox{and} \quad \sigma^i: \mathbb{R}^d \to \mathbb{R}^{d\times d} $$

are $\mathbb{Z}^{d}$-periodic in x and locally Lipschitz continuous in x and in α, the matrix σ ⁱ(x) is nonsingular for any value of x; $\alpha_{t}^{i}$ is an admissible control of the ith player, that is, a locally bounded stochastic process taking values in A ⁱ and adapted to $W^{i}_{t}$.

In view of the assumed periodicity in x ⁱ of all data, we will often consider functions as defined on $Q = \mathbb{T}^{d} (= \mathbb {R}^{d}/{\mathbb{Z}^{d}})$, instead of $\mathbb{R}^{d}$. The ith player seeks to minimize the long-time-average or ergodic cost

$$ J^i \bigl(X_0,\alpha^1,\dots, \alpha^N \bigr)=\liminf_{T\to+\infty} \frac{1}{T} E \biggl[\int _0^TL^i \bigl(X^i_t, \alpha_t^i \bigr) + F^i \bigl(X^1_t, \dots,X^N_t \bigr) \,dt \biggr]. $$

(2)

On the cost of the ith player (2), we assume that

$$ L^i:Q \times A^i \to \mathbb{R}$$

(3)

are measurable and locally bounded, whereas

$$ F^i \bigl(x^1, \dots, x^N \bigr) : Q^{N}\to\mathbb{R}\quad\mbox{is Lipschitz continuous}. $$

(4)

For all x∈Q and $p\in\mathbb{R}^{d}$, define

$$ H^i(x, p) = \sup_{\alpha\in A^i}{ \bigl(-p\cdot f^i(x,\alpha)-L^i \bigl(x^i, \alpha \bigr) \bigr)}. $$

(5)

Of course, we assume that the supremum on the right-hand side is finite for any choice of x∈Q and $p \in\mathbb{R}^{d}$ (this is certainly so if A ⁱ is compact, but beware that we do not make any compactness assumption on A ⁱ here; on the contrary, we are primarily interested in the case $A^{i} =\mathbb{R}^{m_{i}}$ for some $m_{i} \in\mathbb{N}$).

Actually, in order to be able to build a Nash equilibrium in feedback form for the N-player stochastic differential game described above, we assume that there exist functions $\overline {\alpha}^{i}: \mathbb{R}^{d}\times\mathbb{R}^{d}\to A^{i}$ such that

$$ \overline {\alpha}^i\mbox{ is } \mathbb{Z}^d \mbox{-periodic in }x, \mbox{ locally Lipschitz in both variables},\quad \mathrm{and} $$

(6)

$$ \overline {\alpha}^i(x, p) \mbox{ is a maximum point for } \alpha \to-f^i(x, \alpha) \cdot p - L^i(x, \alpha) \quad\forall x,p. $$

(7)

This is the case, e.g., for a class of games where each drift f ⁱ depends affinely on the control $\alpha\in A^{i}=\mathbb{R}^{m_{i}}$, the Lagrangian L ⁱ is of class C ², coercive in $\alpha\in\mathbb{R}^{m_{i}}$, and D _α L ⁱ(x,⋅) is a C ¹-diffeomorphism for all x∈Q (the latter two conditions may be replaced by requiring that L ⁱ satisfy some kind of strict convexity in $\alpha\in\mathbb{R}^{m_{i}}$); see [1, Example 3.1] for the details.

Finally, define also

$$\begin{aligned} g^i(x, p) =& -f^i \bigl(x, \overline { \alpha}^i(x, p) \bigr), \quad i=1, \dots N, \end{aligned}$$

(8)

$$\begin{aligned} a^i =& \sigma^i \bigl( \sigma^i \bigr)^t/2, \quad\mathcal{L}^i = - a^i \cdot D^2, \end{aligned}$$

(9)

and

$$ V^{i,N}[m](x) = \int_{Q^{N-1}} F^i \bigl(x^1,\dots,x^{i-1},x,x^{i+1}, \dots,x^N \bigr)\prod_{j\ne i}m_j^N \bigl(x^j \bigr)\, dx^j $$

(10)

for all N-tuples of Borel probability measures $m^{N} = (m_{1}^{N}, \dots, m_{N}^{N})$ in Q. Here and throughout the paper, the $\mathcal{L}^{i} = - a^{i} \cdot D^{2}$, i=1,…,N, are second-order uniformly elliptic operators with Lipschitz coefficients in Q. (We use the notation $\operatorname {tr} b$ for the trace of a square matrix b and $a\cdot b = \operatorname{tr} a b^{t}$ and |b|:=(b⋅b)^1/2 for the Frobenius scalar product and norm, respectively, in a space of matrices.) We denote by ${\mathcal{L}^{i}}^{*}v = - \sum_{h,k}D_{hk}^{2} (a^{i}_{hk} v)$ the formal adjoint of $\mathcal{L}^{i}$, which is to be interpreted in the sense of distributions:

$$\bigl\langle{\mathcal{L}^i}^* v, \phi \bigr\rangle= \int _Q v \mathcal{L}\phi\,dx \quad\forall\phi\in C^{\infty}(Q). $$

In order to build Nash equilibria for this N-player game, we need to solve the following system of equations:

$$\begin{aligned} &\mathcal{L}^i {v_i^N}+H^i \bigl(x, D {v_i^N} \bigr)+{\lambda_i^N}= V^{i, N} \bigl[ m^N \bigr]\quad\text{in } \mathbb{R}^{d}, \end{aligned}$$

(11)

$$\begin{aligned} &{\mathcal{L}^i}^*{m_i^N}- \mathrm{div} \bigl(g^i \bigl(x, Dv_i^N \bigr){m_i^N} \bigr)=0, \end{aligned}$$

(12)

$$\begin{aligned} &\int_Q{v_i^N \,dx} =0, \qquad\int_Q m_i^N\,dx =1, \quad m_i^N >0, \ i =1, \dots, N. \end{aligned}$$

(13)

In order to assert the existence of Nash equilibria, we need some further assumptions on the Hamiltonians H ⁱ(x,p). (We observe that, in any case, H ⁱ(x,p) are convex in p, but this is not necessary for the following existence result.) We assume that the Hamiltonians H ⁱ=H ⁱ(x,p) (x∈Q, $p \in \mathbb{R} ^{d}$) satisfy one of the following conditions:

• 1. Either they are locally Lipschitz in both variables, superlinear in p uniformly in x, i.e.,

  $$ \inf_{x \in Q}{ \bigl\vert H^i(x, p) \bigr\vert}/|p| \to+\infty\quad\textrm{as} \ |p| \to\infty, $$

  (14)

  and ∃θ ⁱ∈(0,1), C>0:

  $$ \operatorname{tr} \bigl(a^i \bigr) D_xH^i \cdot p + \theta^i \bigl(H^i \bigr)^2 \ge-C|p|^2 \quad \textrm{for} \ |p| \textrm{ large and for a.e. } x\in Q. $$

  (15)

• 2. Or they are locally α-Hölder continuous (0<α<1) and grow at most quadratically:

  $$ \bigl \vert H^i(x, p)\bigr \vert\le C_1 |p|^2 +C_2 \quad\forall x\in Q, p \in \mathbb{R}^d, i=1, \dots, N, $$

  (16)

  for some C ₁, C ₂>0, the so-called natural growth condition.

In [1], under these assumptions on f ⁱ, σ ⁱ, L ⁱ, F ⁱ, H ⁱ, we have shown the existence of Nash equilibria for the games above, precisely, the following:

Theorem 1

(i) There exist $\lambda_{1}^{N}, \dots, \lambda_{N}^{N} \in \mathbb{R}$, $v_{1}^{N}, \dots, v_{N}^{N} \in C^{2}(Q)$, $m_{1}^{N}, \dots$, $m_{N}^{N} \in W^{1,p}(Q)$, 1≤p<∞, that solve (11), (12), (13).

(ii) For any solution $\lambda_{1}^{N}, \dots, \lambda_{N}^{N} \in\mathbb{R}$, $v_{1}^{N}, \dots, v_{N}^{N} \in C^{2}(Q)$, $m_{1}^{N}, \dots, m_{N}^{N}$ of the preceding system (11), (12), (13),

$$ \overline{\alpha}^i (x)= \overline{\alpha}^i \bigl(x, D v_i^N(x) \bigr), \quad i=1, \dots, N, $$

(17)

define a feedback that is a Nash equilibrium for all initial positions X∈Q ^N of the control system (1). In addition, for each X∈Q ^N,

$$ \lambda_i^N = J^i \bigl(X, \overline { \alpha}^1,\dots,\overline {\alpha}^N \bigr)= \liminf_{T\to+\infty} \frac{1}{T} E \biggl[\int_0^TL^i \bigl(\overline {X}^i_t, \overline {\alpha}^i \bigl(\overline {X}_t^i \bigr) \bigr) + F^i \bigl( \overline {X}^1_t,\dots,\overline {X}^N_t \bigr) \,dt \biggr], $$

(18)

where $\overline {X}_{t}^{i}$ is the optimal diffusion corresponding to the feedback $\overline {\alpha}^{i}$, that is, obtained by solving

$$ dX^i_t=f^i \bigl(X^i_t, \overline {\alpha}^i \bigl(X^i_t, D_{x^i}v_i( X_t) \bigr) \bigr) \,dt + \sigma ^i \bigl(X^i_t \bigr) \,dW^i_t ,\qquad X^i_0= X^i\in\mathbb{R}^d , \quad i=1,\dots,N. $$

(19)

We now let the number of the players N→∞, assuming that all the players are similar and indistinguishable. We assume that f ⁱ=f, σ ⁱ=σ, L ⁱ=L, and $\overline {\alpha}^{i} = \overline {\alpha}$ for all 1≤i≤N; as a consequence, H ⁱ=H and g ⁱ=g for all 1≤i≤N. In addition, we assume that the criterion F ⁱ only depends on x ⁱ and the empirical density of the other players, namely $\frac{1}{N-1} \sum_{j \neq i } {\delta _{x^{j}} }$ (we might as well use $\frac{1}{N} \sum_{j } {\delta _{x^{j}} }$). The latter dependence is expressed through an operator defined on the set of Borel probability measures P(Q) with values in a bounded set of C ^0,1(Q) (the Banach space of Lipschitz functions on Q), i.e.,

$$ F^i \bigl(x^i, \dots, x^N \bigr) = V \biggl[ \frac{1}{N-1} \sum_{j \neq i } {\delta_{x^j} } \biggr] \bigl(x^i \bigr), $$

(20)

where V:P(Q)→C ^0,1(Q). Recall that P(Q)⊂C(Q)^∗ becomes a topological space when endowed with the topology of weak^∗-convergence, actually a compact topological space by Prokhorov’s theorem. Furthermore, this topology is metrizable, e.g., by the Rubinstein–Kantorovich distance

$$ d \bigl(m, m' \bigr) =\sup \biggl\{ \int_{Q} f(x) \, d \bigl(m - m' \bigr) (x) : f \in C^{0,1}(Q), \mathrm{Lip} (f) \leq1 \biggr\}, $$

(21)

where Lip(f) denotes the minimal Lipschitz constant for f. Just as a little example and for later reference, the reader is invited to verify that

$$ d \Biggl( \frac{1}{N-1} \sum_{j \neq i } { \delta_{x^j} }, \frac{1}{N} \sum_{j=1}^N {\delta_{x^j} } \Biggr) \leq\frac{ \mathrm{diam}(Q)}{N-1}. $$

(22)

In addition, we assume that

$$ V: P(Q) \to C^{0,1}(Q) \subset C(Q) \mbox{ is continuous}, $$

(23)

i.e., V[m _N ] converges uniformly to V[m] in Q as m _N converges weakly^∗ to m (or, equivalently, as d(m _N ,m)→0).

Theorem 2

Under these assumptions, any Nash equilibrium $\lambda_{1}^{N}, \dots, \lambda_{N}^{N} \in\mathbb{R}$, $v_{1}^{N}, \dots, v_{N}^{N} \in C^{2}(Q)$, $m_{1}^{N}, \dots, m_{N}^{N} \in W^{1,p}(Q)$ satisfies the following properties:

• (i) $\{(\lambda_{i}^{N}, v_{i}^{N}, m_{i}^{N})\}_{i, N}$ is relatively compact in $\mathbb{R}\times C^{2}(Q) \times W^{1, p}(Q)$ (for any 1≤p<∞);

• (ii) $\sup_{i,j} (|\lambda_{i}^{N} -\lambda_{j}^{N}| + \|v_{i}^{N} - v_{j}^{N} \| _{C^{2}(Q)} + \|m_{i}^{N} - m_{j}^{N} \|_{\infty} ) \to0$ as N→∞;

• (iii) any limit point (λ,v,m) of $\{(\lambda_{i}^{N}, v_{i}^{N}, m_{i}^{N})\} _{i, N}$ in $\mathbb{R}\times C^{2}(Q) \times W^{1, p}(Q)$ solves the equations

  $$\begin{aligned} &\mathcal{L} {v}+\lambda+ H(x, D {v})= V[ m], \end{aligned}$$

  (24)
  $$\begin{aligned} &\mathcal{L}^*{m}- \mathrm{div} \bigl(g(x, Dv ){m} \bigr)=0, \end{aligned}$$

  (25)
  $$\begin{aligned} &\int_Q{v\,dx} =0, \qquad\int _Q m\,dx =1, \quad m >0. \end{aligned}$$

  (26)

Proof

(i) This point is a consequence of the a priori estimates obtained during the proof of the existence theorem for solutions to the system of equations (11), (12), (13): under the assumptions of the theorem, we may prove that there exists C>0 independent of N such that $|\lambda_{i}^{N}|$, $\|v_{i}^{N}\| _{C^{2, \alpha}(Q)}$, $\|m_{i}^{N}\|_{W^{1, p}(Q) }\le C$; see [1] for the details.

(ii) For all N=1,2,…, let us introduce an auxiliary system of two equations for the unknowns $\lambda^{N} \in\mathbb{R}$, w ^N∈C ²(Q), μ ^N∈W ^1,p(Q),

$$\begin{aligned} &\mathcal{L} {w^N} +\lambda^N +H \bigl(x, D w^N \bigr)= \int_{Q^N} V \Biggl[ \frac{1}{N} \sum_{j=1}^{N}\delta_{x^j} \Biggr] \prod_{j=1}^{N}m_j^N \bigl(x^j \bigr)\,dx^j, \end{aligned}$$

(27)

$$\begin{aligned} &\mathcal{L}^*{\mu^N}- \mathrm{div} \bigl(g \bigl(x, Dw^N \bigr){\mu^N} \bigr) =0, \end{aligned}$$

(28)

$$\begin{aligned} &\int_Q{w^N \,dx} =0, \qquad \int_Q \mu^N\,dx =1, \quad\mu^N >0. \end{aligned}$$

(29)

If we show that the difference of the right-hand sides of the ith equation in (11), abbr. $\mathit{RHS}^{N}_{i}$, and of (27), abbr. RHS ^N, converges to zero uniformly with respect to x∈Q and also with respect to 1≤i≤N as N→∞, taking into account the continuous dependence of the solutions of the Hamilton–Jacobi–Bellman equation on the right-hand side and of the solutions of the Kolmogorov–Fokker–Planck equations on the vector fields g ⁱ and the continuous dependence of g ⁱ on $Dv_{i}^{N}$, we may conclude that

$$\lim_{N\to\infty} \sup_{i} \bigl( \bigl\vert \lambda_i^N -\lambda^N \bigr\vert+ \bigl \Vert v_i^N - w^N \bigr \Vert _{C^2(Q)} + \bigl \Vert m_i^N - \mu^N \bigr \Vert _{\infty} \bigr) \to0, $$

which, in turn, by the triangle inequality, implies assertion (ii). But this follows easily since

$$\mathit{RHS}_i^N - \mathit{RHS}^N = \int _{Q^N}{ \Biggl( V \biggl[ \frac{1}{N-1} \sum _{j\neq i}\delta_{x^j} \biggr]- V \Biggl[ \frac{1}{N} \sum _{j=1}^{N}\delta_{x^j} \Biggr] \Biggr) \prod_{j=1}^{N}m_j^N \bigl(x^j \bigr)\,dx^j } $$

and, by (22),

$$\bigl \Vert \mathit{RHS}_i^N - \mathit{RHS}^N \bigr \Vert _{\infty} \le\omega \biggl( d \biggl( \frac{1}{N-1} \sum _{j \neq i } {\delta_{x^j} }, \frac{1}{N} \sum_{j } {\delta_{x^j} } \biggr) \biggr) \le\omega \biggl(\frac{ \mathrm{diam}(Q)}{N-1} \biggr), $$

where ω is the modulus of continuity of V, i.e.,

$$\omega(h) = \sup_{d(m,m') \le h } \bigl \Vert V[m] -V \bigl[m' \bigr] \bigr \Vert _{\infty}, \quad h >0. $$

Since P(Q) is compact, the continuous operator V is actually uniformly continuous, that is, ω(h)→0 as h→0⁺. Therefore, $\sup_{i}\|\mathit{RHS}_{i}^{N} - \mathit {RHS}^{N} \| _{\infty} \to0$ as N→∞.

(iii) The proof of this assertion requires a law of large numbers and precisely the following theorem of Hewitt and Savage [4].

Theorem 3

$$ \lim_{N\to\infty} \int_{Q^N}V \Biggl[ \frac{1}{N} \sum_{j=1}^{N} \delta_{x^j} \Biggr] \prod_{j=1}^{N} dm \bigl(x^j \bigr) =V[m] $$

(30)

for all V∈C(P(Q)) and m∈P(Q).

The proof of this fact can also be found in [2], or, even better, the reader can reconstruct the relatively easy proof himself: just noting that by linearity and density (appeal to the Stone–Weierstrass theorem here) one reduces to the case of continuous functions V on P(Q) of the form

$$V[m] = \int_{Q^k} \varphi \bigl(y^1,\dots, y^k \bigr) \prod_{l=1}^{k}dm \bigl(y^l \bigr)\quad\mbox{for } m \in P(Q), $$

where $k \in\mathbb{N}$, φ∈C(Q ^k), and this case can be dealt with by direct calculations.

In order to conclude the proof of Theorem 2 (iii), let (λ,v,m) be a limit point of $\{(\lambda_{i}^{N}, v_{i}^{N}, m_{i}^{N})\}_{i, N}$ in $\mathbb{R} \times C^{2}(Q) \times W^{1, p}(Q)$. The goal is to show that (λ,v,m) satisfy (24), (25), (26). Since $(\lambda_{i_{N}}^{N}, v_{i_{N}}^{N}, m_{i_{N}}^{N}) \to(\lambda, v, m)$ (up to a subsequence and where 1≤i _N ≤N), it is sufficient to show that $\|\mathit{RHS}_{i_{N}}^{N} - V[m]\|_{\infty} \to0$ as N→∞. Since we already saw that $\|\mathit{RHS}_{i_{N}}^{N} - \mathit{RHS}^{N}\| _{\infty} \to 0$ as N→∞ during the proof of (ii), it suffices to show that ∥RHS ^N−V[m]∥_∞→0 as N→∞. On one hand, we have

$$ \Biggl \Vert \int_{Q^N}V \Biggl[\frac{1}{N} \sum _{j=1}^{N} \delta_{x^j} \Biggr] \prod_{j=1}^{N} dm \bigl(x^j \bigr) -V[m] \Biggr \Vert _{\infty} \to0 \quad\mbox{as } N\to\infty; $$

(31)

the pointwise convergence is a consequence of the Hewitt and Savage theorem, Theorem 3, and the convergence is actually uniform for the sequence of functions that are equicontinuous since V[P(Q)] is compact in C(Q).

On the other hand, we have¹

So, if the operator V is Lipschitz continuous with Lipschitz constant L, we deduce

and therefore, taking also into account (ii) and the triangle inequality, we conclude that

$$ \Biggl \Vert \mathit{RHS}^N - \int _{Q^N}V \Biggl[\frac{1}{N} \sum_{j=1}^{N} \delta_{x^j} \Biggr] \prod_{j=1}^{N} dm \bigl(x^j \bigr) \Biggr \Vert _{\infty} \to0 \quad\mbox{as } N \to\infty. $$

(32)

Actually, the convergence above holds for any continuous V:P(Q)→C(Q); one first proves the pointwise convergence by a density argument and then deduces that the convergence is in fact uniform by taking into account the fact that the sequence of functions at hand is equicontinuous (in the compact set Q). By (31), (32), and the triangle inequality we deduce that ∥RHS ^N−V[m]∥_∞→0 as N→∞. This concludes the proof as we already pointed out.  □

3 Several Homogeneous Populations of Players

Now we derive the mean field games equations corresponding to a situation with several groups or populations of players. Each population consists of a large number of identical players, but the characteristics of the players vary from one population to the other. Let us formalize this for the case of only two populations. (Of course, we limit ourselves to this case in order to keep the notation short, but everything extends immediately to the case of an arbitrary number of populations.) We consider again the class of N-player games introduced in the previous section and continue to keep the same notation. We make the same assumptions on f ⁱ, σ ⁱ, L ⁱ, F ⁱ, $\overline {\alpha}^{i}$, and H ⁱ, i=1,…,N, as detailed before Theorem 1, so that this theorem holds. Now we assume in addition that the totality of the N players consists of two groups of players, each of them having N ₁ and N ₂ individuals, respectively, and thus N ₁+N ₂=N. We assume that both populations of players are homogeneous, that is, the individuals composing each of them are identical and indistinguishable. Mathematically, this means that we are assuming that f ⁱ=f ¹, σ ⁱ=σ ¹, L ⁱ=L ¹, $\overline {\alpha}^{i} = \overline {\alpha}^{1}$ for all 1≤i≤N ₁ and $f^{i} = f^{N_{1}+1}$, $\sigma^{i} = \sigma^{N_{1}+1}$, $L^{i}=L^{N_{1}+1}$, $\overline {\alpha}^{i} = \overline {\alpha}^{N_{1}+1}$ for all N ₁+1≤i≤N. As a consequence, $\mathcal{L}^{i} = \mathcal{L}^{1}$, H ⁱ=H ¹, g ⁱ=g ¹ for all 1≤i≤N ₁, and $\mathcal{L}^{i} = \mathcal{L}^{N_{1}+1}$, $H^{i} = H^{N_{1}+1}$, $g^{i} = g^{N_{1}+1}$ for all N ₁+1≤i≤N. It is convenient for us to make the following change of notation: we denote by f ¹, σ ¹, L ¹, $\mathcal{L}^{1}$, H ¹, g ¹, $\overline {\alpha}^{1}$ the expressions f ⁱ, σ ⁱ, L ⁱ, $\mathcal{L}^{i}$, H ⁱ, g ⁱ, $\overline {\alpha}^{i}$, respectively, for 1≤i≤N ₁, and we denote by f ², σ ², L ², $\mathcal{L}^{2}$, H ², g ², $\overline {\alpha}^{2}$ the expressions f ⁱ, σ ⁱ, L ⁱ, $\mathcal{L}^{i}$, H ⁱ, g ⁱ, $\overline {\alpha}^{i}$, respectively, for N ₁+1≤i≤N. For the cost criteria, we assume

(33)

(34)

where

$$ V^i: P(Q)\times P(Q) \to C^{0}(Q), \quad i=1,2,\quad \mbox{are continuous operators}. $$

(35)

We also assume the even more technical hypothesis that the images of the operators V ⁱ, i=1,2, are contained in a bounded subset of the Banach space C ^0,1(Q). This last assumption serves the purpose of obtaining a priori estimates for the solutions of the corresponding systems of Hamilton–Jacobi–Bellman and Fokker–Planck equations and thus proving the existence of Nash equilibria. Furthermore, by this assumption, the estimates are independent of the number of the players of each population N ₁, N ₂.

Under these assumptions, the analogue of Theorem 2 reads as follows.

Theorem 4

All Nash equilibria $\lambda_{1}^{N}, \dots, \lambda_{N}^{N} \in\mathbb {R}$, $v_{1}^{N}, \dots, v_{N}^{N} \in C^{2}(Q)$, $m_{1}^{N}, \dots, m_{N}^{N} \in W^{1,p}(Q)$ satisfy the following properties:

• (i) $\{(\lambda_{i}^{N}, v_{i}^{N}, m_{i}^{N})\}_{i, N}$ is relatively compact in $\mathbb{R}\times C^{2}(Q) \times W^{1, p}(Q)$ (for any 1≤p<∞);

• (ii)
  $$\begin{aligned} &\sup_{1\le i,j\le N_1} \bigl( \bigl\vert\lambda_i^N - \lambda_j^N \bigr\vert+ \bigl \Vert v_i^N - v_j^N \bigr \Vert _{C^2(Q)} + \bigl \Vert m_i^N - m_j^N \bigr \Vert _{\infty} \bigr) \to0 \quad\textit{as } N_1 \to\infty, \\ &\sup_{N_1+1 \le i,j\le N} \bigl( \bigl\vert\lambda_i^N - \lambda_j^N \bigr\vert+ \bigl \Vert v_i^N - v_j^N \bigr \Vert _{C^2(Q)} + \bigl \Vert m_i^N - m_j^N \bigr \Vert _{\infty} \bigr) \to0 \quad\textit{as } N_2 \to\infty; \end{aligned}$$

• (iii) Let (λ ₁,v ₁,m ₁) be a limit point of $\{(\lambda_{i}^{N}, v_{i}^{N}, m_{i}^{N})\}_{ 1\le i \le N_{1}, N}$ in $\mathbb{R}\times C^{2}(Q) \times W^{1, p}(Q)$, and (λ ₂,v ₂,m ₂) be a limit point of $\{(\lambda_{i}^{N}, v_{i}^{N}, m_{i}^{N})\}_{ N_{1} +1 \le i \le N, N}$ in $\mathbb{R}\times C^{2}(Q) \times W^{1, p}(Q)$. Then (λ _i ,v _i ,m _i ), i=1,2, solve the equations

  $$\begin{aligned} &\mathcal{L}^i {v_i}+ \lambda_i + H^i(x, D v_i)= V^i[m_1, m_2], \end{aligned}$$

  (36)
  $$\begin{aligned} &{\mathcal{L}^i}^*{m_i}- \mathrm{div} \bigl(g^i(x, D v_i){m_i} \bigr)=0, \end{aligned}$$

  (37)
  $$\begin{aligned} &\int_Q{v_i\,dx} =0, \qquad \int_Q m_i\,dx =1, \quad m_i >0, \ i=1,2. \end{aligned}$$

  (38)

Proof

It is very similar to the proof of Theorem 2. In any case, we provide the details for the sake of completeness. We need to introduce, for all N=1,2,…, an auxiliary system of four equations for the unknowns $\bar{\lambda}_{i}^{N} \in\mathbb{R}$, $w^{N}_{i} \in C^{2}(Q)$, $\mu_{i}^{N} \in W^{1,p}(Q)$, i=1,2,

(39)

(40)

(41)

We need also the following version of the Hewitt and Savage theorem.

Theorem 5

For all V∈C(P(Q)×P(Q)) and m ₁, m ₂∈P(Q),

$$\begin{aligned} &\lim_{N_1, N_2 \to\infty}\int_{Q^{N_1+N_2}} V \Biggl[ \frac{1}{N_1}\sum_{j =1}^{N_1} \delta_{x^j}, \frac{1}{N_2} \sum_{j = N_1 +1 }^{N } \delta_{x^j} \Biggr] \prod_{j=1}^{N_1} dm_1 \bigl(x^j \bigr) \prod_{j=N_1+ 1}^{N_1+N_2} d m_2 \bigl(x^j \bigr) \\ &\quad= V[m_1, m_2]. \end{aligned}$$

(42)

The proof of this result is immediate since by density and linearity it reduces to showing the result for continuous maps of the form V[m ₁,m ₂]=V ₁[m ₁]V ₂[m ₂] for all m ₁, m ₂∈P(Q), where V _i ∈C(P(Q)), i=1,2. But this is a direct consequence of Theorem 3 and Fubini’s theorem.

Assertion (i) holds precisely for the same reasons why assertion (i) of Theorem 2 holds.

Assertion (ii) is proved similarly to assertion (ii) of Theorem 2: note that P(Q)×P(Q) is a compact metric space with a distance, e.g.,

$$d_1 \bigl( (m_1, m_2),\, \bigl(m'_1, m'_2 \bigr) \bigr) = d \bigl(m_1, m_1' \bigr) + d \bigl(m_2, m_2' \bigr) $$

for all $(m_{1}, m_{2}), (m_{2}, m_{2}') \in P(Q)\times P(Q)$, and the continuous operators V ¹, V ² are in fact uniformly continuous, and therefore have infinitesimal moduli of continuity at zero. For example, the modulus of continuity of V ¹ is

$$\omega^1(h) = \sup_{ d_1 ((m_1, m_2), (m'_1, m'_2) ) \leq h} \bigl \Vert V^1[m_1, m_2] -V^1 \bigl[m'_1, m'_2 \bigr] \bigr \Vert _{\infty} $$

for all h>0, and ω ¹(h)→0 as h→0. If we denote by $\mathit{RHS}^{N}_{i}$ the right-hand side of the ith equation in (36) and by RHS ^N,1 the right-hand side of the first auxiliary equation (i=1) in (39), for any 1≤i≤N ₁, we have

$$ \bigl \Vert \mathit{RHS}_i^N - \mathit{RHS}^{N, 1} \bigr \Vert _{\infty} \le\omega^1 \biggl(\frac{ \mathrm{diam}(Q)}{N_1-1} \biggr) $$

(43)

because

$$ d_1 \Biggl( \Biggl(\frac{1}{N_1}\sum _{ j=1}^{N_1}\delta_{x^j}, \frac{1}{N_2} \sum_{j = N_1 +1 }^{N } \delta_{x^j} \Biggr) , \Biggl(\frac{1}{N_1 -1}\sum _{\substack{ 1\le j \le N_1 \\j \neq i }} \delta_{x^j}, \frac{1}{N_2} \sum _{j = N_1 +1 }^{N } \delta_{x^j} \Biggr) \Biggr) \le\frac{ \mathrm{diam}(Q)}{N_1-1}. $$

(44)

The continuity of the solutions of the Hamilton–Jacobi equations with respect to the right-hand side implies that

$$ \sup_{1\le i\le N_1} \bigl( \bigl\vert\lambda_i^N -\bar{\lambda}^N_1 \bigr\vert+ \bigl \Vert v_i^N - w^N_1 \bigr \Vert _{C^2(Q)} \bigr) \to0 \quad\mbox{as } N_1\to \infty. $$

(45)

The continuous dependence of the solutions of the Kolmogorov–Fokker–Planck equations (37) with respect to the vector field g ⁱ and the continuity of the vector field g ⁱ with respect to $Dv_{i}^{N}$ imply

$$ \sup_{1\le i\le N_1} \bigl \Vert m_i^N - \mu^N_1 \bigr \Vert _{\infty} \to0 \quad\mbox{as } N_1 \to\infty. $$

(46)

By (45), (46), and the triangle inequality, the first part of (ii) follows. Of course, the second part of (ii) has an analogous proof.

Let (λ ₁,v ₁,m ₁) and (λ ₂,v ₂,m ₂) be as prescribed in iii), that is, let $(\lambda_{i}^{N}, v_{i}^{N}, m_{i}^{N}) \to(\lambda_{1}, v_{1}, m_{1})$, 1≤i≤N ₁, in $\mathbb{R}\times C^{2}(Q) \times W^{1, p}(Q)$ as N ₁→∞ (up to a subsequence, i itself depends on N ₁, but we do not write this in order to keep the notation short); let also $(\lambda_{i}^{N}, v_{i}^{N}, m_{i}^{N}) \to(\lambda_{2}, v_{2}, m_{2})$, N ₁+1≤i≤N, in $\mathbb{R}\times C^{2}(Q) \times W^{1, p}(Q)$ as N ₂→∞ (up to a subsequence). In order to show that, e.g., (λ ₁,v ₁,m ₁) solves system (36), (37), (38) for i=1, we need to prove that $\|\mathit{RHS}^{N}_{i} - V^{1}[m_{1}, m_{2}]\|_{\infty} \to0$ as N ₁, N ₂→∞, and by (43) it suffices to show that ∥RHS ^N,1−V ¹[m ₁,m ₂]∥_∞→0 as N ₁, N ₂→∞, But, by Theorem 5 and with a similar reasoning as in the proof of Theorem 2(iii), we have

$$\begin{aligned} &\Biggl \Vert \int_{Q^{N_1+N_2}} V^1 \Biggl[ \frac{1}{N_1}\sum_{j =1}^{N_1}\delta _{x^j}, \frac{1}{N_2} \sum_{j = N_1 +1 }^{N } \delta_{x^j} \Biggr] \prod_{j=1}^{N_1} dm_1 \bigl(x^j \bigr) \prod_{j=N_1+ 1}^{N_1+N_2} d m_2 \bigl(x^j \bigr) \\ &\quad{} - V^1[m_1, m_2] \Biggr\|_{\infty} \to0 \end{aligned}$$

as N ₁, N ₂→∞, so definitely we need to show that

$$\Biggl \Vert \mathit{RHS}^{N,1} - \int_{Q^{N}}V^1 \Biggl[\frac{1}{N_1}\sum_{j =1}^{N_1} \delta_{x^j}, \frac{1}{N_2} \sum_{j = N_1 +1 }^{N } \delta_{x^j} \Biggr] \prod_{j=1}^{N_1} dm_1 \bigl(x^j \bigr) \prod_{j=N_1+ 1}^{N_1+N_2} d m_2 \bigl(x^j \bigr) \Biggr \Vert _{\infty} \to0 $$

as N ₁, N ₂→∞. Let us write simply V ¹ for $V^{1} [\frac{1}{N_{1}}\sum_{j =1}^{N_{1}}\delta_{x^{j}}, \frac{1}{N_{2}} \sum_{j = N_{1} +1 }^{N } \delta_{x^{j}} ]$ in order to keep the notation short. Let us write

(47)

The term in the first parenthesis in the right-hand side of the equality above can be written as follows:

In the same fashion, we have

Let us first assume that the operator V ¹ is Lipschitz continuous with Lipschitz constant L. The last two identities, together with (47), the triangle inequality, and (44), yield

which, in by of assertion (ii) and triangle inequality, goes to zero as N ₁, N ₂→∞. Actually, the said convergence holds for any continuous V ¹:P(Q)×P(Q)→C(Q); the Lipschitzianity assumption is dropped by arguing in the manner as at the end of the proof of Theorem 2. Thus, by our preceding considerations, (λ ₁,v ₁,m ₁) solves Eqs. (36), (37), (38) for i=1; by an identical reasoning (the role of V ¹ being played by V ²), one shows that (λ ₂,v ₂,m ₂) solves (36), (37), (38) for i=2.  □