Abstract
We identify structural assumptions which provide solvability of the Nash system arising from a linear-quadratic closed-loop game, with stable properties with respect to the number of players. In a setting of interactions governed by a sparse graph, both short-time and long-time existence of a classical solution for the Nash system set in infinitely many dimensions are addressed, as well as convergence to the solution to the respective ergodic problem as the time horizon goes to infinity; in addition, equilibria for the infinite-dimensional game are shown to provide \(\epsilon \)-Nash closed-loop equilibria for the N-player game. In a setting of generalized mean-field type (where the number of interactions is large but not necessarily symmetric), directly from the N-player Nash system estimates on the value functions are deduced on an arbitrary large time horizon, which should pave the way for a convergence result as N goes to infinity.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
1 Introduction
Consider a stochastic differential game with N players, indexed by \(i \in [[N]] :=\{ 0,\dots , N-1\}\), where the state \(X^i\) of the i-th player evolves according to the \({\mathbb {R}}^d\)-valued SDE
His/her cost, in the fixed time horizon [0, T], is given by
for some \(F^i = f^i \otimes I_d\) and \(G^i = g^i \otimes I_d\), with \(f^i \in C^0([0,T];\mathscr {S}(N))\) and \(g^i \in \mathscr {S}(N)\).Footnote 1 The \(B^i\)’s are independent \({\mathbb {R}}^d\)-valued Brownian motions, and the \(\alpha ^i\)’s are closed-loop controls in feedback form, that is \(\alpha ^i = \alpha ^i(t,X_t)\).
It is known (see for instance [12, Section 2.1.4], or [22]) that the value functions of the players, \(u^i = u^i(t,x)\) with \(i \in [[N]]\), \(t \in [0,T]\) and \(x = (x^0,\dots ,x^{N-1}) \in ({\mathbb {R}}^d)^N\) solve the so-called Nash system of Hamilton-Jacobi PDEs
where \({\bar{F}}^i = \frac{1}{2} \langle F^i\cdot ,\cdot \rangle \), \({\bar{G}}^i = \langle G^i \cdot ,\cdot \rangle \) and \(i \in [[N]]\). The equilibrium feedbacks are then given by \(\alpha ^i = -D_i u^i\). Since the dynamic of each player is linear, and the costs are quadratic in the state and control variables, it is well-known that the previous system can be recast into a system of ODEs of Riccati type, by making the ansatz that \(u^i\) are quadratic functions of the states. Here, we look for conditions on the solvability of such system, and we focus in particular on properties that are stable as the number of players goes to infinity, with the aim of addressing the limit problem with infinitely many players, whenever possible. The Laplacian appears in (1.2) by the presence of the independent noises \(B^i\), but it will not actually play any relevant role in our analysis. We keep it since the purely deterministic system is not known to be well posed, beyond the linear quadratic setting.
The study of differential games with many players has enjoyed a rapid development in the last two decades, since the introduction of the theory of Mean Field Games (MFGs) independently by Lasry and Lions [31] and Huang, Caines and Malhamé [25]. MFG theory provides an effective paradigm for studying dynamic games with many players that are both symmetric (that is, indistinguishable) and negligible. In such framework, one has a limit model, involving the equilibrium between a typical player versus a mass of agents, which is decentralised and symmetric: in fact, it is realised by a feedback of the state of the player only, it is identical for all the players, and it can be computed just by observing the population of players at the initial time. We refer to the book [12] and the lecture notes [11] for a recent account of the theory of MFGs.
If the MFG assumption is not fullfilled, that is, \(F^i\) and \(G^i\) do not depend just on the empirical measure of the other players, one enters into the broader framework of network games. There has been in the last few years an increasing interest in the understanding of the large population limits of equilibria among players whose interactions are described by graphs. Roughly speaking, when the number of interactions is “large”, the MFG description, or a suitable generalisation based on the notion of graphon effectively characterizes large population limits: see for instance [1, 5, 8, 30] and references therein. See also [16] for the analysis of a model on Erdős-Rényi graphs.
On the other hand, when the underlying network structure is sparse, very few results are available in the literature. In the dense regime, one expects all the players to have an individual negligible influence on a given player, because their running costs involve cumulative functions of many variables, while in sparse regimes, most of the players should have a small impact just because they are “far” with respect to the graph distance. This mechanism of independency between players that are far in the graph has been first observed in [29], by means of correlation estimates.
The framework addressed in [29] is somehow similar to ours. The linear-quadratic setting is considered, and under some symmetry assumptions on the underlying graph, Nash equilibria are computed explicitly (exploiting also the running costs \(\overline{F}^i\) to be identically zero, and \({\overline{G}}^i\) to have a specific structure). Then, a probabilistic information on the covariance of any two players’ equilibrium state processes is derived. A main goal in our work is to derive an analogous information in more analytic terms; that is, we wish to quantify the influence of the j-th player on the i-th one by estimating \(D_j u^i\), with the perspective of developing some ideas that could be applied also beyond the linear-quadratic framework.
We now describe our results more in detail. The first part of the paper is devoted to analyse a sort of sparse regime with a special structure, namely shift-invariance, where basically \(f^i\) and \(g^i\) coincide with \(f^{i-1}\) and \(g^{i-1}\), respectively, after the permutation of variables \(x^i \mapsto x^{i-1}\). Most importantly, we assume players to have nearsighted interactions, that means
where \((\beta _k)_k\) is a suitable sequence in \(\ell ^1(\mathbb {Z})\). Since \(\beta _{k-i}\) decays as \(|k-i|\rightarrow \infty \), this means that \(f^i_{hk}\), which quantifies the influence of the h-th and k-th player on the i-th one, decreases as \(|h-i|\) and \(|k-i|\) increase. A prototypical example is given by the bi-directed chain, where \(f^{i}_{hk}\) is a sparse matrix with zero entries except for \(|h-i| \le 1\) and \(|k-i| \le 1\), so that the i-th player cost depends only on the \((i+1)\)-th and the \((i-1)\)-th state. See also [20] for results on models that build upon a similar structure.
The first statement we get, which sums up Theorems 2.13 and 2.14, is the following.
Theorem
There exists \(T^* > 0\) such that if \(T \le T^*\) then for any \(N \in \mathbb {N}\cup \{\infty \}\) there exists a smooth solution to system (1.2) such that, for any i, j and \(m \in \mathbb {N}\),
Note in particular that we get existence of a smooth classical solution to the infinite-dimensional Nash system (1.2) with \(N=\infty \), \(i \in \mathbb {Z}\). We stress that the key issue in the analysis of our problem is that, despite the cost functions \(f^i, g^i\) may depend on very few variables, the system itself is strongly coupled by the transport terms \(\sum _{j\ne i} D_j u^j D_j u^i\), which become in fact series when \(N = \infty \). The closed-loop structure of equilibria forces the equilibrium feedbacks \(\alpha ^j = -D_j u^j\) to be strongly “nonlocal”, that is, they depend on the full vector state x. Hence, decay estimates on \(D_j u^i\) which are stable as N increases are crucial to pass to the limit \(N \rightarrow \infty \). These are obtained here by a careful choice of \(\beta \): below, we will use the terminology self-controlled for the discrete convolution, or briefly c-self-controlled (where the “c” stands for “convolution”), that fits well with the structure of cyclic discrete convolution appearing in our problem. From the game perspective, the equilibrium feedbacks \(\alpha ^j\) turn out to be almost “local”, in the sense that the influence of “far” players is still negligible in the sense explained above. In other words, despite the strong coupling given by the full information structure of closed-loop equilibria, the property of “unimportance of distance players” is observed (see Remark 2.15 for further discussions).
While the shift-invariance condition can be actually dropped (see Sect. 2.4), one of the main restrictions of the previous result is that it guarantees short-time existence only. Note that, even with a finite number of players N, Riccati systems may fail to have long time solutions in general; therefore we look for further conditions on \(f^i\) and \(g^i\) such that existence holds for any time horizon T, and independently on N. To achieve this goal, we strengthen the previous assumption on nearsighted interactions, that now become of strongly gathering type, and require further directionality conditions. Section 2.5 contains precise definition of this notion and examples. The main existence result is stated in Theorem 2.26. Here, we exploit the possibility to relate a solution to system (1.2) to a flow of generating functions, which works well when \(N=\infty \) but has no clear adaption to the finite N setting.
Within the special setting of systems with cost of strongly gathering type and directionality, we are able to push further our analysis and study the long time limit \(T \rightarrow \infty \), that is, we show that the value functions \(u^i\) converge to solutions of the ergodic problem as the time horizon goes to infinity. To complete this program, estimates on solutions of the Riccati system are obtained at the level of the game with \(N=\infty \) players, uniformly with respect to T. The main result of this part of the work is stated by Theorem 2.42.
We conclude this second part by showing that equilibria of the infinite-dimensional game provide \(\epsilon \)-Nash closed-loop equilibria of suitable N player games, see Sect. 2.6.
In the last part of the work, we come back to the dense regime, and work without any symmetry assumption (like shift-invariance). Our goal is again to deduce some estimates on the Riccati system that do not depend on the number of players, under a mean-field-like condition
and the same for \(g^i\); the previous assumption imposes a smallness condition on the coefficients which is typical of the Mean-Field setting, that is when costs depend on the empirical measure of the players, though here we do not require any symmetry; see Remark 3.2 for further comments on the connection between this condition and the Mean-Field setting. It is crucial to observe that in this dense setting there is no hope to have a limit system of the same form. Indeed, in the sparse case we had an \(\ell ^1\) control of coefficients independent of N (given by \(\beta \)), so that the series \(\sum _{j\ne i} D_j u^j D_j u^i\) carries to the limit by dominated convergence, while now a dominated convergence cannot performed. In fact, at least in the symmetric case, where the costs are of the form \(F^i(x) = V(x^i, \frac{1}{N-1}\sum _{j\ne i}\delta _{x^j}\)), the correct limiting object is the so-called Master Equation, which has been the object of the seminal work [10]. The convergence of \(u^i\) to a limit function defined on the space of probability measures has been shown under monotonicity assumptions of \(f^i\), and its regularity is obtained as a consequence of the stability properties of the MFG system, which characterizes the limit model.
Here, we wish to deduce the estimates on \(u^i\) that allow for a passage to the limit \(N\rightarrow \infty \) directly on the Riccati system (that is, on the Nash system). For short time horizon, one can basically reproduce the same approach of the previous section (cf. Theorem 2.17). To achieve stability for arbitrary values of T, we impose a bound from below on the matrices \((f^i_{ij})_{ij}\) and \((g^i_{ij})_{ij}\):
If \(\kappa \) is small enough, then the existence of a solution to the Nash system is guaranteed for large T, by a mechanism that produces an analogous bound from below on the matrix \((D_{ij} u^i)_{ij}\). Recalling that the equilibrium feedbacks are given by \(\alpha ^j = - D_j u^j\), this is equivalent to the one-sided Lipschitz condition
Our main estimate is contained in Theorem 3.1. Interestingly, the monotonicity (or mild non-monotonicity) condition (1.3) generates the nice structural property (1.4) of the equilibrium drift vector \((-D_ju^j)_j\); it is worth observing that, when \(\kappa = 0\) and in the symmetric MFG case \(f^i(x) = V(x^i, (N-1)^{-1} \sum _{j \ne i} \delta _{x^j})\), (1.3) is “almost” equivalent (that is, up to a correction term of order 1/N) to the displacement monotonicity condition used in [23] to get well-posedness of the Master Equation.
We believe that the estimates obtained here, combined with propagation of chaos arguments, allow for a convergence result of (closed loop) equilibria in the \(N\rightarrow \infty \) limit. We recall that the lack of such estimates so far has motivated the investigation of regularity of solutions to the Master equation, that allows to bypass the difficult analysis of the Nash system; hence they can be of interest by themselves. Analogous estimates for solutions to the Nash system beyond the Linear-Quadratic setting, and their application to the convergence problem will be discussed in the forthcoming work [14].
We now discuss some some further related references. In [3, 19], the convergence problem of open-loop equilibria in symmetric N-person linear-quadratic games is addressed, while [17] deals with the selection problem for models without uniqueness. See also [4] for a convergence result in a finite state model. Linear-quadratic MFGs are studied in several works, see for instance [6, 7, 24, 32, 33] and references therein. Further contributions on the convergence problem are contained in [9, 18, 21, 27, 28], while recent results on the structure of many players cooperative equilibria can be found in [13, 15, 26].
As a final comment, despite trying to pursue the full generality of the linear-quadratic differential game framework, we carefully analysed here a few case studies, with the aim of highlighting some properties that could be investigated beyond the linear-quadratic setting, and directly on the Nash system of PDEs. With this purpose, we tried to avoid as much as possible the use of explicit formulas. Even though these were quite crucial to study the long time behaviour in the sparse regime, there are a few takeaways here that can be starting points for further investigations. On one hand, we are addressing the short-time existence of the infinite-dimensional Nash system with non-quadratic cost functions, by estimating the derivatives of \(u^i\) by means of the coefficients \(\beta \) constructed here. On the other hand, analogous bounds on \(u^i\) in the mean-field-like setting, again by working directly on the Nash system and without making use of limit objects such as the Master Equation, will appear in [14].
2 Shift-Invariant Games
2.1 General Setting
For \(a \in \mathbb {Z}\) and \(b \in \mathbb {Z}_+\), we will use the notation \([a]_b\) to identify the unique natural number \(r \in [[b]]\) such that \(r = a \ \textrm{mod}\,b\). The entries of a \(N \times N\) matrix will be indexed over [[N]]. We will denote by \(L^{(N)} \in \mathscr {S}(N)\) the lower shift matrix \(\textrm{mod}\,N\), defined by
where \(\delta \) is the Kronecker symbol.
Our main assumption in this section, on the structure of the running cost, is that \(f^i\) and \(g^i\) are shift-invariant, in the sense of the following definition.
Definition 2.1
A collection of matrices \((M^i)_{i\in [[N]]} \subset \mathscr {S}(N)\) is shift-invariant if
Note that this is equivalent to requiring that for all \(i \in [[N]]\) and \(x \in {\mathbb {R}}^N\),
Example 2.2
A very basic shift-invariant case is that with \(g^0 = 0\) and
with
that is, by (SI),
When \(\ell = N\), the cost is actually of Mean-Field type, that is, it penalizes the deviation of the private state \(X^i\) from the mean of the vector X.
Here \(f^0\) induces an underlying directed circulant graph structure \(G_\ell \) to the problem; indeed, by assumption (SI)
can be considered as the asymmetric and circulant adjacency matrix of \(G_\ell \), so that (2.2) reads
The same is true in the more general case when
here the \(w_j\)’s are regarded as normalized weights, and we have
which generalizes (2.3).
Note finally that a sort of “directionality” is encoded in the above examples, that is, each player i is affected by the “following” ones \(j > i\) in the chain. This is not yet important at the current stage, namely we may allow for
It is only from Sect. 2.5 that \(m = 0\) will be required.
2.2 The Evolutive Infinite-Dimensional Nash System
The main object of our study is the Nash system
When \(N = \infty \), we need to be a bit careful about the notion of solution. In this case, \(x \in \mathcal {X} = \ell ^\infty (\mathbb {Z};{\mathbb {R}}^d)\). Then, we mean that (2.6) admits a classical solution in the following sense.
Definition 2.3
A sequence of \({\mathbb {R}}\)-valued functions \((u^i)_{i\in \mathbb {Z}}\) defined on \([0,T] \times \mathcal {X}\) is a classical solution to the Nash system (2.6) on \([0,T] \times \mathcal {X}\) if the following hold:
-
(S1)
each \(u^i\) is of class \(C^1\) with respect to \(t \in (0,T)\) and \(C^2\) with respect to \(x \in \mathcal {X}\), in the Fréchet sense;
-
(S2)
for each \(i \in \mathbb {N}\), the Laplacian series \(\Delta u^i = \sum _j \Delta _j u^i\) and the series \(\sum _{j\ne i} D_j u^j D_j u^i\) uniformly converge on all bounded subsets of \([0,T] \times \mathcal {X}\);
-
(S3)
system (2.6) is satisfied pointwise for all \((t,x) \in (0,T) \times \mathcal {X}\);
-
(S4)
\(u^i(T,\cdot ) = {\bar{G}}^i\) for all \(i \in \mathbb {N}\).
Remark 2.4
Here we made a choice of Banach space, namely \(\mathcal {X} = \ell ^\infty (\mathbb {Z}; {\mathbb {R}}^d)\). This seems a natural choice for a twofold reason. First, it is the largest \(\ell ^p\) space contained in the limit set \(({\mathbb {R}}^d)^\infty \cong ({\mathbb {R}}^d)^{\mathbb {Z}}\), so it seems to provide a quite general setting for a Nash system for infinitely many players. Second, in a linear-quadratic setting as the one we are considering, it is fairly clear that (see the upcoming discussion about the standard ansatz \(u^i(t,x) = \frac{1}{2} \langle (c^i(t) \otimes I_d)x,x\rangle _{{\mathbb {R}}^{Nd}} + \eta ^i(t)\)) the uniform convergence of \(\Delta u^i\) when \(N = \infty \) is strictly related to \((c^i_{jj})_{j \in \mathbb {Z}}\) being summable; therefore the variable x is expected to live in the dual space of \(\ell ^1\), that is \(\ell ^\infty \). This is even more evident if one considers the case (and the results we present can indeed be adapted to it) of a more general diffusion such as the one arising from common noise, \(\sum _j \Delta _j u^i + \beta \sum _{jk} {{\,\textrm{tr}\,}}(D^2_{jk} u^i)\).
As it is customary in linear-quadratic N-player games, we look for solutions of the form
for some functions \(c^i :[0,T] \rightarrow \mathscr {S}(N)\) such that \(c^i(T) = g^i\) and \(\eta ^i :[0,T] \rightarrow {\mathbb {R}}\) which vanish at T. We have \(D_j u^i(t,x) = {e^j}{}^\textsf{T}(c^i(t) \otimes I_d) x\), where \(e^j = e_j \otimes I_d\), \(\{e_j\}_{j=0}^{N-1}\) being the canonical basis of \({\mathbb {R}}^N\). Hence, from (2.6) we obtain
that is,
As this must hold for all \(x \in {\mathbb {R}}^{Nd}\), it follows that
and
as \(x^\textsf{T}A x = 0\) for all x if and only if \(A + A^\textsf{T}= 0\) for any square matrix A.
Now, given the shift invariance of \(f^i\) and \(g^i\), one expects a solution to (2.8) to enjoy the same property, hence we look for a solution such that
for some \(c :[0,T] \rightarrow \mathscr {S}(N)\). Clearly, this makes \(\eta ^i\) independent of i, as we will have \(\eta ^i = \eta :=\int _\cdot ^T {{\,\textrm{tr}\,}}c\). By plugging (2.9) into (2.8) and letting \(i=0\) one obtains the following system of ODEs for the entries of c:
where \(f :=f^0\) and \(g :=g^0\).
As we are interested in the limit problem of infinitely many players, which we expect to be indexed by \(\mathbb {Z}\) since we have an undirected structure, it is convenient to shift the indices in such a way that \(i=0\) “stays in the middle”; that is, we let \(i=-N',\dots ,N''\), instead of \(i=0,\dots ,N\), where, for example, \(N'=N'' = (N-1)/2\) if N is odd, and \(N' = N/2 = N''+1\) if N is even. Therefore, we rewrite system (2.10) as
where all indices are understood \(\textrm{mod} \ N\) and between \(-N'\) and \(N''\). Now it is immediate to identify a convenient limit system by letting \(N \rightarrow \infty \).
2.3 Self-Controlled Sequences for the Discrete Convolution
We can recognise a structure of a cyclic discrete convolution in the sums in (2.10); that is,
We wish to exploit this fact in order to prove existence of the solution to system (2.11) for small T, for any (possibly infinite) N.
Our main tool will be the following.
Definition 2.5
A nonnegative sequence \(\beta \in \ell ^2(\mathbb {Z})\) is said to be convolution-self-controlled, or c-self-controlled, if \(\beta \star \beta \lesssim \beta \); that is,
for some constant \(C>0\) independent of i.
We will mainly consider positive c-self-controlled sequences in \(\ell ^1(\mathbb {Z})\) that are even and “weakly decreasing”, in the sense that \(\beta _i = \beta _{-i}\) and there exists \(K>0\) such that \(\beta _j \le K\beta _i\), for all \(i,j \in \mathbb {N}\) with \(j \ge i\). Such sequences, which we will refer to as regular, indeed exist, as shown by the following result.
Lemma 2.6
For any \(\epsilon > 0\), there exists a positive sequence \(\beta \in \ell ^1(\mathbb {Z})\), with \(\Vert \beta \Vert _2 < \epsilon \) and such that \(\beta \star \beta \le 4\beta \). In particular, one can choose \(\beta \) of the form
for some \(\alpha = \alpha (\epsilon ) > 0\), so that \(\beta \) is even and \(\beta _j \le \coth (\frac{\alpha \pi }{2}) \beta _i\) for all \(i,j \in \mathbb {N}\) with \(j \ge i\).
Proof
It is well-known that the Fourier coefficients of a function \(f \in L^2((-\pi ,\pi ))\), given by
satisfy the following property: if \(f,g \in L^2((-\pi ,\pi ))\), then \(\widehat{fg}_j = ({\hat{f}} \star {\hat{g}})_j\) for all \(j \in \mathbb {Z}\). Let now be \(f_\alpha :=e^{-\alpha |\cdot |}\), for any \(\alpha > 0\). It is elementary to compute, for each \(j \in \mathbb {Z}\),
whence \((\widehat{f_\alpha } \star \widehat{f_\alpha })_j = \widehat{f_\alpha ^2}_{\!j} = \widehat{f_{2\alpha }}_j \le 4 \widehat{f_\alpha }_j\), the last inequality being straightforward to check using the explicit expression (2.14). Also, by Parseval’s identity \(\Vert \widehat{f_\alpha }\Vert _{\ell ^2(\mathbb {Z})}^2 = \Vert f_\alpha \Vert _{L^2(-\pi ,\pi )}^2 = \alpha ^{-1}(1-e^{-2\alpha \pi }) \rightarrow 0\) as \(\alpha \rightarrow +\infty \), so that \(\beta = \widehat{f_\alpha }\) has the desired properties for any choice of \(\alpha = \alpha (\epsilon )\) sufficiently large. Finally, note that \(\beta _j \le \coth (\frac{\alpha \pi }{2}) \beta _i\) for all \(i,j \in \mathbb {N}\) with \(j \ge i\). \(\square \)
Remark 2.7
(Variations on a c-self-controlled sequence) Clearly any positive multiple of a c-self-controlled sequence stays self-controlled. This allows to have self-controlled sequences of arbitrarily large \(\ell ^\infty \)-norm, although with a larger constant C in (2.13). On the other hand, one can also build c-self-controlled sequences which decay exponentially faster; indeed, if \(\beta \) is c-self-controlled, then for any \(\gamma > 0\) so is the sequence defined by setting \({\tilde{\beta }}_i :=\beta _i e^{-\gamma |i|}\), with the same implied constant C. This is easily proven as follows. Suppose that \(i\le 0\); then
The case \(i\ge 0\) is analogous.
The next result will be useful to deal with convolution of the form (2.12). It essentially states that c-self-controllability is preserved by suitable perturbations, which include all perturbations with compact support.
Lemma 2.8
Let \(\beta \) be c-self-controlled and let \(\theta = (\theta _{hk})_{h,k \in \mathbb {Z}}\) be a nonnegative sequence such that \(\theta \lesssim \beta \otimes \beta \).Footnote 2 Let d be the sequence given by \(d :=\beta \otimes \beta + \theta \). Then
Proof
It suffices to compute
\(\square \)
Remark 2.9
A straightforward implication of the above inequality is that \((d_{\cdot 0} \star d_{\cdot k})_h \lesssim d_{hk}\).
2.4 Short-Time Existence for Nearsighted Interactions
We are now ready for our first existence and uniqueness results (Theorem 2.13 and 2.14 below), which is a direct consequence of the following proposition.
By a game with nearsighted interactions we are meaning that \(|f| \vee |g| \le \theta \) pointwise (that is, index-wise) for some \(\theta \in \ell ^1(\mathbb {Z}^2)\) satisfying the hypotheses of Lemma 2.8; said differently, we are meaning that \(|f| \vee |g| \lesssim \beta \otimes \beta \) for some regular c-self-controlled \(\beta \).
Remark 2.10
Any compactly supported sequence \(h :\mathbb {Z}^2 \rightarrow [0,+\infty )\) is nearsighted. Indeed, given a positive c-self-controlled sequence \(\beta \), define \({\tilde{\beta }} :=(\max _{(i,j) \in \textrm{spt}\beta } \frac{h_{ij}}{\beta _i\beta _j})\beta \); then \(h \lesssim {\tilde{\beta }} \otimes {\tilde{\beta }}\).
Proposition 2.11
Let \(N \in \mathbb {N}\) and \(\beta \) be a regular c-self-controlled sequence. Let \(f \in C^0([0,T];\mathscr {S}(N))\) and \(g \in \mathscr {S}(N)\) satisfy \(|f| \vee |g| \lesssim \beta \otimes \beta \). Define \(\mathcal {C}_N :=C^0([0,T])^{2N+1}\) and write \(c = (c_{ij})_{i,j=-N}^{N} \in \mathcal {C}_N\). For \(d = \beta \otimes \beta + |g| \vee \sup _{[0,T]} |f|\), set
Let \(J_N :\mathcal {K}_N \rightarrow \mathcal {C}_N\) be the map given, for each \(i,j = -N,\dots ,N\), by
Then there exist \(T^* > 0\), depending on \(\beta \) but independent of N, such that
Proof
Let \(c \in \mathcal {K}_N\). If \(i \ge 0\),
where the last estimate comes from Lemma 2.8. As \(\beta \) is regular and \(|i-2N-1| = 2N + 1 - i > i\), we have that \(\beta _{i-2N-1} \lesssim \beta _i\), and thus \(\left\| (c_{\cdot 0} \star _N c_{\cdot j})_i\right\| _\infty \lesssim d_{ij}\), with an implied constant which does not depend on N. The same holds for \(-N \le i < 0\) by a symmetrical argument. Analogously, \(\left\| ( c_{i\cdot } \star _N c_{0\cdot } )_j \right\| _\infty \lesssim d_{ij}\) and clearly \(\Vert c_{0i}c_{0j}\Vert _\infty \le 4d_{0i}d_{0j} \le 4(d_{\cdot 0} \star d_{\cdot j})_i \lesssim d_{ij}\). Therefore,
where the constant C depends only on \(\beta \). It follows that for \(T>0\) small enough, depending on \(\beta \), one has \(\Vert J_N(c)_{ij}\Vert _{\infty } < 2d_{ij}\) for all \(i,j = -N,\dots ,N\). \(\square \)
Remark 2.12
We stated and proved Proposition 2.11 for an odd number of players. This is just a matter of having expressions that look more symmetrical, yet there is no preference about the parity of the number of players, so that the above result holds, mutatis mutandis, also if the number of players is even. It is also clear that, with a very much analogous proof, the thesis of Proposition 2.11 also holds for \(N = \infty \), where one defines, in a natural way,
and
Theorem 2.13
Under the hypotheses of Proposition 2.11, there exists \(T^* > 0\) such that if \(T \le T^*\) then for any \(N \in \mathbb {N}\cup \{\infty \}\) there exists a unique smooth solution to system (2.11) such that, for any \(i,j \in -N',\dots ,N''\) and \(m \in \mathbb {N}\),
where the implied constants depend only on \(T^*\), f, g and m.
Proof
A fixed point of the map \(J_N\) defined in (2.16) is a solution. We deal with the case of \(J_N\) with \(N = \infty \), as the case with \(N \in \mathbb {N}\) can be included therein. Note that \(\mathcal {K}_\infty \) can be considered as a closed ball of the Banach space \(\ell ^\infty _d(\mathbb {Z}^2; C^0([0,T]))\); that is, the space of functions from \(\mathbb {Z}^2\) to C([0, T]) with finite norm
We prove that the map \(J_\infty \) is a contraction on \(\mathcal {K}_\infty \), provided that T is sufficiently small with respect to d. The conclusion will follow from Proposition 2.11, Remark 2.12 and the contraction mapping theorem; then, once one have a continuous solution, by the structure of equations (2.11), one bootstraps its regularity up to \(C^\infty \), while estimate (2.18) for \(m > 1\) follows by induction differentiating (2.11) and estimating as in the proof of Proposition 2.11. Let now \(c,{\bar{c}} \in \mathcal {K}_\infty \). We have, for i, j fixed,
We have
that is, \(|\!|\!|( {\bar{c}}_{\cdot 0} \star {\bar{c}}_{\cdot j})_i - ( c_{\cdot 0} \star c_{\cdot j})_i|\!|\!| \lesssim |\!|\!|{\bar{c}}-c|\!|\!|\) and analogously for the second term in (2.19). Hence \(|\!|\!|J_\infty ({\bar{c}}) - J_\infty (c)|\!|\!| \lesssim T |\!|\!|{\bar{c}} - c|\!|\!|\). \(\square \)
Theorem 2.14
Suppose that \(f^i\) and \(g^i\) are shift-invariant and there exists a regular c-self-controlled \(\beta \) such that \(|f^0| \vee |g^0| \le C(\beta \otimes \beta )\), \(C > 0\). There exists \(T^* > 0\), depending only on C and \(\beta \), such that if \(T \le T^*\) then there exists a smooth classical solution to the infinite-dimensional Nash system (2.6) with \(i \in \mathbb {Z}\) on \([0,T] \times \mathcal {X}\). Furthermore, such a solution is unique in the class of functions of the form (2.7).
Proof
Let c be the solution given by Theorem 2.13. For \( x = (x^i)_{i\in \mathbb {Z}} \in \ell ^\infty (\mathbb {Z}; {\mathbb {R}}^d)\), define
where we denoted by \(\cdot \) the standard scalar product on \({\mathbb {R}}^d\). U is well-defined for \( x \in \ell ^\infty (\mathbb {Z}; {\mathbb {R}}^d)\), and continuous in t, because the series normally converge thanks to estimate (2.18); for the same reason, also
are well-defined and continuous. Finally, for \( h \in \ell ^\infty (\mathbb {Z}; {\mathbb {R}}^d)\), note that (omitting the dependence on t)
thus \(U(t,\cdot )\) is infinitely many times Fréchet-differentiable in \(\ell ^\infty (\mathbb {Z};{\mathbb {R}}^d)\). Define now \( u = (u^i)_{i\in \mathbb {Z}}\) by setting
where \((\sigma x)_j :=x_{j-1}\) for \(j \in \mathbb {Z}\). We have
hence \(\sum _{j\in \mathbb {Z}} D_ju^j D_j u^i\) locally uniformly converges by estimate (2.18). Hence, by construction, u solves (2.6) in the desired sense. \(\square \)
Remark 2.15
(Unimportance of distant players) What essentially allows the Nash system to have a solution in infinite dimensions is what we call an unimportance of distant players, in the sense that the farther a player is from a given one (say the 0-th player) the smaller the impact it has on it is. This is seen in the fact that, on any bounded \(\mathcal {B} \subset \mathcal {X}\),
is infinitesimal as \(|j| \rightarrow \infty \).
2.5 Beyond Shift-Invariance
We have made the shift-invariance hypothesis to reduce our system of infinitely many equations for c to one equation. Nevertheless, the reader should be aware that the above results can be adapted to a more general setting.
One can suppose that we only have a shift-invariant control on the data; that is
In this case, the most natural limit of (2.8) is indexed over \(\mathbb {N}\) and it suffices to replace \(\mathcal {K}_\infty \) with
which is a closed ball in \(\ell ^\infty _{{\tilde{d}}}(\mathbb {N}^3;C^0([0,T]))\), letting \({\tilde{d}}^i_{hk} :=d_{|h-i|,|k-i|}\). Then, for instance, one obtains the following result.
Theorem 2.16
Assume (2.20). There exists \(T^* > 0\) such that if \(T \le T^*\) then for any \(N \in \mathbb {N}\cup \{\infty \}\) there exists a unique smooth solution to system (2.8) such that, for any \(i,h,k \in [[N]]\) and \(m \in \mathbb {N}\),
where the implied constants depend only on \(T^*\), f, g and m.
Also, one can fix the dimension \(N \ge 2\) and consider \(\beta ^N \in ({\mathbb {R}}_+)^{2N+1}\) given by
which is c-self-controlled in the sense that for \(|j| \in [[N]]\)
In this case one can look for a solution to (2.8) starting with assumption (2.20) with \(\beta = \beta ^N\); that is,
What we get is the following statement.
Theorem 2.17
Let \(N \in \mathbb {N}\), \(N\ge 2\) and assume (2.21). There exists \(T^* > 0\) such that if \(T \le T^*\) then there exists a unique smooth solution to system (2.8) such that, for any \(i,h,k \in [[N]]\) and \(m \in \mathbb {N}\),
where the implied constants depend only on \(T^*\), f, g and m.
Notice that this can be regarded as a result in a mean-field-like setting, as assumption (2.21) is consistent with the fact that we expect the j-th derivative of a mean-field-like cost for the i-th player to scale by a factor of \(N^{-1}\) whenever \(j \ne i\). For further discussion and results related to this specific setting, we invite the reader to proceed directly to the next Sect. 3.
2.6 Long-Time Existence for Shift-Invariant Directed Strongly Gathering Interactions
It is clear that the previous construction follows the standard Cauchy-Lipschitz local (in time) existence argument, and the existence (and uniqueness) of a solution can be as usual continued up to a maximal time \(T^*\), that is when the quantity \(\max _{i,j} \beta _i^{-1}\beta _j^{-1} |c^0_{ij}|\) blows up. So far, we cannot exclude in general that such blow up time \(T^*\) is finite.
Upcoming Definition 2.18 introduces an additional assumption on the running and terminal costs which will allow us to prove long-time existence of a solution to the infinite-dimensional Nash system. This assumption requires some rather abstract properties of the generating function that is built upon \(f_{hk}\). We will state the assumption first, and then discuss its implications on the game structure. We anticipate that, loosely speaking, the i-th player’s cost is affected by the states of the players \(j>i\) only in the chain (directedness) and that the term \(f_{hh}\) is “dominant” with respect to \(f_{hk}\), \(h \ne k\) (strong gathering).
Given \(m \le n\), we will identify \(M \in \mathscr {S}(m) \subset \mathscr {S}(n)\) by considering \({\mathbb {R}}^m \simeq {\mathbb {R}}^m \times \{0\} \subset {\mathbb {R}}^m \times {\mathbb {R}}^{n-m} = {\mathbb {R}}^n\) and then extending \(M = 0\) on \({\mathbb {R}}^{n-m}\); that is, identify \(M \in \mathscr {S}(m)\) with \(\left( \begin{array}{cc} M &{} 0 \\ 0 &{} 0 \end{array}\right) \in \mathscr {S}(n)\). Also, given \(M \in \mathscr {S}(m)\) and \(M' \in \mathscr {S}(n)\) we will say that \(M=M'\) if \(M'\) equals the above-mentioned extension of M over \({\mathbb {R}}^n\).
Definition 2.18
Let \(\mathscr {M} = (M^{(N)})_{N \in \mathbb {N}}\) be a sequence of matrices, with \(M^{(N)} \in \mathscr {S}(N)\). We say that \(\mathscr {M}\) is directed if there exists \(\ell \in \mathbb {N}\) such that \(M^{(N)} = M^{(\ell )} \in \mathscr {S}(\ell )\) for all N large enough.
Given \(\varrho > 1\), we say that \(M \in \mathscr {S}(\ell )\) is \(\varrho \)-strongly gathering if the polynomial
is such that \(\mu (z,0) \notin (-\infty , 0)\) if \(z \in \varrho \bar{\mathbb {D}}\).Footnote 3
Remark 2.19
We anticipate here the main idea behind this definition. We wish to consider the generating function of the coefficients \(c_{hk}\) in (2.11) with \(N = \infty \); that is, formally, \(\Xi (t,z,w) :=\sum _{h,k \in \mathbb {Z}} c_{hk}(t)z^hw^k\). This is a priori singular on \(zw=0\), nevertheless we are going to show that if \(((f_{hk})_{|h|,|k| \le N})_{N\in \mathbb {N}}\) and \(((g_{hk})_{|h|,|k| \le N})_{N \in \mathbb {N}}\) are directed, then one can assume that \(c_{hk} = 0\) if \(h \wedge k < 0\), so that \(\Xi \) is analytic and satisfies a “functional Riccati equation” (see (2.28) below).
At this point the strong gathering condition, which will be put on f (see upcoming Assumptions (\(\varvec{\star }\))), has a twofold utility. First, it ensures that the functional Riccati equation has a solution which is defined in a neighborhood of \((z,w) = (0,0)\) and has real coefficients \(c_{hk}\); this is basically due to the fact that the principal branch of the square root of the function \(\mu (\cdot ,0)\) associated to f is well-defined. Second, as we require \(\varrho > 1\) (and not just \(\varrho > 0\)), it ensures that the \(c_{hk}\)’s will be summable by standard properties of the derivatives of an analytic function.
We also point out that \(\varrho > 1\) is not necessary in order for summability to be satisfied, while \(\varrho = 1\) is not a priori sufficient. This makes the latter a limiting condition that we have found as interesting as difficult to study in the generality we would have wished for; we have devoted to it a brief discussion in Sect. 2.9.
Remark 2.20
The reader could find a first sight quite strange that the crucial condition in the notion of strong gathering regards \(\mu (\cdot ,0)\), which sees only the first column \((M_{h0})_{h \in [[\ell ]]}\) of the matrix M. From a technical point of view, we claimed in Remark 2.19 that it is all we need to solve the functional Riccati equation (2.28). From an interpretative point of view, instead, recalling that we will require f to be strongly gathering, it is interesting to note that the coefficients \(f_{h0}\) quantify the interaction of the 0-th player with the others; in the shift-invariant setting the 0-th player is basically our reference player for the game, hence we are somehow saying that the solvability of the Nash system is related to a condition which sees only the “direct” interactions between the reference player and the others, and not the “indirect” influence of the interactions between pairs of other players.
Remark 2.21
The term directed is related to the fact that the i-th player’s cost is affected by the states of the following players \(j>i\) in the chain, and this fact might not be immediately clear from the previous definition, which just requires the matrices \(M^{(N)}\) to be extensions of a fixed matrix, not depending on N. One may then have a look to the matrices of the form \(M = w \otimes w\) in (the end of) Example 2.2. In particular, \(w = w^{(N)}\) in (2.5) gives rise to a directed family only when \(m = 0\).
Moreover, in the situations described in Example 2.2, the associated sequence of matrices as the dimension N diverges is not strongly gathering. Indeed, even though \(\ell \) stays bounded, one has that \(\mu \) is a polynomial with \(\mu (1,0) = 0\). In fact, as said in Example 2.19, those situations can be seen as limit settings corresponding to taking \(\varrho = 1\) and we will comment on this case later on in Sect. 2.9.
The validity of the strong gathering assumption as it is, can be achieved in different ways; two basic settings are given in the following examples.
Example 2.22
If we want to stick to a matrix \(f^0\) of the form (2.1), in order to have \(({f^0}{(N)})_{N \in \mathbb {N}}\) (where N is the dimension) directed with strongly gathering limit one can require that \(\ell \) is independent of N large and
so that \(\mu (z,0) \ge \epsilon \) if \(|z| \le 1\). Put it differently, it suffices to consider
this means that we are considering an underlying graph where each node is directly connected with itself as well, and such link has a negative weight. As in this model a positive weight is associated to the tendency, in order to reduce their cost, of each player to get closer to their neighbors, a negative connection with themself is to be interpreted as a drift towards self-annihilation. More prosaically, this means that the state of each player will also tend to the common position \(0 \in {\mathbb {R}}^d\).
Regarding this example, it is also worth pointing out that the common attractive position \(0 \in {\mathbb {R}}^d\) cannot be any other arbitrary point, in the sense that the structure of problem is not invariant under translation of the coordinates. This is due to the fact that we are considering a graph whose nodes have outdegree different from 1.
Example 2.23
Another setting with directedness and strong gathering is that of
where w can be given by (2.4) with \(\ell \) independent of N, \(\nu > 0\) and \(E^0x = x^0\) for all \(x = (x^0, \dots , x^{N-1}) \in ({\mathbb {R}}^d)^N\). Here
and with a translation of the coordinates we can lead to this situation also costs like
for any given \(y \in {\mathbb {R}}^d\). This example, more genuinely than the previous one, shows that the strong gathering assumption entails that we are giving some sort of preference about where the players should aggregate. This will strengthen the attractive structure yielded by a graph satisfying only assumption (SI) (Example 2.2), thus providing more stability to our game.
From this section on in this part, the following assumptions will be in force, declined in suitable manners which will be specified.
Assumptions
(\(\varvec{\star }\)) The matrices \(f^i\) and \(g^i\) are shift-invariant and \(f^0 = f^0(N)\) and \(g^0 = g^0(N)\) are directed with limits \(f,g \in \mathscr {S}(\ell )\) for some \(\ell \in \mathbb {N}\). The matrix f is \(\varrho \)-strongly gathering for some \(\varrho > 0\) and g is compatible on I with f for some interval \(I \subset {\mathbb {R}}\), in the following sense: given
and setting
Footnote 4 we have
Remark 2.24
Note that for any \(t \in {\mathbb {R}}\), one has that \(\psi \tanh (t\xi ) + \xi \) is bounded on D, as \(\xi (z)t\) can be a singular point only if \(\xi (z)^2 = \phi (z,0) < 0\), which contradicts that \(z \in D\). Also note that condition (2.22) holds with \(I = {\mathbb {R}}\) if, e.g., \(g = 0\) or \(|\psi | \ge |\xi |\).
Considering system (2.11), we see that for any N sufficiently large with respect to \(\ell \) we have \(f_{hk} = 0 = g_{hk}\) if either h or k is negative; hence the limit system as \(N \rightarrow \infty \) will be given by
where \(f_{hk}\) and \(g_{hk}\) extend to \(h,k \in \mathbb {Z}^2\) by letting \(f_{hk} = 0 = g_{hk}\) if \((h,k) \notin [[\ell ]]^2\).Footnote 5 Given this system, another reduction is possible. Since \(f_{hk} = 0 = g_{hk}\) whenever \(h \wedge k < 0\), the assumption that \(c_{hk} = 0\) if \(h \wedge k < 0\) is not a priori incompatible with the structure of system (2.24). Therefore we will look for a solution with this additional property; in this way, the coefficients which are not a priori null, \((c_{hk})_{h,k \in \mathbb {N}}\), will define in a natural way a symmetric operator \(c(t) \otimes I_d\) on \(\mathcal {X}\) which vanishes on \(\ell ^\infty (\mathbb {Z}_{<0};{\mathbb {R}}^d)\), so that it can also be seen as a trivial extension to \(\mathcal {X}\) of a symmetric operator on \(\ell ^\infty (\mathbb {N};{\mathbb {R}}^d)\). This reduces the system of ODEs in (2.23) to
which will be the object of our following study, and will eventually provide a solution with the particular form presented in the upcoming definition.
Definition 2.25
A quadratic shift-invariant directed (QSD) solution to (2.6) is a classical solution of the form
for some \(c :[0,T] \rightarrow \ell ^1(\mathbb {N}^2) \subset \ell ^1(\mathbb {Z}^2)\).Footnote 6
Theorem 2.26
Under Assumptions (\(\star \)) with \([0,T] \subseteq I\), there exists a unique QSD solution to (2.6) on \([0,T] \times \mathcal {X}\).
The following lemmata, in whose statement the hypotheses of Theorem 2.26 will be implied, provide the steps of our proof of Theorem 2.26.
Lemma 2.27
There exists a unique sequence \((c_{hk})_{h,k \in \mathbb {N}} \subset C([0,T]) \cap C^\infty ((0,T))\), with \(c_{hk} = c_{kh}\), which solves the infinite dimensional system (2.24) on [0, T].
Proof
We perform the change of variables \(t \mapsto T-t\) and prove that there exists a unique solution on [0, T] with initial condition \(c(0)=g\) to the forward system
which smoothly extends to \({\mathbb {R}}\). Then, the solution to (2.24) on [0, T] with \(c(T)=0\) will be given by the restriction to [0, T] of \({\hat{c}}(T-{\cdot })\), where \({\hat{c}}\) is the unique solution to (2.26) on \({\mathbb {R}}\) with \({\hat{c}}(0)=g\). Note that \({\hat{c}}_{00}\) is the solution to the Riccati equation \(\dot{{\hat{c}}}_{00} + {\hat{c}}_{00}^2 = f_{00}\), where \(f_{00} = \varphi (0,0) > 0\) by strong gathering, hence
All the other \({\hat{c}}_{hk}\)’s satisfy first-order ODEs with coefficient which are second-order polynomials depending only on \(f_{hk}\) and \({\hat{c}}_{h'k'}\) with \((h',k') \prec (h,k)\), where \(\prec \) denotes the strict Pareto preference.Footnote 7 Therefore, existence and uniqueness of the solution to the infinite system may be proved by induction. Indeed, suppose \(({\hat{c}}_{0k'})_{0\le k'<k} \subset C^\infty ({\mathbb {R}})\) are given, and note that
then \({\hat{c}}_{0k}\) is unique and smooth as well. This proves the existence and uniqueness of \({\hat{c}}_{0k}\) for all \(k \in \mathbb {N}\); then, looking at this argument as the base step for a new induction over h, one proves analogously the existence and uniqueness of \({\hat{c}}_{hk}\) for all \(h \in \mathbb {N}\) and any \(k \in \mathbb {N}\). Finally, \(c_{hk} = c_{kh}\) since equations (2.10) are invariant with respect to the swap \((h,k) \mapsto (k,h)\). \(\square \)
The arguments below show that the coefficients \(c_{hk}\) can be thought as derivatives of a generating function \({\hat{\Xi }}\), which will play a fundamental role in the long-time analysis.
Lemma 2.28
The solution to (2.24) on [0, T] is given by
for some function \({\hat{\Xi }} :I \times \varrho \mathbb {D}^2 \rightarrow \mathbb {C}\), of class \(C^\infty \) with respect to \(t \in I\) and analytic in \(\varrho \mathbb {D}^2\), such that \({\hat{\Xi }}(\cdot ,z,w) = {\hat{\Xi }}(\cdot ,w,z)\) and \({\hat{\Xi }}(0,\cdot ,\cdot ) = \Psi \).
Proof
Suppose that, for all \(z,w \in D\) fixed, there exists a solution \({\hat{\Xi }}\) on I to
with the desired properties of smoothness, invariant with respect the swap \((z,w) \mapsto (w,z)\) and such that \({\hat{\Xi }}(0,z,w) = \Psi (z,w)\). Then by taking the derivatives \(\partial ^h_z\partial ^k_w|_{(0,0)}\) one recovers equation (2.26), and thus the coefficients given by (2.27) satisfy (2.24) on [0, T].Footnote 8 To see that (2.28) admits such a solution, note that \({\hat{\Xi }}(t,z,0)\) solves the Riccati equation \(\partial _t {\hat{\Xi }}(t,z,0) + {\hat{\Xi }}(t,z,0)^2 = \varphi (z,0)\); hence,
Note that letting
one can write
For any \(t \in I\) fixed, this function is well-defined for \(z \in \varrho \mathbb {D}\) and analytic therein by the compatibility assumption (2.22). At this point, (2.28) becomes a first-order ODE in \(t \in I\), for all \(z,w \in \varrho \mathbb {D}\) fixed, whose solution is given by
Note that \({\hat{\Xi }}(t,\cdot ,\cdot )\) is well-defined for \(z,w \in \varrho \mathbb {D}\) by the same argument we applied to \(\hat{\Xi }(t,\cdot ,0)\). Also, it is trivial that \({\hat{\Xi }}(\cdot ,z,w) \in C^\infty (I)\), and by differentiating under the integral sign one proves the analyticity of \({\hat{\Xi }}(t,\cdot ,\cdot )\). \(\square \)
Remark 2.29
Let \(\tilde{\mathcal {E}}(t,z;\zeta _1,\zeta _2) :={\mathcal {E}}(t,z;\zeta _2,\zeta _1)\). As, omitting the dependence on \(\zeta _1\) and \(\zeta _2\) in \(\mathcal {E}\), we have \(\frac{\partial }{\partial t} \mathcal {E}(t,z) = \xi (z) \tilde{\mathcal {E}}(t,z)\), it is easy to see that
and thus
Letting \((\zeta _1,\zeta _2) \in \{(\xi ,\psi ),(\psi ,\xi )\}\) one can then compute the integral in (2.29). Set
then
where
Lemma 2.30
Let \(c=(c_{hk})_{h,k\in \mathbb {N}}\) be the solution to system (2.24) on [0, T]. Then, for any \(r \in (1,\varrho )\),
for some constant K(r, T) depending only on T, r, f and g. In particular, \(c \in C^0([0,T]; \ell ^1(\mathbb {N}^2))\) and the same is true for \(\dot{c}\).Footnote 9
Proof
By the strong gathering assumption, the function \(\hat{\Xi }(t,\cdot ,\cdot )\) given in Lemma 2.28 is analytic in a neighborhood of \(\mathcal {Q}_r :=\bar{\mathbb {D}}_r \times \bar{\mathbb {D}}_r\), with \(r \in (1,\varrho )\). Then, by Cauchy’s theorem on derivatives,
This proves that \(c \in C^0([0,T]; \ell ^1(\mathbb {N}^2))\), and further regularity is easily proven by induction by exploiting system (2.24). \(\square \)
Lemma 2.31
Let c be the solution to (2.24) on [0, T]. The functions \(u^i\) given by (2.25) are well-defined for \((t,x) \in [0,T] \times \mathcal {X}\), differentiable with respect to t, and twice Fréchet-differentiable with respect to x.
Proof
Since a suitable shift of coordinates in \(\mathcal {X}\) transforms \(u^i(t,\cdot )\) into \(u^0(t,\cdot )\), it is sufficient to prove the result for \(u^0\). The existence of a solution follows from Lemma 2.30, and the differentiability with respect to t follows from the same lemma and the fundamental theorem of calculus. The differentiability with respect to x follows again from Lemma 2.30, as it is trivial to see that for any \(h \in \mathcal {X}\) small one has
where, with an obvious notation, \(\langle (c(t) \otimes I_d)y,z\rangle _{\mathcal {X}} = \sum _{h,k \in \mathbb {N}} c_{hk}(t) \langle y^h,z^k\rangle _{{\mathbb {R}}^d}\). \(\square \)
Lemma 2.32
Let \(u^i\) be given by (2.25), where c is the solution to (2.24) on [0, T]. Let \(\mathcal {B}_R\) be the closed ball of radius R in \(\mathcal {X}\). Then, for any \(r \in (1,\varrho )\),
where \(K=K(r,T)\) is the constant appearing in Lemma 2.30.
Proof
We have \(D_j u^i(t,x) = 0\) if \(j < i\), and \(D_j u^i(t,x) = \sum _{h \in \mathbb {N}} c_{j-i,h}(t) x^h\) if \(j \ge i\). Then, for \(x \in \mathcal {B}_R\), and \(r \in (1,\varrho )\) fixed, estimate (2.32) yields
for all \(t \in [0,T]\). The thesis now follows by computing \(\sum _{j > i} r^{i-j}\). \(\square \)
Since, by construction, choosing \(u^i\) as in (2.25) satisfying (S1) and (S2) in Definition 2.3 yields a classical solution, the proof of Theorem 2.26 is complete.
2.7 Almost-Optimal Controls for the N-Player Game
We did not prove long-time existence of a solution to the Nash system for the N-player game, with \(N > \ell \) finite; nevertheless, in Corollary 2.36 below we show that on the horizon [0, T] the infinite-dimensional optimal control for the i-th player, given by \({\bar{\alpha }}^{*i}(t,X_t) :=\sum _{j\ge i} c_{0,j-i}(t) X_t^j\), if suitably “projected” onto \(({\mathbb {R}}^d)^N\) provides an \(\epsilon \)-Nash equilibrium for the N-player game, with \(\epsilon \rightarrow 0\) as \(N \rightarrow \infty \). In claiming so, we consider classes of admissible controls in the sense of the following definition.
Definition 2.33
Let \(R,L \ge 0\). A control \(\alpha :[0,T] \times ({\mathbb {R}}^d)^N \rightarrow ({\mathbb {R}}^d)^N\) in feedback form belongs to \(\mathcal {A}_{R,L}\) if, for all \(t \in [0,T]\), \(x,y \in ({\mathbb {R}}^d)^N\),
The Lipschitz constant L is said to be admissible if \(L \ge \Vert c_{0\cdot }\Vert _{C^0([0,T];\ell ^1(\mathbb {N}))}\).
Remark 2.34
For such controls, it is known (cf., e.g., [2, Theorems 9.1 and 9.2]) that
has a unique solution, satisfying \(\mathbb {E}\,\sup _{[0,T]} |X|^2 \le C\) where C is a locally bounded function of R, L and T, directly proportional to \(1+|x_0|^2\).
Theorem 2.35
Consider the N-player game on [0, T] with time evolution of the state of the players given by (2.34) and costs given by (1.1) with \(f^0,g^0 \ge 0\). Let Assumptions (\(\star \)) be in force with \([0,T] \subseteq I\). Let c solve (2.24) and define the control \(\alpha ^*\) by
Then, for any \(R \ge 0\) and admissible L, for any \(i \in [[N]]\) and any \((\alpha ^{*,-i},\psi ) \in \mathcal {A}_{R,L}\),Footnote 10
where \(\ell \) is the dimension appearing in Assumptions (\(\star \)) and the constant \({\hat{C}}\) is a locally bounded function of R, L, T and \(\delta \), directly proportional to \(1+|x_0|^2\).
Proof
Since \(f^i\) and \(g^i\) are shift-invariant and \(\alpha ^*\) is linear in the state variable with \(D\alpha ^*\) circulant, without loss of generality we can prove the thesis for \(i = 0\). We denote by \(X^*\) the solution to (2.34) when \(\alpha = \alpha ^*\) and by X the solution to (2.34) when \(\alpha = (\psi , \alpha ^{*1},\dots ,\alpha ^{{*},{N-1}}) =:{\hat{\alpha }}^{*}\). We wish to estimate from above the quantity
where \(F :=\langle F^0\cdot ,\cdot \rangle \) and \(G :=\langle G^0\cdot ,\cdot \rangle \). Let u be the QSD solution to (2.6) on \([0,T] \times \mathcal {X}\). Since G is convex and \(DG = Du^0(T,\cdot )\),
Note that we can replace \(\ell \) with any \(M \ge \ell \) because \(c_{jk}(T) = g_{jk}(T) = 0\) if \(k > \ell \). As c solves (2.24) we obtain
where \((Z_t)_{0\le t\le T}\) is a martingale starting from 0. Straightforward computations show that, omitting the dependence on t,
and
therefore,
where we have set
Note now that by the convexity of \(\frac{1}{2}|\cdot |^2\), omitting the dependence on t,
and by the convexity of F
Using (2.36), (2.37) and (2.38) in (2.35) we obatin
As, whenever \(\{ Y,Z \} \subseteq \{ X^*,X \}\),
where \(C = C(|x_0|,R,L,T)\) is the constant appearing in Remark 2.34, we have
so that by Lemma 2.30, for any \(\delta \in (\varrho ^{-1},1)\), there exists \(K > 0\) such that
On the other hand,
where we used again that \(\alpha ^*, {\hat{\alpha }}^{*} \in \mathcal {A}_{R,L}\). Therefore, from (2.39), (2.40), (2.41) and Lemma 2.30 we obtain
This concludes the proof. \(\square \)
Corollary 2.36
Let \(\Omega \subset {\mathbb {R}}^d\) bounded, and assume the same as in Theorem 2.35, but with \(x_0 \in \Omega ^N\). Let \(R \ge 0\) and L be admissible (in the sense of Definition 2.33); consider as admissible those controls belonging to \(\mathcal {A}_{R,L}\). Then, for any \(\epsilon > 0\) there exists \(N_0 = N_0(\epsilon ,R,L,\varrho , \ell , \Omega )\) such that if \(N \ge N_0\) then the control \(\alpha ^*\) provides an \(\epsilon \)-Nash equilibrium of the game.
Proof
It suffices to require that
where, for instance, one sets \(\delta = \frac{1}{2}(1+\varrho ^{-1})\). Choose \(M = M(N)\) such that \(M \rightarrow \infty \) and \(M = o(N)\) as \(N \rightarrow \infty \); for example, \(M = \lfloor \sqrt{N} \rfloor \) for all \(N > \ell ^2\). Since \(x_0 \in \Omega ^N\) there exists a constant \({\hat{C}}'\), independent of N, such that \({\hat{C}} \le N {\hat{C}}'\). Then the conclusion follows from the fact that the left-hand side of (2.42) goes to 0 as \(N \rightarrow \infty \). \(\square \)
2.8 The Ergodic Nash System
Consider now the related ergodic problem, with costs
where the dynamics and the assumptions on \(F^i\) are the same as before. The corresponding Nash system reads
where \(\mathcal {Y}\) is either \(({\mathbb {R}}^d)^N\), for the N-player game, or \(\mathcal {X}\), for the limit game with infinitely many players. In the latter case we give notions of classical solution and QSD solution which are analogous as those in the previous section.
Definition 2.37
A sequence of pairs \(((\lambda ^i, v^i))_{i\in \mathbb {Z}}\) of real numbers \(\lambda ^i\) and \({\mathbb {R}}\)-valued functions \(v^i\) defined on \(\mathcal {X}\) is a classical solution to the ergodic Nash system (2.43) on \(\mathcal {X}\) if the following hold:
-
(E1)
each \(v^i\) is of class \(C^2\) with respect to \(x \in \mathcal {X}\), in the Fréchet sense;
-
(E2)
for each \(i \in \mathbb {N}\), the series \(\sum _{j\ne i} D_j v^j D_j v^i\) uniformly converges on all bounded subsets of \(\mathcal {X}\);
-
(E3)
system (2.43) is satisfied pointwise for all \(x \in \mathcal {X}\);
A QSD ergodic (QSDE) solution will be a classical solution to (2.43) with
for some \({\bar{c}} \in \ell ^1(\mathbb {N}^2)\).
Remark 2.38
By the structure of (2.43), it is clear that if \(((\lambda ^i, v^i))_{i\in \mathbb {Z}}\) is a classical solution, then so will be \(((\lambda ^i, v^i+\mu ^i))_{i\in \mathbb {Z}}\) for any choice of real numbers \(\mu ^i\). We will prove that there exists a special choice of \(\mu \in {\mathbb {R}}\), and of \({\bar{c}} \in \ell ^1(\mathbb {N}^2)\), such that the solution \(((\lambda , v^i+\mu ))_{i\in \mathbb {Z}}\), with \(\lambda \) and \(v^i\) given by (2.44), is in a precise sense the limit of the QSD solution as \(T \rightarrow +\infty \) (see Theorem 2.42).
Arguing as in the previous section, the coefficients \({\bar{c}}_{hk}\) of a QSDE solution are given by the solutions of the following system:
It is immediate to see that if c solves (2.45), then so does \(- c\), hence we cannot have a unique solution to this limit system.
Lemma 2.39
There are exactly two sequences \((c_{hk}^\pm )_{h,k \in \mathbb {N}}\) which solve (2.45). Such sequences are one the opposite of the other; that is, \(c^- = - c^+\).
Proof
We have \(c_{00}^2 = f_{00} > 0\), hence \(c_{00} \in \{ \pm \sqrt{f_{00}}\, \}\). Once the sign of \(c_{00}\) is chosen, all other \(c_{hk}\) can be uniquely determined by induction on h, k. \(\square \)
Both solutions can be represented also in this case by a generating function.
Lemma 2.40
The two solutions to (2.45) are given by
for some analytic function \(\bar{\Xi }:\varrho \mathbb {D}^2 \rightarrow \mathbb {C}\) such that \(\bar{\Xi }(z,w) = \bar{\Xi }(w,z)\).
Proof
A function \(\bar{\Xi }\) is the desired generating function if it solves the equation
for \(z,w \in \varrho \mathbb {D}\). It follows that \(\bar{\Xi }(z,0)^2 = \varphi (z,0)\), so choose \(\bar{\Xi }(\cdot ,0) = \xi \); then
Note that the real part of the denominator in (2.47) can vanish only if \(|z|,|w| = \varrho \), hence \(\bar{\Xi }\) is well-defined and analytic for \((z,w) \in \varrho \mathbb {D}^2\). \(\square \)
This is sufficient to prove the existence of exactly two opposite QSDE solutions, and thus that the limiting ergodic Nash system in \(\mathcal {X}\) is soluble. We state this as a theorem.
Theorem 2.41
There exist exactly two QSDE solutions to (2.43) on \(\mathcal {X}\). Such solutions are one the opposite of the other.
Proof
Argue as in the first part of the proof of Lemma 2.30 to say that \(c^+ \in \ell ^1(\mathbb {N}^2)\), then argue as in the proof of Lemma 2.31 to build the QSDE solution determined by choosing \({\bar{c}} = c^+\). Finally note that the solution determined by the choice \({\bar{c}} = c^-\) is the opposite function. \(\square \)
2.9 Long-Time Asymptotics
As expected by KAM theory and ergodic control, we are going to prove now that, up to a constant, the QSDE solution corresponding to \(c^+\) (that is, the solution with \(c_{00} > 0\)) describes the long-time asymptotics of the QSD solution as \(T \rightarrow +\infty \), while the opposite solution should be regarded as the result of considering the limit \(T \rightarrow -\infty \) instead. To highlight the dependence of the QSD solution on T, we will write it as \(u^i_T\); on the other hand, since by the shift-invariance property it will suffice to show the convergence of \(u^0_T\) to the QSDE solution \(v^0\), we will omit the superscript 0.
Theorem 2.42
Let Assumptions (\(\star \)) be in force with \([0,+\infty ) \subseteq I\). Let \(u_T\) be the value function of the 0-th player corresponding to the QSD solution to the Nash system on \([0,T] \times \mathcal {X}\); let v be the value function of the 0-th player corresponding to the QSDE solution on \(\mathcal {X}\) determined by \({\bar{c}} = c^+\). Let \(\lambda :={{\,\textrm{tr}\,}}{\bar{c}}\). Then, for any \(t \ge 0\), as \(t < T \rightarrow +\infty \), the following limits hold, locally uniformly in both \(x \in \mathcal {X}\) and t:
and there exists a constant \(\mu \in {\mathbb {R}}\) such that
The proof is based on the following result, which is strictly related to a refinement of Lemma 2.30 (cf. Remark 2.44 below) and is due to the possibility of explicitly compute the integral in formula (2.29) as showed in Remark 2.29.
Lemma 2.43
Under Assumptions (\(\star \)) with \([0,+\infty ) \subseteq I\), there exists a nonnegative function \(\gamma \in C_0^0([0,+\infty )) \cap L^1([0,+\infty ))\), depending only on r, f and g, such that
for all \(t \ge 0\), where \(\sigma ^\pm \) are defined as in (2.30).
Proof
By the continuity of \(\xi \), given \(r' \in (r,\varrho )\),Footnote 11 there exists \(\epsilon > 0\) such that
whence
for some function \(\mathfrak {d}\) which is uniformly positive on \([0,+\infty )\).Footnote 12 By (2.51) there exists a uniformly positive function \({\mathfrak {f}} :[0,+\infty ) \rightarrow {\mathbb {R}}_+\), which depends only on r and f, such that \(|\cosh (\cdot \,t) |\ge {\mathfrak {f}}(t)\) on \(\xi (\bar{\mathbb {D}}_r)\); also, by (2.50) we can suppose that \({\mathfrak {f}}\) be asymptotic to \(\frac{1}{2} e^{3\epsilon |\cdot |}\) at \(\infty \). Since, with \(\mathcal {L}\) defined as in (2.30),
we obtain, also using (2.22),
where the implied constant depends only on r, f and g. At this point, it is easy to see that the desired conclusion follows. \(\square \)
Remark 2.44
This proof shows that if \([0,+\infty ) \subseteq I\), then the constant K appearing in Lemma 2.30 is in fact independent of T, since \(\sup _{{\mathbb {R}}\times \mathcal {Q}_r} |{\hat{\Xi }} |\) is finite. In particular, \(c \in C^0([0,T]; \ell ^1(\mathbb {N}^2))\) along with its derivatives, and their the norms are bounded uniformly with respect to \(T>0\).
Proof of Theorem 2.42
Fix \(r \in (1,\varrho )\). By comparing expressions (2.31) and (2.47) one sees that
where \(\gamma \) is the function given in Lemma 2.43, and the implied constant depends only on the \(L^\infty \)-norms of \(\varphi \) and \(\xi \) on \(\mathbb {D}_r\) and \(\mathcal {Q}_r\), respectively. By Lemma 2.28 and Cauchy’s theorem on derivatives,
where we have used the superscript T to stress the fact that \(c_{hk}(t) = c_{hk}^T(t)\) depends on the horizon T. Plugging (2.53) into (2.54) yields
as well as
where the implied constants depend only on r and f. As \(\gamma \) is integrable on \([0,+\infty )\), by (2.56) so is \({{\,\textrm{tr}\,}}{\hat{c}} - \lambda \), where we use the notation \({\hat{c}} = c(T-\cdot )\) introduced in the proof of Lemma 2.27; thus by the dominated convergence theorem there exists \(\mu \in {\mathbb {R}}\) such that
locally uniformly in t. Along with (2.55), this proves (2.48) and (2.49). \(\square \)
Remark 2.45
The argument of the previous proof also applies to the case when \(t = sT\), with \(s \in [0,1]\). In this case, we can give the following estimate of the rate of convergence of (2.48): for any \(r \in (1,\varrho )\),
The implied constant can be computed quite explicitly, by retracing the proofs above; we confine ourselves to pointing out that it depends only on L, r and f, and that it explodes as \(L \rightarrow \infty \) or \(r \rightarrow 1\).
2.10 Digression on a Delicate Limit Case
We have noted in Example 2.2 that the case of a cost designed according to an underlying directed circulant graph structure is limit among those satisfying our assumptions, in the sense that Definition 2.18 holds with \(\varrho = 1\).
Results like Lemmata 2.27, 2.28, 2.30, 2.39 and 2.40 continue to hold, but the other methods used in the previous sections are not refined enough to successfully prove all the previous theorems for those limit cases, even though, along with Remark 2.44, they are sufficient in order to establish \(\ell ^\infty \)-stability at the level of the system for c; that is, convergence in \(\ell ^\infty (\mathbb {N}^2)\) of the solution to (2.24) to a solution of (2.45).
On the other hand, having a closer look at the simplest limit case, which is the directed chain given by the choice \(g = 0\), \(f^0_{00} = f^0_{11} = 1 = -f^0_{01} = -f^0_{10}\) and \(f^0_{hk} = 0\) for all other h, k, we note that we are also able to compute QSDE solutions quite easily, thanks to formula (2.47). In this case we have \(\varphi (z,0) = 1 - z\), and we find the expansion
yielding
As by Stirling’s formula \(|c^\pm _{hk} |\asymp (h+k)^{\delta _{0,hk}-\frac{5}{2}}\), one easily sees that these coefficients well-define a QSDE solution, hence the limit ergodic Nash system has a solution.
One can also note that the coefficients enjoy the property that
where \({\bar{c}}\) is either \(c^+\) or \(c^-\); this can be seen from (2.57) or proved by induction using system (2.45). In fact, the same can be done for system (2.24), so that property (2.58) also holds for theFootnote 13 solution of (2.24) on [0, T], for any fixed \(T>0\). Therefore, the information about the coefficients of a prospective QSD or QSDE solution is encoded in the functions of one complex variable \(\Xi _0(t,\cdot ) :=\Xi (t,\cdot ,0)\) and \(\bar{\Xi }_0 = \bar{\Xi }(\cdot ,0)\), namely
This peculiar fact is specific of the directed chain. It could be useful as it allows to conclude existence of a solution to the infinite-dimensional evolutive Nash system provided that the sequence of functions \((c_{0k})_{k\in \mathbb {N}}\) is monotone,Footnote 14 yet this would not still be enough to deal with convergence to an ergodic solution.
Another property is instead shared by all problems having f of the form (2.1): the polynomial \(\varphi \) factors as \(\varphi (z,w) = \xi ^2(z)\xi ^2(w)\); this makes \(\Xi \) and \(\bar{\Xi }\) functions of \((\xi (z),\xi (w))\), possibly helping in the analysis of the aforementioned limit cases.
3 Mean-Field-Like Games: Long-Time Existence
In this part we consider the Nash system (2.8) without the shift-invariance assumption. Instead, we make a mean-field-like assumption, and look for conditions that guarantee the existence of solutions on arbitrarily large time horizon T. Given \((a^i_{jk})_{i,j,k} \in \mathscr {S}(N)^N\) we will use the notation B(a) for the matrix \(B(a)_{hk} = a^{h}_{hk}\). Given the i-th vector \(e^i\) of the canonical basis of \({\mathbb {R}}^N\), we will write \(E^i = e^i (e^i)^\textsf{T}\). The indices will range over [[N]].
Note that system (2.8) can be rewritten in a forward form as
where \(c^i(t), f^i(t),g^i \in \mathscr {S}(N)\) for all \(i \in [[N]]\) and \(t \in [0,T]\).
Theorem 3.1
Let \(T>0\) and let \(f \in L^1((0,T);\mathscr {S}(N)^N)\) and \(g \in \mathscr {S}(N)^N\) satisfy
Suppose that \(B(f) \ge -K_f I\) and \(B(g) \ge -K_g I\) for some constants \(K_f\) and \(K_g\) such that
Then there exists \(N_0 \in \mathbb {N}\) such that if \(N > N_0\) there exists a unique absolutely continuous solution c to (3.1) on [0, T), which satisfies
for some constant C which is independent of N.
Note that (3.3) is automatically satisfied when \(K_f, K_g \le 0\) for any \(T > 0\). Otherwise, for fixed \(K_f\) and \(K_g\), it poses a restriction on the size of T (or, similarly, a restriction on the size of \((K_f)_+\) and \((K_g)_+\) once T is fixed). Back to the value functions \(u^i\), the previous estimate reads as follows:
in particular,
Remark 3.2
We use the terminology mean-field-like since any \(f^i\) such that
satisfies (3.2), with \(\kappa _f\) depending on C (and not on N). In turn, the previous inequality is satisfied when \(f^i(x) = V^i(x^i, (N-1)^{-1} \sum _{j \ne i} \delta _{x^j})\), where \(V^i\) is a smooth enough function defined over \({\mathbb {R}}^d \times \mathcal {P}({\mathbb {R}}^d)\) (see for instance [10, Proposition 6.1.1]). Note that \(V^i\) here may depend on i, and it is not the same for all i like in the standard MFG setting.
Remark 3.3
Theorem 2.17 presented in the previous section and Theorem 3.1 can be compared as follows; the former is about short-time solutions for shift-invariant Mean-Field-like problems, while the latter provides (possibly) long-time solutions, under further structural conditions involving B(f) and B(g). Note that the former requires shift-invariance, that is all players are identical (though they do not observe the population through the empirical measure necessarily). Note also that the former is more precise in terms of estimates, that are pointwise with respect to h, k, i, while the latter involves \(\ell ^2\) norms. Clearly, the lines of two proofs are rather different.
The proof of the theorem is based on the following lemmata.
Lemma 3.4
Let c be an absolutely continuous solution to (3.1) with \(B(c) \ge -MI\) on [0, T) for some \(M \in {\mathbb {R}}\). Assume (3.2). Then the following estimates hold on [0, T):
where \(M_+ = M \vee 0\). As a consequence, c continuously extends on [0, T].
Proof
Multiplying the equation for \(c^i_{hk}\) by \(c^i_{hk}\) and summing over h and \(k\ne i\) we have
where we have set \({\hat{c}}^i_{hk} :=c^i_{hk}(1-\delta _{hi})\) and \({\tilde{c}}^i_{hk} = c^i_{hk}(1-\delta _{ki})\). It follows that
thus, by Gronwall’s inequality, (3.5) is proved. Multiplying the equation for \(c^i_{ik}\) by \(c^i_{ik}\) and summing over \(i \ne k\) we obtain
where \({\hat{B}}(c)_{hk} = B(c)_{hk}\) if \(h = k\) and it is null otherwise. By (3.5)
We have
that is,
By Gronwall’s inequality one finds (3.6). Consider at this point that \(c^i_{ii}\) solves
Therefore, multiplying equation (3.8) by \(c^i_{ii}\) one obtains
and thus (3.7) by Gronwall’s inquality. \(\square \)
Lemma 3.5
Under the hypotheses of Theorem 3.1, let c be as in Lemma 3.4. Suppose that
Then there exists \(N_*(T)\) such that \(B(c)(T) > -MI\) provided that \(N > N_*(T)\). Furthermore, the map \(T \mapsto N_*(T)\) is continuous on \([0,+\infty )\).
Proof
Consider that B(c) solves the equation
Let now \(\xi \) solve the linear equation \({\dot{\xi }} = B(c)^\textsf{T}\xi \) on [0, T) with terminal condition \(\xi (T) = \zeta \), for some arbitrary \(\zeta \in \mathbb {S}^{N-1}\); note that since \(\frac{\textrm{d}}{\textrm{d}t} |\xi |^2 \ge -4\,M|\xi |^2\) we have
By (3.9) we get
If \(K_f < 0\) and N is large enough, then \(B(c)(T) > - \nu K_g I\), where
thus it is easily seen that \(B(c)(T) > - MI\). If \(K_f \ge 0\), then
It follows that in order to have \(B(c)(T) > -MI\) it suffices that
where \(h(z) :=(e^{2z}-1)/(2z)\). This is guaranteed by our assumptions on \(K_g\), \(K_f\) and M (by which \(\nu K_g + T K_f {(1 \vee h(MT))} < M\)), provided that N is large enough. The continuity of \(N_*(T)\) is easily seen as one can write it explicitly using the estimates above. \(\square \)
Proof of Theorem 3.1
Fix M according to Lemma 3.5 (note that this implies that \(M > K_g\)). By the Cauchy–Lipschitz theorem there exists \(\tau > 0\) such that (3.1) has a unique absolutely continuous solution on \([0,\tau )\). By taking \(\tau \) smaller if necessary, we may suppose that by continuity \(B(c) > -M I\) on \([0,\tau )\). Then
is well-defined. Seeking for a contradiction, suppose that \(\bar{\tau }< T\). Let \(N_0 :=\max _{[0,T]} N_*\), where \(N_*\) is given by Lemma 3.5. By Lemma 3.4, c continuously extends on \([0,{\bar{\tau }}]\), with \(B(c)({\bar{\tau }}) > -MI\) as guaranteed by Lemma 3.5, thanks to our choice of \(N_0\). By the Cauchy–Lipschitz theorem one can extend the solution on \([0,\tau ')\) for some \(\tau ' > {\bar{\tau }}\) and by continuity we may suppose that \(B(c) > -MI\) on \([0,\tau ')\). This contradicts the maximality of \({\bar{\tau }}\), thus \({\bar{\tau }} \ge T\). Finally, estimate (3.4) follows from (3.5), (3.6) and (3.7). This concludes the proof. \(\square \)
Data availability
No datasets were generated or analysed during the current study.
Notes
The symbol \(\otimes \) denotes the usual Kronecker product.
We mean \((\beta \otimes \beta )_{hk} = \beta _h \beta _k\) for all \(h,k \in \mathbb {Z}\).
We will use the notation \(\mathbb {D}_r(z)\) for the open complex disc of radius r about z, but we will omit the center when it is 0 and the radius when it is 1, so that, e.g., we have \(\mathbb {D}_r = \mathbb {D}_r(0) = r \mathbb {D}\).
The symbol \(\sqrt{\cdot }\) denotes the principal branch of the square root function.
Note that the limit operator is exactly the matrix \(f \in \mathscr {S}(\ell )\) seen as embedded into the space of symmetric linear operators on \({\mathbb {R}}^{\mathbb {Z}}\).
A natural immersion \(\imath :\ell ^1(\mathbb {N}^2) \hookrightarrow \ell ^1(\mathbb {Z}^2)\) is given by \(\imath (x)^i = x^i\) if \(i \in \mathbb {N}\) and \(\imath (x)^i = 0\) otherwise.
That is, \((h',k') \prec (h,k)\) if and only if \(h'\le h\) and \(k'\le k\) with at least one strict inequality.
In other words, we are saying that \(\partial ^h_z \partial ^k_w {\hat{\Xi }}(t,0,0) = h!\, k!\, {\hat{c}}_{hk}(t)\), where the coefficients \({\hat{c}}_{hk}\) are those in the proof of Lemma 2.27.
And for all higher order derivatives.
With this notation we mean that all components are those of \(\alpha ^*\) but the i-th one, which is a suitable \({\mathbb {R}}^d\)-valued function \(\psi = \psi (t,x)\). Note that \(\alpha ^* \in \mathcal {A}_{R,L}\) for any R and admissible L.
E.g., \(r' = \frac{1}{2}(r+\varrho )\).
For instance, \(\mathfrak {d}(t) = \big ( t \vee \tfrac{1}{3}\Vert \xi \Vert _{\infty ;\bar{\mathbb {D}}_{r'}}^{-2} \big )\epsilon \).
Note that it is indeed unique, as Lemma 2.27 is still true with the same proof.
As it would seem by computing the first functions of the sequence....
References
Aurell A, Carmona R, Laurière M (2022) Stochastic graphon games: II. The linear-quadratic case. Appl Math Opt 85(3):26–33
Baldi P (2017) Stochastic calculus. An introduction through theory and exercises. Universitext Springer, Cham, pp 10–12
Bardi M (2012) Explicit solutions of some linear-quadratic mean field games. Netw Heterog Media 7(2):243–261
Bayraktar E, Cecchin A, Cohen A, Delarue F (2022) Finite state mean field games with Wright-Fisher common noise as limits of \(N\)-player weighted games. Math Operat Res 47(4):2840–2890
Bayraktar E, Wu R, Zhang X (2023) Propagation of chaos of forward-backward stochastic differential equations with graphon interactions. Appl Math Opt 88(1):25–44
Bensoussan A, Li B, Yam SCP (2023) Linear quadratic extended mean field games and control problems arXiv:2311.05176
Bensoussan A, Sung KCJ, Yam SCP, Yung SP (2016) Linear-quadratic mean field games. J Opt Theory Appl 169(2):496–529
Caines PE, Huang M (2021) Graphon mean field games and their equations. SIAM J Control Opt 59(6):4373–4399
Cardaliaguet P, Cirant M, Porretta A (2020) Remarks on nash equilibria in mean field game models with a major player. Proc Am Math Soc 148:4241–4255
Cardaliaguet P, Delarue F, Lasry J-M, Lions P-L (2019) The master equation and the convergence problem in mean field games, vol 201. Annals of Mathematics Studies. Princeton University Press, Princeton, NJ
Cardaliaguet P, Porretta A (2020) An introduction to mean field game theory. Mean field games, vol 2281. Lecture Notes in Math. Springer, Cham, pp 1–158
Carmona R , Delarue F (2018) Probabilistic theory of mean field games with applications. I, volume 83 of Probability Theory and Stochastic Modelling. Springer, Cham, Mean field FBSDEs, control, and games
Cecchin A (2021) Finite state \(N\)-agent and mean field control problems. ESAIM Control Opt Calc Var 27:31–33
Cirant, M, Redaelli DF (2024) A priori estimates and large population limits for some nonsymmetric Nash systems with semimonotonicity, In preparation
Daudin S, Delarue F, Jackson J (2023) On the optimal rate for the convergence problem in mean field control, arXiv:2305.08423
Delarue F (2017) Mean field games: a toy model on an Erdös-Renyi graph. Journées MAS 2016 de la SMAI–Phénomènes complexes et hétérogènes, volume 60 of ESAIM Proc. Surveys. EDP Sci, Les Ulis, pp 1–26
Delarue F, Foguen Tchuendom R (2020) Selection of equilibria in a linear quadratic mean-field game. Stochastic Process Appl 130(2):1000–1040
Djete MF (2023) Large population games with interactions through controls and common noise: convergence results and equivalence between open-loop and closed-loop controls. ESAIM Control Opt Calc Var 29:39–42
Feleqi E (2013) The derivation of ergodic mean field game equations for several populations of players. Dyn Games Appl 3(4):523–536
Feng Y, Fouque JP, Ichiba T (2020) Linear-quadratic stochastic differential games on directed chain networks, arXiv:2003.08840
Fischer M (2017) On the connection between symmetric \(N\)-player games and mean field games. Ann Appl Probab 27(2):757–810
Friedman A (1972) Stochastic differential games. J Differ Equ 11:79–108
Gangbo W, Mészáros AR, Mou C, Zhang J (2022) Mean field games master equations with nonseparable Hamiltonians and displacement monotonicity. Ann Probab 50(6):2178–2217
Graber PJ (2016) Linear quadratic mean field type control and mean field games with common noise, with application to production of an exhaustible resource. Appl Math Opt 74(3):459–486
Huang M, Malhamé RP, Caines PE (2006) Large population stochastic dynamic games: closed-loop McKean-Vlasov systems and the Nash certainty equivalence principle. Commun Inf Syst 6(3):221–251
Jackson J, Lacker D (2023) Approximately optimal distributed stochastic controls beyond the mean field setting. arXiv:2301.02901
Jackson J, Tangpi L (2023) Quantitative convergence for displacement monotone mean field games with controlled volatility . arXiv:2304.04543
Lacker D (2020) On the convergence of closed-loop Nash equilibria to the mean field game limit. Ann Appl Probab 30(4):1693–1761
Lacker D, Soret A (2022) A case study on stochastic games on large graphs in mean field and sparse regimes. Math Oper Res 47(2):1530–1565
Lacker D, Soret A (2023) A label-state formulation of stochastic graphon games and approximate equilibria on large networks. Math Oper Res 48(4):1987–2018
Lasry J-M, Lions P-L (2007) Mean field games. Jpn J Math 2(1):229–260
Li M, Mou C, Wu Z, Zhou C (2023) Linear-quadratic mean field games of controls with non-monotone data. Trans Am Math Soc 376(6):4105–4143
Li X, Shi J, Yong J (2021) Mean-field linear-quadratic stochastic differential games in an infinite horizon. ESAIM Control Opt Calc Var 27:81–140
Acknowledgements
The authors are indebted with the anonymous referees for their valuable suggestions
Funding
Open access funding provided by Universitá degli Studi di Padova within the CRUI-CARE Agreement. The authors are partially supported by the Gruppo Nazionale per l’Analisi Matematica, la Probabilità e le loro Applicazioni (GNAMPA) of the Istituto Nazionale di Alta Matematica (INdAM), Italy. M.C. partially supported by the King Abdullah University of Science and Technology (KAUST) project CRG2021-4674 “Mean-Field Games: models, theory, and computational aspects”, and by the project funded by the EuropeanUnion–NextGenerationEU under the National Recovery and Resilience Plan (NRRP), Mission 4 Component 2 Investment 1.1-Call PRIN 2022 No. 104 of February 2, 2022 of Italian Ministry of University and Research; Project 2022W58BJ5 (subject area: PE-Physical Sciences and Engineering) “PDEs and optimal control methods in mean field games, population dynamics and multi-agent models”. D.F.R. partially supported by Fondazione Cassa di Risparmio di Padova e Rovigo, Italy.
Author information
Authors and Affiliations
Contributions
Both authors wrote and reviewed the manuscript.
Corresponding author
Ethics declarations
Conflict of interest
The authors have no Conflict of interest as defined by Springer, or other interests that might be perceived to influence the results and/or discussion reported in this paper.
Ethical approval
Not applicable.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
This article is part of the topical collection “Optimal control and differential games: theory, numerics and applications. In memory of Maurizio Falcone”.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Cirant, M., Redaelli, D.F. Some Remarks on Linear-Quadratic Closed-Loop Games with Many Players. Dyn Games Appl (2024). https://doi.org/10.1007/s13235-024-00568-5
Accepted:
Published:
DOI: https://doi.org/10.1007/s13235-024-00568-5