Abstract
We consider a noncooperative game in infinite time horizon, with linear dynamics and exponentially discounted quadratic costs. Assuming that the state space is one-dimensional, we prove that the Nash equilibrium solution in feedback form is stable under nonlinear perturbations. The analysis shows that, in a generic setting, the linear-quadratic game can have either one or infinitely many feedback equilibrium solutions. For each of these, a nearby solution of the perturbed nonlinear game can be constructed.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
1 Introduction
Noncooperative differential games with linear dynamics and quadratic costs have been widely studied in recent literature [2, 11–14, 16, 19]. The existence of explicit solutions, determined by a finite set of algebraic equations, makes them a very attractive modeling tool. They have been used in a variety of applications, particularly in economics and management science.
In most of these applications, the underlying dynamical equations are nonlinear and the cost functions are not exactly quadratic polynomials. Prices, interest rates, inventory levels, etc., assume only positive values, while in a linear-quadratic game all functions must be defined on the whole real line.
In spite of these discrepancies, linear-quadratic games can be regarded as a natural approximation to more realistic models. In an attempt to justify this approximation, one is led to the following
Question Starting from a linear-quadratic differential game for two players, suppose that some small, nonlinear perturbations are added to the dynamics of the system and to the cost functions. Does the perturbed game still have a Nash equilibrium solution in feedback form, close to the original one?
For noncooperative differential games on a finite time interval [0, T], the analysis in [5, 9] shows that the answer is largely negative. Indeed, the value functions for the two players are determined by a system of two Hamilton–Jacobi equations, with given terminal conditions at \(t=T\). When the state space has dimension \(n=1\), in some cases this leads to a hyperbolic system, which has unique solutions [8]. However, in dimension \(n\ge 2\) the system is not hyperbolic and the backward Cauchy problem is generically ill posed. A small nonlinear perturbation of the terminal costs may produce a large change in the solution.
Aim of the present paper is to prove some positive results, in the case of differential games for two players in infinite time horizon, with exponentially discounted costs.
Let \(u_1,u_2\) be the controls implemented by the two players, and assume that the state of the system \(x\in I\!\!R^n\) evolves according to
Moreover, let
be the exponentially discounted costs. Here and in the sequel, an upper dot denotes a derivative w.r.t. time, while the constant \(\gamma >0\) is a discount rate.
Since the dynamics and the payoffs do not depend explicitly on time, it is natural to seek a Nash equilibrium solution consisting of time-independent feedbacks.
Definition 1
A pair of functions \(x\mapsto u_1^*(x)\in U_1\), \(x\mapsto u_2^*(x)\in U_2\) will be called a Nash equilibrium solution to the noncooperative game (1.1)–(1.2) provided that:
-
(i)
The map \(u_1^*(\cdot )\) is an optimal feedback, in connection with the optimal control problem for the first player:
$$\begin{aligned} \hbox {minimize}\int _0^\infty e^{-\gamma t}\,\phi _1 \bigl (x(t), u_1(t), u_2^*(x(t))\bigr )\, \mathrm{d}t \end{aligned}$$subject to
$$\begin{aligned} \dot{x} =f(x, u_1,u_2^*(x)),\quad u_1(t)\in U_1. \end{aligned}$$ -
(ii)
The map \(u_2^*(\cdot )\) is an optimal feedback, in connection with the optimal control problem for the second player:
$$\begin{aligned} \hbox {minimize}\int _0^\infty e^{-\gamma t}\,\phi _2 \bigl (x(t), u_1^*(x(t)), u_2\bigr )\, \mathrm{d}t \end{aligned}$$subject to
$$\begin{aligned} \dot{x} =f(x, u_1^*(x),u_2),\quad u_2(t)\in U_2. \end{aligned}$$
This equilibrium solution can be found by solving the corresponding PDEs for the value functions. Assuming that the feedbacks \(u_1^*, u_2^*\) implemented the two players are sufficiently regular, let \(t\mapsto x(t, x_0)\) be the trajectory of the system starting from an initial state \(x_0\). This is obtained by solving the Cauchy problem
The value functions for the two players are then computed by
In the following we assume:
-
(H) For any \(x\in I\!\!R^n\) and every pair of vectors \((\xi _1, \xi _2)\in I\!\!R^n\times I\!\!R^n\), there exists a unique pair \((u^\sharp _1,\, u^\sharp _2)\in U_1\times U_2\) such that
$$\begin{aligned} u^\sharp _1(x, \xi _1, \xi _2)= & {} \hbox {arg}\!\min _{\omega \in U_1} \Big \{ \xi _1\cdot f(x, \omega , u^\sharp _2) + \phi _1( x, \omega , u^\sharp _2)\Big \}, \end{aligned}$$(1.5)$$\begin{aligned} u^\sharp _2(x, \xi _1, \xi _2)= & {} \hbox {arg}\!\min _{\omega \in U_2} \Big \{ \xi _2\cdot f(x, \, u^\sharp _1, \omega ) + \phi _2(x, u^\sharp _1,\, \omega )\Big \}. \end{aligned}$$(1.6)
On an open region where \(V_1,V_2\) are continuously differentiable, a dynamic programming argument (see for example [1, 6, 15]) shows that these value functions satisfy a system of Hamilton–Jacobi equations:
where, for \(i=1,2\),
If a solution to system (1.7) can be found, then the Nash equilibrium feedback controls are computed by
In general, system (1.7) is highly nonlinear and difficult to solve (globally on the entire space \(I\!\!R^n\)). Even in one space dimension, examples are known where this system admits infinitely many solutions, or no solution at all [7, 17]. In the case of linear dynamics
and quadratic cost functionals
a particular solution to the system of PDEs (1.7) can be found in terms of quadratic polynomials. Namely
where the superscript \(^T\) denotes transposition. To determine this particular solution, it suffices to solve a system of algebraic equations for the coefficients of the matrices \(M_1,M_2\), the vectors \(\mathbf{n}_1,\mathbf{n}_2\), and the scalars \(e_1,e_2\).
Our present goal is to understand whether this solution is stable w.r.t. small nonlinear perturbations of the dynamics and of the cost functionals. This issue can be studied in two separate settings: either on a bounded domain or on the entire space \(I\!\!R^n\).
-
(I)
Let \(u^*_1, u^*_2\) be the affine feedback controls in a Nash equilibrium for the linear-quadratic game (1.9)–(1.10), and consider any bounded domain \(\Omega \) which is positively invariant for the corresponding dynamical system
$$\begin{aligned} \dot{x}=Ax + B_1 u^*_1(x) + B_2u_2^*(x). \end{aligned}$$(1.11)We recall that \(\Omega \) is positively invariant if, for every solution \(x(\cdot )\) of (1.11), one has the implication
$$\begin{aligned} x(0)\in \Omega \quad \implies \quad x(t)\in \Omega \quad \hbox {for all}~t\ge 0. \end{aligned}$$In this setting a natural question is whether, for a suitably small perturbation of the dynamics and of the cost functions, the system of H–J equations (1.7) admits a solution defined on \(\Omega \), close to the original one. The uniqueness of this solution is also an important issue, both for the linear-quadratic game and for the nonlinear perturbation.
-
(II)
A further question is whether a solution to the perturbed game can be constructed on the entire space \(I\!\!R^n\), under suitable assumptions on how the perturbation grows as \(|x|\rightarrow \infty \).
We remark that, for many applications, the models are meaningful only on a proper subset of \(I\!\!R^n\). For example, state variables should often satisfy nonnegativity constraints. It is thus natural to construct solutions to the system of H–J equations (1.7) on a bounded domain \(\Omega \), as long as this domain is positively invariant for the corresponding dynamics. In this case, the feedback controls \(u_i^*{:}\Omega \mapsto I\!\!R^m\) will still provide a Nash equilibrium for a game where the state is required to remain inside \(\Omega \), assuming that the cost functions \(\phi _i\) in (1.2) satisfy
In this paper we confine the analysis to the one-dimensional case, under generic assumptions (i.e., valid as the coefficients range on an open dense set). Denoting by \(\xi _i=V_i'\in I\!\!R\), \(i=1,2\), the gradients of the value functions, the Hamilton–Jacobi equations (1.7) can be reduced to a system of two implicit ODEs, of the form
where \(\Lambda \) is a suitable \(2\times 2\) matrix.
The position of the line \(\xi =\xi ^*(x)\), relative to the double cone \(\Gamma ^-\) where det\(\Lambda (x,\xi _1,\xi _2)\le 0\). Left: the line is always outside \(\Gamma ^-\). Center: the line is inside \(\Gamma ^-\) for x in some bounded interval [a, b]. Right: the line is inside \(\Gamma ^-\) for x outside a bounded interval \(\,]a,b[\,\) . Notice that trajectories of the implicit ODE (1.12) typically become singular as they approach the boundary of \(\Gamma ^-\) and cannot be prolonged any further (left figure)
In the linear-quadratic case, one can find a solution \((\xi _1^*, \xi _2^*)\) of (1.12), where each \(\xi _i^*(x)=\alpha _i x+\beta _i \) is an affine function. Moreover, the determinant \(\det \Lambda (x,\xi _1,\xi _2)\) is a homogeneous quadratic polynomial of the three variables \(x, \xi _1,\xi _2\). It vanishes on the surface of a double cone \(\Gamma ^-\), shown in Fig. 1. Looking at the relative position of the line \(\xi ^*=\bigl \{(x,\xi _1^*(x),\xi _2^*(x))\,;~~x\in I\!\!R\bigr \}\) w.r.t. the cone \(\Gamma ^-\), under generic conditions two main cases can occur.
If \(\det \Lambda (x, \xi _1^*(x), \xi _2^*(x))\not =0\) for all \(x\in I\!\!R\), then the linear-quadratic game has infinitely many solutions. In one of these solutions the feedbacks \(u_1^*, u_2^*\) are affine, while in all the others they are fully nonlinear functions. All these solutions are stable under small nonlinear perturbations of the dynamics and of the cost functions
A case where \(\det \Lambda (x, \xi _1^*(x), \xi _2^*(x))\) vanishes at two points \(\bar{x}_1<\bar{x}_1\) and the linear-quadratic game has a unique solution, which is stable under small nonlinear perturbations of the dynamics and of the cost functions. The graph of \(\xi ^*(\cdot )\) contains a heteroclinic orbit of a related dynamical system
Case 1: The line \(\xi ^*\) lies entirely outside the cone (Fig. 1, left). As a consequence, the matrix \(\Lambda (x, \xi _1, \xi _2)\) is invertible in a neighborhood of \(\xi ^*\). The system of ODEs (1.12) can thus be written in the standard form
In this case, by solving (1.13) with slightly different initial data at \(x=0\), one can construct infinitely many Nash equilibrium solutions in feedback form for the same linear-quadratic game (Fig. 2). These solutions vary continuously under small nonlinear perturbations of the data.
Case 2: The line \(\xi ^*\) is partly inside, partly outside the cone \(\Gamma ^-\) (Fig. 1, center and right), crossing the surface of the cone at two points \(P_i^* = (\bar{x}_i, \xi _1^*(\bar{x}_i), \xi _2^*(\bar{x}_i))\), \(i=1,2\), with \(\bar{x}_1<\bar{x}_2\). In this case, the equation det \(\Lambda (x,\xi _1,\xi _2)=0\) locally defines two surfaces \(\Sigma _1\), \(\Sigma _2\), near the points \(P_1^*\), \(P_2^*\), respectively (see Fig. 3). Always for the linear-quadratic game, a detailed analysis shows that a smooth solution of (1.12) can cross the surface \(\Sigma _1\) only along a special curve \(\gamma _1\subset \Sigma _1\). By the same argument, it can cross the surface \(\Sigma _2\) only along a curve \(\gamma _2\subset \Sigma _2\).
In general, the graph of a solution to (1.12) can be constructed as a concatenation of orbits of a related dynamical system on \(I\!\!R^3\), where all points on the curves \(\gamma _1\) and \(\gamma _2\) are steady states. Smooth solutions correspond to heteroclinic orbits, connecting a point on \(\gamma _1\) to a point on \(\gamma _2\). Depending on the properties of the two equilibrium points \(P_1^*\), \(P_2^*\) (source, saddle, sink), there can be a single orbit, or else a 1- or a 2-parameter family of heteroclinic orbits. In a typical case, shown in Fig. 3, the points \(P_1^*\in \gamma _1\) and \(P_2^*\in \gamma _2\) are both saddle points. The 2-dimensional center-stable manifold \({\mathcal M}_1\) through \(P_1^*\) and the 2-dimensional center-unstable manifold \({\mathcal M}_2\) through \(P_2^*\) are transversal. Their intersection is the unique heteroclinic orbit joining \(\gamma _1\) with \(\gamma _2\).
Under generic assumptions, all of the above configurations are structurally stable. Namely, the curves \(\gamma _i\) and the manifolds \({\mathcal M}_i\) are still well defined in the presence of a small nonlinear perturbation. The family of heteroclinic orbits is preserved as well.
The remainder of the paper is organized as follows. In Sect. 2 we recall the system of Hamilton–Jacobi equations for a noncooperative 2-player game in infinite time horizon, while in Sect. 3 we review the construction of affine feedback solutions for the linear-quadratic case. Solutions of (1.12) on a bounded interval are studied in Sect. 4. The main tool in the analysis is a representation of their graph as a concatenation of orbits, for a related dynamical system. The case of a heteroclinic orbit joining two saddle points, illustrated in Fig. 3, is worked out in detail in Sect. 5. This is the most pleasing case, where the linear-quadratic game has a unique feedback solution, which is stable under small nonlinear perturbations. In Sect. 6 we prove that, under suitable growth conditions, feedback solutions to the perturbed nonlinear game can be extended to the entire real line. Up to this section, all of our analysis is concerned with the system of ODEs (1.12) for the derivatives of the value functions. Finally, in Sect. 7 we show that, by constructing a solution to (1.12), one can recover the value functions \(V_1,V_2\) in (1.7) and provide a Nash equilibrium solution to the noncooperative differential game.
Remark 1
All of our results hold under “generic” assumptions on the coefficients \(a_0, b_i, R_i,S_i\) of the linear-quadratic game (3.2)–(3.3). In other words, they are valid as these coefficients range in an open dense set. In some cases, this set can be precisely determined. In general, this set is described by a finite set of inequalities:
where the \(\Phi _j\) are analytic functions. Having constructed examples where these inequalities do hold, by analyticity we conclude that they must hold on an open dense set.
In earlier literature, a few examples of feedback equilibria for nonlinear infinite horizon differential games have been studied in [4, 7, 17]
2 The Basic Setting
Throughout the following, we consider a system whose dynamics is linear w.r.t. the controls \(u_1, u_2\). The state \(x\in I\!\!R\) thus evolves according to
Here \(f,g_1,g_2{:}I\!\!R\mapsto I\!\!R\) are smooth maps with sublinear growth:
for some constant C. The cost functionals for the two players are
Call \(V_1,V_2\) the value functions in a Nash equilibrium in feedback form. By the previous analysis these functions satisfy the system of Hamilton–Jacobi equations (1.7), where
The optimal feedback controls are given by the formulas
Inserting (2.5) in (2.4), from (1.7) one obtains the implicit system of ODEs
Differentiating (2.6) w.r.t. x we obtain a system of two ODEs for the derivatives of the value functions: \(\xi _1= V_1'\), \(\xi _2= V_2'\). Namely
where
When both players adopt the equilibrium feedback strategies (2.5), by (2.1) the system evolves according to
3 Review of the Linear-Quadratic Case
In this section we review the construction of feedback equilibrium solutions for the linear-quadratic case. Assume that in (2.1) and (2.3) one has
for some constants \(a_i,b_i, R_ i, S_i\). The dynamics and cost functionals thus become
Notice that, if \(a_0\not = 0\), the more general case where \(f(x) = a_0 x + b_0\) can be reduced to (3.2) by a translation of coordinates. The following assumptions will be used throughout.
-
(A1)
The coefficients \(S_1, S_2, \gamma \) are strictly positive. Moreover, one of the following alternatives holds.
-
Case 1:
\(a_0>{\gamma \over 2}>0\) and \(S_1b_1^2+ S_2 b_2^2~>~ (2a_0-\gamma )^2/2\).
-
Case 2:
\(a_0<0\).
-
Case 1:
In the linear-quadratic case, the matrix \(\Lambda \) and the right-hand side \(\psi \) in the system of ODEs (2.7) take the special form
Under the assumption (A1), one can construct an explicit solution where \(\xi _1,\xi _2\) are affine functions, namely
Indeed, inserting (3.4)–(3.6) in (2.7) one obtains
Since these equalities must hold for every x, we obtain a system of four quadratic equations for the unknowns \(\alpha _1,\alpha _2,\beta _1,\beta _2\).
To solve (3.7), it is convenient to introduce the variables
The first two equations in (3.7) can then be written as
Equivalently,
Introducing the further variables
we obtain
By the first equation, \(Y_2= (K_1-K_2)/Y_1\). Inserting this expression in the second equation, we find
By the assumption (A1) and the definitions (3.8), two cases can arise.
Case 1: \(a_0>{\gamma \over 2}>0\) and \(K_1+K_2> 1/2\). This implies
Hence Eq. (3.12) has a solution \(Y_1^* > 0\).
Case 2: \(a_0<0\). This implies
Hence Eq. (3.12) has a solution \(Y_1^*< -1\).
In both cases, reverting to the original variables we find
As soon as \(\alpha _1,\alpha _2\) have been determined, the last two equations in (3.7) provide a linear system for the variables \(\beta _1,\beta _2\). This can be uniquely solved under the condition
We observe that, under the assumption (A1), this inequality always holds. Indeed, dividing by \(A^2\), (3.16) can be rewritten as
Equivalently,
Two cases are considered:
-
If \(a_0>{\gamma \over 2}>0\) and \(K_1+K_2 > 1/2\) then
$$\begin{aligned} Y_1^*> 0\quad \mathrm {and}\quad {a_0\over A} = {a_0\over 2a_0-\gamma } > {1\over 2}. \end{aligned}$$Thus
$$\begin{aligned} Y_1^*+{a_0\over A}> {1\over 2}(Y_1^*+1) > 0. \end{aligned}$$ -
If \(a_0<0\) then
$$\begin{aligned} Y^*_1 \le -1\quad \mathrm {and}\quad 0< {a_0\over A} = {a_0\over 2a_0-\gamma } < {1\over 2}. \end{aligned}$$Hence
$$\begin{aligned} Y_1^*+{a_0\over A}< {1\over 2}(Y_1^*+1) < 0. \end{aligned}$$
Therefore,
in Case 1 as well as in Case 2. Hence (3.17) holds.
Having determined the constants \(\alpha _i, \beta _i\), the affine functions \(\xi _1^*,\xi _2^*\) in (3.6) yield a solution to the system (2.7). According to (2.5), the corresponding Nash equilibrium feedback controls are
In the following, given a solution \((\alpha _1,\alpha _2,\beta _1,\beta _2)\) of the algebraic system (3.7), recalling (3.8) we shall write
Remark 2
When the Nash equilibrium feedbacks (3.19) are implemented, the system (3.2) evolves according to
Under the assumption (A1), if either \(a_0<0\) or \(0<\gamma \le a_0\), then the stationary point
is an asymptotically stable equilibrium for the ODE (3.21). However, this equilibrium is unstable when \(a_0=\gamma /2\).
To prove the above claims, we examine various cases.
-
If \(a_0<0\), then \(A=2a_0-\gamma <0\), while \(X_1^*+X^*_2=Y_1^*+1< 0\). Therefore
$$\begin{aligned} a_0 - b_1^2 \alpha _1- b_2^2\alpha _2 = a_0 -A(X_1^*+X^*_2) \le a_0 < 0. \end{aligned}$$ -
If \(0<\gamma \le a_0\) and \(K_1+K_2>1/2\), recalling that \(X_1^*+X^*_2=Y_1^*+1 > 1\) and \(A= 2a_0-\gamma >0\), we obtain
$$\begin{aligned} a_0 - b_1^2 \alpha _1- b_2^2\alpha _2 = a_0 -A(X_1^*+X^*_2) < a_0-A = \gamma -a_0 \le 0. \end{aligned}$$ -
On the other hand, if \(0<\gamma = 2 a_0\), then \(A=0\) and
$$\begin{aligned} a_0 - b_1^2 \alpha _1- b_2^2\alpha _2 = a_0 -A(X_1^*+X^*_2) = a_0 > 0. \end{aligned}$$
Remark 3
Under the assumption (A1) one has \(\alpha _1,\alpha _2>0\). Therefore, the value functions of both players approach \(+\infty \) as \(x\rightarrow \pm \infty \).
Indeed, from (3.9) it follows
Without loss of generality we can assume that \(X_1^*\ge X_2^*\). There are two cases:
-
If \(a_0>{\gamma \over 2}>0\) and \(K_1+K_2>1/2\) then \(A=2a_0-\gamma >0\) and \(X^*_1+X^*_2=Y_1^*+1>0\). This implies that \(X^*_1>0\) and
$$\begin{aligned} X^*_2+2X^*_1-1 = X^*_1+Y_1^* > 0. \end{aligned}$$Recalling (3.23) we have \(X^*_2>0\). Thus
$$\begin{aligned} \alpha _i = {AX^*_i\over b_i^2} > 0\qquad i=1,2. \end{aligned}$$(3.24) -
If \(a_0<0\) then \(A=2a_0-\gamma <0\) and \(X^*_1+X^*_2=Y_1^*+1<0\). This implies that \(X^*_2<0\) and
$$\begin{aligned} X^*_1+2X^*_2-1 = X^*_2+Y_1^* < 0. \end{aligned}$$Recalling (3.23) we have \(X^*_1<0\). Hence (3.24) again holds.
4 Perturbed Solutions on a Bounded Domain
Together with the linear-quadratic game (1.9)–(1.10), we now consider a perturbed game, where the dynamics and the cost functions have the form
for some perturbations \(f_0, h_1,h_2,\eta _1,\eta _2\). The gradients of the value functions \(\xi _i(x)= V_i'(x)\) will again satisfy the implicit system of ODEs
where now
Throughout the following, we assume that the condition (A1) holds, and let
be the gradients of the value functions in the affine solution of the linear-quadratic game. We denote by
the corresponding matrix at (3.4). Its determinant is
Under generic assumptions on the coefficients, \(\det \Lambda ^*(x)\) is a polynomial of degree two, either with no real root or with two distinct roots. These two cases are qualitatively very different.
4.1 Det \(\Lambda ^*\) has no Real Roots
In this case, the matrix \(\Lambda ^*(x)\) is invertible for every \(x\in I\!\!R\). In a neighborhood of the line \(\xi ^*\) the system (4.3) can be written in explicit form as
In this setting, the standard theory of uniqueness and continuous dependence for solutions to ODEs yields
Theorem 1
Assume that the quadratic polynomial \(\det \Lambda ^*(x)\) in (4.8) has no real roots, and let \(\Omega \subset I\!\!R\) be any bounded interval. Given \(\varepsilon >0\), there exist \(\delta _0, \delta _1>0\) such that the following holds. Consider any point \(x_0\in \Omega \), any perturbations \(f_0,\eta _1,\eta _2\in {\mathcal C}^1(\Omega )\), \(h_1,h_2\in {\mathcal C}^0(\Omega )\), and any data \(\bar{\xi }_1,\bar{\xi }_2\in I\!\!R\), satisfying
Then the ODE (4.3)–(4.5) has a unique solution on \(\Omega \), with initial data
Moreover, this solution satisfies
Notice that in this case, setting \(f_0=h_1=h_2=\eta _1=\eta _2=0\), one can already find infinitely many solutions to the implicit system of ODEs (4.3) on the interval \(\Omega \). One of these solutions is affine, given by (3.6), while the others are fully nonlinear. In Sect. 6 we shall analyze whether these nonlinear solutions can be extended beyond \(\Omega \), to the entire real line.
Remark 4
Assume that the stationary point \(x^*\) in (3.22) is an asymptotically stable equilibrium for the linear ODE (3.21), contained in the interior of \(\Omega \). Then, choosing \(\delta _0,\delta _1>0\) sufficiently small in (4.10)–(4.11), the interval \(\Omega \) will be positively invariant for the corresponding feedback dynamics
Example 1
Consider the linear system
and the cost functions
The corresponding constants in (3.8) are
Solving (3.12) and (3.11) we obtain
After a few computations, we find that (4.3) admits a linear solution
The corresponding dynamic is
having \(x^*=0\) as asymptotically stable equilibrium point. By (4.8), the determinant of the corresponding matrix \(\Lambda ^*\) is
which is always positive.
4.2 Det \(\Lambda ^*\) has Two Real Roots
We now assume that the determinant of the matrix \(\Lambda ^*(x)\) at (4.7) vanishes at two distinct points \(\bar{x}_1<\bar{x}_2\) and so that
where \(x^*\) is the stationary point in (3.22) and \(C_{\Lambda }\) is a nonzero constant. Notice that, if this assumption holds, the coefficients \(R_i\) in (3.3) and \(\beta _i\) in (4.6) must satisfy
Indeed, if \((R_1,R_2)=(0,0)\), then the last two equations of (3.7) together with (3.16) imply \((\beta _1,\beta _2)=(0,0)\). By (3.6), for \(i=1,2\) we thus have \(\xi _i^*(x)=\alpha _ix\). This yields
Hence \(\bar{x}_1=\bar{x}_2 = 0\), against the assumption.
We rewrite (4.3) as a Pfaffian system
Consider the vectors
and denote by \(\mathbf{v}\wedge \mathbf{w}\) their wedge product. To construct trajectories of (4.3), we seek continuously differentiable functions \(x\mapsto (\xi _1(x), \,\xi _2(x))\) whose graph is obtained by concatenating trajectories of the system
Let us first consider the unperturbed linear-quadratic case, where the matrix \(\Lambda \) and the vector \(\psi \) are given by (3.4) and (3.5), respectively. By (4.16), the equation \(\det \Lambda ^*(x)=0\) has two solutions. These correspond to the two points
Moreover the derivative \({d\over \mathrm{d}x}\det \Lambda ^*(x)\) does not vanish at \(x=\bar{x}_1\) and at \(x=\bar{x}_2\). By the implicit function theorem, there exist two surfaces \(\Sigma _1,\Sigma _2\subset I\!\!R^3\), containing the above two points, where the determinant of \(\Lambda (x, \xi _1,\xi _2)\) vanishes (see Fig. 3).
The following analysis will show that, under generic conditions, there exist
-
Two curves \(\gamma _1\subset \Sigma _1\) and \(\gamma _2\subset \Sigma _2\), containing the points \(P_1^*\) and \(P_2^*\), respectively, where the right-hand side of (4.21) vanishes.
-
Two 2-dimensional invariant manifolds \({\mathcal M}_1,{\mathcal M}_2\), containing \(\gamma _1\) and \(\gamma _2\), respectively, which intersect transversally.
As shown in Fig. 3, the transversal intersection \({\mathcal M}_1\cap {\mathcal M}_2\) defines a heteroclinic orbit connecting \(P_1^*\) with \(P_2^*\). Since this configuration is structurally stable, it is preserved by small \({\mathcal C}^1\) perturbations of the vector fields \(\mathbf{v},\mathbf{w}\) in (4.20).
The next lemma provides generic conditions on the coefficients \(R_i, S_i, b_i\) which guarantee the existence of two smooth curves \(\gamma _1,\gamma _2\) where the vector fields \(\mathbf{v},\mathbf{w}\) are parallel.
Lemma 1
Together with (A1) and (4.16), assume that at least one of the following equalities does not hold:
Then the Jacobian matrix of the vector field \(\mathbf{v}\wedge \mathbf{w}\) has rank 2 at both points \(P_1^*\) and \(P_2^*\). As a consequence, the equation
defines two smooth curves \(\gamma _1,\gamma _2{:}[-s_0,s_0]\mapsto I\!\!R^3\), which we can parameterize by arc length, such that
Proof
1. For \(i=1,2\), since we are assuming \(x_i\not = x^*\), at the point \(P_i^*\) we have
Indeed, by (3.21) the above quantity coincides with the time derivative \(\dot{x}\), which vanishes only at the equilibrium point \(x^*\). In turn, det\(\Lambda ^*(\bar{x}_i)=0\) implies \(\Lambda _{12}\Lambda _{21}\not =0\) as well. By continuity, all four coefficients \(\Lambda _{jk}\) in (4.19)–(4.21) are nonzero on a neighborhood of \(P_i^*\). As a consequence, if the first and second components (or the first and third) of the wedge product in (4.21) vanish, then the remaining component vanishes as well. Next, by (4.16) it follows
By the implicit function theorem, it thus suffices to show that the Jacobian matrix of the map
has rank \(\ge 2\) at the points \(P^*_1\) and \(P^*_2\). In the following we denote by
this \(3\times 3\) Jacobian matrix. Moreover, for a fixed index \(i\in \{1,2\}\), we denote by \({\mathcal J}^{i*}\) this particular matrix computed at the point \(P_i^*\) in (4.22).
2. We claim that the range of the linear map \(\mathbf{u}\mapsto {\mathcal J}^{i*}\mathbf{u}\) has dimension \(\ge 2\).
Indeed, since the function \(\xi ^*=(\xi _1^*, \xi _2^*)\) in (3.6) is an affine solution of (4.3) through \(P_i^*\), the system (4.21) has a corresponding solution of form
This solution satisfies
Therefore, as \(x(t)\rightarrow \bar{x}_i\), we have the convergence
where p is defined in (4.16). This implies
Next, we observe that all functions
are linear homogeneous w.r.t. the variables \(x,\xi _1,\xi _2\). In general, if a function \(\phi {:}I\!\!R^3\mapsto I\!\!R\) is homogeneous of degree \(\beta \), so that \(\phi (t\mathbf{z})=t^\beta \phi (\mathbf{z})\), its directional derivative satisfies
Moreover, since all \(\Lambda _{ij}\) are linear, for every \(\mathbf{z}\in I\!\!R^3\) we trivially have
For convenience, for a fixed \(i\in \{1,2\}\) we shall write
We now claim that
Indeed, applying (4.28) to the function \(\phi =\Lambda _{11}\Lambda _{22}- \Lambda _{12}\Lambda _{21} \) at the point \(P_i^*\) we obtain
Using (4.28)–(4.29), we then compute
After an entirely similar computation is yielded the value of \(\kappa _3\) given in (4.30).
If \((\kappa _2,\kappa _3)\ne (0,0)\), comparing (4.27) with (4.30) we conclude that the Jacobian matrix \({\mathcal J}^{i*}\) has rank 2.
Otherwise, if \((\kappa _2,\kappa _3)= (0,0)\), recalling that \(\Lambda _{11}=\Lambda _{22}\), by (4.30) we obtain
By (4.3) it follows
Since \((R_1,R_2)\ne (0,0) \), we have
for some constant C. Two cases are considered:
-
If \(C\ne 0\) then (4.32) and (4.33) imply
$$\begin{aligned} {R_1\over R_2} = {\psi _1^*\over \psi _2^* } = {\Lambda ^*_{12}\over \Lambda ^*_{11}} = {\alpha _1\over \alpha _2} > 0. \end{aligned}$$(4.34)To achieve a contradiction, let \({\mathcal J}_k^{i*}\) be the k-th row of the Jacobian matrix \({\mathcal J}^{i*}\), as in (4.26). Using the relations
$$\begin{aligned} \Lambda ^*_{11}\psi _1^* = \Lambda _{12}^*\psi _2^*, \quad \Lambda ^*_{11}\psi ^*_2 =\Lambda ^*_{21}\psi ^*_1, \end{aligned}$$one obtains
$$\begin{aligned} \Lambda _{11}^*{\mathcal J}^{i*}_2+\Lambda ^*_{12}{\mathcal J}^{i*}_3= & {} \bigl (*,\psi ^*_1b_2^2\Lambda ^*_{21}, \psi _1^*b_1^2\Lambda ^*_{12}\bigr ), \\\\{\mathcal J}_3^{i*}= & {} \bigl (*, -2b_1^2\Lambda ^*_{11}+b_2^2\Lambda ^*_{21}, -2b_2^2\Lambda ^*_{11}+b_1^2\Lambda ^*_{12}\bigr ). \end{aligned}$$If the Jacobian matrix \({\mathcal J}^{i*}\) has rank \(\le 1\), then the above two vectors must be parallel. Hence
$$\begin{aligned} b_2^4\Lambda ^*_{21}\Lambda ^*_{11} = b_1^4\Lambda ^*_{12}\Lambda ^*_{11}. \end{aligned}$$This yields
$$\begin{aligned} {\Lambda ^*_{12}\over \Lambda _{11}^*} = {\xi _1^*(\bar{x}_i)\over \xi _2^*(\bar{x}_i)}~=~{b_2^2\over b_1^1}. \end{aligned}$$Recalling (4.34), we obtain
$$\begin{aligned} {R_ 1\over R_2} = {\alpha _1\over \alpha _2} = {\beta _1\over \beta _2}={b_2^2\over b_1^2}. \end{aligned}$$Thus, from the first two equations of (3.7), we deduce
$$\begin{aligned} {S_1\over S_2} = {b_1^2\alpha _1^2 + 2b_2^2\alpha _1\alpha _2 +(\gamma -2a_0)\alpha _1\over b_2^2\alpha _2^2 + 2b_1^2\alpha _1\alpha _2 +(\gamma -2a_0) \alpha _2} = {b_2^2\over b_2^1}, \end{aligned}$$contradicting the assumption (4.23).
-
If \(C=0\) then (4.33) implies the identities
$$\begin{aligned} \psi ^*_1 = \psi ^*_2 = 0,\quad \Lambda ^*_{11}\alpha _1 = -\Lambda ^*_{12}\alpha _2,\quad \Lambda ^*_{21}\alpha _1 = -\Lambda ^*_{11}\alpha _2. \end{aligned}$$Therefore
$$\begin{aligned} \begin{pmatrix} \Lambda ^*_{11} &{}&{} \Lambda ^*_{12}\\ \Lambda ^*_{21} &{}&{} \Lambda ^*_{22}\end{pmatrix} \begin{pmatrix}\alpha _1\\ -\alpha _2\end{pmatrix} = 2\Lambda ^*_{11}\begin{pmatrix}\alpha _1\\ -\alpha _2\end{pmatrix}. \end{aligned}$$Recalling (4.32), we obtain
$$\begin{aligned} R_ 1\alpha _2 = -R_ 2\alpha _1. \end{aligned}$$Since \(\alpha _1,\alpha _2>0\), this implies \(R_ 1\not = 0\), \(R_ 2\ne 0\). Recalling (4.30) and (4.17), we thus have \(\Lambda ^*_{11}\ne 0\). On the other hand, using the identities \(\psi ^*_1=\psi ^*_2=0\), a direct computation yields
$$\begin{aligned} {\mathcal J}^{i*} = \begin{pmatrix}*&{}&{} (\gamma -a_0)\Lambda ^*_{11} &{}&{} -(\gamma -a_0)\Lambda ^*_{12} \\ \\ * &{}&{} *&{}&{} *\\ \\ *&{}&{} -2b_1^2\Lambda ^*_{11}+b_2^2\Lambda ^*_{21} &{}&{} -2b_2^2\Lambda ^*_{11} + b_1^2\Lambda ^*_{12} \end{pmatrix}. \end{aligned}$$We claim that this matrix \({\mathcal J}\) at \(P_i^*\) has rank \(\ge 2\). If not, then
$$\begin{aligned} \Lambda ^*_{11}\cdot (-2b_2^2\Lambda ^*_{11} + b_1^2\Lambda ^*_{12})+\Lambda ^*_{12}\cdot (-2b_1^2 \Lambda ^*_{11}+b_2^2\Lambda ^*_{21}) = 0. \end{aligned}$$This implies
$$\begin{aligned} b_2^2\Lambda ^*_{11}+b_1^2\Lambda ^*_{12} = 0,\quad b_1^2\Lambda ^*_{21}+b_2^2\Lambda ^*_{11} = 0. \end{aligned}$$(4.35)Recalling (4.30) we obtain
$$\begin{aligned} -{\alpha _1\over \alpha _2}={R_ 1\over R_ 2} = {\Lambda ^*_{12}\over \Lambda ^*_{11}} = -{b_2^2\over b^2_1}. \end{aligned}$$(4.36)Hence
$$\begin{aligned} b_1^2\alpha _1 = b_2^2\alpha _2. \end{aligned}$$(4.37)Moreover, (4.35) also implies
$$\begin{aligned} b_1^2\cdot \xi _1^*(\bar{x}_i) = b_2^2\cdot \xi _2^*(\bar{x}_i), \end{aligned}$$which yields
$$\begin{aligned} b_1^2\beta _1 = b_2^2\beta _2. \end{aligned}$$(4.38)Inserting (4.37) and (4.38) in the last two equations of the system (3.7), we obtain \(b_1^2R_ 1=b_2^2R_ 2\), reaching a contradiction with (4.36).
In all cases, we conclude that the Jacobian matrix \({\mathcal J}\) has rank \(\ge 2\) at the points \(P_1^*\) and \(P_2^*\). Hence the curves \(\gamma _1,\gamma _2\) are well defined. \(\square \)
The same conclusion remains true also in the presence of sufficiently small nonlinear perturbations. Indeed, a straightforward application of the implicit function theorem yields:
Lemma 2
Under the same assumptions as in Lemma 1, there exists \(\delta _0>0\) sufficiently small such that the following holds. Let \(f_0, \eta _1,\eta _2,h_1,h_2\), be perturbations such that
Then there exist equilibrium points \(\overline{P}_1,\overline{P}_2\) for the ODE (4.21), with \(\Lambda ,\psi \) given at (4.4)–(4.5), such that
Moreover, there exist two surfaces \(\Sigma _1,\Sigma _2\subset I\!\!R^3\), containing the above two points, where the determinant of \(\Lambda (x, \xi _1,\xi _2)\) vanishes. Equation (4.24) defines two curves \(\gamma _i{:}[-s_0,s_0]\rightarrow \Sigma _i\), parameterized by arc length, such that \(\gamma _i(0)=\overline{P}_i\) and
for some constant \(C>0\).
We observe that
-
Since the vector field \(\mathbf{v}\wedge \mathbf{w}\) vanishes on the curve \(\gamma _i\) and \(\overline{P}_i\in \gamma _i\), the curve \(\gamma _i\) describes the unique local center manifold for \(\overline{P}_i\).
-
At every point \(Q\in \Sigma _i{\setminus }\gamma _i\), the vector field \(\mathbf{v}\wedge \mathbf{w}\) is vertical (i.e., its first component is zero, while the last two components do not both vanish). Hence, a trajectory through Q cannot be the graph of a \({\mathcal C}^1\) solution of (4.3).
To construct a global, continuously differentiable solution to (4.3), we need to concatenate orbits of (4.21) which cross the surfaces \(\Sigma _i\) through some point along the curves \(\gamma _i\), \(i=1,2\). This can be achieved by a trajectory which lies in the intersection of two invariant manifolds \({\mathcal M}_1,{\mathcal M}_2\), as shown in Fig. 3.
4.3 Constructing a Solution as a Concatenation of Orbits.
Let us first consider the unperturbed, linear-quadratic case. As before, for \(i=1,2\), we denote by \({\mathcal J}^{i*}\) the Jacobian matrix \({\mathcal J}\) at the point \(P_i^*= \bigl (\bar{x}_i, \xi _1^*(\bar{x}_i), \xi _2^*(\bar{x}_i) \bigr )\). Two of the eigenvalues of \({\mathcal J}^{i*}\) are known, namely
On the other hand, at an arbitrary point \((x,\xi _1,\xi _2)\), by (3.4) the Jacobian matrix in (4.26) has the form
At the point \(P_i^*\), the trace of the matrix \({\mathcal J}^{i*} = {\mathcal J}(P_i^*)\) is
The third eigenvalue of \({\mathcal J}^{i*}\) is thus computed as
To proceed, we now impose generic conditions implying that, at \(P_i^*\), the three eigenvalues of the Jacobian matrix \({\mathcal J}\) are distinct. By (4.41) and (4.42), this will be the case if
Let \(x=\hat{x}_i\) be the solution to the equation
The first condition in (4.43) is equivalent to
We observe that, since \(\bar{x}_i\) and \(\hat{x}_i\) depend on the coefficients of the dynamics and the cost functions (3.2)–(3.3), the inequality (4.43) will be satisfied for generic values of these coefficients.
In order to construct a solution \(\xi (\cdot )\) as a concatenation of orbits of (4.21), one needs to study the flow in a neighborhood of the equilibrium points \(P_1^*\) and \(P_2^*\). We recall that, since all points on the curve \(\gamma _i\), \(i=1,2\), are steady states, the \(3\times 3\) Jacobian matrix \({\mathcal J}\) in (4.26) always has a zero eigenvalue at \(P_i^*\). Different cases arise, depending on the sign of the two remaining eigenvalues \(\lambda _{i,2}^{*}, \lambda _{i,3}^{*}\). In turn, these signs depend on the relative position of \(\hat{x}_i\) with respect to \(\bar{x}_i\).
Case 1 If \(C_{\Lambda }>0\), then (4.41) implies
Recalling that
with \((a_0-b_1^2 \alpha _1 - b_2^2\alpha _2)<0\), from (4.44) we obtain \(\hat{x}_1>\hat{x}_2\). Moreover, for \(i=1,2\),
Three subcases can occur:
-
If \(\bar{x}_1<\hat{x}_1\) and \(\bar{x}_2>\hat{x}_2\), then
$$\begin{aligned} \lambda ^{*}_{1,2}<0< \lambda ^{*}_{1,3}\quad \mathrm {and}\quad \lambda ^{*}_{2,3}<0< \lambda ^{*}_{2,2}; \end{aligned}$$hence both \(P_1^*\) and \(P_2^*\) are saddle points.
-
If \(\bar{x}_1>\hat{x}_1\), then \(\bar{x}_2>\hat{x}_2\) and
$$\begin{aligned} \lambda ^{*}_{1,2}, \lambda ^{*}_{1,3}< 0\quad \mathrm {and}\quad \lambda ^{*}_{2,3}<0< \lambda ^{*}_{2,2}. \end{aligned}$$In this case \(P^*_1\) is a stable point, while \(P^*_2\) is a saddle point.
-
If \(\bar{x}_2<\hat{x}_2\) then \(\bar{x}_1<\hat{x}_1\) and
$$\begin{aligned} \lambda ^{*}_{1,2}<0< \lambda ^{*}_{1,3}\quad \mathrm {and}\quad \lambda ^{*}_{2,2}, \lambda ^{*}_{2,3} > 0. \end{aligned}$$In this case \(P^*_1\) is a saddle point and \(P^*_2\) is an unstable source.
Case 2 If \(C_{\Lambda }<0\) then (4.41) implies
Recalling (4.45), with \((a_0-b_1^2 \alpha _1 - b_2^2\alpha _2)<0\), from (4.44) we obtain \(\hat{x}_1<\hat{x}_2\), together with (4.46).
Four subcases can now occur:
-
If \([\bar{x}_1,\bar{x}_2]\subset [\hat{x}_1,\hat{x}_2]\), then
$$\begin{aligned} \lambda ^{*}_{1,3}<0< \lambda ^{*}_{1,2}\quad \mathrm {and}\quad \lambda ^{*}_{2,2}<0< \lambda ^{*}_{2,3}, \end{aligned}$$In this case, both \(P_1^*\) and \(P_2^*\) are saddle points.
-
If \([\hat{x}_1,\hat{x}_2]\subset [\bar{x}_1,\bar{x}_2]\), then
$$\begin{aligned} \lambda ^{*}_{1,3},\lambda ^{*}_{1,2} > 0\quad \mathrm {and}\quad \lambda ^{*}_{2,3},\lambda ^{*}_{2,2} < 0. \end{aligned}$$In this case, \(P^*_1\) is an unstable source, while \(P^*_2\) is stable.
-
If \(\hat{x}_1<\bar{x}_1\le \hat{x}_2<\bar{x}_2\), then
$$\begin{aligned} \lambda ^{*}_{1,3}<0< \lambda ^{*}_{1,2}\quad \mathrm {and}\quad \lambda ^{*}_{2,3}, \lambda ^{*}_{2,2} < 0. \end{aligned}$$Hence \(P^*_1\) is a saddle point, while \(P^*_2\) is stable.
-
If \(\bar{x}_1< \hat{x}_1\le \bar{x}_2<\hat{x}_2\), then
$$\begin{aligned} \lambda ^{*}_{1,3}, \lambda ^{*}_{1,2} > 0\quad \mathrm {and}\quad \lambda ^{*}_{2,2}<0< \lambda ^{*}_{2,3}. \end{aligned}$$Hence \(P^*_1\) is an unstable source, while \(P^*_2\) is a saddle point.
Consider again the affine solution \(\xi ^*=(\xi _1^*,\xi _2^*)\) of the system (4.3) in the linear-quadratic case, constructed in (3.6). Depending on the signs of the eigenvalues \(\lambda _{i,2}^{*}, \lambda _{i,3}^{*}\), by looking at the dynamics generated by (4.21) we see that the following cases can occur, under generic assumptions.
-
(I)
Assume that \(\lambda _{1,2}^{*}\cdot \lambda _{1,3}^{*}<0\) and \(\lambda _{2,2}^{*}\cdot \lambda _{2,3}^{*}<0\), so that both \(P^*_1\) and \(P^*_2\) are saddle points. Then in the linear-quadratic case \(\xi ^*\) is the unique globally smooth solution to the system (4.3). If a small nonlinear perturbation is present, so that \(\Lambda , \psi \) are given by (4.4)–(4.5), then (4.3) has a unique globally smooth solution close to \(\xi ^*\) (see Fig. 3).
-
(II)
Assume that \(\lambda _{1,2}^{*}<0<\lambda _{1,3}^{*}\) and \(\lambda _{2,2}^{*},\,\lambda _{2,3}^{*}<0\), so that \(P_1^*\) is a saddle and \(P_2^*\) is a sink. Then in the linear-quadratic case system (4.3) has a 1-parameter family of globally smooth solutions close to \(\xi ^*\). The same holds in the presence of a small nonlinear perturbation.
-
(III)
Assume that \(\lambda _{1,2}^{*}, \lambda _{1,3}^{*}>0\) and \(\lambda _{2,2}^{*},\lambda _{2,3}^{*}<0\), so that \(P^*_1\) is a source and \(P^*_2\) is a sink. Then in the linear-quadratic case system (4.3) has a 2-parameter family of globally smooth solutions close to \(\xi ^*\). The same holds in the presence of a small nonlinear perturbation.
To conclude this section we give three examples corresponding to three cases I, II and III.
Example 2
(Two saddle points). Consider the linear system (4.14), but with cost functions
In this case, (4.3) admits the affine solution
This yields equilibrium point \(x^*=-{2\over 7}\), which is asymptotically stable for the dynamics
The determinant of the corresponding matrix \(\Lambda ^*\) is computed by
It vanishes at the two points \(\bar{x}_1=-{1\over 2}\), \(\bar{x}_2=-{1\over 4}\). We thus have \(x^*\in \,]\bar{x}_1,\bar{x}_2[\,\). Moreover, the equilibrium points
On the other hand, the constant in (4.16) is \(C_{\Lambda }=24>0\). Hence, (4.41) implies that \(\lambda _{1,2}^{*}=-6\) and \(\lambda _{2,2}^{*}=6\). By (4.42), the remaining eigenvalues are then computed as \(\lambda _{1,3}^{*}=9\) and \(\lambda _{2,3}^{*}=-{13\over 2}\). Therefore, both \(P_1^*\) and \(P_2^*\) are saddle points.
Example 3
(One saddle point). Consider the linear system (4.14), but with cost functions
In this case, (4.3) admits the affine solution
This yields equilibrium point \(x^*=-2\), which is asymptotically stable for the dynamics
The determinant of the corresponding matrix \(\Lambda ^*\) is computed by
It vanishes at the two points \(\bar{x}_1=-1\), \( \bar{x}_2=1\). Notice that now \(x^*\notin [\bar{x}_1,\bar{x}_2]\). At the equilibrium points \(P_1^*=(-1,-1,-1)\), \(P_2^*=(1,3,3)\), the second eigenvalues of the Jacobian matrix \({\mathcal J}\) are found to be \(\lambda _{1,2}^{*}=6\) and \( \lambda _{2,2}^{*}=-6\). Moreover, recalling (4.42), the third eigenvalues are computed as \(\lambda _{1,3}^{*}=-14\) and \(\lambda _{2,3}^{*}=-18\). We thus conclude that \(P_1^*\) is a saddle and \(P_2^*\) is a sink.
Example 4
(No saddle points). Consider the linear system (4.14), but with cost functions
In this case, we find that (4.3) admits the affine solution
This yields equilibrium point \(x^*=0\), which is asymptotically stable for the dynamics
The determinant of the corresponding matrix \(\Lambda ^*\) is computed by
It vanishes at the two points \( \bar{x}_1=-1 /\sqrt{3}\), \(\bar{x}_2=1 /\sqrt{3}\). Notice that \(x^*\in [\bar{x}_1,\bar{x}_2]\). Moreover,
The eigenvalues of \({\mathcal J}^*\) at \(P_1^*\) and at \(P_2^*\) are found to be
Therefore, \(P_1^*\) is a source and \(P_2^*\) is a sink.
5 The Case with Unique Solutions
In this section we study in greater detail the case where both \(P_1^*\) and \(P_2^*\) are saddle points, for the auxiliary dynamical system (4.21). We consider first the linear-quadratic case. To fix the ideas, let \(\lambda _{1,1}^{*} =\lambda _{2,1}^{*}=0\) and assume that
Notice that this will be the case if the constant \(C_{\Lambda }\) in (4.16) is positive and the equilibrium point \(x^*\) is asymptotically stable and lies in the interior of the interval \([\bar{x}_1,\bar{x}_2]\). Indeed, in this case
Moreover, the matrix \(\Lambda ^*\) in (4.7) satisfies
Therefore
By continuity, we can then choose \(\bar{s} \in \,]0, s_0]\) sufficiently small such that for every \(s\in [-\bar{s} ,\bar{s} ]\) the Jacobian matrix \({\mathcal J}\) at the equilibrium point \(\gamma _i(s)\) defined at (4.25) has three real distinct eigenvalues \(\lambda _{i,j}(s)\), \(j=1,2,3\), which satisfy \(\lambda _{i,j}(0)= \lambda ^{*}_{i,j}\), together with
Let \({\mathcal M}_1\) be the center-stable manifold for the equilibrium point \(P_1^*\). In other words, \({\mathcal M}_1\) is an invariant 2-dimensional manifold, whose tangent space at \(P_1^*\) is spanned by the eigenvectors
corresponding to the eigenvalues \(\lambda _{1,1}^{*}=0\) and \(\lambda _{2,2}^{*}<0\), respectively.
We observe that, for general systems, a center manifold is not uniquely defined [3, 10, 18]. However, in the present setting it is clear that the local center manifold through \(P_1^*\) is unique, because it must coincide with the curve of steady states \(\gamma _1\). Indeed, the 2-dimensional manifold \({\mathcal M}_1\) can be uniquely determined as the union of the 1-dimensional stable manifolds of all equilibrium points \(\gamma _1(s)\).
Similarly, let \({\mathcal M}_2\) be the center-unstable manifold at \(P_2^*\). Notice that \({\mathcal M}_2\) is uniquely determined as the union of the 1-dimensional unstable manifolds of all equilibrium points \(\gamma _2(s)\). Its tangent space at \(P_2^*\) is spanned by the two eigenvectors
The segment joining \(P_2^*\) with \(P_1^*\) (with endpoints removed) is a heteroclinic orbit of system (4.21), contained in the intersection \({\mathcal M}_1\cap {\mathcal M}_2\). To proceed, we now make the key assumption that this intersection is transversal:
-
(A2)
At any point P along the segment joining \(P_1^*\) with \(P_2^*\), the manifolds \({\mathcal M}_1\)and \({\mathcal M}_2\) intersect transversally. Namely, the union of the tangent spaces to \({\mathcal M}_1\) and \({\mathcal M}_2\) spans the whole space \(I\!\!R^3\).
Clearly, if the intersection is transversal at one such point P, then it is transversal at every other point.
For reader’s convenience, we collect all the assumptions that will be used in our main theorem.
-
(i)
The assumption (A1) holds, providing the existence of the affine solution \((\xi ^*_1,\xi _2^*)\) in (3.6).
-
(ii)
The determinant of the matrix \(\Lambda (x,\xi _1,\xi _2)\) at (3.4) vanishes at the two points \(P_i^*=(\bar{x}_i, \xi _1^*(\bar{x}_i), \xi _2^*(\bar{x}_i))\), \(i=1,2\).
-
(iii)
At least one of the equalities in (4.23) does NOT hold. By Lemma 1, the curves of steady states \(\gamma _1,\gamma _2\) are thus well defined.
-
(iv)
At the points \(P_1^*\) and \(P_2^*\), the eigenvalues of the Jacobian matrix \({\mathcal J}\) in (4.26) satisfy the sign conditions (5.1).
-
(v)
As in (A2), the manifolds \({\mathcal M}_1,{\mathcal M}_2\) intersect transversally.
We are now ready to state our main uniqueness and stability theorem, for feedback solutions to the noncooperative differential game.
Theorem 2
Let the above assumptions (i)–(v) hold, and let \(\Omega \subset I\!\!R\) be any bounded interval containing \(\bar{x}_1, \bar{x}_2\) in its interior. Then, for any \(\varepsilon >0\) there exists \(\delta >0\) such that the following holds. If the perturbations \(f_0, h_1,h_2,\eta _1,\eta _2\) are small enough, i.e., if they satisfy (4.39), then system (4.3)–(4.5) has a unique \({\mathcal C}^1\) solution \((\xi _1,\xi _2)\) defined on the whole interval \(\Omega \), such that
Remark 5
In the above theorem, the interval \(\Omega \) can be taken large enough so that it contains the stable equilibrium point \(x^*\) in (3.22). Choosing \(\delta >0\) sufficiently small, the set \(\Omega \) will then be positively invariant for the corresponding feedback dynamics (4.13).
Proof of Theorem 2
Consider the vectors \(\mathbf{v},\mathbf{w}\) introduced at (4.20). To prove the result, it suffices to check that all steps in the construction of the heteroclinic orbit connecting \(P_1^*\) with \(P_2^*\) remain valid also in the presence of a small \({\mathcal C}^1\) perturbation of the vector fields \(\mathbf{v},\mathbf{w}\).
-
1.
Thanks to the assumption (iii), by the implicit function theorem the two curves \(\gamma _1,\gamma _2\) where \(\mathbf{v}\wedge \mathbf{w}=0\) (i.e., where \(\mathbf{v}\) and \(\mathbf{w}\) are parallel) are still well defined also in the presence of a small \({\mathcal C}^1\) perturbation. By continuity, at every point along these curves, the eigenvalues of the Jacobian matrix \({\mathcal J}\) in (4.26) still satisfy the strict inequalities (5.2).
-
2.
Given the dynamical system
$$\begin{aligned} \dot{x} = \mathbf{v}(x)\wedge \mathbf{w}(x), \end{aligned}$$(5.4)let \({\mathcal M}_1\) be the 2-dimensional manifold obtained as the union of all the 1-dimensional stable manifolds of the points along \(\gamma _1\). Similarly, let \({\mathcal M}_2\) be the 2-dimensional manifold obtained as the union of all the 1-dimensional unstable manifolds of the points along \(\gamma _2\). By the regularity theory of invariant manifolds (see Chapter 4 in [10]), the tangent space to these manifolds varies continuously with the vector field \(\mathbf{v}\wedge \mathbf{w}\), w.r.t. the \({\mathcal C}^1\) norm. In particular, if a small \({\mathcal C}^1\) perturbation is added to \(\mathbf{v}\) and \(\mathbf{w}\), then the intersection \({\mathcal M}_1\cap {\mathcal M}_2\) is still transversal. This 1-dimensional intersection provides the unique heteroclinic orbit connecting a point \(P_1\in \gamma _1\) with a point \(P_2\in \gamma _2\).
-
3.
The graph of the desired solution \(x\mapsto (\xi _1,\xi _2)(x)\) of (4.3)–(4.5) is now obtained concatenating the 1-dimensional stable manifold for \(P_1\) with the 1-dimensional unstable manifold for \(P_2\). \(\square \)
We conclude this section by providing an example where the crucial transversality condition (A2) can be directly checked.
Example 5
(continued). Consider again the linear-quadratic system with dynamics (4.14) and cost functionals (4.47). In this case, we have
and
At the point \(P(x)=(x, \xi _1^*(x), \xi _2^*(x))\), the Jacobian matrix takes the form
The eigenvalues are
with corresponding eigenvectors
Notice that \(\mathbf{v}_1\) and \(\mathbf{v}_2\) are independent of x. As in Fig. 3, let \({\mathcal M}_1\) be the center-stable manifold for the point \(P_1^*\), and let \({\mathcal M}_2\) be the center-unstable manifold for the point \(P_2^*\). Then the line \(\{P(x)=(x, \xi ^*_1(x), \xi ^*_2(x)); x\in I\!\!R\}\) lies in the intersection \({\mathcal M}_1\cap {\mathcal M}_2\). To check that this intersection is transversal we observe that, at every point \(P(x)=(x,\xi _1^*(x), \xi _2^*(x))\), the tangent space to the manifold \({\mathcal M}_1\) is spanned by the vectors \(\mathbf{v}_1\) and \(\mathbf{v}_2\), independent of x. On the other hand, at \(P(\bar{x}_2)\), the tangent space to \({\mathcal M}_2\) is spanned by \(\mathbf{v}_2\) and \(\mathbf{v}_3(\bar{x}_2)\). Hence \({\mathcal M}_1\) and \({\mathcal M}_2\) have transversal intersection.
6 Perturbed Solutions on the Entire Real Line
In the previous sections we studied the existence of solutions to the perturbed system (4.3) on a bounded interval \(\Omega \subset I\!\!R\), possibly containing the two points where det \(\,\Lambda ^*(x)\) vanishes. We now examine whether these perturbed solutions can be extended to the whole real line. The heart of the matter is illustrated in Fig. 1. In the linear-quadratic case, the determinant of the matrix \(\Lambda (x,\xi _1,\xi _2)\) in (3.4) is negative inside the double cone
To fix the ideas, assume that det \(\Lambda ^*(x)\not = 0\) for all \(x\in [x_0,\,+\infty [\,\). Notice that trajectories of the implicit ODE (4.3) can become singular near the boundary of \(\Gamma ^-\). In this case, they cannot be prolonged any further (Fig. 1, left). To make sure that a small perturbation of the trajectory \(\xi ^*(\cdot )\) is well defined for all \(x\in [x_0, +\infty [\,\) we must check that it never approaches the surface where \(\det \Lambda (x,\xi _1,\xi _2)=0\).
Recalling (3.8), set \(\bar{a}_0\doteq {a_0\over A}\). By (3.6) and (4.7) all entries of the matrix \(\Lambda ^*(x)\) are polynomials of degree one. Recalling (3.20), the matrix of leading coefficients is computed as
Throughout the following, we assume that \(\Lambda _2^*\) is invertible, i.e.,
The inverse matrix of \(\Lambda ^*_2\) is
Let \(\lambda _1, \lambda _2\) be the eigenvalues of \((\Lambda ^*_2)^{-1}\) and denote by \(r_1,r_2 \) the corresponding normalized eigenvectors, so that \(|r_1|=|r_2|=1\). We have
We introduce the constant
Lemma 3
Under the assumption (A1), if
then there exists \(x_0>0\) such that the affine solution (3.6) satisfies
for all \(x\in ]-\infty ,-x_0]\cup [x_0,\infty [\,\). Moreover, one has
Proof
For |x| large, the determinant of \(\Lambda ^*(x)\) has the same sign as the determinant of the matrix of leading-order coefficients \(\Lambda _2^*\). Hence the first assertion of the lemma is clear.
To prove (6.7), we consider two cases.
-
If \(a_0<0\), then
$$\begin{aligned} A< 0,\quad \bar{a}_0>0\quad \mathrm {and}\quad X_1^*+X_2^* < 0. \end{aligned}$$Recalling (6.6) and (6.4), we obtain
$$\begin{aligned} \lambda _1\cdot \lambda _2 > 0\quad \mathrm {and}\quad \lambda _1+\lambda _2 < 0. \end{aligned}$$(6.8) -
If \(0<{\gamma \over 2}<a_0\) and \(K_1+K_2>1/2\), then
$$\begin{aligned} A> 0,\quad \bar{a}_0>0,\quad X_1^*+X_2^*> 1> \bar{a}_0\quad \mathrm {and}\quad X^*_i > 0. \end{aligned}$$
In both cases, (6.8) implies (6.7). \(\square \)
Lemma 4
Under the assumption (A1), if
then there exists \(x_0>0\) such that the affine solution (3.6) satisfies
for all \(x\in \,]-\infty ,-x_0]\cup [x_0,\infty [\). Moreover, the eigenvalues \(\lambda _1\) and \(\lambda _2\) of \((\Lambda _2^*)^{-1}\) are real, with opposite signs. In particular,
In the case \(a_0>0\) there exists a constant \(0<\gamma ^*_1<2a_0\), depending on \(S_1,S_2,\alpha _1,\alpha _2\), such that
Proof
As in Lemma 3, the first statement is clear, because for |x| large det\(\Lambda ^*(x)\) and det\(\Lambda _2^*\) have the same sign.
Hence the eigenvalues are real, and (6.10) holds.
To prove the last statement, we observe that
This yields (6.11), for some \(\gamma _1^*\in \,]0, \, 2a_0[\,\). \(\square \)
The following theorem provides the existence of solutions to the perturbed ODE (4.3) on an unbounded interval \([x_0, +\infty [\). An entirely similar result holds on a domain of the form \(]-\infty ,\,-x_0]\). We recall that \(\lambda _{max}\) is the constant introduced at (6.5).
Theorem 3
Let the assumption (A1) holds together with (6.3).
If
then there exist \(\delta _0, \delta _1>0\) sufficiently small and \(x_0>0\) such that the following holds. For any perturbations such that
and any initial data
the implicit ODE in (4.3)–(4.5) admits a unique solution \((\xi _1,\xi _2)\) which is defined for all \(x\in [x_0, \, +\infty [\).
Remark 6
There are two main cases where Theorem 3 applies.
-
(i)
If (6.6) holds, then Lemma 3 implies that \(\gamma \lambda _{max}<0<2\). In this case, for |x| sufficiently large, we have det\(\Lambda ^*(x)>0\) , and the solution \(\xi ^*\) lies outside the cone \(\Gamma ^-\). This is illustrated in Fig. 1, left and center.
-
(ii)
If (6.9) holds, then Lemma 4 yields \(\gamma \lambda _{max}<2\), provided that \(\gamma <2a_0\). In this case, for |x| sufficiently large, we have det\(\Lambda ^*(x)<0\) , and the solution \(\xi ^*\) lies inside the cone \(\Gamma ^-\). This is illustrated in Fig. 1, right.
Proof of Theorem 3
By a rescaling of coordinates, without loss of generality we can assume that \(x_0=1\). The proof will be given in several steps.
1. We first consider the unperturbed case, where \(f_0= g_1=g_2=h_1=h_2=0\).
From (4.3) it follows
Notice that \(\xi '\) is determined by the implicit ODE (4.3), while the derivative \((\xi ^*)' = \begin{pmatrix}\alpha _1\\ \alpha _2\end{pmatrix}\) is constant, according to (3.6). Introducing the variable \(\zeta =\xi -\xi ^*\), by a direct computation we obtain
This can be written as
where \(\Lambda ^*_2\) is the matrix of leading-order terms (6.2) and I is the \(2\times 2\) identity matrix. We thus have
To bring out the leading-order terms in the right-hand side of (6.15), we estimate
and
where the Landau symbol \(\mathcal O(1)\) denotes a uniformly bounded matrix-valued function.
Observing that \(|(\Lambda ^*(x))^{-1}|\le C/x\), for some constant C and all x sufficiently large, we can thus write (6.15) as
Here the remainder term satisfies the bound
as long as
for some large constant \(C_2>0\) and for \(\delta _2>0\) sufficiently small, depending on all coefficients \(a_i, b_i\).
2. Given any solution \(\xi (\cdot )\) to the implicit ODE (2.7), we introduce the auxiliary variables
Here \(x\in [1,\,+\infty [\,\), \(t\in [-1,0[\,\), while \(\zeta = (\zeta _1,\zeta _2)(x)\) is a solution to (6.15). Denoting the derivative w.r.t. t by an upper dot, we obtain
To bring out the leading-order terms in the above equation, we use (6.17) and obtain
where
By (6.18), the matrix K(t, z) remains uniformly bounded, as long as \(|z|= |\zeta |/|x|\) remains sufficiently small. Notice that \(\gamma (\Lambda ^*_2)^{-1}- 2I\) is a matrix with eigenvalues \(\gamma \lambda _i-2\), \(i=1,2\). By the assumption (6.12) the real parts of these eigenvalues satisfy
for some constant \(\lambda >0\). By a classical result on the stability of linear systems (see for example Theorem 2.61 in [10]), there exists an equivalent norm \(\Vert \cdot \Vert \) on \(I\!\!R^2\) such that every solution to the homogeneous equation
satisfies
Let \(C_3\) be an upper bound on the corresponding operator norm of the matrix K, as long as \(\Vert z\Vert \le \delta _3\), for some constant \(\delta _3>0\). Then the solution of (6.21) satisfies
as long as \(\Vert z(t)\Vert \le \delta _3\).
We now choose \(t_0<0\) such that
For \(t\in [t_0,0[\), (6.25) implies
hence
If now \(\Vert z(-1)\Vert \) is sufficiently small, then \(\Vert z(t_0) \Vert \le \delta _3\). In turn, (6.26) implies that z(t) is well defined for all \(t\in [t_0,0[\) and \(z(t)\rightarrow 0\) as \(t\rightarrow 0-\,\). Returning to the original variables \(x,\xi \), we conclude that if \(\zeta (1) = \xi (1)-\xi ^*(1)\) is sufficiently small, then the solution \(x\mapsto \xi (x)\) of (4.3) is well defined for all \(x\in [1,\,+\infty [\,\). This proves the theorem in the unperturbed case, where \(f_0=h_1=h_2=\eta _1=\eta _2=0\) (Fig. 4).
3. In the remainder of the proof, we check that the same conclusion holds for the perturbed problem, where the dynamics is determined by (4.3)–(4.5). A minor modification of the argument in step 2 yields \(\square \)
For \(t\in [t_0, 0[\,\), the domain \(\Vert z\Vert \le \delta _3\) is positively invariant for the ODE (6.21)
Lemma 5
Assume that both eigenvalues of the matrix \(\gamma (\Lambda ^*_2)^{-1}-2I\) have strictly negative real part. Given \(C_3\) and \(\delta _3>0\), there exist sufficiently small constants \(\delta _4, \delta _5\in \,]0, \delta _3]\) such that the following holds. Let \(z{:}[-1,0[\,\mapsto I\!\!R^2\) be a solution of the Cauchy problem
with \(|\bar{z}| \le \delta _5\) and
Then z satisfies
Indeed, let \(\lambda >0\) be as in (6.22). Using the equivalent norm introduced at (6.23)–(6.24), we obtain
for some constant C, depending on \(C_3\) and on the equivalent norm \(\Vert \cdot \Vert \). Choosing \(\delta _4\) and then \(t_0<0\) so that
for all \(t\in [t_0, 0[\,\) we obtain
By possibly taking a smaller constant \(\delta _4\), from (6.30) we deduce that the ball \(\{z\in I\!\!R^2; \Vert z\Vert \le \delta _3\}\) is positively invariant for the ODE in (6.27), for \(t\in [t_0, 0[\,\). Next, by choosing the initial condition \(z(-1)=|\bar{z}|\) sufficiently small, we can achieve the bound
By positive invariance, this implies \(\Vert z(t)\Vert \le \delta _3\) for all \(t\in [-1, 0[\,\).
We have thus proved (6.29), with the Euclidean norm replaced by the equivalent norm \(\Vert \cdot \Vert \). Since the constant \(\delta _3\) was arbitrary, this achieves a proof of Lemma 5.
4. To complete the proof of the theorem it now suffices to check that, if (6.13)–(6.14) hold with \(\delta _0,\delta _1>0\) sufficiently small, then the change of variables (6.19) yields a function z(t) which satisfies the assumptions of Lemma 5. This is achieved by a lengthy but straightforward computation.
Repeating the argument in step 1, one obtains
where
and
As long as that matrix \(\Lambda \bigl (x,\xi _1,\xi _2\bigr )\) is invertible, we can thus write
We now have
for some matrix \(K_2\) whose norm satisfies the bound
provided
for some constant \(C_6>0\) and some \(\delta _6>0\) sufficiently small.
Recalling (6.16), one can write (6.31) in the form
We now have the estimates
In turn, these yield
where the matrix \(K_3\) and the vector \(\mathbf{b}\) satisfy
and
as long as (6.32) holds. Under the assumptions (6.13) and (6.32) we thus have
for some constant \(C_7\).
Performing the change of coordinates (6.19) we obtain
where
By (6.35) one has the bounds
Given the constants \(C_7\) and \(\delta _3>0\), set \(C_3\doteq C_7(1+\delta _3)\) and let \(\delta _4,\delta _5\) be as in Lemma 5. By choosing \(\delta _1>0\) sufficiently small, we achieve
while \(\Vert \psi (t,z)\Vert \le C_7 \delta _1\le \delta _4\). An application of Lemma 5 now yields the existence of a solution \(z:[-1,0[\,\mapsto I\!\!R^2\). Going back to the original variables, we obtain a solution \(x\mapsto \xi (x)\), globally defined for all \(x\in [1,+\infty [\,\).
Remark 7
From the inequality (6.30) it follows
In terms of the original variables, this implies that the perturbed solutions satisfy
7 Equilibrium Solutions to Differential Games
All previous analysis has been concerned with the system of ODEs (1.12) describing the derivatives of value functions for the two players. We conclude the paper by showing that, starting with a smooth solution to these ODEs, one obtains a pair of value functions \(V_1,V_2\) satisfying the H–J equations (2.5). In turn, this yields the existence of a Nash equilibrium solution to the differential game (1.1)–(1.2), in the sense of Definition 1. Since here we are dealing with smooth value functions, all these claims can be proved by standard arguments.
Proposition 1
Let \(f, g_1,g_2{:}I\!\!R\mapsto I\!\!R\) be smooth maps with sublinear growth, as in (2.2), and let \(\varphi _i\) be such that
for some positive constant \(C_1, C_2\). Let \(x\mapsto (\xi _1(x),\xi _2(x))\) provide a solution to the implicit system of ODEs (2.7)–(2.9), and assume that the corresponding feedback dynamics (2.10) has a unique equilibrium point \(x^*\), which is stable. If in addition
for some positive constants \(\alpha ,\beta \), then the feedback controls
provide a Nash equilibrium solution to the differential game with dynamics (2.1) and cost functionals (2.3).
Proof
1. Define the functions \(V_1,V_2\) by setting
By (2.7)–(2.9), a direct computation yields
Since \((\xi _1,\xi _2)\) is a solution to the implicit system of ODEs (2.7)–(2.9),for \(i=1,2\) we have
Therefore \(V_1, V_2\) satisfy the H–J equations in (2.6).
2. Consider the optimal control problem for the first player:
subject to
Here \(u_2^*(\cdot )\) is the feedback control of the second player, defined at (7.3). We claim that \(V_1(\cdot )\) is the value function for this problem.
Indeed, if the first player implements the feedback control \(u=u^*_1(\cdot )\) in (7.3), then for any initial datum \(x(0)=x_0\) the system evolves according to
Hence, for every \(\tau \),
Letting \( \tau \rightarrow \infty \) and recalling that \(\lim _{ \tau \rightarrow \infty }{x}( \tau ) = x^*\), we conclude that the right-hand side converges to \(V_1(x(0))\). Calling \(V_1^*(\cdot ) \) the minimum value function for the problem (7.6)–(7.7), we thus have
To prove the converse inequality we need to show that, for any control \(u(\cdot )\) such that the Cauchy problem (7.7) admits a global solution in time, one has
Indeed, consider the function
For a.e \(\tau \in [0,+\infty )\), we compute
Hence, \(\Phi (\tau )\) is a nondecreasing function. We now estimate
If now
then the inequality (7.9) follows. Assume that, on the contrary, there exist \(\tau _0,\varepsilon >0\) such that
for some constant \(c_1,c_2>0\). Hence, for all \(t\ge \tau _0\),
Recalling (7.1), we now estimate
Letting \(\tau \rightarrow +\infty \), we thus obtain
Hence (7.9) trivially holds.
3. Having proved that \(V_1(\cdot )\) is the value function, it is clear that the feedback control \(u^*_1(\cdot )\) in (7.3) is optimal for player 1. An entirely similar argument shows that \(u_2^*(\cdot )\) is an optimal feedback control for player 2. \(\square \)
A similar result holds in the case where the solution \((\xi _1,\xi _2)\) of (2.7)–(2.9) is constructed only on a bounded interval.
Proposition 2
Let \(f, g_1,g_2{:}I\!\!R\mapsto I\!\!R\) be smooth maps with sublinear growth, as in (2.2). Let \(x\mapsto (\xi _1(x),\xi _2(x))\) provide a solution to the implicit system of ODEs (2.7)–(2.9) restricted to some interval [a, b], and assume that the corresponding feedback dynamics (2.10) has a unique equilibrium point \(x^*\in [a,b]\), which is stable. Then the controls \(u_i^*\) in (7.3) are optimal for the game with dynamics (2.1) and cost functionals
where
Indeed, by (7.12) both players face an optimization problem with state constraint \(x\in [a,b]\). The proof is achieved by the same arguments used in the previous case.
Remark 8
As a consequence of Proposition 1, the perturbed solution \((\xi _1,\xi _2)\) constructed in Sect. 6 on the whole real line yields a Nash equilibrium solution to the differential game with dynamics (2.1) and cost functionals (2.3). Indeed, (6.13) implies (7.1) and (2.2). By Remark 7, there exists a constant \(C>0\) such that
Hence the condition (7.2) holds.
On the other hand, by Proposition 2 the perturbed solution \((\xi _1,\xi _2)\) constructed in Sects. 4 and 5 on a bounded interval [a, b] yields a Nash equilibrium solution to the differential game with dynamics (2.1) and cost functionals (7.11)–(7.12).
References
Bardi M, Capuzzo Dolcetta I (1997) Dolcetta, optimal control and viscosity solutions of Hamilton–Jacobi–Bellman equations. Birkhäuser, Boston
Basar T, Olsder GJ (1995) Dynamic noncooperative game theory, 2nd edn. Academic, London
Bressan A (2007) A tutorial on the center manifold theorem. In: Hyperbolic systems of balance laws, 327–344. Lecture Notes in Math. 1911, Springer, Berlin
Bressan A (2009) From optimal control to non-cooperative differential games: a homotopy approach. Control Cybern 38:1081–1106
Bressan A (2011) Noncooperative differential games. Milan J Math 79:357–427
Bressan A, Piccoli B (2007) Introduction to the mathematical theory of control. AIMS Series in Applied Mathematics, Springfield
Bressan A, Priuli F (2006) Infinite horizon noncooperative differential games. J Differ Equ 227:230–257
Bressan A, Shen W (2004) Small BV solutions of hyperbolic non-cooperative differential games. SIAM J Control Optim 43:104–215
Bressan A, Shen W (2004) Semi-cooperative strategies for differential games. Int J Game Theory 32:561–593
Chicone C (2006) Ordinary differential equations with applications, 2nd edn. Springer, New York
Dockner E, Jorgensen S, Van Long N, Sorger G (2000) Differential games in economics and management science. Cambridge University Press, Cambridge
Engwerda JC (2009) Linear quadratic differential games: an overview. In: Advances in dynamic games and their applications, 37–70. Ann Int Soc Dyn Games, 10. Birkhäuser, Boston
Engwerda JC (2000) Feedback Nash equilibria in the scalar infinite horizon LQ-game. Autom J IFAC 36:135–139
Engwerda JC, Salmah Y (2013) Necessary and sufficient conditions for feedback Nash equilibria for the affine-quadratic differential game. J Optim Theory Appl 157:552–563
Fleming W, Rishel R (1975) Deterministic and stochastic optimal control. Springer, Berlin
Lukes D (1971) Equilibrium feedback control in linear games with quadratic costs. SIAM J Control Optim 9:234–252
Priuli F (2007) Infinite horizon noncooperative differential games with nonsmooth costs. J Math Anal Appl 336:156–170
Vanderbauwhede A (1989) Centre manifolds, normal forms and elementary bifurcations. Dyn Rep 2:89–169
Weeren A, Schumacher J, Engwerda J (1999) Asymptotic analysis of linear feedback Nash equilibria in nonzero-sum linear-quadratic differential games. J Optim Theory Appl 101:693–722
Acknowledgments
The authors would like to thank the anonymous referee, whose suggestions and comments helped to improve various aspects of the paper.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Bressan, A., Nguyen, K.T. Stability of Feedback Solutions for Infinite Horizon Noncooperative Differential Games. Dyn Games Appl 8, 42–78 (2018). https://doi.org/10.1007/s13235-016-0206-2
Published:
Issue Date:
DOI: https://doi.org/10.1007/s13235-016-0206-2