Abstract
We study a differential game where two players separately control their own dynamics, pay a running cost, and moreover pay an exit cost (quitting the game) when they leave a fixed domain. In particular, each player has its own domain and the exit cost consists of three different exit costs, depending whether either the first player only leaves its domain, or the second player only leaves its domain, or they both simultaneously leave their own domain. We prove that, under suitable hypotheses, the lower and upper values are continuous and are, respectively, the unique viscosity solution of a suitable Dirichlet problem for a Hamilton–Jacobi–Isaacs equation. The continuity of the values relies on the existence of suitable non-anticipating strategies respecting the domain constraint. This problem is also treated in this work.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
1 Introduction
In some quite recent authors’ works (see, for example, Bagagiolo [1], Bagagiolo–Danieli [3], Bagagiolo–Maggistro [4] and the references therein), some optimal control problems for systems governed by thermostatic dynamics are studied in the framework of dynamic programming methods and viscosity solutions of Hamilton–Jacobi–Bellman (HJB) equations. By system governed by thermostatic dynamics, here we mean an equation as \(y'=f(y,w,\alpha )\) where \(\alpha \) is the measurable control and \(w\in \{-1,1\}\) is the output of a hysteretical thermostat (delayed relay) subject to the evolution of some suitable components of the state variable y. Hence, the switching evolution of the parameter w is not directly at disposal of the external controller, but it follows some internal switching rules which are intrinsic to the system. In [1, 3], the value function is proven to be the unique viscosity solution of a suitably coupled system of HJB equations, where the coupling is given by the boundary conditions in the regions where the thermostat certainly assumes a constant value (cannot switch). This is done by splitting the optimal control problem in some problems of exit time kind: in every space-region where the thermostat is constant, the problem is equivalent to an exit time problem with unknown exit cost given by the value function itself evaluated in the other region of constancy for w. Then, an ad hoc fixed point procedure is applied. Hence, a crucial starting point for such a procedure is a good theory for exit time/exit cost optimal control problems, in particular for what concerns the identification of the value function as the unique viscosity solution of a Dirichlet problem for HJB equations with suitable boundary conditions. Fortunately, such a good theory was quite already at disposal.
In [1, 3, 4] some motivations and applications for studying optimal control problems with thermostatic dynamics are given. Similar motivations certainly suggest the study of differential games with thermostatic dynamics. Just think to a pursuit–evasion game between two cars with automatic gears, where the switching variable(s) w may represent the position of the gears. We also point out that such thermostatic dynamics is a special case of a so-called hybrid dynamics, and we refer to Gromov–Gromova [19], for a recent study of hybrid differential games in the framework of necessary optimality conditions.
In order to apply to differential games some similar procedures as the ones applied to HJB for thermostatic optimal control problems, we need a good theory for exit time differential games. Unfortunately, for differential games, the situation is rather different than from optimal control: differential games problem with exit time and exit costs are not so well studied in the framework of viscosity solutions for Hamilton–Jacobi equations. Hence, before studying differential games for systems with thermostatic dynamics we need first, at least, uniqueness results for some suitable Dirichlet problems for Hamilton–Jacobi–Isaacs (HJI) equations, in order to be able to identify the (upper and lower) values of the exit time differential game as the unique viscosity solutions. Up to the knowledge of the authors, this paper may represent the first attempt of studying an exit time differential games in the framework of viscosity solutions theory for Isaacs equations with boundary conditions in the viscosity sense. The possible applications to differential games with thermostatic dynamics will be the subject of a future work. However, we would like to point out that the differential games with exit time and exit costs are interesting by themselves, and not only for applications to thermostatic dynamics. To this purpose, we are going to give an example at the end of the next section.
The studied problem is the following one. We are given the controlled system
with \(\alpha \) and \(\beta \) controls, X and Y state variables for the two players, respectively (i.e., for example, the time-dependent function \(t\mapsto X(t)\) is the solution (trajectory) of the first system, when the initial point x and the measurable control \(\alpha \) are fixed). The closed sets \({\overline{\varOmega }}_X\) and \({\overline{\varOmega }}_Y\) (closure of the open ones \(\varOmega _X\) and \(\varOmega _Y\)) are, respectively, the state space for X and Y. Denoting by \(\tau _X\) and \(\tau _Y\) the first exit time from \({\overline{\varOmega }}_X\) and \({\overline{\varOmega }}_Y\), respectively, we define \(\tau =\min \{\tau _X,\tau _Y\}\) and consider the following integral discounted cost
where, in particular, the exit cost \(\psi :\partial (\varOmega _X\times \varOmega _Y)\rightarrow [0,+\infty [\) is a given function, which is not required to be continuous but only separately continuous on \(\partial \varOmega _X\times \varOmega _Y\), \(\varOmega _X\times \partial \varOmega _Y\) and \(\partial \varOmega _X\times \partial \varOmega _Y\). Player X wants to minimize the cost, whereas player Y wants to maximize. The problem has then a pursuit–evasion structure.
The “weak” continuity hypothesis on the exit cost \(\psi \) is assumed in order to take account of the possible application to the thermostatic case. Indeed, in that case, \({\overline{\varOmega }}_X\) and \({\overline{\varOmega }}_Y\) represent two regions where the thermostats (one per every player) assume constant values. Hence, the first exit time represents the first switching time for the thermostat, and so exiting from \(\partial \varOmega _X\times \varOmega _Y\) means that only the thermostat of X switches, exiting from \(\varOmega _X\times \partial \varOmega _Y\) means that only the thermostat of Y switches and finally exiting from \(\partial \varOmega _X\times \partial \varOmega _Y\) means that both thermostats switch. In every one of such three cases, the new scenario of the game (which continues to run after switching) may be completely different and not related to each other.
There are three main points which are going to be treated in this paper:
- (a)
continuity of the lower and upper values,
- (b)
derivation of suitable boundary conditions for the corresponding HJI equations,
- (c)
uniqueness results for those Dirichlet problems, in the sense of viscosity solutions.
We refer the reader to Bardi–Capuzzo Dolcetta [5] for a comprehensive account to viscosity solutions theory and applications to optimal control problems and differential games (for differential games see also Buckdahn–Cardaliaguet–Quincampoix [13])
Point (a): Sections 3, 4 and 7.
The lower and upper values are, respectively, defined as
where \(\gamma \) and \(\xi \) are non-anticipating strategies in the sense of Elliot–Kalton [16]. They are functions from the set of controls for one player to the set of controls of the other one, which do not depend on the future behavior of the control, in the sense that, if two controls coincide in the time interval [0, t], then their images also coincide in [0, t] (see Definition 1).
At this level, a subpoint a1): state-constraint non-anticipating strategies, must be treated.
After a necessary suitable compatibility assumptions on the exit cost (see (6) and Remark 4), the main problem concerning the continuity of the values is the existence of suitable “state-constraint non-anticipating strategies.” Simplifying (see Assumption 2 for more precise details), this means that, for example, for every non-anticipating strategy \(\gamma \) for player X, there exists a non-anticipating strategy \({\tilde{\gamma }}\) for X such that, for every control \(\beta \) for Y, the control \({\tilde{\gamma }}[\beta ]\) makes the evolution of X remain inside \({\overline{\varOmega }}_X\) as long as \(\beta \) makes the evolution of Y remaining inside \({\overline{\varOmega }}_Y\), and the cost paid by the couple of controls \(({\tilde{\gamma }}[\beta ],\beta )\) is not so different from the cost paid by the couple \((\gamma [\beta ],\beta )\). The difficult here is to construct such a state-constraint non-anticipating strategy \({\tilde{\gamma }}\). The problem of the construction of a new state-constraint control that makes the state-constraint be respected and that pays a cost not so different from the cost paid by the old control, also occurs in the case of constrained optimal control problems, where the trajectory is constrained to remain inside a fixed set for all the times. In Soner [21], starting from any control, a possible construction of a state-constraint control with such properties is given. The main assumption is the controllability on the boundary of the set (which we also assume). However, in that paper, the construction of the state-constraint control is done in an “anticipating” way, that is at time t the new control is constructed taking also account of the behavior of the old control for suitable times after t. In particular, starting from two controls that coincide in the time interval [0, t], it may happen that the two corresponding state-constraint controls actually differ in [0, t] (see Remark 11). However, in [21], there is no need of non-anticipating properties, because the argument is an optimal control problem and not a differential game, and this is why the author was not concerned with non-anticipating behaviors. In the case of non-anticipating strategies, a similar construction as in [21] is forbidden. In the present paper, using the fact that the dynamics of the two players are decoupled with respect to the space variables and to the controls (see (1)), and also adding a decoupled feature in the controls for the cost (see (48)), we are able to suitably adapt Soner’s construction in order to get the desired state-constraint non-anticipating strategy. Such assumptions on decoupled dynamics, at the present moment, seem almost necessary in order to get this kind of results. In Sect. 7 we actually assume a sort of more general weak decoupling of the dynamics with respect to the controls (see (32)) for at most one of the two players, and we still get the desired state-constraint non-anticipating strategy for the player with weakly coupled dynamics. However, that weak decoupling seems to be not immediately suitable for the results of Sects. 5 and 6. The same problem of constructing that kind of non-anticipating strategy is also studied in Bettiol–Cardaliaguet–Quincampoix [7], where the decoupled dynamics assumption is also used, and other hypotheses on the running cost are made. Other studies on constrained trajectories and non-anticipating strategies as well as on possible relations with optimal control problems and differential games can be found in Koike [20], Bardi–Koike–Soravia [6], Cardaliaguet–Quincampoix–Saint Pierre [14], Bettiol–Bressan–Vinter [8, 9], Bressan–Facchi [12], Bettiol–Facchi [10], Bettiol–Frankowska–Vinter [11], and Frankowska–Marchini–Mazzola [18].
Point (b): Section 5.
In that section, using the dynamic programming principle, we prove that \({\overline{V}}\) and \({\underline{V}}\) are viscosity solutions of the corresponding HJI equation with suitable boundary conditions in the viscosity sense. As expected, such boundary conditions are determined by the exit costs on the boundary. However, in our formulation of the differential game, we are considering different exist costs, depending on which of the two players is exiting (in a state-constraint framework: which of the two players are violating the constraint). This is an important feature of a state-constraint differential game, and hence of an exit time differential game. Which player is in charge in order to respect the constraint? Which player must be penalized when the constraint is violated? When the dynamics are not decoupled such questions have no evident answers, they may depend on the particular model under analysis. However, even if the game has a zero-sum structure (min–max), the definition of the right players’ responsibility with respect to the constraint is almost always not of that kind. In our case, the dynamics are decoupled and we have different exit costs and these facts allow to rightly assign the responsibility of exit from the constraint. The compatibility condition (6) helps to write a coherent and useful boundary condition for HJI. It says that, on the common boundary, the exit costs for the maximizing player are not larger than the cost of the minimizing one.
Point (c): Section 6.
We show uniqueness of \({\underline{V}}\) and \({\overline{V}}\) as viscosity solution of the corresponding Dirichlet problem for the upper and lower HJI, respectively. This is done by a rather standard double-variable technique for proving a comparison result between sub- and supersolutions, where the boundary conditions must be treated in a non-standard way.
2 The Problem
Let \(\varOmega _X\subseteq {\mathbb {R}}^{n}\) and \(\varOmega _Y\subseteq {\mathbb {R}}^{m}\) be two open regular sets with n, m positive integers; let \(A\subset {\mathbb {R}}^{n'},B\subset {\mathbb {R}}^{m'}\) be two compact sets; let \(f:{\mathbb {R}}^n\times A\rightarrow {\mathbb {R}}^n\) and \(g:{\mathbb {R}}^m\times B\rightarrow {\mathbb {R}}^m\) be two regular functions (i.e., bounded, continuous and Lipschitz continuous with respect to the state variable (their first entry) uniformly with respect to the control (their second entry)). We consider the system (1) where \(\alpha \) and \(\beta \) are, respectively, the measurable controls \(\alpha :[0,+\infty [\rightarrow A\), \(\beta :[0,+\infty [\rightarrow B\) (i.e., \(\alpha \in {{{\mathcal {A}}}}\) and \(\beta \in {{{\mathcal {B}}}}\)).
The player X uses the measurable control \(\alpha \) and governs the state variable \(X(t)\in {\mathbb {R}}^n\). On the other hand, the player Y uses the measurable control \(\beta \) and governs the state variable \(Y(t)\in {\mathbb {R}}^m\).
We are also given of a suitably regular running cost \(\ell :{\mathbb {R}}^n\times {\mathbb {R}}^m\times A\times B\rightarrow [0,+\infty [\) (i.e., bounded, continuous and Lipschitz continuous with respect to the state variables (its first two entries) uniformly with respect to the controls (its second two entries)) and of three suitably regular exit costs (i.e., bounded and continuous)
which, respectively, represent the costs for the exit of X only (from \({\overline{\varOmega }}_X\)), for the exit of Y only (from \({\overline{\varOmega }}_Y\)) and for the simultaneous exit of X and Y. Finally, we have a discount factor \(\lambda >0\).
Here we collect all such hypotheses, better specifying some of them and some notations.
In (2), \(\Vert \cdot \Vert \) stays, time by time, for the corresponding Euclidean norm. Moreover, some of the hypotheses may be relaxed, as it is quite common: the Lipschitz continuity of the running cost \(\ell \) with respect to the space variable may be relaxed to a simple uniform continuity, and the regularity of the boundaries of \(\varOmega _X\), \(\varOmega _Y\) may be relaxed to a suitable piece-wise \(C^2\) regularity (see, for example, Bagagiolo–Bardi [2] for such relaxation in the context of a constrained optimal control problem). Finally, in Sect. 7 we are going to relax a little bit the “decoupled” feature of the dynamics f and g with respect to the controls.
When a control \(\alpha \) is fixed, we define the corresponding trajectory of the first system in (1) as \(X(\cdot ;x,\alpha )\); similarly, we use the notation \(Y(\cdot ;y,\beta )\). We define the first exit time of X from \({\overline{\varOmega }}_X\) as
and, similarly, the first exit time of Y from \({\overline{\varOmega }}_Y\) as
with the convention \(\inf \emptyset =+\infty \).
In the following formulas, we use the notation \(\tau =\min \{(\tau _X(x,\alpha ),\tau _Y(y,\beta )\}\). We consider the cost functional J, defined on \({\overline{\varOmega }}_X\times {\overline{\varOmega }}_Y\times {\mathcal {A}}\times {\mathcal {B}}\),
where
The game consists in the fact that player X wants to minimize the cost J and the player Y wants to maximize J.
Definition 1
- i)
Let \(k,{\tilde{k}}\) be two nonnegative integers, and \({{\mathcal {U}}}\) be a set of measurable functions \(u:[0,+\infty [\rightarrow {\mathbb {R}}^{k}\). A map that sends any \(u\in {{{\mathcal {U}}}}\) to a measurable function \({\tilde{u}}:[0,+\infty [\rightarrow {\mathbb {R}}^{{\tilde{k}}}\) is a “non-anticipating tuning” if, for every \(u_1,u_2\in {{\mathcal {U}}}\) and for every \(t\ge 0\), the following holds
$$\begin{aligned} u_1=u_2\ \mathrm{a.e.\ in }\ [0,t]\ \Longrightarrow \ {\tilde{u}}_1={\tilde{u}}_2\ \mathrm{a.e\ in }\ [0,t]. \end{aligned}$$ - ii)
The “non-anticipating strategies for player X” (respectively, for player Y) are the elements of the set
$$\begin{aligned} \begin{array}{l} \displaystyle \varGamma =\left\{ \gamma :{{\mathcal {B}}}\rightarrow {{\mathcal {A}}},\ \beta \mapsto \gamma [\beta ]\ \Big |\ \forall t\ge 0,\right. \\ \displaystyle \ \ \ \ \ \ \ \ \ \beta _1=\beta _2\ \text{ a. } \text{ e. } \text{ in } [0,t]\Longrightarrow \gamma [\beta _1]=\gamma [\beta _2]\ \text{ a. } \text{ e. } \text{ in } [0,t]\Big \}; \end{array} \end{aligned}$$
(respectively,
Note that a non-anticipating strategy for player X is a non-anticipating tuning that sends measurable controls for player Y to measurable controls for player X. The concept of non-anticipating tuning will be used in the next sections. The concept of non-anticipating strategies is the one introduced by Elliot–Kalton in [16], and it is used for defining the lower and the upper value functions of the differential game, respectively, as
We say that the game has a value if \({\underline{V}}(x,y)={\overline{V}}(x,y)\) for all \((x,y)\in {\overline{\varOmega }}_X\times {\overline{\varOmega }}_Y\).
One of the possible interesting motivations/applications of differential games with exit cost can be seen in the so-called surge tank problem, as described in Vinter–Clark [22], and in Falugi–Kountouriotis–Vinter [17]. Surge tanks are flow control devices, whose purpose is to prevent flow rate fluctuations for fluids passing from one process unit to another one. In [17, 22] the authors, using a method given by Dupuis–McEneaney [15], regard the problem as a differential game, involving dynamics with two players X and Y, where the objective of the X player is to keep the state within a specified safe region, despite the best efforts of the Y player to drive the state out of this region. The dynamic equations of an ideal surge tank are
where \(x_1\) and \(x_2\) can be identified with the volume and rate of change of volume of fluid in the tank, respectively, \(\alpha \) is the control which regulates the rate of change of outflow and \(\beta \) is the disturbance. A possible upper game is given by
where \(\tau \) denotes the first exit time from a suitable open set. The X player wants to maximize the cost (to maintain the state in the safe region), whereas Y wants to minimize. Note that here the dynamics are not decoupled; however, if the disturbance enters the system in a “bounded manner,” then this case can be cast in the situation assumed in Sect. 7. In [17, 22] the authors are interested in bang-bang controls and in the decomposition of the problem into a collection of one player optimal control problems.
3 Controllability
In the next section, we are going to give some regularity results and properties of the value functions. Of course, suitable, but general, hypotheses are needed.
First of all, we assume a controllability hypothesis on the boundaries.
Assumption 1
For every \(x\in \partial \varOmega _X\) there exist two constant controls \(a_1,a_2\in A\) such that \(f(x,a_1)\) is strictly entering in \(\varOmega _X\) and \(f(x,a_2)\) is strictly entering in \({\mathbb {R}}^n{\setminus }{\overline{\varOmega }}_X\). That is, denoted by \(\eta _X\) the outer unit normal to the \(C^2\)-boundary set \(\varOmega _X\), then \(f(x,a_1)\cdot \eta _X(x)<0\) and \(f(x,a_2)\cdot \eta _X(x)>0\). Similarly, for every \(y\in \partial \varOmega _Y\) there exist two constant controls \(b_1,b_2\in B\) such that \(g(y,b_1)\) is strictly entering in \(\varOmega _Y\) and \(g(y,b_2)\) is strictly entering in \({\mathbb {R}}^m{\setminus }{\overline{\varOmega }}_Y\).
Such a controllability hypothesis is essential for having the continuity of the value functions. In particular, it is linked to the existence of suitable constrained non-anticipating strategy. Indeed, the continuity of the value functions for the exit time case presents similar features as the case of state-constraint. When we evaluate, for instance, the difference \({\underline{V}}(x_1,y_1)-{\underline{V}}(x_2,y_2)\) we need, for instance, the possibility of driving the state \(X(\cdot ;\cdot ,x_1)\) in such a way that it remains inside \({\overline{\varOmega }}_X\) until the state \(X(\cdot ;\cdot ,x_2)\) stays inside \({\overline{\varOmega }}_X\). This must be done in a way such that the variation of the cost is controlled, but the main difficulty here is the fact that it must be done in a non-anticipating way.
Definition 2
A modulus of continuity is an increasing and continuous function \(\omega :[0,+\infty [\rightarrow [0,+\infty [\) such that \(\omega (0)=0\). Given a function \(u:{\mathbb {R}}^n\rightarrow {\mathbb {R}}^m\), a modulus of continuity for u is a modulus of continuity \(\omega \) such that
It is well known that the existence of a modulus of continuity for u is equivalent to the fact that u is uniformly continuous.
Assumption 2
For every \(T>0\), for every \(K\subseteq {\overline{\varOmega }}_X\times {\overline{\varOmega }}_Y\) compact, there exists a modulus of continuity \({\mathcal {O}}_{T,K}\), and:
I) for every \((x_1,y_1),(x_2,y_2)\in K\), there exists a non-anticipating tuning \(\beta \mapsto {\tilde{\beta }}\) from \({{\mathcal {B}}}\) to itself (i.e., satisfying next point i), that is Definition 1), and there exists a way to associate \({\tilde{\gamma }}\in \varGamma \) to any \(\gamma \in \varGamma \), such that, for every \(\beta ,\beta _1,\beta _2\in {{{\mathcal {B}}}}\), \(\gamma \in \varGamma \), \(t\ge 0\), we have
where \({\tilde{\tau }}=\min {\{}\tau _X(x_2,\gamma [{\tilde{\beta }}]),\tau _Y(y_1,\beta ),T{\}}\), and \(J_{{\tilde{\tau }}}\) is the integral of the discounted running cost up to the time \({\tilde{\tau }}\).
II) A similar condition holds reversing the roles of X and Y, \(\gamma \in \varGamma \) and \(\xi \in \chi \), \(\alpha \in {{\mathcal {A}}}\) and \(\beta \in {{\mathcal {B}}}\).
Assumption 2 is required in order to guarantee the existence of suitable non-anticipating strategies and then prove the continuity of the values. We only need trajectories estimates on compact sets of time because the cost is discounted (the presence of the term \(e^{-\lambda t}\)), see the proof of Proposition 3. Under our hypotheses, in particular the decoupling of the dynamics, and the controllability on the boundaries, Assumption 2 holds, as it is proven in Bettiol–Cardaliaguet–Quincampoix [7]. More precisely, i), iv) and v) are treated in Proposition 3.1 and of vi) in Proposition 2.3 of [7]. However, note that in [7] such estimates are given for all times (and not only for compact sets) and indeed they are in a exponential fashion, which of course implies our uniform estimates on compact sets. On the other hand, conditions ii) and iii) just say that the constructed trajectories do not exit before the given ones. In Sect. 7 we are going to give a different proof of the validity of Assumption 2, modifying, in a non-anticipating manner, the proof of Soner [21] for the construction of constrained controls.
4 The Lower and the Upper Value Functions
By standard calculations (see, for example, Bardi–Capuzzo Dolcetta [5]), \({\underline{V}}\) and \({\overline{V}}\) satisfy the usual dynamic programming principle (DPP). For example, for every \(t\ge 0\)
where \(\tau =\min \{t,\tau _X(x,\gamma [\beta ]),\tau _Y(y,\beta )\}\).
We now assume that (see Remark 4 for comments on it)
Proposition 3
Given Assumption 1, Assumption 2, hypothesis (6) and hypotheses (2), the value functions are continuous in \({\overline{\varOmega }}_X\times {\overline{\varOmega }}_Y\).
Proof
We only prove the continuity of the lower value \({\underline{V}}\). In particular, we are going to prove its uniform continuity in every compact set. We proceed by some steps.
- (1)
Let us fix \(\varepsilon >0\) and take \(T>0\) such that, for every trajectories and controls entering the costs, \(\int _T^{+\infty } e^{-\lambda t}\ell \mathrm{d}t+e^{-\lambda T}\psi \le \varepsilon \), where \(\psi \) is any one of the exit costs \(\psi _X,\psi _Y,\psi _{XY}\). This is possible, independently from the trajectories and controls inside the costs, because of the boundedness hypotheses (2), and by the fact that the cost is discounted, i. e. \(\lambda >0\).
- (2)
Let \(K\subseteq {\overline{\varOmega }}_X\times {\overline{\varOmega }}_Y\) be a compact set (where we are going to prove the uniform continuity). Take another compact set \(K'\), with \(K\subseteq K'\subseteq {\overline{\varOmega }}_X\times {\overline{\varOmega }}_Y\), such that all the trajectories starting from points of K belong to \(K'\), for times not greater than T before they possibly exit from \({\overline{\varOmega }}_X\) and \({\overline{\varOmega }}_Y\), respectively. That is, for example, \(X(t;x,\alpha )\in K'\) for all \(t\in [0,\min \{\tau _X(x,\alpha ),T\}]\). Such a compact set exists by the hypotheses (2).
- (3)
By the compactness of \(K'\) and by the continuity regularities (2), there exist \(\zeta >0\) and a modulus of continuity \(\omega \) such that, for every \((x,y)\in \left( \partial \varOmega _X\times \partial \varOmega _Y\right) \cap K'\), taking the constant controls \(a_2\in A\) and \(b_2\in B\) as in the Assumption 1 with respect to \(x\in \partial \varOmega _X\) and \(y\in \partial \varOmega _Y\), respectively, we have the following:
- for every \(x'\in {\overline{\varOmega }}_X\) with \(\Vert x-x'\Vert \le \zeta \), the trajectory starting from \(x'\) with constant control \(a_2\) exits from \({\overline{\varOmega }}_X\) in a time interval whose length is less than \(\omega (\Vert x-x'\Vert )\); for every \(y'\in {\overline{\varOmega }}_Y\) with \(\Vert y-y'\Vert \le \zeta \), the trajectory starting from \(y'\) with constant control \(b_2\) exits from \({\overline{\varOmega }}_Y\) in a time interval whose length is less than \(\omega (\Vert y-y'\Vert )\).-
This in particular comes, besides the controllability Assumption 1 and the Lipschitz regularity of the dynamics, from the \(C^2\) regularity of the boundaries, which implies that, for any piece of boundary in a compact set, the signed distance function from the boundary, d, is \(C^2\) in a neighborhood of it. Hence, one can argue estimating the signed of the composed function \(t\mapsto d(z(t))\) where z is the considered trajectory (see Bardi–Capuzzo Dolcetta [5] p. 272, for a similar treatment of that function).
- (4)
Let \({{{\mathcal {O}}}}_{T,K}\) as in Assumption 2 with respect to T and K fixed above, and \({\overline{\omega }}\) be a modulus of continuity for \(\psi _X,\psi _Y,\psi _{XY}\) in their domains inside \(K'\). We define, the following modulus of continuity
$$\begin{aligned} {\tilde{\omega }}(r)=\max \{{{{\mathcal {O}}}}_{T,K}(r),(M+1) \omega ({{{\mathcal {O}}}}_{T,K}(r)), {\overline{\omega }}((M+1)\omega ({{{\mathcal {O}}}}_{T,K}(r)))\} \end{aligned}$$where M is the bound of f, g, \(\ell \) and of the exits costs as in (2), and \(\omega \) is as in point 3).
- (5)
Our goal is to show that, there exists \(\delta >0\) such that, for all \((x_1,y_1),(x_2,y_2)\in K\) with \(\Vert (x_1,y_1)-(x_2,y_2)\Vert \le \delta \), we have \(|{\underline{V}}(x_1,y_1)-{\underline{V}}(x_2,y_2)|\le 4\varepsilon \).
- (6)
In the sequel, we can be concerned with the behavior of the trajectories in the time interval [0, T] only. In particular, all the exit time we are going to consider will be assumed to be less than T. Indeed, by the previous point 1), we are going to perform the comparison analysis of the costs up to the time T, because, even if the game runs after T, then all the costs (the integrated one as well as exit ones) that will be accumulated after that time, will differ for a quantity not greater than \(2\varepsilon \). Also note that, when the game run up to the time T, the estimate of the difference of the accumulated running costs (the integrated ones only) is standard as in the infinite-horizon case (no exit time), see, for example, Bardi–Capuzzo Dolcetta [5], Chapter VIII Proposition 1.8 and Chapter III Proposition 2.1. Hence, in the following points, and in particular, in the next formulas (7) and (9), we will assume, respectively,
$$\begin{aligned} 0\le \tau _Y(y_1,\beta )\le T,\ \ 0\le \tau _X(x_2,\gamma [\beta ])\le T \end{aligned}$$ - (7)
Now, take \(\delta >0\) such that \({\tilde{\omega }}(\delta )\le \zeta \) and take two arbitrary points \((x_1,y_1),(x_2,y_2)\in K\) such that \(\Vert (x_1,y_1)-(x_2,y_2)\Vert \le \delta \), where \(\zeta \) is given in point 3). In the sequel, \({\tilde{\gamma }}\), \({\tilde{\beta }}\) are the ones defined in Assumption 2.
- (8)
Now, we exhibit a suitable non-anticipating tuning (see Definition 1) \(\beta \mapsto {\overline{\beta }}\). For every \(\beta \in {\mathcal {B}}\) let us define \({\overline{\beta }}\in {\mathcal {B}}\) for \(t\in [0,T]\) as
$$\begin{aligned} {\overline{\beta }}(t)=\left\{ \begin{array}{ll} \displaystyle {\tilde{\beta }}(t)&{}\quad \text{ if } 0\le t\le \tau _Y(y_1,\beta ),\\ \displaystyle b_2&{}\quad \text{ otherwise }, \end{array} \right. \end{aligned}$$(7)where \(b_2\in B\) is as in Assumption 1 with respect to \(Y(\tau _Y(y_1,\beta );y_1,\beta )\in \partial \varOmega _Y\). Hence, by points 3)–7) and Assumption 2 (points iii) and v)), if we use the control \({\overline{\beta }}\) starting from \(y_2\), then the trajectory exits from \({\overline{\varOmega }}_Y\) with exit time \(\tau _Y(y_2,{\overline{\beta }})\) satisfying
$$\begin{aligned} 0\le \tau _Y(y_2,{\overline{\beta }})-\tau _Y(y_1,\beta )\le {\tilde{\omega }}(\delta ). \end{aligned}$$(8)
In particular, \(\tau _Y(y_2,{\overline{\beta }})\) is not less than \(\tau _Y(y_1,\beta )\) because, up to the time \(\tau _X(y_1,\beta )\), starting from \(y_2\) and using the control \({\tilde{\beta }}\), the trajectory does not exit from \({\overline{\varOmega }}_Y\) (point iii) of Assumption 2). Moreover, by point v) of Assumption 2, \(Y(\tau _Y(y_1,\beta );y_2,{\tilde{\beta }})\) is sufficiently close to the boundary (because \(Y(\tau _Y(y_1,\beta );y_1,\beta )\in \partial \varOmega _Y\)), and hence, by point 3), the trajectory “rapidly” exits from \({\overline{\varOmega }}_Y\) using the constant control \(b_2\), that is (8).
Also note that, being the dynamics bounded by M, it is
Finally, such a construction of \({\overline{\beta }}\) is a non-anticipating tuning. Indeed, if \(\beta _1=\beta _2\) a. e. in the time interval [0, T], then the controls \({\tilde{\beta }}_1\) and \({\tilde{\beta }}_2\) are also equal a.e. in [0, T] by point i) of Assumption 2. Moreover, the trajectories \(Y(\cdot ;y_1,\beta _1)\) and \(Y(\cdot ;y_1,\beta _2)\) are also equal in the time interval [0, t]. Then in the interval [0, t], they possibly generate the same exit time \(\tau _Y(y_1,\beta _1)=\tau _Y(y_1,\beta _2)\), and hence, we must have \({\overline{\beta }}_1={\overline{\beta }}_2\) a.e. in [0, t].
- (9)
Similarly, as in point 8), starting from a non-anticipating strategy for X\(\gamma \) we define a new non-anticipating strategy for X, \({\overline{\gamma }}\):
$$\begin{aligned} {\overline{\gamma }}[\beta ](t)=\left\{ \begin{array}{ll} \displaystyle {\tilde{\gamma }}[\beta ](t)&{}\quad \text{ if } 0\le t\le \tau _X(x_2,\gamma [\beta ]),\\ \displaystyle a_2&{}\quad \text{ otherwise }, \end{array} \right. \end{aligned}$$(9)where \(a_2\in A\) is as in Assumption 1 with respect to \(X(\tau _X(x_2,\gamma [\beta ]);x_2,\gamma [\beta ])\in \partial \varOmega _X\). In this case, by Assumption 2 points ii) and iv), and by the previous points of this proof, we have, as in point 8),
$$\begin{aligned} \begin{array}{ll} \displaystyle 0\le \tau _X(x_1,{\overline{\gamma }}[\beta ])-\tau _X(x_2,\gamma [\beta ])\le {\tilde{\omega }}(\delta )\\ \displaystyle \Vert X(\tau _X(x_1,{\overline{\gamma }}[\beta ]);x_1,{\overline{\gamma }}[\beta ])-X(\tau _X(x_2,\gamma [\beta ]);x_2,\gamma [\beta ])\Vert \le {\tilde{\omega }}(\delta ). \end{array} \end{aligned}$$(10)
Note that, by our hypotheses, in particular because \({\tilde{\gamma }}\) is a non-anticipating strategy (the one given by Assumption 2), we have that \({\overline{\gamma }}\) is also a non-anticipating strategy. Moreover, for every \(\gamma \in \varGamma \), we also consider the following non-anticipating strategy \(\overline{{\overline{\gamma }}}\in \varGamma \), defined as \(\overline{{\overline{\gamma }}}[\beta ]={\overline{\gamma }}[{\overline{\beta }}]\), for all \(\beta \in {\mathcal {B}}\).
- (10)
For suitable \(\gamma _2\in \varGamma \) and \(\beta _1\in {{{\mathcal {B}}}}\), by definition of infimum and supremum, we have
$$\begin{aligned} \begin{array}{l} \displaystyle {\underline{V}}(x_1,y_1)-{\underline{V}}(x_2,y_2)\le \inf _{\gamma \in \varGamma } \sup _{\beta \in {\mathcal {B}}}J(x_1,y_1, \gamma [\beta ],\beta )-\sup _{\beta \in {\mathcal {B}}}J(x_2,y_2, \gamma _2[\beta ],\beta )+\varepsilon \\ \qquad \displaystyle \le \sup _{\beta \in {\mathcal {B}}}J(x_1,y_1,\overline{{ \overline{\gamma }}}_2[\beta ],\beta )-\sup _{\beta \in {\mathcal {B}}} J(x_2,y_2,\gamma _2[\beta ],\beta )+\varepsilon \\ \qquad \displaystyle \le J(x_1,y_1,\overline{{\overline{\gamma }}}_2[\beta _1],\beta _1)- \sup _{\beta \in {\mathcal {B}}}J(x_2,y_2,\gamma _2[\beta ],\beta )+2\varepsilon \\ \qquad \le J(x_1,y_1,\overline{{\overline{\gamma }}}_2[\beta _1], \beta _1)-J(x_2,y_2,\gamma _2[{\overline{\beta }}_1],{\overline{\beta }}_1)+2\varepsilon \\ \qquad = J(x_1,y_1,{\overline{\gamma }}_2[{\overline{\beta }}_1],\beta _1)-J(x_2,y_2, \gamma _2[{\overline{\beta }}_1],{\overline{\beta }}_1)+2\varepsilon . \end{array} \end{aligned}$$Now, we define \({\overline{\tau }}_{12}=\min (\tau _X(x_2,\gamma _2[{\overline{\beta }}_1]),\tau _Y(y_1,\beta _1))\). Hence, we have
$$\begin{aligned} \begin{array}{l} \displaystyle {\underline{V}}(x_1,y_1)-{\underline{V}}(x_2,y_2)\\ \qquad \displaystyle \le J_{{\overline{\tau }}_{12}}(x_1,y_1,{\overline{\gamma }}_2[{\overline{\beta }}_1],\beta _1)+\varPsi _1-J_{{\overline{\tau }}_{12}}(x_2,y_2,\gamma _2[{\overline{\beta }}_1],{\overline{\beta }}_1)-\varPsi _2+3\varepsilon , \end{array} \end{aligned}$$where \(J_{{\overline{\tau }}_{12}}\) stays for the integral cost up to the time \({\overline{\tau }}_{12}\), and \(\varPsi _1\) and \(\varPsi _2\) stay to indicate two possible sums of remaining integral cost and discounted exit costs paid by the trajectories (their explicit formulations will be given in the following points).
By our definition, note that, up to the time \({\overline{\tau }}_{12}\), it is \({\overline{\gamma }}_2[{\overline{\beta }}_1]={\tilde{\gamma }}_2[{\tilde{\beta }}_1]\), \(\gamma _2[{\overline{\beta }}_1]=\gamma _2[{\tilde{\beta }}_1]\), and \({\overline{\beta }}_1={\tilde{\beta }}_1\). Hence, by Assumption 2 point vi), and by the previous points of this proof, we have
- (11)
We now analyze \(\varPsi _{1}-\varPsi _{2}\). We have some subcases.
- (11a)
\({\overline{\tau }}_{12}=\tau _X(x_2,\gamma _2[{\overline{\beta }}_1])\le \tau _X(x_1,{\overline{\gamma }}_2[{\overline{\beta }}_1])<\tau _Y(y_1,\beta _1)\). Then
$$\begin{aligned} \begin{array}{l} \displaystyle \varPsi _1=\int _{{\overline{\tau }}_{12}}^{\tau _X(x_1,{\overline{\gamma }}_2 [{\overline{\beta }}_1])}e^{-\lambda s}\ell \mathrm{d}s\\ \qquad \displaystyle \ +e^{-\lambda \tau _X(x_1,{\overline{\gamma }}_2[{\overline{\beta }}_1])}\psi _X(X(\tau _X(x_1,{ \overline{\gamma }}_2[{\overline{\beta }}_1]);x_1,{\overline{\gamma }}_2[{\overline{\beta }}_1]), Y(\tau _X(x_1,{\overline{\gamma }}_2[{\overline{\beta }}_1]);y_1,\beta _1)),\\ \qquad \displaystyle \varPsi _2=e^{-\lambda {\overline{\tau }}_{12}}\psi _X(X({\overline{\tau }}_{12};x_2,\gamma _2[{\overline{\beta }}_1]),Y({\overline{\tau }}_{12};y_2,{\overline{\beta }}_1)) \end{array} \end{aligned}$$From this, by the previous points, the uniform continuity of \(\psi _X\) in \({\overline{\varOmega }}_X\cap K\), and the definition of \({\tilde{\omega }}\), we get
$$\begin{aligned} |\varPsi _1-\varPsi _2|\le 2{\tilde{\omega }}(\delta ), \end{aligned}$$ - (11b)
\({\overline{\tau }}_{12}=\tau _X(x_2,\gamma _2[{\overline{\beta }}_1])\le \tau _Y(y_1,\beta _1)\le \tau _X(x_1,{\overline{\gamma }}_2[{\overline{\beta }}_1])\). Hence, \(\varPsi _1\) is the integral of the cost on the interval \([{\overline{\tau }}_{12},\tau _Y(y_1,\beta _1)]\) (whose length is not greater than \({\tilde{\omega }}(\delta )\), (10)), plus the discounted exit cost \({\tilde{\varPsi }}_1\), the latter being (respectively, for \(\tau _Y(y_1,\beta _1)<\tau _X(x_1,{\overline{\gamma }}_2[{\overline{\beta }}_1])\) and for \(\tau _Y(y_1,\beta _1)=\tau _X(x_1,{\overline{\gamma }}_2[{\overline{\beta }}_1])\)):
$$\begin{aligned} {\tilde{\varPsi }}_1= & {} e^{-\lambda \tau _Y(y_1,\beta _1)} \psi _Y(X(\tau _Y(y_1,\beta _1);x_1,{\overline{ \gamma }}_2[{\overline{\beta }}_1]),Y(\tau _Y(y_1,\beta _1);y_1,\beta _1)),\\ {\tilde{\varPsi }}_1= & {} e^{-\lambda \tau _Y(y_1,\beta _1)}\psi _{XY}(X( \tau _Y(x_1,{\overline{\gamma }}_2[{\overline{\beta }}_1]);x_1,{ \overline{\gamma }}_2[{\overline{\beta }}_1]),Y(\tau _Y(y_1,\beta _1);y_1,\beta _1)) \end{aligned}$$Similarly, \(\varPsi _2\), which has no integral part, is (respectively, for \({\overline{\tau }}_{12}=\tau _X(x_2,\gamma _2[{\overline{\beta }}_1])<\tau _Y(y_2,{\overline{\beta }}_1)\) and for \({\overline{\tau }}_{12}=\tau _X(x_2,\gamma _2[{\overline{\beta }}_1])=\tau _Y(y_2,{\overline{\beta }}_1)\))
$$\begin{aligned} \varPsi _2= & {} e^{-\lambda {\overline{\tau }}_{12}}\psi _X(X({\overline{\tau }}_{12};x_2,\gamma _2[{\overline{\beta }}_1]),Y({\overline{\tau }}_{12};y_2,{\overline{\beta }}_1)\\ \varPsi _2= & {} e^{-\lambda {\overline{\tau }}_{12}}\psi _{XY}(X({\overline{\tau }}_{12};x_2,\gamma _2[{\overline{\beta }}_1]),Y({\overline{\tau }}_{12};y_2,{\overline{\beta }}_1). \end{aligned}$$Using (6), all the estimates of the previous points, and the uniform continuity in the compact sets of the exit costs, together with the definition of \({\tilde{\omega }}\), we have
$$\begin{aligned} \displaystyle {\tilde{\varPsi }}_1\le & {} e^{-\lambda {\overline{\tau }}_{12}}\psi _{XY}(X(\tau _X(x_2,\gamma _2[{\overline{\beta }}_1]);x_2,\gamma _2[{\overline{\beta }}_1]),Y(\tau _Y(y_1,\beta _1);y_1,\beta _1))+{\tilde{\omega }}(\delta ),\\ \varPsi _2\ge & {} e^{-\lambda {\overline{\tau }}_{12}}\psi _{XY}(X(\tau _X(x_2,\gamma _2[{\overline{\beta }}_1]);x_2,\gamma _2[{\overline{\beta }}_1]),Y(\tau _Y(y_1,\beta _1);y_1,\beta _1))-{\tilde{\omega }}(\delta ), \end{aligned}$$from which
$$\begin{aligned} |\varPsi _1-\varPsi _2|\le 3{\tilde{\omega }}(\delta ). \end{aligned}$$ - (11c)
The cases \({\overline{\tau }}_{12}=\tau _Y(y_1,\beta _1)\le \tau _Y(y_2,{\overline{\beta }}_1)<\tau _X(x_2,\gamma _2[{\overline{\beta }}_1])\) and \({\overline{\tau }}_{12}=\tau _Y(y_1,\beta _1)\le \tau _X(x_2,\gamma _2[{\overline{\beta }}_1])\le \tau _Y(y_2,{\overline{\beta }}_1)\) are similar to points 11a) and 11b), respectively.
- (12)
Putting together the points 10) and 11), and reversing the role of \((x_1,y_1)\) and \((x_2,y_2)\), we then get point 5) taking \(\delta \) sufficiently small. The proof is concluded. \(\square \)
Remark 4
Note that, whenever hypothesis (6) is not satisfied, then the continuity is not guaranteed. For example, if \(\psi _X<\psi _Y\) on \(\partial \varOmega _X\times \partial \varOmega _Y\), then, using also the controllability hypothesis on the boundaries, we can approximate points on \(\partial \varOmega _X\times \partial \varOmega _Y\) by points in \(\partial \varOmega _X\times \varOmega _Y\) and by points in \(\varOmega _X\times \partial \varOmega _Y\) where, respectively, \({\underline{V}}\le \psi _X\) and \({\underline{V}}\ge \psi _Y\). And this fact makes immediately fail the continuity of \({\underline{V}}\) on points of \(\partial \varOmega _X\times \partial \varOmega _Y\).
5 The Lower and the Upper Hamilton–Jacobi–Isaacs Equations and Boundary Conditions
For every \((x,y)\in {\overline{\varOmega }}_X\times {\overline{\varOmega }}_Y\) and for every \((p,q)\in {\mathbb {R}}^n\times {\mathbb {R}}^m\), we introduce the Upper Hamiltonian (“\(\cdot \)” stays for the scalar product)
and the lower Hamiltonian
In the sequel, for a function u, \(u_x(x,y)\) and \(u_y(x,y)\) will denote the gradient with respect to x and with respect to y, respectively.
Theorem 5
Given Assumption 1, regularities (2) and condition (6), the lower value \({\underline{V}}\) satisfies the following (upper) problem in the viscosity sense (here expressed for a generic function \(u:{\overline{\varOmega }}_X\times {\overline{\varOmega }}_Y\rightarrow {\mathbb {R}}\))
By solutions in the viscosity sense, we mean the following: let \(\varphi \in C^1({\overline{\varOmega }}_X\times {\overline{\varOmega }}_Y)\) and \((x_0,y_0)\in {\overline{\varOmega }}_X\times {\overline{\varOmega }}_Y\), then the following facts i) and ii) hold true:
i) if \((x_0,y_0)\) is a point of local maximum for \(u-\varphi \), with respect to \({\overline{\varOmega }}_X\times {\overline{\varOmega }}_Y\), then we have the following four implications (one per every line)
ii) if \((x_0,y_0)\) is a point of local minimum for \(u-\varphi \), with respect to \({\overline{\varOmega }}_X\times {\overline{\varOmega }}_Y\), then we have the following four implications (one per every line)
If u satisfies i), it is said to be a subsolution; if it satisfies ii), it is said to be a supersolution. The equation in the first line of (11) is called the (upper) Hamilton–Jacobi–Isaacs equation (or simply Isaacs). The implications given by the second, third and fourth lines of (12)–(13) represent the boundary conditions in the viscosity sense.
Note that, in the formulation of (11), the intermediate exit cost \(\psi _{XY}\) does not play any role. We refer the reader to Remark 8 for more details on this fact.
Remark 6
Under the same hypotheses of Theorem 5, and with the same definitions for solutions and the boundary conditions, the upper value function \({\overline{V}}\) is a solution in the viscosity sense of
Proof of Theorem 5
Let us note that, by our hypothesis of controllability on the boundaries, we have the following inequalities
This is because both players, from their own boundary, can immediately exit, stopping the game and paying the corresponding exit cost.
Hence, for the subsolution case, we only need to prove the validity of the Isaacs equation in \(\varOmega _X\times \varOmega _Y\), the boundary condition on \(\varOmega _X\times \partial \varOmega _Y\) and the boundary condition on \(\partial \varOmega _X\times \partial \varOmega _Y\) for only the case \(\psi _Y<{\underline{V}}<\psi _X\). In the same way, for the supersolution case, we only need to prove the Isaacs equation in \(\varOmega _X\times \varOmega _Y\), the boundary condition on \(\partial \varOmega _X\times \varOmega _Y\) and the boundary condition on \(\partial \varOmega _X\times \partial \varOmega _Y\) for only the case \(\psi _X>{\underline{V}}>\psi _Y\). Since the validity of the Isaacs equation in \(\varOmega _X\times \varOmega _Y\) is standard (see for instance Bardi–Capuzzo Dolcetta [5] p. 438), we only concentrate on the boundary conditions.
Supersolution Let \((x_0,y_0)\in \partial \varOmega _X\times \varOmega _Y\) be of minimum for \({\underline{V}}-\varphi \), and by absurd hypothesis, let us suppose that
Of course, it is not restrictive to suppose that \({\underline{V}}(x_0,y_0)=\varphi (x_0,y_0)\) and that \(\lambda =1\). Also using this assumption, we have that, for some \(\varepsilon >0\), and for every \((x,y)\in B((x_0,y_0),\varepsilon )\cap {\overline{\varOmega }}_X\times {\overline{\varOmega }}_Y\) (here B((x, y), r) stays for a ball of \({\mathbb {R}}^n\times {\mathbb {R}}^m\) with center in (x, y) and radius \(r>0\))
Moreover, we can also suppose that \(y\in \varOmega _Y\) for all \((x,y)\in B((x_0,y_0),\varepsilon )\). Now, let \(t>0\) be such that, for every \(\gamma \in \varGamma \) and \(\beta \in {\mathcal {B}}\), \((X(s;x,\gamma [\beta ]),Y(s;y,\beta ))\in B((x_0,y_0),\varepsilon )\) for all \((x,y)\in B((x_0,y_0),\varepsilon /2)\), for all \(0\le s\le t\). Let us define \(\delta =\varepsilon (1-e^{-t})/2>0\) and take \(\gamma \in \varGamma \) such that, by the dynamic programming principle (5), for any \(\beta \in {\mathcal {B}}\) (note that it is \(\tau _Y(y_0,\beta )\ge t\))
In particular, we can take \(\beta \equiv b\), with \(b\in B\) arbitrary. Note that we can certainly suppose that \(0<t<\tau _X(x_0,\gamma [\beta ])\) for all \(\beta \). Indeed, if not, for three sequences \(t_n\rightarrow 0^+\), \(\gamma _n\in \varGamma \), and \(\beta _n\in {{{\mathcal {B}}}}\) we would have \([0,t_n]\ni \tau _X(x_0,\gamma _n[\beta _n])\rightarrow 0^+\) and (dropping the notations of the trajectories in the entries)
where \(\delta _n=\varepsilon (1-e^{-t_n})/2\). But then, in the limit, this implies \({\underline{V}}(x_0,y_0)=\psi _X(x_0,y_0)\), against our absurd hypothesis (18). Hence, we get, for every \(b\in B\), (here, \(X(\cdot )\) and \(Y(\cdot )\) stay for the trajectories starting from \(x_0\) and \(y_0\) with controls \(\gamma [b]\) and b, respectively)
In the previous formula, we used (16), the fact that \(\gamma [b](s)\) is almost everywhere an element of A, and the time derivative of the function \(s\mapsto e^{-s}\varphi (X(s),Y(s))\). By the arbitrariness of \(b\in B\), and by the definitions of the Hamiltonian UH and of \(\delta \), we then get the following contradiction
Let us now consider the case \((x_0,y_0)\in \partial \varOmega _X\times \partial \varOmega _Y\). We only have the case \(\psi _Y(x_0,y_0)<{\underline{V}}(x_0,y_0)<\psi _X(x_0,y_0)\), and we can again restrict ourselves to the case \(0<t<\min \{\tau _Y,\tau _X\}\), and then, arguing as before, we get the conclusion.
Subsolution We only treat the case \((x_0,y_0)\in \varOmega _X\times \partial \varOmega _Y\) of maximum for \({\underline{V}}-\varphi \), the other cases being similar. Let us take \(\varepsilon \), \(t>0\) and \(\delta >0\) in a similar way as before (changing the role of \(\partial \varOmega _X\) and of \(\partial \varOmega _Y\)). For every \(a\in A\) (and so for every constant strategies \(\gamma \equiv a\)) we find \(\beta \in {\mathcal {B}}\) such that \(0<t<\tau _Y(y_0,\beta )\) independently on a (otherwise we get the contradiction \({\underline{V}}(x_0,y_0)\le \psi _Y(x_0,y_0)\)) and
Hence, for every \(a\in A\), we get
from which the contradiction (by the arbitrariness of \(a\in A\))
\(\square \)
6 Uniqueness
We are going to use the following inner-cone property of the boundaries. There exist two bounded continuous functions \(\eta _X:{\overline{\varOmega }}_X\rightarrow {\mathbb {R}}^n\), \(\eta _Y:{\overline{\varOmega }}_Y\rightarrow {\mathbb {R}}^m\), and two real positive continuous functions \(c,d:{\overline{\varOmega }}_X\cup {\overline{\varOmega }}_Y\rightarrow ]0,+\infty [\) such that, for all \(x\in {\overline{\varOmega }}_X\) (respectively, \(y\in {\overline{\varOmega }}_Y\)) and for all \(s\in ]0,d(x)]\) (respectively, \(s\in ]0,d(y)]\)),
Condition (17) roughly means that at every point there is a small cone with vertex in that point and contained, besides the vertex, in the interior of the set. Note that this is essentially a boundary regularity property, and it is certainly satisfied under the \(C^2\)-regularity hypothesis (2), where you can take, on the boundary, the unit interior normal as \(\eta _X\) and \(\eta _Y\). Finally, note that in any compact subset, \(\eta _X\) and \(\eta _Y\) can be assumed uniformly continuous, and c and d just two positive constants.
Theorem 7
Let Assumptions 1, (2) (and hence (17)), and (6) hold. Then the lower value \({\underline{V}}\) (respectively, the upper value function \({\overline{V}}\)) is the unique bounded and continuous function on \({\overline{\varOmega }}_X\times {\overline{\varOmega }}_Y\) which is a solution of (11) (respectively, of (14)) in the viscosity sense.
We are going to only prove uniqueness for (11) among continuous and bounded functions, from which the theorem follows because \({\underline{V}}\) is a continuous and bounded solution in the viscosity sense. As usual, we prove such a uniqueness result by proving a comparison result between sub- and supersolutions, and we will refer to the standard double-variable technique, and in particular to the “constrained” double-variable technique of Soner (see Bardi–Capuzzo Dolcetta [5] pp. 278–281) Let \(u,v:{\overline{\varOmega }}_X\times {\overline{\varOmega }}_Y\rightarrow {\mathbb {R}}\) be two bounded and continuous sub- and supersolution, respectively. We are going to prove that \(u\le v\) on \({\overline{\varOmega }}_X\times {\overline{\varOmega }}_Y\), from which the conclusion follows because every solution in the viscosity sense is simultaneously a sub- and a supersolution.
The standard procedure is as here explained. By contradiction, let us suppose
and try to construct two test functions \(\varphi _1\) and \(\varphi _2\) such that, for some \(({\overline{x}}_1,{\overline{y}}_1),({\overline{x}}_2,{\overline{y}}_2)\in {\overline{\varOmega }}_X\times {\overline{\varOmega }}_Y\) and some \(k>0\),
We then get the contradiction
The main ingredients for this procedure are some continuity and uniform continuity properties satisfied by the Hamiltonian UH (see Bardi–Capuzzo Dolcetta [5] p. 443 formula (2.1)), and the construction of a suitable penalizing function \({\tilde{\phi }}:({\overline{\varOmega }}_X\times {\overline{\varOmega }}_Y)\times ({\overline{\varOmega }}_X\times {\overline{\varOmega }}_Y)\rightarrow {\mathbb {R}}\). Given such a function \({\tilde{\phi }}\), the standard double-variable technique is to consider the function
and a point of maximum \((({\overline{x}}_1,{\overline{y}}_1),({\overline{x}}_2,{\overline{y}}_2))\) of it. Such a maximum point gives the candidate points for the above explained procedure with
As for the state-constraint optimal control problem, the main difficult is to be able to guarantee that, in the points \(({\overline{x}}_1,{\overline{y}}_1),({\overline{x}}_2,{\overline{y}}_2)\), both equation inequalities hold (the last two lines of (19)). This is made by a suitable use of the boundary conditions (12),(13), and by the use of a suitable penalizing term inside \({\tilde{\phi }}\) which avoids \(({\overline{x}}_1,{\overline{y}}_1),({\overline{x}}_2,{\overline{y}}_2)\), or some components of them, to belong to the boundaries.
For this reason, in the following proof, we are mostly going to put in evidence the suitable penalizing term and the use of the boundary conditions for our particular case of problem (12), (13), and to refer to Bardi–Capuzzo Dolcetta [5] (pp. 279–280) for the “standard” part in order to obtain all the lines of (19) and the contradiction (20). The point c) of the proof is more detailed.
Proof of Theorem 7
Let us assume the absurd hypothesis (18) and take \(\delta >0\) and \((x_0,y_0)\in {\overline{\varOmega }}_X\times {\overline{\varOmega }}_Y\) such that the other following absurd hypothesis holds
We only treat the boundary case \((x_0,y_0)\in \partial (\varOmega _X\times \varOmega _Y)\) and, if \((x_0,y_0)\in \partial \varOmega _X\times \partial \varOmega _Y\), \(\psi _Y(x_0,y_0)<\psi _X(x_0,y_0)\). The other cases are similar or easier. For example, when \(\psi _Y(x_0,y_0)=\psi _{XY}(x_0,y_0)=\psi _X(x_0,y_0)\), then we are in a sort of continuity case, all exit costs almost coincide around \((x_0,y_0)\), and the situation is similar to the one with continuous datum on the whole boundary.
We take \(\varepsilon >0\) and, for every one of the following cases, we consider a suitable “double-variable” function \(\phi :({\overline{\varOmega }}_X\times {\overline{\varOmega }}_Y)\times ({\overline{\varOmega }}_X\times {\overline{\varOmega }}_Y)\rightarrow {\mathbb {R}}\), as here explained. In the following, \(\zeta \) is a \(C^1\) positive function on \({\mathbb {R}}^n\times {\mathbb {R}}^m\) with bounded gradient and such that \(\zeta (x_0,y_0)=0\) and that \(\zeta \rightarrow +\infty \) when \(\Vert (x,y)\Vert \rightarrow +\infty \), and \(\mu >0\) is a constant whose value will be fixed later (see Bardi–Capuzzo Dolcetta [5], p. 54, for the use of this kind of functions in the comparison results in case of an unbounded domain).
a) i) \((x_0,y_0)\in \partial \varOmega _X\times \varOmega _Y\) and \(v(x_0,y_0)<\psi _X(x_0,y_0)\); or ii) \((x_0,y_0)\in \varOmega _X\times \partial \varOmega _Y\) and \(v(x_0,y_0)<\psi _Y(x_0,y_0)\); or iii) \((x_0,y_0)\in \partial \varOmega _X\times \partial \varOmega _Y\) and \(\psi _Y(x_0,y_0)<v(x_0,y_0)<\psi _X(x_0,y_0)\); or iv) \((x_0,y_0)\in \partial \varOmega _X\times \partial \varOmega _Y\) and \(\psi _Y(x_0,y_0)\ne v(x_0,y_0)<\psi _X(x_0,y_0)\):
b) i) \((x_0,y_0)\in \partial \varOmega _X\times \varOmega _Y\) and \(v(x_0,y_0)\ge \psi _X(x_0,y_0)\); or ii) \((x_0,y_0)\in \varOmega _X\times \partial \varOmega _Y\) and \(v(x_0,y_0)\ge \psi _Y(x_0,y_0)\); or iii) \((x_0,y_0)\in \partial \varOmega _X\times \partial \varOmega _Y\) and \(v(x_0,y_0)\ge \psi _X(x_0,y_0)\):
c) \((x_0,y_0)\in \partial \varOmega _X\times \partial \varOmega _Y\) and \(v(x_0,y_0)=\psi _Y(x_0,y_0)\):
Note the differences: from \(\phi _a\) to \(\phi _b\): in all penalizing terms the role of indexes 1 and 2 are mutually exchanged; from \(\phi _a\) to \(\phi _c\): in the second penalizing terms the role of indexes 1 and 2 are mutually exchanged. This means that, when performing the usual double-variable technique for comparison results, for suitable test functions \(\varphi _1\) and \(\varphi _2\) defined as in (21), in the case a) we are going to detach maxima for \(u-\varphi _1\) (i.e., \((x_1,y_1)\)) from the boundary; in the case b) we are going to detach minima for \(v-\varphi _2\) (i.e., \((x_2,y_2)\)) from the boundary; in the case c) we are going to detach the x-component of the maxima for \(u-\varphi _1\) (i.e., \(x_1\)) and the y-component of the minima for \(v-\varphi _2\) (i.e., \(y_2\)) from the boundary. We briefly treat some of the above cases.
In the following \(C>0\) is a suitable constant and \(\omega \) is a suitable modulus of continuity, whose choices are independent from \(\delta \) and \(\varepsilon \) (depending on \(\mu \) only).
Case a). We use the absurd hypothesis (22), the hypothesis on \(\zeta \), the penalizing terms involving \(\eta _X\) and \(\eta _Y\), and the hypothesis (17). Using standard estimates, we have that, at least for small \(\delta \) and \(\varepsilon \), \(\phi _a\) has a maximum point in \(({\overline{\varOmega }}_X\times {\overline{\varOmega }}_Y)\times ({\overline{\varOmega }}_X\times {\overline{\varOmega }}_Y)\), let us say \(((x_1^\varepsilon ,y_1^\varepsilon ),(x_2^\varepsilon ,y_2^\varepsilon ))\), and that
i) For small \(\varepsilon \) we have \((x_2^\varepsilon ,y_2^\varepsilon )\in {\overline{\varOmega }}_X\times \varOmega _Y\) and, if \(x_2^\varepsilon \in \partial \varOmega _X\), then \(v(x_2^\varepsilon ,y_2^\varepsilon )<\psi _X(x_2^\varepsilon ,y_2^\varepsilon )\). Hence, both equation inequalities hold in (12) and in (13) for u and v, respectively, when tested with test functions obtained from \(\phi _a\) as in (21). We then get the required contradiction, because, by the absurd hypothesis (22) and the infinitesimal estimates here above, we get the first and the fourth lines of (19), just taking \(\mu ,\delta ,\varepsilon \) sufficiently small.
The points ii), iii) and iv) are similarly treated.
b) As before, let \(((x_1^\varepsilon ,y_1^\varepsilon ),(x_2^\varepsilon ,y_2^\varepsilon ))\) be a point of maximum for \(\phi _b\). In this case, for at least small \(\delta \) and \(\varepsilon \), it is
i) The hypothesis \(v(x_0,y_0)\ge \psi _X(x_0,y_0)\) and the absurd hypothesis (22) imply \(u(x_0,y_0)>\psi _X(x_0,y_0)\) and so \(u(x_1^\varepsilon ,y_1^\varepsilon )>\psi _X(x_1^\varepsilon ,y_1^\varepsilon )\), for small \(\delta \) and \(\varepsilon \). Hence, both equation inequalities hold in (12) and in (13) for u and v, respectively, when tested with suitable test functions obtained from \(\phi _b\) as in (21). We then get the contradiction as before.
The points ii) and iii) are similarly treated.
c) Again, let \(((x_1^\varepsilon ,y_1^\varepsilon ),(x_2^\varepsilon ,y_2^\varepsilon ))\) be a point of maximum for \(\phi _c\). For at least small \(\delta \) and \(\varepsilon \), it is
The hypothesis \(v(x_0,y_0)=\psi _Y(x_0,y_0)\) and absurd hypothesis (22) imply that, for small \(\delta \) and \(\varepsilon \), \(u(x_1^\varepsilon ,y_1^\varepsilon )>\psi _Y(x_1^\varepsilon ,y_1^\varepsilon )\) and \(v(x_2^\varepsilon ,y_2^\varepsilon )<\psi _X(x_2^\varepsilon ,y_2^\varepsilon )\).
For a seek of completeness, we show here the calculation for this case. In particular, let us note that in this case we are not detaching from the boundary both points of maximum and of minimum, which is in general not possible, but instead we are detaching the first n components of the point of maximum and the second m components of the point of minimum. And this is possible because the domain \(\varOmega =\varOmega _X\times \varOmega _Y\subset {\mathbb {R}}^n\times {\mathbb {R}}^m\) is indeed a Cartesian product. By definition of \(\phi _c\) we have
where M is a bound for \(\Vert \eta _X\Vert \) and for \(\Vert \eta _Y\Vert \) (and also for |u| and |v|, in subsequent calculations). By the coercivity of \(\zeta \) and the boundedness of u and v, we get that \(\phi _c\) reaches its maximum in a point \(((x_1^\varepsilon ,y_1^\varepsilon ),(x_2^\varepsilon ,y_2^\varepsilon ))\) and that there exists two compact subsets (depending on \(\mu \)) \(K^\mu _X\subseteq {\overline{\varOmega }}_X\), \(K^\mu _Y\subseteq {\overline{\varOmega }}_Y\) such that \(((x_1^\varepsilon ,y_1^\varepsilon ),(x_2^\varepsilon ,y_2^\varepsilon ))\in (K^\mu _X\times K_Y)\times (K^\mu _X\times K_Y)\), for every \(\varepsilon \) sufficiently small. We can also suppose \(x_0+\varepsilon \eta _X(x_0)\in K^\mu _X\), \(y_0+\varepsilon \eta _Y(y_0)\in K^\mu _Y\) for all \(\varepsilon >0\) sufficiently small. Let \(\omega ^\mu \) be a modulus of continuity for both u and v and for \(\zeta \) in \((K^\mu _X\cap {\overline{\varOmega }}_X)\times (\times K^\mu _Y\cap {\overline{\varOmega }}_Y)\). We have
where \(C>0\) is a suitable constant independent from \(\varepsilon \). Now, we have the inequalities, for \(\delta \) and \(\varepsilon \) small,
From the definition of \(\phi _c\) and from (26), we get
By the boundedness of \(\omega ^\mu \), when its argument is the distance of points in \(K_X^\mu \times K_Y^\mu \), and the boundedness of \(\eta _X,\eta _Y\), from (28) we get (for another constant independent from \(\varepsilon \), and still denoted by C)
which, again by (28) and for another \(C>0\) independent from \(\varepsilon \), gives
By the inward cone hypothesis (17), from (30), in a standard way (Bardi-Capuzzo Dolcetta [5], p. 280), we get, for sufficiently small \(\delta \) and \(\varepsilon \),
Now, since we are in the case \(v(x_0,y_0)=\psi _Y(x_0,y_0)<\psi _X(x_0,y_0)\) and since \(u(x_0,y_0)>v(x_0,y_0)\), by (29)–(30) we can suppose that
This, together with (31) and the definition of \(((x_1^\varepsilon ,y_1^\varepsilon ),(x_2^\varepsilon ,y_2^\varepsilon ))\), implies that both equation inequalities hold, in \((x_1^\varepsilon ,y_1^\varepsilon )\) for u as in (12) and in \((x_2^\varepsilon ,y_2^\varepsilon )\) for v as in (13), respectively, when we take as test functions
respectively. We have
and then, if \(\mu \) is sufficiently small, we can conclude in the standard way getting the conclusion by contradiction to (18). \(\square \)
Remark 8
As already remarked, the exit cost \(\psi _{XY}\), for simultaneous exit of X and Y, does not play any role in the formulation of the Dirichlet problem (11). Indeed, it can never happen that the simultaneous exit cost \(\psi _{XY}\) is a “good” choice for both players (i.e., an equilibrium) without being already equal to \(\psi _X\) or to \(\psi _Y\) or to \(V_{int}^\delta \) for some \(\delta >0\) where the latter is defined as the lower value function restricted to controls \(\beta \) and to non-anticipating strategies \(\gamma \) which make Y and X remain inside \({\overline{\varOmega }}_Y\) and \({\overline{\varOmega }}_X\) for times in \([0,\delta ]\), respectively. For instance, let us suppose that, in a point \((x,y)\in \partial \varOmega _X\times \partial \varOmega _Y\), we have
Then, player II (the maximizing one) has certainly no interest in exit, and so the “really paid cost” is \(\psi _X\) or \(V_{int}^\delta \). A similar conclusion holds for the case
In the case that \(\psi _{XY}=V_{int}^\delta \) is a “good choice” for both players, then dynamic programming leads to the Isaacs equation and so the exit cost \(\psi _{XY}\) does not really influence the problem.
By the way, even in a strategic static minmax game where two players may independently choose to “stay” or to “exit” and the first player wants to minimize, if the utility u(exit, exit) stays between the utilities \(u(stay,exit)\le u(exit,stay)\), then the choice (exit, exit) is never a Nash equilibrium, whichever u(stay, stay) is.
Remark 9
Since the dynamics are decoupled, in order to have the classical Isaacs’ condition for the existence of a value of the game (see, for example, Bardi–Capuzzo Dolcetta [5]), we only need some further hypotheses on the running cost \(\ell \). The simplest one is that it is also decoupled with respect to controls (i.e., \(\ell (x,y,a,b)=\ell _1(x,y)+\ell _2(a)+\ell _3(b)\)) . In this case the two Hamiltonians UH and LH are the same and hence, by uniqueness of the corresponding Dirichlet problems (11), (14), \({\overline{V}}={\underline{V}}\).
7 On Constrained Non-anticipating Strategies
We give a possible construction of state-constraint non-anticipating tuning as well as non-anticipating strategies satisfying Assumption 2. We follow Soner’s [21] construction of state-constraint controls (see also Bardi–Capuzzo Dolcetta [5], pp. 272–274), modifying it in a non-anticipating way, in order to adapt such a construction to our purposes (see Remark 11).
We are now considering only point I) of Assumption 2, point II) being similar. Note that point I) is concerning with non-anticipating strategies \(\gamma \in \varGamma \) for player X. With respect to (2) (and to (1)), we are going to relax a little bit the hypotheses of decoupled dynamics and we are going to consider the following hypothesis: the dynamics f, the one for player X, is affine with respect to the controls (coherently with Bettiol–Cardaliaguet–Quincampoix [7]) and “weakly” depends on the control of Y. This means that (here, \({\tilde{f}}\) stays for the dynamics of the first player X)
where \(D \in {\mathbb {R}}^{n' \times m'}\) is a fixed constant matrix, B is as in (2), and \(f:{\mathbb {R}}^n\rightarrow {\mathbb {R}}^n\) is bounded and Lipschitz continuous. Note that, the dynamics \({\tilde{f}}\) then satisfies similar regularity hypotheses as in (2), in particular, for some \(M,L>0\) and for all \(x_1,x_2\in {\mathbb {R}}^n,a\in A,b\in B\):
The system for the trajectories of the first player is
Remark 10
We point out that here and in what follows, we are assuming such a weak decoupling for the dynamics of the player X only. The dynamics g of the player Y will be still considered decoupled, i.e., depending on the pair (y, b) only, as in (2). In this situation, we are going to prove that all the hypotheses i)–vi) of item I) only, of Assumption 2, are satisfied. The fact that the dynamics of the second player is completed decoupled, i. e. it does not depend on the control of the first player, enters in what follows because \({\tilde{\beta }}\) in (37) is constructed independently on the behavior and on the controls of the first player.
On the other hand, if we assume that the dynamics g is weakly decoupled (similarly to (32), changing the role of a and b), and we maintain the decoupled feature of f as in (2), then we can prove that item II) of Assumption 2 is satisfied.
Also note that, the validity of item I) (respectively, item II)) permits to conclude that the lower value function \({\underline{V}}\) (respectively, the upper value function \({\overline{V}}\)) is continuous. However, all the proofs of the results in Sects. 5 and 6 (as well as the simultaneous validity of both items I) and II)) hold in the case of decoupling of both f and g as in (2), that is when \(D=0\) in (32). The extension of such results to the weak decoupled case may be the subject of future works.
However, we point out that, in order to get the estimate on the costs, we are going to also assume a decoupled feature of the running cost with respect to the controls (see (48)).
Assuming (32), we need a modification of the controllability Assumption 1.
Assumption 3
Similarly, as in Assumption 1, we assume here that, for every \(x\in \partial \varOmega _X\), there exist two constant controls \(a_1,a_2\in A\) such that \({\tilde{f}}(x,a_1, b)\) is strictly entering in \(\varOmega _X\) and \({\tilde{f}}(x,a_2, b)\) is strictly entering in \({\mathbb {R}}^m{\setminus }{\overline{\varOmega }}_X\)\(\forall b \in B\).
Assuming \(C^2\)-regularity of \(\partial \varOmega _X\) (2), by Assumption 3, by the weak decoupling (32), and by the Lipschitz continuity of f, for every compact K, there exist \(\zeta >0\) and \(r>0\) and, for any \({\overline{x}} \in K\cap \partial \varOmega _X\), there exists \(a({\overline{x}})\in A\) such that, for every \(x\in B({\overline{x}},r)\cap {\overline{\varOmega }}_X\),
where \(\xi (x)\) is the inward normal unit vector to \(\varOmega _X\) at \(x\in \partial \varOmega _X\). In what follows, in view of possible future applications to thermostatically switching systems, we assume that the boundaries of \(\varOmega _X\) and \(\varOmega _Y\) are hyperplanes passing through the origin, and that \(\varOmega _X\) and \(\varOmega _Y\) are just one of the two semi-space defined by the hyperplane. In this way, the unit vector inward normal \(\xi \) is constant on \(\partial \varOmega _X\) (as well as on \(\partial \varOmega _Y\), in the sequel denoted by the same letter \(\xi \)). However, everything done here can be easily generalized to the case in which the boundaries of \(\varOmega _X\) and \(\varOmega _Y\) are finite intersections of hyperplanes not necessarily passing through the origin (see, for example, what done in Bagagiolo–Bardi [2]). Moreover, it can be extended to more general regular domains.
Now, we are going to prove that, in the situation described above, all the points i)–vi) of item I) of Assumption 2 are satisfied. In what follows, we will indicate by K a generic compact set of the form \(K=K^X\times K^Y\), where \(K^X\subseteq {\overline{\varOmega }}_X\) and \(K^Y\subseteq {\overline{\varOmega }}_Y\) are compact.
i), iii) and v). Take \(T>0\), and take \(T\ge t_Y^*>0\) to be fixed later on and \(y_1,y_2\in {K^Y}\subseteq {\overline{\varOmega }}_Y\) compact, and define
where \((r)^+=\max (r,0)\) is the positive part, and \(\xi \) is the unit internal normal to \(\varOmega _Y\). Note that, since \(\varOmega _Y\) is a semi-space, the quantity inside the suprema over \(\beta \) is just the maximal distance from \({\overline{\varOmega }}_Y\) reached by the trajectory starting from \(y_2\) with control \(\beta \), before that the trajectory starting from \(y_1\) with the same control \(\beta \) exits from \({\overline{\varOmega }}_Y\), or the time \(t^*_Y\) is reached. However, the presence of the supremum over \(\beta \) makes \(\varepsilon _Y\) independent from \(\beta \), and this is the essential feature for the fact that next formula (37) defines a non-anticipating tuning, as we are going to explain in the comments after (37) and in Remark 11.
Note that we have the estimate (with C depending only on T, K, and \(t_Y^*\))
Indeed, for every \(\beta \) and for every \(0\le t\le \min \{\tau _Y(y_1,\beta ),t_Y^*\}\), we have \(Y(t;y_1,\beta )\in {\overline{\varOmega }}_Y\), i.e., \(\xi \cdot Y(t;y_1,\beta )\ge 0\). Then there exists a constant \(C(t_Y^*)\) such that
and hence, (35) holds.
Now, take \(\beta \in {\mathcal {B}}\), let \(t_{0_Y}\ge 0\) be the first time the trajectory \(Y(\cdot ):=Y(\cdot ;y_2,\beta )\) hits the boundary \(\partial \varOmega _Y\) and let \(b_0\in B\) be such that \(g(Y(t_{0_Y}),b_0)\) strictly enters in \(\varOmega _Y\) at \(Y(t_{0_Y})\): the one given by Assumption 1. Now, let us take \(k_Y>0\) and define the measurable control \({\tilde{\beta }}\in {\mathcal {B}}\) as
By our definition of \(\varepsilon _Y\) (34) which is independent on \(\beta \), due to the presence of the supremum over the controls, the construction in (37) is a non-anticipating tuning (see Definition 1, see also Remark 11 for other comments). This means that, whenever \(\beta _1=\beta _2\) in [0, t] a.e., then also \({\tilde{\beta }}_1={\tilde{\beta }}_2\) in [0, t] a.e., in other words i) holds. Indeed, in such a case, in the time interval [0, t], \(\beta _1\) and \(\beta _2\) generate the same trajectory Y starting from \(y_2\). If, in the time interval [0, t] the trajectory Y does not hit the boundary, then, by definition (25), \({\tilde{\beta }}_1=\beta _1=\beta _2={\tilde{\beta }}_2\) in [0, t]. If instead the trajectory Y hits the boundary at \(t_{0_Y}\le t\), then in the interval \([0,t_{0_Y}[\) we still have the equality \({\tilde{\beta }}_1={\tilde{\beta }}_2\), and in the time interval \([t_{0_Y},t_{0_Y}+k_Y\varepsilon _Y]\cap [0,t]\) (whose length by (34) is independent from \(\beta _1\) and \(\beta _2\)), we have \({\tilde{\beta }}_1=b_0={\tilde{\beta }}_2\), with \(b_0\) given by Assumption 1 in \(Y(t_{0_Y})\), which is the same for both controls. Finally, in the (possibly empty) time interval \([0,t]{\setminus }[t_{0_Y},t_{0_Y}+k_Y\varepsilon _Y]\) we have \({\tilde{\beta }}_1(s)=\beta _1(s-k_Y\varepsilon _Y)=\beta _2(s-k_Y\varepsilon _Y)={\tilde{\beta }}_2(s)\).
Now we want to suitably choose \(t_Y^*\) and \(k_Y\) such that, for every \(t\in [0,t_Y^*]\), it is \(Y(t;y_2,{\tilde{\beta }})\in {\overline{\varOmega }}_Y\), at least for \(t\le \tau _Y(y_1,\beta )\). This can be done (independently on \(\beta \)) just following Bardi–Capuzzo Dolcetta [5] p. 273, with \(\varepsilon \) given by our \(\varepsilon _Y\) (a more detailed construction is given for the similar question in the next point). Repeating the construction for every needed time interval \([nt_Y^*,(n+1)t_Y^*]\), in order to cover the interval \([0,{\tilde{\tau }}]\), we get iii). Finally, using (35), we also get v).
Remark 11
Observe that (34) is different from the one defined by Soner [21] (see also Bardi–Capuzzo Dolcetta [5], p. 273) since here we are building non-anticipating tuning as well as non-anticipating strategies, and this feature is guaranteed by the supremum over \(\beta \) in definition (34). Indeed, in our framework and notations, if we just follow what done in [21], we would have, for every control \(\beta \),
This means that, even if \(\beta _1=\beta _2\) in [0, t], then they may generate different values of \(\varepsilon (\beta _1)\) and \(\varepsilon (\beta _2)\) because they, and their corresponding trajectories, may differ after the time t. In particular, it may happen that the trajectories hit the boundary at the same instant \(0\le t_{0_{Y}}<\min \{\tau _Y(y_1,\beta _1),\tau _Y(y_1,\beta _2),t^*_Y,t\}\) and that \([t_{0_Y},t_{0_Y}+k_Y\varepsilon (\beta _1)]\subset [t_{0_Y},t_{0_Y}+k_Y\varepsilon (\beta _2)]\subset [0,t]\). Applying (37), we would get two different behavior of \({\tilde{\beta }}_1\) and \({\tilde{\beta }}_2\) in [0, t], that is an anticipating construction: (37) is not more a non-anticipating tuning (see Definition 1). Considering instead the supremum over all controls \(\beta \), as we do in (34), makes us to avoid this behavior because the length of \(\varepsilon _Y\) does not depend by the single control \(\beta \).
Of course, as already said in the Introduction, in [21] and [5], the non-anticipating structure is not taken into consideration because the reference problem is an optimal control problem. Also observe that the construction (37) is exactly the same as in [21] and [5]. But here, the non-anticipating feature is given by the definition of \(\varepsilon _Y\), which is independent on the single control.
Finally, observe that we may have \(\varepsilon _Y=0\). By its very definition (22) (do not consider here \(t^*_Y\)), \(\varepsilon _Y=0\) means that, whatever the control \(\beta \) is, the trajectory \(Y(\cdot ;y_2,\beta )\) does not exit from \({\overline{\varOmega }}_Y\) before the trajectory \(Y(\cdot ;y_1,\beta )\) exits from \({\overline{\varOmega }}_Y\). And this is exactly what we need in our proof of continuity of the value function (i. e. requirement iii) of Assumption 2). Hence, in this case, we do not need to modify the control: in (37), for every control \(\beta \), it is \({\tilde{\beta }}=\beta \), coherently with the fact that the time interval of length \(k_Y\varepsilon _Y\) (where we should make the modification of \(\beta \)) is just a point (not a true interval).
ii) and iv). Take \(T>0\), and take \(T\ge t_X^*\) to be fixed later on, \(x_1, x_2 \in {K^X}\subseteq {\overline{\varOmega }}_X\) compact and define, similarly as for \(\varepsilon _Y\),
where \(\xi \) is the unit internal normal to \(\varOmega _X\), and \({\tilde{\beta }}\) is defined as in (37), with respect to the previously fixed \(y_1,y_2\in K^Y\). In (38) we use both the supremum over \(\beta \) and over \(\gamma \) in order to build non-anticipating strategies. Note that now, in the notations of the trajectory X and of the exit time \(\tau _X\) we are taking account that the dynamics f is only weakly decoupled (32). In particular, in this case, the estimates ii) and iv) of Assumption 2 must be replaced by (see also point 10) of the proof of Proposition 3)
We refer to Remark 12 for comments on the simpler case of strongly decoupled dynamics f.
As before, inside the suprema in (38), the scalar product is the distance from the semi-space \({\overline{\varOmega }}_X\). In this case we have (with C depending only on T, K, \(t_X^*\) and \(t_Y^*\))
Indeed, for t as in (38) and using (35),
and we conclude by the Gronwall estimate. Note that, in the estimate of the integral of \(D(\beta -{\tilde{\beta }})\), we have used the equality
Now, take \(\gamma \in \varGamma \), and we want to construct the strategy \({\tilde{\gamma }}\) acting on \({\tilde{\beta }}\) (see (39)). Take \(\beta \in {{{\mathcal {B}}}}\) and let \(t_{0_X}\ge 0\) be the first time the trajectory \(X(\cdot ;x_1,\gamma [{{\tilde{\beta }}}], \beta )\) hits the boundary \(\partial \varOmega _X\) and let \(a_0\in A\) be such that f strictly enters in \(\varOmega _X\) as in Assumption 3. Now, let us take \(k_X>0\) and define \(\tilde{{\tilde{\gamma }}}:{{\mathcal {B}}}\rightarrow {{\mathcal {A}}}\) as
Being \(\gamma \) and the tuning \(\beta \mapsto {\tilde{\beta }}\) non-anticipating, by the definition of \(\varepsilon _X\) we get that \({\tilde{{\tilde{\gamma }}}}\) is also non-anticipating. We use the following notations, which also give notational coherence with (38) and (39),
As before we want to suitably choose \(t_X^*\) and \(k_X\) such that, for every \(t \in [0, t_X^*]\), it is \({\tilde{X}}(t)\in {\overline{\varOmega }}_X\). Of course we are interested in the case \(t\le \tau _X(x_2,\gamma [{{\tilde{\beta }}}],{{\tilde{\beta }}})\). We then prove that
Again, we follow [5] but, due to presence of both controls (32), we now show some explicit calculations. Note that if \(\min \{t_{0_X},t_X^*\}=t_X^*\) then \({\tilde{\gamma }}[{\tilde{\beta }}]=\gamma [{{\tilde{\beta }}}]\) and \(X(t)\in {\overline{\varOmega }}_X\). If instead \(\min \{t_{0_X},t_X^*\}=t_{0_X}\) then, for \(0\le t\le t_{0_X}\), \( X(t)\in {\overline{\varOmega }}_X\). We consider only the case (the other one being easier) \(t_{0_X}+k_X\varepsilon _X\le t\le t_X^*\). Since \(X(t_{0_X})={\tilde{X}}(t_{0_X})\in \partial \varOmega _X\) and so \(X(t_{0_X})\cdot \xi ={\tilde{X}}(t_{0_X})\cdot \xi =0\), we have
We estimate the first integral in (43) using (33). Indeed, we first assume \(t^*_X\) small enough such that \(\Vert {\tilde{X}}(s)-X(t_{0_X})\Vert <r\) as in (33), for all \(s\in [t_{0_X},t^*_X]\), which is possible, independently on the controls and on the points in K, because the dynamics are bounded. Here \(\zeta \) and r are as in (33) with respect to \(K'\), which is a compact set such that any trajectory starting from a point of \(K^X\) does not exit from \(K'\) in the time interval [0, T].
The second integral in (43) is estimated as
where in the last inequality (for suitable \(M,L>0\) depending on f, A, B and D) we have used the Lipschitz continuity of f, standard estimates on trajectories (coming from Gronwall inequality), and the fact that the dynamics is affine in the controls (32). In particular, we have used the following estimate for \(s\ge t_{0_X}\)
using also the equality
Adding (44) to (45), and using the definition of \(\varepsilon _X\), we get
where \({\tilde{C}}\) is an upper bound for \(Db\cdot \xi \) and \(\zeta -{\tilde{C}}>0\) by (33). Consequently, if \(t^*_X\) is sufficiently small, \({\tilde{X}}(t)\cdot \xi \ge \displaystyle \frac{(\zeta -{\tilde{C}}) k_X\varepsilon _X}{2}-\varepsilon _X\), and hence, taking \(k_X:=2/(\zeta -{\tilde{C}})\), we obtain \(\xi \cdot {\tilde{X}}(t)\ge 0\). This proves ii), and iv) is proven in a standard way using (40).
Remark 12
Note that \(t^*_X\) (as well as \(t^*_Y\)) only depends, besides the dynamics, on \(T>0\) and on the compact set K (via the compact set \(K'\) such that, any trajectory starting from K, does not exit from \(K'\) in the time interval [0, T]) and not on controls and non-anticipating strategy in use as well as not on the chosen initial points inside K. Hence, repeating, if necessary, the procedure a finite number of time, we can cover the whole interval \([0,{\tilde{\tau }}]\subseteq [0,T]\) and obtain the estimates in Assumption 2, where the modulus of continuity \({{{\mathcal {O}}}}_{T,K}\) only depends on T and K. For example, after the first interval \([0,t^*_X]\), we can consider the points \(X(t^*_X)\) and \({\tilde{X}}(t^*_X)\) and repeat all the construction with those points as initial points (and note that, starting from them, in the time interval \([t^*_X,T]\) we do not exit from \(K'\)).
We point out once again that the problem is an exit time problem, and hence, when a player firstly exits from its domain, the game stops and what happens after that moment is not influencing anymore. This is the reason why the constructed non-anticipating strategies, even if they map controls on \([0,+\infty [\) to controls on \([0,+\infty [\), are mainly constructed looking to what happens up to the exit time only. Indeed, after the exit time, controls and strategies may be arbitrarily defined, for example, in any a-priori constant manner (which is obviously non-anticipating). Indeed, the cost J does not change if we take controls that coincide up to the exit time and possibly differ from the exit time on, since it only depends on controls and strategies used up to exit time. In particular, a similar situation is in Definitions (7) and (9), where, after the exit time (of the homologous trajectory), the controls are defined in a constant manner (in that case, in a suitable constant manner, using the outward-pointing control).
In the case when the dynamics f is also strongly decoupled, as in (2) and in the rest of the paper, the definition of \(\varepsilon _X\) (38) is simply replaced by
the definition of \(\tilde{{\tilde{\gamma }}}\) (42) is replaced by
in place of (39) we maintain the corresponding ones ii) and iv) of Assumption 2, and finally, (40) turns out as depending only on \(\Vert x_1-x_2\Vert \).
We finally point out the obvious fact that the estimates iv) and v) of Assumption 2, as well as iv’) in (39), also hold for all \(0\le t\le {\tilde{\tau }}\).
vi) Now we assume that the dependence of the running cost \(\ell \) on the controls is separated, that is
where \(\ell _1,\ell _2\) are continuous, bounded, and Lipschitz continuous in (x, y) uniformly with respect to a and b, respectively. Note that a similar separated feature is also assumed in Bettiol–Cardaliaguet–Quincampoix [7].
We have to estimate
where \({\tilde{\tau }}=\min (\tau _X(x_2,\gamma [{\tilde{\beta }}],{\tilde{\beta }}),\tau _Y(y_1,\beta ),T)\) and \(J_{{\tilde{\tau }}}\) is the integral of the discounted running cost up to time \({\tilde{\tau }}\). We will sketch the computation in the case in which \({\tilde{\tau }}=T\ge t^*_Y\ge t_{0_X}+k_X\varepsilon _X\ge t_{0_Y}+k_Y\varepsilon _Y\ge t_{0_X}\ge t_{0_Y}\), since the other cases are similar. The quantity in (49) is majorized by
where we used the expressions of \({\tilde{\beta }}\) (37) and \({\tilde{\gamma }}[{\tilde{\beta }}]\) (see (42) and four lines below it)) and dropped the notations of the controls in the trajectories \(X(\cdot ;x_1)=X(\cdot ;x_1,{\tilde{\gamma }}[{\tilde{\beta }}],\beta )\), \(Y(\cdot ;y_1)=Y(\cdot ;y_1,\beta )\), \(X(\cdot ;x_2)=X(\cdot ;x_2,\gamma [{\tilde{\beta }}],{\tilde{\beta }})\), \(Y(\cdot ;y_2)=Y(\cdot ;y_2,{\tilde{\beta }})\). Note that we passed the absolute value under the integral sign in the first four integrals only. We are going to use the boundedness and the Lipschitz continuity of \(\ell \) (and of \(\ell _1,\ell _2\)), the estimates (35), (40), and standard estimates on trajectories.
The first integral in (50) is majorized by standard procedure (the difference of the running costs inside the integral is evaluated in the same control values). The other three integrals can be easily estimated since the time interval size is small (it is less than \(k_X\varepsilon _X\) or than \(k_Y\varepsilon _Y\)), while, using (48), the last integral is majorized by
Now, using the fact that the dynamics are bounded, the fact that the running costs and the function \(t\mapsto e^{-\lambda t}\) are bounded and Lipschitz continuous, and the estimates v) of Assumption 2 and iv’) of (39) (see also Remark 12), arguing by a change of variable as in the previous subsection (see, for example, (41), (46), and also see Bardi–Capuzzo Dolcetta [5] p. 274), the first integral in (51) is estimated by (here \({\tilde{M}}\) depends on L, the Lipschitz constant of costs and dynamics, on M, the bound of costs and dynamics, and on \(\lambda \), the discount factor)
where in the last inequality we have used (40), and the last \({{{\mathcal {O}}}}_{T,K}\) is an infinitesimal function, sum of the infinitesimal functions in the line before. We similarly estimate the second integral in (51).
Again, the estimate only depends on T and K, being independent on controls, strategies, and starting points in K. Hence, possibly repeating such procedure a finite number of times, we get the estimate vi) of Assumption 2.
Finally, we point out that hypothesis (48) is used, here above, in order to separately treat the change of variable in the controls. The extension to the case of non-decoupled cost \(\ell (x,y,a,b)\) seems to be not obvious. However, we think that (48) may be probably amended by suitably modifying (37) and (42), but we did not check any details.
References
Bagagiolo F (2007) Minimum time for a hybrid system with thermostatic switchings. In: Bemporad A, Bicchi A, Buttazzo G (eds) Hybrid systems: computation and control, vol 4416. Lecture notes in computer sciences. Springer, Berlin, pp 32–45
Bagagiolo F, Bardi M (1998) Singular perturbation of a finite horizon problem with state-space constraints. SIAM J Control Optim 36:2040–2060
Bagagiolo F, Danieli K (2012) Infinite horizon optimal control problems with multiple thermostatic hybrid dynamics. Nonlinear Anal Hybrid Syst 6:824–838
Bagagiolo F, Maggistro R (2019) Hybrid thermostatic approximations of junctions for some optimal control problems on networks. SIAM J Control Optim 57:2415–2442
Bardi M, Capuzzo Dolcetta I (1997) Optimal control and viscosity solutions of Hamilton–Jacobi–Bellman equations. Birkhäuser, Boston
Bardi M, Koike S, Soravia P (2000) Pursuit-evasion games with state constraints: dynamic programming and discrete-time approximations. Discrete Contin Dyn Syst 6:361–380
Bettiol P, Cardaliaguet P, Quincampoix M (2006) Zero-sum state constraint differential game: existence of a value for Bolza problem. Int J Game Theory 34:495–527
Bettiol P, Bressan A, Vinter R (2011) Estimates for trajectories confined to a cone in \({\mathbb{R}}^n\). SIAM J Control Optim 49:21–41
Bettiol P, Bressan A, Vinter R (2010) On trajectories satisfying a state constraint: \(W^{1,1}\) estimates and counterexamples. SIAM J Control Optim 48:4664–4679
Bettiol P, Facchi G (2014) Linear estimates for trajectories of state-constrained differential inclusions and normality conditions in optimal control. J Math Anal Appl 414:914–933
Bettiol P, Frankowska H, Vinter R (2015) Improved sensitivity relations in state constrained optimal control. Appl Math Optim 71:353–377
Bressan A, Facchi G (2011) Trajectories of differential inclusions with state constraints. J Differ Equ 250:2267–2281
Buckdahn R, Cardaliaguet P, Quincampoix M (2011) Some recent aspects of differential game theory. Dyn Games Appl 1:74–114
Cardaliaguet P, Quincampoix M, Saint-Pierre P (2000) Pursuit differential games with state constraints. SIAM J Control Optim 39:1615–1632
Dupuis P, McEneaney WM (1997) Risk-sensitive and robust escape criteria. SIAM J Control Optim 35:2021–2049
Elliot RJ, Kalton NJ (1972) The existence of value in differential games, vol 126. Memoirs of American Mathematical Society. AMS, Providence
Falugi P, Kountouriotis PA, Vinter R (2012) Differential games controllers that confine a system to a safe region in the state space, with applications to surge tank control. IEEE Trans Autom Control 57:2778–2788
Frankowska H, Marchini EM, Mazzola M (2016) A relaxation result for state constrained inclusions in infinite dimension. Math Control Relat Fields 6:113–141
Gromov D, Gromova E (2016) On a class of hybrid differential games. Dyn Games Appl 7:266–288
Koike S (1995) On the state constraint problem for differential games. Indiana Univ Math J 44:467–487
Soner HM (1986) Optimal control problems with state-space constraints I. SIAM J Control Optim 24:552–561
Vinter RB, Clark JMC (2003) A differential games approach to flow control. In: Proceedings of the 42nd IEEE conference on decision and control Maui, Hawaii, USA
Acknowledgements
This work was partially supported by the Italian INDAM-GNAMPA project 2017.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Bagagiolo, F., Maggistro, R. & Zoppello, M. A Differential Game with Exit Costs. Dyn Games Appl 10, 297–327 (2020). https://doi.org/10.1007/s13235-019-00334-y
Published:
Issue Date:
DOI: https://doi.org/10.1007/s13235-019-00334-y
Keywords
- Differential games
- Exit costs
- Dirichlet problems for Hamilton–Jacobi–Isaacs equations
- Viscosity solutions
- Uniqueness
- Non-anticipating strategies