Hamilton–Jacobi–Bellman Equations on Multi-domains

Abstract

A system of Hamilton–Jacobi (HJ) equations on a partition of $\mathbb {R}^{d}$ is considered, and a uniqueness and existence result of viscosity solution is analyzed. While the notion of viscosity solution is by now well known, the question of uniqueness of solution, when the Hamiltonian is discontinuous, remains an important issue. A uniqueness result has been derived for a class of problems, where the behavior of the solution, in the region of discontinuity of the Hamiltonian, is assumed to be irrelevant and can be ignored (see (Camilli, Siconolfi in Adv. Differ. Equ. 8(6):733–768, 2003)). Here, we provide a new uniqueness result for a more general class of Hamilton–Jacobi equations.

The work has been partially supported by the European Union under the 7th Framework Programme “FP7-PEOPLE-2010-ITN”, GA number 264735-SADCO.

1 Introduction

The present work aims at investigating a system of Hamilton–Jacobi–Bellman equations on multi-domains. Consider the repartition of $\mathbb {R}^{d}$ by disjoint subdomains (Ω _i)_i=1,…,m with

$$\mathbb {R}^d=\overline{\varOmega }_1\cup\cdots\cup\overline{ \varOmega }_m, \qquad\varOmega_i\cap\varOmega_j= \emptyset\quad \mbox{for }i\neq j. $$

Consider a collection of Hamilton–Jacobi–Bellman (HJB) equations

$$ \left\{\begin{array}{l@{\quad }l} -\partial_t u(t,x)+H_i(x,Du(t,x))=0, & \mbox{for}\ t\in(0,T),\ x\in \varOmega _i,\\[2pt] u(T,x)=\varphi (x), & \mbox{for}\ x\in \varOmega _i, \end{array}\right. $$

(1.1)

with the different Hamiltonians H _i satisfying standard assumptions, and where $\phi:\mathbb {R}^{d}\to \mathbb {R}$ is a Lipschitz continuous function. We address the question to know what condition should be considered on the interfaces (i.e., the intersections of the sets $\overline{\varOmega}_{i}$) in order to get the existence and uniqueness of solution, and also what should be the precise notion of solution.

In order to identify a global solution satisfying (1.1) on each subdomain Ω _i, one can define a global HJB equation with the Hamiltonian H defined on the whole $\mathbb {R}^{d}$ with H(x,p)=H _i(x,p) whenever x∈Ω _i. However, H can not be expected to be continuous and the definition of H on the interfaces between the subdomains Ω _i is not clear.

The viscosity notion has been introduced by Crandall–Lions to give a precise meaning to the HJ equations with continuous Hamiltonians. This notion has been extended to the discontinuous case by Ishii (see [14]), and later to the case where the Hamiltonian is measurable with respect to the space variable (see [7]). The main difficulty remains the uniqueness of viscosity solution when the Hamiltonian is not continuous.

In [16], a stationary HJ equations with discontinuous Lagrangian have been studied where the Hamiltonian is the type of H(x,p)+g(x) with continuous H and discontinuous g. A uniqueness result is proved under rather restrictive assumptions on g. In [7], the viscosity notion has been extended for the HJ equations with space-measurable Hamiltonians, and a uniqueness result has been established under a transversality assumption. Roughly speaking, this transversality condition amounts saying that the behavior of the solution on the interfaces is not relevant and can be ignored. In the present work, we will consider some more general situations where the transversality condition may not be satisfied. Our aim is to derive some junction conditions that have to be considered on the interfaces in order to guarantee the existence and uniqueness of the viscosity solution of (1.1).

Let us mention that the first work dealing with the case where the whole space is separated into two subdomains by one interface has been studied in [4]. In the context, general results on the viscosity sense and uniqueness of solution are analyzed. Even though the problem in [4] considers the problem of a steady equation, the paper shares the same difficulty as the ones we are presenting here. In the present work, our approach is completely different from the one used in [4] and seems to be easy to generalize for two or multi-domains problems. Other papers related to the topic of HJB equations with discontinuous Hamiltonians are [1, 13], where some HJB equations are studied on networks (union of a finite number of half-lines with a single common point), motivated by some traffic flow problems. An inspiring result is the strong comparison principle of [13] leading to the uniqueness result by considering a HJ equation on the junction point.

In the present work, we will investigate the junction conditions on the interfaces. For this, by using the Filippov regularization of the multifunctions F _i, we shall introduce a particular optimal control problem on $\mathbb {R}^{d}$. The main feature of this control problem is that its value function is solution to the system of (1.1). By investigating the transmission conditions satisfied by the value function on the interfaces between the subdomains Ω _i, we obtain the equations which are defined on the interfaces. Then the system (1.1) is completed by these equations on the interfaces and the existence and uniqueness of solution is guaranteed. No transversality requirement is needed in this paper. The main idea developed here follows the concept of Essential Hamiltonian introduced in [5], and provides a new viscosity notion that is quite different from the notion of Ishii [14]. This new definition gives a precise meaning to the transmission conditions between Ω _i and provides the uniqueness of viscosity solution.

The paper is organized as follows. In Sect. 2, the setting of the problem is described and the main results are presented. Section 3 is devoted to the link with optimal control problem and the study of the properties of the value function, and the proofs for the main results are given in Sect. 4.

2 Main Results

2.1 Setting of the Problem

Consider the following structure on $\mathbb {R}^{d}$: given $m\in \mathbb {N}$, let {Ω ₁,…,Ω _m} be a finite collection of C ² open d-manifolds embedded in $\mathbb {R}^{d}$. For each i=1,…,m, the closure of Ω _i is denoted as $\overline{\varOmega }_{i}$. Assume that this collection of manifolds satisfies the following:

(H1) :

$\displaystyle \begin{cases} \mbox{(i)}&\mathbb {R}^d=\bigcup_{i=1}^m \overline{\varOmega }_i\quad \mbox{and}\quad \varOmega _i\cap \varOmega _j=\emptyset\quad \mbox{when }i\not= j,\ i,j\in\{1,\ldots, m\}; \\ \mbox{(ii)} &\mbox{Each}\ \overline{\varOmega }_i\mbox{ is proximally smooth and wedged.} \\ \end{cases}$

The concepts of proximally smooth and wedged are introduced in [9]. For any set $\varOmega \subseteq \mathbb {R}^{d}$, we recall that $\overline{\varOmega }$ is proximally smooth means that the signed distance function to $\overline{\varOmega }$ is differentiable on a tube neighborhood of $\overline {\varOmega }$. $\overline{\varOmega }$ is said to be wedged means that the interior of the tangent cone of $\overline{\varOmega }$ at each point of $\overline{\varOmega }$ is nonempty. The precise definitions and properties are presented in Appendix B.

Let $\varphi :\mathbb {R}^{d}\rightarrow \mathbb {R}$ be a given function satisfying:

(H2) :

φ is a bounded Lipschitz continuous function.

Let T>0 be a given final time, for i=1,…,m, consider the following system of Hamilton–Jacobi (HJ) equations:

$$ \begin{cases}\displaystyle -\partial_t u(t,x)+H_i(x,Du(t,x))=0, & \mbox{for}\ t\in(0,T),\ x\in \varOmega _i,\\ u(T,x)=\varphi (x), & \mbox{for}\ x\in \varOmega _i. \end{cases} $$

(2.1)

The system above implies that on each d-manifold Ω _i, a classical HJ equation is considered. However, there is no information on the boundaries of the d-manifolds which are the junctions between Ω _i. We then address the question to know what condition should be considered on the boundaries in order to get the existence and uniqueness of solution to all the equations.

In the sequel, we call the singular subdomains contained in the boundaries of the d-manifolds the interfaces. Let $\ell\in \mathbb {N}$ be the number of the interfaces and we denote Γ _j, j=1,…,ℓ the interfaces which are also open embedded manifolds with dimensions strictly smaller than d. Assume that the interfaces satisfy the following:

(H3) :

$\displaystyle \begin{cases} \mbox{(i)}\hspace*{-4pt} &\mathbb {R}^d= (\bigcup_{i=1}^m \varOmega _i )\cup(\bigcup_{j=1}^{\ell} \varGamma_j ),\ \ \varGamma_j\cap\varGamma_k=\emptyset,\ \ j\not= k,\ j,k=1,\ldots ,\ell; \\ \mbox{(ii)}\hspace*{-4pt} &\mbox{If } \varGamma_j\cap\overline{\varOmega }_i\neq\emptyset, \ \ \mbox{then }\varGamma_j\subseteq\overline{\varOmega }_i,\ \ \mbox{for}\ i=1,\ldots , m,\ j=1,\ldots,\ell; \\ \mbox{(iii)}\hspace*{-4pt} &\mbox{If } \varGamma_k\cap\overline{\varGamma}_j\neq\emptyset ,\ \ \mbox{then }\varGamma_k\subseteq\overline{\varGamma}_j,\ \ \mbox{for}\ j,k\in \{1,\ldots,\ell\}; \\ \mbox{(iv)}\hspace*{-4pt} &\mbox{Each }\overline{\varGamma}_j\mbox{ is proximally smooth and relatively wedged.} \end{cases}$

For any open embedded manifold Γ with dimension p<d, $\overline{\varGamma}$ is said to be relatively wedged if the relative interior (in $\mathbb {R}^{p}$) of the tangent cone of $\overline{\varGamma}$ at each point of $\overline{\varGamma}$ is nonempty, see Appendix B for the precise definition.

Example 1

A simple example is shown in Fig. 1 with d=1, m=2 and ℓ=1. Here $\mathbb {R}=\varOmega _{1}\cup\varGamma_{1}\cup \varOmega _{2}$ with

$$\varOmega _1=\{x:x<0\},\qquad \varOmega _2=\{x:x>0\},\qquad \varGamma_1=\{0\}. $$

Note that Ω ₁, Ω ₂ are two one dimensional manifolds, and the only interface is the zero dimensional manifold Γ ₁.

Fig. 1

A multi-domain in 1d

Other possible examples in $\mathbb {R}^{2}$ are depicted in Fig. 2.

Fig. 2

Other possible examples in $\mathbb {R}^{2}$

We are interested particularly in the HJ equations with the Hamiltonians $H_{i}:\overline{\varOmega }_{i}\times \mathbb {R}^{d}\rightarrow \mathbb {R}$, i=1,…,m of the following Bellman form: for $(x,q)\in\overline{\varOmega }_{i}\times \mathbb {R}^{d}$,

$$ H_i(x,q)=\sup_{p\in F_i(x)}\{-p\cdot q\}, $$

where $F_{i}:\overline{\varOmega }_{i}\rightsquigarrow \mathbb {R}^{d}$ are multifunctions defined on $\overline{\varOmega }_{i}$ and satisfy the following assumptions:

(H4) :

$\displaystyle \begin{cases} {\mbox{(i)}} &\forall x\in\overline{\varOmega }_i,\quad F_i(x) \mbox{ is a nonempty, convex, and compact set;} \\ {\mbox{(ii)}} & F_i \mbox{ is Lipschitz continuous on }\varOmega _i\mbox{ with respect to the}\\&\mbox{Hausdorff metric;} \\ {\mbox{(iii)}} &\exists \mu>0 \mbox{ so that}\quad \max\{|p|:p\in F_i(x)\} \leq\mu(1+\|x\|) \forall x\in\overline{\varOmega }_i; \\ {\mbox{(iv)}} & \exists \delta>0 \mbox{ so that}\quad \forall x\in \overline{\varOmega }_i,\ \delta\overline{B(0,1)}\subseteq F_i(x). \end{cases}$

The hypothesis (H4)(i)–(iii) are classical for the study of HJB equations, whereas (H4)(iv) is a strong controllability assumption. Although this controllability assumption is restrictive, we use it here in order to ensure the continuity of solutions for the system (2.1). The continuity property plays an important role in our analysis, but it can be obtained under weaker assumption than (H4)(iv), see [15].

Remark 2.1

For the simplicity, we define the multifunction F _i on $\overline{\varOmega }_{i}$. In fact, if F _i is only defined on Ω _i and satisfies (H4), it can be extended to the whole $\overline{\varOmega }_{i}$ by its local Lipschitz continuity.

2.2 Essential Hamiltonian

The main goal of this work is to identify the junction conditions that ensure the uniqueness of the solution for the HJ system (2.1). In [7], the uniqueness of the solution of space-measurable HJ equations has been studied under some special conditions, called “transversality” conditions. Roughly speaking, this transversality condition would mean, in the case of problem (2.1), that the interfaces can be ignored and the behavior of the solution on the interfaces is not relevant. Here we consider the case when no transversality condition is assumed and we analyze the behavior of the solution on the interfaces.

First of all, in order to define a multifunction on the whole $\mathbb {R}^{d}$, an immediate idea is to consider the approach of Filippov regularization [11] of (F _i)_i=1,…,m. For this consider the multifunction $G:\mathbb {R}^{d}\rightsquigarrow \mathbb {R}^{d}$ given by:

$$\forall x\in \mathbb {R}^d,\quad G(x):=\mathrm{co} \bigl\{F_i(x): i\in\{1,\ldots,m\}, x\in\overline{\varOmega }_i\bigr\}. $$

G is the smallest upper semi-continuous (usc) envelope of (F _i)_i=1,…,m such that G(x)=F _i(x) for x∈Ω _i. Consider the Hamiltonian associated to G:

$$H_G(x,q)=\sup_{p\in G(x)}\{-p\cdot q\}. $$

If H _G(⋅,q) is Lipschitz continuous, then one could define the HJB equations on the interfaces with the Hamiltonian H _G and the uniqueness result would follow from the classical theory. However, G is not necessarily Lipschitz continuous and the characterization by means of HJB equations is not valid, see [10].

The next step is to define the multifunctions on the interfaces Γ _j. We first recall the notion of tangent cone. For any C ² smooth $\mathcal {C}\subseteq \mathbb {R}^{p}$ with 1≤p≤d, the tangent cone $\mathcal {T}_{\mathcal {C}}(x)$ at $x\in \mathcal {C}$ is defined as

$$\mathcal {T}_{\mathcal {C}}(x)=\biggl\{v\in \mathbb {R}^p:\mathop{\lim \inf}\limits _{t\rightarrow 0^+} \frac{d_{\mathcal {C}}(x+tv)}{t}=0\biggr\}, $$

where $d_{\mathcal {C}}(\cdot)$ is the distance function to $\mathcal {C}$. For j=1,…,ℓ, we define the multifunction $\widetilde {G}_{j}:\varGamma_{j}\rightsquigarrow \mathbb {R}^{d}$ on the interface Γ _j by

$$\forall x\in\varGamma_j,\quad \widetilde {G}_j(x):=G(x)\cap \mathcal {T}_{\varGamma_j}(x). $$

Note that $\mathcal {T}_{\varGamma_{j}}(x)$ agrees with the tangent space of Γ _j at x, and the dimension of $\mathcal {T}_{\varGamma_{j}}(x)$ is strictly smaller than d. On $\widetilde {G}_{j}$ we have the following regularity result for which the proof is postponed to Appendix A.

Lemma 2.2

Under the assumptions (H1), (H2) and (H4), $\widetilde {G}_{j}(\cdot):\varGamma_{j}\rightsquigarrow \mathbb {R}^{d}$is locally Lipschitz continuous onΓ _j.

Through this paper, and for the sake of simplicity of the notations, for k=1,…,m+ℓ we set

$$\mathcal {M}_k= \begin{cases}\displaystyle \varOmega _k, & \mbox{for}\ k=1,\ldots,m;\\ \varGamma_{k-m}, & \mbox{for}\ k=m+1,\ldots,m+\ell, \end{cases} $$

and we define a new multifunction $F^{\mathit {new}}:\mathbb {R}^{n}\rightsquigarrow \mathbb {R}^{n}$ by

$$F^{\mathit {new}}_k(x):= \begin{cases}\displaystyle F_k(x) & \mbox{for}\ x\in \mathcal {M}_k,\ k=1,\ldots,m;\\ \widetilde {G}_{k-m}, & \mbox{for}\ x\in \mathcal {M}_k,\ k=m+1,\ldots,m+\ell. \end{cases} $$

In all the sequel, we will also need the “essential multifunction” F ^E which will be used in the junction conditions:

Definition 2.3

(The essential multifunction)

The essential multifunction $F^{E}:\mathbb {R}^{d}\rightsquigarrow \mathbb {R}^{d}$ is defined by

$$F^{E}(x):= \bigcup_{k\in\{1,\ldots,m+\ell\}} \bigl\{F^{E}_k(x): x\in\overline{\mathcal {M}}_k \bigr\},\quad \forall x\in \mathbb {R}^d, $$

where $F^{E}_{k}:\overline{\mathcal {M}}_{k}\rightsquigarrow \mathbb {R}^{d}$ is defined by

$$F^{E}_k(x)=F^{\mathit {new}}_k(x)\cap \mathcal {T}_{\overline{\mathcal {M}}_k}(x),\quad \mbox{\textit{for}} \ x\in\overline{\mathcal {M}}_k. $$

F ^E is called essential velocity multifunction in [5]. According to the definition, F ^E(x) is the union of the corresponding inward and tangent directions to each subdomain near x. We note that

$$F^{E}|_{\mathcal {M}_i}=F_i,\quad \mbox{for}\ i=1,\ldots,m,\quad \mbox{and}\quad F^{E}(x)\subseteq G(x),\quad \mbox{for}\ x\in \mathbb {R}^d. $$

Example 2

Suppose the following dynamic data for the domain in Example 1:

$$F_1(x)=\biggl[-\frac{1}{2},1\biggr],\quad \forall x\in \varOmega _1,\quad \mbox{and}\quad F_2(x)=\biggl[-1,\frac{1}{2} \biggr],\quad \forall x\in \varOmega _2. $$

On this simple example, one can easily see that G and F ^E are different on the interface {0}:

$$G(0)=[-1,1],\qquad F^{E}(0)=\biggl[-\frac{1}{2},\frac{1}{2}\biggr]. $$

Now, define the “essential” Hamiltonian $H^{E}:\mathbb {R}^{d}\times \mathbb {R}^{d}\rightarrow \mathbb {R}$ by:

$$ H^{E}(x,q)=\sup_{p\in F^{E}(x)}\{-p\cdot q\},\quad \forall (x,q) \in \mathbb {R}^d\times \mathbb {R}^d. $$

We point out that on each d-manifold Ω _i, for each $q\in \mathbb {R}^{d}$

$$H^E(x,q)=H_i(x,q),\quad \mbox{whenever}\ x\in \varOmega _i. $$

In general, H ^E is not Lipschitz continuous with respect to the first variable. Some properties of H ^E will be discussed in Sect. 3.

2.3 Main Results

We now state the main existence and uniqueness result.

Theorem 2.4

Assume that (H1)–(H4) hold. The following system:

(2.2a)

(2.2b)

(2.2c)

has a unique viscosity solution in the sense of Definition 2.6.

Note that the system (2.2a)–(2.2c) can be rewritten as

$$ \begin{cases}\displaystyle -\partial_t u(t,x)+H^E(x,Du(t,x))=0, & \mbox{for}\ t\in(0,T),\ x\in \mathbb {R}^d,\\ u(T,x)=\varphi (x), & \mbox{for}\ x\in \mathbb {R}^d, \end{cases} $$

which is an HJB equation on the whole space with a discontinuous Hamiltonian H ^E.

Before giving the definition of viscosity solution, we need the following notion of extended differentials.

Definition 2.5

(Extended differential)

Let $\phi:(0,T)\times \mathbb {R}^{d}\rightarrow \mathbb {R}$be a continuous function, and let $\mathcal {M}\subseteq \mathbb {R}^{d}$be an openC ²embedded manifold in $\mathbb {R}^{d}$. Suppose that $\phi\in C^{1}((0,T)\times \mathcal {M})$. Then we define the differential ofϕon any $(t,x)\in(0,T)\times\overline{\mathcal {M}}$by

$$\nabla_{\overline{\mathcal {M}}}\phi(t,x):=\lim_{x_n\rightarrow x,x_n\in \mathcal {M}} \bigl( \phi_t(t,x_n),D\phi(t,x_n) \bigr). $$

Note that ∇ϕ is continuous on $(0,T)\times \mathcal {M}$, the differential defined above is nothing but the extension of ∇ϕ to the whole $\overline{\mathcal {M}}$.

Definition 2.6

(Viscosity solution)

Let $u:(0,T]\times \mathbb {R}^{d}\rightarrow \mathbb {R}$be a bounded local Lipschitz continuous function. For any $x\in \mathbb {R}^{d}$, let $\mathbb{I}(x):=\{i,x\in\overline{\mathcal {M}}_{i}\}$be the index set.

• (i) We say thatuis a supersolution of (2.2a)–(2.2b) if for any $(t_{0},x_{0})\in(0,T)\times \mathbb {R}^{d}$, $\phi\in C^{1}((0,T)\times \mathbb {R}^{d})$such thatu−ϕattains a local minimum on (t ₀,x ₀), we have

  $$-\phi_t(t_0,x_0)+H^E\bigl(x_0,D \phi(t_0,x_0)\bigr)\geq0. $$

• (ii) We say thatuis a subsolution of (2.2a)–(2.2b) if for any $(t_{0},x_{0})\in(0,T)\times \mathbb {R}^{d}$, any continuous $\phi:(0,T)\times \mathbb {R}^{d}\rightarrow \mathbb {R}$with $\phi|_{(0,T)\times \mathcal {M}_{k}}$beingC ¹for any $k\in \mathbb{I}(x)$such thatu−ϕattains a local maximum at (t ₀,x ₀) on $(0,T)\times\overline{\mathcal {M}}_{k}$, we have

  $$-q_t+\sup_{p\in F^{E}_k(x_0)}\{-p\cdot q_x\}\leq 0,\quad \mbox{\textit{with}}\ (q_t,q_x)=\nabla_{\overline{\mathcal {M}}_k} \phi(t_0,x_0). $$

• (iii) We say thatuis a viscosity solution of (2.2a)–(2.2c) ifuis both a supersolution and a subsolution, andusatisfies the final condition

  $$u(T,x)=\varphi (x),\quad \forall x\in \mathbb {R}^d. $$

2.4 Comments

The problem (2.1) is formally linked to some hybrid control problems where the dynamics depend on the state region. Theorem 2.4 indicates a new characterization of the value function of hybrid control problems without transition cost. More details are presented in Sect. 3.

Another application related to the addressed problem in this paper is the traffic flow problems where the structure of multi-domains is composed by one-dimensional half-lines and a junction point. On each half-line an HJ equation is imposed to describe the density of the traffic and it is interesting to understand what happens at the junction point. See [13] for more details.

A similar topic with one interface (hyperplane) separating two subdomains has been studied in [4]. The work [4] deals with an infinite horizon problem which leads to the stationary HJB equations with running cost. In this context, a complete analysis of the uniqueness of solutions for (2.1) is provided in [4].

In the present work, we consider a more general situation where the intersection of the domains are interfaces with different dimensions from d−1 to zero. In order to focus only on the difficulty arising from this general structure, we consider a time-dependent equations without running cost. The presence of running costs arises to further difficulties that will be addressed in [15].

Optimal control problems on stratified domains have been studied by Bressan–Hong [6] and Barnard–Wolenski [5]. The stratified domains are the multi-domains provided with dynamic data on each subdomain under some structural conditions. The work [5] focuses on the flow invariance on stratified structure. The junction condition established in our work is inspired by the notion of essential dynamics introduced in [5].

3 Link with Optimal Control Problems

Recall that for the classical optimal control problems of the Mayer’s type, the value function can be characterized as the unique viscosity solution of the equations of the type (2.1) with Lipschitz continuous Hamiltonians. In our settings of problem, the multifunctions F _i are defined separately on $\overline{\varOmega }_{i}$. A first idea would be to consider the “regularization” of F _i. However, the regularized multifunction G is only usc in general, and this is not enough to guarantee the existence and uniqueness of solution for (2.1). So in our framework, in order to link the Hamilton–Jacobi equation with a Mayer’s optimal control problem, we need to well define the global trajectories driven by the dynamics (F _i)_i=1,…,m. Consider the following differential inclusion

$$ \begin{cases}\displaystyle \dot{y}(s)\in G(y(s)), & \mbox{for}\ s\in(t,T),\\ y(t)=x. \end{cases} $$

(3.1)

Since G is usc, (3.1) admits an absolutely continuous solution defined on [τ,T]. For any $(t,x)\in[0,T]\times \mathbb {R}^{d}$, we denote the set of absolutely continuous trajectories by

$$S_{[t,T]}(x):=\bigl\{y_{t,x}, y_{t,x}\ \mbox{satisfies}\ (3.1)\bigr\}. $$

Now consider the following Mayer’s problem

$$ v(t,x):=\min\bigl\{\varphi \bigl(y(T)\bigr), y(\cdot)\in S_{[t,T]}(x)\bigr\}. $$

(3.2)

Since G is usc and convex, the set S _[t,T](x) of absolutely continuous arcs is compact in $C(t,T;\mathbb {R}^{d})$ (see Theorem 1, [2] pp. 60). And then the problem (3.2) has an optimal solution for any t∈[0,T], $x\in \mathbb {R}^{d}$.

As in the classical case, v satisfies a Dynamical programming principle (DPP).

Proposition 3.1

Assume that (H1)–(H3) hold. Then for any $(t,x)\in[0,T]\times \mathbb {R}^{d}$the following holds.

• (i) The super-optimality. $\exists \bar{y}_{t,x}\in S_{[t,T]}(x)$such that

  $$v(t,x)\geq v\bigl(t+h,\bar{y}_{t,x}(t+h)\bigr),\quad \mbox{\textit{for}}\ h \in[0,T-t]. $$

• (ii) The sub-optimality. ∀y _t,x∈S _[t,T](x) such that

  $$v(t,x)\leq v\bigl(t+h,y_{t,x}(t+h)\bigr),\quad \mbox{\textit{for}}\ h\in[0,T-t]. $$

An important fact resulting from the assumptions (H2) and (H4)(iv) is the local Lipschitz continuity of the value function v.

Proposition 3.2

Assume that (H1)–(H4) hold. Then the value functionvis locally Lipschitz continuous on $[0,T]\times \mathbb {R}^{d}$.

Proof

For any t∈[0,T], we first prove that v(t,⋅) is locally Lipschitz continuous on $\mathbb {R}^{d}$. Let $x,z\in \mathbb {R}^{d}$, without loss of generality, suppose that

$$v(t,x)\geq v(t,z). $$

There exists $\overline{y}_{t,z}\in S_{[t,T]}(z)$ such that

$$v(t,z)=\varphi \bigl(\overline{y}_{t,z}(T)\bigr). $$

We set

$$h=\frac{\|x-z\|}{\delta},\qquad \xi(s)=x+\delta\frac{z-x}{\|z-x\|}(s-t)\quad \mbox{for}\ s \in[t,t+h]. $$

Note that ξ(t)=x, ξ(t+h)=z. By the controllability assumption (H4)(iv), we can define the following trajectory

$$ \widetilde{y}_{t,x}(s)= \begin{cases}\displaystyle \xi(s), & \mbox{for}\ s\in[t,t+h],\\ \overline{y}_{t,z}(s-h), & \mbox{for}\ s\in[t+h,T]. \end{cases} $$

By denoting L _φ>0 the Lipschitz constant of φ, we have

$$ \begin{aligned} v(t,x)-v(t,z)&\leq \varphi \bigl(\widetilde{y}_{t,x}(T) \bigr)-\varphi \bigl(\overline{y}_{t,z}(T)\bigr) \\ &\leq L_{\varphi }\bigl\|\widetilde{y}_{t,x}(T)-\overline{y}_{t,z}(T)\bigr\| \\ &\leq L_{\varphi }\bigl\|\overline{y}_{t,z}(T-h)-\overline{y}_{t,z}(T)\bigr\| \\ &\leq L_{\varphi }\|G\|h=\frac{L_{\varphi }\|G\|}{\delta}\|x-z\|, \end{aligned} $$

where we deduce the local Lipschitz continuity of v(t,⋅).

Then for $x\in \mathbb {R}^{d}$, we prove the Lipschitz continuity of v(⋅,x) on [0,T]. For any t,s∈[0,T], without loss of generality suppose that t<s. By the super-optimality, there exists y ^op∈S _[t,T](x) such that

$$v(t,x)=v\bigl(s,y^{\mathit{op}}(s)\bigr). $$

Then

$$ \bigl|v(t,x)-v(s,x)\bigr|=\bigl|v\bigl(s,y^{\mathit{op}}(s)\bigr)-v(s,x)\bigr|\leq L_v \|G\|(s-t), $$

where L _v is the local Lipschitz constant of v(s,⋅). And the proof is complete. □

Remark 3.3

Assumption (H4)(iv) plays an important role in our proof for the Lipschitz continuity of the value function. However, it is worth mentioning that the Lipschitz continuity can also be satisfied in some cases where (H4)(iv) is not satisfied. In Example 1, if one take F ₁=F ₂ Lipschitz continuous dynamics, then the value function will be Lipschitz continuous without assuming any controllability property. For multi-domains problems, some weaker assumptions of controllability are analyzed in [15].

The following result analyzes the structure of the dynamics and makes clear the behavior of the trajectories.

Proposition 3.4

Suppose $y(\cdot):[t,T]\rightarrow \mathbb {R}^{d}$is an absolutely continuous arc. Then the following are equivalent.

• (i) y(⋅) satisfies (3.1);

• (ii) For eachk=1,…,m+ℓ, y(⋅) satisfiesy(t)=xand

  $$\dot{y}(s)\in F^{\mathit {new}}_k\bigl(y(s)\bigr),\quad \mbox{\textit{a.e. whenever}}\ y(s)\in \mathcal {M}_k, $$

• (iii) y(⋅) satisfies

  $$\begin{cases}\displaystyle \dot{y}(s)\in F^{E}(y(s)) & \mbox{\textit{for}}\ s\in(t,T),\\ y(t)=x. \end{cases} $$

Proof

It is clear that (ii) implies (i) since $F^{\mathit {new}}_{k}(x)\subseteq G(x)$ whenever $x\in \mathcal {M}_{k}$. So assume that (i) holds, and let us show that (ii) holds as well.

The proof is essentially the same as in Proposition 2.1 of [5]. For any k=1,…,m+ℓ, let $J_{k}:=\{s\in[t,T]:y(s)\in \mathcal {M}_{k}\}$. Without loss of generality, suppose that the Lebesgue measure mes(J _k)≠0. We set

$$\tilde{J}_k:=\bigl\{s\in J_k: \dot{y}(s)\ \mbox{exists in}\ G\bigl(y(s)\bigr)\ \mbox{and}\ s\ \mbox{is a Lebesgue point of } J_k\bigr\}. $$

It is clear that $\tilde{J}_{k}$ has full measure in J _k. For any $s\in \tilde{J}_{k}$, then being a Lebesgue point implies that there exists a sequence {s _n} such that s _n→s as n→∞ with $s\neq s_{n}\in\tilde{J}_{k}$ for all n. Since $y(s_{n})\in \mathcal {M}_{k}$, we have

$$\dot{y}(s)=\lim_{n\rightarrow \infty}\frac{y(s_n)-y(s)}{s_n-s}\in \mathcal {T}_{\mathcal {M}_k} \bigl(y(s)\bigr). $$

Then by the definition of $F^{\mathit {new}}_{k}$, we have

$$\dot{y}(s)\in G\bigl(y(s)\bigr)\cap \mathcal {T}_{\mathcal {M}_k}\bigl(y(s)\bigr)= F^{\mathit {new}}_k\bigl(y(s)\bigr),\quad \forall s\in\tilde{J}_k, $$

which proves (ii).

It is clear that (ii) ⇒ (iii) ⇒ (i) since $F^{\mathit {new}}_{k}(\cdot)\subseteq F^{E}(\cdot)\subseteq G(\cdot)$, which ends the proof. □

Proposition 3.4 will be very useful in the characterization of the super-optimality and the sub-optimality by HJ equations involving the essential Hamiltonian H ^E.

3.1 The Supersolution Property

The following proposition shows the characterization of the super-optimality by the supersolutions of HJ equations. This is a classical result since G is usc.

Proposition 3.5

Suppose $u:[0,T]\times \mathbb {R}^{d}\rightarrow \mathbb {R}$is continuous. Thenusatisfies the super-optimality if and only if for any $(t_{0},x_{0})\in(0,T)\times \mathbb {R}^{d}$, $\phi\in C^{1}((0,T)\times \mathbb {R}^{d})$such thatu−ϕattains a local minimum on (t ₀,x ₀), we have

$$ -\phi_t(t_0,x_0)+H_G \bigl(x_0, D\phi(t_0,x_0)\bigr)\geq0. $$

(3.3)

Proof

This is a straightforward consequence of Theorem 3.2 and Lemma 4.3 in [12] (see also [3]). □

Due to the structure of the dynamics G illustrated in Proposition 3.4, it is possible to replace G by F ^E to get a more precise HJB inequality since the set of trajectories driven by G or F ^E is the same. But the difficulty here is that in general F ^E is not usc.

At first, we have the following result concerning the dynamics of the optimal trajectories.

Lemma 3.6

Lety(⋅)∈S _[t,T](x) be an absolutely continuous arc along which the value functionvsatisfies the super-optimality. For any $p\in \mathbb {R}^{d}$such that there existst _n→0⁺with $\frac {y(t_{n})-x}{t_{n}}\rightarrow p$, by denoting co F ^E(x) the convex hull ofF ^E(x) we have

$$p\in \text {co }F^{E}(x). $$

The proof of Lemma 3.6 is presented in Appendix A. In the next theorem, we will use the statement of Lemma 3.6 to show that the functions satisfying the super-optimality condition is also a solution to a more precise HJB equation with H ^E than the HJB equation (3.3) with the Hamiltonian H _G even if F ^E is not usc.

Theorem 3.7

Suppose $u:[0,T]\times \mathbb {R}^{d}\rightarrow \mathbb {R}$is continuous andu(T,x)=φ(x) for all $x\in \mathbb {R}^{d}$. usatisfies the super-optimality if and only ifuis a supersolution of (2.2a)–(2.2c), i.e. for any $(t_{0},x_{0})\in(0,T)\times \mathbb {R}^{d}$, $\phi\in C^{1}((0,T)\times \mathbb {R}^{d})$such thatu−ϕattains a local minimum on (t ₀,x ₀), we have

$$-\phi_t(t_0,x_0)+\sup_{p\in F^{E}(x_0)} \bigl\{-p\cdot D\phi(t_0,x_0)\bigr\}\geq0. $$

Proof

(⇒) Let $\bar{y}_{t_{0},x_{0}}$ be the optimal trajectory along which u satisfies the super-optimality. Then for any $(t_{0},x_{0})\in(0,T)\times \mathbb {R}^{d}$, $\phi\in C^{1}((0,T)\times \mathbb {R}^{d})$ such that u−ϕ attains a local minimum on (t ₀,x ₀), by the same argument in Proposition 3.5, we obtain

$$\frac{1}{h}\bigl(\phi(t_0,x_0)-\phi \bigl(t_0+h,\bar{y}_{t_0,x_0}(t_0+h)\bigr)\bigr)\geq 0, $$

i.e.,

$$ \frac{1}{h}\int^h_0 \bigl[- \phi_t\bigl(t_0+s,\bar{y}_{t_0,x_0}(t_0+s) \bigr) -D\phi\bigl(t_0+s,\bar{y}_{t_0,x_0}(t_0+s)\bigr) \cdot\dot{\bar{y}}_{t_0,x_0}(t_0+s) \bigr]ds\geq0. $$

Up to a subsequence, let h _n→0⁺ so that $x_{n}:=\bar {y}_{t_{0},x_{0}}(t_{0}+h_{n})$ satisfies $\frac{x_{n}-x}{h_{n}}\rightarrow p$ for some $p\in \mathbb {R}^{d}$. We then get

$$-\phi_t(t_0,x_0)-p\cdot D \phi(t_0,x_0)\geq0. $$

Lemma 3.6 leads to

$$ p\in \text {co }F^{E}(x_0). $$

(3.4)

Then we deduce that

$$-\phi_t(t_0,x_0)+\sup_{p\in \text {co }F^{E}(x_0)} \bigl\{-p\cdot D\phi(t_0,x_0)\bigr\}\geq0. $$

By the separation theorem

$$-\phi_t(t_0,x_0)+\sup_{p\in F^{E}(x_0)} \bigl\{-p\cdot D\phi(t_0,x_0)\bigr\}\geq0. $$

(⇐) For any $(t_{0},x_{0})\in(0,T)\times \mathbb {R}^{d}$, $\phi\in C^{1}((0,T)\times \mathbb {R}^{d})$ such that u−ϕ attains a local minimum on (t ₀,x ₀), since u is a supersolution, we have

$$-\phi_t(t_0,x_0)+\sup_{p\in F^E(x_0)} \bigl\{-p\cdot D\phi(t_0,x_0)\bigr\}\geq0. $$

Note that F ^E(x ₀)⊆G(x ₀), then we deduce that

$$-\phi_t(t_0,x_0)+\sup_{p\in G(x_0)} \bigl\{-p\cdot D\phi(t_0,x_0)\bigr\}\geq0. $$

Then we deduce the desired result by Proposition 3.5. □

3.2 The Subsolution Property

As mentioned before, if G is Lipschitz continuous, one can characterize the sub-optimality by the opposite HJB inequalities:

$$-u_t(t_0,x_0)+H_G \bigl(x_0,Du(t_0,x_0)\bigr)\leq0 $$

in the viscosity sense. However, G is only usc on the interfaces. And the characterization using H _G fails because there are dynamics in G which are not “essential”, which means for some p∈G(x), there does not exist any trajectory coming from x using the dynamic p. For instance in Example 2, at the point 0, G(0)=[−1,1]. Consider the dynamic p=1∈G(0), if there exists a trajectory y starting from 0 using the dynamic 1, y goes immediately into Ω ₂ and y is not admissible since 1 is not contained in the dynamics F ₂.

In the sequel, we consider the essential dynamic multifunction F ^E to replace G by eliminating the useless nonessential dynamics. Note that F ^E in general is not Lipschitz either. The significant role of F ^E is shown in the following result.

Lemma 3.8

For anyp∈F ^E(x), there existsτ>tand a solutiony(⋅) of (3.1) which isC ¹on [t,τ] with $\dot{y}(t)=p$.

Proof

This is a partial result of in [5, Proposition 5.1]. For the convenience of reader, a sketch of the proof is given in Appendix B. □

More precisely, Lemma 3.8 can be rewritten as:

Lemma 3.9

Letk∈{1,…,m+ℓ}, $x\in\overline{\mathcal {M}}_{k}$. Then for any $p\in F^{E}_{k}(x)$, there existτ>tand a trajectory of (3.1) y(⋅) which isC ¹on [t,τ] with $\dot{y}(t)=p$and $y(s)\in \overline{\mathcal {M}}_{k}$fors∈[t,τ].

The following two results give the characterization of sub-optimality by HJB inequalities.

Proposition 3.10

Let $u:[0,T]\times \mathbb {R}^{d}\rightarrow \mathbb {R}$be locally Lipschitz continuous andu(T,x)=φ(x) for all $x\in \mathbb {R}^{d}$. Suppose thatusatisfies the sub-optimality, thenuis a subsolution of (2.2a)–(2.2c) in the sense of Definition 2.6.

Proof

Given $(t_{0},x_{0})\in[0,T]\times \mathbb {R}^{d}$, for any $k\in\mathbb{I}(x_{0}),p\in F^{E}_{k}(x_{0})$, by Lemma 3.9, there exists h>0 and a solution y(⋅) of (3.1) C ¹ on [t ₀,t ₀+h] with $\dot {y}(t_{0})=p,y(t_{0})=x_{0}$ and $y(s)\in\overline{\mathcal {M}}_{k},\forall s\in [t_{0},t_{0}+h]$. By the sub-optimality of u

$$u(t_0,x_0)\leq u\bigl(t_0+h,y(t_0+h) \bigr). $$

For any $\phi\in C^{0}((0,T)\times \mathbb {R}^{d})\cap C^{1}((0,T)\times \mathcal {M}_{k})$ such that u−ϕ attains a local maximum at (t ₀,x ₀) on $(0,T)\times \overline{\mathcal {M}}_{k}$, we have

$$u\bigl(t_0+h,y(t_0+h)\bigr)-\phi\bigl(t_0+h,y(t_0+h) \bigr)\leq u(t_0,x_0)-\phi(t_0,x_0). $$

Then we deduce that

$$\frac{1}{h}\bigl(\phi(t_0,x_0)-\phi \bigl(t_0,y(t_0+h)\bigr)\bigr)\leq0. $$

By taking h→0 we have

$$-q_t-p\cdot q_x\leq0,\quad \mbox{where}\ p\in F^{E}_k(x_0),\ (q_t,q_x)\in \nabla_{\overline{\mathcal {M}}_k}\phi(t_0,x_0), $$

i.e.

$$ -q_t+\sup_{p\in F^{E}_k(x_0)}\{-p\cdot q_x\}\leq 0. $$

 □

We present a precise example to illustrate that H ^E is the proper Hamiltonian for the subsolution characterization of the value function.

Example 3

Consider again the same 1d structure as in Example 1 and Example 2, i.e. $\mathbb {R}=\varOmega _{1}\cup \varOmega _{2}\cup\varGamma_{1}$ with

$$\varOmega _1=(-\infty,0),\qquad \varOmega _2=(0,+\infty),\qquad \varGamma_1=\{0\}, $$

and the dynamics

$$F_1(x)=\biggl[-\frac{1}{2},1\biggr],\quad \forall x\in \varOmega _1,\quad \mbox{and}\quad F_2(x)=\biggl[-1,\frac{1}{2} \biggr],\quad \forall x\in \varOmega _2. $$

At the point 0, the convexified dynamics G(0)=[−1,1] and the essential dynamics $F^{E}(0)=[-\frac{1}{2},\frac{1}{2}]$. Let T>0 be a given final time and the final cost function φ ₂(x)=x. Then from any initial data $(t,x)\in[0,T]\times \mathbb {R}$, the optimal strategy is to go on the left as far as possible. Thus the value function is given by

$$v_2(t,x):=\min\bigl\{\varphi _2\bigl(y_{t,x}(T) \bigr)\bigr\}= \begin{cases} x-\frac{1}{2}(T-t) & x\leq0,\\ -\frac{1}{2}(T-t-x) & 0\leq x\leq T,\\ x-(T-t) & x\geq T-t. \end{cases} $$

At the point (t,x)=(0,0), $\partial_{t} v_{2}(0,0)=\frac{1}{2}$, Dv ₂(0,0⁻)=1, $Dv_{2}(0,0^{+})=\frac{1}{2}$, $D^{+}v_{2}(0,0)=[\frac {1}{2},1]$. Then we have

$$-\partial_t v_2(0,0)+\max_{p\in F^E(0)}\bigl \{-p\cdot D^+v_2(0,0)\bigr\}=0\leq0, $$

while

$$-\partial_t v_2(0,0)+\max_{p\in G(0)}\bigl \{-p\cdot D^+v_2(0,0)\bigr\}=\frac{1}{2}> 0. $$

We see that the subsolution property fails if we replace F ^E by G which is larger.

Proposition 3.7 indicates that any function satisfying the sub-optimality is a subsolution of (2.2a)–(2.2c). The inverse result needs more elaborated arguments. The difficulty arises mainly from handling the trajectories oscillating near the interfaces, i.e. the trajectories cross the interfaces infinitely in finite time which exhibit a type of “Zeno” effect. The proofs of Theorem 3.12 and of Proposition 3.11 contain details on how to construct the “nice” approximate trajectories to deal with Zeno-type trajectories.

At first, we give the following result containing the key fact of Zeno-type trajectories.

Proposition 3.11

Letube a Lipschitz continuous subsolution of (2.2a)–(2.2c). Suppose $\mathcal {M}_{k}$is a subdomain and $\mathcal {M}$is a union of subdomains with $\mathcal {M}_{k}\subseteq\overline{\mathcal {M}}$. Assume $\mathcal {M}$has the following property: for every trajectoryy(⋅) of (3.1) defined on [a,b]⊆[t,t+h] with $y(\cdot)\subseteq \mathcal {M}$, we have

$$ u\bigl(a,y(a)\bigr)\leq u\bigl(b,y(b)\bigr). $$

(3.5)

Then for any trajectoryy(⋅) of (3.1) defined on [a,b]⊆[t,t+h] lying totally within $\mathcal {M}_{k}\cup \mathcal {M}$, we have

$$u\bigl(a,y(a)\bigr)\leq u\bigl(b,y(b)\bigr). $$

Proof

Here we adapt an idea introduced in [5] in a context of stratified control problems. Let y(⋅) be a trajectory of (3.1) with $y(\cdot)\subseteq \mathcal {M}_{k}\cup \mathcal {M}$ satisfying (3.5). Without loss of generality, suppose that $y(a)\in \mathcal {M}_{k}$ and $y(b)\in \mathcal {M}_{k}$. By (H3), we have $\mathcal {M}_{k}\cap \mathcal {M}=\emptyset$. Let $J:=\{s\in[a,b]:y(s)\notin \mathcal {M}_{k}\}$, which is an open set and so can be written as

$$J=\bigcup_{n=1}^{\infty} (a_n,b_n) $$

where the intervals are pairwise disjoint. For a fixed p, we set

$$J_{ p} := \bigcup_{n=1}^{ p} (a_n,b_n), $$

which after re-indexing can be assumed to satisfy

$$b_0:=a\leq a_1<b_1\leq a_2 < b_2\leq\dots\leq a_{ p} < b_{ p}\leq a_{ p+1}:=b. $$

Choose p sufficiently large so that

$$ {\mathit {meas}}(J\backslash J_{ p})<\frac{r}{2 e^{LT}\|G\|}, $$

where ∥G∥ is an upper bound of the norm of any velocity that may appear, and r>0 is given by

$$r:=\inf_{\substack{s\in[b_0,b]\\ w\in\overline{\mathcal {M}}_k\backslash \mathcal {M}_k}}\bigl\| y(s)-w\bigr\|. $$

For n=1,…,p, $y(s)\in \mathcal {M}$ for s∈(a _n,b _n). Let ε>0 small enough such that [a _n+ε,b _n−ε]⊆(a _n,b _n), then by (3.5)

$$u\bigl(a_n+\varepsilon ,y(a_n+\varepsilon )\bigr)\leq u\bigl(b_n- \varepsilon ,y(b_n-\varepsilon )\bigr). $$

Taking ε→0 and by the continuity of u and y(⋅), we deduce that

$$u\bigl(a_n,y(a_n)\bigr)\leq u\bigl(b_n,y(b_n) \bigr). $$

Next we need to deal with y(⋅) restricted to [b _n,a _n+1]. For n=0,…,p, by Proposition 3.4 $\dot{y}(s)\in F^{\mathit {new}}_{k} (y(s) )$ for almost all s∈[b _n,a _n+1]∖J. For n=0,…,p, set ε _n:=meas([b _n,a _n+1]∩J), and note that $\sum_{n=0}^{ p}\varepsilon_{n}={\mathit {meas}}(J\backslash J_{ p})$. We calculate how far y(⋅) is from a trajectory lying in $\mathcal {M}_{k}$ with dynamics $F^{\mathit {new}}_{k}$ by

$$\xi_n:=\int_{b_n}^{a_{n+1}}\text {dist}\bigl(\dot{y}(s),F^{\mathit {new}}_k \bigl(y(s) \bigr) \bigr) ds \leq2 \|G\| \varepsilon_n. $$

By the Filippov approximation theorem (see [8, Theorem 3.1.6] and also [9, Proposition 3.2]), there exists a trajectory z _n(⋅) of $F^{\mathit {new}}_{k}$ defined on the interval [b _n,a _n+1] that lies in $\mathcal {M}_{k}$ with z _n(b _n)=y(b _n) and satisfies

$$ \bigl\|z_n(a_{n+1})-y(a_{n+1}) \bigr\|\leq e^{L(a_{n+1}-b_n)}\xi_n \leq2\|G\|e^{L(a_{n+1}-b_n)} \varepsilon_n. $$

(3.6)

Since u is subsolution of (2.2a)–(2.2c), then for any $x\in \mathcal {M}_{k}$, note that $F^{\mathit {new}}_{k}(x)\subseteq \mathcal {T}_{\mathcal {M}_{k}}(x)$ and $\mathcal {T}_{\mathcal {M}_{k}}(x)=\mathcal {T}_{\overline{\mathcal {M}}_{k}}(x)$ by Definition 2.6

$$ -\partial_t \phi(t,x)+\sup_{p\in F^{\mathit {new}}_k(x)} \bigl\{-p\cdot D\phi(t,x)\bigr\}\leq0 $$

(3.7)

with $\phi\in C^{0}((0,T)\times \mathbb {R}^{d})\cap C^{1}((0,T)\times \mathcal {M}_{k})$ and u−ϕ attains a local maximum at (t,x) on $(0,T)\times \mathcal {M}_{k}$. Since z _n(⋅) lies in $\mathcal {M}_{k}$ on [b _n,a _n+1] driven by the Lipschitz dynamics $F^{\mathit {new}}_{k}$, then (3.7) implies that the sub-optimality of u is satisfied on $z_{n}(\cdot)|_{[b_{n},a_{n+1}]}$, i.e.

$$u\bigl(b_n,z_n(b_n)\bigr)\leq u \bigl(a_{n+1},z_n(a_{n+1})\bigr). $$

Then by (3.6) we have

$$ \begin{aligned} u\bigl(b_n,y(b_n)\bigr)&= u \bigl(b_n,z_n(b_n)\bigr)\leq u \bigl(a_{n+1},z_n(a_{n+1})\bigr) \\ &\leq u\bigl(a_{n+1},y(a_{n+1})\bigr)+2L_u\|G \|e^{L(a_{n+1}-b_n)}\varepsilon _n. \end{aligned} $$

We set ε ^p:=meas(J∖J _p), and we deduce that

$$ \begin{aligned} u\bigl(a,y(a)\bigr)&\leq u\bigl(a_1,y(a_1) \bigr)+2L_u\|G\|e^{L(a_1-b_0)}\varepsilon _1 \\ &\leq u\bigl(a_2,y(a_2)\bigr)+2L_u\|G \|e^{L(a_2-b_0)}(\varepsilon _1+\varepsilon _2) \\ &\vdots \\ &\leq u\bigl(a_{ p+1},y(a_{ p+1})\bigr)+2L_u\|G \|e^{L(a_{p+1}-b_0)}\varepsilon ^p \\ &= u\bigl(b,y(b)\bigr)+2L_u\|G\|e^{L(b-a)}\varepsilon ^p. \end{aligned} $$

By taking p→+∞, we have ε ^p→0 and the desired result is obtained. □

Theorem 3.12

Supposeuis a locally Lipschitz continuous subsolution of (2.2a)–(2.2c). Thenusatisfies the sub-optimality, i.e. for any trajectoryy(⋅)∈S _[t,T](x), one has

$$u(t,x)\leq u\bigl(t+h,y(t+h)\bigr),\quad \forall h\in[0,T-t]. $$

Proof

Let $\mathcal {M}$ be a union of subdomains (manifolds or interfaces). Let $\bar{d}_{\mathcal {M}}\in\{0,\ldots,d\}$ be the minimal dimension of the subdomains in $\mathcal {M}$. We claim that for any h∈[0,T−t] and any trajectory y(⋅) of (3.1) lying totally within $\mathcal {M}$, we have

$$ u\bigl(a,y(a)\bigr)\leq u\bigl(b,y(b)\bigr),\quad \mbox{for any}\ [a,b]\subseteq[t,t+h]. $$

(3.8)

The proof of (3.8) is based on an induction argument with regard to the minimal dimension $\bar{d}_{\mathcal {M}}$:

(HR) :

for $\tilde{d}\in\{1,\ldots,d\}$, suppose that for any $\mathcal {M}$ with $\bar{d}_{\mathcal {M}}\geq\tilde{d}$ and for any trajectory y(⋅) that lies within $\mathcal {M}$, (3.8) holds.

Step (1): Let us first check the case when $\tilde{d}=d$. In this case, $\bar{d}_{\mathcal {M}}=d$, then $\mathcal {M}$ is a union of d-manifolds which are disjoint by (H1). For any trajectory y(⋅) of (3.1) lying within $\mathcal {M}$, since y(⋅) is continuous, y(⋅) lies entirely in one of the d-manifolds, denoted by Ω _i. The subsolution property of u implies that

$$-\partial_t u(t,x)+\sup_{p\in F_i(x)}\bigl\{-p\cdot Du(t,x)\bigr\}\leq0 $$

holds in the viscosity sense. Since the dynamics on Ω _i is F _i which is Lipschitz continuous, then by the classical theory u satisfies the sub-optimality along y(⋅) and (3.8) holds true.

Step (2): Now assume that (HR) is true for $\tilde{d}\in\{1,\ldots,d\}$, and let us prove that (HR) is true for $\tilde{d}-1$. In this case, the minimal dimension of subdomains in $\mathcal {M}$ is $\bar{d}_{\mathcal {M}}=\tilde{d}-1$, $\tilde{d}\in\{1,\ldots,d\}$. As an induction hypothesis, assume that for any trajectory that lies within a union of subdomains each with dimension greater than $\tilde{d}$, then (3.8) holds. Three cases can occur.

• If $\mathcal {M}$ contains only one subdomain, i.e. $\mathcal {M}=\mathcal {M}_{k}$ with dimension $\bar{d}_{\mathcal {M}}$ for some k∈{1,…,m+ℓ}, then for any trajectory y(⋅) lying within $\mathcal {M}_{k}$, the subsolution property of u implies that u satisfies the sub-optimality along y(⋅) since the dynamics $F^{\mathit {new}}_{k}$ is Lipschitz continuous on $\mathcal {M}_{k}$.

• If $\mathcal {M}$ contains more than one subdomain and $\mathcal {M}$ is connected, let $\mathcal {M}'_{1},\ldots,\mathcal {M}'_{p}$ be all the subdomains contained in $\mathcal {M}$ with dimension $\bar{d}_{\mathcal {M}}$. Then $\widetilde{\mathcal {M}}:=\mathcal {M}\backslash(\cup _{k=1}^{p}\mathcal {M}'_{k} )$ is a union of subdomains with dimension greater than $\tilde{d}$. We note that $\mathcal {M}'_{k}\subseteq\overline{\widetilde{\mathcal {M}}}$ for each k=1,…,p. Then by the induction hypothesis and Proposition 3.11, (3.8) holds true for any trajectory lying entirely within $\widetilde{\mathcal {M}}\cup \mathcal {M}'_{1}$. Then by applying Proposition 3.11 for $\widetilde{\mathcal {M}}\cup \mathcal {M}'_{1}$ and $\mathcal {M}'_{2}$, (3.8) holds true for any trajectory lying entirely within $\widetilde {\mathcal {M}}\cup \mathcal {M}'_{1}\cup \mathcal {M}'_{2}$. We continue this process and finally we have (3.8) holds true for any trajectory lying entirely within $\mathcal {M}=\widetilde{\mathcal {M}}\bigcup(\cup_{k=1}^{p}\mathcal {M}'_{k} )$.

• If $\mathcal {M}$ is not connected, for any trajectory y(⋅) lying within $\mathcal {M}$, since y(⋅) is continuous, then y(⋅) lies within one connected component of $\mathcal {M}$. Then by the same argument as above, (3.8) holds true for y(⋅). And the induction step is complete.

Finally, to complete the proof of the theorem, we remark that for any trajectory y(⋅) of (3.1), by considering $\mathcal {M}=\mathbb {R}^{d}$ with $\bar{d}_{\mathcal {M}}=0$, taking a=t, b=t+h in (3.8) we have

$$u(t,x)\leq u\bigl(t+h,y(t+h)\bigr), $$

which ends the proof. □

4 Proof of Theorem 2.4

Since v satisfies the super-optimality and sub-optimality, by Theorem 3.7 and Theorem 3.10 v is a viscosity solution of (2.2a)–(2.2c).

The uniqueness result is obtained by the following result of comparison principle.

Proposition 4.1

Suppose that $u:[0,T]\times \mathbb {R}^{d}\rightarrow \mathbb {R}$is Lipschitz continuous andu(T,x)=φ(x) for any $x\in \mathbb {R}^{d}$.

• (i) Ifusatisfies the super-optimality, thenv(t,x)≤u(t,x) for all $(t,x)\in[0,T]\times \mathbb {R}^{d}$;

• (ii) Ifusatisfies the sub-optimality, thenv(t,x)≥u(t,x) for all $(t,x)\in[0,T]\times \mathbb {R}^{d}$.

Proof

(i) For any $(t,x)\in[0,T]\times \mathbb {R}^{d}$, by the super-optimality of u, there exists a trajectory $\overline{y}_{t,x}$ such that

$$u(t,x)\geq u\bigl(T,\overline{y}_{t,x}(T)\bigr)=\varphi \bigl( \overline{y}_{t,x}(T)\bigr). $$

By the sub-optimality of v, we have

$$v(t,x)\leq v\bigl(T,\overline{y}_{t,x}(T)\bigr)=\varphi \bigl( \overline{y}_{t,x}(T)\bigr). $$

Then we deduce that

$$v(t,x)\leq u(t,x). $$

(ii) The proof is completed by the same argument by considering the super-optimality of v and the sub-optimality of u. □

5 Conclusion

In this paper, we have studied the system (1.1) in a general framework of the multi-domains with several interfaces. The existence and uniqueness result of the solution is studied under some junction conditions on the interfaces. The latter are derived by considering a control problem for which the value function satisfies the system (1.1) on each sub-domain Ω _i. The analysis of this value function indicates the information that should be considered on the interfaces in order to guarantee a continuous solution of the system.

Appendices

Appendix A

Proof of Lemma 2.2

Note that although G is only usc on $\mathbb {R}^{d}$, G is Lipschitz continuous on Γ _j since G is the convexification of a finite group of Lipschitz continuous multifunctions on Γ _j. For any x∈Γ _j, there exists α>0 and a diffeomorphism $g\in C^{1,1}(\mathbb {R}^{d})$ such that

$$\overline{B(x,\alpha)\cap\varGamma_j}=\bigl\{x:g(x)=0\bigr\}\quad \mbox{and}\quad \nabla g(y)\neq0,\quad \forall y\in\overline{B(x,\alpha)}. $$

We can take g as the signed distance function to Γ _j for instance. Then there exists β>0 such that

$$\bigl\|\nabla g(y)\bigr\|\geq\beta,\quad \forall y\in B(x,\alpha)\cap\varGamma_j. $$

For any $w\in G(x)\cap \mathcal {T}_{\varGamma_{j}}(x)$, by the Lipschitz continuity of G there exists v∈G(y) such that

$$\|w-v\|\leq L_G\|x-y\|, $$

where L _G is the Lipschitz constant of G(⋅). Since $w\in \mathcal {T}_{\varGamma_{j}}(x)$, we have

$$w\cdot\nabla g(x)=0. $$

Then

$$\bigl\|v\cdot\nabla g(x)\bigr\|=\bigl\|(v-w)\cdot\nabla g(x)\bigr\|\leq L_G\|\nabla g\| \|x-y\|. $$

Thus,

$$ \begin{aligned} \bigl\|v\cdot\nabla g(y)\bigr\|&\leq\bigl\|v\cdot\nabla g(x)\bigr\|+\bigl\| v\cdot\bigl( \nabla g(y)-\nabla g(x)\bigr)\bigr\| \\ &\leq\bigl(L_G\|\nabla g\|+\|G\|L_g' \bigr)\|x-y\|, \end{aligned} $$

where $L_{g}'$ is the Lipschitz constant of ∇g(⋅). We consider the following three cases:

• If v⋅∇g(y)=0, then $v\in \mathcal {T}_{\varGamma_{j}}(y)$ and we deduce that

  $$w\in G(y)\cap \mathcal {T}_{\varGamma_j}(y)+L_G\|x-y\|B(0,1). $$

• If v⋅∇g(y):=−γ<0, let p:=δ∇g(y)/∥∇g(y)∥, then by (H4)(iv),

  $$p\in G(y)\quad \mbox{and}\quad p\cdot\nabla g(y):=\tilde{\beta}\geq\delta\beta>0. $$

We set

$$q:=\frac{\tilde{\beta}}{\tilde{\beta}+\gamma}v +\frac{\gamma}{\tilde {\beta}+\gamma}p, $$

then q⋅∇g(y)=0, i.e. $q\in \mathcal {T}_{\varGamma_{j}}(y)$. And since G(y) is convex, we have q∈G(y). Then we obtain

$$ \begin{aligned} \|w-q\|&\leq\|w-v\|+\|v-q\| \\ &\leq L_G\|x-y\|+\frac{\gamma}{\tilde{\beta}+\gamma}\|v-p\| \\ &\leq\biggl(L_G+\frac{L_G\|\nabla g\|+\|G\|L_g'}{\delta\beta}2\|G\| \biggr)\|x-y\|, \end{aligned} $$

where we deduce that

$$ w\in G(y)\cap \mathcal {T}_{\varGamma_j}(y)+ L\|x-y\|B(0,1), $$

(A.1)

with $L:=L_{G}+2\|G\|(L_{G}\|\nabla g\|+\|G\|L_{g}')/\delta\beta$.

If v⋅∇g(y)>0, then by the same argument taking p=−δ∇g(y)/∥∇g(y)∥, (A.1) holds true as well.

Finally, (A.1) implies the local Lipschitz continuity of $G(\cdot)\cap \mathcal {T}_{\varGamma_{j}}(\cdot)$ on Γ _j with the local constant L. □

Proof of Lemma 3.6

For k=1,…,m+ℓ, we set

For each $k\in\mathbb{K}(x)$, we have $x\in \mathcal {M}_{k}$. Up to a subsequence, there exists 0≤λ _k≤1 and $p_{k}\in \mathbb {R}^{d}$ so that

$$\frac{\mu^n_k}{t_n}\rightarrow \lambda_k,\qquad \sum _{k\in\mathbb{K}(x)}\lambda_k=1,\qquad \frac{1}{\mu^n_k}\int _{J^n_k}\dot{y}(s)ds\rightarrow p_k $$

as n→+∞. By Proposition 3.4 and the Lipschitz continuity of $F^{\mathit {new}}_{k}$, we have

$$ \begin{aligned} p_k&=\lim_{n\rightarrow \infty}\frac{1}{\mu^n_k} \int_{J^n_k}\dot{y}(s)ds \\ &\in\lim_{n\rightarrow \infty}\frac{1}{\mu^n_k}\int_{J^n_k} F^{\mathit {new}}_k\bigl(y(s)\bigr)ds \\ &\subseteq\lim_{n\rightarrow \infty} \biggl[\frac{1}{\mu^n_k}\int _{J^n_k}F^{\mathit {new}}_k(x)ds+ \frac{1}{\mu^n_k}\int _{J^n_k}L_k\bigl\|y(s)-x\bigr\|B(0,1)ds \biggr] \\ &\subseteq\lim_{n\rightarrow \infty} \biggl[F^{\mathit {new}}_k(x)+L_k \|F\| \biggl[\frac{1}{\mu^n_k}\int_{J^n_k}sds \biggr]B(0,1) \biggr]=F^{\mathit {new}}_k(x). \end{aligned} $$

We then have

$$ \begin{aligned} p&=\lim_{n\rightarrow \infty}\frac{y(t_n)-x}{t_n} =\lim _{n\rightarrow \infty}\frac{1}{t_n}\int^{t_n}_0 \dot{y}(s)ds \\ &=\sum_{k\in\mathbb{K}(x)}\lim_{n\rightarrow \infty} \frac{\mu^n_k}{t_n} \biggl[\frac{1}{\mu^n_k}\int_{J^n_k} \dot{y}(s)ds \biggr] \\ &=\sum_{k\in\mathbb{K}(x)}\lambda_k p_k \in\sum_{k\in\mathbb{K}(x)}\lambda_k F^{\mathit {new}}_k(x) \subseteq \text {co }\bigcup_{k\in\mathbb{K}(x)} F^{\mathit {new}}_k(x). \end{aligned} $$

Now set $\mathcal {M}:=\cup_{k\in\mathbb{K}(x)}\mathcal {M}_{k}$, and since $y(t_{n})\in \mathcal {M}$ for all large n, we have $p\in \mathcal {T}_{\overline{\mathcal {M}}}(x)$. Then we obtain

$$p\in\biggl(\text {co }\bigcup_{k\in\mathbb{K}(x)}F^{\mathit {new}}_k(x) \biggr) \cap \mathcal {T}_{\overline{\mathcal {M}}}(x). $$

The fact that $F^{\mathit {new}}_{k}(z)\subseteq \mathcal {T}_{\mathcal {M}_{k}}(z)$ whenever $z\in \mathcal {M}_{k}$ implies

$$F^{\mathit {new}}_k(x)\cap \mathcal {T}_{\overline{\mathcal {M}}}(x)=F^{\mathit {new}}_k(x) \cap \mathcal {T}_{\overline{\mathcal {M}}_k}(x) $$

whenever $x\in\overline{\mathcal {M}}_{k}$. Hence

 □

Appendix B

We review the background in nonsmooth analysis required in our analysis. A closed set $\mathcal {C}\subseteq \mathbb {R}^{d}$ is called proximally smooth of radius δ>0 provided the distance function $d_{\mathcal {C}}(x):=\inf_{c\in \mathcal {C}}\|c-x\|$ is differentiable on the open neighborhood $\mathcal {C}+\delta B(0,1)$ of $\mathcal {C}$. For any $c\in \mathcal {C}$, we denote the Clarke normal cone by $\mathcal{N}_{\mathcal {C}}(c)$. Recall the tangent cone $\mathcal {T}_{\mathcal {C}}(c)$ at $c\in \mathcal {C}$ is defined as

$$ \mathcal {T}_{\mathcal {C}}(c)= \biggl\{v:\liminf_{t\rightarrow 0^-} \frac{d_{\mathcal {C}}(c+tv)}{t}=0 \biggr\}, $$

and in the case of $\mathcal {C}$ proximally smooth, equals the Clarke tangent cone as the negative polar of $\mathcal{N}_{\mathcal {C}}(c)$:

$$v\in \mathcal {T}_{\mathcal {C}}(c)\quad\Longleftrightarrow\quad\langle\zeta,v\rangle \leq0\quad\forall\zeta\in\mathcal{N}_{\mathcal {C}}(c). $$

If $\mathcal {M}$ is an embedded C ² manifold, $\mathcal {C}:=\overline{\mathcal {M}}$, and $c\in \mathcal {M}$, then $\mathcal {T}_{\mathcal {C}}(c)$ agrees with the usual tangent space $\mathcal {T}_{\mathcal {M}}(c)$ to $\mathcal {M}$ at c from differential geometry (see [9, Proposition 1.9]). If in addition $\overline{\mathcal {M}}$ is proximally smooth, then for each $x\in\overline{\mathcal {M}}$, the tangent cone $\mathcal {T}_{\overline{\mathcal {M}}}(x)$ is closed and convex, and thus has a relative interior denoted by $\text {\emph {r-int} }\mathcal {T}_{\overline{\mathcal {M}}}(x)$. Its relative boundary is defined as $\text {\emph {r-bdry} }\mathcal {T}_{\overline{\mathcal {M}}}(x):=\mathcal {T}_{\overline{\mathcal {M}}}(x)\backslash \text {\emph {r-int} }\mathcal {T}_{\overline{\mathcal {M}}}(x)$.

Another key assumption on the multi-domains is each domain being relatively wedged. A set $\mathcal {C}\subseteq \mathbb {R}^{N}$ is wedged (see [9, p.166]) if at every $x\in \text {bdry }\mathcal {C}$, $\mathit{int}\mathcal {T}_{\overline{\mathcal {C}}}\not=\emptyset$. If $\mathcal {C}=\overline{\mathcal {M}}$ is the closure of an embedded manifold $\mathcal {M}$, then $\mathcal {C}$ relatively wedged means the dimension of $\text {\emph {r-int} }\mathcal {T}_{\overline{\mathcal {M}}_{k}}(x)$ is equal to d _k.

The following result is [5, Lemma 3.1] and is the key geometrical ingredient that permits the construction of boundary trajectories of (DI).

Lemma B.1

If $x\in\overline{\mathcal {M}_{k}}\backslash \mathcal {M}_{k}$and $v\in \text {\emph {r-bdry} }\mathcal {T}_{\overline{\mathcal {M}_{k}}}(x)$, then there exists an indexjfor which $\mathcal {M}_{j}\subseteq \overline{\mathcal {M}_{j}}$, $x\in\overline{\mathcal {M}}_{j}$, and $v\in \mathcal {T}_{\overline{\mathcal {M}_{k}}}(x)$. Of course in this case, one hasd _j<d _k.

Proof

See [5, Lemma 3.1]. □

We finally give a sketch of the proof for Lemma 3.8.

Proof

A key fact is that for any $p\in G(x)\cap \text {\emph {r-int} }\mathcal {T}_{\overline{\mathcal {M}}_{k}}(x)$, there exist τ>0 and a C ¹ trajectory $y(\cdot):[t,\tau]\to{ \mathcal {M}}_{k}\cup\{x\}$ so that

$$ y(t)=x \quad\mbox{and}\quad\dot{y}(t)=p. $$

(B.1)

Let k be such that $x\in\overline{\mathcal {M}}_{k}$ and $p\in G(x)\cap \mathcal {T}_{\overline{\mathcal {M}}_{k}}(x)$. If $p\in \text {\emph {r-int} }\mathcal {T}_{\overline{\mathcal {M}}_{k}}(x)$, then the result follows by the key fact (B.1). If $p\notin \text {\emph {r-int} }\mathcal {T}_{\overline{\mathcal {M}}_{k}}(x)$, then $p\in \text {\emph {r-bdry} }\mathcal {T}_{\overline{\mathcal {M}}_{k}}(x)$ and hence by Lemma B.1, there exists another subdomain $\mathcal {M}_{j}\subseteq \overline{\mathcal {M}}_{k}$ with $x\in\overline{\mathcal {M}}_{j}$ and $p\in \mathcal {T}_{\overline{\mathcal {M}}_{j}}(x)$. If $p\in \text {\emph {r-int} }\mathcal {T}_{\overline{\mathcal {M}}_{j}}(x)$, then the result follows from (B.1), otherwise the argument just given can be repeated with k replaced by j. The process must eventually terminate since the dimension is decreasing at each step. □