Control problems on infinite horizon subject to time-dependent pure state constraints

Abstract

In the last decades, control problems with infinite horizons and discount factors have become increasingly central not only for economics but also for applications in artificial intelligence and machine learning. The strong links between reinforcement learning and control theory have led to major efforts toward the development of algorithms to learn how to solve constrained control problems. In particular, discount plays a role in addressing the challenges that come with models that have unbounded disturbances. Although algorithms have been extensively explored, few results take into account time-dependent state constraints, which are imposed in most real-world control applications. For this purpose, here we investigate feasibility and sufficient conditions for Lipschitz regularity of the value function for a class of discounted infinite horizon optimal control problems subject to time-dependent constraints. We focus on problems with data that allow nonautonomous dynamics, and Lagrangian and state constraints that can be unbounded with possibly nonsmooth boundaries.

1 Introduction

Infinite time horizon models arising in mathematical economics and engineering typically involve control systems with restrictions on both controls and states. Models of optimal allocation of economic resources were, in the late 50 s, among the key incentives for the creation of the mathematical theory of optimal control. Constrained optimal control problems are often solved in practical control applications, which are more challenging to deal with than unconstrained ones.

Over the last few decades, an increasingly central role has been given to infinite horizon control problems with discount factors not only for applications in finance but for applications to artificial intelligence and machine learning. Strong connections between reinforcement learning and control theory have prompted a major effort toward developing algorithms to learn optimal solutions. Discounting plays a role in addressing the challenges that come with models where unbounded disturbances are present. The discount configuration is common in many stochastic control problems [7, 23, 24, 33], reinforcement learning [8], and financial engineering [19, 30]. Discount factors ensure the feasibility of constrained optimal control problems with potentially unbounded perturbations. In dynamic programming, discounting is often used to ensure well-posedness of problems with infinite horizons and possibly unlimited costs [6, 9]. Moreover, with an appropriate value of the discount factor, stability is guaranteed [31].

Much of the present works in the literature focuses on manage constraints in control problems. In general, state constraints imply non-convex feasible sets. So, there are several ways to provide amenable approximations in deterministic and probabilistic frameworks, e.g., using available informations from probability distributions [32], deterministic approximation methods jointly with confidence sets [25], deterministic and stochastic tubes [5, 12], attainable sets [22] or conservative approach with probabilistic inequalities [17, 21] and random methods [11, 26]. Although deterministic algorithms have been widely investigated to solve the optimal regulation problems, few results consider the solution of optimal synthesis in the presence of time-dependent state constraints, needed for most real-world control applications (cfr. Sect. 2). A fundamental point in constrained cases is how to ensure desirable properties, as existence of viable solutions and stability, including regularity of the value function. This is particularly evident in reinforcement learning, where value functions necessitates employing a function approximator with a limited set of parameters. Several researchers have emphasized that integrating reinforcement learning algorithms subject to state constraints with general approximation systems, such as neural networks, fuzzy sets, or polynomial approximators, can lead to unstable or divergent outcomes, even for straightforward problems (cfr. [2, 10, 20]).

In this settings, a key role is given by the dynamic programming principle and the Hamilton–Jacobi–Bellman (HJB) equation associated with the control problem [6, 34]. The value function, when differentiable, solves the HJB equation in the classical sense. However, it is well known that such a kind of notion turns out to be quite unsatisfactory for HJB equations arising in control theory and the calculus of variations (we refer the interested reader to the pioneer works [13, 14] and [3] for further discussions). Indeed, the value function loses the differentiability property whenever there are multiple optimal solutions at the same initial condition or additional state constraints are present. The lack of classical (smooth) solutions to HJB equations for regular data led to the need of a new notion—i.e., weak or viscosity solution—of this equation, in the class of Lipschitz continuous functions. Such regularity is not taken for granted especially when time-dependent state constraints are imposed on the control problem [4, 6].

In this paper, we focus on analyzing the Lipschitz regularity of the value function of infinite horizon control problems, specifically those with discount factors and time-dependent state constraints of a functional type. Our approach is designed to provide a comprehensive and rigorous analysis of this problem. We carefully consider the impact of the presence of time-dependent state constraints and discount factors, as these factors can significantly alter the optimal control strategy and lead to unexpected system behavior. To ensure feasibility and obtain neighboring estimates on the set of feasible trajectories, sufficient conditions on the constraint set by means of inward pointing conditions are imposed (cfr. Sects. 3 and 4). More specifically, by employing recent viability results that were investigated in [3], we establish Lipschitz regularity of the value function and viability of the system. We also demonstrate that the value function vanishes at infinity on the feasible set for all sufficiently large discount factors. This result is significant, as it implies that the value function is bounded on such set, which has important implications for the stability of the system. Overall, our analysis sheds light on the behavior and regularity of weak—or viscosity—solutions of HJB equations.

The outline of the present paper is as follows. In Sect. 2 we describe the general formulation of the optimal control problem addressed here, with notations and backgrounds on nonsmooth analysis. Section 3 is devoted to a controllability condition on constraint set. We give a viability and neighboring estimate results in Sect. 4 for feasible trajectories on infinite horizon. Meanwhile, in Sect. 5 we show the desiderate Lipschitz continuity for the value function.

2 Problem’s formulation and backgrounds

In this paper, we address the following infinite horizon control problem subject to functional constraints

$$\begin{aligned} \text {minimize}\quad \int _t^{+\infty } e^{-\lambda s}{} \textbf{L}(s,x,u) ds \qquad \qquad \qquad \qquad \qquad \qquad \qquad \qquad \qquad \qquad \qquad (\mathcal {P}) \end{aligned}$$

$$\begin{aligned} \text {subject to}\quad&x'=\textbf{f}(s,x,u) \quad \text {a.e. }s\\ \quad&x(t)={\bar{x}}\\ \quad&u(s)\in U(s)\quad \text {a.e. }s\\ \quad&h_1(s, x(s))\le 0 \quad \forall s\ge t \\ \quad&\vdots \\ \quad&h_m(s, x(s))\le 0 \quad \forall s\ge t.\\ \end{aligned}$$

We assume:

• the controls u takes values in $\mathbb {R}^m$ and are Lebesgue measurable;

• U is a measurable set-valued map with nonempty closed images in $\mathbb {R}^m$;

• $h_i$’s are real-valued functions, measurable in time and space-$\Gamma ^{1,\theta }$ regular, uniformly in time;

where $\Gamma ^{1,\theta }$ stands for the class of continuously differentiable functions with $\theta$-Höelder continuous and bounded differential, i.e., for $\theta \in ]0,1[$

$$\begin{aligned} \psi \in \Gamma ^{1,\theta } \quad \Longleftrightarrow \quad&\psi \in C^1 \text { with } \nabla \psi \text { bounded,}\\&\exists k>0:\,|\nabla \psi (x)-\nabla \psi (y)| \le k |x-y|^\theta . \end{aligned}$$

The optimal control problem described above is applicable to several scenarios within the fields of economics and engineering sciences (cfr. [15, 18, 27]). In these fields of applications, functional constraints often appear as functions affine in space with measurable time-dependent terms, specifically, $h_i(s,x)=A(s)x_k+B(s)$ which falls under the framework of the proposed model. This family of functions extends to include $h_i(s,x)=A(s)\psi _i(x)+B(s)$, with $c_i\in {\mathbb {R}}^n$ a parameter and $\psi _i\in \Gamma ^{1,\theta }$. It is worth to notice that the autonomous case with $\theta =1$ was previously studied in [6].

2.1 Preliminaries and notations

Let $B(x,\delta )$ stand for the closed ball in ${\mathbb {R}}^n$ with radius $\delta >0$ centered at $x\in {\mathbb {R}}^n$ and set ${\mathbb {B}}=B(0,1)$, $S^{n-1}=\partial {\mathbb {B}}$. Denote by $|\,\cdot \,|$ and $\langle \cdot , \cdot \rangle$ the Euclidean norm and scalar product, respectively. Let $C\subset {\mathbb {R}}^n$ be a nonempty set. We denote the interior of C by $\textrm{int}\,C$ and the convex hull of C by $\textrm{co}\,C$. The distance from $x\in {\mathbb {R}}^n$ to C is defined by $d_C(x):=\inf \{|x-y|:\,y\in C\}$. If C is closed, we let $\Pi _C(x)$ be the set of all projections of $x\in {\mathbb {R}}^n$ onto C.

For $p\in {\mathbb {R}}^+\cup \{\infty \}$ and a Lebesgue measurable set $I\subset {\mathbb {R}}$ we denote by $L^p(I;{\mathbb {R}}^n)$ the space of ${\mathbb {R}}^n$-valued Lebesgue measurable functions on I endowed with the norm $\Vert \cdot \Vert _{p,I}$. We say that $f\in L^p_{\textrm{loc}}(I;{\mathbb {R}}^n)$ if $f\in L^p(J;{\mathbb {R}}^n)$ for any compact subset $J\subset I$. Let I be an open interval in ${\mathbb {R}}$. For any $f\in L^1_{\textrm{loc}}({\overline{I}};{\mathbb {R}}^n)$ we define $\theta _f:[0,\mu (I))\rightarrow \mathbb {R}^+$ by

$$\begin{aligned} \theta _{ f}(\sigma )=\sup \left\{ \int _{J} |{ f}(\tau )|\,d\tau \,:\, J\subset {\overline{I}},\, \text {Lebesgue measure of }J\leqslant \sigma \right\} . \end{aligned}$$

We denote by ${\mathcal {L}}_{\textrm{loc}}$ the set of all functions $f\in L^1_{\textrm{loc}}(\mathbb {R}^+;{\mathbb {R}}^+)$ such that $\lim _{\sigma \rightarrow 0}\theta _{ f}(\sigma )=0$. Notice that $L^{\infty }(\mathbb {R}^+;{\mathbb {R}}^+)\subset {\mathcal {L}}_{\textrm{loc}}$ and, for any $f\in {\mathcal {L}}_{\textrm{loc}}$, $\theta _{ f}(\sigma )<\infty$ for every $\sigma >0$.

Let $\Omega :\mathbb {R}\rightsquigarrow \mathbb {R}^n$, $F:{\overline{I}}\times {\mathbb {R}}^n\rightsquigarrow {\mathbb {R}}^n$, and $G:{\mathbb {R}}^m\rightsquigarrow {\mathbb {R}}^n$ be set-valued maps with nonempty values. G is said to be L-Lipschitz continuous, for some $L\geqslant 0$, if $G(x)\subset G({\tilde{x}})+ L|x-{\tilde{x}}|{\mathbb {B}}$ for all $x,\,{\tilde{x}}\in {\mathbb {R}}^m$. We say that F has a sub-linear growth (in x) if, for some $c\in L^1_{\textrm{loc}}({\overline{I}};{\mathbb {R}}^+)$,

$$\begin{aligned} \sup _{v\in F(t,x)}|v|\leqslant c (t) (1+|x|) \quad \text {a.e. }t\in {\overline{I}},\; \forall x\in {\mathbb {R}}^n. \end{aligned}$$

Definition 2.1

Let $\gamma \in L^1_{\textrm{loc}}({\overline{I}};{\mathbb {R}}^+)$. We say that F is $\gamma$-left absolutely continuous, uniformly wrt $\Omega$, if

$$\begin{aligned} F(s,x)\subset F(t,x)+\int _s^t \gamma (\tau )\,d\tau {\mathbb {B}}\qquad \forall s,t\in {\overline{I}} :\, s<t,\, \forall x\in \cup _{\tau \in [s,t]}\Omega (\tau ).\nonumber \\ \end{aligned}$$

(2.1)

If F does not depends explicitly from x, in that case we simply say that F is $\gamma$-left absolutely continuous.

If ${\overline{I}}=[S,T]$, then we have the following characterization of uniform absolute continuity from the left: F is $\gamma$-left absolutely continuous, uniformly wrt $\Omega$, for some $\gamma \in L^1_{\textrm{loc}}({\overline{I}};{\mathbb {R}}^+)$, if and only if for every $\varepsilon >0$ there exists $\delta >0$ such that for any finite partition $S\leqslant t_1<\tau _1\leqslant t_2<\tau _2\leqslant ...\leqslant t_m<\tau _m\leqslant T$ of [S, T],

$$\begin{aligned} \sum _{i=1}^m (\tau _i-t_i)<\delta \; \Longrightarrow \; \sum _{i=1}^m exc(F(t_i,x) |F(\tau _i,x)) <\varepsilon \quad \forall \; x\in \cup _{\tau \in [S,T]}\Omega (\tau ) \end{aligned}$$

where the excess of A given B is defined by

$$\begin{aligned} exc(A|B):=\sup \{d_B(a)|a\in A\}\in \mathbb {R}^+\cup \{+\infty \}. \end{aligned}$$

3 Controllability

In what follows, we take the notation

$$\begin{aligned} \Omega (t):=\bigcap _{i=1}^m \Omega _i(t),\quad \Omega _i(t):=\{x\in {\mathbb {R}}^n\,:\, h_i(t,x)\leqslant 0\}. \end{aligned}$$

Consider the following condition

Assumptions 3.1

Let $\theta \in ]0,1[$ and $h_i:\mathbb {R}^+ \times {\mathbb {R}}^n \rightarrow {\mathbb {R}}$ be m real-valued functions satisfying for any $i=1,...,m$:

• $h_i(.,x)$ is measurable for any x.

• $h_i(t,.)$ is $\Gamma ^{1,\theta }$ regular, uniformly wrt t.

The proposition below states a geometric result for a Inward Pointing Field Condition (also known as Inward Pointing Condition) on infinite horizon wrt the constraints $\Omega (t)$ and a vector fields F(t, x).

Proposition 3.1

[Inward Pointing Fields Condition] Consider the Assumptions 3.1. Let $F:\mathbb {R}\times \mathbb {R}^n \rightsquigarrow \mathbb {R}^n$ be a set-valued map with nonempty closed values satisfying

$$\begin{aligned}&\exists M\geqslant 0, \varphi >0:\\&(a) \; \sup \{|v|\,:\,v\in F(t,x),\, t \in \mathbb {R}^+, x\in \partial \Omega (t)\}\leqslant M\\&(b)\;F(t,\cdot ) \text { is }\varphi \text {-Lipschitz continuous for any } t\geqslant 0. \end{aligned}$$

Assume that

$$\begin{aligned} \begin{aligned}&\text {for some} \, \delta>0,\,r>0 \text {and for all } \, t\in \mathbb {R}^+,\,x\in \partial \Omega (t)\\&\text {there exists } \, v\in \textrm{co}\,F(t,x)\, \text {satisfying}\\&\langle \nabla h_i(t,x), v \rangle \leqslant -r\qquad \forall \, i\in \bigcup _{z\in B(x,\delta )} I(z) \end{aligned} \end{aligned}$$

(3.1)

where $I(z)=\{i\in I:\, z\in \partial \Omega _i(t)\}$ and $I:=\{1,...,m\}$. Then

$$\begin{aligned} \begin{aligned}&\text {for some}\,\, \varepsilon>0,\,\eta >0\, \text {and every }\,\, t\in \mathbb {R}^+,\,x\in (\partial \Omega (t)+\eta {\mathbb {B}})\cap \Omega (t)\\&\text {there exists }\, v\in \textrm{co}\,F(t,x)\, \text {satisfying}\\&\{y+[0,\varepsilon ](v+\varepsilon {\mathbb {B}}):\, y\in (x+\varepsilon {\mathbb {B}})\cap \Omega (t)\}\subset \Omega (t). \end{aligned} \end{aligned}$$

(3.2)

Proof

Let us set $J(x):=\bigcup _{z\in B(x,\delta )} I(z)$ for all $x\in \partial \Omega (s)$ and $s\ge 0$. Fix $t\in \mathbb {R}^+,\,x\in \partial \Omega (t)$, and $v\in \textrm{co}\,F(t,x)$ satisfying $\langle \nabla h_i(t,x), v \rangle \leqslant -r$ for all $i\in J(x)$. Pick

$$\begin{aligned}&k> \max _{i\in I} \sup _{x\ne y}\frac{|\nabla h_i(t,x)-\nabla h_i(t,y)|}{|x-y|^\theta }\\&L> \max _{i\in I}\sup _{x\in {\mathbb {R}}^n}|\nabla h_i(t,x)|. \end{aligned}$$

We proceed by steps.

(i): We claim that there exists $\eta '>0$, not depending on (t, x), such that for all $y\in B(x,\eta ')$ we can find $w\in \textrm{co}\,F(t,y)$, with $|w-v|\leqslant r/4\,L$, satisfying for all $i\in J(x)$,

$$\begin{aligned} \langle \nabla h_i(t,y), w \rangle \leqslant -r/2. \end{aligned}$$

Indeed, for all $i\in J(x)$ and $y \in B(x,\root \theta \of {r/4\,kM})$ we have

$$\begin{aligned} \langle \nabla h_i(t,y),v \rangle&= \langle \nabla h_i(t,y)-\nabla h_i(t,x), v \rangle +\langle \nabla h_i(t,x),v \rangle \\&\leqslant kM|y-x|-r\leqslant - \frac{3r}{4} \end{aligned}$$

and for all $w\in {\mathbb {R}}^n$ such that $|w-v|\leqslant r/4L$

$$\begin{aligned} \langle \nabla h_i(t,y), w \rangle&= \langle \nabla h_i(t,y), w-v\rangle + \langle \nabla h_i(t,y), v \rangle \\&\leqslant L|w-v|-3r/4\leqslant - \frac{r}{2}. \end{aligned}$$

Since $F(t,\cdot )$ is $\varphi$-Lipschitz continuous, there exists $w\in \textrm{co}\,F(t,y)$ such that $|w-v|\leqslant r/4L$ whenever $|y-x|\leqslant r/4\varphi L$. So the claim follows with $\eta '=\min \{r/4\varphi L, \root \theta \of {r/4kM}\}$.

(ii): We claim that there exists $\varepsilon '>0$, not depending on (t, x), such that for all $y \in B(x,\eta ')$ we can find $w\in \textrm{co}\,F(t,y)$ such that

$$\begin{aligned} \langle \nabla h_i(t,z),{\tilde{w}} \rangle \leqslant -r/4 \qquad \forall \, z\in B(y,\varepsilon ')\,\forall \, {\tilde{w}} \in B(w,\varepsilon ')\, \forall \, i\in J(x). \end{aligned}$$

Indeed, let $y\in B(x,\eta ')$ and $w\in \textrm{co}\, F(t,y)$ be as in (i). Then for any ${\tilde{w}}\in {\mathbb {R}}^n$ such that $|\tilde{w}-w|\leqslant r/8\,L$ and for all $i\in J(x)$ and $z\in {\mathbb {R}}^n$,

$$\begin{aligned}&\langle \nabla h_i(t,z),{\tilde{w}} \rangle \\&=\langle \nabla h_i(t,z)-\nabla h_i(t,y),{\tilde{w}} \rangle +\langle \nabla h_i(t,y),{\tilde{w}}-w \rangle + \langle \nabla h_i(t,y), w \rangle \\&\leqslant k (M+r/4L+r/8L)|z-y|+r/8-r/2. \end{aligned}$$

So the claim follows with $\varepsilon '= \min \{k^{-1}(M+r/2\,L)^{-1} r/8,\,r/8\,L\}$.

(iii): We prove that there exist $\eta>0,\,\varepsilon >0$, not depending on (t, x), such that for all $y\in B(x,\eta )\cap \Omega (t)$ we can find $w\in \textrm{co}\,F(t,y)$ satisfying

$$\begin{aligned} z+\tau {\tilde{w}}\in \Omega (t)\qquad \forall \, z\in B(y,\varepsilon )\cap \Omega (t),\, \forall \, {\tilde{w}}\in B(w,\varepsilon ),\, \forall \, 0\leqslant \tau \leqslant \varepsilon . \end{aligned}$$

(3.3)

Let $y\in B(x,\eta ')\cap \Omega (t)$ and $w\in \textrm{co}\, F(t,y)$ be as in (ii). Then, by the mean value theorem, for any $\tau \geqslant 0$, any $z\in B(y,\varepsilon ')\cap \Omega (t)$, any ${\tilde{w}}\in B(w,\varepsilon ')$, and any $i\in J(x)$ there exists $\sigma _\tau \in [0,1]$ such that

$$\begin{aligned} h_i(t,z+\tau {\tilde{w}})&=h_i(t,z)+\tau \langle \nabla h_i(t,z+\sigma _\tau \tau {\tilde{w}}), {\tilde{w}} \rangle \\&\leqslant \tau \langle \nabla h_i(t,z),{\tilde{w}} \rangle +k(M+r/4L+\varepsilon ')^2\tau ^2\\&\leqslant -\frac{r\tau }{4}+k(M+r/4L+\varepsilon ')^2\tau ^2. \end{aligned}$$

Choosing $\eta \in ]0, \eta ']$ and $\varepsilon \in ]0, \varepsilon ']$ such that $\eta +\varepsilon (M+r/4L+\varepsilon )\leqslant \delta$ and $\varepsilon \leqslant {k^{-1}(M+r/4L+\varepsilon ')^{-2}r/4}$, it follows that for all $z\in B(y,\varepsilon )\cap \Omega (t),\,{\tilde{w}}\in B(w,\varepsilon )$, and all $0\leqslant \tau \leqslant \varepsilon$

$$\begin{aligned} z+\tau {\tilde{w}}\in B(x,\delta ) \end{aligned}$$

(3.4)

and

$$\begin{aligned} h_i(t,z+\tau {\tilde{w}})\leqslant 0\qquad \forall \, i\in J(x). \end{aligned}$$

(3.5)

Furthermore, by (3.4) and since $B(x,\delta )\subset \Omega _j(t)$ for all $j\in I\backslash J(x)$, we have for all $z\in B(y,\varepsilon )\cap \Omega (t),\,{\tilde{w}}\in B(w,\varepsilon )$, and all $0\leqslant \tau \leqslant \varepsilon$

$$\begin{aligned} h_i(t,z+\tau {\tilde{w}})\leqslant 0\qquad \forall \, i\in I\backslash J(x). \end{aligned}$$

(3.6)

The conclusion follows from (3.5) and (3.6). $\square$

4 Viability and distance estimates on trajectories

We provide here sufficient conditions for uniform linear $L^\infty$ estimates on intervals of the form $I=[t_0,t_1]$, with $0\leqslant t_0<t_1$, for the state constrained differential inclusion

$$\begin{aligned}&x' (t) \in F(t,x (t) )\quad {\text {a.e. } } t\in I\\&x (t) \in \Omega (t)\quad \forall \, t\in I \end{aligned}$$

where $F:\mathbb {R}^+ \times {\mathbb {R}}^n \rightsquigarrow {\mathbb {R}}^n$ is a given set-valued map. A function $x:[t_0,t_1] \rightarrow {\mathbb {R}}^n$ is said to be:

• F-trajectory if it is absolutely continuous and $x' (t) \in F(t,x (t) )$ for a.e. $t\in [t_0,t_1]$.

• feasible F-trajectory if $x (\cdot )$ is an F-trajectory and $x (t) \in \Omega (t)$ for all $t\in I$.

Assumptions 4.1

We assume the following on $F(\cdot ,\cdot )$:

• (1) F has closed and nonempty values, a sub-linear growth, and $F(\cdot ,x)$ is Lebesgue measurable for all $x\in {\mathbb {R}}^n$.

• (2) There exist $M\geqslant 0$ and $\alpha >0$ such that

  $$\begin{aligned} \sup \{|v|\,:\,v \in F(t,x),\,t\in \mathbb {R}^+,\,x\in {\partial \Omega (t)+\alpha {\mathbb {B}}}\} \le M. \end{aligned}$$

• (3) There exists $\varphi \in {\mathcal {L}}_{\textrm{loc}}$ such that $F(t,\cdot )$ is $\varphi (t)$-Lipschitz continuous for all $t\in {\mathbb {R}}^+$.

• (4) There exist ${\tilde{\eta }}>0$ and $\gamma \in \mathcal L_{\textrm{loc}}$ such that F is $\gamma$-left absolutely continuous, uniformly wrt ${\partial \Omega +{\tilde{\eta }} {\mathbb {B}}}$.

Before to state the main result of this section, we recall a definition and a viability result for tubes ([3]-Corollary 4.5).

Definition 4.1

Consider a closed interval $I\subset \mathbb {R}$. We say that a set-valued map $\Phi :I\rightsquigarrow \mathbb {R}^k$ is of locally bounded variations if:

• $\Phi$ takes nonempty closed images.

• For any $[a,b]\subset I$

  $$\begin{aligned} \sup \;\sum _{i=1}^{m-1} exc(\Phi (t_{i+1})\cap \mathscr {K}| {\Phi (t_i)})\vee exc(\Phi (t_i)\cap \mathscr {K}|{\Phi (t_{i+1})})<+\infty \end{aligned}$$

where the supremum is taken over all compact subset $\mathscr {K}\subset \mathbb {R}^k$ and all finite partition $a=t_1< t_2<...< t_{m-1}<t_m= b$.

In the next result, we need to recall the definition of Boulingad (or contingent) cone. Consider a closed set $G\subset \mathbb {R}^n$. The Boulingad tangent cone at $x\in G$ is defined by $\mathscr {T}_G(x):=\{ v\in \mathbb {R}^n:\exists t_i\rightarrow 0+, \exists v_i\rightarrow v, x+t_iv_i \in G \,\forall i \}$.

Proposition 4.1

(Existence of Viable Trajectories, [3] Let $E:\mathbb {R}^+ \rightsquigarrow \mathbb {R}^{d}$ be continuous,¹ of locally bounded variations in the sense of Definition 4.1, and consider $\Phi :\mathbb {R}^+ \times \mathbb {R}^d\rightsquigarrow \mathbb {R}^d$ a set-valued map with nonempty convex closed values such that:

$$\begin{aligned} -&\quad \Phi (.,x) \text { is measurable for any }\, x \\ -&\quad \exists \varrho \in L^1_{loc}(\mathbb {R}^+ ;\mathbb {R}^+ )\\&\quad \forall r>0\,\exists \,k_r: \mathbb {R}^+ \rightarrow \mathbb {R}^+ \text { locally bounded: for a.e. } t\\&\sup _{v\in \Phi (t,x)}|v|\le \varrho (t)(1+|{x}|) \text { and } \Phi (t,.) \text { is }\,\, k_r(t)-\text {Lipschitz on } r\mathbb {B}. \end{aligned}$$

If for a.e. $t>0$ and all $y\in E(t)$ it holds

$$\begin{aligned} \begin{aligned} {\text {cl co }} \mathscr {T}_{\text{ graph } E}(t, y)\cap (\{1\} \times \Phi (t,y))\ne \emptyset , \end{aligned} \end{aligned}$$

(4.1)

then for any $t_0\in \mathbb {R}^+$ and $x_0\in E(t_0)$ there exists an absolutely continuous viable solution

$$\begin{aligned}&x'(t)\in \Phi (t,x(t))\; \text { a.e. }t>t_0\\&x(t_0)=x_0\\&x(t) \in E(t)\quad \forall t>t_0. \end{aligned}$$

Remark 4.1

• (1) Proposition 4.1 extends classical viability results under restricted conditions on the regularity of the tube E (we refer the interested reader to the bibliography therein [3]). Furthermore, it is straightforward to see that Lipschitz continuity for set-valued maps imply the locally bounded variations property.

• (2) We notice that, whenever Proposition 3.1 applies, then condition (3.1) on $\Phi (t,x)=\{\mathbf{{f}}(t,x,u):u\in U(t)\}$ ensure the non-triviality intersection (4.1) for $E(t)=\Omega (t)$.

We have the following

Theorem 4.1

(Neighboring Trajectories Estimates) Consider Assumptions 4.1. Suppose that $h_i$’s satisfy the viability condition (3.1) and there exists $L\ge 0$ such that

$$\begin{aligned} \begin{aligned} \text {the set-valued map } \Omega :\mathbb {R}^+\rightsquigarrow \mathbb {R}^n \text {is }\, L-\text {Lipschitz continuous.} \end{aligned} \end{aligned}$$

(4.2)

Then for every $\delta >0$ there exists a constant $\beta >0$ such that for any $[t_0,t_1]\subset \mathbb {R}^+$ with $t_1-t_0 = \delta$, any F-trajectory ${\hat{x}} (\cdot )$ defined on $[t_0,t_1]$ with ${\hat{x}}(t_0)\in \Omega (t_0)$, and any $\varrho >0$ satisfying

$$\begin{aligned} \sup _{t\in [t_0,t_1]}d_{\Omega (t)}({\hat{x}} (t) )\le \varrho \end{aligned}$$

we can find an F-trajectory $x (\cdot )$ on $[t_0,t_1]$ such that

$$\begin{aligned}&x(t_0)={\hat{x}}(t_0)\\&\Vert {\hat{x}}-x\Vert _{\infty ,[t_0,t_1]}\leqslant \beta \varrho \\&x (t) \in \textrm{int}\; \Omega (t)\qquad \forall \, t\in ]t_0,t_1]. \end{aligned}$$

Proof

Fix $\delta >0$ and let $[t_0,t_1]\subset \mathbb {R}^+$ with $t_1-t_0=\delta$.

We first show the statement whenever $F=\text {co }F$. Let

$$\begin{aligned} \varepsilon>0,\, k>0,\,\Delta>0,\,{\bar{\varrho }}>0,\textrm{and } \;\;m\in {\mathbb {N}}^+ \end{aligned}$$

(4.3)

be such that

$$\begin{aligned} \begin{aligned} \Delta \leqslant \varepsilon ,\quad {\bar{\varrho }}+M\Delta<\varepsilon ,\quad k{\bar{\varrho }} <\varepsilon , \quad k>1/\varepsilon ,\quad 4\Delta M\leqslant {\hat{\eta }} \end{aligned} \end{aligned}$$

(4.4)

$$\begin{aligned} \begin{aligned} e^{\theta _\varphi (\Delta )}(\theta _\gamma (\Delta )+\theta _\varphi (\Delta )M)<\varepsilon ,\; 2e^{\theta _\varphi (\Delta )}(\theta _\gamma (\Delta )+\theta _\varphi (\Delta )M)k<(k\varepsilon -1) \end{aligned} \end{aligned}$$

(4.5)

and

$$\begin{aligned} \frac{\delta }{m}\leqslant \Delta . \end{aligned}$$

(4.6)

Notice that all the constants appearing in (4.3) do not depend on the time interval $[t_0,t_1]$, the trajectory ${\hat{x}} (\cdot )$, and $\varrho$.

1) $\varrho \leqslant {\bar{\varrho }}$ and $\delta \leqslant \Delta$.

We observe that, by the last inequality in (4.4), if

$$\begin{aligned} {\hat{x}}(t_0)\in \Omega (t_0) \backslash (\partial \Omega (t_0)+\frac{{\hat{\eta }} }{2}{\mathbb {B}}) \end{aligned}$$

then $x (\cdot ) ={\hat{x}} (\cdot )$ is as desired. Indeed, without loss of generality, assume ${\hat{x}}(t_0)\in (\partial \Omega (t_0)+\frac{{\hat{\eta }} }{2}S^{n-1} )\cap \Omega (t_0)$ and suppose by contradiction that

$$\begin{aligned} \mathcal {R}:=\{ t\in ]t_0,t_1] : {\hat{x}}(t)\in \text {cl }(\mathbb {R}^n\backslash \Omega (t))\}\ne \emptyset . \end{aligned}$$

Put $s:=\inf R$ and notice that $s\ne t_0$. Then, we have

$$\begin{aligned} d_{\partial \Omega (t_0)}({\hat{x}}(s))\le \text {dist}({\partial \Omega (t_0)}, \partial \Omega (s))+d_{\partial \Omega (s)}( {\hat{x}}(s))\le L |s-t_0| \end{aligned}$$

where $\text {dist}(A,B)$ stands for the standard Euclidean distance between two sets A and B. Since

$$\begin{aligned} d_{\partial \Omega (t_0)}( {\hat{x}}(t_0))-d_{\partial \Omega (t_0)}({\hat{x}}(s))\le |{\hat{x}}(s)-{\hat{x}}(t_0)| \end{aligned}$$

it follows that

$$\begin{aligned} {\hat{\eta }}/4-L |s-t_0|\le M\Delta \le \hat{\eta }/4, \end{aligned}$$

a contradiction. Next we assume that ${\hat{x}}(t_0)\in (\partial \Omega (t_0)+\frac{{\hat{\eta }} }{2}{\mathbb {B}})\cap \Omega (t_0)$.

From Proposition 3.1, let $v\in F(t_0,{\hat{x}}(t_0))$ be as in (3.2) and define

$$\begin{aligned} y:[t_0,t_1] \rightarrow {\mathbb {R}}^n \end{aligned}$$

by

$$\begin{aligned} y(t_0)={\hat{x}}(t_0),\quad y' (t) = \left\{ \begin{array}{ll} v &{}t\in [t_0,(t_0+k\varrho )\wedge t_1]\\ {\hat{x}}'(t-k\varrho )&{}t\in ]t_0+k\varrho ,t_1]\cap J \end{array}\right. \end{aligned}$$

(4.7)

where $J=\{s\in ]t_0+k\varrho ,t_1]:\,{\hat{x}}'(s-k\varrho )\; \; \textrm{exists}\}$. Hence

$$\begin{aligned} \Vert {\hat{x}} -y\Vert _{\infty ,[t_0,t_1]}\leqslant 2Mk\varrho . \end{aligned}$$

(4.8)

By Filippov’s theorem (cfr. [1]) there exists an F-trajectory $x (\cdot )$ on $[t_0,t_1]$ such that $x(t_0)=y(t_0)$ and

$$\begin{aligned} \Vert y-x\Vert _{\infty ,[t_0,t]}\leqslant e^{\int _{t_0}^t \varphi (\tau )\,d\tau }{\int _{t_0}^t d_{F(s,y(s))}(y'(s))}\,ds \end{aligned}$$

(4.9)

for all $t\in [t_0,t_1]$. Then, using Assumptions 4.1–3, (2.1), and (4.7), it follows that

$$\begin{aligned} \qquad \;\;d_{F(s,y(s))}(y'(s))\leqslant \left\{ \begin{array}{ll} \theta _\gamma (\Delta )+\varphi (s)M(s-t_0) &{}\mathrm{a.e. }\; s\in [t_0,(t_0+k\varrho )\wedge t_1]\\ \varphi (s)Mk\varrho +\int _{s-k\varrho }^{s}\gamma (\tau )\,d\tau &{}\mathrm{a.e. }\; s\in ]t_0+k\varrho ,t_1]. \end{array}\right. \end{aligned}$$

Hence, we obtain for any $t\in [t_0,(t_0+k\varrho )\wedge t_1]$

$$\begin{aligned} \int _{t_0}^t d_{F(s,y(s))}(y'(s))\,ds\leqslant (\theta _\gamma (\Delta )+\theta _\varphi (\Delta )M)(t-t_0) \end{aligned}$$

and, using the Fubini theorem, for any $t\in ]t_0+k\varrho ,t_1]$

$$\begin{aligned} \int _{t_0+k\varrho }^t d_{F(s,y(s))}(y'(s))\,ds\leqslant (\theta _\varphi (\Delta )M+\theta _\gamma (\Delta ))k\varrho . \end{aligned}$$

Thus, by (4.9), for all $t\in [t_0,(t_0+k\varrho )\wedge t_1]$

$$\begin{aligned} \Vert y-x\Vert _{\infty ,[t_0,t]}\leqslant e^{\theta _\varphi (\Delta )}(\theta _\gamma (\Delta )+\theta _\varphi (\Delta )M)(t-t_0) \end{aligned}$$

(4.10)

and

$$\begin{aligned} \Vert y-x\Vert _{\infty ,[t_0,t_1]}\leqslant 2e^{\theta _\varphi (\Delta )}(\theta _\gamma (\Delta )+\theta _\varphi (\Delta )M)k\varrho . \end{aligned}$$

(4.11)

Finally, taking note of (4.8), it follows that

$$\begin{aligned} \Vert {\hat{x}}-x\Vert _{\infty ,[t_0,t_1]}\leqslant \beta _1 \varrho , \end{aligned}$$

where we put $\beta _1=2 (M+e^{\theta _\varphi (\Delta )}(\theta _\gamma (\Delta )+\theta _\varphi (\Delta )M))k$.

We claim next that

$$\begin{aligned} x (t) \in \textrm{int}\,\Omega (t)\quad \forall t\in ]t_0,t_1]. \end{aligned}$$

Indeed, if $t\in ]t_0,(t_0+k\varrho )\wedge t_1]$, then from (3.2), the first condition in (4.4), and (4.7) it follows that

$$\begin{aligned} y (t) +(t-t_0)\varepsilon {\mathbb {B}}={\hat{x}}(t_0)+(t-t_0)(v+\varepsilon {\mathbb {B}})\subset \Omega (t) \end{aligned}$$

and it is enough to use (4.10) and the first inequality in (4.5).

On the other hand, if $t\in ]t_0+k\varrho ,t_1]$, then for $\pi (t) \in \Pi _{\Omega (t)}({\hat{x}}(t-k\varrho ))$ we have $|{\hat{x}}(t-k\varrho )-\pi (t) |= d_{\Omega (t)}({\hat{x}}(t-k\varrho ))\leqslant \varrho$, and, from (4.7), it follows that

$$\begin{aligned} y (t) \in \pi (t) +k\varrho v +\varrho {\mathbb {B}}. \end{aligned}$$

(4.12)

Now, since $|\pi (t) -{\hat{x}}(t_0)|\leqslant |{\hat{x}}(t-k\varrho )-\pi (t) |+ |{\hat{x}}(t-k\varrho )-{\hat{x}}(t_0)| \leqslant {\bar{\varrho }}+M\Delta$, from Proposition 3.1 and the $2^{\text {nd}}$ inequality in (4.4)

$$\begin{aligned} \pi (t) +k\varrho v+k\varrho \varepsilon {\mathbb {B}}=\pi (t) +k \varrho (v+\varepsilon {\mathbb {B}})\subset \Omega (t). \end{aligned}$$

(4.13)

Finally, (4.12) and (4.13) imply that $y (t) +(k\varepsilon -1)\varrho {\mathbb {B}}\subset \Omega (t).$ So, the claim follows from (4.5)-(ii) and (4.11).

2) $\varrho > {\bar{\varrho }}$ and $\delta \leqslant \Delta$.

By Proposition 4.1, there exists a feasible F-trajectory ${\bar{x}} (\cdot )$ on $[t_0,t_1]$ starting from ${\hat{x}}(t_0)$. Note that $d_{\Omega (t)}({\bar{x}}(t))=0$ for all $t\in [t_0,t_1]$. By the Case 1, replacing ${\hat{x}} (\cdot )$ with ${\bar{x}} (\cdot )$, it follows that there exists a feasible F-trajectory $x (\cdot )$ on $[t_0,t_1]$ such that $x(t_0)={\hat{x}}(t_0)$ and $x(t)\in \text {int }\Omega (t)$ for all $t\in ]t_0,t_1]$. Hence, by Assumption 4.1-2, we have $\Vert {\hat{x}}-x\Vert _{\infty ,[t_0,t_1]}\leqslant 2\,M\Delta \leqslant \beta _2\varrho ,$ with $\beta _2=\frac{2\,M\Delta }{{\bar{\varrho }}}$.

3) $\delta > \Delta$.

The above proof implies that in Cases 1 and 2, $\beta _1, \, \beta _2$ can be taken the same if $\delta$ is replaced by any $0< \delta _1 < \delta$. Define ${\tilde{\beta }}=\beta _1\vee \beta _2$ and let $\{[\tau _-^i,\tau _+^i]\}_{i=1}^m$ be a partition of $[t_0,t_1]$ by the intervals with the length at most $\delta /m$. Put $x_0 (\cdot ):={\hat{x}} (\cdot )$. From Cases 1 and 2, replacing $[t_0,t_1]$ by $[\tau _-^1,\tau _+^1]$ and setting

$$\begin{aligned} \varrho _0=\varrho \end{aligned}$$

we conclude that there exists an F-trajectory $x_1 (\cdot )$ on $[\tau _-^1,\tau _+^1]=[t_0,\tau _+^1]$ such that $x_1(t_0)={\hat{x}}(t_0)$, $x_1(t)\in \text {int } \Omega (t)$ for all $t\in ]t_0,\tau _+^1]$, and

$$\begin{aligned} \Vert x_1-x_0\Vert _{\infty ,[\tau _-^1,\tau _+^1]}\leqslant {\tilde{\beta }}\varrho _0. \end{aligned}$$

Using Filippov’s theorem, we can extend the trajectory $x_1 (\cdot )$ on whole interval $[t_0,t_1]$ so that

$$\begin{aligned} \Vert x_1-x_0\Vert _{\infty ,[t_0,t_1]}\leqslant e^{\int _{t_0}^{t_1} \varphi (\tau )\,d\tau } {\tilde{\beta }} \varrho _0\leqslant K \tilde{\beta }\varrho _0 \end{aligned}$$

where $K:=e^{\theta _{\varphi }(\delta )}$. Repeating recursively the above argument on each time interval $[\tau _-^i,\tau _+^i]$, we conclude that there exists a sequence of F-trajectories $\{x_i (\cdot ) \}_{i=1}^m$ on $[t_0,t_1]$, such that:

• $x_i(t_0)={\hat{x}}(t_0)$ for all $i=1,...,m$;

• $x_i(t)\in \text {int }\Omega (t)$ for all $t\in ]t_0,\tau _+^i]$ and all $i=1,...,m$;

• $x_j (\cdot ) |_{[t_0,\tau _+^{j-1}]}=x_{j-1} (\cdot )$ for all $j=2,...,m$;

and

$$\begin{aligned} \Vert x_{i}-x_{i-1}\Vert _{\infty ,[t_0,t_1]}\leqslant K {\tilde{\beta }} \varrho _{i-1}\qquad \forall \, i=1,...,m \end{aligned}$$

(4.14)

where

$$\begin{aligned} \varrho _{i-1}=\max \{\varrho , \sup _{t\in [t_0,t_1]}d_{\Omega (t)}(x_{i-1} (t) )\}. \end{aligned}$$

Notice that

$$\begin{aligned} \varrho _{i}\leqslant \varrho _{i-1}+\Vert x_{i}-x_{i-1}\Vert _{\infty ,[t_0,t_1]} \qquad \forall \, i=1,...,m. \end{aligned}$$

(4.15)

Taking note of (4.14) and (4.15) we get for all $i=1,...,m$

$$\begin{aligned} \Vert x_{i}-x_{i-1}\Vert _{\infty ,[t_0,t_1]}&\leqslant K{\tilde{\beta }} (\varrho _{i-2}+\Vert x_{i-1}-x_{i-2}\Vert _{\infty ,[t_0,t_1]})\\&\leqslant K{\tilde{\beta }} (1+K{\tilde{\beta }})\varrho _{i-2}\\&\leqslant ...\\&\leqslant K{\tilde{\beta }} (1+K{\tilde{\beta }})^{i-1}\varrho _0. \end{aligned}$$

Then, letting $x (\cdot ):=x_m (\cdot )$ and observing that $\varrho _0\leqslant \varrho$, we obtain

$$\begin{aligned} \Vert x-{\hat{x}}\Vert _{\infty ,[t_0,t_1]}&\leqslant \sum _{i=1}^{m}\Vert x_i-x_{i-1}\Vert _{\infty ,[t_0,t_1]}\\&\leqslant K{\tilde{\beta }} \varrho _0 \sum _{i=1}^{m}(1+K{\tilde{\beta }})^{i-1}\leqslant \beta _3\varrho , \end{aligned}$$

where $\beta _3=(1+K{\tilde{\beta }})^m-1.$

Then all conclusions of the theorem follow with $\beta ={\tilde{\beta }} \vee \beta _3$. Observe that $\beta$ depends only on $\varepsilon ,{\hat{\eta }}$, M, $\delta$, and on functions $\gamma (\cdot )$ and $\varphi (\cdot )$.

Now, assume $F\ne \text {co }F$. From the first part of the proof, we have that there exist $\beta >0$ (that does not depend on the reference trajectory $\hat{x}(\cdot )$ on $[t_0, t_1]$) and a $\text {co }F$ trajectory ${\bar{x}} (\cdot ):[t_0, t_1] \rightarrow {\mathbb {R}}^{n}$, strictly feasible on $]t_0, t_1]$, such that

$$\begin{aligned} \left\| {\bar{x}} -\hat{x} \right\| _{\infty , [t_0,t_1]} \leqslant \beta \varrho . \end{aligned}$$

Let $\left\{ s_{i}\right\} _i\subset ]t_0, t_1]$ with $s_{1}=t_1$ be a decreasing sequence such that $s_{i} \rightarrow t_0$. Since ${\bar{x}}(\cdot )$ is strictly feasible on $]t_0, t_1]$ we can find a sequence of decreasing numbers $\{\varepsilon _{i}\}_i \subset ]0, \varrho [$ such that $\varepsilon _{i} \rightarrow 0$ and

$$\begin{aligned} \begin{aligned} {\bar{x}} (\sigma )+\varepsilon _{i} {\mathbb {B}} \subset \Omega (\sigma ) \quad \forall \sigma \in \left[ s_{i}, t_1\right] \; \forall i\ge 2. \end{aligned} \end{aligned}$$

(4.16)

Without loss of generality, we can assume that $\varepsilon _i\le \frac{1}{4}\wedge \varrho$ for all $i\in \mathbb {N}^+$. Put $C:=e^{ \int _{t_0}^{t_1} \varphi (\sigma ) d \sigma }$ and define $a_k:=\frac{\varepsilon _k}{C^k}$ for all $k\in \mathbb {N}$. Notice that

$$\begin{aligned} \sum _{k=i}^{\infty } C^{k} a_{k}<\frac{\varepsilon _{i}}{ 2} \quad \forall i \geqslant 2 . \end{aligned}$$

We recall the following known relaxation result.

Lemma 4.1

(Relaxation, [34]) Consider a measurable set-valued map $F: [S,T]\times \mathbb {R}^n \rightsquigarrow \mathbb {R}^{n}$ with closed and nonempty values. Assume that there exist $\varphi ,\psi \in L^{1}(S,T;\mathbb {R})$ such that

$$\begin{aligned} F (t, x)&\subset F (t, y)+\varphi (t) \mathbb {B} \quad \forall t\in [S,T] \; \forall x,y\in \mathbb {R}^n\\ F(t, x)&\subset \psi (t)\mathbb {B} \quad \forall (t, x) \in [S,T]\times \mathbb {R}^n. \end{aligned}$$

Take any feasible $\text {co }F$-trajectory $x(\cdot )$ and any $\varepsilon >0$. Then there exists an F-trajectory $y(\cdot )$ that satisfies $y(S)=x(S)$ and

$$\begin{aligned} \left\| y-x \right\| _{\infty ,[S,T]}<\varepsilon . \end{aligned}$$

From the above lemma, there exist a sequence of F-trajectories $x_{i}:\left[ s_{i}, t_1\right] \rightarrow {\mathbb {R}}^{n}$ such that, for all $i \geqslant 2$, we have $x_{i}\left( s_{i}\right) = {\bar{x}} \left( s_{i}\right)$ and

$$\begin{aligned} \begin{aligned} \left\| x_{i} - {\bar{x}} \right\| _{\infty , [s_i,t_1]} \leqslant a_{i}. \end{aligned} \end{aligned}$$

(4.17)

For each integer $j \geqslant 2$, we construct an F-trajectory $y_{j}:\left[ s_{j}, t_1\right] \rightarrow {\mathbb {R}}^{n}$ as follows: $y_2(\cdot ):=x_2(\cdot )$ and for all $j>2$

• $y_{j}(\cdot )$ is the restriction of $x_{j}(\cdot )$ on $]s_{j}, s_{j-1} ]$.

• for all $1\le k\le j-2$, $y_{j}(\cdot )$ restricted to $]s_{j-k}, s_{j-k-1}]$ is an F-trajectory with initial state $y_{j}\left( s_{j-k}\right)$, obtained by applying Filippov’s theorem with reference trajectory $y_{j-1}(\cdot )$.

Now fix an integer $j>2$. From Filippov’s theorem and since $x_{i} (s_{i} )= {\bar{x}} (s_{i} )$, for any $2 \leqslant i<j$

$$\begin{aligned} \left\| y_{j}-x_{i} \right\| _{\infty , [s_i,s_{i-1}] }&\leqslant C\left| y_{j}\left( s_{i}\right) - {\bar{x}} \left( s_{i}\right) \right| \\ \left\| y'_{j}- x'_{i} \right\| _{1, [s_i,s_{i-1}]}&\leqslant C\left| y_{j}\left( s_{i}\right) - {\bar{x}} \left( s_{i}\right) \right| . \end{aligned}$$

From these relations and (4.17), it follows that for each $2 \leqslant i<j$ and any $l\in \mathbb {N}^+$

$$\begin{aligned} \begin{aligned} \left\| y_{j}- {\bar{x}} \right\| _{\infty , [s_i, s_{i-1}] }&\leqslant \sum _{k=i}^{j} C^{k-i} a_{k} \end{aligned} \end{aligned}$$

(4.18)

$$\begin{aligned} \begin{aligned} \left\| y'_{j+l}- y'_{j} \right\| _{1, [s_i,s_{i-1}]}&\leqslant 2 \sum _{k=i+1}^{j+l} C^{k-i} a_{k}. \end{aligned} \end{aligned}$$

(4.19)

Notice that $y_{j} (s_{j} )= {\bar{x}} (s_{j} )$ for any $j \geqslant 2$. Hence, we can extend each F-trajectory $y_{j}$ as an co F trajectory to whole interval $[t_0, t_1]$, by setting $y_{j}(\sigma )= {\bar{x}} (\sigma )$ for $\sigma \in \left[ t_0, s_{j}\right]$. Since the trajectories $\{y_{i}\}_i$ have initial value $\hat{x}(t_0)$ and owing the sub-linear growth of F, taking a subsequence and keeping the same notation, we have

$$\begin{aligned} \exists \; \text {co } F\text {-trajectory }x(\cdot ): \; y_i\rightarrow x \text { uniformly on }[t_0, t_1], \text { with } x(t_0)=\hat{x}(t_0). \end{aligned}$$

We conclude to show that $x(\cdot )$ satisfy all the conclusions with $\beta$ replaced by $\beta +1$. Indeed, due to (4.19), for each $k \geqslant 2$ the F-trajectories $\{y_{i}\}_i$, restricted to $\left[ s_{k}, s_{k-1}\right]$, forms a Cauchy sequence on $W^{1,1}\left( s_{k}, s_{k-1}\right)$.² So, it follows that the limiting co F-trajectory $x(\cdot )$ is an F-trajectory and, since $\varepsilon _{i} \leqslant \varrho$ for all $i\ge 2$,

$$\begin{aligned} \Vert \hat{x}-x\Vert _{\infty , [t_0,t_1]} \leqslant \left\| \hat{x}- {\bar{x}} \right\| _{\infty , [t_0,t_1]}+\left\| x- {\bar{x}} \right\| _{\infty , [t_0,t_1]} \leqslant ({\beta }+1) \varrho . \end{aligned}$$

Moreover, notice that $x(\cdot )$ is strictly feasible on $]t_0,t_1]$. Indeed, consider $\sigma \in ]t_0, t_1]$. We have $\sigma \in ]s_{i}, s_{i-1}]$ for some $i \geqslant 2$. From (4.18), (4.17), and (4.16) we get

$$\begin{aligned} y_{j}(\sigma ) \in {\bar{x}} (\sigma )+\frac{\varepsilon _{i}}{2} {\mathbb {B}} \subset \text{ int } \Omega (\sigma ) \quad \text{ for } \text{ all } j \geqslant i . \end{aligned}$$

Since the $\{y_{j}\}_j$ converge uniformly to x,

$$\begin{aligned} x(\sigma ) \in {\bar{x}} (\sigma )+\frac{\varepsilon _{i}}{2} {\mathbb {B}} \subset {\text {int}} \Omega (\sigma ) . \end{aligned}$$

This concludes our proof. $\square$

Now, consider the following state constrained differential inclusion:

$$\begin{aligned}&x' (t) \in F(t,x (t) )\quad {\text {a.e. } }t\in [t_0,+\infty [\\&x (t) \in \Omega (t)\quad \forall \, t\in [t_0,+\infty [, \end{aligned}$$

where $t_0\geqslant 0$. A function $x:[t_0,+\infty [ \rightarrow {\mathbb {B}}^n$ is said to be:

• $F_{\infty }$-trajectory if $x|_{[t_0,t_1]} (\cdot )$ is an F-trajectory.

• feasible $F_{\infty }$-trajectory if $x|_{[t_0,t_1]} (\cdot )$ is a feasible F-trajectory for all $t_1> t_0$.

Theorem 4.2

Consider Assumptions 4.1. Suppose that conditions in (3.2) hold true and

$$\begin{aligned} \limsup _{t \rightarrow \infty } \frac{1}{t}\int _0^t \varphi (\tau )\,d\tau <\infty . \end{aligned}$$

Then there exist $C>1$ and $K>0$ such that for any $t_0\geqslant 0$, any $x^0,x^1\in \Omega (t_0)$, and any feasible $F_{\infty }$-trajectory $x:[t_0,+\infty [ \rightarrow {\mathbb {R}}^n$, with $x(t_0)=x^0$, we can find a feasible $F_{\infty }$-trajectory $\tilde{x}:[t_0,+\infty [ \rightarrow {\mathbb {R}}^n$, with ${\tilde{x}}(t_0)=x^1$, such that

$$\begin{aligned} |{\tilde{x}} (t) -x (t)|\leqslant Ce^{{K}t}|x^1-x^0|\qquad \forall \, t\geqslant t_0. \end{aligned}$$

Proof

Let $\delta =1$ and $\beta >0$ be as in Theorem 4.1. Consider $K_1>0,K_2>0,$ and ${\tilde{k}}>0$ such that

$$\begin{aligned} 2\beta +1 <e^{K_1}\quad \textrm{and}\quad \int _{0}^{t+1}\varphi (s)\,ds\leqslant K_2t+{\tilde{k}} \qquad \forall \, t\geqslant 0. \end{aligned}$$

(4.20)

Fix $t_0\ge 0$, $x^0,x^1\in \Omega (t_0)$, with $x^1\ne x^0$, and a feasible ${F}_{\infty }$-trajectory $x:[t_0,+\infty [ \rightarrow {\mathbb {R}}^n$ with $x(t_0)=x_0$. By Filippov’s theorem, there exists an F-trajectory $y_0:[t_0,t_0+1] \rightarrow {\mathbb {R}}^n$ such that $y_0(t_0)=x^1$ and

$$\begin{aligned} \Vert y_0-x\Vert _{\infty ,[t_0,t_0+1]}\leqslant e^{\int _{t_0}^{t_0+1}\varphi (s)\,ds} |x^1-x^0|. \end{aligned}$$

Denote by $x_0:[t_0,t_0+1] \rightarrow {\mathbb {R}}^n$ the feasible F-trajectory, with $x_0(t_0)=x^1$, satisfying the conclusions of Theorem 4.1 with ${\hat{x}} (\cdot ) =y_0 (\cdot )$. Thus

$$\begin{aligned} \Vert x_0-y_0\Vert _{\infty ,[t_0,t_0+1]}&\leqslant \beta (\max _{t\in [t_0,t_0+1]} d_{\Omega (t)}(y_0 (t) )+|x^1-x^0|)\\&\leqslant \beta (\Vert y_0-x\Vert _{\infty ,[t_0,t_0+1]}+|x^1-x^0|)\\&\leqslant 2\beta e^{\int _{t_0}^{t_0+1}\varphi (s)\,ds}|x^1-x^0| \end{aligned}$$

and therefore

$$\begin{aligned} \begin{aligned} \Vert x_0-x\Vert _{\infty ,[t_0,t_0+1]}&\leqslant \Vert x_0-y_0\Vert _{\infty ,[t_0,t_0+1]}+\Vert y_0-x\Vert _{\infty ,[t_0,t_0+1]}\\&\leqslant (2\beta +1)e^{\int _{t_0}^{t_0+1}\varphi (s)\,ds}|x^1-x^0|. \end{aligned} \end{aligned}$$

(4.21)

Now, applying again Filippov’s theorem on $[t_0+1,t_0+2]$, there exists an F-trajectory $y_1:[t_0+1,t_0+2] \rightarrow {\mathbb {R}}^n$, with $y_1(t_0+1)=x_0(t_0+1)$, such that, thanks to (4.21),

$$\begin{aligned} \Vert y_1-x\Vert _{\infty ,[t_0+1,t_0+2]}\leqslant (2\beta +1)e^{\int _{t_0}^{t_0+2}\varphi (s)\,ds}|x^1-x^0|. \end{aligned}$$

(4.22)

Denoting by $x_1:[t_0+1,t_0+2] \rightarrow {\mathbb {R}}^n$ the feasible F-trajectory, with $x_1(t_0+1)=x_0(t_0+1)$, satisfying the conclusions of Theorem 4.1, for ${\hat{x}} (\cdot ) =y_1 (\cdot )$, we deduce from (4.22), that

$$\begin{aligned} \Vert x_1-y_1\Vert _{\infty ,[t_0+1,t_0+2]}\leqslant \beta (2\beta +1)e^{\int _{t_0}^{t_0+2}\varphi (s)\,ds} |x^1-x^0|. \end{aligned}$$

(4.23)

Hence, taking note of (4.22) and (4.23),

$$\begin{aligned} \Vert x_1-x\Vert _{\infty ,[t_0+1,t_0+2]}\leqslant (2\beta +1)^2 e^{\int _{t_0}^{t_0+2}\varphi (s)\,ds}|x^1-x^0|. \end{aligned}$$

Continuing this construction, we obtain a sequence of feasible F-trajectories $x_i:[t_0+i,t_0+i+1] \rightarrow {\mathbb {R}}^n$ such that $x_j(t_0+j)=x_{j-1}(t_0+j)$ for all $j\geqslant 1$, and

$$\begin{aligned} \Vert x_i-x\Vert _{\infty ,[t_0+i,t_0+i+1]}\leqslant (2\beta +1)^{i+1}e^{\int _{t_0}^{t_0+i+1}\varphi (s)\,ds}|x^1-x^0|\;\; \forall \, i\in {\mathbb {N}}.\qquad \quad \end{aligned}$$

(4.24)

Define the feasible ${F}_{\infty }$-trajectory ${\tilde{x}}:[t_0,+\infty [ \rightarrow {\mathbb {R}}^n$ by ${\tilde{x}}(t):=x_i(t)$ if $t\in [t_0+i,t_0+i+1]$ and observe that ${\tilde{x}}(t_0)=x^1$. Let $t\geqslant t_0$. Then there exists $i\in {\mathbb {N}}$ such that $t\in [t_0+i,t_0+i+1]$. So, from (4.24) and (4.20), it follows that

$$\begin{aligned} |{\tilde{x}}(t)-x(t)|&\leqslant (2\beta +1)^{i+1}e^{\int _{t_0}^{t_0+i+1}\varphi (s)\,ds}|x^1-x^0|\\&\leqslant e^{{\tilde{k}}}(2\beta +1) e^{(K_1+K_2)(t_0+i)}|x^1-x^0|\\&\leqslant Ce^{K t}|x^1-x^0|, \end{aligned}$$

where $K=K_1+K_2$ and $C=e^{{\tilde{k}}}(2\beta +1)$. $\square$

5 Lipschitz continuity

Now we give an application of the results of previous sections to the Lipschitz regularity of the value function for a class of infinite horizon optimal control problems subject to state constraints.

Let us consider³ the problem ($\mathcal {P}$) stated in Sect. 2.

Assumptions 5.1

We take the following assumptions on $\textbf{f}$ and $\textbf{L}$:

• (1) For all $x\in {\mathbb {R}}^n$ the mappings $\textbf{f}(\cdot ,x,\cdot ),\, \textbf{L}(\cdot ,x,\cdot )$ are Lebesgue-Borel measurable.

• (2) There exists $\alpha >0$ such that $\textbf{f}$ and $\textbf{L}$ are bounded functions on

  $$\begin{aligned} \{(t,x,u)\,:\,t\geqslant 0,\, x\in (\partial \Omega (t)+\alpha {\mathbb {B}}),\, u\in U (t) \}. \end{aligned}$$

• (3) For all $(t,x)\in \mathbb {R}^+ \times {\mathbb {R}}^{n}$ the set

  $$\begin{aligned} \{(\textbf{f}(t,x,u), \textbf{L}(t,x,u))\,:\,u\in U (t) \} \end{aligned}$$

  is closed.

• (4) There exist $c\in L^1_{\textrm{loc}}(\mathbb {R}^+;{\mathbb {R}}^+)$ and $k\in {\mathcal {L}}_{\textrm{loc}}$ such that for any $t\in {\mathbb {R}}^+,\, x,\,y\in {\mathbb {R}}^n$, and $u\in U (t)$,

  $$\begin{aligned} |\textbf{f}(t,x,u)-\textbf{f}(t,y,u)|+ | \textbf{L}(t,x,u)- \textbf{L}(t,y,u)|\leqslant k (t) |x-y|, \end{aligned}$$
  $$\begin{aligned} |\textbf{f}(t,x,u)|+ | \textbf{L}(t,x,u)|\leqslant c (t) (1+|x|). \end{aligned}$$

• (5) there exist $\tilde{\eta }>0$ and $\gamma \in {\mathcal {L}}_{\text{ loc } }$ such that

  $$\begin{aligned} t\rightsquigarrow \{(\textbf{f}(t,x,u), \textbf{L}(t,x,u)): u\in U(t)\} \end{aligned}$$

  is $\gamma$-left absolutely continuous, uniformly wrt $\partial \Omega +\tilde{\eta } {\mathbb {B}}$.

• (6) $\limsup _{t \rightarrow \infty }\,\frac{1}{t}\int _0^{t} (c(s)+k(s))\,ds <\infty$.

We consider, for any $\lambda >0$, the relaxed infinite horizon state constrained problem

$$\begin{aligned} \text {minimize } \int _{t}^{\infty } e^{-\lambda s}\textbf{L}^\star (s,x,w )\,ds, \qquad \qquad \qquad \qquad \qquad \qquad \qquad \qquad \qquad \qquad \qquad (\mathcal {P}^\star ) \end{aligned}$$

$$\begin{aligned} \text {subject to}\quad&x' = \textbf{f}^\star (s,x ,w ) \quad {\text { a.e. }} s \\ \quad&x(t)={\bar{x}}\\ \quad&w (s) \in W (s) \quad {\text { a.e. }} s \\ \quad&h_1(s, x(s))\le 0 \quad \forall s\ge t \\ \quad&\vdots \\ \quad&h_m(s, x(s))\le 0 \quad \forall s\ge t.\\ \end{aligned}$$

where

• $W:\mathbb {R}^+ \rightsquigarrow {\mathbb {R}}^{(n+1)m}\times {\mathbb {R}}^{n+1}$ is the measurable set-valued map defined by:

  $$\begin{aligned} W (s) :=(\times _{i=0}^n U (s) ) \times \{(\alpha _0,...,\alpha _n)\in {\mathbb {R}}^{n+1}\,:\, \sum _{i=0}^n \alpha _i=1,\, \alpha _i\geqslant 0\,\, \forall \, i\} \end{aligned}$$

  for all $s \geqslant 0$.

• $\textbf{f}^\star : \mathbb {R}^+ \times {\mathbb {R}}^n\times {\mathbb {R}}^{(n+1)m}\times {\mathbb {R}}^{n+1} \rightarrow {\mathbb {R}}^n$ and $\textbf{L}^\star : \mathbb {R}^+ \times {\mathbb {R}}^n\times {\mathbb {R}}^{(n+1)m}\times {\mathbb {R}}^{n+1} \rightarrow {\mathbb {R}}$ are defined by:

  $$\begin{aligned} \textbf{f}^\star (s,x,w)&:=\sum _{i=0}^n \alpha _i \textbf{f}(s,x,u_i)\\ \textbf{L}^\star (s,x,w)&:=\sum _{i=0}^n\alpha _i \textbf{L}(s,x,u_i) \end{aligned}$$

  for all $s \geqslant 0$, $x\in {\mathbb {R}}^n$, and $w=(u_0,...,u_n,\alpha _0,...,\alpha _n)\in {\mathbb {R}}^{(n+1)m}\times {\mathbb {R}}^{n+1}$.

Remark 5.1

• 1. For control systems, the condition (3.2) take the following form: for some $\varepsilon>0,\,\eta >0$ and every $t\in \mathbb {R}^+,\,x\in (\partial \Omega (t)+\eta {\mathbb {B}})\cap \Omega (t)$ there exist $\{\alpha _i\}_{i=0}^n\subset [0,1]$, with $\sum _{i=0}^n\alpha _i=1$, and $\{u_i\}_{i=0}^n\subset U(t)$ satisfying

  $$\begin{aligned} \{y+[0,\varepsilon ] (\sum _{i=0}^n\alpha _i\textbf{f}(t,x,u_i)+\varepsilon {\mathbb {B}} )\,:\, y\in (x+\varepsilon {\mathbb {B}})\cap \Omega (t)\}\subset \Omega (t). \end{aligned}$$

• 2. If there exist ${\tilde{\eta }}>0,\,\gamma ,\,{\tilde{\gamma }}\in {\mathcal {L}}_{\textrm{loc}}$, and $k\geqslant 0$ such that $(\textbf{f},\textbf{L} )$ is $\gamma$-left absolutely continuous, uniformly wrt $(\partial \Omega +{\tilde{\eta }} {\mathbb {B}})\times {\mathbb {R}}^m$, $U (\cdot )$ is ${\tilde{\gamma }}$-left absolutely continuous, and $\textbf{f}(t,x,\cdot )$ is k-Lipschitz continuous for all $t\in \mathbb {R}^+,\,x\in (\partial \Omega (t)+{\tilde{\eta }} {\mathbb {B}})$, then Assumption 5.1-5 holds true.

Definition 5.1

We denote by

$$\begin{aligned} V:\mathcal {Q}_\Omega \rightarrow \mathbb {R}\cup \{\pm \infty \}\quad \text {and} \quad V^\star :\mathcal {Q}_\Omega \rightarrow \mathbb {R}\cup \{\pm \infty \} \end{aligned}$$

the value functions of the infinite horizon control problems ($\mathcal {P}$) and ($\mathcal {P}^\star$), respectively, where

$$\begin{aligned} \mathcal {Q}_\Omega :=\{(t,x)\in \mathbb {R}^+\times \mathbb {R}^n: t\in \mathbb {R}^+,\, x\in \Omega (t)\}. \end{aligned}$$

Next, we state the main result of this section

Theorem 5.1

Consider Assumptions 5.1. Suppose that (3.1) and (4.2) hold true. Then there exist $b>1$ and $K>0$ such that for all $\lambda >K$ we have

• (i) $V^\star (t,\cdot )$ is $b \cdot e^{-(\lambda -K)t}$-Lipschitz continuous on $\Omega (t)$, for any $t\geqslant 0$.

• (ii) $\lim _{t \rightarrow \infty } V^\star (t,x (t) )=0$ for any feasible trajectory $x (\cdot )$.

• (iii) $V^\star =V$ on $\mathcal {Q}_\Omega$.

Proof

We notice that, by Proposition 4.1, the problem ($\mathcal {P}^\star$) admits feasible trajectory-control pairs for any initial condition; using the sub-linear growth of $\textbf{f}$ and the Gronwall lemma, we have $1+|x (t) |\leqslant (1+|x_0|)e^{\int _{t_0}^{t}c(s)\,ds}$ for all $t\geqslant t_0$ and for any trajectory-control pair $(x (\cdot ),u (\cdot ) )$ at $t_0\in \mathbb {R}^+,\,x_0\in \Omega (t_0)$.

In what follows, we define for all $(t,x,z)\in \mathbb {R}^+ \times {\mathbb {R}}^n\times {\mathbb {R}}$ the time-measurable set-valued maps

$$\begin{aligned} G(t,x,z)&=\{(\textbf{f}(t,x,u),e^{-\lambda t} \textbf{L}(t,x,u))\,:\,u\in U (t) \}\\ G^\star (t,x,z)&=\{(\textbf{f}^\star (t,x,u), \textbf{L}^\star (t,x,w))\,:\,w\in W (t) \}. \end{aligned}$$

Next, we show (i). Let $a_1>0,\,a_2>0$ be such that

$$\begin{aligned} \int _{0}^{t}c(s)\,ds\leqslant a_1t+a_2\qquad \forall t\geqslant 0. \end{aligned}$$

(5.1)

For all $T>t_0$, we have

$$\begin{aligned} \begin{array}{ll} \int _{t_0}^{T}e^{-\lambda t}| \textbf{L}^\star (t,x (t),w (t) )|\,dt &{} \leqslant (n+1) \int _{t_0}^{T}e^{-\lambda t} c (t) (1+|x_0|)e^{\int _{t_0}^{t}c(s)\,ds}\,dt\\ &{} \leqslant (n+1)(1+|x_0|)e^{a_2}\int _{t_0}^{T}e^{-(\lambda -a_1)t}c (t) \,dt. \end{array}\nonumber \\ \end{aligned}$$

(5.2)

Then, by (5.1) and denoting $\psi (t) =\int _{t_0}^{t}c(s)\,ds$, for any $\lambda >a_1$

$$\begin{aligned} \begin{array}{ll} &{}\int _{t_0}^{T}e^{-\lambda t}| \textbf{L}^\star (t,x (t),w (t) )|\,dt\\ &{}\quad \leqslant (n+1)(1+|x_0|)e^{a_2}\left( \left[ e^{-(\lambda -a_1)t} \psi (t) \right] _{t_0}^T+(\lambda -a_1)\int _{t_0}^{T}e^{-(\lambda -a_1)t}{\psi (t) }\,dt \right) \\ &{}\quad \leqslant (n+1)(1+|x_0|) e^{a_2}\left( e^{-(\lambda -a_1)T}(a_1T+a_2)+\left( a_1t_0+\frac{a_1}{\lambda -a_1}+a_2\right) e^{-(\lambda -a_1)t_0} \right) . \end{array}\nonumber \\ \end{aligned}$$

(5.3)

Passing to the limit when $T \rightarrow \infty$, we deduce that for every feasible trajectory-control pair $(x (\cdot ),w (\cdot ) )$ at $(t_0,x_0)$

$$\begin{aligned} \int _{t_0}^{\infty }e^{-\lambda t}| \textbf{L}^\star (t,x (t) ,w (t) )|\,dt<+\infty \qquad \forall \lambda >a_1. \end{aligned}$$

From now on, assume that $\lambda >a_1$. Fix $t\geqslant 0$ and $x^1,x^0\in \Omega (t)$ with $x^1\ne x^0$. Then, for any $\delta >0$ there exists a feasible trajectory-control pair $(x_\delta (\cdot ),w_\delta (\cdot ) )$ at $(t,x^0)$ such that

$$\begin{aligned} V^\star (t,x^0)+e^{- \delta t}|x^1-x^0|>\int _{t}^{\infty }e^{-\lambda s} \textbf{L}^\star (s,x_\delta (s),w_\delta (s))\,ds. \end{aligned}$$

Hence

$$\begin{aligned} \begin{array}{ll} &{} V^\star (t,x^1)- V^\star (t,x^0) \leqslant e^{- \delta t}|x^1-x^0| +\\ &{} \lim _{\tau \rightarrow \infty }\left| \int _{t}^{\tau }e^{-\lambda s} \textbf{L}^\star (s, x(s), w(s))\,ds-\int _{t}^{\tau }e^{-\lambda s} \textbf{L}^\star (s,x_\delta (s),w_\delta (s))\,ds\right| \end{array}\qquad \end{aligned}$$

(5.4)

for any feasible trajectory-control pair $( x (\cdot ), w (\cdot ) )$ satisfying $x(t)=x^1$. Consider the following state constrained differential inclusion in ${\mathbb {R}}^{n+1}$

$$\begin{aligned}&(x,z)' (s) \in G^\star (s,x (s) ,z (s) ) \quad {\text { a.e. }}s\in [t,+\infty [\\&x (s) \in \Omega (s)\quad \forall \, s\in [t,+\infty [. \end{aligned}$$

Putting $z_\delta (s)=\int _{t}^{s} \textbf{L}^\star (\xi ,x_\delta (\xi ),w_\delta (\xi ))\,d\xi$, by Theorem 4.2 applied on $\Omega (t)\times {\mathbb {R}}$ and the measurable selection theorem, there exist $C>1$ and $K>0$ such that for all $\delta >0$ we can find a $G^\star _{\infty }$-trajectory $({\tilde{x}}_\delta (\cdot ), {\tilde{z}}_\delta (\cdot ) )$ on $[t,+\infty [$, and a measurable selection ${\tilde{w}}_\delta (s)\in W (s)$ a.e. $s\geqslant t$, satisfying

$$\begin{aligned}&({\tilde{x}}_\delta ,{\tilde{z}}_\delta )' (s) =(\textbf{f}^\star (s,\tilde{x}_\delta (s) , {\tilde{w}}_\delta (s) ), \textbf{L}^\star (s,\tilde{x}_\delta (s),{\tilde{w}}_\delta (s)))\qquad \mathrm{a.e. }\; s\geqslant t,\\&({\tilde{x}}_\delta (t),{\tilde{z}}_\delta (t))=(x^1,0)\\&{\tilde{x}}_\delta (s)\in \Omega (s)\quad \forall s\in [t,+\infty [ \end{aligned}$$

and for any $s\geqslant t$

$$\begin{aligned} |{\tilde{x}}_\delta (s) -x_\delta (s) |+|{\tilde{z}}_\delta (s) -z_\delta (s) |\leqslant Ce^{K s}|x^1-x^0|. \end{aligned}$$

(5.5)

Now, relabelling by K the constant $K\vee a_1$, by (5.5) and integrating by parts, for all $\lambda >K$, all $\tau \geqslant t$, and all $\delta >0$

$$\begin{aligned} \begin{aligned}&\left| \int _{t}^{\tau }e^{-\lambda s} \textbf{L}^\star (s, \tilde{x}_\delta (s),{\tilde{w}}_\delta (s))\,ds-\int _{t}^{\tau }e^{-\lambda s} \textbf{L}^\star (s,x_\delta (s),w_\delta (s))\,ds \right| \\&\leqslant \left| \left[ e^{-\lambda s}\left( \int _t^s \textbf{L}^\star (\xi ,{\tilde{x}}_\delta (\xi ),\tilde{w}_\delta (\xi ))\,d\xi -\int _t^s \textbf{L}^\star (\xi , x_\delta (\xi ),w_\delta (\xi ))\,d\xi \right) \right] _t^\tau \right| \\&\quad +\lambda \left| \int _t^\tau e^{-\lambda s}\left( \int _t^s \textbf{L}^\star (\xi ,{\tilde{x}}_\delta (\xi ),\tilde{w}_\delta (\xi ))\,d\xi -\int _t^s \textbf{L}^\star (\xi , x_\delta (\xi ),w_\delta (\xi ))\,d\xi \right) \,ds \right| \\&\leqslant e^{-\lambda \tau }|\tilde{z}_\delta (\tau )-z_\delta (\tau )|+\lambda \int _t^\tau e^{-\lambda s}|{\tilde{z}}_\delta (s) -z_\delta (s) | ds \\&\leqslant Ce^{-\lambda \tau }e^{K\tau }|x^1-x^0|+\lambda C \int _{t}^\tau e^{-(\lambda -K)s}|x^1-x^0|\,ds\\&=\left( Ce^{-(\lambda -K)\tau }+\lambda C \left[ -\frac{e^{-(\lambda -K)s}}{\lambda -K}\right] _t^\tau \right) |x^1-x^0| \\&=\left( -\frac{CK}{\lambda -K}e^{-(\lambda -K)\tau }+\frac{\lambda C}{\lambda -K}e^{-(\lambda -K)t}\right) |x^1-x^0|\\&\leqslant {\frac{\lambda C}{\lambda -K}e^{-(\lambda -K)t}}|x^1-x^0|. \end{aligned} \end{aligned}$$

(5.6)

Taking note of (5.4), (5.6), and putting $\delta =\lambda -K$, for all $\lambda >K$ we get

$$\begin{aligned} V^\star (t,x^1)- V^\star (t,x^0) \leqslant \left( \frac{\lambda C}{\lambda -K}+1\right) e^{-(\lambda -K)t}|x^1-x^0|. \end{aligned}$$

By the symmetry of the previous inequality with respect to $x^1$ and $x^0$, and since $\lambda$, C, and K do not depend on t, $x^1$, and $x^0$, the statement (i) follows.

Now, let $(t_0,x_0)\in \mathcal {Q}_\Omega$ and consider a feasible trajectory $X (\cdot )$ at $(t_0,x_0)$. Let $t> t_0$ and $(x (\cdot ),w (\cdot ) )$ be a feasible trajectory-control pair at (t, X(t) ) such that $V^\star (t,X (t) )>\int _{t}^{\infty }e^{-\lambda s} \textbf{L}^\star (s,x (s),w (s) )\,ds-\frac{1}{t}$. Then

$$\begin{aligned} | V^\star (t,X (t) )|\leqslant \int _{t}^{\infty }e^{-\lambda s}| \textbf{L}^\star (s,x (s) ,w (s) )|\,ds+\frac{1}{t}. \end{aligned}$$

From (5.1) and (5.2), we have for all $T>t$

$$\begin{aligned}&\int _{t}^{T}e^{-\lambda s}| \textbf{L}^\star (s,x (s) ,w (s) )|\,ds \leqslant \int _{t}^{T}e^{-\lambda s} (1+|X(t)|)e^{\int _{t}^{s}c(\omega )\,d\omega }c (s) \,ds.\\&\leqslant (1+|x_0|)\int _{t}^{T}e^{-\lambda s} e^{\int _{t_0}^{t}c(\omega )\,d\omega }e^{\int _{t}^{s}c(\omega )\,d\omega }c (s) \,ds\\&\leqslant (1+|x_0|)\int _{t}^{T}e^{-\lambda s} e^{\int _{0}^{s}c(\omega )\,d\omega }c (s) \,ds\\&\leqslant (1+|x_0|)e^{a_2}\int _{t}^{T}e^{-(\lambda -a_1) s} c (s) \,ds. \end{aligned}$$

Then, arguing as in (5.3) with $t_0$ replaced by t and taking the limit when $T \rightarrow \infty$, we deduce that

$$\begin{aligned} | V^\star (t,X (t) )|\leqslant (1+|x_0|)e^{a_2} \left( a_1t+\frac{a_1}{\lambda -a_1}+a_2\right) e^{-(\lambda -a_1)t}+\frac{1}{t}. \end{aligned}$$

Since $K\geqslant a_1$, (ii) follows passing to the limit when $t \rightarrow + \infty$.

Next, we show (iii). Notice that $V^\star (t,x)\leqslant V(t,x)$ for any $(t,x)\in \mathcal {Q}_\Omega$, and $V^\star (t,\cdot )$ is Lipschitz continuous on $\Omega (t)$ for all $t\geqslant 0$ whenever $\lambda >0$ is sufficiently large. Fix $t_0\in \mathbb {R}^+,\,x_0\in \Omega (t_0)$, and $\varepsilon >0$. We claim that: for all $j\in {\mathbb {N}}^+$ there exists a finite set of trajectory-control pairs $\{(x_k (\cdot ),u_k (\cdot ) )\}_{k=1,...,j}$ satisfying the following: $x_k' (s) =\textbf{f}(s,x_k' (s),u_k' (s) )$ a.e. $s\in [t_0,t_0+k]$ and $x_k(s)\in \Omega (s)$ for all $s\in [t_0,t_0+k]$ and for all $k=1,...,j$; if $j\geqslant 2$, $x_k|_{[t_0,t_0+k-1]} (\cdot ) =x_{k-1} (\cdot )$ for all $k=2,...,j$; and for all $k=1,...,j$

$$\begin{aligned} V^\star (t_0,x_0)\geqslant V^\star (t_0+k,x_k(t_0+k))+\int _{t_0}^{t_0+k} e^{-\lambda t} \textbf{L}(t,x_k(t),u_k(t))\,dt-\varepsilon \sum _{i=1}^{k}\frac{1}{2^i}.\nonumber \\ \end{aligned}$$

(5.7)

We prove the claim by the induction argument with respect to $j\in {\mathbb {N}}^+$. By the dynamic programming principle, there exists a trajectory-control pair $({\tilde{x}} (\cdot ),{\tilde{w}} (\cdot ) )$ on $[t_0,t_0+1]$, feasible for the problem ($\mathcal {P}^\star$) at $(t_0,x_0)$, such that

$$\begin{aligned} V^\star (t_0,x_0)+\frac{\varepsilon }{4}> V^\star (t_0+1,\tilde{x}(t_0+1))+\int _{t_0}^{t_0+1}e^{-\lambda t} \textbf{L}^\star (t,\tilde{x} (t),{\tilde{w}} (t) )\,dt. \end{aligned}$$

(5.8)

By the relaxation theorem for finite horizon problems (cfr. [34]), for any $h>0$ there exists a measurable control ${\hat{u}}^h(t)\in U(t)$ a.e. $t\in [t_0,t_0+1]$ such that the solution of the equation $({\hat{x}}^h)' (t) =\textbf{f}(t,{\hat{x}}^h (t),{\hat{u}}^h (t) )$ a.e. $t\in [t_0,t_0+1]$, with ${\hat{x}}^h(t_0)=x_0$, satisfies

$$\begin{aligned} \Vert {\hat{x}}^h-{\tilde{x}}\Vert _{\infty ,[t_0,t_0+1]}< h \end{aligned}$$

and

$$\begin{aligned} \left| \int _{t_0}^{t_0+1}e^{-\lambda t} \textbf{L}^\star (t,\tilde{x} (t) ,{\tilde{w}} (t) )\,dt-\int _{t_0}^{t_0+1}e^{-\lambda t} \textbf{L}(t,{\hat{x}}^h (t) ,{\hat{u}}^h (t) )\,dt\right| < h. \end{aligned}$$

Now, consider the following state constrained differential inclusion in ${\mathbb {R}}^{n+1}$

$$\begin{aligned} (x,z)' (s)&\in G(s,x (s) ,z (s) ) \quad \mathrm{a.e. }\; s\in [t_0,t_0+1]\\ x (s)&\in \Omega (s)\quad \forall \, s\in [t_0,t_0+1]. \end{aligned}$$

Letting ${\hat{X}}^h(\cdot )=({\hat{x}}^h(\cdot ),{\hat{z}}^h(\cdot ))$, with ${\hat{z}}^h (t) =\int _{t_0}^{t} e^{-\lambda s} \textbf{L}(s,{\hat{x}}^h (s),{\hat{u}}^h (s) )\,ds$, by Theorem 4.1 and the measurable selection theorem, there exist $\beta >0$ (not depending on $(t_0,x_0)$) such that for any $h>0$ we can find a feasible G-trajectory $X^h (\cdot ) =(x^h (\cdot ),z^h (\cdot ) )$ on $[t_0,t_0+1]$, with $X^h(t_0)=(x_0,0)$, and a measurable control $u^h (s) \in U (s)$ a.e. $s\in [t_0,t_0+1]$, such that

$$\begin{aligned}&( x^h, z^h)'(s)=(\textbf{f}(s, x^h (s) , u^h (s) ),e^{-\lambda s} \textbf{L}(s, x^h(s), u^h(s)))\\&\qquad \mathrm{a.e. }\; s\in [t_0,t_0+1] \end{aligned}$$

and

$$\begin{aligned} \Vert X^h-{\hat{X}}^h\Vert _{\infty ,[t_0,t_0+1]} \leqslant \beta ( \sup _{s\in [t_0,t_0+1]}d_{\Omega (s)\times {\mathbb {R}}}({\hat{X}}^h(s))+h). \end{aligned}$$

Since $\sup _{s\in [t_0,t_0+1]}d_{\Omega (s)\times {\mathbb {R}}}({\hat{X}}^h(s))\leqslant \Vert {\tilde{x}}-{\hat{x}}^h\Vert _{\infty ,[t_0,t_0+1]}$, we have

$$\begin{aligned}&\left| \int _{t_0}^{t_0+1} e^{-\lambda t} \textbf{L}(t, x^h (t) , u^h (t) )\,dt-\int _{t_0}^{t_0+1}e^{-\lambda t} \textbf{L}^\star (t,\tilde{x} (t) ,{\tilde{w}} (t) )\,dt\right| \\&\leqslant \left| \int _{t_0}^{t_0+1}e^{-\lambda t} \textbf{L}^\star (t,{\tilde{x}} (t) ,{\tilde{w}} (t) )\,dt-\int _{t_0}^{t_0+1}e^{-\lambda t} \textbf{L}(t,{\hat{x}}^h (t) ,{\hat{u}}^h (t) )\,dt\right| \\&\qquad +\left| \int _{t_0}^{t_0+1} e^{-\lambda t} \textbf{L}(t, x^h (t) , u^h (t) )\,dt-\int _{t_0}^{t_0+1}e^{-\lambda t} \textbf{L}(t,{\hat{x}}^h (t) ,{\hat{u}}^h (t) )\,dt \right| \\&< h(2\beta +1) \end{aligned}$$

and

$$\begin{aligned} \Vert x^h-{\tilde{x}}\Vert _{\infty ,[t_0,t_0+1]}&\leqslant \Vert \tilde{x}-{\hat{x}}^h\Vert _{\infty ,[t_0,t_0+1]}+\Vert x^h-{\hat{x}}^h\Vert _{\infty ,[t_0,t_0+1]}\\&< h(2\beta +1). \end{aligned}$$

Hence, choosing $0<h<\varepsilon /4(2\beta +1)$ sufficiently small, we can find a trajectory-control pair $(x^h (\cdot ),u^h (\cdot ) )$ on $[t_0,t_0+1]$, with $u^h (s) \in U (s)$ and $(x^h)' (s) =\textbf{f}(s,x^h (s),u^h (s) )$ a.e. $s\in [t_0,t_0+1]$, $x^h(t_0)=x_0$, and $x^h(s)\in \Omega (s)$ for $s\in [t_0,t_0+1]$, such that, by (5.8) and continuity of $V^\star (t_0+1,\cdot )$

$$\begin{aligned} V^\star (t_0,x_0)> V^\star (t_0+1, x^h(t_0+1))+\int _{t_0}^{t_0+1} e^{-\lambda t} \textbf{L}(t, x^h (t) , u^h (t) )\,dt-\frac{\varepsilon }{2}. \end{aligned}$$

Letting $(x_1 (\cdot ),u_1 (\cdot ) ):=(x^h (\cdot ),u^h (\cdot ) )$, the conclusion follows for $j=1$. Now, suppose we have shown that there exist $\{(x_k (\cdot ),u_k (\cdot ) )\}_{k=1,...,j}$ satisfying the claim. Let us to prove it for $j+1$. By the dynamic programming principle there exists a trajectory-control pair $({\tilde{x}} (\cdot ),{\tilde{w}} (\cdot ) )$ on $[t_0+j,t_0+j+1]$, feasible for the problem ($\mathcal {P}^\star$) at $(t_0+j,x_j(t_0+j))$, such that

$$\begin{aligned} \begin{array}{ll} V^\star (t_0+j,x_j(t_0+j))+\frac{\varepsilon }{2^{j+2}}&{}> V^\star (t_0+j+1,{\tilde{x}}(t_0+j+1))\\ &{}\qquad +\int _{t_0+j}^{t_0+j+1}e^{-\lambda t} \textbf{L}^\star (t,{\tilde{x}} (t),{\tilde{w}} (t) )\,dt. \end{array}\qquad \end{aligned}$$

(5.9)

As before, for every $h>0$ there exist a feasible G-trajectory $X^h (\cdot ) =(x^h (\cdot ),z^h (\cdot ) )$ on $[t_0+j,t_0+j+1]$, with $X^h(t_0)=(x_j(t_0+j),0)$, and a measurable control $u^h (s) \in U (s)$ a.e. $s\in [t_0+j,t_0+j+1]$, such that

$$\begin{aligned}&( x^h, z^h)'(s)=(\textbf{f}(s, x^h (s) , u^h (s) ),e^{-\lambda s} \textbf{L}(s, x^h(s), u^h(s))) \\&\qquad \mathrm{a.e. }\;s\in [t_0+j,t_0+j+1] \end{aligned}$$

satisfying

$$\begin{aligned}&\left| \int _{t_0+j}^{t_0+j+1} e^{-\lambda t} \textbf{L}(t, x^h (t) , u^h (t) )\,dt-\int _{t_0+j}^{t_0+j+1}e^{-\lambda t} \textbf{L}^\star (t,{\tilde{x}} (t) ,{\tilde{w}} (t) )\,dt \right| \\&<h(2\beta +1) \end{aligned}$$

and

$$\begin{aligned} \Vert x^h-{\tilde{x}}\Vert _{\infty ,[t_0+j,t_0+j+1]}<h(2\beta +1). \end{aligned}$$

Putting

$$\begin{aligned} (x_{j+1} (\cdot ),u_{j+1} (\cdot ) ):= \left\{ \begin{array}{ll} (x_{j} (\cdot ),u_j (\cdot ) ) &{}\textrm{on }\;\;[t_0,t_0+j]\\ (x^h (\cdot ),u^h (\cdot ) )&{}\textrm{on }\;\;[t_0+j,t_0+j+1] \end{array} \right. \end{aligned}$$

(5.10)

and choosing $0<h<\varepsilon /2^{j+2}(2\beta +1)$ sufficiently small, it follows from (5.9) that

$$\begin{aligned} \begin{array}{ll} V^\star (t_0+j,x_j(t_0+j))&{}\geqslant V^\star (t_0+j+1, x_{j+1}(t_0+j+1))\\ &{}\qquad +\int _{t_0+j}^{t_0+j+1}e^{-\lambda t} \textbf{L}(t, x_{j+1} (t), u_{j+1} (t) )\,dt-\frac{2\varepsilon }{2^{j+2}}. \end{array}\nonumber \\ \end{aligned}$$

(5.11)

So, taking note of (5.10) and (5.11), we obtain

$$\begin{aligned}&V^\star (t_0,x_0)\\&\geqslant V^\star (t_0+j,x_j(t_0+j))+\int _{t_0}^{t_0+j}e^{-\lambda t} \textbf{L}(t,x_j(t),u_j(t))\,dt-\varepsilon \sum _{i=1}^{j}\frac{1}{2^i}\\&\geqslant V^\star (t_0+j+1, x_{j+1}(t_0+j+1)) -\varepsilon \sum _{i=1}^{j}\frac{1}{2^i}-\frac{\varepsilon }{2^{j+1}}\\&\qquad +\int _{t_0+j}^{t_0+j+1}e^{-\lambda t} \textbf{L}(t, x_{j+1} (t) , u_{j+1} (t) )\,dt\\&\qquad +\int _{t_0}^{t_0+j}e^{-\lambda t} \textbf{L}(t,x_j(t),u_j(t))\,dt\\&= V^\star (t_0+j+1, x_{j+1}(t_0+j+1))\\&\qquad +\int _{t_0}^{t_0+j+1}e^{-\lambda t} \textbf{L}(t,x_{j+1}(t),u_{j+1}(t))\,dt-\varepsilon \sum _{i=1}^{j+1}\frac{1}{2^i}. \end{aligned}$$

Hence $\{(x_k (\cdot ),u_k (\cdot ) )\}_{k=1,...,j+1}$ also satisfy our claim. Now, let us define the trajectory-control pair $(x (\cdot ),u (\cdot ) )$ by $(x (t),u (t) ):=(x_k (t),u_k (t) )$ if $t\in [t_0+k-1,t_0+k]$. Then $(x (\cdot ),u (\cdot ) )$ is a feasible trajectory-control pair for the problem ($\mathcal {P}$) at $(t_0,x_0)$. Since, by (ii), $V^\star (t,x (t) ) \rightarrow 0$ when $t \rightarrow +\infty$, from (5.7) we have

$$\begin{aligned} V^\star (t_0,x_0)&\geqslant \int _{t_0}^{\infty }e^{-\lambda t} \textbf{L}(t,x(t),u(t))\,dt-\varepsilon . \end{aligned}$$

Hence, we deduce that $(t_0,x_0)$ lays in the domain of the value function V and so $V^\star (t_0,x_0)\geqslant V(t_0,x_0)-\varepsilon$. From the arbitrariness of $\varepsilon$, the conclusion follows. $\square$

Corollary 5.1

Consider any $N>0$ with

$$\begin{aligned} N \ge \sup \{|\textbf{f}(t,x,u)|+| \textbf{L}(t,x,u)|\,:\, t\geqslant 0,\,x\in {\mathbb {R}}^n,\, u\in U (t) \}<\infty . \end{aligned}$$

Then, for any $\lambda >0$ sufficiently large, for any $t\geqslant 0$ and any $x\in \Omega (t)$, the function $V(\cdot ,x)$ is Lipschitz continuous on $[t,+\infty [$ with constant $\left( L(t)+2e^{-\lambda t}\right) N$ and $L(t):=b e^{-(\lambda -K)t}$.

Proof

From Theorem 5.1, when $\lambda >0$ is large enough, $V(t,\cdot )$ is L(t) -Lipschitz continuous on $\Omega (t)$. Fix $t\geqslant 0$ and $x\in \Omega (t)$. Let $s,{\tilde{s}}\in [t,+\infty [$.

Suppose that $s\geqslant {\tilde{s}}$. Then, by the dynamic programming principle, there exists a feasible trajectory-control pair $({\bar{x}} (\cdot ),{\bar{u}} (\cdot ) )$ at $({\tilde{s}},x)$ such that

$$\begin{aligned} \begin{aligned} V(s,x)-V({\tilde{s}},x)&\leqslant |V(s,x)-V(s,{\bar{x}}(s))|+\int _{\tilde{s}}^{s}e^{-\lambda \xi }| \textbf{L}(\xi ,{\bar{x}}(\xi ),\bar{u}(\xi ))|\,d\xi \\&\qquad +N|s-{\tilde{s}}|e^{-\lambda t}\\&\leqslant L(s)N|s-{\tilde{s}}|+N|s-{\tilde{s}}|e^{-\lambda {\tilde{s}}}+N|s-{\tilde{s}}|e^{-\lambda t}\\&\leqslant {\left( L(t)+2e^{-\lambda t}\right) N}|s-{\tilde{s}}|. \end{aligned} \end{aligned}$$

(5.12)

Arguing in a similar way, we get (5.12) when $s<{\tilde{s}}$. Hence, by the symmetry with respect to s and $\tilde{s}$ in (5.12), the conclusion follows. $\square$

Remark 5.2

The relaxation result in Theorem 5.1 assumes crucial significance when convex data assumptions are absent. By transitioning to the relaxed problem, we ensure that both convergence and Lipschitz regularity remain guaranteed. This approach naturally aligns with the need of machine learning algorithms, in which the desirable property of Lipschitz regularity of the value function improve significantly convergence rates (cfr. [7, 8]). As mentioned earlier in the Introduction, it is well known that incorporating state constraints in the learning process can introduce instability or lead to error divergence in function approximation techniques. Hence, to bolster the overall robustness and reliability of the methods, inward point conditions such as those in (3.1) play a critical role. We refer the reader to [16, 28, 29, 35] for a more comprehensive understanding of the roles of inward pointing conditions and Lipschitz continuity of value functions in convergence guarantees for reinforcement learning with uncertainties and path planning algorithms for autonomous vehicles.

6 Conclusions

This paper presents a method for recovering the feasibility and Lipschitz regularity of the value function for control problems with time-dependent state constraints and infinite horizon discount factor. These results are essential for addressing optimal synthesis and weak solutions to the Hamilton–Jacobi–Bellman equation. We establish sufficient conditions on the constraint set to ensure feasibility and obtain estimates on the neighboring set of feasible trajectories, based on recent viability results. An important contribution of this paper is the demonstration of the equivalence between the master and relaxed infinite horizon problems. Additionally, we prove that the value function approaches zero at infinity for all feasible sets and large discount factors.