From Pure State and Input Constraints to Mixed Constraints in Nonlinear Systems

Abstract

We survey the results on the problem of pure/mixed state and input constrained control, with multidimensional constraints, for finite dimensional nonlinear differential systems with focus on the so-called admissible set and its boundary. The admissible set is the set of initial conditions for which there exist a control and an integral curve satisfying the constraints for all time. Its boundary is made of two disjoint parts: the subset of the state constraint boundary on which there are trajectories pointing towards the interior of the admissible set or tangentially to it; and a barrier, namely a semipermeable surface which is constructed via a generalized minimum-like principle with nonsmooth terminal conditions. Comparisons between pure state constraints and mixed ones are presented on a series of simple academic examples.

1 Introduction

Though constrained systems, namely with restrictions on the control and the state, are present in many applications due to actuator limitations and obstacles, they are not generally studied on their own and are more often studied in the context of optimal control or differential games [8]. We focus here on a fully qualitative approach, i.e., without any optimisation framework where the aim is the construction of the set of initial conditions such that the system variables can satisfy the constraints for all time, called admissible set, and we show how to compute its boundary. Other approaches based on flow computation, or Lyapunov functions, or other variants, may be found in [1, 2, 11,12,13,14, 16,17,18,19,20].

We first review the results of [6] for pure state and input constraints (Sect. 5.2) and present a simple example of double integrator. In a second part (Sect. 5.3), we review their extension to mixed constraints (see [7]) and show, on the double integrator example, how mixed constraints may modify the previously presented behavior. Then another simple example of a spring system is presented in two versions with different mixed constraints and again, we compare their consequences on the respective solutions.

2 Recalls on Pure State and Input Constrained Systems

The material of this section is a summary of [6]. We consider the constrained nonlinear system

$$\begin{aligned}&\dot{x} = f(x,u), \end{aligned}$$

(5.1)

$$\begin{aligned}&x(t_0) = x_0, \end{aligned}$$

(5.2)

$$\begin{aligned}&u \in {\mathcal U}, \end{aligned}$$

(5.3)

$$\begin{aligned}&g_{i}\big (x(t)\big ) \le 0 \quad \forall t \in [t_0, \infty ), \quad \forall i \in \{1, \ldots , p \} \end{aligned}$$

(5.4)

where $x(t)\in {\mathbb R}^{n}$. ${\mathcal U}$ is the set of Lebesgue measurable functions from $[t_0, \infty )$ to U, where U is a compact convex subset of ${\mathbb R}^{m}$, and not a singleton.

The constraint set is defined by

$$ G \triangleq \{x \in {\mathbb R}^n: g_i(x) \le 0, i = 1,\ldots ,p\} $$

The notation $g(x) \circeq 0$ indicates that there exists an $i\in \{1,\ldots ,p\}$ such that x satisfies $g_i(x) = 0$ and $g_j(x)\le 0$ for all $j\in \{1,\ldots ,p\}$, and ${\mathbb I}(x)$ denotes the set of all indices $i\in \{ 1, \ldots , p \}$ such that $g_{i}(x)=0$. Also, $g(x) \prec 0$ (resp. $g(x) \preceq 0$) indicates that $g_{i}(x) < 0$ (resp. $g_{i}(x) \le 0$) for all $i\in \{ 1, \ldots , p \}$.

The sets

$$\begin{aligned} G_0 \triangleq \{x \in {\mathbb R}^n: g(x)\circeq 0 \}, \qquad G_- \triangleq \{x \in {\mathbb R}^n: g(x) \prec 0\}. \end{aligned}$$

(5.5)

are indeed such that $G = G_0 \cup G_-$.

We further assume (see [6])

• (A1) f is at least $C^{2}$ on ${\mathbb R}^{n}\times \tilde{U}$ where $\tilde{U}$ in an open subset of ${\mathbb R}^{m}$, $U\subset \tilde{U}$.

• (A2) There exists a positive and finite constant C such that

  $$ \sup _{u\in U}\vert x^{T}f(x,u) \vert \le C(1+ \Vert x \Vert ^{2} ), \quad {\text {for all}}~x $$

• (A3) The set f(x, U), called the vectogram in [10], is convex for all $x\in {\mathbb R}^{n}$.

• (A4) For each $i=1,\ldots , p$, $g_{i}$ is an at least $C^{2}$ function from ${\mathbb R}^{n}$ to ${\mathbb R}$,

• (A5) the set of points given by $g_{i}(x)=0$ defines an $n-1$ dimensional manifold.

In the sequel we will denote by $x^{(u,x_0)}$ the solution of the differential equation (5.1) with input $u\in {\mathcal U}$ and initial condition $x_0$, and by $x^{(u,x_0)}(t)$ its solution at time t. We also use the notation $x^u$ and $x^u(t)$ when the initial condition is unambiguous or unimportant.

2.1 The Admissible Set

Following [6], we define:

Definition 5.1

(Admissible Set) We say that the point $\bar{x}\in G$ is admissible if, and only if, there exists at least one input function $v\in {\mathcal U}$, such that (5.1)–(5.4) are satisfied for $x_0=\bar{x}$ and $u=v$. The set of all such $\bar{x}$ is called the admissible set:

$$\begin{aligned} {\mathcal A}\triangleq \{\bar{x} \in G: \exists u\in {\mathcal U},~ g\big (x^{(u,\bar{x})}(t)\big ) \preceq 0, \forall t\in [t_0,\infty )\}. \end{aligned}$$

(5.6)

Clearly, if $\bar{x}$ is admissible, any point of the integral curve, $x^{(v,\bar{x})}(t_1)$, $t_1 \in [t_0, \infty )$, with $v\in {\mathcal U}$ as in the above definition, is also an admissible point.

We now recall from [6] the following results:

Proposition 5.1

Assume that (A1)–(A4) are valid. The set ${\mathcal A}$ is closed.

Denote by $\partial \mathcal {A}$ the boundary of the admissible set and define

$$\begin{aligned} \left[ \partial \mathcal {A}\right] _{0}= \partial \mathcal {A}\cap G_0, \quad \left[ \partial \mathcal {A}\right] _{\mathcal {-}}= \partial \mathcal {A}\cap G_-. \end{aligned}$$

(5.7)

We indeed have $\partial \mathcal {A}= \left[ \partial \mathcal {A}\right] _{0}\cup \left[ \partial \mathcal {A}\right] _{\mathcal {-}}$.

2.2 The Barrier

We next consider the subset $\left[ \partial \mathcal {A}\right] _{\mathcal {-}}$ of the boundary of the admissible set.

Definition 5.2

The set $\left[ \partial \mathcal {A}\right] _{\mathcal {-}}$ is called the barrier of the set ${\mathcal A}$.

Still following [6], $\left[ \partial \mathcal {A}\right] _{\mathcal {-}}$ is “fibered” by arcs of integral curves:

Proposition 5.2

Assume that (A1)–(A4) hold. The barrier $\left[ \partial \mathcal {A}\right] _{\mathcal {-}}$ is made of points $\bar{x}\in G_{-}$ for which there exists $\bar{u}\in {\mathcal U}$ and an arc of integral curve $x^{(\bar{u},\bar{x})}$ entirely contained in $\left[ \partial \mathcal {A}\right] _{\mathcal {-}}$ until it intersects $G_0$ at a point $x^{(\bar{u},\bar{x})}(\bar{t})$ for some $\bar{t}\in [t_0,+\infty )$.

Corollary 5.1

(Semi-permeability) From any point on the boundary $\left[ \partial \mathcal {A}\right] _{\mathcal {-}}$, there cannot exist a trajectory penetrating the interior of ${\mathcal A}$, denoted by $\mathsf {int}({\mathcal A})$, before leaving $G_{-}$.

The intersection of $\mathsf {cl}(\left[ \partial \mathcal {A}\right] _{\mathcal {-}})$, the closure of $\left[ \partial \mathcal {A}\right] _{\mathcal {-}}$, with $G_0$ is remarkable:

Proposition 5.3

(Ultimate Tangentiality Condition [6]) Assume that (A1)–(A5) hold and consider $\bar{x} \in \left[ \partial \mathcal {A}\right] _{\mathcal {-}}$ and $\bar{u}\in {\mathcal U}$ as in Proposition 5.2, i.e., such that $x^{(\bar{u},\bar{x})}(t) \in \left[ \partial \mathcal {A}\right] _{\mathcal {-}}$ for all t in some time interval until it reaches $G_0$. Then, there exists a point $z= x^{(\bar{u},\bar{x})}(\bar{t})\in \mathsf {cl}(\left[ \partial \mathcal {A}\right] _{\mathcal {-}})\cap G_0$ for some finite time $\bar{t}\ge t_0$ such that

$$\begin{aligned} \min _{u\in U} \max _{i \in {\mathbb I}(z)} L_{f} g_{i}(z,u) = 0. \end{aligned}$$

(5.8)

where $L_{f}g_{i}(x,u)\triangleq Dg_{i}(x).f(x,u)$ is the Lie derivative of $g_{i}$ along the vector field $f(\cdot ,u)$ at the point x.

Let $H(x,\lambda ,u) = \lambda ^Tf(x,u)$ denote the Hamiltonian.

Theorem 5.1

(Minimum-like principle [6]) Under the assumptions of Proposition 5.3, every integral curve $x^{\bar{u}}$ on $\left[ \partial \mathcal {A}\right] _{\mathcal {-}}\cap \mathsf {cl}(\mathsf {int}({\mathcal A}))$ and the corresponding control function $\bar{u}$, as in Proposition 5.2, satisfies the following necessary condition.

There exists a (nonzero) absolutely continuous maximal solution $\lambda ^{\bar{u}}$ to the adjoint equation

$$\begin{aligned} \dot{\lambda }^{\bar{u}}(t)= - \left( \frac{\partial f}{\partial x}(x^{\bar{u}}(t),\bar{u}(t))\right) ^T\lambda ^{\bar{u}}(t) , \quad \lambda ^{\bar{u}}(\bar{t}) = \left( Dg_{i^{*}}(z)\right) ^{T} \end{aligned}$$

(5.9)

such that the Hamiltonian is minimized

$$\begin{aligned} \min _{u\in U} \left\{ (\lambda ^{\bar{u}}(t))^T f(x^{\bar{u}}(t),u)\right\} =(\lambda ^{\bar{u}}(t))^T f(x^{\bar{u}}(t),\bar{u}(t))=0 \end{aligned}$$

(5.10)

at every Lebesgue point t of $\bar{u}$ (i.e., for almost all $t \le \bar{t}$).

In (5.9), $\bar{t}$ denotes the time at which z is reached, i.e., $x^{\bar{u}}(\bar{t})=z$, with $z\in G_0$ satisfying the ultimate tangentiality condition

$$\begin{aligned} g_{i}(z)=0, \quad i\in {\mathbb I}(z), \quad \min _{u\in U} \max _{i \in {\mathbb I}(z)} L_f g_{i}(z,u) \triangleq L_f g_{i^{*}}(z,\bar{u}(\bar{t})) = 0. \end{aligned}$$

(5.11)

We illustrate this result by the next particularly simple example (double integrator).

2.3 Double Integrator, Pure State Constraint

Let us consider the double integrator subjected to a pure state constraint

$$\begin{aligned} \dot{x}_1 = x_2 , \quad \dot{x}_2 = u , \quad |u|\le 1, \quad x_1 - 1\le 0 \end{aligned}$$

(5.12)

The ultimate tangentiality condition reads $\min _{|u|\le 1} Dg(z).f(z,u) = z_2 = 0$ with $z \triangleq (z_1,z_2) = (x_1^{\bar{u}}(\bar{t}),x_2^{\bar{u}}(\bar{t})) = (1,0)$, $\bar{t}$ indicating the time of tangential arrival on $G_0$ and $\bar{u}$ denoting the control associated to the barrier trajectory. The costate satisfies

$$ \dot{\lambda } = \left( \begin{array}{cc} 0 &{} 0\\ -1 &{} 0 \end{array}\right) \lambda ,\quad \lambda ^{\bar{u}}(\bar{t}) = (1,0) $$

and we deduce $\lambda _1^{\bar{u}}(t)\equiv 1$ and $\lambda _2^{\bar{u}}(t) = -t + \bar{t} > 0$ for all $t\in (-\infty ,\bar{t}]$. We find that the control is given by $\bar{u}(t) = -{\mathsf {sign}}(\lambda _2(t)) \equiv -1$. Integrating backwards from z with $\bar{u}$ gives the parabola-shaped barrier in Fig. 5.1.

Fig. 5.1

Admissible set and barrier for system (5.12)

3 Dynamical Control Systems with Mixed Constraints

The material of this section is borrowed from [7]. We now consider the following constrained nonlinear system:

$$\begin{aligned}&\dot{x} = f(x,u), \end{aligned}$$

(5.13)

$$\begin{aligned}&x(t_0) = x_0, \end{aligned}$$

(5.14)

$$\begin{aligned}&u \in {\mathcal U}, \end{aligned}$$

(5.15)

$$\begin{aligned}&g_i\big (x(t), u(t)\big ) \le 0 \quad \mathrm {for}\,\, a.e.~ t \in [t_0, \infty ) \quad i=1,\ldots ,p \end{aligned}$$

(5.16)

where $x(t)\in {\mathbb R}^{n}$.

As before, ${\mathcal U}$ is the set of Lebesgue measurable functions from $[t_0, \infty )$ to U, with U a given compact convex subset of ${\mathbb R}^{m}$, expressible as

$$U \triangleq \{u\in {\mathbb R}^m : \gamma _j(u)\le 0, j = 1,\ldots ,r\}$$

with $r\ge m$, the functions $\gamma _{j}$ being convex and of class $C^2$.

Let us stress that the constraints (5.16), called mixed constraints [3, 9], depend both on the state and the control. We denote by g(x, u) the vector-valued function whose i-th component is $g_i(x,u)$. As before, by $g(x,u)\prec 0$ (resp. $g(x,u)\preceq 0$) we mean $g_i(x,u) < 0$ (resp. $g_i(x,u) \le 0$) for all i and by $g(x,u)\circeq 0$, we mean $g_i(x,u) = 0$ for at least one i.

We define the following sets:

$$\begin{aligned} \displaystyle G \triangleq \bigcup _{u\in U} \{x\in {\mathbb R}^n : g(x,u)\preceq 0\} \end{aligned}$$

(5.17)

$$\begin{aligned} \displaystyle G_0 \triangleq \{x \in G : \min _{u\in U} \max _{i\in \{1,\ldots ,p\}}g_i(x,u)=0 \} \end{aligned}$$

(5.18)

$$\begin{aligned} \displaystyle G_{-} \triangleq \bigcup _{u\in U} \{x\in {\mathbb R}^n : g(x,u) \prec 0\} \end{aligned}$$

(5.19)

$$\begin{aligned} \displaystyle U(x) \triangleq \{u \in U : g(x,u) \preceq 0 \}\quad \forall x \in G. \end{aligned}$$

(5.20)

Given a pair $(x,u)\in {\mathbb R}^n \times U$, we denote by ${\mathbb I}(x,u)$ the set of indices, possibly empty, corresponding to the “active” mixed constraints, namely

$${\mathbb I}(x,u) = \{i_1,\ldots ,i_{s_{1}} \} \triangleq \{ i\in \{1,\ldots ,p\} : g_{i}(x,u) = 0\}$$

and by ${\mathbb J}(u)$ the set of indices, possibly empty, corresponding to the “active” input constraints:

$${\mathbb J}(u) = \{ j_1,\ldots ,j_{s_{2}}\} \triangleq \{j\in \{1,\ldots r\} : \gamma _{j}(u) = 0\}.$$

The integer $s_{1} \triangleq \#({\mathbb I}(x,u)) \le p$ (resp. $s_{2}\triangleq \#({\mathbb J}(u)) \le r$) is the number of elements of ${\mathbb I}(x,u)$ (resp. of ${\mathbb J}(u)$). Thus, $s_1 + s_2$ represents the number of “active” constraints, among the $p + r$ constraints, at (x, u).

In addition to (A1)–(A4) of the previous section, we assume

• (A6) For all $i=1,\ldots , p$, the mapping $u\mapsto g_{i}(x,u)$ is convex for all $x\in {\mathbb R}^{n}$.

• (A7) The (row) vectors

  $$\begin{aligned} \left\{ \frac{\partial g_{i}}{\partial u}(x,u), \frac{\partial \gamma _{j}}{\partial u}(u) : i\in {\mathbb I}(x,u), j\in {\mathbb J}(u)\right\} \end{aligned}$$

  (5.21)

  are linearly independent at every $(x,u) \in {\mathbb R}^{n}\times U$ for which ${\mathbb I}(x,u)$ or ${\mathbb J}(u)$ is non empty.¹ We say, in this case, that the point x is regular with respect to u (see e.g., [9, 15]).

Given $u\in {\mathcal U}$, we say that an integral curve $x^u$ of Eq. (5.13) defined on $[t_0,T]$ is regular if, and only if, at each Lebesgue point, or shortly L-point, t of u, $x^u(t)$ is regular in the aforementioned sense w.r.t. u(t), and, if t is a point of discontinuity of u, $x^u(t)$ is regular in the aforementioned sense w.r.t. $u(t_-)$ and $u(t_+)$, with $u(t_-) \triangleq \lim _{\tau \nearrow t, t\notin I_0}u(\tau )$ and $u(t_+)\triangleq \lim _{\tau \searrow t, t\notin I_0}u(\tau )$, $I_0$ being a suitable zero measure set of ${\mathbb R}$.

Since system (5.13) is time-invariant, the initial time $t_0$ may be taken as 0. When clear from the context, “$\forall t$” or “for a.e t” will mean “$\forall t \in [0, \infty )$” or “for a.e. $t\in [0, \infty )$”. Note that throughout this paper a.e. is understood with respect to the Lebesgue measure.

3.1 The Admissible Set in the Mixed Case: Topological Properties

Definition 5.1

(Admissible States, Mixed Case) We say that the point $\bar{x} \in G$ is admissible if, and only if, there exists $v\in {\mathcal U}$, such that (5.13)–(5.16) are satisfied for $x_0=\bar{x}$ and $u=v$:

$$\begin{aligned} {\mathcal A}\triangleq \{\bar{x} \in G: \exists u\in {\mathcal U},~ g\big (x^{(u,\bar{x})}(t), u(t)\big ) \preceq 0, \mathrm {for~} a.e.~ t\}. \end{aligned}$$

(5.22)

As before, any point of the integral curve, $x^{(v,\bar{x})}(t')$, $t' \in [0, \infty )$, is also an admissible point.

We assume that both ${\mathcal A}$ and ${\mathcal A}^{\mathsf C}$ contain at least one element to discard the trivial cases ${\mathcal A}= \emptyset$ and ${\mathcal A}^{\mathsf C}= \emptyset$.

We use the notations $\mathsf {int}(S)$ (resp. $\mathsf {cl}(S)$) (resp. $\mathsf {co}(S)$) for the interior (resp. the closure) (resp. the closed and convex hull) of a set S.

Proposition 5.4

Assume that (A1)–(A5) are valid. The set ${\mathcal A}$ is closed.

3.2 Boundary of the Admissible Set (Mixed Case)

3.2.1 Geometric Description of the Barrier

As before, we define the barrier as $\left[ \partial \mathcal {A}\right] _{\mathcal {-}}= \partial \mathcal {A}\cap G_-$.

Proposition 5.5

Assume (A1)–(A4) and (A6) hold. $\left[ \partial \mathcal {A}\right] _{\mathcal {-}}$ is made of points $\bar{x}\in G_-$ for which there exists $\bar{u}\in {\mathcal U}$ and an integral curve $x^{(\bar{u},\bar{x})}$ entirely contained in $\left[ \partial \mathcal {A}\right] _{\mathcal {-}}$ until it intersects $G_0$, i.e., at a point $z = x^{(\bar{u},\bar{x})}(\tilde{t})$, for some $\tilde{t}$, such that $\min _{u\in U} \max _{i=1,\ldots ,p} g_{i}(z,u) = 0$.

The “fibered” nature of the barrier thus extends to the mixed case. Note however that $G_0$ is now modified: it is not defined as the set of x for which there exists $u\in U$ such that $g(x,u)\circeq 0$ but is given by (5.18). Note that $\tilde{t}$ may be infinite, in which case the barrier does not intersect $G_0$ as shown in the next double integrator with mixed constraint example.

Corollary 5.2

(Semi-permeability) Assume (A1)–(A4) and (A6) hold. Then from any point on the boundary $\left[ \partial \mathcal {A}\right] _{\mathcal {-}}$, there cannot exist a trajectory penetrating the interior of ${\mathcal A}$ before leaving $G_{-}$.

3.2.2 Ultimate Tangentiality

We now characterize the intersection of $\left[ \partial \mathcal {A}\right] _{\mathcal {-}}$ with $G_0$ at the point z defined in Proposition 5.5. We define

$$\begin{aligned} \tilde{g}(x)\triangleq \min _{u\in U} \max _{i\in \{1,\ldots ,p\}}g_i(x,u). \end{aligned}$$

(5.23)

Comparing to (5.18) we readily see that $G_0 = \{x\in G : \tilde{g}(x)=0 \}$. According to a result of Danskin [5], $\tilde{g}$ is locally Lipschitz and thus absolutely continuous and almost everywhere differentiable, on every open and bounded subset of ${\mathbb R}^{n}$.

We now recall basic notions from nonsmooth analysis [4] that are used in the next proposition. Consider $h: {\mathbb R}^{n} \rightarrow {\mathbb R}$ Lipschitz near a given point $x \in {\mathbb R}^n$. The generalized directional derivative of h at x in the direction v is defined as follows:

$$\begin{aligned} h^0(x;v) \triangleq \limsup _{y\rightarrow x,\ t\rightarrow 0 ^+} \frac{h(y + tv) - h(y)}{t}. \end{aligned}$$

(5.24)

We also need to introduce the generalized gradient of h at x, labeled $\partial h(x)$. It is well-known that the generalized gradient of a locally Lipschitz function $h:{\mathbb R}^n \rightarrow {\mathbb R}$ is the compact and convex set

$$\begin{aligned} \partial h(x) = \mathsf {co}\{\lim _{i\rightarrow \infty } Dh(x_i): x_i \rightarrow x, x_i \notin \varOmega _1 \cup \varOmega _2 \}\end{aligned}$$

(5.25)

where Dh(x) denotes the row vector Dh(x) at x, $\varOmega _1$ is a zero measure set where h is nondifferentiable and $\varOmega _2$ is an arbitrary zero measure set.

The relationship between the generalized directional derivative and the generalized gradient is given by

$$\begin{aligned} h^0(x;v) = \max _{\xi \in \partial h(x)} \xi v . \end{aligned}$$

(5.26)

Proposition 5.6

(Ultimate Generalized Tangentiality Condition [7]) Assume (A1)–(A4) and (A6)–(A7) hold. Consider $\bar{x} \in \left[ \partial \mathcal {A}\right] _{\mathcal {-}}$ and $\bar{u}\in {\mathcal U}$ as in Proposition 5.5, i.e., such that the integral curve $x^{(\bar{u},\bar{x})}(t)$ remains in $\left[ \partial \mathcal {A}\right] _{\mathcal {-}}$ for all t in some time interval until it reaches $G_{0}$ at some finite time $\bar{t}\ge 0$. Then, the point $z= x^{(\bar{u},\bar{x})}(\bar{t})\in \mathsf {cl}(\left[ \partial \mathcal {A}\right] _{\mathcal {-}})\cap G_{0}$, satisfies

$$\begin{aligned} 0= \max _{\xi \in \partial \tilde{g}(z)} \xi f(z, \bar{u}(\bar{t})) = \min _{v\in U(z)} \max _{\xi \in \partial \tilde{g}(z)} \xi f(z, v) = \max _{\xi \in \partial \tilde{g}(z)} \min _{v\in U(z)} \xi f(z, v). \end{aligned}$$

(5.27)

Moreover, if the function $\tilde{g}$ is differentiable at z, then (5.27) reduces to

$$\begin{aligned} 0 = L_{f}\tilde{g}(z,\bar{u}(\bar{t})) = \min _{u\in U(z)} L_{f}\tilde{g}(z,u). \end{aligned}$$

(5.28)

Remark 5.1

Note that (5.28) significantly differs from (5.8) on several aspects: in (5.28), U(z) replaces U, where z is such that $\tilde{g}(z)=0$; moreover, in (5.28), if $g_{i}$ effectively depends on u for $i\in {\mathbb I}(z)$, $L_{f}\tilde{g}(z,u)$ is not generally equal to $\max _{i\in {\mathbb I}(z)} L_{f} g_{i}(z,u)$.

3.3 The Barrier Equation (Mixed Case)

The next necessary conditions are essential to construct the integral curves running along the barrier.

Theorem 5.2

(Minimum-like Principle (Mixed Case) [7]) Under the assumptions of Proposition 5.6, consider an integral curve $x^{\bar{u}}$ on $\left[ \partial \mathcal {A}\right] _{\mathcal {-}}\cap \mathsf {cl}(\mathsf {int}({\mathcal A}))$ and assume that the control function $\bar{u}$ is piecewise continuous. Then $\bar{u}$ and $x^{\bar{u}}$ satisfy the following necessary conditions.

There exists a nonzero absolutely continuous adjoint $\lambda ^{\bar{u}}$ and piecewise continuous multipliers $\mu _i^{\bar{u}} \ge 0$, $i=1,\ldots ,p$, such that

$$\begin{aligned} \dot{\lambda }^{\bar{u}}(t) = -\left( \frac{\partial f}{\partial x}(x^{\bar{u}}(t),\bar{u}(t)) \right) ^T \lambda ^{\bar{u}}(t) - \sum _{i=1}^{p}\mu _i^{\bar{u}}(t)\frac{\partial g_i}{\partial x}(x^{\bar{u}}(t),\bar{u}(t)) \end{aligned}$$

(5.29)

with the “complementary slackness condition”

$$\begin{aligned} \mu _i^{\bar{u}}(t)g_i(x^{\bar{u}}(t),\bar{u}(t)) = 0, \quad i=1,\ldots , p \end{aligned}$$

(5.30)

and final conditions

$$\begin{aligned} \lambda ^{\bar{u}}(\bar{t})^T \in \arg \max _{\xi \in \partial \tilde{g}(z)} \xi . f(z,\bar{u}(\bar{t})) \end{aligned}$$

(5.31)

where $z = x^{\bar{u}}(\bar{t})$ with $\bar{t}$ such that $z\in G_0$, i.e., $\min _{u\in U}\max _{i=1,\ldots ,p} g_i(z,u) = 0$, $\partial \tilde{g}(z)$ being the generalized gradient of $\tilde{g}$, defined by (5.23), at z.

Moreover, at almost every t, the Hamiltonian

$$H(\lambda ^{\bar{u}}(t),x^{\bar{u}}(t),u) = \left( \lambda ^{\bar{u}}(t) \right) ^Tf(x^{\bar{u}}(t),u)$$

is minimized over the set $U(x^{\bar{u}}(t))$ and equals zero

$$\begin{aligned} \begin{aligned} \min _{u\in U(x^{\bar{u}}(t))} \lambda ^{\bar{u}}(t)^T f(x^{\bar{u}}(t),u)&=\min _{u\in U} \left[ \left( \lambda ^{\bar{u}}(t) \right) ^T f(x^{\bar{u}}(t),u) + \sum _{i=1}^{p} \mu _{i}^{\bar{u}}(t)g_{i}(x^{\bar{u}}(t),u)\right] \\&= \lambda ^{\bar{u}}(t)^T f(x^{\bar{u}}(t),\bar{u}(t)) = 0 \end{aligned} \end{aligned}$$

(5.32)

Remark 5.2

If $\tilde{g}$ is differentiable at the point z, condition (5.31) indeed reduces to its smooth counterpart, i.e., $\lambda ^{\bar{u}}(\bar{t})^T = D\tilde{g}(z)$

Remark 5.3

The assumption that $x^{(\bar{u},\bar{x})}\in \left[ \partial \mathcal {A}\right] _{\mathcal {-}}\cap \mathsf {cl}(\mathsf {int}({\mathcal A}))$ means that we possibly miss isolated trajectories which are in ${\mathcal A}\setminus \mathsf {cl}(\mathsf {int}({\mathcal A}))$. The existence and computation of such trajectories, if they exist, are open questions.

4 Examples

4.1 Double Integrator, Mixed Constraint

Let us go back to the double integrator introduced in Sect. 5.2.3, the pure state constraint $x_{1} \le 1$ being now replaced by the mixed constraint $x_{1} \le u$

$$\begin{aligned} \dot{x}_1 = x_2 , \quad \dot{x}_2 = u , \quad |u|\le 1, \quad x_1 - u \le 0 \end{aligned}$$

(5.33)

We will show that this apparently innocuous change dramatically modifies the admissible set and its barrier since, in the mixed case, the latter does not intersect $G_0$ anymore (compare Figs. 5.1 and 5.2).

We readily get $\tilde{g}(x) = x_1 - 1$ and $G_0 = \{(x_1,x_2) : x_1 = 1 \}$. The ultimate tangentiality condition reads $\min _{u\in U(z)} D\tilde{g}(z).f(z,u) = z_2 = 0$, or $z \triangleq (z_1,z_2) = (1,0)$. Note that, at this point, $U(z)=\{1\}$ is reduced to a single element. The minimal Hamiltonian is given by

$$ \min _{u\in U(x)} \lambda _1 x_2 + \lambda _2 u = 0,\quad \mathrm {a.e.~}t. $$

Thus:

$$ \begin{array}{lll}\mathrm {if}~\lambda _2(t) < 0, &{} &{}\bar{u}(t)=1 \mathrm {~if~}x_1\in ]\infty ,1] \\ \mathrm {if}~\lambda _2(t) > 0, &{} &{}\bar{u}(t)=\left\{ \begin{array}{lll} x_1 &{} \mathrm {if} &{} x_1 \in [-1,1]\\ -1 &{} \mathrm {if} &{} x_1 \in ]-\infty ,-1[ \end{array}\right. \\ \mathrm {if}~\lambda _2(t) = 0, &{} &{}\bar{u}(t)=\mathrm {arbitrary.} \end{array} $$

The costate equations are given by

$$ \dot{\lambda } = \left( \begin{array}{cc} 0 &{} 0 \\ -1 &{} 0 \end{array}\right) \lambda $$

with $\lambda (\bar{t}) = D\tilde{g}(z) = (1,0)^T$. From here we deduce that $\lambda _2(t) = -t + \bar{t}$ for all $t\in (-\infty ,\bar{t}]$, and thus $\lambda _2(t)>0$ for all $t\in (-\infty ,\bar{t}]$. Integrating backwards from the point $z = (1,0)$, we find that the integral curve immediately leaves G, and so this curve cannot be part of the barrier.

However, let us show that the barrier indeed exists and that it remains in $G_-$ for all time.

When the control $\hat{u}(t) = x_1(t)$ is applied to (5.33), the analytic solution initiating at $t= 0$ from $x_0 \triangleq (x_{1,0},x_{2,0})$ is given by

$$\begin{aligned} x_1^{(\hat{u},x_0)}(t)&= \frac{x_{1,0} + x_{2,0}}{2} e^{t} + \frac{x_{1,0} - x_{2,0}}{2} e^{-t} \\ x_2^{(\hat{u},x_0)}(t)&= \frac{x_{1,0} + x_{2,0}}{2} e^{t} - \frac{x_{1,0} - x_{2,0}}{2} e^{-t}. \end{aligned}$$

It is thus immediately seen that, with this control, the origin is a saddle point, the line $x_1 + x_2 = 0$ being the associated stable manifold, and $x_1 - x_2 = 0$ the unstable one.

We now prove that the line segment ${\mathcal L}\triangleq \{(x_1,x_2) : x_1 + x_2 = 0, -1 \le x_1 < 1\}$ is a subset of $\left[ \partial \mathcal {A}\right] _{\mathcal {-}}$.

Clearly, $\mathcal {L}$ is positively invariant and every integral curve starting on it asymptotically approaches the origin. Moreover, $g(x^{(\hat{u},x_0)}(t),\hat{u}(t)) = x_{1}^{(\hat{u},x_0)}(t) - \hat{u}(t) = 0$ for all t such that $-1 \le x_{1}^{(\hat{u},x_0)}(t) < 1$. Let $h(x) \triangleq x_1 + x_2$ and denote $x_i(t) \triangleq x_i^{(\hat{u},x_0)}(t)$, $i=1,2$, for simplicity’s sake. If at a suitable time $t_1$, the state satisfies $x(t_1)\in {\mathcal L}$, i.e. with $h(x(t_1)) = 0$, using any other admissible control $v > \hat{u}(t_1) = x_1(t_1)$, with $\vert v \vert \le 1$, we get

$$ Dh(x(t_1)).f(x(t_1),v) = x_2(t_1) + v > -x_1(t_1) + x_1(t_1) = 0. $$

Therefore, any other admissible control results in the state entering the set ${\mathcal B}\triangleq \{ h(x)=x_1 + x_2 > 0 \}$. Moreover, in ${\mathcal B}$, all trajectories are such that h is non-decreasing for all admissible control v: $L_f h (x, v)= x_{2} + v > -x_1 + x_1 = 0$ as long as $x_1 \le 1$, which implies that all trajectories starting from ${\mathcal B}$ cross the constraint $x_1=1$ and hence are not admissible, i.e., ${\mathcal B}\subset {\mathcal A}^{\mathsf C}$. Moreover, starting from any point in the complement, i.e., such that $x_1 + x_2 \le 0$, denoted by ${\mathcal C}$ in Fig. 5.2, it is straightforward to verify that $\hat{u}$ ensures that the corresponding integral curve remains in G for all time which proves the assertion that ${\mathcal L}$ is a subset of $\left[ \partial \mathcal {A}\right] _{\mathcal {-}}$.

We now prove that the barrier extends, for $x_{2}> 1$, by the integral curve starting backwards from the point $(x_1,x_2) = (-1,1)$, with the control $\bar{u}(t) \equiv -1$ for all $t\in ]-\infty ,\bar{t}]$.

By Theorem 5.2, assuming that $\bar{u}$ is piecewise continuous, any trajectory running along the barrier, generated by $\bar{u}$, satisfies Eqs. (5.29), (5.30) and (5.32) with absolutely continuous costate $\lambda ^{\bar{u}}$ and piecewise continuous multipliers $\mu ^{\bar{u}}$.

Consider the end point of ${\mathcal L}$, denoted by $\xi$, of coordinates $\xi _1 = -1, \xi _2=1$. The set $U(\xi )$ at that point is equal to $[-1,1]$. By (5.32) we must have

$$ \min _{u\in U(\xi )} \lambda ^{\bar{u}}(t)^T f(\xi ,u) = \min _{u\in [-1,1]} \lambda ^{\bar{u}}_1(t) + \lambda ^{\bar{u}}_2(t)u =0 $$

and, by continuity of the Hamiltonian on ${\mathcal L}$, since we had $\bar{u}=x_1$, considering the limit of the Hamiltonian for $x\rightarrow \xi$, $x\in {\mathcal L}$, we deduce that the costate $(\lambda ^{\bar{u}}_1(t), \lambda ^{\bar{u}}_2(t))^T$ is orthogonal to the vector $(1,-1)^T$, i.e., $\lambda (\bar{t}) = k(1,1)^T$, with k a positive constant, and the minimizing $\bar{u}$ is thus $\bar{u}(t) = -{\mathsf {sign}}(\lambda _2(t)) = -1$. Therefore, in $\left[ \partial \mathcal {A}\right] _{\mathcal {-}}\setminus {\mathcal L}$, since $x_1 < -1$, the constraint $x_1 - u$ is nowhere active and $\mu ^{\bar{u}}=0$ by (5.30). Thus the costate equation reads

$$ \dot{\lambda } = \left( \begin{array}{cc} 0 &{} 0 \\ -1 &{} 0 \end{array}\right) \lambda ,\quad \lambda (\bar{t}) = k(1,1) $$

from which we deduce that $\lambda _1(t)\equiv k$ and $\lambda _2(t) = -k(t - \bar{t}) + k$, $t\in (-\infty ,\bar{t}]$ and $\bar{u}(t) = -{\mathsf {sign}}(\lambda _2(t)) \equiv -1$. Note that this solution indeed satisfies the piecewise continuous assumption of $\bar{u}$ in Theorem 5.2. The barrier is thus further extended backwards as in Fig. 5.2. We have also included a few of the vectograms along the extension of the barrier in order to emphasize that this is indeed an “extremal” trajectory and that as we approach the point $(-1,1)$, the vectogram points towards the set $\mathcal {B}$, which we have shown to be a subset of ${\mathcal A}^{\mathsf C}$.

Fig. 5.2

Figure showing some of the sets referred to in Sect. 5.4.1, along with a curve obtained by backward integration from the point $(-1,1)$ which we have shown to be the backward extension of the barrier

4.2 Constrained Spring I

Consider the following constrained mass–spring–damper model:

$$ \left( \begin{array}{c} \dot{x}_{1}\\ \dot{x}_{2} \end{array} \right) = \left( \begin{array}{cc} 0 &{} 1\\ -2 &{} -2 \end{array} \right) \left( \begin{array}{c} x_{1}\\ x_{2} \end{array} \right) +\left( \begin{array}{c} 0\\ 1 \end{array} \right) u ,\,\,\,\,\,|u|\le 1, \,\,\,\,\,x_2 - u \le 0 $$

where $x_1$ is the mass’s displacement. The spring stiffness is here equal to 2 for a mass equal to 1 and the friction coefficient is equal to 2. u is the force applied to the mass.

We identify $g(x,u) = x_2 - u$, $U = [-1,1]$ and $\tilde{g}(x) = x_2 - 1$. We also identify the following sets: $G = \{x\in \mathbb {R}^2:x_2\le 1\}$, $G_{0} = \{x\in G: x_2 = 1\}$ and $U(x) = \{u\in U : x_2\le u \le 1\}$. Note that if $z \triangleq (z_1,z_2)\in G_0$, i.e. $z_2 = 1$, then U(z) is the singleton $U(z) = \{1\}$.

We have $\partial \tilde{g}(z) = \{ (0,1) \} = D\tilde{g}(z)$ ($\tilde{g}$ being indeed differentiable everywhere) and the ultimate tangentiality condition reads

$$ \min _{u\in U(z)} D \tilde{g}(z) f(z,u) = 0 $$

which gives

$$ \min _{u\in U(z)} -2z_1 - 2z_2 + u = -2z_1 - 2 + 1 = 0 $$

Thus $z = (-\frac{1}{2},1)$.

The final costate $\lambda (\bar{t})$, according to (5.31), which here reduces to (5.28), is given by $\lambda ^T(\bar{t}) = D\tilde{g}(z) = (0,1)$.

The Hamiltonian being here $H(x,\lambda ,u) = \lambda _1x_2 + \lambda _2(-2x_1 - 2x_2 + u)$, condition (5.32) reads

$$\begin{aligned} \min _{x_2\le u \le 1} \lambda _1x_2 + \lambda _2(-2x_1 - 2x_2 + u) = 0 \end{aligned}$$

(5.34)

which gives the control $\bar{u}$ associated with the barrier

$$ \begin{array}{lll} \mathrm {if}~\lambda _2(t) < 0, &{}~ &{}\bar{u}(t)=1 \\ \mathrm {if}~\lambda _2(t) > 0, &{} &{}\bar{u}(t)=\left\{ \begin{array}{lll} x_2 &{} \mathrm {if} &{} x_2 \in ]-1,1]\\ -1 &{} \mathrm {if} &{} x_2 \in ]-\infty ,-1] \end{array}\right. \\ \mathrm {if}~\lambda _2(t) = 0, &{} &{}\bar{u}(t)=\mathrm {arbitrary} \end{array} $$

We note from condition (5.29) that if the constraint is active (i.e., $g(x,u) = 0$), the costate differential equation is given by

$$\begin{aligned} \dot{\lambda }^{\bar{u}} = -\frac{\partial f}{\partial x}^T\lambda ^{\bar{u}} - \mu ^{\bar{u}}\frac{\partial g}{\partial x} = \left( \begin{array}{cc} 0 &{} 2\\ -1 &{} 2 \end{array}\right) \lambda ^{\bar{u}} - \mu ^{\bar{u}} \left( \begin{array}{c} 0 \\ 1 \end{array}\right) \end{aligned}$$

(5.35)

and is otherwise (when $g(x,u)<0$) given by

$$\begin{aligned} \dot{\lambda }^{\bar{u}} = -\frac{\partial f}{\partial x}^T\lambda ^{\bar{u}} = \left( \begin{array}{cc} 0 &{} 2\\ -1 &{} 2 \end{array}\right) \lambda ^{\bar{u}}. \end{aligned}$$

(5.36)

Recall that $\lambda _2(\bar{t})>0$ and $x_2(\bar{t})= z_2=1 > 0$. Therefore, because $\lambda$ and x are continuous, $\bar{u}(t) = x_2(t)$ over an interval before $\bar{t}$. We can show that $\bar{u}(t) \ne 1$ over this interval: if $x_2 = 1$ and $u = 1$ over an interval before $\bar{t}$, then we get $\dot{x}_2 = -2x_1 - 2 + 1 = 0$, or $x_1 = -\frac{1}{2}$, meaning that $x_1$ remains constant over the same interval. Thus $\dot{x}_1 = 0$ for all $t\in ]\bar{t} - \eta , \bar{t}],\,\,\eta >0$, which contradicts the fact that $\dot{x}_1 = 1$ over $t\in ]\bar{t} - \eta , \bar{t}]$.

Therefore, only the constraint g can be active over an interval before $\bar{t}$, and by (5.32), we obtain $\mu$ over this interval

$$ \frac{\partial H}{\partial u} + \mu \frac{\partial g}{\partial u} = \lambda _2 - \mu = 0 $$

thus $\lambda _2 = \mu$ and the adjoint equation (5.35) reads

$$\begin{aligned} \dot{\lambda } = \left( \begin{array}{cc} 0 &{} 2\\ -1 &{} 1 \end{array}\right) \lambda ,\quad \forall t\in ]\bar{t} - \eta ,\bar{t}] \end{aligned}$$

(5.37)

Let us next analyze the switching condition of $\bar{u}$, or more precisely the change of signum of $\lambda _2$. We know that, in an interval $] \bar{t}-\eta , \bar{t}]$ with $\eta >0$, we have $\lambda _2 > 0$ and we want to characterize $\eta$ such that $\lambda _2(t) < 0$ for $t\le \bar{t}-\eta$ and $\lambda _{2}(\bar{t}-\eta )=0$. Note that $\lambda _2$ cannot vanish over a nonempty open interval since then, according to (5.36) or (5.37), we would also get $\lambda _1=0$ which is impossible since the vector $\lambda$ cannot vanish. Thus, since $\lambda _2$ is locally increasing in a neighborhood of $\bar{t}-\eta$, we must have $\dot{\lambda }_2(\bar{t}-\eta ) > 0$, which is equivalent to $\lambda _{1}(\bar{t}-\eta ) < 0$. Thus, expressing (5.34) at time $\bar{t}-\eta$, we get $x_{2}(\bar{t}-\eta ) =0$ and $\bar{u}(t) =1$ for $t <\bar{t}-\eta$.

As long as $\lambda _2$ remains different from zero we keep $\bar{u} = 1$. As seen on Fig. 5.3, $x_2$ crosses for a second time the $x_1$ axis and it can be checked that, at this time, $\lambda _2$ also vanishes. Therefore, the last section of the barrier is made of the trajectory generated by $\bar{u} = x_2$ from this time.

Fig. 5.3

Admissible set of the constrained spring from Sect. 5.4.2

Remark 5.4

Note that Assumption (A7) does not hold true at the final point z since there are two active constraints for only one control. However, since this condition is violated only at this point, we may conclude by continuity that condition (5.31) still holds.

4.3 Constrained Spring II

Consider the same mass–spring–damper system with the same constants as in the previous example, but with a richer constraint

$$\begin{aligned} \left( \begin{array}{c} \dot{x}_{1}\\ \dot{x}_{2} \end{array} \right) = \left( \begin{array}{cc} 0 &{} 1\\ -2 &{} -2 \end{array} \right) \left( \begin{array}{c} x_{1}\\ x_{2} \end{array} \right) +\left( \begin{array}{c} 0\\ 1 \end{array} \right) u ,\,\,\,\,\,|u|\le 1, \,\,\,\,\,x_2(x_2 - u) \le 0 \end{aligned}$$

(5.38)

We identify $\tilde{g}(x) = x_2^2 - |x_2|$, and $G_0 = \{x:x_2 = 0 ~\mathrm {or~} x_2 = \pm 1\}$. $\tilde{g}$ is differentiable for $x_2 \ne 0$ and from (5.31) and (5.32) we identify, in same manner as in the previous example, two points of ultimate tangentiality, namely $z=(-\frac{1}{2},1)$ along with $\lambda (\bar{t}) = (0,1)$, and $z = (\frac{1}{2},-1)$ along with $\lambda (\bar{t}) = (0,-1)$. We defer the treatment of the $x_1$ axis, which is also in $G_0$, to the discussion below.

From the minimisation of the Hamiltonian, which is the same as in the previous example except that U(x) now corresponds to $u\ge x_2$ if $x_2 \ge 0$ and $u\le x_{2}$ if $x_{2} \le 0$, we find the control $\bar{u}$

$$ \begin{array}{lll}\mathrm {if}~\lambda _2(t) < 0 &{}~ &{}\bar{u}(t)=\left\{ \begin{array}{lll} 1 &{} \mathrm {if} &{} x_2 \in ]0,1]\\ x_2 &{} \mathrm {if} &{} x_2 \in ]-1,0[ \end{array}\right. \\ \mathrm {if}~\lambda _2(t) > 0 &{} &{}\bar{u}(t)=\left\{ \begin{array}{lll} x_2 &{} \mathrm {if} &{} x_2 \in ]0,1]\\ -1 &{} \mathrm {if} &{} x_2 \in ]-1,0[ \end{array}\right. \\ \mathrm {if}~\lambda _2(t) = 0 &{} &{}\bar{u}(t)=\mathrm {arbitrary} \end{array} $$

If we now integrate backwards from the points $(-\frac{1}{2},1)$ and $(\frac{1}{2},-1)$ with the control $\bar{u}(t)$ we obtain the barrier as in Fig. 5.4. It turns out that $\bar{u}(t) = x_2(t)$ all along both curves: the reader may easily check that, the necessary condition $\frac{\partial H}{\partial u} + \mu \frac{\partial g}{\partial u} = 0$ yields $\lambda _2 - \mu x_{2} =0$ and, since $\frac{\partial g}{\partial x} = (0,2x_2 -u)^T$, we get the same adjoint equation as (5.37) when $\bar{u}=x_{2}$, and conclude that $\lambda _2 (t)$ is positive as long as $x_2(t)$ is positive, which implies that $\bar{u}=x_{2}$, and $\lambda _2 (t)$ must be negative as long as $x_2(t)$ is negative, which again implies that $\bar{u}=x_{2}$, hence the result.

Fig. 5.4

Admissible set of the constrained spring from Sect. 5.4.3

Let us now turn to the $x_1$ axis, where $\tilde{g} = x_2^2 - |x_2|$ is non-differentiable. For any z on the $x_1$ axis, we have $U(z) = [-1,1]$ and $\partial \tilde{g} (z) = \bar{\mathsf {co}}\left( (0,-1)^T, (0,1)^T \right) = \{0\} \times [-1,1]$ and we must have

$$\begin{aligned} \min _{u\in [-1,1]} \max _{\xi \in \partial \tilde{g}(\tilde{z})} \xi .f(\tilde{z},u) = 0 = \min _{u\in [-1,1]} \max _{\xi _2\in [-1,1]} \xi _2 (-2x_1 + u) \end{aligned}$$

(5.39)

For each $-\frac{1}{2}\le z_1\le \frac{1}{2}$ Eq. (5.39) has a solution given by $\xi = (0,{\mathsf {sign}}(-2z_1 + u))$ from which we deduce that $\bar{u} = 2z_1$. However, one can directly verify that the integral curves of (5.38) with endpoints in the set $[-\frac{1}{2}, \frac{1}{2}] \times \{ 0 \}$ with the control $u = x_2$ all correspond to admissible curves (integrated backwards) and therefore do not belong to the barrier, but that they make the constraint $g(x^{(\bar{u},\bar{x})}(t),\bar{u}(t))$ equal to 0 for $\bar{u} = x_2$ for all $\bar{x} \in [-\frac{1}{2}, \frac{1}{2}] \times \{ 0 \}$ and for all t. This attests that our conditions are only necessary and far from being sufficient.

Remark 5.5

Note that, as in Sect. 5.4.2, Assumption (A7) does not hold true at the final points $z \in G_0$ since there are two active constraints for only one control. Again, we conclude by a continuity argument that condition (5.31) still holds.

5 Conclusion

In this paper, we have demonstrated on elementary examples of systems subject to pure or mixed constraints, the effectiveness of the results obtained in [6, 7], which allowed us to give a complete construction of their barriers and admissible sets. We also pointed out some significant differences in these constructions. In particular, we have shown, in the mixed constrained case, that the barrier does not need to intersect the boundary $G_0$ of the constraint set; that, according to the feedback nature of the control, due to the state dependence of the control set, the equilibria and their stability could be modified to be repelled from $G_0$; that the nonsmooth version of the necessary ultimate tangentiality condition, though useful, is far from being sufficient; and that Assumption (A7) is, even in simple examples, not everywhere satisfied. Higher dimensional examples are presently under investigation and will be published elsewhere.