Keywords

1 Statement of the Problem and Preliminaries

Consider the following problem (P):

$$\begin{aligned} J(T,x(\cdot ))=\varphi (T,x(0),x(T))+\lambda \int _{0}^{T}\delta _M(x(t))\,dt\rightarrow \mathrm{min}\,, \end{aligned}$$
(1)
$$\begin{aligned} \dot{x}(t)\in F(x(t)), \end{aligned}$$
(2)
$$\begin{aligned} x(0)\in M_0,\qquad x(T)\in M_1. \end{aligned}$$
(3)

Here \(x\in \mathbb {R}^n\) is a state vector, \(M_0\), \(M_1\) are nonempty closed sets in \(\mathbb {R}^n\), \(\lambda \) is a positive real, \(F:\mathbb {R}^n\rightrightarrows \mathbb {R}^n\) is a locally Lipschitz multivalued mapping with nonempty convex compact values, \(\varphi :[0,\infty )\times \mathbb {R}^n\times \mathbb {R}^n\mapsto \mathbb {R}^1\) is a locally Lipschitz function; \(\delta _M(\cdot )\) is the characteristic function of a set M (“risk zone”) in \(\mathbb {R}^n\), i.e.

$$\begin{aligned} \delta _M(x)= {\left\{ \begin{array}{ll} 1, &{} x\in M,\\ 0, &{} x\notin M. \end{array}\right. } \end{aligned}$$
(4)

We assume that M is a nonempty open set, \(G=\mathbb {R}^n\setminus M\ne \emptyset \), and for any \(x\in G\) the Clarke tangent cone \(T_G(x)\) (see [9]) has nonempty interior, i.e. \(\mathrm{int}\,\, T_G(x)\ne \emptyset \). The terminal time \(T>0\) in problem (P) is assumed to be free; accordingly, the class of admissible trajectories in (P) consists of all absolutely continuous solutions \(x(\cdot )\) of differential inclusion (2) defined on corresponding time intervals [0, T], \(T>0\), and satisfying boundary conditions (3). An admissible trajectory \(x_*(\cdot )\) defined on a time interval \([0,T_*]\), \(T_*>0\), is optimal in problem (P) if the functional \(J(\cdot ,\cdot )\) (see (1)) reaches the minimal possible value at \((T_*,x_*(\cdot ))\).

Notice, that the peculiarity of problem (P) consists of the presence of discontinuous integrand \(\delta _M(\cdot )\) in the integral term in the functional \(J(\cdot ,\cdot )\). Substantially, the integral term penalizes the states in the risk zone M. Such risk zones could appear in statements of different applied problems when there is an admissible but unfavorable set M in the state space \(\mathbb {R}^n\). In economics the set M can correspond to the states with high probability of bankruptcy; in ecology the set M can correspond to the states with high probability of the system degradation; in engineering such sets can correspond to the states of overloading or instability of the system.

In classical optimal control theory the presence of such unfavorable set M is modeled usually via introducing an additional state constraint (see [15, Chapt. 6])

$$ x(t)\in G=\mathbb {R}^n\setminus M,\qquad t\in [0,T]. $$

Substantially, this means that presence of the state variable \(x(\cdot )\) in the set M is prohibited. The set G (“safety zone”) is assumed to be closed in this case (i.e. the set M is open).

An optimal control problem with a closed convex risk zone M was initially considered in [16] in the case of linear control system, and under some a priori regularity assumptions on behavior of an optimal trajectory \(x_*(\cdot )\). In particular, it was assumed in [16] that the optimal trajectory \(x_*(\cdot )\) had a finite number of intersection points with the boundary of the set M. In [17] under the same linearity and regularity assumptions the case of time dependent closed convex set \(M = M(t)\), \(t\in [0,T]\), was considered. In [7, 8] the problem of optimal crossing a given closed risk zone M was studied and necessary optimality conditions for affine in control system were developed without any a priori assumptions on the behavior of the optimal trajectory. In [18] this result (in the case of closed set M) was generalized to the case of more general integral utility functional. The main novelty of the present work is that the risk zone M is assumed to be open. In this case introducing of the risk zone M in the statement of problem (P) can be considered as a weakening of the classical concept of the state constraint in optimal control. Notice also, that the approach developed in [7, 8, 18] for the case of the closed set M does not work if the set M is open.

In that follows \(N_A(a)=T_A^*(a)\) and \(\hat{N}_A(a)\) are the Clarke normal cone [9] and the cone of generalized normals [13] to the closed set \(A\subset \mathbb {R}^{n}\) at a point \(a\in A\), respectively; \(\partial A\) is the boundary of the set A; \(H(F(x),\psi )=\mathrm{max}\,_{f\in F(x)}\langle f,\psi \rangle \) is the value of the Hamiltonian \(H(F(\cdot ),\cdot )\) of differential inclusion (2) at a point \((x,\psi )\in \mathbb {R}^{n}\times \mathbb {R}^{n}\); \(\partial H(F(x),\psi )\) is the Clarke subdifferential of the locally Lipschitz function \(H(F(\cdot ),\cdot )\) at a point \((x,\psi )\in \mathbb {R}^{n}\times \mathbb {R}^{n}\) [9], and \(\partial \hat{\varphi }(T,x_1,x_2)\) is the generalized gradient of locally Lipschitz function \(\varphi (\cdot ,\cdot ,\cdot )\) at a point \((T,x_1,x_2)\in [0,\infty )\times \mathbb {R}^{n}\times \mathbb {R}^{n}\) [13].

For \(i\in \mathbb {N}\) and an arbitrary \(x\in \mathbb {R}^n\) set \(\tilde{\delta }_i(x)=\mathrm{min}\,\{i\rho (x,G),\delta _M(x)\}\) where \(\rho (x,G)=\mathrm{min}\,\{\Vert x-\xi \Vert :\xi \in G\}\) is the distance from a point x to the nonempty closed set \(G=\mathbb {R}^n\setminus M\) and the function \(\delta _M(\cdot )\) is defined by equality (4).

Further, for \(i\in \mathbb {N}\) let us define the function \(\delta _i:\mathbb {R}^n\mapsto \mathbb {R}^1\) by equality

$$\begin{aligned} \delta _i(x)=\int _{\mathbb {R}^n}\tilde{\delta }_i(x+y)\omega _i(y)\, dy. \end{aligned}$$
(5)

Here \(\omega _i(\cdot )\) is a smooth (\(C^{\infty }(\mathbb {R}^n\))) probabilistic density such that \(\mathrm{supp}\,\omega _i(\cdot )\subset 1/2^iB\) where B is the closed unit ball in \(\mathbb {R}^n\) with the center in 0. Then for any \(i\in \mathbb {N}\) the function \(\delta _i(\cdot )\) is smooth as a convolution with \(\omega _i(\cdot )\).

The following auxiliary statements hold.

Lemma 1

For any \(x\in \mathbb {R}^n\) we have

$$\begin{aligned} \delta _i(x)\le \delta _M(x)+\frac{i}{2^i},\qquad i\in \mathbb {N}. \end{aligned}$$
(6)

Proof

Indeed, if \(x\in M\) then \(\delta _M(x)=1\). Since \(\delta _i(x)\le 1\), \(i\in \mathbb {N}\), inequality (6) is obviously satisfied. Now assume \(x\notin M\). Then \(\delta _M(x)=0\), and for any \(y\in \mathrm{supp}\,\omega _i(\cdot )\), \(i\in \mathbb {N}\), we have \(\tilde{\delta }_i(x+y)\le i\rho (x+y,G)\le iy\le i/2^i\). Due to the definition of the function \(\delta _i(\cdot )\) (see (5)) we get

$$ \delta _i(x)=\int _{\mathbb {R}^n}\tilde{\delta }_i(x+y)\omega _i(y)\, dy\le \frac{i}{2^i},\qquad i\in \mathbb {N}. $$

Since \(\delta _M(x)=0\) inequality (6) also holds in this case.    \(\Box \)

Lemma 2

Let a sequence \(\{x_i(\cdot )\}_{i=1}^{\infty }\) of continuous functions \(x_i:[0,T]\mapsto \mathbb {R}^n\) defined on some time interval [0, T], \(T>0\), converges uniformly to a continuous function \(\tilde{x}:[0,T]\mapsto \mathbb {R}^n\). Then

$$\begin{aligned} \liminf _{i\rightarrow \infty }\int _0^{T}\delta _i(x_i(t))\, dt\ge \int _0^{T}\delta _M(\tilde{x}(t))\, dt. \end{aligned}$$
(7)

Proof

Assume that for some \(t\in [0, T]\) we have \(\tilde{x}(t)\in M\). Then \(\delta _M(\tilde{x}(t))= 1\), and since the set M is open and the sequence \(\{x_i(\cdot )\}_{k=1}^{\infty }\) converges uniformly to \(\tilde{x}(\cdot )\) there are \(\varepsilon _0>0\) and \(i_0\ge 1/\varepsilon _0\) such that for all \(i\ge i_0\) we have \(x_i(t)+\varepsilon _0B\subset M\). Then for all \(i\ge i_0\) due to definition of function \(\delta _i(\cdot )\) (see (5)) we get equality \(\delta _i(x_i(t))=1\). Hence, \(\lim _{i\rightarrow \infty }\delta _i(x_i(t))=\delta _M(\tilde{x}(t))=1\) in this case. Now, assume that \(t\in [0,T]\) is such that \(\tilde{x}(t)\not \in M\). Then \(\delta _M(\tilde{x}(t))=0\). As far as \(\delta _i(x_i(t))\ge 0\) for any \(t\in [0,T]\) and all \(i\in \mathbb {N}\) (see (5)) we have \(\liminf _{i\rightarrow \infty }\delta _i(x_i(t))\ge \delta _M(\tilde{x}(t))\) in this case.

Thus, for any \(t\in [0,T]\) the following inequality holds:

$$ \liminf _{i\rightarrow \infty }\,\delta _i(x_i(t))\ge \delta _M(\tilde{x}(t)). $$

From this inequality due to Fatou’s lemma (see [10, Lemma 8.7.i.]) we get (7).

   \(\Box \)

As an immediate corollary of the lemmas above we get the following result.

Theorem 1

The integral functional \(J_M:C([0,T],\mathbb {R}^n)\mapsto \mathbb {R}^1\), \(T>0\), defined by the equality

$$ J_M(x(\cdot ))=\int _0^T\delta _M(x(t))\, dt $$

is lower semicontinuous.

Proof

Indeed, let \(T>0\) and a sequence \(\{x_i(\cdot )\}_{i=1}^{\infty }\) of continuous functions \(x_i:[0,T]\mapsto \mathbb {R}^n\) converges to a continuous function \(\tilde{x}(\cdot )\) in \(C([0,T],\mathbb {R}^n)\). Then due to Lemma 1 we have

$$ J_M(x_i(\cdot ))=\int _0^T\delta _M(x_i(t))\, dt\ge \int _0^T\delta _i(x_i(t))\, dt-\frac{iT}{2^i},\qquad i\in \mathbb {N}. $$

Hence, due to Lemma 2 passing to a limit as \(i\rightarrow \infty \) we get

$$ \liminf _{i\rightarrow \infty } J_M(x_i(\cdot ))\ge \liminf _{i\rightarrow \infty }\int _0^T\delta _i(x_i(t))\, dt\ge \int _0^T\delta _M(\tilde{x}(t))\, dt =J_M(\tilde{x}(\cdot )). $$

   \(\Box \)

2 Main Result

Let \(x_*(\cdot )\) be an optimal admissible trajectory in (P), and let \(T_*>0\) be the corresponding optimal terminal time. In that follows we always assume that \(x_*(\cdot )\) is defined on the time interval \([T_*,\infty )\) as a constant: \(x_*(t)\equiv x_*(T_*)\), \(t\ge 0\). Define also the sets \(\tilde{M}_0\) and \(\tilde{M}_1\) by the equalities

$$\begin{aligned} \tilde{M}_0= {\left\{ \begin{array}{ll} M_0, &{} x_*(0)\in M,\\ M_0\bigcap G, &{} x_*(0)\in G \end{array}\right. }\quad \text{ and }\quad \tilde{M}_1= {\left\{ \begin{array}{ll} M_1, &{} x_*(T_*)\in M,\\ M_1\bigcap G, &{} x_*(T_*)\in G. \end{array}\right. } \end{aligned}$$
(8)

Next theorem is the main result of the present paper.

Theorem 2

Let \(x_*(\cdot )\) be an optimal admissible trajectory in problem (P), and let \(T_*>0\) be the corresponding optimal terminal time. Then there are a constant \(\psi ^0\ge 0\), an absolutely continuous function \(\psi :[0,T_*]\mapsto \mathbb {R}^n\) and a bounded regular Borel vector measure \(\eta \) on \([0,T_*]\) such that the following conditions hold:

  1. (1)

    the measure \(\eta \) is concentrated on the set \(\mathfrak {M}=\{t\in [0,T_*]:x_*(t)\in \partial G\}\), and it is nonpositive on the set of continuous functions \(y:\mathfrak {M}\mapsto \mathbb {R}^n\) with values \(y(t)\in T_G(x_*(t))\), \(t\in \mathfrak {M}\), i.e.

    $$ \int _{\mathfrak {M}} y(t)\, d\eta \le 0; $$
  2. (2)

    for a.e. \(t\in [0,T_*]\) the Hamiltonian inclusion holds:

    $$ (-\dot{\psi }(t),\dot{x}_*(t))\in \partial H(x_*(t),\psi (t)+\lambda \int _0^t\, d\eta ); $$
  3. (3)

    for \(t=T_*\) and for any \(t\in [0,T_*)\) which is a point of right approximate continuityFootnote 1 of the function \(\delta _M(x_*(\cdot ))\) the following stationarity condition holds:

    $$ H(x_*(t),\psi (t)+\lambda \int _0^t\, d\eta )-\psi ^0\lambda \delta _M(x_*(t))= H(x_*(0),\psi (0))-\psi ^0\lambda \delta _M(x_*(0)); $$
  4. (4)

    the transversality condition holds:

  5. (5)

    the nontriviality condition holds:

    $$ \psi ^0+\Vert \psi (0)\Vert +\Vert \eta \Vert \ne 0. $$

The proof of Theorem 2 is based on approximation of problem (P) by a sequence of approximating problems with Lipschitz data for which the corresponding necessary optimality conditions are known (see [9, Theorem 5.2.1]).

Let \(x_*(\cdot )\) be an optimal admissible trajectory in problem (P), and let \(T_*>0\) be the corresponding optimal terminal time. For \(i\in \mathbb {N}\) consider the following optimal control problem \((P_i)\):

(9)
$$\begin{aligned} \dot{x}(t)\in F(x(t)), \end{aligned}$$
(10)
$$\begin{aligned} |T-T_*|\le 1,\qquad \Vert x(t)-x_*(t)\Vert \le 1,\quad t\in [0,T], \end{aligned}$$
(11)
$$\begin{aligned} x(0)\in \tilde{M}_0,\qquad x(T)\in \tilde{M}_1. \end{aligned}$$
(12)

Here the function \(\varphi (\cdot ,\cdot ,\cdot )\), the multivalued mapping \(F(\cdot )\) and the number \(\lambda >0\) are the same as in (P). The sets \(\tilde{M}_0\) and \(\tilde{M}_1\) are defined in (8). As in the problem (P), the set of admissible trajectories in \((P_i)\), \(i\in \mathbb {N}\), consists of all absolutely continuous solutions \(x(\cdot )\) of differential inclusion (10) defined on their own time intervals [0, T], \(T>0\), and satisfying constraints in (11) and boundary conditions in (12).

For any \(i\in \mathbb {N}\) the problem \((P_i)\) is a standard optimal control problem for the differential inclusion with Lipschitz data, state and terminal constraints (see [9, Sect. 3.6]). Since \(x_*(\cdot )\) is an admissible trajectory in \((P_i)\), \(i\in \mathbb {N}\), due to Filippov’s existence theorem (see, [10, Theorem 9.3.i]) for any \(i\in \mathbb {N}\) there is an optimal admissible trajectory \(x_i(\cdot )\) in \((P_i)\) which is defined on the corresponding time interval \([0,T_i]\), \(T_i>0\). We will assume bellow that for any \(i\in \mathbb {N}\) the trajectory \(x_i(\cdot )\) is extended to the infinite time interval \([T_i,\infty )\) as a constant: \(x_i(t)\equiv x_i(T_i))\), \(t\ge T_i\).

We will call \(\left\{ (P_i)\right\} _{k=1}^\infty \) a sequence of approximating problems corresponding to the optimal trajectory \(x_*(\cdot )\).

Theorem 3

Let \(x_*(\cdot )\) be an optimal admissible trajectory in problem (P), and let \(T_*\) be the corresponding optimal terminal time. Let \(\{(P_i)\}_{i=1}^{\infty }\) be the sequence of approximating problems corresponding to \(x_*(\cdot )\), and let \(x_i(\cdot )\), \(T_i>0\), be an optimal admissible trajectory and the corresponding optimal time, respectively in \((P_i)\), \(i\in \mathbb {N}\). Then

$$\begin{aligned} \lim _{i\rightarrow \infty } \, T_i=T_*, \end{aligned}$$
(13)
$$\begin{aligned} \lim _{i\rightarrow \infty }\, x_i(\cdot ) = x_*(\cdot )\qquad {in} \quad C([0,T_*],\mathbb {R}^n), \end{aligned}$$
(14)
$$\begin{aligned} \lim _{i\rightarrow \infty }\, \dot{x}_i(\cdot ) = \dot{x}_*(\cdot )\qquad {weakly\, in}\quad L^1([0,T_*],\mathbb {R}^n), \end{aligned}$$
(15)
$$\begin{aligned} \lim _{i\rightarrow \infty }\, \int _0^{T_i}\delta _i(x_i(t))\, dt =\int _0^{T_*}\delta _M(x_*(t))\, dt. \end{aligned}$$
(16)

Proof

Since \(x_i(\cdot )\) is an optimal admissible trajectory in \((P_i)\), \(i\in \mathbb {N}\), and \(x_*(\cdot )\) is an admissible trajectory in \((P_i)\), due to Lemma 1 we have (see (9) and (6)):

(17)

Since \(|T_i-T_*|\le 1\), \(i\in \mathbb {N}\), without loss of generality we can assume that \(\lim _{i\rightarrow \infty }T_i = \tilde{T}\le T_*+1\). Further, the set of all admissible trajectories of (10) satisfying the state constraint (11) is a compactum in \(C([0,\tilde{T}],\mathbb {R}^n)\). Let \(\tilde{x}(\cdot )\) be a limit point of \(\{x_i(\cdot )\}_{i=1}^\infty \) in \(C([0,\tilde{T}],\mathbb {R}^n)\). Then \(\tilde{x}(\cdot )\) is an admissible trajectory in (P), and passing to a subsequence we can assume that \(\lim _{i\rightarrow \infty } x_i(\cdot ) = \tilde{x}(\cdot )\) in \(C([0,\tilde{T}],\mathbb {R}^n)\). Further, \(x_*(\cdot )\) is an optimal trajectory in (P), while \(\tilde{x}(\cdot )\) is an admissible one in this problem. Hence,

$$ \varphi (T_*,x_*(0),x_*(T_*)) +\lambda \int _0^{T_*} \delta _M(x_*(t))\, dt\le \varphi (\tilde{x}(0),\tilde{x}(\tilde{T})) +\lambda \int _0^{\tilde{T}} \delta _M(\tilde{x}(t))\, dt. $$

Hence, for \(i\in \mathbb {N}\) due to (17) we get

(18)

Since \(\lim _{i\rightarrow \infty }T_i=\tilde{T}\) and \(\lim _{i\rightarrow \infty } x_i(\cdot ) = \tilde{x}(\cdot )\) in \(C([0,\tilde{T}],\mathbb {R}^n)\) due to Lemma 2 for any \(\varepsilon >0\) there is a natural \(i_0\) such that for all \(i\ge i_0\) we have

From these inequalities due to (18) for any \(i\ge i_0\) we get

$$ (T_i-T_*)^2 + \int _0^{T_i} \Vert x_i(t)-x_*(t)\Vert ^2\,dt \le \varepsilon (1+\lambda ) + \frac{i\lambda T_*}{2^i}. $$

Passing to a limit as \(i\rightarrow \infty \) in the inequality above we get

$$ \limsup _{i\rightarrow \infty }\left[ (T_i-T_*)^2 +\int _0^{T_i} \Vert x_i(t)-x_*(t)\Vert ^2\,dt\right] \le \varepsilon (1+\lambda ). $$

Since \(\varepsilon >0\) is an arbitrary positive number this implies

$$ \lim _{i\rightarrow \infty }T_i=T_*,\qquad \lim _{i\rightarrow \infty }\int _0^{T_*} \Vert x_i(t)-x_*(t)\Vert ^2\,dt =0. $$

Thus, equality (13) is proved. Since \(\lim _{i\rightarrow \infty }T_i=\tilde{T}=T_*\) and \(\tilde{x}(\cdot )\) is an arbitrary limit point of the sequence \(\{x(\cdot )\}_{i=1}^{\infty }\) in \(C([0,\tilde{T}],\mathbb {R}^n)\) we get (14). Equality (15) is followed by (14) and the fact that the sequence \(\{\dot{x}_i(\cdot )\}_{i=1}^{\infty }\) is bounded in \(L_{\infty }([0,T_*],\mathbb {R}^n)\). Finally, due to Lemma 2 equality (16) follows from (13), (14) and (18).    \(\Box \)

Due to condition (14) of Theorem 3 for all sufficiently large numbers i the terminal time and state constraints in (11) hold as strict ones. Hence, the Clarke necessary conditions (see [9, Theorem 5.2.1])) hold for optimal trajectories \(x_i(\cdot )\) in problems \((P_i)\) for all sufficiently large numbers i. The subsequent proof of Theorem 2 is based on the limiting procedure in these necessary optimality conditions applied to problems \((P_i)\), \(i\in \mathbb {N}\), as \(i\rightarrow \infty \). It is similar to the proof of analogous results for problems with state constraints (see [3, 5, Theorem 1]). The detailed proof of a similar result for problem (P) in the case of a fixed time interval [0, T], \(T>0\), is presented in [6].

Notice, that Theorem 2 is similar to the necessary conditions for optimality for an optimal control problem for the differential inclusion with state constraints proved in [3]. As in [3], the stationarity condition (3) allows one to get sufficient conditions for nondegeneracy of the developed necessary optimality conditions (Theorem 2). Other results on nondegeneracy of different versions of the maximum principle for problems with state constraints and further references can be found in [1,2,3,4,5, 11, 12].