Solutions to Constrained Optimal Control Problems with Linear Systems

Abstract

This paper is devoted to present solutions to constrained finite-horizon optimal control problems with linear systems, and the cost functional of the problem is in a general form. According to the Pontryagin’s maximum principle, the extremal control of such problem is a function of the costate trajectory, but an implicit function. We here develop the canonical backward differential flows method and then give the extremal control explicitly with the costate trajectory by canonical backward differential flows. Moreover, there exists an optimal control if and only if there exists a unique extremal control. We give the proof of the existence of the optimal solution for this optimal control problem with Green functions.

1 Introduction

It is well known that there is a close relationship between the theory of optimization and the technique of optimal control [1,2,3]. This paper is devoted to the study of constrained finite-horizon optimal control problems as known as problem (P) by the Pontryagin’s maximum principle and backward differential flows. These two theories have been widely used in the research of optimizations and optimal control problems [4,5,6,7]. In problem (P), an integral cost functional is minimized across the set of controls and trajectories of a linear finite-dimensional dynamical system operating over a bounded interval of time. Using standard notations of control theory, we represent the optimal control problem (P) which has a quadratic part in its cost function and the constraints of the control are a ball and a linear differential equation.

A piecewise continuous function mapping the time to the constraint zone, a ball, is said to be an admissible control. And the initial point is a given vector. Specially, the quadratic part of the cost function is symmetric positive definite. Let the rest part of the cost function be second-order continuously differentiable and the second derivative is great than 0 for all controls.

By the classical optimal control theory, we have the Hamiltonian function [8]. In general, it is difficult to obtain an analytic form of the optimal feedback control for the optimal control problem (P). It is well known that, in the unconstrained case, if the part of the cost function except the quadratic part is a positive semi-definite quadratic form, then a perfect optimal feedback control is obtained by the solution of a Riccati matrix differential equation. The primal goal of this paper is to present an analytic form of the optimal feedback control to the optimal control problem (P).

We know from the Pontryagin principle that the optimal trajectory and corresponding parameter denoting the state and the costate which correspond to the optimal control. Particularly, when the control is an extremal control, it satisfies the Pontryagin principle. By means of the Pontryagin principle and the dynamic programming theory, many numerical algorithms have been suggested to approximate the solution to the problem (P). This is due to the nonlinear integrand in the cost functional. In this paper, combining the backward differential flows with the Pontryagin principle, we solve problem (P) which has nonlinear integrand on the control variable in the cost functional and present the optimal control expressed by the costate via canonical dual variables.

2 Problem Formulation

Using standard notations of control theory, we represent the optimal control problem (P) as follows:

$$\begin{aligned} (P) ~~ \begin{array}{ll} \min \quad &{} J(u)=\int ^\mathrm{T}_{0} [F(x(t)) + \frac{1}{2} u^\mathrm{T}(t)Ru(t) + b^\mathrm{T}u(t)]\mathrm{d}t\\ s.t.\quad &{}{\dot{x}}(t)=Ax(t)+Bu(t),\\ \quad &{}x(0)=x_0\in {\mathbb {R}}^n,\\ \quad &{} u(t)\in U:=\{u:u^\mathrm{T}u\le 1\}\subset {\mathbb {R}}^m, t\in [0,T],\\ \end{array} \end{aligned}$$

(1)

where $A \in {\mathbb {R}}^{n\times n}$, $B \in {\mathbb {R}}^{n\times m}$ and $b\in {\mathbb {R}}^m$ are constant matrices, and $R\in {\mathbb {R}} ^{m\times m}$ is a symmetric positive definite matrix. The initial point $x_0$ is a given vector in ${\mathbb {R}}^n$. Let F(x) be second-order continuously differentiable and $F_{xx}(x)\ge 0$ for all $x\in {\mathbb {R}}^n$. A piecewise continuous function, $u(\cdot ):[ 0 , T ] \rightarrow U$, is said to be an admissible control. By the classical optimal control theory, we have the following Hamiltonian function [8]:

$$\begin{aligned} H(t,x,u,\lambda )= F(x(t)) + \frac{1}{2}u^\mathrm{T}(t)Ru(t) + b^\mathrm{T}u(t) + \lambda ^\mathrm{T}(t) ( Ax(t) + Bu(t) ). \end{aligned}$$

(2)

The state and costate systems are

$$\begin{aligned} {\dot{x}}= & {} H_\lambda (t,x,u,\lambda ) =Ax(t)+Bu(t),x(0)=x_0, \end{aligned}$$

(3)

$$\begin{aligned} {\dot{\lambda }}= & {} -H_x = - F_x(x(t)) -A^\mathrm{T}\lambda ,\lambda (T)=0. \end{aligned}$$

(4)

In general, it is difficult to obtain an analytic form of the optimal feedback control for the problem (P). It is well known that, in the unconstrained case, if F(x(t)) is a positive semi-definite quadratic form, then a perfect optimal feedback control is obtained by the solution of a Riccati matrix differential equation. The primal goal of this paper is to present an analytic form of the optimal feedback control to the optimal control problem (P).

We know from the Pontryagin principle the ${\hat{x}}(\cdot )$ and ${\hat{\lambda }}(\cdot )$ denoting the state and costate corresponding to ${\hat{u}}(\cdot )$. Particularly, when ${\hat{u}}$ is an extremal control, we have

$$\begin{aligned}&\dot{{\hat{x}}}=H_\lambda (t,{\hat{x}},{\hat{u}},{\hat{\lambda }}) =A{\hat{x}}+B{\hat{u}},{\hat{x}}(0)=x_0, \end{aligned}$$

(5)

$$\begin{aligned}&\dot{{\hat{\lambda }}} =-H_x (t,{\hat{x}},{\hat{u}},{\hat{\lambda }})= - F_{{\hat{x}}}({\hat{x}}) -A^\mathrm{T}{\hat{\lambda }},{\hat{\lambda }}(T)=0, \end{aligned}$$

(6)

$$\begin{aligned}&H(t,{\hat{x}}(t),{\hat{u}}(t),{\hat{\lambda }}(t))=\min _{\Vert u\Vert \le 1} H(t,{\hat{x}}(t),u,{\hat{\lambda }}(t)), a.e.t\in [0,T]. \end{aligned}$$

(7)

By means of the Pontryagin principle and the dynamic programming theory, many numerical algorithms have been suggested to approximate the solution to the problem (P). This is due to the nonlinear integrand in the cost functional. In this paper, combining the backward differential flows with the Pontryagin principle, we solve problem (P), which has nonlinear integrand on the control variable in the cost functional and present the optimal control expressed by the costate via canonical dual variables.

3 A Differential Flow with Lagrangian Function

In this section, we present a differential flow to deal with the problem (P), which is used to find the optimal control expressed by the costate in the next section. For the problem (P), the Lagrangian function can be written as

$$\begin{aligned} L(u,\rho ) := \frac{1}{2} u^\mathrm{T} R u + (B^\mathrm{T}\lambda + b)^\mathrm{T} u + \frac{\rho }{2} (u^\mathrm{T} u - 1), \end{aligned}$$

(8)

where ${\rho }$ is a Lagrangian multiplier. The corresponding partial derivatives with respect to u are as follows

$$\begin{aligned} L_{u}(u,\rho ) = (R + \rho I)u +B^\mathrm{T} \lambda +b, \end{aligned}$$

(9)

and

$$\begin{aligned} L_{uu}(u,\rho ) = R + \rho I. \end{aligned}$$

(10)

Let R be a ${m\times m}$ symmetric positive definite matrix. We define a set G as

$$\begin{aligned} G:=\{\rho \ge 0 : R + \rho I>0\} \subset {\mathbb {R}}. \end{aligned}$$

(11)

Obviously, G is nonempty.

Then, we introduce a backward differential flow over G.

Definition 3.1

For each $\rho$ in G, define

$$\begin{aligned} {\hat{u}}(\rho ) := - (R + \rho I)^{-1} (B^\mathrm{T} \lambda + b), \end{aligned}$$

(12)

which is called a differential flow.

It can be verified easily that the differential flow satisfies the following differential system

$$\begin{aligned} \frac{ \mathrm{d}{\hat{u}}(\rho )}{\mathrm{d}\rho } = -(R+\rho I)^{-1} {\hat{u}}(\rho ). \end{aligned}$$

(13)

Theorem 3.1

If ${\hat{u}}(\rho )$ is the differential flow over G, then, for given $\rho \in G$, ${\hat{u}}(\rho )$ is the unique minimizer of $L(u,\rho )$ over ${\mathbb {R}}^n$, i.e.,

$$\begin{aligned} L(u,\rho )\ge L({\hat{u}}(\rho ),\rho ), \forall u \in {\mathbb {R}}^n. \end{aligned}$$

(14)

Proof

For given $\rho \in G$, $L_{uu}(u,\rho ) = R + \rho I > 0$, for any $u \in {\mathbb {R}}^n$, $L_{uu}({\hat{u}}(\rho ),\rho ) > 0, \forall \rho \in G$. On the other hand, by (9), it is clear that

$L_{u}({\hat{u}}(\rho ),\rho ) = - (R+\rho )(R+\rho )^{-1}(B^\mathrm{T} \lambda + b) + (B^\mathrm{T} \lambda + b) = 0$. Then, the conclusion of the theorem follows by elementary calculus. $\square$

The dual function with respect to a given ${\hat{u}}(\rho )$ is defined as

$$\begin{aligned} \begin{aligned} P_d(\rho )&:= L({\hat{u}}(\rho ),\rho )\\&= \frac{1}{2} {\hat{u}}^\mathrm{T}(\rho )R{\hat{u}}(\rho ) + (B^\mathrm{T} \lambda +b)^\mathrm{T} {\hat{u}}(\rho ) + \frac{\rho }{2} ({\hat{u}}^\mathrm{T}(\rho ) {\hat{u}}(\rho )-1)\\&= -\frac{1}{2}( B^\mathrm{T}\lambda +b )^\mathrm{T} (R + \rho )^{-1} (B^\mathrm{T}\lambda +b) - \frac{1}{2}\rho . \end{aligned} \end{aligned}$$

(15)

The first derivative function of $P_d(\rho )$ with respect to $\rho$ can be expressed as

$$\begin{aligned} \frac{\mathrm{d} P_d(\rho )}{\mathrm{d} \rho } = \frac{1}{2}({\hat{u}}^\mathrm{T}(\rho ) {\hat{u}}(\rho ) -1). \end{aligned}$$

(16)

Furthermore, we have the following result.

Lemma 3.1

For all $\rho \in G$, $\frac{\mathrm{d} P_d(\rho )}{\mathrm{d} \rho } \le 0$ if and only if

${\hat{u}}(\rho ) \in D: = \{ u : u^\mathrm{T}u \le 1,u \in {\mathbb {R}}^n \}$.

Proof

By the definition of the feasible set D, a point $u \in D$ if and only if ${\hat{u}}^\mathrm{T}(\rho ) {\hat{u}}(\rho ) -1\le 0$. Then, by (16) we see that for $\rho \in G$ , ${{\hat{u}}}(\rho )\in D$ if and only if $\frac{\mathrm{d}P_d(\rho )}{\mathrm{d} \rho } \le 0$. $\square$

4 Solving Quadratically Constrained Quadratic Programming by Dual Problem

For a given quadratic function $P(u): = \frac{1}{2} u^\mathrm{T}Ru + (B^\mathrm{T}\lambda +b)^\mathrm{T}u$, we have the differential flow with respect to the quadratically constrained quadratic programming problem (P)

$$\begin{aligned} {\hat{u}}(\rho ) = -(R+\rho I)^{-1} ( B^\mathrm{T}\lambda +b). \end{aligned}$$

(17)

According to the dual function (15), we have the dual problem

$$\begin{aligned} (P_d):\max _{\rho \ge 0,\>\> \rho \in G}\{P_d(\rho )\}. \end{aligned}$$

(18)

Theorem 4.1

If $\rho ^* \in G$ is an optimal solution to the dual problem (18), then the quadratic function P(u) with $u\in D$ has a minimizer at ${\hat{u}}(\rho ^*)$.

Proof

First, the optimal solution of the dual problem (18) $\rho ^*$ is also the maximizer of the function $P_d(\rho )$ with $\rho \ge 0$. If it is not true, then there exists a sequence $(\rho ^{(k)})$, $k\ge 1$, which tends to $\rho ^*$ and satisfies $P_d(\rho ^{(k)})>P_d(\rho ^*)$. Recall that $\rho ^* \in G$, and therefore $R+\rho ^* I>0$. For k is sufficiently large, we have $R+\rho ^{(k)} I\ge 0$, which implies that $\rho ^{(k)} \in G$. It contradicts our assumption that $\rho ^* =\arg {\min _{\rho \in G}\{P_d(\rho )}\}$. $\square$

The fact that $\rho ^*$ is a local maximizer of $P_d(\rho )$ with $\rho \ge 0$ implies that it is a K-T point of the problem $\max _{\rho \ge 0}\{P_d(\rho )\}$. It means that $\frac{\mathrm{d} P_d(\rho ^*)}{\mathrm{d} \rho }\le 0$ and $\rho ^* \frac{\mathrm{d} P_d(\rho ^*)}{\mathrm{d}\rho }=0$. By (15) and Lemma 2.1, we have ${\hat{u}}^\mathrm{T}(\rho ^*) {\hat{u}}(\rho ^*) - 1 \le 0$, i.e., ${\hat{u}}(\rho ^*) \in D$ and

$$\begin{aligned} \rho ^*({\hat{u}}^\mathrm{T}(\rho ^*) {\hat{u}}(\rho ^*) - 1 )= 0. \end{aligned}$$

(19)

For any $\rho \in G$ and $u\in D$, we have

$$\begin{aligned} \begin{aligned} P(u)&\ge P(u) +\frac{\rho }{2}(u^\mathrm{T} u - 1)= L(u,\rho ) \\&\ge \min _{u\in {\mathbb {R}}^n} L(u,\rho )=L({\hat{u}}(\rho ),\rho )= P_d(\rho ). \end{aligned} \end{aligned}$$

(20)

Since $\rho ^*\in G$, it follows that

$$\begin{aligned} \begin{aligned} P(u)&\ge P_d(\rho ^*)= L({\hat{u}}(\rho ^*),\rho ^*)\\&= P({\hat{u}}(\rho ^*)) + \frac{\rho ^*}{2} ({\hat{u}}^\mathrm{T}(\rho ^*) {\hat{u}}(\rho ^*) - 1 )\\&=P({\hat{u}}(\rho ^*)). \end{aligned} \end{aligned}$$

(21)

Thus, ${\hat{u}}(\rho ^*)$ is a global minimum point of P(u) over D.

5 Pontryagin Extremal Control and the Canonical Backward Differential Flow

According to the Pontryagin’s maximum principle [8, 9], an optimal control is an extremal control. For (2), the Hamiltonian function of (P), an extremal control ${\hat{u}}(\cdot )$ with the associated state ${\hat{x}}(\cdot )$ and costate ${\hat{\lambda }}(\cdot )$ together, satisfies

$$\begin{aligned} \dot{{\hat{x}}}= & {} H_\lambda =A{\hat{x}}+B{\hat{u}},{\hat{x}}(0)=x_0, \nonumber \\ \dot{{\hat{\lambda }}}= & {} -H_x = - F_{{\hat{x}}}({\hat{x}}) -A^\mathrm{T}{\hat{\lambda }},{\hat{\lambda }}(T)=0, \end{aligned}$$

(22)

and for almost every given $t\in [0,T]$,

$$\begin{aligned} H(t,{\hat{x}}(t), {\hat{u}}(t), {\hat{\lambda }}(t) ) = \min _{ u\in U } H(t, {\hat{x}}(t), u, {\hat{\lambda }}(t) ). \end{aligned}$$

(23)

Here,

$$\begin{aligned} H(t,x,u,\lambda )= F(x(t)) + \frac{1}{2}u^\mathrm{T}(t)Ru(t) + b^\mathrm{T}u(t) + \lambda ^\mathrm{T}(t) ( Ax(t) + Bu(t) ). \end{aligned}$$

(24)

Since the global optimization in (23) is solved for a fixed t, noting that the variable x and u are separating, it is helpful to consider the following optimization problem with a given parameter vector $\lambda$ at first

$$\begin{aligned} (P_1)~~ \begin{array}{c} \min P(u)=\frac{1}{2}u^\mathrm{T}Ru+(B^\mathrm{T}\lambda +b)^\mathrm{T}u\\ s.t.~~u^\mathrm{T}u\le 1. \end{array} \end{aligned}$$

(25)

In what follows, we use the theory of canonical differential flow [6] to solve the optimization problem $(P_1)$. Then, we can prove that the minimizer of the problem $(P_1)$ is on the backward differential flow, and we can point it out by a nonnegative parameter $\rho$. The details are as follows. Since $R > 0$, $R+\rho I>0$ when $\rho \ge 0$. If $B^\mathrm{T}\lambda +b \ne 0$, then there is a $\rho ^* > 0$ satisfying

$$\begin{aligned} 0<\Vert (R+\rho ^*I)^{-1}(B^\mathrm{T}\lambda +b)\Vert <1. \end{aligned}$$

(26)

Let $u^*=-(R+\rho ^*I)^{-1}(B^\mathrm{T}\lambda +b)$. Solving the backward differential equation

$$\begin{aligned} \begin{array}{c} \frac{\mathrm{d} u}{\mathrm{d} \rho }+[R+\rho I]^{-1}u=0, \\ u(\rho ^*)=u^*, \end{array} \end{aligned}$$

(27)

one may get the so-called canonical differential flow [6] for the optimization problem $(P_1)$

$$\begin{aligned} {\hat{u}}(\rho )=-(R+\rho I)^{-1}(B^\mathrm{T}\lambda +b). \end{aligned}$$

(28)

It is easy to verify, when $\rho >0$,

$$\begin{aligned}&\frac{\mathrm{d} (P({\hat{u}}(\rho )) + \frac{\rho }{2} ( {\hat{u}}^\mathrm{T} (\rho ) {\hat{u}} (\rho ) -1))}{\mathrm{d} {{\hat{u}}}}\nonumber \\&\quad =\frac{\mathrm{d} (P({\hat{u}}(\rho ) ))}{\mathrm{d} {{\hat{u}}}}+ \rho {\hat{u}} (\rho )\nonumber \\&\quad =R{\hat{u}} (\rho ) + B^\mathrm{T} \lambda +b + \rho {\hat{u}} (\rho )=0. \end{aligned}$$

(29)

$$\begin{aligned}&\frac{\mathrm{d}^2 ( P({\hat{u}} (\rho )) + \frac{\rho }{2} ({\hat{u}}^\mathrm{T} (\rho ){\hat{u}} (\rho )-1 ))}{\mathrm{d} {{\hat{u}}}^2}=R+ \rho I>0. \end{aligned}$$

(30)

Lemma 5.1

$\parallel {\hat{u}} (\rho ) \parallel$ monotonically decreases when $\rho \in [0,+\infty )$.

Proof

Since

$$\begin{aligned} \Vert {\hat{u}}(\rho )\Vert ^2 = (B^\mathrm{T}\lambda + b)^\mathrm{T}(R+\rho I)^{-2}(B^\mathrm{T}\lambda + b), \end{aligned}$$

(31)

and $R+\rho I > 0$ for $\rho \ge 0$, it follows that

$$\begin{aligned} \frac{\mathrm{d}\Vert {\hat{u}}(\rho )\Vert ^2}{\mathrm{d}\rho } = -2( B^\mathrm{T}\lambda + b )^\mathrm{T} (R+\rho I)^{-3} (B^\mathrm{T}\lambda + b) \le 0. \end{aligned}$$

(32)

It is deduced that $\Vert {\hat{u}}\Vert$ monotonically decreases when $\rho \in [0,+\infty )$. $\square$

6 Solution to the Optimization Problem $(P_1)$

Theorem 6.1

• (i) When $\Vert R^{-1}(B^\mathrm{T}\lambda +b)\Vert >1$, the optimization problem $(P_1)$ has a minimizer

  $$\begin{aligned} {\hat{u}}_\lambda = -[R + \rho _\lambda I]^{-1}(B^\mathrm{T}\lambda +b), \end{aligned}$$

  (33)

  where $\rho _\lambda$ is the only positive root of $\Vert [R+\rho I ] ^{-1}(B^\mathrm{T}\lambda +b)\Vert =1$.

• (ii) When $\Vert R^{-1}(B^\mathrm{T}\lambda +b)\Vert \le 1$, the optimization problem $(P_1)$ has a minimizer

  $$\begin{aligned} {\hat{u}}_\lambda =-R ^{-1}(B^\mathrm{T}\lambda +b). \end{aligned}$$

  (34)

Proof

• (i) Let $\Vert R^{-1}(B^\mathrm{T}\lambda +b)\Vert >1$. For $\lim _{\rho \rightarrow \infty }\Vert {\hat{u}}(\rho )\Vert =0$ and based on Lemma 5.1, there is a $\rho _\lambda >0$ so that

  $$\begin{aligned} \Vert {\hat{u}}(\rho _\lambda )\Vert =1. \end{aligned}$$

  (35)

  Let ${\hat{u}}_\lambda = {\hat{u}}(\rho _\lambda )=-(R+\rho _\lambda I)^{-1}(B^\mathrm{T}\lambda + b)$. Then, we have

  $$\begin{aligned} \frac{\mathrm{d} (P({\hat{u}}_\lambda ) + \frac{\rho _\lambda }{2}({\hat{u}}^\mathrm{T}_\lambda {\hat{u}}_\lambda - 1))}{d {\hat{u}}_\lambda }= \frac{\mathrm{d} P({\hat{u}}_\lambda )}{d {\hat{u}}_\lambda } +\rho _\lambda {\hat{u}}_\lambda = (R + \rho _\lambda I){\hat{u}}_\lambda + B^\mathrm{T}\lambda + b =0.\nonumber \\ \end{aligned}$$

  (36)

  Further, we have

  $$\begin{aligned} \frac{\mathrm{d}^2 (P({\hat{u}}_\lambda ) + \frac{\rho _\lambda }{2}({\hat{u}}^\mathrm{T}_\lambda {\hat{u}}_\lambda - 1))}{d {\hat{u}}^2 _\lambda }= R + \rho _\lambda I >0. \end{aligned}$$

  (37)

  Thus, for any $u\in U=\{u^\mathrm{T}u \le 1 \}$,

  $$\begin{aligned} P(u) \ge P(u) + \frac{\rho _\lambda }{2} ( u^\mathrm{T}u - 1 ) \ge P({\hat{u}}_\lambda )+\frac{\rho _\lambda }{2} ( {\hat{u}}^\mathrm{T}_\lambda {\hat{u}}_\lambda - 1 ) = P({\hat{u}}_\lambda ). \end{aligned}$$

  (38)

  This shows that for the case(i) the optimization problem ($P_1$) has a minimizer

  $$\begin{aligned} {\hat{u}}_\lambda = -[R+\rho _\lambda I]^{-1}(B^\mathrm{T}\lambda +b). \end{aligned}$$

  (39)

• (ii) Let $\Vert R^{-1}(B^\mathrm{T}\lambda +b)\Vert \le 1$. By Lemma 5.1, for this case, $\Vert {\hat{u}}(\rho )\Vert \le 1$ in $[0,+\infty )$. Noting (29) and (30), for any $u\in U$ and $\rho \ge 0$, we have

  $$\begin{aligned} P(u) \ge P(u) + \frac{\rho }{2}(u^\mathrm{T}u-1) \ge P({\hat{u}}) + \frac{\rho }{2}({\hat{u}}^\mathrm{T}(\rho ){\hat{u}}(\rho )-1). \end{aligned}$$

  (40)

  Let ${\hat{u}}_\lambda = -R^{-1}(B^\mathrm{T}\lambda + b)$. Consequently,

  $$\begin{aligned} P(u) \ge P({\hat{u}}_\lambda ) + \frac{0}{2}({\hat{u}}^\mathrm{T}_\lambda {\hat{u}}_\lambda -1) = P({\hat{u}}_\lambda ). \end{aligned}$$

  (41)

  This shows that for the case (ii) the optimization problem ($P_1$) has a minimizer ${\hat{u}}_\lambda = -R^{-1}(B^\mathrm{T}\lambda +b)$. $\square$

7 Solution to the Linear Optimal Control Problem (P)

In what follows, we use the notation $\arg \{f(\rho )=0,\rho \ge 0\}$ to stand for the positive root of the equation $f(\rho )=0$. For a given vector $\lambda$ such that

$$\begin{aligned} \Vert R^{-1}(B^\mathrm{T}\lambda +b)\Vert >1, \end{aligned}$$

(42)

we denote

$$\begin{aligned} \rho _\lambda : = \arg \{\Vert [R+\rho I]^{-1}(B^\mathrm{T}\lambda +b)\Vert =1,\rho \ge 0\}. \end{aligned}$$

(43)

For the optimal control problem (P), we define the control $u(\lambda )$ as follows.

Definition 7.1

For a vector $\lambda$, define

$$\begin{aligned} u( \lambda ) := {\left\{ \begin{array}{ll} -[R+\rho _\lambda I]^{-1}(B^\mathrm{T} \lambda + b),&{} \text {if}~ \Vert R^{-1}(B^\mathrm{T} \lambda +b)\Vert > 1,\\ -R^{-1}(B^\mathrm{T} \lambda + b),&{} \text {if}~ \Vert R^{-1}(B^\mathrm{T} \lambda + b)\Vert \le 1, \end{array}\right. } \end{aligned}$$

(44)

where

$$\begin{aligned} \rho _\lambda = \arg \{ \Vert [ R+ \rho I ] ^ {-1} (B^\mathrm{T} \lambda +b)\Vert = 1, \rho \ge 0 \}. \end{aligned}$$

Lemma 7.1

$u( {\hat{\lambda }} ) \in C( {\mathbb {R}}^n , {\mathbb {R}}^m)$.

Recall the linear system

$$\begin{aligned} x(t) = e^{At}x_0 + \int _0^t e^{A(t-s)} Bu(s) \mathrm{d}s. \end{aligned}$$

(45)

It is not difficult to see that

$$\begin{aligned} \Vert x(t)\Vert&\le e^{\Vert A\Vert t} \Vert x_0\Vert + \int _0^t e^{\Vert A\Vert (t-s)} \Vert B\Vert \Vert u(s)\Vert \mathrm{d}s \end{aligned}$$

(46)

$$\begin{aligned}&\le e^{\Vert A\Vert t} \Vert x_0\Vert + \Vert B\Vert \int _0^t e^{\Vert A\Vert (t-s)}\mathrm{d}s \end{aligned}$$

(47)

$$\begin{aligned}&\le e^{\Vert A\Vert T} \left( \Vert x_0\Vert + \frac{\Vert B\Vert }{\Vert A\Vert }\right) :=C_1. \end{aligned}$$

(48)

Let $\alpha :=\max \{C_1,Te^{\Vert A\Vert T}(\Vert B\Vert +1)\}$. Because $F(x) \in C^2({\mathbb {R}}^n)$, $F_x(x)$ is bounded in $S:=\{x:\Vert x\Vert \le \alpha \}$ and $\sup _{x\in S} \Vert F_x(x)\Vert := C_2$. The original optimal control problem (P) is equivalent to the following optimal control problem

$$\begin{aligned} (P_2)~~ \begin{array}{l} \min J(u)=\frac{1}{C_2}\int ^\mathrm{T}_{0} [F(x(t)) + \frac{1}{2} u^\mathrm{T}(t)Ru(t) + b^\mathrm{T}u(t)]\mathrm{d}t,\\ s.t.~ {\dot{x}}(t)=Ax(t)+Bu(t),\\ ~~~~~x(0)=x_0\in {\mathbb {R}}^n,\\ ~~~~~u(t)\in U=\{u:u^\mathrm{T}u\le 1\}\subset {\mathbb {R}}^m, t\in [0,T]. \end{array} \end{aligned}$$

(49)

According to Theorem 6.1, the optimal control of problem (49) is

$$\begin{aligned} {\hat{v}}( {\hat{\omega }} ) = {\left\{ \begin{array}{ll} -[R+\rho _{{\hat{\omega }}} I]^{-1}(B^\mathrm{T} {\hat{\omega }} + b),&{} \text {if}~ \Vert R^{-1}(B^\mathrm{T} {\hat{\omega }}+b)\Vert > 1,\\ -R^{-1}(B^\mathrm{T} {\hat{\omega }} + b),&{} \text {if}~ \Vert R^{-1}(B^\mathrm{T} {\hat{\omega }} + b)\Vert \le 1. \end{array}\right. } \end{aligned}$$

(50)

The Pontryagin boundary value problem can be rewritten as

$$\begin{aligned} (BVP)~~ \begin{array}{l} {\dot{Y}}={\hat{A}}Y+f(Y),\\ H_1Y(0)+H_2Y(T)=Y_0. \end{array} \end{aligned}$$

(51)

Theorem 7.1

There is a solution to the problem (51).

Proof

Note that the matrix function $\varPhi (t): = e^{{\hat{A}}t}$ satisfies

$$\begin{aligned} det \big ( H_1\varPhi (0) + H_2\varPhi (T) \big ) = det \left( \begin{array}{cc} I&{}0\\ 0&{}e^{-A^\mathrm{T}T} \end{array} \right) \ne 0. \end{aligned}$$

The solution to the corresponding homogeneous problem

$$\begin{aligned} {\dot{Y}}={\hat{A}}Y,\>\>U(Y)=H_1Y(0) + H_2Y(T) =Y_0 \end{aligned}$$

(52)

is

$$\begin{aligned} Y(t) = \varPhi (t) \big ( H_1\varPhi (0) + H_2\varPhi (T) \big )^{-1} Y_0 = \varPhi (t) \left( \begin{array}{cc} I&{}0\\ 0 &{}e^{-A^\mathrm{T}T} \end{array} \right) ^{-1}Y_0. \end{aligned}$$

The Green function [10] to the homogeneous BVP is

$$\begin{aligned} G(t,s):= {\left\{ \begin{array}{ll} -\varPhi (t)\big ( H_1\varPhi (0) + H_2\varPhi (T) \big )^{-1} H_2 \varPhi (T) \varPhi ^{-1}(s),&{}0<t<s<T,\\ \varPhi (t) \big [ I-\big ( H_1\varPhi (0) + H_2\varPhi (T) \big )^{-1} H_2 \varPhi (T) \big ] \varPhi ^{-1}(s),&{}0<s<t<T. \end{array}\right. }\nonumber \\ \end{aligned}$$

(53)

This is to say

$$\begin{aligned}&G(t,s) = {\left\{ \begin{array}{ll} -\left( \begin{array}{cc} 0&{}0\\ 0&{}e^{-A^\mathrm{T}(t-s)} \end{array} \right) ,&{}0<t<s<T,\\ \left( \begin{array}{cc} e^{A(t-s)}&{}0\\ 0&{}0 \end{array} \right) ,&0<s<t<T, \end{array}\right. } \end{aligned}$$

(54)

$$\begin{aligned}&\Vert G(t,s)\Vert \le e^{\Vert A\Vert t} \le e^{\Vert A\Vert T}. \end{aligned}$$

(55)

In what follows, we show that there exists one solution Y(t) of the Pontryagin boundary value problem (51), which is equivalent to the solvability of the following integral equation

$$\begin{aligned} Y(t) = \int _0^\mathrm{T}{G(t,s)f(Y(s))\mathrm{d}s}, \end{aligned}$$

where $f(Y): = \left( \begin{array}{c} Bv((0,I)Y)\\ -\frac{1}{C_2} F_x((I,0)Y) \end{array} \right)$.

Let $X:=C([0,T],{\mathbb {R}}^{2n})$ and $\varOmega :=\{Y\in X:\Vert Y(\cdot )\Vert \le \alpha \}$. Define an operator $T:X\rightarrow X$, for given $t\in [0,T]$, each $Y\in X$,

$$\begin{aligned} (TY)(t) = \int _0^\mathrm{T}{G(t,s)f(Y(s))\mathrm{d}s}. \end{aligned}$$

(56)

Since $\Vert v((0,I)Y)\Vert \le 1$, if $Y(\cdot )\in \varOmega$, then $\Vert -\frac{1}{C_2} F_x((I,0)Y)\Vert \le 1$.

Consequently,

$$\begin{aligned} \Vert TY(t)\Vert =\Vert \int _0^\mathrm{T} G(t,s)f(Y(s))\mathrm{d}s\Vert \le Te^{\Vert A\Vert T}(\Vert B\Vert +1)\le \alpha . \end{aligned}$$

(57)

For $\Vert TY\Vert = \max _{t\in [0,T]} \Vert (TY)(t)\Vert \le \alpha$, $T\varOmega \subset \varOmega$. By Schaefer’s fixed-point theorem [11], there is a ${\hat{Y}}\in \varOmega$ such that $T{\hat{Y}}={\hat{Y}}$, i.e.,

$$\begin{aligned} T{\hat{Y}}=\int _0^\mathrm{T} {G(t,s)f({\hat{Y}}(s))\mathrm{d}s}. \end{aligned}$$

It follows that the Pontryagin boundary value problem (51) has a solution. $\square$

Theorem 7.2

Let ${\hat{Y}}(\cdot ):=({{\hat{y}}}(\cdot ),{{\hat{\omega }}}(\cdot ))^\mathrm{T}$ be a solution to the problem (51). Then, the control ${\hat{v}}(t) := v \big ( (0,I) {\hat{Y}}(t) \big )=v({{\hat{\omega }}}(t))$ is a Pontryagin extremal control.

Proof

Note that ${\hat{y}}(\cdot )$ and ${\hat{\omega }}(\cdot )$ satisfy the equations

$$\begin{aligned} \dot{{\hat{y}}}= & {} A{\hat{y}}+B{\hat{v}}({\hat{\omega }}),{\hat{y}}(0)=x_0,\nonumber \\ \dot{{\hat{\omega }}}= & {} -\frac{1}{C_2} F_{{\hat{y}}} ({\hat{y}}) -A^\mathrm{T}{\hat{\omega }},{\hat{\omega }}(T)=0. \end{aligned}$$

(58)

It implies that ${\hat{y}}(\cdot )$, ${\hat{\omega }}(\cdot )$ and ${\hat{\omega }}$ satisfy the Pontryagin boundary value equations. From Theorem 6.1 and Definition (50), it follows that

$$\begin{aligned} {\hat{v}}(t)=\arg \{\min _{v\in D} H^1(t,{\hat{y}}(t),v,{\hat{\omega }}(t))\},~~a.e. ~t\in [0,T], \end{aligned}$$

(59)

where $H^1(t,{\hat{y}}(t),v,{\hat{\omega }}(t))$ is the Hamiltonian function of the optimal control problem ($P_2$). So ${\hat{v}}(t)$ is a Pontryagin extremal control. $\square$

Theorem 7.3

The control ${\hat{v}}(t) = v \big ( (0,I){\hat{Y}}(t) \big )$ defined by (50) is an optimal control to the problem $(P_2)$ and (P).

Proof

Based on Theorem 7.2 and Definition (50), the extremal control ${\hat{v}}(t) = {\hat{v}} \big ( (0,I){\hat{Y}}(t) \big )$ can be expressed as a function of the costate variable ${\hat{\omega }}$, i.e., ${\hat{v}} ({\hat{\omega }})$. Bringing it back into the Hamiltonian function of problem ($P_2$), we get

$$\begin{aligned}&H^1(t,{\hat{y}}(t),{\hat{v}}({\hat{\omega }}(t)),{\hat{\omega }}(t) ) \nonumber \\&\quad =\frac{1}{C_2}(F({\hat{y}}(t)) + \frac{1}{2}{\hat{v}}({\hat{\omega }}(t))^\mathrm{T}R{\hat{v}}({\hat{\omega }}(t)) + b^\mathrm{T}{\hat{v}}({\hat{\omega }}(t))) + {\hat{\omega }}^\mathrm{T}(t) ( A{\hat{y}}(t) + B{\hat{v}}({\hat{\omega }}(t)) ).\nonumber \\ \end{aligned}$$

(60)

Since ${\hat{v}}({\hat{\omega }})$ is independent of the state variable ${\hat{y}}$ and $F_{{\hat{y}}{\hat{y}}}({\hat{y}}) \ge 0$, $H ^1(t,{\hat{y}} ,{\hat{v}}({\hat{\omega }}),{\hat{\omega }} )$ is convex with respect to the variable ${\hat{y}}$. Referring to the classical optimal control theory [9, 12], we see that ${{\hat{v}}}(t)$ is an optimal control to the singular optimal control problem $(P_2)$, i.e., ${{\hat{u}}}(t)= {\hat{v}}$ is an optimal control to the singular optimal control problem (P). $\square$

8 Example

Example 8.1

Consider the following optimal control problem

$$\begin{aligned} \begin{array}{l} \min J(u)=\int ^\mathrm{T}_{0} [ x^4(t)+x^2(t) + \frac{1}{2} ru^2(t) + cu(t)]\mathrm{d}t,\\ s.t.~ {\dot{x}}(t)=ax(t)+bu(t),\\ ~~~~~x(0)=x_0\in {\mathbb {R}}^n,\\ ~~~~~u(t)\in U=\{u:u^\mathrm{T}u\le 1\}\subset {\mathbb {R}}^m, t\in [0,T]. \end{array} \end{aligned}$$

(61)

According to the Pontryagin principle and Theorem 6.1, the Pontryagin pair $({\hat{x}}(t),{\hat{u}}(t),{\hat{\lambda }}(t))$ satisfies

$$\begin{aligned} \dot{{\hat{x}}}= & {} H_\lambda (t,{\hat{x}},{\hat{u}},{\hat{\lambda }}) =A{\hat{x}}+B{\hat{u}},{\hat{x}}(0)=x_0 \nonumber \\ \dot{{\hat{\lambda }}}= & {} -H_x (t,{\hat{x}},{\hat{u}},{\hat{\lambda }})= - F_x({\hat{x}}) -A^\mathrm{T}{\hat{\lambda }},{\hat{\lambda }}(T)=0 \nonumber \\ {\hat{u}}_\lambda= & {} {\left\{ \begin{array}{ll} -[R + \rho _\lambda I]^{-1}(B^\mathrm{T}\lambda +b), &{} \text {if}~ \Vert R^{-1}(B^\mathrm{T}\lambda +b)\Vert >1\\ -R ^{-1}(B^\mathrm{T}\lambda +b), &{} \text {if}~ \Vert R^{-1}(B^\mathrm{T}\lambda +b)\Vert \le 1 \end{array}\right. }. \end{aligned}$$

(62)

By computation, we have

$$\begin{aligned} \dot{{\hat{x}}}= & {} a{\hat{x}}+b{\hat{u}} \nonumber \\ \dot{{\hat{\lambda }}}= & {} -4{\hat{x}}^3-2{\hat{x}} - a{\hat{\lambda }} \nonumber \\ {\hat{u}}= & {} {\left\{ \begin{array}{ll} -\frac{b{\hat{\lambda }}+c}{r},&{} \text {if}~|b{\hat{\lambda }}+c| \le |r| \\ -\frac{b{\hat{\lambda }}+c}{|b{\hat{\lambda }}+c|},&{} \text {if}~|b{\hat{\lambda }}+c| > |r| \end{array}\right. }. \end{aligned}$$

(63)

By an iterative algorithm, we can solve this equation. For example, $r=10$, $c=1$, $a=1$, $b=2$, $T=1$ and $x_0= 1$. We figure out the optimal control ${\hat{u}}(\cdot )$ and coordinate trajectory ${\hat{x}}(\cdot )$ in Fig. 1, and the optimal value is 130.3525.

Fig. 1

Optimal control and optimal trajectory of Example 8.1

Example 8.2

Consider another example where $m=2$, i.e., $u\in {\mathbb {R}}^2$.

$$\begin{aligned} \begin{array}{l} \min J(u)=\int ^\mathrm{T}_{0} [ \frac{1}{6}x_1^6(t)+\frac{1}{4}x_2^4(t) + \frac{1}{2} u^\mathrm{T}(t)u(t) + b^\mathrm{T}u(t)]\mathrm{d}t,\\ s.t.~ {\dot{x}}(t)=Ax(t)+Bu(t),\\ ~~~~~x(0)=x_0\in {\mathbb {R}}^n,\\ ~~~~~u(t)\in U=\{u:u^\mathrm{T}u\le 1\}\subset {\mathbb {R}}^m, t\in [0,T], \end{array} \end{aligned}$$

(64)

where $T=1$, $b=\left( \begin{array}{cc} 2 \\ 4 \end{array} \right)$, $A=\left( \begin{array}{cc} 1&{}3\\ 2&{}5 \end{array} \right)$, $B=\left( \begin{array}{cc} 2&{}2\\ 3&{}7 \end{array} \right)$ and $x_0=\left( \begin{array}{cc} 1\\ 2 \end{array} \right)$.

Same as Example 8.1, the Pontryagin pair $({\hat{x}}(t),{\hat{u}}(t),{\hat{\lambda }}(t))$ satisfies

$$\begin{aligned} \dot{\hat{x_1}}= & {} \hat{x_1}+ 3\hat{x_2} + 2\hat{u_1} + 4\hat{u_2} \nonumber \\ \dot{\hat{x_2}}= & {} 2\hat{x_1}+ 5\hat{x_2} + 2\hat{u_1} + 4\hat{u_2}\nonumber \\ \dot{\hat{\lambda _1}}= & {} -\hat{x_1}^5 - \hat{\lambda _1} - 2\hat{\lambda _1} \nonumber \\ \dot{\hat{\lambda _2}}= & {} -\hat{x_2}^3 - 2\hat{\lambda _1} - 5\hat{\lambda _1} \end{aligned}$$

(65)

and

$$\begin{aligned} {\hat{u}}= {\left\{ \begin{array}{ll} -\frac{B^\mathrm{T}{\hat{\lambda }}+b}{||B^\mathrm{T}{\hat{\lambda }}+b||},&{} \text {if}~||B^\mathrm{T}{\hat{\lambda }}+b|| >1, \\ -(B^\mathrm{T}{\hat{\lambda }}+b),&{} \text {if}~||B^\mathrm{T}{\hat{\lambda }}+b|| <1. \end{array}\right. } \end{aligned}$$

(66)

Figure 2 shows the optimal control and corresponding trajectory, and the optimal value is $3.4688\times 10^5$.

Fig. 2

Optimal control and the optimal trajectory of Example 8.2

9 Conclusions

In this paper, a new approach to constrained finite-horizon optimal control problems with linear systems has been investigated using the canonical backward differential flows method, which can produce an analytic form of the optimal feedback control to the optimal control problems. Meanwhile, we give the extremal control explicitly with the trajectory by canonical backward differential flows. The existence of the optimal solution for this optimal control problem has been proved (see Theorem 7.2). More research needs to be done for the development of applicable canonical differential flows theory.