Keywords

Introduction

The problem of finding sufficient conditions for optimality for a stochastic optimal control problem with finite-dimensional state equation had been well studied since the pioneering work of Bismut (1976, 1978). In particular, Bismut introduced linear backward stochastic differential equations (BSDEs) which have become an active domain of research since the seminal paper of Pardoux and Peng in 1990 concerning (nonlinear) BSDEs in Pardoux and Peng (1990).

The first results on SMP concerned only the stochastic systems where the control domain is convex or the diffusion coefficient does not contain control variable. In this case, only the first-order expansion is needed. This kind of SMP was developed by Bismut (1976, 1978), Kushner (1972), and Haussmann (1986). It is important to note that (Bismut 1978) introduced linear BSDE to represent the first-order adjoint process.

Peng made a breakthrough by establishing the SMP for the general stochastic optimal control problem where the control domain need not to be convex and the diffusion coefficient can contain the control variable. He solved this general case by introducing the second-order expansion and second-order BSDE. We refer to the book Yong and Zhou (1999) for the account of the theory of SMP in finite-dimensional spaces and describe Peng’s SMP in the next section.

Despite the fact that the problem has been solved in complete generality more than 20 years ago, the infinite-dimensional case still has important open issues both on the side of the generality of the abstract model and on the side of its applicability to systems modeled by stochastic partial differential equations (SPDEs). The last section is devoted to the recent development of SMP in infinite-dimensional space.

Statement of SMP

Formulation of Problem

Let \((\Omega,\mathcal{F}, \mathbb{P})\) be a complete probability space, on which an m-dimensional Brownian motion W is given. Let \(\{\mathcal{F}_{t}\}_{t\geq 0}\) be the natural completed filtration of W.

We consider the following stochastic controlled system:

$$\displaystyle\begin{array}{rcl} dx(t)& =& b(x(t),u(t))dt +\sigma (x(t),u(t))dW(t), \\ & & \ \ x(0) = x_{0}, {}\end{array}$$
(1)

with the cost functional

$$\displaystyle\begin{array}{rcl} J(u(\cdot )) = \mathbb{E}\left \{\int _{0}^{T}f(x(t),u(t))dt + h(x(T))\right \}.& &{}\end{array}$$
(2)

In the above, b, σ, f, h are given functions with appropriate dimensions. (U, d) is a separable metric space.

We define

$$\displaystyle\begin{array}{rcl} \mathcal{U}& =& \{u : [0,T] \times \Omega \\ & & \ \ \rightarrow U\ \vert \ u\mbox{ is }\{\mathcal{F}_{t}\}_{t\geq 0} -\mbox{ adapted }\}.{}\end{array}$$
(3)

The optimal problem is: Minimize J(u(⋅ )) over \(\mathcal{U}\).

Any \(\bar{u} \in \mathcal{U}\) satisfying

$$\displaystyle{ J(\bar{u}) =\inf _{u\in \mathcal{U}}J(u) }$$
(4)

is called an optimal control. The corresponding \(\bar{x}\) and \((\bar{x},\bar{u})\) is called an optimal state process/trajectory and optimal pair, respectively. In this section, we assume the following standard hypothesis:

Hypothesis 1

  1. 1.

    The functions \(b : \mathbb{R}^{n} \times U\mapsto \mathbb{R}^{n}\) , \(\sigma = (\sigma ^{1},\cdots \,,\sigma ^{m}) : \mathbb{R}^{n} \times U\mapsto \mathbb{R}^{n\times m}\) , \(f : \mathbb{R}^{n} \times U\mapsto \mathbb{R}\) and \(h : \mathbb{R}^{n}\mapsto \mathbb{R}\) are measurable functions.

  2. 2.

    For \(\varphi = b,\sigma ^{j},j = 1,\cdots \,,m,f\) , the functions \(x\mapsto \varphi (x,u)\) and \(x\mapsto h(x)\) are C 2 , denoted \(\varphi _{x}\) and \(\varphi _{xx}\) (respectively, h x and h xx ), which are also continuous functions of (x,u).

  3. 3.

    There exists a constant K > 0 such that

    $$\displaystyle{\vert \varphi _{x}\vert + \vert \varphi _{xx}\vert + \vert h_{x}\vert + \vert h_{xx}\vert \leq K,}$$

    and

    $$\displaystyle{\vert \varphi \vert + \vert h\vert \leq K(1 + \vert x\vert + \vert u\vert ).}$$

Adjoint Equations

Let us first introduce the following backward stochastic differential equations (BSDEs).

$$\displaystyle\begin{array}{rcl} dp(t)& =& -\{b_{x}(\bar{x}(t),\bar{u}(t))^{T}p(t) \\ & & +\sum _{j=1}^{m}\sigma _{ x}^{j}(\bar{x}(t),\bar{u}(t))^{T}q_{ j}(t) \\ & & \quad - f_{x}(\bar{x}(t),\bar{u}(t))\}dt + q(t)dW(t), \\ p(T)& =& -h_{x}(\bar{x}(T)). {}\end{array}$$
(5)

The solution (p, q) to the above BSDE (first-order BSDE) is called the first-order adjoint process.

$$\displaystyle\begin{array}{rcl} dP(t)& =& -\{b_{x}(\bar{x}(t),\bar{u}(t))^{T}P(t) + P(t)b_{ x}(\bar{x}(t),\bar{u}(t)) +\sum _{ j=1}^{m}\sigma _{ x}^{j}(\bar{x}(t),\bar{u}(t))^{T}P(t)\sigma _{ x}^{j}(\bar{x}(t),\bar{u}(t)) \\ & & \quad +\sum _{ j=1}^{m}\{\sigma _{ x}^{j}(\bar{x}(t),\bar{u}(t))^{T}Q_{ j}(t) + Q_{j}(t)\sigma _{x}^{j}(\bar{x}(t),\bar{u}(t))\} \\ & & \quad \quad + H_{xx}(\bar{x}(t),\bar{u}(t),p(t),q(t))\}dt +\sum _{ j=1}^{m}Q_{ j}(t)dW^{j}(t), \\ P(T)& =& -h_{xx}(\bar{x}(T)), {}\end{array}$$
(6)

where the Hamiltonian H is defined by

$$\displaystyle\begin{array}{rcl} H(x,u,p,q)& =& \langle p,b(x,u)\rangle \\ & & +\mbox{ tr}[q^{T}\sigma (x,u)] - f(x,u).{}\end{array}$$
(7)

The solution (P, Q) to the above BSDE (second-order BSDE) is called the second-order adjoint process.

Stochastic Maximum Principle

Let us now state the stochastic maximum principle.

Theorem 1

Let\((\bar{x},\bar{u})\)be an optimal pair of problem. Then there exist a unique couple (p,q) satisfying (5) and a unique couple (P,Q) satisfying (6) , and the following maximum condition holds:

$$\displaystyle\begin{array}{rcl} & & H(\bar{x}(t),\bar{u}(t),p(t),q(t)) - H(\bar{x}(t),u,p(t),q(t)) \\ & & \quad -\frac{1} {2}\mbox{ tr}(\{\sigma (\bar{x}(t),\bar{u}(t)) -\sigma (\bar{x}(t),u)\}^{T}P(t)\{\sigma (\bar{x}(t),\bar{u}(t)) -\sigma (\bar{x}(t),u)\}) \geq 0.{}\end{array}$$
(8)

SMP in Infinite-Dimensional Space

The problem of finding sufficient conditions for optimality for a stochastic optimal control problem with infinite-dimensional state equation, along the lines of the Pontryagin maximum principle, was already addressed in the early 1980s in the pioneering paper (Bensoussan 1983).

Whereas the Pontryagin maximum principle for infinite-dimensional stochastic control problems is a well-known result as far as the control domain is convex (or the diffusion does not depend on the control; see Bensoussan 1983; Hu and Peng 1990), for the general case (that is when the control domain need not be convex and the diffusion coefficient can contain a control variable), existing results are limited to abstract evolution equations under assumptions that are not satisfied by the large majority of concrete SPDEs.

The technical obstruction is related to the fact that (as it was pointed out in Peng 1990) if the control domain is not convex, the optimal control has to be perturbed by the so-called spike variation. Then if the control enters the diffusion, the irregularity in time of the Brownian trajectories imposes to take into account a second variation process. Thus, the stochastic maximum principle has to involve an adjoint process for the second variation. In the finite-dimensional case, such a process can be characterized as the solution of a matrix-valued backward stochastic differential equation (BSDE), while in the infinite-dimensional case, the process naturally lives in a non-Hilbertian space of operators and its characterization is much more difficult. Moreover, the applicability of the abstract results to concrete controlled SPDEs is another delicate step due to the specific difficulties that they involve such as the lack of regularity of Nemytskii-type coefficients in Lp spaces.

Concerning results on the infinite-dimensional stochastic Pontryagin maximum principle, as we already mentioned, in Bensoussan (1983) and Hu and Peng (1990), the case of diffusion independent on the control is treated (with the difference that in Hu and Peng (1990) a complete characterization of the adjoint to the first variation as the unique mild solution to a suitable BSDE is achieved).

The paper Tang and Li (1994) is the first one in which the general case is addressed with, in addition, a general class of noises possibly with jumps. The adjoint process of the second variation \((P_{t})_{t\in [0,T]}\) is characterized as the solution of a BSDE in the (Hilbertian) space of Hilbert-Schmidt operators. This forces to assume a very strong regularity on the abstract state equation and control functional that prevents application of the results in Tang and Li (1994) to SPDEs.

Then in the papers by Fuhrman et al. (2012, 2013), the state equation is formulated, only in a semiabstract way in order, on one side, to cope with all the difficulties carried by the concrete nonlinearities and, on the other, to take advantage of the regularizing properties of the leading elliptic operator.

Recently in Lü and Zhang (2012), P t was characterized as “transposition solution” of a backward stochastic evolution equation in \(\mathcal{L}(L^{2}(\mathcal{O}))\). Coefficients are required to be twice Fréchet differentiable as operators in \(L^{2}(\mathcal{O})\). Finally, even more recently in a couple of preprints (Du and Meng (2012, 2013)), the process P t is characterized in a similar way as it is in Fuhrman et al. (2012, 2013). Roughly speaking it is characterized as a suitable stochastic bilinear form. As it is the case in Lü and Zhang (2012), in Du and Meng (2012, 2013) as well, the regularity assumptions on the coefficients are too restrictive to apply directly the results in Lü and Zhang (2012), Du and Meng (2012, 2013) to controlled SPDEs.

Cross-References