1 Introduction

In this paper, we consider optimal mixed stochastic regular-singular control problems, where the state process satisfies the following stochastic differential equation:

$$\begin{aligned} \left\{ \begin{array}{l} dx_{t}=b\left( t,x_{t},u_{t},\omega \right) dt+\sigma \left( t,x_{t} ,u_{t},\omega \right) dB_{t}+\lambda \left( t,\omega \right) d\xi _{t};\\ x_{0}=x\in \mathbb {R}. \end{array} \right. \end{aligned}$$
(1.1)

The control is a pair \(\left( u_{t},\xi _{t}\right) \) such that \(u_{t}\) stands for the regular, called also the absolutely continuous part and \(\xi _{t}\) is the singular part.

The expected cost has the form

$$\begin{aligned} J\left( u,\xi \right) =E\left[ g\left( x_{T},\omega \right) + {\displaystyle \int \limits _{0}^{T}} f\left( t,x_{t},u_{t},\omega \right) dt+ {\displaystyle \int \limits _{0}^{T}} h\left( t,\omega \right) d\xi _{t}\right] ,\text { }\left( u,\xi \right) \in \mathcal {A}_{\mathcal {E}},\qquad \quad \end{aligned}$$
(1.2)

A major approach to deal with stochastic control problems is to derive optimality necessary conditions satisfied by some optimal control, known as the stochastic maximum principle. The first fundamental result on this subject was obtained by Kushner [24], for classical regular or absolutely continuous controls. Since then, a huge literature has been produced on this subject, among them, in particular, those by Benssoussan [10], Bismut [11], Haussmann [21] and Peng [30]. One can refer to the excellent book by Yong and Zhou [31] for a complete account on the subject and the references therein.

In this paper, we study general regular-singular stochastic control problems, in which the controller has only partial information. The control has two components, the first one is a classical regular control and the second one is a singular control. We consider systems driven by random coefficients and the running and the final costs are allowed to be random. It is clear that for such systems the dynamic programming does not hold, as the state process is no longer a Markov process. Our goal is to obtain necessary conditions for optimality satisfied by some optimal control.

We use Malliavin calculus techniques [27], to express the adjoint process in an explicit form. Our result extends those by Baghery and Oksendal [2], Meyer-Brandis et al. [25] and Øksendal and Sulem [29], to mixed regular-singular control problems. See also [26] for the mean field control problems. Note that in the stochastic maximum principle, a serious drawback is the computation at least numerically of the adjoint process. This process is given by a conditional expectation and satisfies a linear backward stochastic differential equation (BSDE). Numerical and Monte Carlo methods have been developed recently to deal with BSDEs by using Malliavin calculus, see [12, 13, 16, 19]. This could be seen as a step forward to solve numerically stochastic control problems by using these methods.

Stochastic control problems of singular type, have been studied extensively in the literature, as they model numerous situations in different areas, see [18, 28, 29]. A typical example in mathematical finance is the so called portfolio optimization problem, under transaction costs [17, 20]. These problems were studied through dynamic programming principle, see [22], where it was shown in particular that, the value function is continuous and is the unique viscosity solution of the HJB variational inequality. In particular the value function satisfies a variational inequality, which gives rise to a free boundary problem, and the optimal state process is a diffusion reflected at the free boundary. Bather and Chernoff [8] were the first to study such a problem. Benĕs et al. [9] solved a one dimensional example by observing that the value function in their example satisfies the so called the principle of smooth fit. Davis and Norman [17] solved the two dimensional problem, arising in portfolio selection models, under transaction costs. The case of diffusions with jumps has been studied in Øksendal and Sulem [28].

The first maximum principle for singular stochastic control problems was derived by Cadenillas and Haussmann [14], for systems with linear dynamics, convex cost criterion and convex state constraints. An extension to non linear systems has been developed via convex perturbations method for both absolutely continuous and singular components by Bahlali and Chala [3]. The second order stochastic maximum principle for nonlinear SDEs with a controlled diffusion matrix was obtained by Bahlali and Mezerdi [7], extending the Peng maximum principle [30] to singular control problems. Similar techniques have been used by Anderson [1] and Bahlali et al. [6], to study the stochastic maximum principle for relaxed-singular controls. The case of systems with non smooth coefficients has been treated by Bahlali et al. [4], where the classical derivatives are replaced by the generalized ones in the definition of adjoint processes. See also the recent paper by \(\varnothing \)ksendal and Sulem [29], where Malliavin calculus techniques have been used to define the adjoint process. The relationship between the stochastic maximum principle and dynamic programming has been investigated in [5, 15]. See also [28] for some worked examples.

2 Introduction to Malliavin calculus

In this section, we give some properties of the Malliavin derivatives, which will be useful for the definition of the adjoint process. The detailed proofs can be found in Nualart [27].

Let \(\left( B_{t}\right) \) be a \(d\)-dimensional Brownian motion, defined on a probability space \(\left( \Omega ,\mathcal {F},P\right) \) and let \(\left( \mathcal {F}_{t}\right) \) be its natural filtration. The following theorem gives the Wiener chaos expansion of a square integrable random variable, see [27] page 13.

Theorem 2.1

Any square integrable random variable \(F\in L^{2}\left( \Omega ,\mathcal {F} _{T},P\right) \) can be expanded into a series of multiple stochastic integrals:

$$\begin{aligned} F=\sum \limits _{n=0}^{\infty }I_{n}\left( f_{n}\right) , \end{aligned}$$
(2.1)

for a unique sequence of symmetric deterministic functions \(f_{n}\in L^{2}\left( \lambda ^{n}\right) ,\) where \(\lambda \) is the Lebesgue measure on \(\left[ 0,T\right] \) and

$$\begin{aligned} I_{n}\left( f_{n}\right) =n!\int \limits _{0}^{T}\int \limits _{0}^{t_{n} }\ldots \int \limits _{0}^{t_{2}}f_{n}\left( t_{1},\ldots ,t_{n}\right) dB_{t_{1} }dB_{t_{2}}\ldots dB_{t_{n}}, \end{aligned}$$

(the \(n\)-times iterated integral of \(f_{n}\) with respect to \(B)\) for \(n=1,2,\ldots \) and \(I_{0}\left( f_{0}\right) =f_{0}\) when \(f_{0}\) is a constant.)

Moreover, we have the isometry

$$\begin{aligned} E\left[ F^{2}\right] =\left\| F\right\| _{L^{2}\left( P\right) } ^{2}=\sum \limits _{n=0}^{\infty }n!\left\| f_{n}\right\| _{L^{2}\left( \lambda ^{n}\right) }^{2}. \end{aligned}$$
(2.2)

Definition 2.2

(Malliavin derivative \(D_{t}\)). Let \(F\in L^{2}\left( P\right) \) be \(\mathcal {F}_{T}-\)measurable.

  1. (i)

    We say that \(F\in \mathbb {D}_{1,2}\) if

    $$\begin{aligned} \left\| F\right\| _{\mathbb {D}_{1,2}}^{2}:=\sum \limits _{n=1}^{\infty }nn!\left\| f_{n}\right\| _{L^{2}\left( \lambda ^{n}\right) } ^{2}<\infty . \end{aligned}$$
    (2.3)
  2. (ii)

    For any \(F\in \mathbb {D}_{1,2}\), we define the Malliavin derivative \(D_{t}F\) of \(F\) at time \(t\), as the expansion

    $$\begin{aligned} D_{t}F:=\sum \limits _{n=1}^{\infty }nI_{n-1}\left( f_{n}\left( .,t\right) \right) , \end{aligned}$$
    (2.4)

where \(I_{n-1}\left( f_{n}\left( .,t\right) \right) \) is the \(\left( n-1\right) -\)fold iterated integral of \(f_{n}\left( t_{1},\ldots ,t_{n-1} ,t\right) \) with respect to the first \(n-1\) variables \(t_{1},\ldots ,t_{n-1}\) and \(t_{n}=t\) is left as parameter.

Note that \(\left\| D_{.}F\right\| _{L^{2}\left( P\times \lambda \right) }=\left\| F\right\| _{\mathbb {D}_{1,2}}^{2}<\infty \), thus the derivative \(D_{t}F\) is well-defined as an element of \(L^{2}\left( P\times \lambda \right) .\)

Example

Let \(F=\int \limits _{0}^{T}f\left( s\right) dB_{s}\), where \(f\in L^{2}\left( \left[ 0,T\right] \right) \), then:

  1. 1.

    \(D_{t}F=f\left( t\right) ,\)

  2. 2.

    \(D_{t}\left( F^{n}\right) =nF^{n-1}D_{t}F=nF^{n-1}f\left( t\right) .\)

Now, we shall give a few rules that will be needed in this paper

  • Integration by parts and duality formula

Suppose that \(\left( u_{t}\right) \) is \(\mathcal {F}_{t}-\)adapted with \(E\left( \int \limits _{0}^{T}u_{t}^{2}dt\right) <+\infty \) and let \(F\in \mathbb {D}_{1,2}.\) Then

$$\begin{aligned} E\left[ F\int \limits _{0}^{T}u_{t}dB_{t}\right] =E\left[ \int \limits _{0} ^{T}u_{t}D_{t}Fdt\right] . \end{aligned}$$
(2.5)

  • Clark-Ocone representation formula (see [27], Proposition 1.3.14 page 46).

Let \(F\in \mathbb {D}_{1,2},\) then

$$\begin{aligned} F=E\left( F\right) +\int \limits _{0}^{1}E\left( D_{t}F/\mathcal {F}_{t}\right) dB_{t}. \end{aligned}$$
(2.6)

  • A generalized Clark-Ocone formula (see [27], Theorem 6.3.1, page 337).

Suppose that

$$\begin{aligned} \tilde{B}_{t}=B_{t}+\int \limits _{0}^{t}\theta _{s}ds, \end{aligned}$$

where \(\theta =\left\{ \theta _{t},t\in \left[ 0,T\right] \right\} \) is an adapted and measurable process such that

$$\begin{aligned} \int \limits _{0}^{T}\theta _{t}^{2}dt<+\infty ,\text { }P-\text {a.s.} \end{aligned}$$

Suppose that \(E\left( Z_{T}\right) =1\), where the process \(Z_{t}\) is given by

$$\begin{aligned} Z_{t}=\exp \left( -\int \limits _{0}^{t}\theta _{s}dB_{s}-\frac{1}{2} \int \limits _{0}^{t}\theta _{s}^{2}ds\right) . \end{aligned}$$

By the Girsanov theorem \(\tilde{B}=\left\{ \tilde{B}_{t},t\in \left[ 0,T\right] \right\} \) is a Brownian motion under the probability \(Q\) on \(\mathcal {F}_{T},\) with density\(\dfrac{dQ}{dP}=Z_{T}.\) Let \(F\) be an \(\mathcal {F}_{T}\)-measurable random variable such that \(F\in \mathbb {D}_{1,2}\) and let \(\theta \in L^{1,2}.\) Assume that

  1. (i)
    $$\begin{aligned} E\left( Z_{T}^{2}F^{2}\right) +E\left( Z_{T}^{2}\int \limits _{0}^{T}\left( D_{t}F\right) ^{2}dt\right) <\infty , \end{aligned}$$
  2. (ii)
    $$\begin{aligned} E\left( Z_{T}^{2}F^{2}\int \limits _{0}^{T}\left( \theta _{t}+\int \limits _{t}^{T}D_{t}\theta _{s}dB_{s}+\int \limits _{t}^{T}\theta _{s}D_{t} \theta _{s}ds\right) ^{2}dt\right) <\infty . \end{aligned}$$

Then

$$\begin{aligned} F=E_{Q}\left( F\right) +\int \limits _{0}^{T}E_{Q}\left( D_{t}F-F\int \limits _{t}^{T}D_{t}\theta _{s}d\tilde{B}_{s}/\mathcal {F}_{t}\right) d\tilde{B}_{s}. \end{aligned}$$
(2.7)

See also [23] for applications to finance.

3 Formulation of the problem

Suppose the state process \(x_{t}=x_{t}^{\left( u,\xi \right) }\); \(t\ge 0\), satisfies the following stochastic differential equation:

$$\begin{aligned} \left\{ \begin{array}{l} dx_{t}=b\left( t,x_{t},u_{t}\right) dt+\sigma \left( t,x_{t},u_{t}\right) dB_{t}+\lambda _{t}d\xi _{t};\\ x_{0}=x\in \mathbb {R} . \end{array} \right. \end{aligned}$$
(3.1)

Here \(\left( B_{t}\right) \) is \(1\)-dimensional Brownian motion, defined on a filtered probability space \(\left( \Omega ,\mathcal {F},\left( \mathcal {F} _{t}\right) _{t\ge 0},P\right) ,\) satisfying the usual conditions. Assume that \(\left( \mathcal {F}_{t}\right) \) is the natural filtration of \(\left( B_{t}\right) \). The coefficients

$$\begin{aligned}&b:\left[ 0,T\right] \times \mathbb {R}\times U\times \Omega \rightarrow \mathbb {R},\\&\sigma :\left[ 0,T\right] \times \mathbb {R}\times U\times \Omega \rightarrow \mathbb {R},\\&\lambda :\left[ 0,T\right] \times \Omega \rightarrow \mathbb {R}, \end{aligned}$$

are given \(\mathcal {F}_{t}-\)predictable processes.

Suppose in addition that we are given a subfiltration \(\mathcal {E}_{t} \subset \mathcal {F}_{t},\) \(t\in \left[ 0,T\right] \), representing the information available to the controller at time t and satisfying the usual conditions.

  • Let \(T\) be a strictly positive real number and consider the following sets.

  • \(\mathcal {U}_{1}^{\mathcal {E}}\) is the class of measurable, \(\mathcal {E}_{t}\)-adapted processes \(u:\left[ 0,T\right] \times \Omega \rightarrow U,\) where \(U\) is some Borel subset of \(\mathbb {R}^{k}.\)

  • \(\mathcal {U}_{2}^{\mathcal {E}}\) is the class of measurable, \(\mathcal {E}_{t}\)-adapted processes \(\xi :\left[ 0,T\right] \times \Omega \rightarrow \) \([0,\infty )\) such that \(\xi \) is nondecreasing, right-continuous with left hand limits and \(\xi _{0}=0.\)

Definition 3.1

An admissible control is an \(\mathcal {E}_{t}\)-adapted process \(\left( u,\xi \right) \in \mathcal {U}_{1}^{\mathcal {E}} \times \mathcal {U} _{2}^{\mathcal {E}}\) such that

$$\begin{aligned} E\left[ {\displaystyle \int \nolimits _{0}^{T}} \left| u_{t}\right| ^{2}dt+\left| \xi _{T}\right| ^{2}\right] <\infty . \end{aligned}$$

We denote by \(\mathcal {A}_{\mathcal {E}}\) the set of all admissible controls.

The expected reward to be maximized has the form

$$\begin{aligned} J\left( u,\xi \right) =E\left[ g\left( x_{T}\right) + {\displaystyle \int \limits _{0}^{T}} f\left( t,x_{t},u_{t}\right) dt+ {\displaystyle \int \limits _{0}^{T}} h\left( t\right) d\xi _{t}\right] , \left( u,\xi \right) \in \mathcal {A}_{\mathcal {E}}, \end{aligned}$$
(3.2)

where

$$\begin{aligned}&f:[0,T]\times \mathbb {R} \times U\times \Omega \rightarrow \mathbb {R},\\&g: \mathbb {R} \times \Omega \rightarrow \mathbb {R},\\&h:[0,T]\times \Omega \rightarrow \mathbb {R} , \end{aligned}$$

are given \(\mathcal {F}_{t}\)-adapted processes.

The goal of the controller is to maximize the functional \(J\left( u,\xi \right) \) over \(\mathcal {A}_{\mathcal {E}}\). An admissible control \(\left( \hat{u},\hat{\xi }\right) \in \mathcal {A}_{\mathcal {E}}\) is optimal if:

$$\begin{aligned} J\left( \hat{u},\hat{\xi }\right) =\sup \limits _{\left( u,\xi \right) \in \mathcal {A}_{\mathcal {E}}}J\left( u,\xi \right) . \end{aligned}$$
(3.3)

Our objective is to derive necessary conditions satisfied by \(\left( \hat{u},\hat{\xi }\right) \).

Note that since we allow \(b\), \(\sigma \), \(h\), \(f\) and \(g\) to be random coefficients and also because our controls must be \(\mathcal {E}_{t}\)-adapted, this problem is no longer of Markovian type and hence cannot be solved by dynamic programming. Our attention will be focused on the stochastic maximum principle, for which an explicit form for the adjoint process is obtained. Malliavin calculus techniques will be used to get an explicit form of the adjoint process.

Assumptions The following assumptions will be in force throughout this paper.

\((\) H \(_{\mathbf {1}})\) \(b,\) \(\sigma \), \(g,\) \(f\) are adapted processes such that there exists a positive constant \(C\) satisfying:

$$\begin{aligned} \left| b(t,x,u)\right| +\left| \sigma (t,x,u)\right| +\left| f(t,x,u)\right| +\left| g(x)\right| \le C(1+\left| x\right| +\left| u\right| ). \end{aligned}$$

\((\) H \(_{\mathbf {2}})\) \(b,\) \(\sigma \), \(g,\) \(f\) are continuously differentiable with respect to \(x\in \mathbb {R}\) and \(u\in U\) for each \(t\in \left[ 0,T\right] ,\) and a.s. \(\omega \in \Omega ,\) with bounded derivatives.

\((\) H \(_{\mathbf {3}})\) \(\lambda \), \(h\) are bounded continuous processes.

\((\) H \(_{\mathbf {4}})\)For all bounded \(\mathcal {F}_{t}-\)measurable random variables \(\alpha =\alpha \left( \omega \right) \) the process \(v_{s}^{\alpha }=\alpha \left( \omega \right) 1_{\left( t,r\right] }\left( s\right) ;\) \(s\in \left[ 0,T\right] \) belongs to \(\mathcal {U}_{1} ^{\varepsilon }.\)

\((\) H \(_{\mathbf {5}})\)For \(u\), \(v\) \(\in \) \(\mathcal {U}_{1}^{\mathcal {E} }\) with \(v\) bounded, there exists \(\delta >0\) such that

$$\begin{aligned} u^{\theta }=u+\theta v\in \mathcal {U}_{1}^{\mathcal {E}} \hbox { for all } \theta \in \left[ -\delta ,\delta \right] . \end{aligned}$$

Under the above assumptions, for every \(\left( u,\xi \right) \in \mathcal {A}_{\mathcal {E}}\), Eq. (3.1) admits a unique strong solution given by

$$\begin{aligned} x_{t}^{\left( u,\xi \right) }=x+\int \limits _{0}^{t}b\left( s,x_{s}^{\left( u,\xi \right) },u_{s}\right) ds+\int \limits _{0}^{t}\sigma \left( s,x_{s}^{\left( u,\xi \right) },u_{s}\right) dB_{s}+\int \limits _{0} ^{t}\lambda \left( t\right) d\xi _{s}, \end{aligned}$$
(3.4)

and the reward functional \(J\) is well defined from \(\mathcal {A}_{\mathcal {E}}\) into \( \mathbb {R} \).

We list some notations which will be used throughout this paper.

Notations For \(\xi \in U_{2}^{\mathcal {E}},\) let denotes the set of \({\mathcal {E}}_{t}\)-adapted processes \(\eta \) of finite variation such that there exists \(\delta >0\) such that \(\xi \) \(+\,\theta \eta \) \(\in U_{2}^{\mathcal {E}},\) for all \(\theta \in [0,\delta ].\) For all \(u\in {\mathcal {U}}_{1}^{\mathcal {E}}\) and \(0\le t\le s\le T,\) we denote the following processes

$$\begin{aligned} R\left( t\right):= & {} g^{\prime }\left( x_{T}\right) + {\displaystyle \int \limits _{t}^{T}} \frac{\partial f}{\partial x}\left( s,x_{s},u_{s}\right) ds,\end{aligned}$$
(3.5)
$$\begin{aligned} D_{t}\left( R\left( t\right) \right):= & {} D_{t}g^{\prime }\left( x_{T}\right) + {\displaystyle \int \limits _{t}^{T}} D_{t}\frac{\partial f}{\partial x}\left( s,x_{s},u_{s}\right) ds,\end{aligned}$$
(3.6)
$$\begin{aligned} H_{0}\left( s,x,u\right)= & {} R\left( s\right) b\left( s,x,u\right) +D_{s}R\left( s\right) \sigma \left( s,x,u\right) , \end{aligned}$$
(3.7)
$$\begin{aligned} G\left( t,s\right):= & {} \exp \left( {\displaystyle \int \limits _{t}^{s}} \left\{ \frac{\partial b}{\partial x}\left( r,x_{r},u_{r}\right) -\frac{1}{2}\left( \frac{\partial \sigma }{\partial x}\right) ^{2}\left( r,x_{r},u_{r}\right) \right\} dr\right. \nonumber \\&\left. + {\displaystyle \int \limits _{t}^{s}} \frac{\partial \sigma }{\partial x}\left( r,x_{r},u_{r}\right) dB_{r}\right) ,\end{aligned}$$
(3.8)
$$\begin{aligned} p\left( t\right):= & {} R\left( t\right) + {\displaystyle \int \limits _{t}^{T}} \frac{\partial H_{0}}{\partial x}\left( s,x_{s},u_{s}\right) G\left( t,s\right) ds, \end{aligned}$$
(3.9)
$$\begin{aligned} q\left( t\right):= & {} D_{t}p\left( t\right) . \end{aligned}$$
(3.10)

We define the usual Hamiltonian of the control problem (3.1)–(3.2) by:

$$\begin{aligned} H:\left[ 0,T\right] \times \mathbb {R\times }U\times \times \mathbb {R} \times \mathbb {R}\times \Omega \rightarrow \mathbb {R}, \end{aligned}$$

where

$$\begin{aligned} H\left( t,x,u,p,q,\omega \right) =f\left( t,x,u,\omega \right) +p\left( t\right) b\left( t,x,u,\omega \right) +q\left( t\right) \sigma \left( t,x,u,\omega \right) ,\qquad \end{aligned}$$
(3.11)

4 The stochastic maximum principle

The purpose of the stochastic maximum principle is to find necessary conditions for optimality satisfied by an optimal control. Suppose that \(\left( \hat{u},\hat{\xi }\right) \in \mathcal {A}_{\mathcal {E}}\) is an optimal control and let \(\hat{x}_{t}\) denotes the optimal trajectory, that is, the solution of (3.1) corresponding to \(\left( \hat{u},\hat{\xi }\right) .\) As it is well known the stochastic maximum principle is based on the computation of the derivative of the reward functional with respect to some perturbation parameter. Let us define the perturbed controls as follows.

  • \(u^{\theta }=\hat{u}+\theta v,\) where \(v\) is some bounded \(\mathcal {E} _{t}-\)adapted process. We know by (H \(_{\mathbf {5}}\)) that there exists \(\delta >0\) such that \(u^{\theta }=\hat{u}+\theta v\in \mathcal {U} _{1}^{\mathcal {E}}\) for all \(\theta \in \left[ -\delta ,\delta \right] \)

  • \(\xi ^{\theta }=\) \(\hat{\xi }\) \(+\,\theta \eta ,\) where the set of \(\mathcal {E}_{t}-\)adapted processes of finite variation, for which there exists \(\delta =\delta (\hat{\xi })>0\) such that \(\hat{\xi }\) \(+\,\theta \eta \) \(\in \mathcal {U}_{2}^{\mathcal {E}}.\)

Since \(\left( \hat{u},\hat{\xi }\right) \) is an optimal control it holds that:

  1. (1)

    \( \begin{array}{l} \lim \limits _{\theta \rightarrow 0^{+}}\frac{1}{\theta }\left( J\left( \hat{u},\xi ^{\theta }\right) -J\left( \hat{u},\hat{\xi }\right) \right) \le 0 \end{array} \), where \(\xi ^{\theta }=\) \(\hat{\xi }\) \(+\,\theta \eta \), and

  2. (2)

    \( \begin{array}{l} \lim \limits _{\theta \rightarrow 0}\frac{1}{\theta }\left( J\left( u^{\theta },\hat{\xi }\right) -J\left( \hat{u},\hat{\xi }\right) \right) \le 0 \end{array},\) where \(u^{\theta }=\hat{u}+\theta v\).

We use the two limits to obtain the variational inequalities. To achieve this goal, we need the following technical Lemmas.

We define the derivative process \(\mathcal {Y}\left( t\right) \) by

$$\begin{aligned} \mathcal {Y}\left( t\right) =\lim \limits _{\theta \rightarrow 0^{+}}\frac{1}{\theta }\left( x_{t}^{\left( \hat{u},\xi ^{\theta }\right) } -x_{t}^{\left( \hat{u},\hat{\xi }\right) }\right) , \end{aligned}$$
(4.1)

Since that \(\mathcal {Y}\left( 0\right) =0\), then

$$\begin{aligned} d\mathcal {Y}\left( t\right) =\frac{\partial b}{\partial x}\left( t\right) \mathcal {Y}\left( t\right) dt+\frac{\partial \sigma }{\partial x}\left( t\right) \mathcal {Y}\left( t\right) dB_{t}+\lambda \left( t\right) d\eta _{t}, \end{aligned}$$
(4.2)

where we use the abbreviated notation:

\(\dfrac{\partial b}{\partial x}\left( t\right) =\dfrac{\partial b}{\partial x}\left( t,\hat{x}_{t},\hat{u}_{t},\omega \right) ,\) \(\dfrac{\partial \sigma }{\partial x}\left( t\right) =\dfrac{\partial \sigma }{\partial x}\left( t,\hat{x}_{t},\hat{u}_{t},\omega \right) \).

Lemma 4.1

The solution of Eq. (4.2) is given by

$$\begin{aligned} \mathcal {Y}\left( t\right) =Z\left( t\right) \left[ {\displaystyle \int \limits _{0}^{t}} Z^{-1}\left( s\right) \lambda \left( s\right) d\eta _{s}\right] ;\text { }t\in \left[ 0,T\right] , \end{aligned}$$
(4.3)

where \(Z\left( t\right) \) is the solution of the homogeneous version of (4.2), i.e.

$$\begin{aligned} \left\{ \begin{array}{l} dZ\left( t\right) =\dfrac{\partial b}{\partial x}\left( t\right) Z\left( t\right) dt+\dfrac{\partial \sigma }{\partial x}\left( t\right) Z\left( t\right) dB_{t},\\ Z\left( 0\right) =1. \end{array} \right. \end{aligned}$$
(4.4)

Proof

We set \(\mathcal {Y}\left( t\right) =Z\left( t\right) A_{t}\) where

$$\begin{aligned} A_{t}= {\displaystyle \int \limits _{0}^{t}} Z^{-1}\left( s\right) \lambda \left( s\right) d\eta _{s}. \end{aligned}$$

By using Itô’s formula for semimartingales, we get

$$\begin{aligned} d\mathcal {Y}\left( t\right)&=Z\left( t\right) dA_{t}+A_{t}dZ\left( t\right) +d\left\langle A,Z\right\rangle _{t},\\ d\mathcal {Y}\left( t\right)&=\lambda \left( t\right) d\eta _{t} +A_{t}\left( \dfrac{\partial b}{\partial x}\left( t\right) Z\left( t\right) dt+\dfrac{\partial \sigma }{\partial x}\left( t\right) Z\left( t\right) dB_{t}\right) \\&=\dfrac{\partial b}{\partial x}\left( t\right) \mathcal {Y}\left( t\right) dt+\dfrac{\partial \sigma }{\partial x}\left( t\right) \mathcal {Y}\left( t\right) dB_{t}+\lambda \left( t\right) d\eta _{t}. \end{aligned}$$

This completes the proof. \(\square \)

In the sequel, we use the abbreviated notation:

$$\begin{aligned} Q\left( t,s\right) =\frac{Z\left( s\right) }{Z\left( t\right) }\text { for }t<s. \end{aligned}$$

Lemma 4.2

Let \(\left( \hat{u},\hat{\xi }\right) \) be an optimal control. Then

$$\begin{aligned} \lim \limits _{\theta \rightarrow 0^{+}}\frac{1}{\theta }\left( J\left( \hat{u},\xi ^{\theta }\right) -J\left( \hat{u},\hat{\xi }\right) \right) =E\left[ {\displaystyle \int \limits _{0}^{T}} \left( \lambda \left( t\right) \hat{P}\left( t\right) +h\left( t\right) \right) d\eta _{t}\right] , \end{aligned}$$
(4.5)

where

$$\begin{aligned}&\hat{P}\left( t\right) :=\hat{R}\left( t\right) + {\displaystyle \int \limits _{t}^{T}} \frac{\partial H_{0}}{\partial x}\left( s\right) Q\left( t,s\right) ds,\end{aligned}$$
(4.6)
$$\begin{aligned}&\hat{R}\left( t\right) =R^{\left( \hat{u},\hat{\xi }\right) }\left( t\right) =g^{\prime }\left( \hat{x}_{T}\right) + {\displaystyle \int \limits _{t}^{T}} \frac{\partial f}{\partial x}\left( s\right) ds,\end{aligned}$$
(4.7)
$$\begin{aligned}&H_{0}(s,x)=R(s)+D_{s}R(s)\sigma (s,x). \end{aligned}$$
(4.8)

Proof

We have

$$\begin{aligned} \lim \limits _{\theta \rightarrow 0^{+}}\frac{1}{\theta }\left( J\left( \hat{u},\xi ^{\theta }\right) -J\left( \hat{u},\hat{\xi }\right) \right)&=E\left[ g^{\prime }\left( \hat{x}_{T}\right) \mathcal {Y}\left( T\right) + {\displaystyle \int \limits _{0}^{T}} \frac{\partial f}{\partial x}\left( t\right) \mathcal {Y}\left( t\right) dt\right. \nonumber \\&\quad \left. + {\displaystyle \int \limits _{0}^{T}} h\left( t\right) d\eta _{t}\right] . \end{aligned}$$
(4.9)

We have from (4.2)

$$\begin{aligned} E\left[ {\displaystyle \int \limits _{0}^{T}} \frac{\partial f}{\partial x}\left( t\right) \mathcal {Y}\left( t\right) dt\right]&=E\left[ {\displaystyle \int \limits _{0}^{T}} \frac{\partial f}{\partial x}\left( t\right) {\displaystyle \int \limits _{0}^{t}} \left\{ \mathcal {Y}\left( s\right) \frac{\partial b}{\partial x}\left( s\right) ds+\mathcal {Y}\left( s\right) \frac{\partial \sigma }{\partial x}\left( s\right) dB_{s}\right. \right. \\&\quad \left. \left. +\,\lambda \left( s\right) d\eta _{s}\right\} dt\right] . \end{aligned}$$

Since \(\mathcal {Y}\left( 0\right) =0\), we have by the duality formulae for the Malliavin derivatives,

$$\begin{aligned} E\left[ {\displaystyle \int \limits _{0}^{T}} \frac{\partial f}{\partial x}\left( t\right) \mathcal {Y}\left( t\right) dt\right]&=\left[ E {\displaystyle \int \limits _{0}^{T}} {\displaystyle \int \limits _{0}^{t}} \left\{ \frac{\partial f}{\partial x}\left( t\right) \mathcal {Y}\left( s\right) \frac{\partial b}{\partial x}\left( s\right) ds+D_{s}\left( \frac{\partial f}{\partial x}\left( t\right) \right) \mathcal {Y}\left( s\right) \frac{\partial \sigma }{\partial x}\left( s\right) ds\right. \right. \\&\quad \left. \left. +\,\frac{\partial f}{\partial x}\left( t\right) \lambda \left( s\right) d\eta _{s}\right\} dt\right] , \end{aligned}$$

by using Fubini theorem

$$\begin{aligned} E\left[ {\displaystyle \int \limits _{0}^{T}} \frac{\partial f}{\partial x}\left( t\right) \mathcal {Y}\left( t\right) dt\right]&=E\left[ {\displaystyle \int \limits _{0}^{T}} {\displaystyle \int \limits _{s}^{T}} \left\{ \frac{\partial f}{\partial x}\left( t\right) \mathcal {Y}\left( s\right) \frac{\partial b}{\partial x}\left( s\right) dt+D_{s}\left( \frac{\partial f}{\partial x}\left( t\right) \right) \mathcal {Y}\left( s\right) \frac{\partial \sigma }{\partial x}\left( s\right) dt\right\} ds\right. \nonumber \\&\quad \left. + {\displaystyle \int \limits _{0}^{T}} \left( {\displaystyle \int \limits _{s}^{T}} \frac{\partial f}{\partial x}\left( t\right) \lambda \left( s\right) dt\right) d\eta _{s}\right] , \end{aligned}$$
(4.10)

changing the notation \(s\rightarrow t\) , this becomes

$$\begin{aligned} E\left[ {\displaystyle \int \limits _{0}^{T}} \frac{\partial f}{\partial x}\left( t\right) \mathcal {Y}\left( t\right) dt\right]&=E\left[ {\displaystyle \int \limits _{0}^{T}} {\displaystyle \int \limits _{t}^{T}} \left\{ \frac{\partial f}{\partial x}\left( s\right) \mathcal {Y}\left( t\right) \frac{\partial b}{\partial x}\left( t\right) ds+D_{t}\left( \frac{\partial f}{\partial x}\left( s\right) \right) \mathcal {Y}\left( t\right) \frac{\partial \sigma }{\partial x}\left( t\right) ds\right\} dt\right. \nonumber \\&\quad \left. + {\displaystyle \int \limits _{0}^{T}} \left( {\displaystyle \int \limits _{t}^{T}} \frac{\partial f}{\partial x}\left( s\right) \lambda \left( t\right) ds\right) d\eta _{t}\right] \nonumber \\&=E\left[ {\displaystyle \int \limits _{0}^{T}} \left\{ \left( {\displaystyle \int \limits _{t}^{T}} \frac{\partial f}{\partial x}\left( s\right) ds\right) \mathcal {Y}\left( t\right) \frac{\partial b}{\partial x}\left( t\right) \right. \right. \nonumber \\&\quad \left. \left. +\,D_{t}\left( {\displaystyle \int \limits _{t}^{T}} \left( \frac{\partial f}{\partial x}\left( s\right) \right) ds\right) \mathcal {Y}\left( t\right) \frac{\partial \sigma }{\partial x}\left( t\right) \right\} dt\right. \nonumber \\&\quad \left. + {\displaystyle \int \limits _{0}^{T}} \left( {\displaystyle \int \limits _{t}^{T}} \frac{\partial f}{\partial x}\left( s\right) ds\right) \lambda \left( t\right) d\eta _{t}\right] . \end{aligned}$$
(4.11)

Similarly we get

$$\begin{aligned} E\left[ g^{\prime }\left( X_{T}\right) \mathcal {Y}\left( T\right) \right]&=E\left[ g^{\prime }\left( X_{T}\right) \left\{ {\displaystyle \int \limits _{0}^{T}} \mathcal {Y}\left( t\right) \frac{\partial b}{\partial x}\left( t\right) dt+\mathcal {Y}\left( t\right) \frac{\partial \sigma }{\partial x}\left( t\right) dB_{t}+\lambda \left( t\right) d\eta _{t}\right\} \right] \nonumber \\&=E\left[ {\displaystyle \int \limits _{0}^{T}} \mathcal {Y}\left( t\right) \left\{ g^{\prime }\left( X_{T}\right) \frac{\partial b}{\partial x}\left( t\right) +D_{t}\left( g^{\prime }\left( X_{T}\right) \right) \frac{\partial \sigma }{\partial x}\left( t\right) \right\} dt+g^{\prime }\left( X_{T}\right) \lambda \left( t\right) d\eta _{t}\right] . \end{aligned}$$
(4.12)

Combining (4.10) and (4.11) and using the notations (3.5) and (3.7), we obtain

$$\begin{aligned} \lim \limits _{\theta \rightarrow 0^{+}}\frac{1}{\theta }\left( J\left( \hat{u},\xi ^{\theta }\right) -J\left( \hat{u},\hat{\xi }\right) \right)&=E\left[ {\displaystyle \int \limits _{0}^{T}} \mathcal {Y}\left( t\right) \left\{ R\left( t\right) \frac{\partial b}{\partial x}\left( t\right) +D_{t}R\left( t\right) \frac{\partial \sigma }{\partial x}\left( t\right) \right\} dt\right. \\&\quad \left. +\left\{ R\left( t\right) \lambda \left( t\right) +h\left( t\right) \right\} d\eta _{t}\right] \\&=A_{1}\left( \eta \right) +A_{2}\left( \eta \right) , \end{aligned}$$

where

$$\begin{aligned} A_{1}\left( \eta \right) =E\left[ {\displaystyle \int \limits _{0}^{T}} \mathcal {Y}\left( t\right) \left\{ R\left( t\right) \frac{\partial b}{\partial x}\left( t\right) +D_{t}R\left( t\right) \frac{\partial \sigma }{\partial x}\left( t\right) \right\} dt\right] , \end{aligned}$$

and

$$\begin{aligned} A_{2}\left( \eta \right) =\left\{ R\left( t\right) \lambda \left( t\right) +h\left( t\right) \right\} d\eta _{t}. \end{aligned}$$

We set

$$\begin{aligned} d\Lambda _{t}=\frac{\partial H_{0}}{\partial x}\left( t\right) dt, \end{aligned}$$

then by using Lemma 4.1 it follows that

$$\begin{aligned} A_{1}\left( \eta \right)&=E\left[ {\displaystyle \int \limits _{0}^{T}} \mathcal {Y}\left( t\right) \frac{\partial H_{0}}{\partial x}\left( t\right) dt\right] \\&=E\left[ {\displaystyle \int \limits _{0}^{T}} \mathcal {Y}\left( t\right) d\Lambda _{t}\right] \\&=E\left[ {\displaystyle \int \limits _{0}^{T}} \left( Z\left( t\right) {\displaystyle \int \limits _{0}^{t}} Z^{-1}\left( s\right) \lambda \left( s\right) d\eta _{s}\right) d\Lambda _{t}\right] . \end{aligned}$$

Hence by using Fubini’s theorem we get by changing the notation \(s\rightarrow t\)

$$\begin{aligned} A_{1}\left( \eta \right)&=E\left[ {\displaystyle \int \limits _{0}^{T}} \left( \left( {\displaystyle \int \limits _{t}^{T}} Z\left( s\right) d\Lambda _{s}\right) Z^{-1}\left( t\right) \lambda \left( t\right) d\eta _{t}\right) \right] \\&=E\left[ {\displaystyle \int \limits _{0}^{T}} {\displaystyle \int \limits _{t}^{T}} Q\left( t,s\right) \frac{\partial H_{0}}{\partial x}\left( s\right) ds\lambda \left( t\right) d\eta _{t}\right] . \end{aligned}$$

Finaly

$$\begin{aligned} \lim \limits _{\theta \rightarrow 0^{+}}\frac{1}{\theta }\left( J\left( \hat{u},\xi ^{\theta }\right) -J\left( \hat{u},\hat{\xi }\right) \right)&=A_{1}\left( \eta \right) +A_{2}\left( \eta \right) \\&=E\left[ {\displaystyle \int \limits _{0}^{T}} \left( \lambda \left( t\right) \hat{P}\left( t\right) +h\left( t\right) \right) d\eta _{t}\right] . \end{aligned}$$

This completes the proof. \(\square \)

We define the derivative process \(Y\left( t\right) \) by

$$\begin{aligned} Y\left( t\right) =Y^{v}\left( t\right) =\lim \limits _{\theta \rightarrow 0}\frac{1}{\theta }\left( x_{t}^{\left( u^{\theta },\hat{\xi }\right) } -x_{t}^{\left( \hat{u},\hat{\xi }\right) }\right) , \end{aligned}$$
(4.13)

then \(Y\left( t\right) \) satisfies the following equation

$$\begin{aligned} dY\left( t\right)&=Y\left( t\right) \left[ \frac{\partial b}{\partial x}\left( t\right) dt+\frac{\partial \sigma }{\partial x}\left( t\right) dB_{t}\right] \\&\quad +\,v_{t}\left[ \frac{\partial b}{\partial u}\left( t\right) dt+\frac{\partial \sigma }{\partial u}\left( t\right) dB_{t}\right] ,\nonumber \\ Y\left( 0\right)&=0,\nonumber \end{aligned}$$
(4.14)

Lemma 4.3

The following identity holds

$$\begin{aligned} \lim \limits _{\theta \rightarrow 0}\frac{1}{\theta }\left( J\left( u^{\theta },\hat{\xi }\right) -J\left( \hat{u},\hat{\xi }\right) \right)&=\,E\left[ {\displaystyle \int \limits _{0}^{T}} \left\{ R\left( t\right) \left\{ \frac{\partial b}{\partial x}\left( t\right) Y\left( t\right) +\frac{\partial b}{\partial u}\left( t\right) v_{t}\right\} \right. \right. \\&\quad \left. \left. +\,D_{t}R\left( t\right) \left\{ \frac{\partial \sigma }{\partial x}\left( t\right) Y(t)+\frac{\partial \sigma }{\partial u}\left( t\right) v_{t}\right\} +\frac{\partial f}{\partial u}\left( s\right) v_{t}\right\} \right] dt. \end{aligned}$$

Proof

We have

$$\begin{aligned} \frac{d}{d\theta }J\left( u^{\theta },\hat{\xi }\right) \mid _{\theta =0}&=\,E\left[ {\displaystyle \int \limits _{0}^{T}} \left\{ \frac{\partial f}{\partial x}\left( t\right) Y(t)+\frac{\partial f}{\partial u}\left( t\right) v_{t}\right\} dt\right. \nonumber \\&\quad \left. +\,g^{\prime }\left( \hat{x}_{T}\right) Y(T)\right] , \end{aligned}$$
(4.15)

where \(Y\left( t\right) =Y^{v}\left( t\right) \) is the solution of the linear equation

$$\begin{aligned} \left\{ \begin{array}{l} dY\left( t\right) =\left[ \frac{\partial b}{\partial x}\left( t\right) Y\left( t\right) +\frac{\partial b}{\partial u}\left( t\right) v_{t}\right] dt+\left[ \frac{\partial \sigma }{\partial x}\left( t\right) Y\left( t\right) +\frac{\partial \sigma }{\partial u}\left( t\right) v_{t}\right] dB_{t}\\ Y\left( 0\right) =0 \end{array} \right. \end{aligned}$$
(4.16)

By the duality formula we get

$$\begin{aligned} E\left( g^{\prime }\left( \hat{x}_{T}\right) Y(T)\right)&=E\left[ g^{\prime }\left( \hat{x}_{T}\right) {\displaystyle \int \limits _{0}^{T}} \left\{ \frac{\partial b}{\partial x}\left( t\right) Y(t)+\frac{\partial b}{\partial u}\left( t\right) v_{t}\right\} dt\right. \\&\quad \left. +\,g^{\prime }\left( \hat{x}_{T}\right) {\displaystyle \int \limits _{0}^{T}} \left\{ \frac{\partial \sigma }{\partial x}\left( t\right) Y(t)+\frac{\partial \sigma }{\partial u}\left( t\right) v_{t}\right\} dB_{t}\right] \\&=E\left[ {\displaystyle \int \limits _{0}^{T}} g^{\prime }\left( \hat{x}_{T}\right) \left\{ \frac{\partial b}{\partial x}\left( t\right) Y(t)+\frac{\partial b}{\partial u}\left( t\right) v_{t}\right\} dt\right. \\&\quad \left. + {\displaystyle \int \limits _{0}^{T}} D_{t}g^{\prime }\left( \hat{x}_{T}\right) \left\{ \frac{\partial \sigma }{\partial x}\left( t\right) Y(t)+\frac{\partial \sigma }{\partial u}\left( t\right) v_{t}\right\} dt\right] . \end{aligned}$$

Using similar arguments and Fubini’s theorem it follows that,

$$\begin{aligned} E\left[ {\displaystyle \int \limits _{0}^{T}} \frac{\partial f}{\partial x}\left( t\right) Y(t)dt\right]&=E\left[ {\displaystyle \int \limits _{0}^{T}} \left( {\displaystyle \int \limits _{0}^{t}} \frac{\partial f}{\partial x}\left( t\right) \left\{ \frac{\partial b}{\partial x}\left( s\right) Y(s)+\frac{\partial b}{\partial u}\left( s\right) v_{s}\right\} ds\right) dt\right. \nonumber \\&\quad +\left. {\displaystyle \int \limits _{0}^{T}} \left( {\displaystyle \int \limits _{0}^{t}} D_{s}\frac{\partial f}{\partial x}\left( t\right) \left\{ \frac{\partial \sigma }{\partial x}\left( t\right) Y(t)+\frac{\partial \sigma }{\partial u}\left( t\right) v_{t}\right\} ds\right) dt\right] \nonumber \\&=E\left[ {\displaystyle \int \limits _{0}^{T}} \left( {\displaystyle \int \limits _{s}^{T}} \frac{\partial f}{\partial x}\left( t\right) \left\{ \frac{\partial b}{\partial x}\left( s\right) Y(s)+\frac{\partial b}{\partial u}\left( s\right) v_{s}\right\} dt\right) ds\right. \nonumber \\&\quad +\left. {\displaystyle \int \limits _{0}^{T}} \left( {\displaystyle \int \limits _{s}^{T}} D_{s}\frac{\partial f}{\partial x}\left( t\right) \left\{ \frac{\partial \sigma }{\partial x}\left( s\right) Y(s)+\frac{\partial \sigma }{\partial u}\left( s\right) v_{s}\right\} dt\right) ds\right] . \end{aligned}$$
(4.17)

Changing the notation \(s\rightarrow t,\) we get

$$\begin{aligned}&E\left[ {\displaystyle \int \limits _{0}^{T}} \frac{\partial f}{\partial x}\left( t\right) Y(t)dt\right] \nonumber \\&=E\left[ {\displaystyle \int \limits _{0}^{T}} \left( \left( {\displaystyle \int \limits _{t}^{T}} \frac{\partial f}{\partial x}\left( s\right) ds\right) \left\{ \frac{\partial b}{\partial x}\left( t\right) Y(s)+\frac{\partial b}{\partial u}\left( t\right) v_{t}\right\} \right) dt\right. \nonumber \\&\qquad +\left. {\displaystyle \int \limits _{0}^{T}} \left( {\displaystyle \int \limits _{t}^{T}} \left( D_{t}\frac{\partial f}{\partial x}\left( s\right) ds\right) \left\{ \frac{\partial \sigma }{\partial x}\left( t\right) Y(t)+\frac{\partial \sigma }{\partial u}\left( t\right) v_{t}\right\} \right) dt\right] . \end{aligned}$$
(4.18)

Using the notation

$$\begin{aligned} R\left( t\right) :=g^{\prime }\left( X_{T}\right) + {\displaystyle \int \limits _{t}^{T}} \frac{\partial f}{\partial x}\left( s\right) ds, \end{aligned}$$

and combining (4.17) and (4.18), we get

$$\begin{aligned} \lim \limits _{\theta \rightarrow 0}\frac{1}{\theta }\left( J\left( u^{\theta },\hat{\xi }\right) -J\left( \hat{u},\hat{\xi }\right) \right)&=E\left[ {\displaystyle \int \limits _{0}^{T}} \left\{ R\left( t\right) \left\{ \frac{\partial b}{\partial x}\left( t\right) Y\left( t\right) +\frac{\partial b}{\partial u}\left( t\right) v_{t}\right\} \right. \right. \nonumber \\&\quad +D_{t}R\left( t\right) \left\{ \frac{\partial \sigma }{\partial x}\left( t\right) Y(t)+\frac{\partial \sigma }{\partial u}\left( t\right) v_{t}\right\} \nonumber \\&\quad \left. \left. +\frac{\partial f}{\partial u}\left( t\right) v_{t}\right\} dt\right] , \end{aligned}$$
(4.19)

which completes the proof. \(\square \)

Now, we are ready to state the main result of this paper. Note that the following theorem extends in particular [25] Theorem 3.4 and [29] Theorem 2.4 to mixed regular-singular control problems.

Theorem 4.4

(The stochastic maximum principle) Let \(\left( \hat{u},\hat{\xi }\right) \in \mathcal {A}_{\mathcal {E}}\) be an optimal control maximizing the reward \(J\) over \(\mathcal {A}_{\mathcal {E}}\) and \(\hat{x}_{t}\) denotes the optimal trajectory, then for a.e. \(t\in \left[ 0,T\right] \) we have:

  1. i)

    \(E\left[ V_{\left( \hat{u},\hat{\xi }\right) }(t)/\mathcal {E}_{t}\right] \le 0,\) and \(E\left[ V_{\left( \hat{u},\hat{\xi }\right) }(t)/\mathcal {E} _{t}\right] d\hat{\xi }_{t}=0\) where

    $$\begin{aligned} V_{\left( \hat{u},\hat{\xi }\right) }(t)=\lambda \left( t\right) \hat{p}\left( t\right) +h\left( t\right) , \end{aligned}$$
  2. ii)

    \(E\left[ \dfrac{\partial H}{\partial u}\left( t,\hat{x}_{t},\hat{u} _{t}\right) /\mathcal {E}_{t}\right] =0,\) where

    $$\begin{aligned} H\left( t,\hat{x}_{t},\hat{u}_{t},\hat{p}\left( t\right) ,\hat{q}\left( t\right) \right) =f\left( t,\hat{x}_{t},\hat{u}_{t}\right) +\hat{p}\left( t\right) b\left( t,\hat{x}_{t},\hat{u}_{t}\right) +\hat{q}\left( t\right) \sigma \left( t,\hat{x}_{t},\hat{u}_{t}\right) , \end{aligned}$$

is the usual Hamiltonian.

Proof

First, we start to prove \((i)\). By Lemma \(4.2\) we have

$$\begin{aligned} \lim \limits _{\theta \rightarrow 0^{+}}\frac{1}{\theta }\left( J\left( \hat{u},\xi ^{\theta }\right) -J\left( \hat{u},\hat{\xi }\right) \right) =E\left[ {\displaystyle \int \limits _{0}^{T}} V_{\left( \hat{u},\hat{\xi }\right) }(t)d\eta _{t}\right] \le 0, \end{aligned}$$

for all \(\eta \in \mathcal {U}_{2}^{\mathcal {E}}.\) In particular, this holds if we choose \(\eta \) such that \(d\eta \left( t\right) =a\left( t\right) dt,\) where \(a\left( t\right) \ge 0\) is continuous and \(\mathcal {E}_{t}-\)adapted, then

$$\begin{aligned} E\left[ {\displaystyle \int \limits _{0}^{T}} V_{\left( \hat{u},\hat{\xi }\right) }(t)a\left( t\right) dt\right] \le 0. \end{aligned}$$

Since this holds for all such \(\mathcal {E}_{t}-\)adapted processes, we deduce that

$$\begin{aligned} E\left[ V_{\left( \hat{u},\hat{\xi }\right) }(t)/\mathcal {E}_{t}\right] \le 0;\text { }a.e.t\in \left[ 0,T\right] . \end{aligned}$$
(4.20)

Then, choosing \(\eta _{t}=-\hat{\xi }_{t}\) we get

$$\begin{aligned} E\left[ {\displaystyle \int \limits _{0}^{T}} V_{\left( \hat{u},\hat{\xi }\right) }(t)\left( -d\hat{\xi }_{t}\right) \right] \le 0. \end{aligned}$$

Next, choosing \(\eta _{t}=\hat{\xi }_{t}\) we get

$$\begin{aligned} E\left[ {\displaystyle \int \limits _{0}^{T}} V_{\left( \hat{u},\hat{\xi }\right) }d\hat{\xi }_{t}\right] \le 0. \end{aligned}$$

Hence

$$\begin{aligned} E\left[ {\displaystyle \int \limits _{0}^{T}} V_{\left( \hat{u},\hat{\xi }\right) }(t)d\hat{\xi }_{t}\right] =E\left[ {\displaystyle \int \limits _{0}^{T}} E\left( V_{\left( \hat{u},\hat{\xi }\right) }(t)/\mathcal {E}_{t}\right) d\hat{\xi }_{t}\right] =0, \end{aligned}$$

which combined with (4.20) gives

$$\begin{aligned} E\left( V_{\left( \hat{u},\hat{\xi }\right) }(t)/\mathcal {E}_{t}\right) d\hat{\xi }_{t}=0. \end{aligned}$$

Now let us prove \((ii)\).

We have

$$\begin{aligned} \lim \limits _{\theta \rightarrow 0}\frac{1}{\theta }\left( J\left( u^{\theta },\hat{\xi }\right) -J\left( \hat{u},\hat{\xi }\right) \right) \le 0. \end{aligned}$$

Then by Lemma 4.3 we get

$$\begin{aligned} 0&\ge E\left[ {\displaystyle \int \limits _{0}^{T}} \left\{ R\left( t\right) \left\{ \frac{\partial b}{\partial x}\left( t\right) Y\left( t\right) +\frac{\partial b}{\partial u}\left( t\right) v_{t}\right\} \right. \right. \\&\quad \left. \left. +D_{t}R\left( t\right) \left\{ \frac{\partial \sigma }{\partial x}\left( t\right) Y(t)+\frac{\partial \sigma }{\partial u}\left( t\right) v_{t}\right\} +\frac{\partial f}{\partial u}\left( s\right) v_{t}\right\} \right] dt.\nonumber \end{aligned}$$
(4.21)

Now we apply the above to \(v=v_{\alpha }\in \mathcal {U}_{1}^{\mathcal {E}}\) of the form \(v_{\alpha }\left( s\right) =\alpha 1_{\left[ t,t+h\right] }\left( s\right) ,\) for some \(t,h\in \left( 0,T\right) \), \(t+h\le T,\) where \(\alpha =\alpha \left( \omega \right) \) is bounded and \(\mathcal {E}_{t} \)-measurable. Then \(\ Y^{v_{\alpha }}\left( s\right) =0\) for \(\ 0\le s\le t\), hence (4.21) becomes

$$\begin{aligned} A_{1}+A_{2}\le 0, \end{aligned}$$
(4.22)

where

$$\begin{aligned} A_{1}=E\left[ {\displaystyle \int \limits _{t}^{T}} \left\{ R\left( s\right) \frac{\partial b}{\partial x}\left( s\right) Y\left( s\right) +D_{s}R\left( s\right) \frac{\partial \sigma }{\partial x}\left( s\right) Y(s)\right\} ds\right] , \end{aligned}$$

and

$$\begin{aligned} A_{2}=\left[ {\displaystyle \int \limits _{t}^{t+h}} \left\{ R\left( s\right) \frac{\partial b}{\partial u}\left( t\right) +D_{s}R\left( s\right) \frac{\partial \sigma }{\partial u}\left( t\right) +\frac{\partial f}{\partial u}\left( s\right) \right\} \alpha ds\right] . \end{aligned}$$

Note that by (4.14), with \(Y\left( s\right) =Y^{v_{\alpha }}\left( s\right) \), \(s\ge t+h\) the process \(Y\left( s\right) \) satisfies the following dynamics

$$\begin{aligned} dY(s)=Y(s)\left\{ \frac{\partial b}{\partial x}\left( s\right) ds+\frac{\partial \sigma }{\partial x}\left( s\right) dB_{s}\right\} , \end{aligned}$$
(4.23)

for \(s\ge t+h\) with initial condition \(Y\left( t+h\right) \) at time \(t+h.\) An application of Itô’s formula yields

$$\begin{aligned} Y\left( s\right) =Y\left( t+h\right) G\left( t+h,s\right) ;\quad s\ge t+h, \end{aligned}$$
(4.24)

where, for \(s\ge t\),

$$\begin{aligned} G\left( t,s\right) =\exp \left( {\displaystyle \int \limits _{t}^{s}} \left\{ \frac{\partial b}{\partial x}\left( r\right) -\frac{1}{2}\left( \frac{\partial \sigma }{\partial x}\right) ^{2}\left( r\right) \right\} dr+ {\displaystyle \int \limits _{t}^{s}} \frac{\partial \sigma }{\partial x}\left( r\right) dB_{r}\right) . \end{aligned}$$

Note that \(G\left( t,s\right) \) does not depend on \(h,\) but \(Y\left( s\right) \) does. We have by (3.7)

$$\begin{aligned} A_{1}=E\left[ {\displaystyle \int \limits _{t}^{T}} \frac{\partial H_{0}}{\partial x}\left( s\right) Y(s)ds\right] . \end{aligned}$$

Differentiating with respect to \(h\) at \(h=0\) we get

$$\begin{aligned} \left. \frac{d}{dh}A_{1}\right| _{h=0}=\frac{d}{dh}\left. E\left[ {\displaystyle \int \limits _{t}^{t+h}} \frac{\partial H_{0}}{\partial x}\left( s\right) Y(s)ds\right] \right| _{h=0}+\frac{d}{dh}\left. E\left[ {\displaystyle \int \limits _{t+h}^{T}} \frac{\partial H_{0}}{\partial x}\left( s\right) Y(s)ds\right] \right| _{h=0}. \end{aligned}$$

Using the fact that \(Y\left( t\right) =0,\) we see that

$$\begin{aligned} \frac{d}{dh}E\left[ {\displaystyle \int \limits _{t}^{t+h}} \frac{\partial H_{0}}{\partial x}\left( s\right) Y(s)ds\right] _{h=0}=0. \end{aligned}$$

Therefore, using (4.24) and the fact that \(Y\left( t\right) =0\) it holds that,

$$\begin{aligned} \left. \dfrac{d}{dh}A_{1}\right| _{h=0}&=\dfrac{d}{dh}\left. E\left[ {\displaystyle \int \limits _{t+h}^{T}} \dfrac{\partial H_{0}}{\partial x}\left( s\right) Y\left( t+h\right) G\left( t+h,s\right) ds\right] \right| _{h=0}\nonumber \\&= {\displaystyle \int \limits _{t}^{T}} \dfrac{d}{dh}\left. E\left[ \dfrac{\partial H_{0}}{\partial x}\left( s\right) Y\left( t+h\right) G\left( t+h,s\right) \right] \right| _{h=0}ds\nonumber \\&= {\displaystyle \int \limits _{t}^{T}} \dfrac{d}{dh}\left. E\left[ \dfrac{\partial H_{0}}{\partial x}\left( s\right) G\left( t,s\right) Y\left( t+h\right) \right] \right| _{h=0}ds. \end{aligned}$$
(4.25)

By (4.16)

$$\begin{aligned} Y\left( t+h\right) =\alpha {\displaystyle \int \limits _{t}^{t+h}} \left\{ \frac{\partial b}{\partial u}\left( s\right) ds+\frac{\partial \sigma }{\partial u}\left( s\right) dB_{s}\right\} + {\displaystyle \int \limits _{t}^{t+h}} Y_{s}\left\{ \frac{\partial b}{\partial x}\left( s\right) ds+\frac{\partial \sigma }{\partial x}\left( s\right) dB_{s}\right\} .\qquad \quad \end{aligned}$$
(4.26)

Therefore, by the duality formulae\(,\) \(\left. \dfrac{d}{dh}A_{1}\right| _{h=0}=\Lambda _{1}+\Lambda _{2},\) where

$$\begin{aligned} \Lambda _{1}&= {\displaystyle \int \limits _{t}^{T}} \frac{d}{dh}\left. E\left[ \frac{\partial H_{0}}{\partial x}\left( s\right) G\left( t,s\right) \alpha \left( {\displaystyle \int \limits _{t}^{t+h}} \frac{\partial b}{\partial u}\left( r\right) dr+\frac{\partial \sigma }{\partial u}\left( r\right) dB_{r}\right) \right] \right| _{h=0}ds\nonumber \\&= {\displaystyle \int \limits _{t}^{T}} \frac{d}{dh}\left. E\left[ F\left( t,s\right) \alpha \left( {\displaystyle \int \limits _{t}^{t+h}} \frac{\partial b}{\partial u}\left( r\right) dr+\frac{\partial \sigma }{\partial u}\left( r\right) dB_{r}\right) \right] \right| _{h=0}ds\nonumber \\&= {\displaystyle \int \limits _{t}^{T}} \frac{d}{dh}\left. E\left[ \alpha \left( {\displaystyle \int \limits _{t}^{t+h}} \left\{ F\left( t,s\right) \frac{\partial b}{\partial u}\left( r\right) dr+D_{r}F\left( t,s\right) \frac{\partial \sigma }{\partial u}\left( r\right) \right\} dr\right) \right] \right| _{h=0}ds\nonumber \\&= {\displaystyle \int \limits _{t}^{T}} E\left[ \alpha \left\{ F\left( t,s\right) \frac{\partial b}{\partial u}\left( t\right) dt+D_{t}F\left( t,s\right) \frac{\partial \sigma }{\partial u}\left( t\right) \right\} \right] ds, \end{aligned}$$
(4.27)

\(F\left( t,s\right) =\frac{\partial H_{0}}{\partial x}\left( s\right) G\left( t,s\right) ,\) and

$$\begin{aligned} \Lambda _{2}= {\displaystyle \int \limits _{t}^{T}} \frac{d}{dh}E\left[ \frac{\partial H_{0}}{\partial x}\left( s\right) G\left( t,s\right) \left( {\displaystyle \int \limits _{t}^{t+h}} Y_{r}\left\{ \frac{\partial b}{\partial x}\left( r\right) dr+\frac{\partial b}{\partial x}\left( r\right) dB_{r}\right\} \right) \right] ds. \end{aligned}$$

Using the fact that \(Y\left( t\right) =0\), we see that

$$\begin{aligned} \Lambda _{2}=0. \end{aligned}$$

We conclude that

$$\begin{aligned} \frac{d}{dh}\left. A_{1}\right| _{h=0}=\Lambda _{1}. \end{aligned}$$

Moreover, we see directly that

$$\begin{aligned} \frac{d}{dh}\left. A_{2}\right| _{h=0}=E\left[ \alpha \left\{ R\left( t\right) \frac{\partial b}{\partial u}\left( t\right) +D_{t}R\left( t\right) \frac{\partial \sigma }{\partial u}\left( t\right) +\frac{\partial f}{\partial u}\left( t\right) \right\} \right] . \end{aligned}$$

Therefore, differentiating (4.26) with respect to \(h\) at \(h=0,\) gives the inequality

$$\begin{aligned}&E\left[ \alpha \left\{ \left( R\left( t\right) +\int \limits _{t} ^{T}F\left( t,s\right) ds\right) \dfrac{\partial b}{\partial u}\left( t\right) \right. \right. \\&\left. \left. +\,D_{t}\left( R\left( t\right) +\int \limits _{t}^{T}F\left( t,s\right) ds\right) \dfrac{\partial \sigma }{\partial u}\left( t\right) +\dfrac{\partial f}{\partial u}\left( t\right) \right\} \right] \le 0. \end{aligned}$$

We can reformulate this by using the notation (3.9) and (3.10)

$$\begin{aligned} E\left[ \alpha \left\{ p\left( t\right) \frac{\partial b}{\partial u}\left( t\right) +q\left( t\right) \frac{\partial \sigma }{\partial u}\left( t\right) +\frac{\partial f}{\partial u}\left( t\right) \right\} \right] \le 0. \end{aligned}$$

Using the definition of the Hamiltonian (3.11) the last inequality can be rewritten

$$\begin{aligned} E\left[ \frac{\partial H}{\partial u}\left( t,\hat{x}_{t},\hat{u} _{t}\right) \alpha \right] \le 0. \end{aligned}$$

Since this holds for all bounded \(\ \mathcal {E}_{t}\)-measurable random variable \(\alpha \), we conclude that

$$\begin{aligned} E\left[ \frac{\partial H}{\partial u}\left( t,\hat{x}_{t},\hat{u} _{t}\right) /\mathcal {E}_{t}\right] =0. \end{aligned}$$

This completes the proof. \(\square \)