Abstract
This paper studies singular optimal control problems for systems described by nonlinear-controlled stochastic differential equations of mean-field type (MFSDEs in short), in which the coefficients depend on the state of the solution process as well as of its expected value. Moreover, the cost functional is also of mean-field type. The control variable has two components, the first being absolutely continuous and the second singular. We establish necessary as well as sufficient conditions for optimal singular stochastic control where the system evolves according to MFSDEs. These conditions of optimality differs from the classical one in the sense that here the adjoint equation turns out to be a linear mean-field backward stochastic differential equation. The proof of our result is based on convex perturbation method of a given optimal control. The control domain is assumed to be convex. A linear quadratic stochastic optimal control problem of mean-field type is discussed as an illustrated example.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
1 Introduction
In this paper, we study optimal stochastic singular control for systems driven by nonlinear-controlled stochastic differential equations of mean-field type, which is also called McKean–Vlasov equations
for some functions \(f,\sigma \), and \(G.\) Noting that mean-field dynamics (1.1) are obtained as the mean-square limit, when \(n\rightarrow +\infty \) of a system of interacting particles of the form
where \(\left( W^{k}(\cdot ):k\ge 1\right) \) is a collection of independent Brownian motions. The expected cost to be minimized over the class of admissible controls is also of mean-field type, which has the form
where the initial time \(s\) and the initial state \(\zeta \) of the system are fixed. Any admissible control \((u^{*}(\cdot ),\eta ^{*}(\cdot ))\) satisfying
is called an optimal control. The corresponding state process, solution of MFSDE-(1.1), is denoted by \(x^{*}(\cdot )=x^{u^{*},\eta ^{*}}(\cdot )\).
The stochastic maximum principle for singular control was considered by many authors, see for instance [2–4, 9–11, 13, 14, 16]. The first version of maximum principle for singular stochastic control problems was obtained by Cadenillas et al. [9]. The first-order weak stochastic maximum principle has been studied in [4]. In [11], the authors derived stochastic maximum principle where the singular part has a linear form. Sufficient conditions for the existence of optimal singular control have been studied in Dufour et al. [10]. The necessary and sufficient conditions for near-optimal singular control were obtained by Hafayed et al. [13]. For this type of problem, the reader may consult the papers by Haussmann et al. [14] and the list of references therein.
Many authors made contributions on SDEs of mean-field type and applications, see for instance [1, 6–8, 12, 15, 17, 18, 20, 21]. Mean-field stochastic maximum principle of optimality was considered by many authors, see for instance [7, 17, 18, 21]. In Buckdahn et al. [8], the authors obtained mean-field backward stochastic differential equations. In Buckdahn et al. [7], the general maximum principle was introduced for a class of stochastic control problems involving MFSDEs, where the authors obtained a stochastic maximum principle differs from the classical one in the sense that the first-order adjoint equation turns out to be a linear mean-field backward SDE, while the second-order adjoint equation remains the same as in Peng’s [19] stochastic maximum principle. In Mayer-Brandis et al. [18], a stochastic maximum principle of optimality for systems governed by controlled Itô-Levy process of mean-field type is proved by using Malliavin calculus. The local maximum principle of optimality for mean-field stochastic control problem has been derived by Li [17]. The linear quadratic optimal control problem for MFSDEs has been studied by Yong [21]. The maximum principle for mean-field jump diffusion processes has been studied in Hafayed et al. [12].
Our main goal in this paper is to derive a set of necessary as well as sufficient conditions for the optimal singular stochastic control of mean-field problem (1.1)–(1.2). Following the standard idea of deriving necessary and sufficient conditions for optimal control processes, due to the fact that the control domain is assumed to be convex, one needs to use convex perturbations for both continuous and singular parts of our control process. The problem under consideration where the coefficients depend on the marginal probability law of the solution is not only simple extensions from the mathematical point of view, but also provides interesting models in applications. To streamline the presentation, we only study the one-dimensional case.
The rest of the paper is organized as follows. Section 2 begins with a general formulation of a mean-field singular control problem and gives the notations and assumptions used throughout the paper. In Sects. 3 and 4 we establish our necessary and sufficient conditions of optimality, respectively. In the last section an example is given to illustrate the theoretical results.
2 Assumptions and Statement of the Control Problem
We consider mean-field stochastic singular control problem of the following kind. Let \(T\) be a fixed strictly positive real number and \( (\Omega ,\mathcal {F},\left\{ \mathcal {F}_{t}\right\} _{t\in \left[ s,T\right] },\mathbb {P})\) be a fixed filtered probability space satisfying the usual conditions in which one-dimensional Brownian motion \(W(t)=\left\{ W(t):s\le t\le T\right\} \) and \(W(s)=0\) is defined. Let \(\mathbb {A}_{1}\) be a closed convex subset of \(\mathbb {R}\) and \(\mathbb {A}_{2}:=\left[ 0,\infty \right) .\) Let \(\mathcal {U}_{1}\) be the class of measurable, \( \mathcal {F}_{t}-\)adapted processes \(u(\cdot ):\left[ s,T\right] \times \Omega \rightarrow \mathbb {A}_{1}\), and \(\mathcal {U}_{2}\) is the class of measurable, \(\mathcal {F}_{t}-\)adapted processes \(\eta (\cdot ):\left[ s,T \right] \times \Omega \rightarrow \mathbb {A}_{2}\).
Since the objective of this paper is to study optimal singular stochastic control, we give here the precise definition of the singular part of an admissible control.
Definition 2.1
An admissible control is a pair \(\left( u(\cdot ),\eta (\cdot )\right) \) of measurable \(\mathbb {A} _{1}\times \mathbb {A}_{2}\)-valued, \(\mathcal {F}_{t}-\) adapted processes, such that
-
1.
\(\eta (\cdot )\) is of bounded variation, nondecreasing continuous on the left with right limits and \(\eta (s)=0.\)
-
2.
\(\mathbb {E}\left[ \sup _{t\in \left[ s,T\right] }\left| u(t)\right| ^{2}+\left| \eta (T)\right| ^{2}\right] <\infty .\)
We denote \(\mathcal {U}_{1}\times \mathcal {U}_{2},\) the set of all admissible controls. We note that since \(\mathrm{d}\eta (t)\) may be singular with respect to Lebesgue measure \(\mathrm{d}t,\) we call \(\eta (\cdot )\) the singular part of the control and the process \(u(\cdot )\) its absolutely continuous part.
We denote by \(\mathbb {L}_{\mathcal {F}}^{2}\left( \left[ s,T\right] ;\mathbb {R}\right) =\left\{ \Phi (\cdot ):=\Phi (t,w)\text { is an }\mathcal {F }_{t}\text {-adapted }\mathbb {R}\text {-valued}\right. \) \(\left. \text {measurable process on }\left[ s,T\right] \text { such that }\mathbb {E}\left( \int \nolimits _{s}^{T}\left| \Phi (t)\right| ^{2}\mathrm{d}t\right) <\infty \right\} .\) We denote by \(\chi _{\mathcal {R}}\) the indicator function of \(\mathcal {R}\). In what follows, \(C\) represents a generic constants, which can be different from line to line.
Conditions: Throughout this paper we assume the following.
- (H1):
-
The functions \(f,\sigma ,\ell :\left[ s,T\right] \times \mathbb {R}\times \mathbb {R\times \mathbb {A}}_{1}\mathbb {\rightarrow R}\), and \(h:\mathbb {R}\times \mathbb {R\rightarrow R}\) are continuously differentiable with respect to \(\left( x,\tilde{x},u\right) \). Moreover, \(f,\sigma ,h\), and \(\ell \) and all their derivatives with respect to \(\left( x,\tilde{x} ,u\right) \) are continuous and bounded.
- (H2):
-
The function \(G:\left[ s,T\right] \rightarrow \mathbb {R} ,K:\left[ s,T\right] \rightarrow \left[ 0,\infty \right) \), for each \( t\in [s,T]:G\) is continuous and bounded, also \(K\) is continuous.
Under the above assumptions, the MFSDE-(1.1) has a unique strong solution \(x^{u,\eta }(t)\) which is given by
Moreover, by standard arguments it is easy to show that for any \(p>0\), it holds that
where \(C_{p}\) is a constant depending only on \(p\) and the functional \( J^{s,\zeta }\) is well defined.
We define the usual Hamiltonian associated with the mean-field stochastic control problem (1.1)–(1.2) as follows
where \((t,x,\tilde{x},u)\in [s,T]\times \mathbb {R}\times \mathbb {R }\times \mathbb {A}_{1},x\) is a random variable such that \(x\in \mathbb {L} ^{1}\left( \Omega ,\mathcal {F},\mathbb {R}\right) \) and \(\left( \Psi (\cdot ),Q(\cdot )\right) \in \mathbb {R\times R}\) given by equation mean-field BSDE-(2.3).
We introduce the adjoint equations involved in the stochastic maximum principle for our singular mean-field control problem. The adjoint equation turns out to be a linear mean-field BSDE. So for any \(\left( u(\cdot ),\eta (\cdot )\right) \in \mathcal {U}_{1}\times \mathcal {U}_{2}\) and the corresponding state trajectory \(x(t)=x^{u,\eta }(t)\), we consider the following adjoint equations
If we denote by
then the adjoint Eq. (2.3) can be rewritten as follows
As it is well known that under conditions (H1) and (H2), the adjoint Eq. (2.3) admits one and only one \(\mathcal {F}_{t}-\) adapted solution pair \(\left( \Psi (\cdot ),Q(\cdot )\right) \in \mathbb { L}_{\mathcal {F}}^{2}\left( \left[ s,T\right] ;\mathbb {R}\right) \times \mathbb {L}_{\mathcal {F}}^{2}\left( \left[ s,T\right] ;\mathbb {R}\right) \). This equation reduces to the standard one as in [5], when the coefficients do not explicitly depend on the expected value (or the marginal law) of the underlying diffusion process.
We note that since the derivatives \(f_{x},f_{\tilde{x}}, \sigma _{x},\sigma _{\tilde{x}},\ell _{x},\ell _{\tilde{x} },h_{x}\), and \(h_{\tilde{x}}\) are bounded, by assumptions (H1), we have the following estimate
3 Mean-Field Maximum Principle for Optimal Singular Control
Our purpose in this section is to establish a stochastic maximum principle for optimal singular stochastic control for systems driven by nonlinear-controlled MFSDEs. Since the control domain is assumed to be convex, the proof of our result based on convex perturbation for both continuous and singular parts of the control process.
The main result of this paper is stated in the following theorem.
Theorem 3.1
(Mean-field maximum principle for optimal singular control in integral form). Let Conditions (H1) and (H2) hold. Then there exists a unique pair of \(\mathcal {F}_{t}-\) adapted processes \(\left( \Psi ^{*}(\cdot ),Q^{*}(\cdot )\right) \) such that for all \(\left( u,\eta \right) \in \mathbb {A} _{1}\times \mathbb {A}_{2}:\)
where \(\left( \Psi ^{*}(\cdot ),Q^{*}(\cdot )\right) \) be the solution of adjoint equation (2.3) corresponding to \((u^{*}(\cdot ),\eta ^{*}(\cdot ),x^{*}(\cdot ))\).
Corollary 3.2
(Mean-field maximum principle for optimal singular control). Under Conditions of Theorem 3.1 Then there exists a unique pair of \(\mathcal {F}_{t}\mathcal {-}\)adapted processes \(\left( \Psi ^{*}(\cdot ),Q^{*}(\cdot )\right) \) solution of mean-field FBSDE-(2.3)such that for all \(\left( u,\eta \right) \in \mathbb {A}_{1}\times \mathbb {A}_{2}:\)
To prove Theorem 3.1 and Corollary 3.2 we need the following results which we have to translate to our mean-field singular problem.
Let \(\left( u^{*}(\cdot ),\eta ^{*}(\cdot ),x^{*}(\cdot )\right) \) be the optimal solution of the control problem (1.1)–(1.2).We derive the variational inequality (3.1) in several steps, from the fact that
where \((u^{\varepsilon }(\cdot ),\eta ^{\varepsilon }(\cdot ))\) is the so-called convex perturbation of \(\left( u^{*}(\cdot ),\eta ^{*}(\cdot )\right) \) defined as follows: \(t\in \left[ s,T\right] \),
where \(\varepsilon >0\) is sufficiently small and \(\left( u(\cdot ),\eta (\cdot )\right) \) is an arbitrary element of \(\mathcal {U}_{1}\times \mathcal { U}_{1}\). We emphasize that the convexity of \(\mathbb {A}_{1}\times \mathbb {A} _{1}\) has the consequence that \((u^{\varepsilon }(t),\eta ^{\varepsilon }(t))\in \mathcal {U}_{1}\times \mathcal {U}_{1}\).
Let \(x^{\varepsilon }(\cdot )=x^{(u^{\varepsilon },\eta ^{\varepsilon })}(\cdot )\) be the solutions of MFSDE-(1.1) corresponding to admissible control \((u^{\varepsilon }(t),\eta ^{\varepsilon }(t))\).
Lemma 3.3
Let Conditions (H1) and (H2) hold. Then we have
Proof
From standard estimates and the Burkholder–Davis–Gundy inequality we obtain
by applying assumption (H2) and the Lipschitz conditions on the coefficients \(f,\sigma \) with respect to \(x,\tilde{x},u\) we get
from Definition 2.1 and Gronwall’s inequality, the desired result follows. \(\square \)
Lemma 3.4
Let \(\mathcal {Z}(t)\) be the solution of the following linear MFSDE
then the following estimation holds
Proof
Noting that since under conditions (H1) and (H2) then linear MFSDE-(3.5) has a unique solution. We put
by a simple computations we show that
then from the above equation and (3.7) we conclude that if \(\gamma ^{\varepsilon }(t)\) is independent of singular part, then we can use similar method developed in Li [17] for the rest of proof.\(\square \)
Lemma 3.5
For any \((u(\cdot ),\eta (\cdot ))\in \mathcal { U}_{1}\times \mathcal {U}_{1}\) we have
Proof
By applying similar arguments developed in [17] for \(\mathbb {I}_{1}, \mathbb {I}_{2}\), and \(\mathbb {I}_{3}\). Let us turn to estimate \( \mathbb {I}_{4}.\) From (3.4), we get for any \(\eta (\cdot )\in \mathcal { U}_{2}\)
then we can easily prove that
Finally, we conclude that \(\lim _{\varepsilon \rightarrow 0}\mathbb {I}_{4}=0,\) which completes the proof of Lemma 3.5. \(\square \)
Proof of Theorem 3.1
By applying Itô’s formula to \( \Psi ^{*}(t)\mathcal {Z}(t)\) and take expectation, where \(\mathcal {Z} (s)=0\), then a simple computations show that
where
and
where \(b_{\rho }(t)=\dfrac{\partial b}{\partial \rho }(t,x^{*}(t), \mathbb {E}(x^{*}(t)),u^{*}(t))\) for \(b=f,\sigma ,\ell \) and \(\rho =x, \tilde{x},u\).
Combining (3.8)–(3.11) and the fact that \(\Psi ^{*}(T)=h_{x}\left( x^{*}(T),\mathbb {E}(x^{*}(T))\right. +\mathbb { E}\left[ h_{\tilde{x}}\left( x^{*}(T),\mathbb {E}(x^{*}(T))\right. \right] \) we get
Finally, applying Lemma 3.5 we get
This completes the proof of Theorem 3.1. \(\square \)
4 Mean-Field Sufficient Conditions for Optimal Singular Control
The specific purpose of this section is to derive sufficient conditions for optimal stochastic singular control for systems governed by MFSDE. We prove that under certain convexity conditions on the Hamiltonian and on the function \(h\), the necessary conditions also become sufficient for optimality. We assume
Conditions (H3): The functions \(h(\cdot ,\cdot ): \mathbb {R}\times \mathbb {R\rightarrow R},\) and \(H(t,\cdot ,\cdot ,\cdot ,\Psi ,Q):\mathbb {R}\times \mathbb {R}\times \mathbb {A}_{1}\rightarrow \mathbb {R}\) satisfy
Let \(\left( v(\cdot ),\xi (\cdot )\right) \in \mathcal {U}_{1}\times \mathcal { U}_{2}\) be an admissible control, and \(x^{v,\xi }(\cdot ),(\Psi ^{v}(\cdot ),Q^{v}(\cdot ))\) be the solution of (1.1), (2.4), respectively, corresponding to \(\left( v(\cdot ),\xi (\cdot )\right) \).
Theorem 4.1
(Mean-field sufficient conditions for optimal singular control). Let conditions (H1)–(H3) hold. Suppose that the singular control \(\left( v(\cdot ),\xi (\cdot )\right) \) satisfies: for any \(\left( u(\cdot ),\eta (\cdot )\right) \in \mathcal {U}_{1}\times \mathcal {U}_{2}:\)
then \(\left( v(\cdot ),\xi (\cdot )\right) \) is an optimal control, i.e.,
Proof
For any \(\left( u(\cdot ),\eta (\cdot )\right) \in \mathcal {U}_{1}\times \mathcal {U}_{2}\), and from (1.2) we get
Using (4.1) then we get
Now, by noting that
and by using integration by parts formula to \(\Psi ^{v}(t)(x^{u,\eta }(t)-x^{v,\xi }(t))\) we get
where
from (2.4) we get
and
By applying (4.2) it holds that
from (4.11) we obtain
Finally, since \(\left( u(\cdot ),\eta (\cdot )\right) \) is an arbitrary element of \(\mathcal {U}_{1}\times \mathcal {U}_{2}\) the desired result (4.4) follows immediately by combining (4.3) and (4.12). This completes the proof of Theorem 4.1. \(\square \)
5 Application: Singular Mean-Field Linear Quadratic Control Problem
In this section, optimal singular stochastic control problem for linear MFSDEs is considered. The optimal control is represented by a state feedback form involving both \(x(\cdot )\) and \(\mathbb {E(}x(\cdot ))\) , via the solutions of Riccati ordinary differential equations.
We assume \(T=1,s=0,\mathbb {A}_{1}=\left[ 0,1\right] ,\mathbb { A}_{2}=\left[ 0,1\right] \),
We consider the following mean-field singular stochastic control problem:
Minimize
subject to
For a given optimal control \(\left( u^{*}(\cdot ),\eta ^{*}(\cdot )\right) \), then due to (2.3) the corresponding adjoint equation gets the form
The Hamiltonian function corresponding to control problem (5.1)–(5.2) gets the form
By applying Theorem 4.1, and the fact that
we deduce that the optimal control is given by
In order to solve explicitly the above equation, we try the adjoint process \( \Psi ^{*}(\cdot )\) as follows
Applying (5.3) and by comparing the coefficient of \(x^{*}(t)\) and \(\mathbb {E}\left( x^{*}(t)\right) \) we can show that \(\Phi _{1}(t)\) and \(\Phi _{2}(t)\) are given by the following Riccati equations
then we get
However, from Theorem 4.1, the singular part \(\eta ^{*}(\cdot )\) satisfying: for any \(\eta (\cdot )\in \mathcal {U}_{2}\),
Noting that Eqs. (5.7) are Riccati ordinary differential equations admit one and only one solution (see also Yong [21] and Li [17]). In particular case, if we define
and let \(\eta (\cdot )\in \mathcal {U}_{2}\) such that
then by a simple computations it is easy to see that
which implies that \(\eta ^{*}(t)\) satisfies for any \(t\in \left[ 0,1 \right] \)
5.1 Some Discussions
In Sects. 3 and 4 we established mean-field type necessary and sufficient conditions, respectively, for optimal singular control for dynamics evolves according to nonlinear-controlled MFSDE. Moreover, the cost functional is also of mean-field type. In Sect. 5 a linear quadratic singular control problem of mean-field type was solved explicitly, where the optimal control was given in feedback form by means of Riccati differential equations.
- Case 1::
-
If the coefficients \(f,\sigma \), and \(\ell \) of the underlying diffusion process and the cost functional do not explicitly depend on the expected value, Theorem 3.1 reduces to Theorem 8 proved in [4].
- Case 2::
-
If we assume \(G(\cdot )\equiv K(\cdot )\equiv 0,\) our necessary and sufficient conditions (Theorem 3.1, Theorem 4.1) reduce to (Theorem 3.1 and Theorem 3.3), respectively, proved in Li [17].
- Case 3::
-
Let Case 1 and Case 2 hold, then our necessary conditions of optimality (Theorem 3.1) reduce to Theorem 2.1 proved in Bensoussan [5].
References
Ahmed, N.U.: Nonlinear diffusion governed by McKean–Vlasov equation on Hilbert space and optimal control. SIAM J. Control Optim. 46, 356–378 (2007)
Alvarez, L.H.R.: A class of solvable singular stochastic control problems. Stoch. Stoch. Rep. 67, 83–122 (1999)
Alvarez, L.H.R.: Singular stochastic control linear diffusion and optimal stopping: a class of solvable problems. SIAM J. Control Optim. 39, 1697–1710 (2001)
Bahlali, S., Chala, A.: The stochastic maximum principle in optimal control of singular diffusions with non linear coefficients. Random Oper. Stoch. Equ. 13, 1–10 (2005)
Bensoussan, A.: Lectures on stochastic control. In: Mitter, S.K., Moro, A. (eds.) Lecture Notes in Mathematics, vol. 972, pp. 1–62. Springer, Berlin (1983)
Borkar, V.S., Kumar, K.S.: Mean-field limit in portfolio optimization. Stoch. Anal. Appl. 28, 884–906 (2010)
Buckdahn, R., Djehiche, B., Li, J.: A general stochastic maximum principle for SDEs of mean-field type. Appl. Math. Optim. 64, 197–216 (2011)
Buckdahn, R., Li, J., Peng, S.: Mean-field backward stochastic differential equations and related partial differential equations. Stoch. Process. Appl. 119, 3133–3154 (2009)
Cadenillas, A., Haussman, U.: The stochastic maximum principle for singular control problem. Stoch. Stoch. Rep. 49(3–4), 211–237 (1994)
Dufour, F., Miller, B.: Singular stochastic control problem. SIAM J. Control Optim. 43(2), 705–730 (2004)
Dufour, F., Miller, B.: Maximum principle for singular stochastic control problem. SIAM J. Control Optim. 45(2), 668–698 (2006)
Hafayed, M., Abbas, S.: A general maximum principle for stochastic differential equations of mean-field type with jump processes. Technical Report (2013)
Hafayed, M., Abbas, S., Veverka, P.: On necessary and sufficient conditions for near-optimal singular stochastic controls. Optim. Lett. 7(5), 949–966 (2013)
Haussmann, U.G., Suo, W.: Singular optimal control I, II. SIAM J. Control Optim. 33(3), 916–959 (1995)
Kac, M.: Foundations of kinetic theory. In: Proceedings of the 3rd Berkeley Symposium on Mathematical Statistics and Probability, vol. 3, pp. 171–197 (1956)
Karatzas, I., Ocone, D., Wang, H., Zervos, M.: Finite-fuel singular control with discretionary stopping. Stoch. Stoch. Rep. 71, 1–50 (2000)
Li, J.: Stochastic maximum principle in the mean-field controls. Automatica 48, 366–373 (2012)
Meyer-Brandis, T., \(\emptyset \)ksendal, B., Zhou, X.Y.: A mean-field stochastic maximum principle via Malliavin calculus. Stochastics. 84(5–6), 643–666 (2012)
Peng, S.: A general stochastic maximum principle for optimal control problems. SIAM J. Control Optim. 28(4), 966–979 (1990)
Veretennikov, A.Y.: On Ergodic Measures for Mean-Field Stochastic Equations, from Stochastic Calculus to Mathematical Finance. Springer, Berlin (2006)
Yong, J.: A linear-quadratic optimal control problem for mean-field stochastic differential equations. SIAM J. Control Optim. 51(4), 2809–2838 (2013)
Acknowledgments
The author would like to thank the editor, the associate editor, and anonymous referees for their constructive corrections and valuable suggestions that improved the manuscript. The author was partially supported by Algerian PNR Project Grant 08/u07/857, ATRST-(ANDRU) 2011–2013.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Hafayed, M. A Mean-Field Necessary and Sufficient Conditions for Optimal Singular Stochastic Control. Commun. Math. Stat. 1, 417–435 (2013). https://doi.org/10.1007/s40304-014-0023-0
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s40304-014-0023-0
Keywords
- Stochastic optimal singular control
- Mean-field stochastic maximum principle
- Mean-field necessary and sufficient conditions of optimality
- McKean–Vlasov SDEs
- Convex perturbation