Abstract
This paper deals with a class of time inconsistent stochastic linear quadratic optimal control problems in Markovian framework. Three notions, i.e., closed-loop equilibrium strategies, open-loop equilibrium controls and open-loop equilibrium strategies, are characterized in unified manners. These results indicate clearer and deeper distinctions among these notions. For example, in particular time consistent setting, the open-loop equilibrium controls are fully characterized by first-order, second-ordernecessaryoptimalityconditions, and are not optimal in general, while the closed-loop equilibrium controls naturally reduce into closed-loopoptimalcontrols.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
1 Introduction
Through out this paper, \((\Omega ,{\mathcal {F}},{\mathbb {P}},{\mathbb {F}})\) is a complete filtered probability space, on which one-dimensional standard Brownian motion \(W(\cdot )\) is defined. Here \({\mathbb {F}}\equiv \{{\mathcal {F}}_t\}_{t\ge 0}\) is the natural filtration of \(W(\cdot )\) augmented by \({\mathbb {P}}\)-null sets.
1.1 Formulation of Time Inconsistent Optimal Control Problems
For any \(t\in [0,T)\), we consider the following stochastic differential equation (SDE):
and the cost functional defined by
Here A, B, C, D, Q, S, R, G are suitable matrix-valued (deterministic) functions, \(b,\sigma \) are proper stochastic processes, and \({\mathbb {E}}_t(\cdot ):={\mathbb {E}}[\,\cdot \,|{\mathcal {F}}_t]\) stands for conditional expectation operator. In the above, \(X(\cdot )\), valued in \({\mathbb {R}}^n\), is called the state process, \(u(\cdot )\), valued in \({\mathbb {R}}^m\), is called the control process, and \((t,\xi )\in {\mathscr {D}}\) is called the initial pair where
We denote the set of all control processes by
Under some mild conditions on the coefficients, for any initial pair \((t,\xi )\) and a control \(u(\cdot )\in {\mathscr {U}}[t,T]\), the state equation (1.1) admits a unique solution \(X(\cdot )=X(\cdot \,;t,x,u(\cdot ))\), and the cost functional \(J(t,\xi ;u(\cdot ))\) is well-defined. We pose the following stochastic linear quadratic (SLQ) optimal control problem.
Problem (SLQ) For any given \((t,\xi )\), find a \(\bar{u}(\cdot )\in {\mathscr {U}}[t,T]\) such that
Any \({\bar{u}}(\cdot )\in {\mathscr {U}}[t,T]\) satisfying (1.3) is called an optimal control for the given initial pair \((t,\xi )\), the corresponding state process \({\bar{X}}(\cdot )\) is called an optimal state process for \((t,\xi )\), \(({\bar{X}}(\cdot ),{\bar{u}}(\cdot ))\) is called an optimal pair for \((t,\xi )\), and \(V(\cdot \,,\cdot )\) is called the value function of Problem (SLQ).
For above optimal control problem, it is reasonable to keep the state process stable with respect to possible variation of random factors. To this end, one effective way is to add the variation of \(X(\cdot )\), i.e.
into the cost functional (e.g., [2, 3, 11,12,13,14, 21, 25], etc).Therefore, it is natural to propose the following general modified cost functional
Here \(\widetilde{S},\widetilde{R}, \widetilde{G},\widetilde{Q}\) are deterministic matrices-valued functions and g is a vector.
In this scenario, the optimal controls become time-inconsistent, i.e., the “optimal” control based on this moment may not keep optimality in future. We refer to [25] for some explicit examples.
1.2 Related Literature
The study on time inconsistency by economists actually dates back to Strotz [17] in the 1950s. One possible way to treat time inconsistency is to discuss the pre-committed controls for which the solutions are verified to be optimal only at the initial time.
In this paper, we shall discuss above optimal control problem from another viewpoint. More precisely, we investigate the time inconsistency within a game-theoretic framework and analyze the time-consistent equilibrium solution (e.g., [10, 15]). Recently, people began to treat the equilibrium controls using the ideas of stochastic control theories, and developed several different approaches in the existing papers. These methods range from dynamic programming principles and verification procedures to maximum principles and variational techniques.
In Björk-Murgoci [1], Björk et al [2], the authors examined a general class of time inconsistent problems under Markovian framework by equilibrium value functions. In the continuous case, they formally derived the extended HJB equations, and then rigorously proved the verification theorem by the conclusions of discrete time case, see Theorem 5.2 in [2]. They also present some special cases including a linear quadratic control problem in which equilibrium solutions are constructed. This method was also used to treat investment-reinsurance problems with mean-variance criterion, see e.g., [14, 27].
In Yong [23, 25], the author discussed a class of time inconsistent optimal control problems by multi-person differential games approach, where a new kind of equilibrium HJB equations/sytems of Riccati equations were introduced. Unlike [1, 2], they started the investigations in continuous time setting, made partition on time intervals and used tricks of forward-backward stochastic differential equations (FBSDEs). Further study along this can be found in [19, 22], and so on.
In Ekeland and Lazrak [8, 9], they considered some financial problems such as investment and consumption model with time-inconsistency feature. They used the variational ideas to introduce certain feedback/closed-loop equilibrium controls, and spread out discussions via equilibrium value functions. Compared with the general situation in [1, 2], the particular form of equilibrium value functions were proposed according to the given cost functional, while the complex convergence arguments were avoided.
Inspired by the ideas of stochastic maximum principles in optimal control theories, Hu et al. [11] studied a class of time inconsistent SLQ problems in Markovian setting, introduced open-loop equilibrium controls and their closed-loop representations, derived general sufficient conditions through a flow of FBSDEs or systems of backward ordinary differential equations (ODEs). Just recently, the same authors continued to discuss the uniqueness of open-loop equilibrium controls in [12]. More related details can also be found in [7, 20, 21].
1.3 Unified Approach and Contributions
As to Problem (SLQ), in this article we propose a unified method to characterize the open-loop equilibrium controls, open-loop equilibrium strategies, closed-loop equilibrium strategies. We combines the ideas from variational analysis, forward-backward stochastic differential equations and forward-backward decoupling procedures. In the following, we provide a brief outline of our approach.
For any \((\Theta _1,\Theta _2,\varphi )\in L^2(0,T;{\mathbb {R}}^{m\times n})\times L^2(0,T;{\mathbb {R}}^{m\times n})\times L^2_{{\mathbb {F}}}(0,T;{\mathbb {R}}^m),\) we start with control processes
They can reduce into the required equilibrium controls and perturbed controls in various settings (see Sect. 4.4).
In view of the definitions for equilibrium controls, we proceed to consider the difference of the cost functional at u, \(u^\varepsilon \). To do so, given X and \(X^\varepsilon \), we introduce, respectively, backward stochastic differential equations (BSDEs) with conditional expectations. We point out that the one associated with \(X^\varepsilon \) appears for the first time in the literature. As a result, we obtain two forward-backward systems in which the terminal parts and generators of backward systems rely respectively on X, \(X^\varepsilon \).
To tackle the limit part in the definitions of both open-loop and closed-loop equilibrium controls (i.e., Definitions 2.1, 2.3 next), we continue to decouple the above two forward-backward systems. More precisely, we make conjectures on the solutions of backward systems, formally obtain a class of systems of BSDEs merely depending on given coefficients, and then verify our arguments rigorously. At last we establish our characterizations with proper convergence procedures.
At this very moment, it is worth mentioning that the previous proposed approach demonstrates several new advantages on the treatment of both open-loop equilibrium controls, closed-loop equilibrium controls/strategies. Unlike [1, 2, 23, 25], our procedures on closed-loop equilibrium strategy in continuous time drop the reliance on complex convergence arguments from discrete time to continuous case. Comparing with [11, 12], our methodology on open-loop equilibrium controls neither requires any non-definite assumptions on the involved coefficients, nor directly uses the conclusions of stochastic maximum principles. Moreover, it can be adjusted into the random coefficients case, see [21].
Even though both open-loop equilibrium controls and closed-loop equilibrium controls are widely investigated in the literature, there is no paper discussing their differences to our best. In this paper, we give a clear picture by the obtained characterizations. For example, in the classical SLQ setting, open-loop equilibrium controls are fully characterized by first-order, second-order necessary conditions. In other words, they are weaker than optimal controls (Remark 3.1). However, in the same situation, the closed-loop equilibrium controls happen to reduce exactly into closed-loop optimal controls (Remark 3.3). Eventually, we point out that the characterizations on open-loop, closed-loop equilibrium strategies, respectively, include two different second-orderequilibriumconditions, which are absent in nearly all the relevant articles.
1.4 Outline of the Article
The remainder of this article of structured as follows. In Sect. 2, an overview of assumptions, notation used in the sequel is provided. In Sect. 3, the main conclusions of this article are gathered and some important remarks are demonstrated. In Sect. 4, the proofs of the main results in Sect. 3 are given. Section 5 concludes this article.
2 Preliminary Notations
Given \(H:={\mathbb {R}}^n,{\mathbb {R}}^{n\times n},{\mathbb {S}}^{n\times n},\) etc, \(0\le s\le t\le T\), we define some spaces as follows.
We also need the following hypotheses on coefficients of (1.1), (1.4).
(H1) Suppose \(A, \ B,\ C,\ D, \ R,\ \widetilde{R}, \ Q,\ \widetilde{Q},\ S,\ \widetilde{S} \in L^{\infty }(0,T;H)\), \(G,\ \widetilde{G},\ g\in H\), \(b \in L^2_{{\mathbb {F}}}(\Omega ;L^1(0,T;H))\), \(\sigma \in L^2_{{\mathbb {F}}}(0,T;H)\).
To begin with, we look at Problem (SLQ) from an open-loop equilibrium control viewpoint. The following definition is adapted from [11, 12].
Definition 2.1
Given \(X^*(0)=x_0\in {\mathbb {R}}^n\), a state-control pair
is called an open-loop equilibrium pair if for any \(t\in [0,T)\), \(\varepsilon >0\), \(v\in L^2_{{\mathcal {F}}_t}(\Omega ;{\mathbb {R}}^m)\),
where \(u^{v,\varepsilon } =u^*+vI_{[t,t+\varepsilon ]}\). Here \(u^*\) and \(X^*\) are called open-loop equilibrium control and open-loop equilibrium state process.
Roughly speaking, the definition shows the dynamiclocaloptimality in some sense. In this paper we will explore deeper properties of such equilibrium controls via their characterizations.
Due to our particular linear quadratic structure, we also introduce the notion of open-loop equilibrium strategy, which is independent of initial state \(x_0\).
Definition 2.2
\((\Theta ^*,\varphi ^*)\in L^2(0,T;{\mathbb {R}}^{m\times n})\times L^2_{{\mathbb {F}}}(0,T;{\mathbb {R}}^{m})\) is called an open-loopequilibriumstrategy of Problem (SLQ), if for any \(X^*(0)=x_0\in {\mathbb {R}}^n\), \(u^*:=\Theta ^*X^*+\varphi ^*\), with \(X^*\) being the associated state process, is an open-loop equilibrium control.
The open-loop equilibrium strategy enable us to capture the explicit feedback representation of open-loop equilibrium control. However, it is different from the following one.
Definition 2.3
\((\Theta ^*,\varphi ^*)\in L^2(0,T;{\mathbb {R}}^{m\times m}) \times L^2 _{{\mathbb {F}}}(0,T;{\mathbb {R}}^{m}) \) is called a closed-loop equilibrium strategy, if for any initial state \(x_0\in {\mathbb {R}}^n\), \(t\in [0,T)\), \(\varepsilon >0\), \(v\in L^2_{{\mathcal {F}}_t}(\Omega ;{\mathbb {R}}^m)\),
where
\(X^*\), \(X^\varepsilon \) are the state process on [0, T] associated with \(u^*\), \(u^\varepsilon \), respectively.
We emphasize that both open-loop equilibrium strategy and closed-loop equilibrium strategy are independent of initial state \(x_0\). However, the perturbed control \(u^{v,\varepsilon }\) in Definition 2.1 is actually different from \(u^\varepsilon \) in Definition 2.3. In this paper, we will demonstrate further connections between these two kinds of strategies.
In the following, let K be a generic constant which varies in different context and
3 Characterizations of Equilibrium Controls/Strategies
In this part, we state the main results of this article. To begin with, recall the notation in (1.5), we introduce the following system which is useful next,
It is easy to check that [18]
We start with the case of open-loop equilibrium controls. Recall (1.5), we choose \(\Theta _1\equiv 0,\)\(\Theta _2\equiv 0\), which indicates that \(u=\varphi \in L^2_{{\mathbb {F}}}(0,T;{\mathbb {R}}^m)\). Moreover, (3.1) reduces to
For later clarification, we replace (Y, Z) by (P, L), and omit the reference to the time variable for simplicity.
Above \(P_1, P_2\) do not rely on u while \(P_3, P_4\) do. As to (3.2), it is easy to see
Considering X in (1.1), we define
Theorem 3.1
Suppose (H1) holds, \(P_1\) satisfies (3.2). Then \({\bar{u}} \) is an open-loop equilibrium control for Problem (SLQ) associated with initial state \({\bar{X}}(0)=x_0\in {\mathbb {R}}^n\) if and only if
- (i)
the following inequality holds,
$$\begin{aligned} \begin{array}{ll} \displaystyle {\mathscr {R}}(s)+D(s)^{\top }P_1(s)D(s)\ge 0,\qquad s\in [0,T], \ \ a.e. \ \ \end{array} \end{aligned}$$(3.4) - (ii)
given \(({\bar{M}},{\bar{N}})\) in (3.3) associated with \({\bar{u}} \),
$$\begin{aligned} \displaystyle {\mathscr {R}}(s) {\bar{u}}(s)+{\mathscr {S}}(s){\bar{X}}(s)+ B(s)^{\top } {\bar{M}}(s,s) + D(s)^{\top } {\bar{N}}(s)=0, \ \ s\in [0,T]. \ \ a.e. \nonumber \\ \end{aligned}$$(3.5)
Above (3.4), (3.5) are named as first-order, second-orderequilibriumconditions of open-loop equilibrium controls for Problem (SLQ).
Remark 3.1
As to Theorem 3.1, let us make the following comments,
- (1)
Above \(P_1\) is indeed the unique solution of classical second-order adjoint equation in control theories of mean-field SDEs. That is to say, (3.4) coincides with the corresponding second-ordernecessaryoptimalitycondition [4]. To our best knowledge, (3.4) has not been discussed seriously in [11, 12], and other related papers on open-loop equilibrium controls.
- (2)
If we denote \(\widehat{v}(\cdot ,t)\) the (time inconsistent) optimal control of Problem (SLQ), then the first-order adjoint equation [24] is
$$\begin{aligned} \left\{ \begin{array}{ll} \displaystyle d \widehat{Y}(s,t)=-\Big [A(s)^{\top }\widehat{Y}(s,t)+C(s)^{\top }\widehat{Z}(s,t)+Q(s)\widehat{X}(s,t)+S(s)^{\top } \widehat{v}(s,t)\\ \displaystyle \qquad \qquad \quad \quad +\,\widetilde{Q}(s){\mathbb {E}}_t \widehat{X}(s,t)+\widetilde{S}(s)^{\top }{\mathbb {E}}_t\widehat{v}(s,t)\Big ]ds+\widehat{Z}(s,t)dW(s),\\ \displaystyle \widehat{Y}(T,t)=G X (T,t)+\widetilde{G} {\mathbb {E}}_tX (T,t), \end{array}\right. \end{aligned}$$(3.6)and the first-order necessary optimality condition is
$$\begin{aligned} \begin{array}{ll} \displaystyle R(s)\widehat{v}(s,t)+\widetilde{R}(s){\mathbb {E}}_t \widehat{v}(s,t)+S(s)\widehat{X}(s,t)+\widetilde{S}(s){\mathbb {E}}_t \widehat{X}(s,t)\\ \displaystyle \qquad -B(s)^{\top }\widehat{Y}(s,t)-D(s)^{\top }\widehat{Z}(s,t)=0,\ \ s\in [t,T].\ \ \ a.e.\ \ a.s. \end{array} \end{aligned}$$(3.7)Let us return back to our framework. Given \(({\bar{X}},{\bar{u}})\) in (1.1), we see that \(({\bar{M}},{\bar{N}})\) satisfies
$$\begin{aligned} \left\{ \!\!\!\!\begin{array}{ll} \displaystyle d{\bar{M}}(s,t)=-\Big [A(s)^{\top }{\bar{M}}(s,t) + C(s)^{\top }{\bar{N}}(s)+Q(s){\bar{X}}(s)+S(s)^{\top } {\bar{u}}(s) \\ \displaystyle \qquad \qquad \qquad +\,\widetilde{Q}(s){\mathbb {E}}_t {\bar{X}}(s)+\widetilde{S}(s)^{\top }{\mathbb {E}}_t {\bar{u}}(s)\Big ]dr+{\bar{N}}(s)dW(s), \\ \displaystyle M(T,t)=G X (T)+\widetilde{G} {\mathbb {E}}_tX (T). \end{array}\right. \end{aligned}$$(3.8)Obviously, above (3.6), (3.7) are in general different from our (3.8), (3.5). But if there is no time-inconsistency, i.e., \(\widetilde{R}=\widetilde{Q}=\widetilde{S}=\widetilde{G}=0\), then they coincide with each other.
- (3)
If \(\widetilde{R}=\widetilde{S}=S=0,\)R, Q, G are positive definite matrices, then (3.4) is obvious to see. In this scenario, a characterization of open-loop equilibrium control, which is different yet equivalent with (3.5), was given in Theorem 3.5 of [12] without involving systems (3.2).
- (4)
We compare our equilibrium controls with optimal controls when \(\widetilde{R}=\widetilde{Q}=\widetilde{S}=\widetilde{G}=0\). Recall that the characterization of open-loop optimal controls includes first-order necessary condition and the following convexity condition [5, 6, 26]
$$\begin{aligned} \displaystyle {\mathbb {E}}_t\int _t^T u^{\top }\big [R u+S X^{0}+B^{\top }Y^{0}+D^{\top }Z^{0}\big ]dr\ge 0,\ \ \ \forall u\in L^2_{{\mathbb {F}}}(t,T;{\mathbb {R}}^m), \nonumber \\ \end{aligned}$$(3.9)where \(X^0\) satisfies (1.1) with \(\xi =0\), \((Y^0,Z^0)\) solves (3.8) with \(\widetilde{G}=\widetilde{S}=\widetilde{Q}=0\) and \(X\equiv X^0\). In contrast, Theorem 3.1 indicates that the open-loop equilibrium controls are fully characterized by first-order, second-order necessary optimality conditions. Therefore, when there is no time inconsistency in Problem (SLQ), the exact difference between open-loop equilibrium controls and open-loop optimal controls lies in (3.4) and (3.9).
Next we characterize the open-loop equilibrium strategy. Recall (1.5), we choose \(\Theta _1\equiv 0\), which implies that \(u=\Theta _2X+\varphi \). Moreover, (3.1) reduces to
We suppress the time variable for simplicity. We also define processes \(({\mathcal {M}},{\mathcal {N}})\) as follows,
Theorem 3.2
Suppose (H1) holds, \(P_1\) satisfies (3.2). Then \((\Theta ^*,\varphi ^*)\) is a pair of open-loop equilibrium strategy if and only if
- (i)
condition (3.4) holds true,
- (ii)
there exist \( {\mathcal {P}}_i^*,\)\( {\mathcal {L}}^*_j\) satisfying BSDEs (3.10) with \((\Theta _2,\varphi )\equiv (\Theta ^*,\varphi ^*)\) and
$$\begin{aligned} \left\{ \!\!\! \begin{array}{ll} \displaystyle \big [{\mathscr {R}}+ D^{\top }{\mathcal {P}}_1^* D\big ]\Theta ^*+ B^{\top }\big [{\mathcal {P}}_1^*+{\mathcal {P}}_2^*\big ]+D^{\top } {\mathcal {P}}_1^* C+{\mathscr {S}}=0,\ \ a.s. \ \ a.e. \\ \displaystyle \big [{\mathscr {R}}+D^{\top }{\mathcal {P}}_1^* D\big ]\varphi ^*+ D^{\top }\big [{\mathcal {P}}_1^*\sigma +{\mathcal {L}}_4^*\big ]+ B^{\top } \big [{\mathcal {P}}_3^*+{\mathcal {P}}_4^*\big ]=0.\ \ a.s. \ \ a.e. \end{array}\right. \qquad \end{aligned}$$(3.12)
Above (3.4), (3.12) are named as first-order, second-orderequilibriumconditions of open-loop equilibrium strategy for Problem (SLQ). Different from (3.5), the conclusion (3.12) focuses on the coefficients and there are no state process or control variable involved.
Remark 3.2
As to Theorem 3.2, we point out two useful facts.
- (1)
Given \((\Theta ^*,\varphi ^*)\), it is easy to check that \(({\mathcal {M}}^*,{\mathcal {N}}^*)\) solves (3.8) with \(u^*:=\Theta ^* X^*+\varphi ^*\). Since \(u^*\) is an open-loop equilibrium control, and one can define \((M^*,N^*)\) as in (3.3). By the uniqueness of BSDEs, we end up with \(({\mathcal {M}}^*,{\mathcal {N}}^*)\equiv (M^*,N^*)\). In other words, the unique solution of (3.8) has two different forms of representations.
- (2)
From (3.12), there exists \(\theta '\in L^2(0,T;{\mathbb {R}}^{n\times n})\), \(\varphi '\in L^2_{{\mathbb {F}}}(0,T;{\mathbb {R}}^m) \) s.t.
$$\begin{aligned} \left\{ \begin{array}{ll} \displaystyle \Theta ^*=-\big [{\mathscr {R}}+D^{\top }{\mathcal {P}}_1^* D\big ]^{\dagger }\big [B^{\top }({\mathcal {P}}_1^*+{\mathcal {P}}_2^* )+D^{\top }{\mathcal {P}}_1^* C+{\mathscr {S}}\big ]\\ \displaystyle \qquad \quad +\,\Big \{I-\big [{\mathscr {R}}+D^{\top }{\mathcal {P}}_1^*D\big ]^{\dagger }\big [{\mathscr {R}}+D^{\top } {\mathcal {P}}_1^*D\big ]\Big \}\theta ',\\ \displaystyle \varphi ^*= -\big [{\mathscr {R}}+D^{\top }{\mathcal {P}}_1^*D\big ]^{\dagger }\big [B^{\top }[{\mathcal {P}}_4^*+{\mathcal {P}}_3^*] +D^{\top }[{\mathcal {P}}_1^*\sigma +{\mathcal {L}}_4^*]\big ]\\ \displaystyle \qquad \quad +\,\Big \{I-\big [{\mathscr {R}}+D^{\top }{\mathcal {P}}_1^*D\big ]^{\dagger }\big [{\mathscr {R}}+D^{\top }{\mathcal {P}}_1^*D\big ]\Big \}\varphi '. \end{array}\right. \end{aligned}$$(3.13)Moreover,
$$\begin{aligned} \left\{ \begin{array}{ll} \displaystyle {\mathcal {R}}\Big (B^{\top }({\mathcal {P}}_1^*+{\mathcal {P}}_2^*)+D^{\top }{\mathcal {P}}_1^* C+{\mathscr {S}}\Big )\subset {\mathcal {R}}\Big ({\mathscr {R}}+D^{\top }{\mathcal {P}}_1^* D\Big ),\ \ a.e. \\ \displaystyle \Big [B^{\top }[{\mathcal {P}}_4^*+{\mathcal {P}}_3^*] +D^{\top }[{\mathcal {P}}_1^*\sigma +{\mathcal {L}}_4^*]\Big ]\in {\mathcal {R}}\Big ({\mathscr {R}}+D^{\top }{\mathcal {P}}_1^* D\Big ), \ a.e. \ a.s. \\ \displaystyle \big [{\mathscr {R}}+D^{\top }{\mathcal {P}}_1^* D\big ]^{\dagger }\big [B^{\top }({\mathcal {P}}_1^*+{\mathcal {P}}_2^*)+D^{\top }{\mathcal {P}}_1^* C+{\mathscr {S}}\big ]\in L^2(0,T;{\mathbb {R}}^{m\times n}),\\ \displaystyle \Big [\big [{\mathscr {R}}+D^{\top }{\mathcal {P}}_1^*D\big ]^{\dagger }\big [B^{\top }[{\mathcal {P}}_4^*+{\mathcal {P}}_3^*] +D^{\top }[{\mathcal {P}}_1^*\sigma +{\mathcal {L}}_4^*]\big ] \in L^2_{{\mathbb {F}}}(0,T;{\mathbb {R}}^m). \end{array}\right. \qquad \end{aligned}$$(3.14)In the above, \({\mathcal {R}}(A)\), \(A^{\dagger }\) are the range, pseudo-inverse of matrix A, respectively. As a result, we obtain the explicit forms of \((\Theta ^*,\varphi ^*)\), as well as some intrinsic relations among coefficients.
At last, we give the characterizations of closed-loop equilibrium strategies. Recall (1.5), we choose \(\Theta _2\equiv 0\) which implies that \(u=\Theta _1X+\varphi \). Moreover, (3.1) reduces to
We also define two processes \({\mathscr {M}},\ {\mathscr {N}}\) as follows,
Theorem 3.3
A pair of \((\Theta ^*,\varphi ^*)\in L^2(0,T;{\mathbb {R}}^{m\times n}) \times L^2_{{\mathbb {F}}}(0,T;{\mathbb {R}}^{m})\) is a closed-loop equilibrium strategy if and only if there exists \({\mathscr {P}}_i^*\) satisfies (3.15) with \((\Theta _1,\varphi )\equiv (\Theta ^*,\varphi ^*)\) such that
For the closed-loop equilibrium strategy \((\Theta ^*,\varphi ^*)\), the first inequality in (3.17) is referred as the second-orderequilibriumcondition, while the other two conditions are named as first-orderequilibriumcondition.
Remark 3.3
If \(\widetilde{G}=\widetilde{S}=\widetilde{Q}=\widetilde{R}=0\), above (3.17) reduces to
where \(({\mathscr {P}}^*_1,{\mathscr {P}}^*_4,{\mathscr {L}}_4^*)\) are described as,
According to [17, 18], (3.18) is equivalent to the optimality of strategy pair \((\Theta ^*,\varphi ^*)\). Therefore, we find the following two useful aspects.
- (1)
Our defined closed-loop equilibrium controls/strategies are natural extension of closed-loop optimal controls/strategies.
- (2)
From the optimality viewpoint, closed-loop equilibrium controls/strategies are essentially different and stronger than open-loop equilibrium controls/strategies.
To conclude this section, we clarify the relations among above three characterizations in the following three manners.
Firstly, we make comparisons among (3.2), (3.10), (3.15). On the one hand, they are basically the same in the sense that all of them are particular cases of system (3.1). On the other hand, they also differ from each other in the following three ways. In the first place, the solutions of the first two equations in (3.2), (3.15) are symmetric, while the analogue of (3.10) are non-symmetric [25]. In the second place, the first two equations in (3.2) merely depends on given coefficients, while the counterparts in (3.10) and (3.15) are determined by \(\Theta _1\) or \(\Theta _2\). In the third place, the last two equations in (3.2) rely on control process u, while the analogue equations in (3.10) and (3.15) are determined by \(\varphi \).
Secondly, we make the following comments on the second-order equilibrium conditions. For both open-loop equilibrium controls and open-loop equilibrium strategies, we use \({\mathscr {R}}+D^{\top } P_1 D\ge 0\), where \(P_1\) satisfies the second-order adjoint equation in LQ optimal control problems of mean-field SDEs. This condition was missing in [11, 12, 20, 21]. As to closed-loop equilibrium strategies, we introduce \({\mathscr {R}}+D^{\top }{\mathscr {P}}_1^* D\ge 0\) where \({\mathscr {P}}_1^*\) satisfies one backward ordinary differential equation that contains Riccati equation as special case. Notice that this condition has not been discussed in [1, 2, 23, 25].
Thirdly, let us give three examples.
First we consider the relations between open-loop equilibrium controls and open-loop equilibrium strategies. Given open-loop equilibrium strategy \((\Theta ^*,\varphi ^*)\), for any initial state \(x_0\in {\mathbb {R}}^n\), Problem (SLQ) admits an open-loop equilibrium control \(u^*:=\Theta ^* X^*+\varphi ^*\). Conversely, the conclusion is not true, even when there is no time consistency feature in Problem (SLQ). To see it, we look at the following example.
Example 3.1
Suppose \(m=n=1\), function B is continuous, \(B^{-1}\) exists and is bounded, and
By introducing
we can represent \(X(\cdot )\) by
Since \(G>0\), for any \(x_0\in {\mathbb {R}}\), we see that \({\bar{u}}\) is an optimal control as along as the corresponding \({\bar{X}}(T)=0\). To this end, we set
Moreover, \({\bar{u}}\) is also an open-loop equilibrium control satisfying (3.4), (3.5).
Next we claim that the open-loop equilibrium strategy does not exists. Actually, if there exists \((\Theta ^*,\varphi ^*)\), it then follows from (3.12) that \(B {\mathcal {P}}_1^*\equiv 0\). a.s. a.e. On the other hand, \({\mathcal {P}}^*_1(T)=G>0\), by the continuity of \({\mathcal {P}}^*_1\) and B, as well as the existence of \(B^{-1}\), there exists \(T_1<T\) such that for any \(t\in [T_1,T]\), \(B(t){\mathcal {P}}_1^*(t)\ne 0\). A contradiction arises.
Now let us turn to the connections between open-loop equilibrium strategies and closed-loop equilibrium strategy. The following example shows that open-loop equilibrium strategy equals to closed-loop equilibrium strategy.
Example 3.2
Suppose \(m=n=1\), \(G>0\), \(Q(\cdot )\ge 0,\)
From Theorem 3.2, we have \({\mathcal {P}}_1^* \Theta ^*+{\mathcal {P}}_1^*+{\mathcal {P}}_2^*=0\) where
It is easy to see that \(({\mathcal {P}}_1^*,{\mathcal {P}}_2^*):=(P,-P)\) satisfies (3.21) with \(\Theta \equiv 0\), where
Suppose there is another \(\Theta '\) and \(({\mathcal {P}}_1',{\mathcal {P}}_2')\in C([0,T];{\mathbb {R}}^2)\) such that (3.21) is satisfied and
Notice that
By uniqueness, \(\widehat{\mathcal {P}}'\equiv 0\). By (3.20), \(\frac{1}{{\mathcal {P}}_1'}\) exists and is bounded. Hence \(\Theta '=0\).
Due to \(b=\sigma =0\), it is easy to check there exists a unique \(\varphi ^*\equiv 0\). Moreover, condition (3.4) holds.
To sum up, Problem (SLQ) admits a unique pair of open-loop equilibrium strategy \((\Theta ^*,\varphi ^*)\equiv (0,0)\) under condition (3.20).
Now we look at the closed-loop equilibrium strategies \((\Xi ^*,\phi ^*)\). Here we change the notation for later comparisons. From Theorem 3.3, one has \({\mathscr {P}}_1^* \Xi ^*+({\mathscr {P}}_1^*+{\mathscr {P}}_2^*)=0,\) where
For P in (3.22), we see that \(({\mathscr {P}}_1^*,{\mathscr {P}}_2^*)\equiv (P,-P)\) satisfies (3.24) with \(\Xi ^*\equiv 0\). Moreover, \({\mathscr {P}}_1^*\ge 0.\) Suppose there is another \(\Xi '\) and \(({\mathscr {P}}_1',{\mathscr {P}}_2')\) such that (3.24) is satisfied and
Notice that
By (3.20), \(\frac{1}{{\mathscr {P}}_1'}\) exists and is bounded. Hence \(\Xi '=-\frac{\widehat{\mathscr {P}}'}{{\mathscr {P}}_1'}\) and (3.25) can be rewritten as,
By uniqueness, \(\widehat{\mathscr {P}}'\equiv 0\), which yields \(\Xi '=0\).
Due to \(b=\sigma =0\), it is easy to check there exists a unique \(\phi ^*\equiv 0\).
To sum up, Problem (SLQ) admits a unique pair of closed-loop equilibrium strategy \((\Xi ^*,\phi ^*)\equiv (0,0)\) under condition (3.20), which is the same as open-loop equilibrium strategy.
The following example shows that open-loop equilibrium strategy are also distinctive from closed-loop equilibrium strategy in some cases.
Example 3.3
Suppose \(m=n=1\),
As to open-loop equilibrium strategy \((\Theta ^*,\varphi ^*)\), from Theorem 3.2, \({\mathcal {P}}_1^* \Theta ^*+{\mathcal {P}}_1^*+{\mathcal {P}}_2^*=0\), where \(({\mathcal {P}}_1^*,{\mathcal {P}}_2^*)\) satisfies (3.21) with \(Q=\widetilde{Q}=0\). As to closed-loop equilibrium strategies \((\Xi ^*,\phi ^*)\), from Theorem 3.3, one has \({\mathscr {P}}_1^* \Xi ^*+({\mathscr {P}}_1^*+{\mathscr {P}}_2^*)=0,\) where \(({\mathscr {P}}_1^*,{\mathscr {P}}_2^*)\) satisfies (3.24) with \(Q=\widetilde{Q}=0.\) We illustrate out point by three steps.
- Step 1:
Under (3.27), we look at the solvability of system (3.21). Consider an ODE of
$$\begin{aligned} \left\{ \begin{array}{ll} \displaystyle dP(s)=-(2A(s)-1-\frac{\widetilde{G}}{G})P(s)ds,\ \ s\in [0,T],\\ \displaystyle P(T)=1. \end{array}\right. \end{aligned}$$It is easy to see that \(({\mathcal {P}}_1^*(\cdot ),{\mathcal {P}}_2^*(\cdot ))\equiv (G P(\cdot ),\widetilde{G} P(\cdot ))\) is a solution of (3.27).
- Step 2:
Under (3.27), we claim that system (3.24) is solvable with \({\mathscr {P}}_1^*>{\mathscr {P}}_2^*\). By a simplification, it is suffice to consider the regularity of the following system
$$\begin{aligned} \left\{ \begin{array}{ll} \displaystyle P_1(t)=G\exp \Big [\int _t^T \big [2A(s)-1+ \frac{ |P_2(s)|^2 }{|P_1(s)|^2} \big ]ds\Big ],\ \ t\in [0,T],\\ \displaystyle P_2(t)=\widetilde{G}\exp \Big [\int _t^T 2\big \{A(s)-1-\frac{P_2(s)}{P_1(s)}\big \}ds\Big ], \ \ t\in [0,T]. \end{array}\right. \end{aligned}$$(3.28)For later usefulness, we make the following conventions,
$$\begin{aligned} \left\{ \begin{array}{ll} \displaystyle \Vert p\Vert _{[\tau _1,\tau _2]}:=\sup _{t\in [\tau _1,\tau _2]}|p(t)|,\ \ \tau _1,\ \tau _2\in [0,T],\\ \displaystyle L_1:= \widetilde{G} e^{-2T(\Vert A\Vert _{[0,T]}+2)},\ \ L_2:= G e^{2T( \Vert A\Vert _{[0,T]}+2)},\ \ K_1:= L_2 e^{2(\Vert A\Vert _{[0,T]}+1)T},\\ \displaystyle C_{L_1,L_2}([0,T];{\mathbb {R}}^2):=\big \{(x_1,x_2){\in } C([0,T];{\mathbb {R}}^2), \ L_1\le x_i(\cdot ){\le } L_2,\ \ x_1(\cdot ){\ge } x_2(\cdot ) \big \}. \end{array}\right. \end{aligned}$$For \(i=1,2,\) we choose \( (p_1^{(i)}, p_2^{(i)})\in C_{L_1,L_2}([0,T];{\mathbb {R}}^2), \) and define
$$\begin{aligned} \left\{ \begin{array}{ll} \displaystyle P_2^{(i)}(t):=\widetilde{G}\exp \Big [\int _t^T 2\big \{A(s)-1-\frac{p_2^{(i)}(s)}{p_1^{(i)}(s)}\big \}ds\Big ],\ \ t\in [0,T],\\ \displaystyle P_1^{(i)}(t):=G\exp \Big [\int _t^T \big [2A(s)-1+ \frac{ |p_2^{(i)}(s)|^2 }{|p_1^{(i)}(s)|^2} \big ]ds\Big ],\ \ t\in [0,T]. \end{array}\right. \end{aligned}$$Under (3.27), it is easy to see that \((P_1^{(i)},P_2^{(i)})\in C_{L_1,L_2}([0,T];{\mathbb {R}}^2)\). We denote by
$$\begin{aligned} \begin{array}{ll} \displaystyle \widehat{k}_1(s):=k_1^{(1)}(s)-k_1^{(2)}(s), \ \ \widehat{k}_2(s):=k_2^{(1)}(s)-k_2^{(2)}(s), \ \ s\in [0,T], \ \ k:=P, \ p. \end{array} \end{aligned}$$After some calculation, one has
$$\begin{aligned} \left\{ \begin{array}{ll} \displaystyle \big |\widehat{P}_2(t)\big | \le 4K_1 e^{2\frac{L_2}{L_1}(T-t)} \frac{L_2(\Vert \widehat{p}_1\Vert _{[T-t,T]}+\Vert \widehat{p}_2\Vert _{[T-t,T]})}{L_1^2}(T-t),\\ \displaystyle \big |\widehat{P}_1(t)\big | \le 2 K_1 e^{2\frac{L_2^2}{L_1^2}(T-t)} \frac{L_2^2(\Vert \widehat{p}_1\Vert _{[T-t,T]}+\Vert \widehat{p}_2\Vert _{[T-t,T]})}{L_1^3}(T-t). \end{array}\right. \end{aligned}$$We can choose \(T_1\) such that for \(\delta :=T-T_1\),
$$\begin{aligned} \begin{array}{ll} \displaystyle 2 K_1\frac{L_2}{L_1^2} \Big [2 e^{2\frac{L_2}{L_1}T}+ e^{2\frac{L_2^2}{L_1^2}T}\frac{L_2}{L_1} \Big ]\delta =\frac{1}{2}. \end{array} \end{aligned}$$By contraction, one has the existence and uniqueness of \((P_1,P_2)\) satisfying (3.28) on \([T_1,T]\). Now let us look at the case of \([T_1-\delta ,T_1] \), where
$$\begin{aligned} \left\{ \begin{array}{ll} \displaystyle P_2(t)=P_2(T_1)\exp \Big [\int _t^{T_1} 2\big \{A(s)-1-\frac{P_2(s)}{P_1(s)}\big \}ds\Big ], \ \ t\in [0,T_1],\\ \displaystyle P_1(t)=P_1(T_1)\exp \Big [\int _t^{T_1}\big [2A(s)-1+ \frac{ |P_2(s)|^2 }{|P_1(s)|^2} \big ]ds\Big ],\ \ t\in [0,T_1]. \end{array}\right. \end{aligned}$$Given \((p_1^{(i)},p_2^{(i)})\in C_{L_1,L_2}([0,T_1];{\mathbb {R}}^2)\), we see that \((P_1^{(i)},P_2^{(i)})\in C_{L_1,L_2}([0,T_1];{\mathbb {R}}^2)\), and
$$\begin{aligned} \left\{ \begin{array}{ll} \displaystyle \big |\widehat{P}_2(t)\big | \le 4K_1 e^{2\frac{L_2}{L_1}(T_1-t)} \frac{L_2(\Vert \widehat{p}_1\Vert _{[T_1-t,T_1]}+\Vert \widehat{p}_2\Vert _{[T_1-t,T_1]})}{L_1^2}(T_1-t),\ \ t\in [0,T_1],\\ \displaystyle \big |\widehat{P}_1(t)\big |\le 2 K_1 e^{2\frac{L_2^2}{L_1^2}(T_1-t)} \frac{L_2^2(\Vert \widehat{p}_1\Vert +\Vert \widehat{p}_2\Vert )}{L_1^3}(T_1-t),\ \ t\in [0,T_1]. \end{array}\right. \end{aligned}$$By the choice of \(\delta \), we obtain the solvability in \([T-2\delta ,T_1]\). By induction, one has the conclusion in [0, T].
- Step 3:
We claim that \(\Theta ^*\ne \Xi ^*\). To prove this result, we first recall that \(\widehat{\mathcal {P}}^*:={\mathcal {P}}_1^*+{\mathcal {P}}_2^*\), \(\widehat{\mathscr {P}}^*:={\mathscr {P}}_1^*+{\mathscr {P}}_2^*\) satisfy
$$\begin{aligned} \left\{ \begin{array}{ll} \displaystyle d\widehat{\mathcal {P}}^*(s)\!=\!- \big [(2A(s) +\Theta ^*(s)) \widehat{\mathcal {P}}^*(s)\big ] ds,\ \ s\in [0,T],\\ \displaystyle d\widehat{\mathscr {P}}^*(s)=- \Big [(2 A(s)+ \Xi ^*(s))\widehat{\mathscr {P}}^*(s)\Big ] ds,\ \ s\in [0,T],\\ \displaystyle \widehat{\mathcal {P}}^*(T)=(\widetilde{G}+G),\ \ \widehat{\mathscr {P}}^*(T)= \widetilde{G}+G. \end{array}\right. \end{aligned}$$(3.29)If \(\Theta ^* \equiv \Xi ^* \), then by the uniqueness, \(\widehat{\mathcal {P}}^*\equiv \widehat{\mathscr {P}}^*\). According to above two steps, \(\frac{1}{{\mathcal {P}}_1^*},\)\(\frac{1}{{\mathscr {P}}_1^*}\) exist. Therefore, due to the definitions of \(\Theta ^*,\)\(\Xi ^*\), one has \({\mathscr {P}}_1^*\equiv {\mathcal {P}}_1^*\) which implies that
$$\begin{aligned} \begin{array}{ll} \displaystyle \Theta ^* {\mathcal {P}}_1^*=\Xi ^*{\mathscr {P}}_1^*=2{\mathscr {P}}_1^*\Xi ^*+|\Xi ^*|^2{\mathscr {P}}_1^*. \end{array} \end{aligned}$$Hence \( \Xi ^* {\mathscr {P}}_1^*(\Xi ^*+1)=0, \) which leads to \(\Xi ^*=-1\) or \(\Xi ^*=0.\) This indicates that \(\widehat{\mathscr {P}}^*\equiv 0\) or \({\mathscr {P}}_2^*\equiv 0\). Since \(\widehat{\mathscr {P}}^*(T)=G+\widetilde{G}>0\), by the continuity of \(\widehat{\mathscr {P}}^*\) and the fact of \(\Xi ^*=-\frac{\widehat{\mathscr {P}}^*}{{\mathscr {P}}_1^*}\), we see that \(\Xi ^*\equiv 0\) does not hold. Similarly, since \(\widetilde{G}\ne 0\), by the continuity of \({\mathscr {P}}_2^*\), we see that \({\mathscr {P}}_2^*\equiv 0\) does not hold as well. We finish Step 3 by contradiction.
4 Proofs of the Main Results
In this section, we prove Theorems 3.1–3.3.
For \((\Theta _1,\Theta _2,\varphi )\in L^2(0,T;{\mathbb {R}}^{m\times m})\times L^2(0,T;{\mathbb {R}}^{m\times m})\times L^2_{{\mathbb {F}}} (0,T;{\mathbb {R}}^{m})\), we consider
For \(t\in [0,T)\), \(\varepsilon >0\), \(v\in L^2_{{\mathcal {F}}_t}(\Omega ;{\mathbb {R}}^m)\), let \( X^\varepsilon \) solve the following perturbed system:
with \(s\in [0,T]\). Hence we see that \(X_0^\varepsilon :=X^\varepsilon -X\) satisfies
By Proposition 2.1 in [18], we have the following estimate of \(X_0^\varepsilon \)
We also define
Lemma 4.1
Suppose (H1) holds, \((\Theta _1,\Theta _2,\varphi )\) are given as above, u, \(u^\varepsilon \) are defined in (4.4). Then we have
where \({\mathscr {R}},\ {\mathscr {S}}\) are defined in (2.3),
Proof
By above definitions of X, \(X^{\varepsilon }\) and \(X_0^{\varepsilon }\), we deal with the terms in the cost functional one by one. First let us treat the term associated with Q,
Then we look at the one with S. From the definitions of u and \(u^\varepsilon \), we have
We also have
Similarly one can obtain the terms involving \(\widetilde{Q}\), \(\widetilde{S}\), \(\widetilde{R}\) as,
At last we have the follows results on the terms associated with G and \(\widetilde{G} \),
To sum up, we deduce above (4.5). \(\square \)
Next we carry out further study on \(J_1(t)\) and \(J_2(t)\) by making some equivalent transformations. In fact, from the definitions of equilibrium controls it is unavoidable to take certain convergence arguments. Fortunately, in above we derive the important and useful structure of \({\mathbb {E}}_t\int _t^{t+\varepsilon }\mathop {\langle }F_2(r),v\mathop {\rangle }dr\). Consequently, we will derive similar expressions for other terms in \(J_1(t)\), \(J_2(t)\). This is the starting point for our later investigations.
4.1 A New Decoupling Result
Inspired by the decoupling techniques in the literature (e.g., [11, 24], etc), we present one conclusion which serves our purpose of this paper. It is interesting in its own right and may be potentially useful for (among others) various problems.
Given \(t\in [0,T]\), we consider
(H1) For \(H:={\mathbb {R}}^{m},\ {\mathbb {R}}^{n},\ {\mathbb {R}}^{n\times n}\), etc, suppose
For \(t\in [0,T]\) and \(s\in [t,T]\), suppose that
where \(P_1,\ P_2\) are deterministic, \(P_3, \ P_4\) are stochastic processes satisfying
Here \(\Pi _i\) are to be determined. It is easy to see
Using Itô’s formula, we derive that
As a result, we have
Consequently, it is necessary to have
In this case, from (4.7), (4.8), we see that
On the other hand,
At this moment, we can choose \(\Pi _i(\cdot )\) in the following ways,
Next we make above arguments rigorous. Given the notations in (2.3), for \(s\in [0,T]\), we consider the following systems of equations
From Proposition 2.1 in [18], under (H1) we see the following regularities,
At this moment, for \(s\in [0,T]\), and \(t\in [0,s]\), we define a pair of processes
By the results of \(P_i\), we can conclude that
where \(M_{d}(s)\equiv M(s,s)\) with \(s\in [0,T]\). We present the following result.
Lemma 4.2
Given \((\Theta ,\varphi )\in L^2(0,T;{\mathbb {R}}^{m\times n})\times L^2_{{\mathbb {F}}}(0,T;{\mathbb {R}}^m)\), suppose (X, Y, Z) is the unique solution of (4.6) and (M, N) is defined in (4.10). Then for any \(t\in [0,T]\),
Proof
Given (4.10), it is easy to see that
Using Itô’s formula, we know that
Consequently, after some calculations one has
Considering \(P_i(T)\) in (4.9), we see that for any \(t\in [0,T]\), \((M,N)\in L^2_{{\mathbb {F}}}(\Omega ;C([t,T];{\mathbb {R}}^n))\times L^2_{{\mathbb {F}}}(0,T;{\mathbb {R}}^n)\) satisfies the backward equation in (4.6). The conclusion is followed by the uniqueness of BSDEs. \(\square \)
4.2 A New Expression of \(J_1\)
In this part, we deal with \(J_1(t)\) in Lemma 4.1. For convenience, we rewrite the equation of \(X_0^\varepsilon :=X^\varepsilon -X\) as
where \(s\in [0,T],\) and
We introduce
where X satisfies (4.1), \(F_1\) is in Lemma 4.1. From Proposition 2.1 in [18], (4.12) is solvable with
By Itô’s formula on [t, T], we have
From (4.12) we then arrive at
Inspired by Lemma 4.2, we introduce
Moreover, the following equalities hold on [t, T],
Consequently,
This shows that
By the definition of \(J_1(t)\) and above (4.13), we see that
Lemma 4.3
Suppose (H1) holds, X solves (4.1) associated with \((\Theta _1,\Theta _2,\varphi )\), and \(J_1(t)\) is defined in Lemma 4.1. Then (4.15) is true, where \({\mathcal {P}}_i\) satisfies (4.14).
4.3 A New Expression of \(J_2\)
In the following, we turn to treating \(J_2\). To this end, we introduce
where \(F_1^\varepsilon \) is defined in Lemma 4.1. From Proposition 2.1 in [18], we see that
Recall \(X_0^\varepsilon \) in (4.11), we obtain the following by Itô’s formula,
As a result, we then have
By the decoupling tricks in Lemma 4.2, we introduce
Moreover, from Lemma 4.2, the following holds on [t, T],
At this moment, we take a closer look at \(({{\bar{{\mathcal {P}}}}}_3,{{\bar{{\mathcal {L}}}}}_3),\)\(({{\bar{{\mathcal {P}}}}}_4,{{\bar{{\mathcal {L}}}}}_4).\) By the uniqueness of BSDEs in Proposition 2.1 of [18], we have the following equalities
where
Consequently, on [t, T] we conclude that
As a result,
By the estimate of \(X_0^\varepsilon \), for almost \(t\in [0,T),\)
From the equations of \((\widetilde{\mathcal {P}}_3,\widetilde{\mathcal {P}}_4)\),
To sum up, by the definition of \(J_2\) and (4.16), for almost \(t\in [0,T)\) we deduce that
Lemma 4.4
Suppose (H1) holds, \(X_0^\varepsilon \) is in (4.11) associated with \((\Theta _1,\Theta _2,\varphi )\), and \(J_2(t)\) is defined in Lemma 4.1. Then (4.17) is true.
4.4 Proofs of the Main Results
We are in the position to give the proofs of the main results in Sect. 3.
To begin with, we give the proof of Theorem 3.1.
Proof
In Lemmas 4.1, 4.3, and 4.4, we take \(\Theta _1\equiv \Theta _2\equiv 0\). Hence for the notations in (4.4), \(u\equiv \varphi \) and
where \(P_i\), \(i=1,2\), \((P_j,L_j)\), \(j=3,4,\) satisfies (3.2). Moreover, for any \(t\in [0,T),\) by the estimate of \(X_0^\varepsilon \),
We set out to define \({\bar{X}}\) the state process associated with \({\bar{u}}\), \(u^{v,\varepsilon }:={\bar{u}}+vI_{[t,t+\varepsilon ]}\), and for any \(t\in [0,T)\)
with \(({\bar{M}},{\bar{N}})\) in (3.3) corresponding to \({\bar{u}}.\) To sum up, \(u\equiv {\bar{u}}={{\bar{\varphi }}}\) is an equilibrium control associated with \(x_0\) if and only if for any \(t\in [0,T),\)\(v\in L^2_{{\mathcal {F}}_t}(\Omega ;{\mathbb {R}}^m)\),
Given \(t\in [0,T)\), this holds if and only if both \({\mathscr {H}}_0(t)=0\) and \({\mathscr {D}}_0(t)\ge 0\). Since both \({\mathscr {R}}\) and \(P_1\) are bounded and deterministic, we thus know that
If \({\mathscr {H}}_0(t)=0\), then by Lemma 3.4 in [12], above (3.5) holds. Conversely, if (3.5) is true, we immediately obtain \({\mathscr {H}}_0(t)=0\). \(\square \)
Next we present the proof of Theorem 3.2.
Proof
In Lemmas 4.1, 4.3, and 4.4, we take \(\Theta _1\equiv 0\). Hence for the notations in (4.4), we have \(u\equiv \Theta _2 X+\varphi \) and
where \({\mathcal {P}}_i\), \(i=1,2\), \(({\mathcal {P}}_j,{\mathcal {L}}_j)\), \(j=3,4,\) satisfies (3.10). Moreover, by the estimate of \(X_0^\varepsilon \),
For open-loop equilibrium strategy pair \((\Theta ^*,\varphi ^*)\) and associated equilibrium control \(u^*\), we define \(X^*\) the corresponding state process as,
and perturbed control \(u^{v,\varepsilon }:=\Theta ^* X^*+\varphi ^*+vI_{[t,t+\varepsilon ]}\). Moreover, for \(({\mathcal {M}}^*,{\mathcal {N}}^*)\) in (3.11) corresponding to \(u^*,\) let
To sum up, \(u^*=\Theta ^*X^*+\varphi ^*\) is an equilibrium control associated with \(x_0\in {\mathbb {R}}^n\) if and only if for any \(t\in [0,T],\)\(v\in L^2_{{\mathcal {F}}_t}(\Omega ;{\mathbb {R}}^m)\),
where \({\mathscr {D}}_0\) is in (4.18). Given \(t\in [0,T)\), this holds if and only if both \({\mathscr {H}}_1(t)=0\) and \({\mathscr {D}}_0(t)\ge 0\). Since both \({\mathscr {R}}\) and \(P_1\) are bounded and deterministic,
\(\Longrightarrow \) If \({\mathscr {H}}_1(t)=0\), then by Lemma 3.4 in [12], for almost \(s\in [0,T],\) we have
Notice that (4.20) holds for any \(x_0\in {\mathbb {R}}^n\). We choose \(x_0=0\), and denote the state process by \(X^*_0\). As a result,
At this moment, given \(I\in {\mathbb {R}}^{n\times n}\) the unit matrix, we consider the following equation
the solvability of which is easy to see. Moreover, \( {\mathscr {X}}^{-1}\) also exists. By the standard theory of SDEs,
Using the existence of \({\mathscr {X}}^{-1}\), it is easy to see above (3.12).
\(\Longleftarrow \) In this case, it is easy to see (4.20) with \(u^*:=\Theta ^*X^*+\varphi ^*\). Consequently, the conclusion is followed by (4.19), (3.4) and the fact of \({\mathscr {H}}_1(t)=0\). \(\square \)
At last, we show the proof of Theorem 3.3.
Proof
In Lemmas 4.1, 4.3, and 4.4, we take \(\Theta _2\equiv 0\). Hence for the notations in (4.4), \(u\equiv \Theta _1 X+\varphi \) and
where \({\mathscr {P}}_i\), \(i=1,2\), \(({\mathscr {P}}_j,{\mathscr {L}}_j)\), \(j=3,4,\) satisfies (3.15). Moreover, in view of the estimate of \(X_0^\varepsilon \), it is straightforward to see
For closed-loop equilibrium strategy pair \((\Theta ^*,\varphi ^*)\) in the sense of Definition 2.3 and associated equilibrium control \(u^*:=\Theta ^* X^*+\varphi ^*\), we define \(X^*\) the corresponding state process as,
and perturbed control variable \(u^{v,\varepsilon }:=\Theta ^* X^{v,\varepsilon }+\varphi ^*+vI_{[t,t+\varepsilon ]}\). In addition, for \(({\mathscr {M}}^*,{\mathscr {N}}^*)\) in (3.16) corresponding to \(u^*,\) we denote by
To sum up, \(u^*=\Theta ^*X^*+\varphi ^*\) is a closed-loop equilibrium control associated with \(x_0\in {\mathbb {R}}^n\) if and only if for any \(t\in [0,T],\)\(v\in L^2_{{\mathcal {F}}_t}(\Omega ;{\mathbb {R}}^m)\),
Given \(t\in [0,T)\), this holds if and only if both \({\mathscr {H}}_2(t)=0\) and \({\mathscr {D}}_1(t)\ge 0\).
\(\Longrightarrow \) Given equilibrium strategy pair \((\Theta ^*,\varphi ^*)\), we conclude that \({\mathscr {P}}_1^*\) is bounded and deterministic. Recall the requirement on \({\mathscr {R}}\), it is clear that
If \({\mathscr {H}}_2(t)=0\), then by Lemma 3.4 in [12], for almost \(s\in [0,T],\) we have
Notice that (4.24) holds for any \(x_0\in {\mathbb {R}}^n\). We choose \(x_0=0\), and denote the state process by \(X^*_0\). As a result,
As in Theorem 3.2, we introduce \({\mathscr {X}}\) satisfying (4.21), and therefore obtain (3.17) by following the same spirit of that in Theorem 3.2.
\(\Longleftarrow \) In this case, it is easy to see (4.20) with \(u^*:=\Theta ^*X^*+\varphi ^*\). Consequently, the conclusion is followed by (4.22), (4.23) and the fact of \({\mathscr {H}}_1(t)=0\). \(\square \)
5 Concluding Remarks
In the Markovian setting, a unified approach by variational technique is developed to build the characterizations for three notions, i.e., closed-loop equilibrium controls/strategies, open-loop equilibrium controls, as well as the closed-loop representations of open-loop equilibrium controls. The intrinsic differences among different equilibrium controls are also revealed clearly and deeply. Related studies with random coefficients or in mean-field setting are under consideration. We hope to do some relevant research in future.
References
Björk, T., Murgoci, A.: A theory of Markovian time-inconsistent stochasitic control in discrete time. Financ. Stoch. 18, 545–592 (2014)
Björk, T., Khapko, M., Murgoci, A.: On time-inconsistent stochastic control in continuous time. Financ. Stoch. 21, 331–360 (2017)
Björk, T., Murgoci, A., Zhou, X.: Mean-variance portfolio optimization with state-dependent risk aversion. Math. Financ. 24, 1–24 (2014)
Buckdahn, R., Djehiche, B., Li, J.: A general stochastic maximum principle of SDEs of mean-field type. Appl. Math. Optim. 64, 197–216 (2011)
Chen, S., Li, X., Zhou, X.: Stochastic linear quadratic regulators with indefnite control weight costs. SIAM J. Control Optim. 36, 1685–1702 (1998)
Chen, S., Yong, J.: Stochastic linear quadratic opitmal control problems. Appl. Math. Optim. 43, 21–45 (2001)
Djehiche, B., Huang, M.: A characterization of sub-game perfect equilibria for SDEs of mean-field type. Dyn. Games Appl. 55, 55–81 (2016)
Ekeland, I., Pirvu, T.: Investment and consumption without commitment. Math. Financ. Econ. 2, 57–86 (2008)
Ekeland, I., Mbodji, O., Pirvu, T.: Time-consistent portfolio management. SIAM J. Financ. Math. 3, 1–32 (2012)
Goldman, S.: Consistent plans. Rev. Econ. Stud. 47, 533–537 (1980)
Hu, Y., Jin, H., Zhou, X.: Time-inconsistent stochastic linear-quadratic control. SIAM J. Control Optim. 50, 1548–1572 (2012)
Hu, Y., Jin, H., Zhou, X.: Time-inconsistent stochastic linear-quadratic control: characterization and uniqueness of equilibrium. SIAM J. Control Optim. 55, 1261–1279 (2017)
Huang, J., Li, X., Wang, T.: Characterizations of closed-loop equilibrium solutions for dynamic mean-variance optimization problems. Syst. Control Lett. 110, 15–20 (2017)
Li, Y., Li, Z.: Optimal time-consistent investment and reinsurance strategies for mean-variance insurers with state dependent risk aversion. Insurance Math. Econ. 53, 86–97 (2013)
Peleg, B., Yaari, M.: On the existence of a consistent course of action when tastes are changing. Rev. Econ. Stud. 40, 391–401 (1973)
Strotz, R.: Myopia and inconsistency in dynamic utility maximization. Rev. Econ. Stud. 23, 165–180 (1955)
Sun, J., Li, X., Yong, J.: Open-loop and closed-loop solvabilities for stochastic linear quadratic optimal control problems. SIAM J. Control Optim. 54, 2274–2308 (2016)
Sun, J., Yong, J.: Linear quadratic stochastic differential games: open-loop and closed-loop saddle points. SIAM J. Control Optim. 52, 4082–4121 (2014)
Wang, H., Wu, Z.: Time-inconsistent optimal control problem with random coefficients and stochastic equilibrium HJB equation. Math. Control Relat. Fields 3, 651–678 (2015)
Wang, H., Wu, Z.: Partially observed time-inconsistency recursive optimization problem and application. J. Optim. Theory Appl. 161, 664–687 (2014)
Wei, J., Wang, T.: Time-consistent mean-variance asset-liability management with random coefficients. Insurance Math. Econ. 77, 84–96 (2017)
Wei, Q., Yong, J., Yu, Z.: Time-inconsistent recursive stochastic optimal control problems. SIAM J. Control Optim. 55, 4156–4201 (2017)
Yong, J.: Time-inconsistent optimal control problem and the equilibrium HJB equation. Math. Control Relat. Fields 2, 271–329 (2012)
Yong, J.: Linear-quadratic optimal control problems for mean-field stochastic differential equations. SIAM J. Control Optim. 51, 2809–2838 (2013)
Yong, J.: Linear-quadratic optimal control problems for mean-field stochastic differential equations-time-consistent solutions. Trans. Am. Math. Soc. 369, 5467–5523 (2017)
Yong, J., Zhou, X.: Stochstic Control: Hamiltonian Systems and HJB Equations. Springer, New York (1999)
Zeng, Y., Li, Z.: Optimal time-consistent investment and reinsurance policies for mean-variance insurers. Insurance Math. Econ. 49, 145–154 (2011)
Author information
Authors and Affiliations
Corresponding author
Additional information
The research was supported by the NSF of China under Grant 11231007, 11401404 and 11471231, and the Fundamental Research Funds for the central Universities.
Rights and permissions
About this article
Cite this article
Wang, T. Equilibrium Controls in Time Inconsistent Stochastic Linear Quadratic Problems. Appl Math Optim 81, 591–619 (2020). https://doi.org/10.1007/s00245-018-9513-x
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00245-018-9513-x