Abstract
This paper discusses discrete-time stochastic linear quadratic (LQ) problem in the infinite horizon with state and control dependent noise, where the weighting matrices in the cost function are assumed to be indefinite. The problem gives rise to a generalized algebraic Riccati equation (GARE) that involves equality and inequality constraints. The well-posedness of the indefinite LQ problem is shown to be equivalent to the feasibility of a linear matrix inequality (LMI). Moreover, the existence of a stabilizing solution to the GARE is equivalent to the attainability of the LQ problem. All the optimal controls are obtained in terms of the solution to the GARE. Finally, we give an LMI -based approach to solve the GARE via a semidefinite programming.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
R. E. Kalman. Contributions to the theory of optimal control. Boletin de la Sociedad Matematica Mexicana, 1960, 5: 102–119.
B. D. O. Anderson, J. B. Moore. Optimal Control–Linear Quadratic Methods. New York: Prentice-Hall, 1989.
F. L. Lewis. Optimal Control. New York: John Wiley & Sons, 1986.
L. Ntogramatzidis, A. Ferrante. On the solution of the Riccati differential equation arising from the LQ optimal control problem. Systems & Control Letters, 2010, 59(2): 114–121.
A. Ferrante, L. Ntogrammatzidis. The generalized continuous algebraic Riccati equation and impulse-free continuous-time LQ optimal control. Automatica, 2014, 50(4): 1176–1180.
W. M. Wonham. On a matrix Riccati equation of stochastic control. SIAM Journal on Control, 1968, 6: 312–326.
D. H. Jacobson, D. H. Martin, M. Pachter, et al. Extensions of Linear–Quadratic Control Theory. Lecture Notes in Control and Information Sciences. Berlin: Springer, 1980.
A. Bensoussan. Stochastic Control of Partially Observed Systems. Cambridge, U.K.: Cambridge University Press, 1992.
M. H. A. Davis. Linear Estimation and Stochastic Control. London: Chapman and Hall, 1977.
E. Yaz. Infinite horizon quadratic optimal control of a class of nonlinear stochastic systems. IEEE Transactions on Automatic Control, 1989, 34(11): 1176–1180.
W. Zhang, B. Chen. On stabilizability and exact observability of stochastic systems with their applications. Automatica, 2004, 40(1): 87–94.
W. Zhang, H. Zhang, B. Chen. Generalized Lyapunov equation approach to state-dependent stochastic stabilization/detectability criterion. IEEE Transactions on Automatic Control, 2008, 53(7): 1630–1642.
Y. Hu, H. Jin, X. Zhou. Time-inconsistent stochastic linearquadratic control. SIAM Journal on Control and Optimization, 2012, 50(3): 1548–1572.
J. M. Yong. Linear-quadratic optimal control problems for meanfield stochastic differential equations. SIAM Journal on Control and Optimization, 2013, 51(4): 2809–2838.
S. Chen, X. Li, X. Zhou. Stochastic linear quadratic regulators with indefinite control weight costs. SIAM Journal on Control and Optimization, 1998, 36(5): 1685–1702.
Z. Qian, X. Zhou. Existence of solutions to a class of indefinite stochastic Riccati equations. SIAM Journal on Control and Optimization, 2013, 51(1): 221–229.
Q. Meng. Linear quadratic optimal stochastic control problem driven by a Brownian motion and a Poisson random martingale measure with random cofficients. Stochastic Analysis and Applications, 2014, 32(1): 88–109.
R. S. Burachik, C. Y. Kaya, S. N. Majeed. A duality approach for solving control-constrained linear-quadratic optimal control problems. SIAM Journal on Control and Optimization, 2014, 52(3): 1423–1456.
Y. Huang, W. Zhang, H. Zhang. Infinite horizon linear quadratic optimal control for discrete-time stochastic systems. Asian Journal of Control, 2008, 10(5): 608–615.
R. T. Ku, M. Athans. Further results on the uncertainty threshold principle. IEEE Transactions on Automatic Control, 1977, 22(5): 866–868.
A. Beghi, D. D. Alessandro. Discrete-time optimal control with control-dependent noise and generalized Riccati difference equations. Automatica, 1998, 34(8): 1031–1034.
M. Ait Rami, X. Chen, X. Zhou. Discrete-time indefinite LQ control with state and control dependent noises. Journal of Global Optimization, 2002, 23(3/4): 245–265.
G. Li, W. Zhang. Study on indefinite stochastic linear quadratic optimal control with inequality constraint. Journal of Applied Mathematics, 2013: DOI 10.1155/2013/805829.
W. Zhang, G. Li. Discrete-time indefinite stochastic linear quadratic optimal control with second moment constraints. Mathmatical Problems in Engineering, 2014: DOI 10.1155/2014/278142.
X. Liu, Y. Li, W. Zhang. Stochastic linear quadratic optimal control with constraint for discrete-time systems. Applied Mathematics and Computation, 2014, 228: 264–270.
R. Penrose. A generalized inverse for matrices. Mathematical Proceedings of the Cambridge Philosophical Society, 1955, 51(3): 406–413.
A. Albert. Conditions for positive and nonnegative definiteness in terms of pseudo-inverse. SIAM Journal on Applied Mathematics, 1969, 17(2): 434–440.
L. Vandenberghe, S. Boyd. Semidefinite programming. SIAM Review, 1996, 38(1): 49–95.
M. Ait Rami, X. Zhou, J. B. Moore. Well-posedness and attainability of indefinite stochastic linear quadratic control in infinite time horizon. Systems & Control Letters, 2000, 41(2): 123–133.
B. R. Copeland, M. G. Safonov. Zero cancelling compensation for singular control problems and their application to the innerouter factorization problem. International Journal of Robust and Nonlinear Control, 1992, 2(2): 139–164.
Author information
Authors and Affiliations
Corresponding author
Additional information
This work was supported by the National Natural Science Foundation of China (Nos. 61174078, 61170054, 61402265) and the Research Fund for the Taishan Scholar Project of Shandong Province of China.
Weihai ZHANG received his Ph.D. degree from Zhejiang University in 1998, Hangzhou, China. He is currently a professor of Shandong University of Science and Technology. His research interests include stochastic optimal control, robust H∞ control, stochastic stability and stabilization.
Yan LI received the M.Sc. and Ph.D. degrees from Shandong University of Science and Technology, China, in 2006 and 2015, respectively. She is a lecturer of Shandong University of Science and Technology. Her research interests include linear and nonlinear stochastic control.
Xikui LIU received the M.Sc. degree from Shandong University of Science and Technology, and the Ph.D. degree from Huazhong University of Science and Technology, China, in 2000 and 2004, respectively. He is a associate professor of Shandong University of Science and Technology. His interests include graph theory and DNA computing.
Rights and permissions
About this article
Cite this article
Zhang, W., Li, Y. & Liu, X. Infinite horizon indefinite stochastic linear quadratic control for discrete-time systems. Control Theory Technol. 13, 230–237 (2015). https://doi.org/10.1007/s11768-015-4147-x
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11768-015-4147-x