Abstract
In this paper, we propose a novel event-triggered near-optimal control for nonlinear continuoustime systems. The receding horizon principle is utilized to improve the system robustness and obtain better dynamic control performance. In the proposed structure, we first decompose the infinite horizon optimal control into a series of finite horizon optimal problems. Then a learning strategy is adopted, in which an actor network is employed to approximate the cost function and an critic network is used to learn the optimal control law in each finite horizon. Furthermore, in order to reduce the computational cost and transmission cost, an event-triggered strategy is applied. We design an adaptive trigger condition, so that the signal transmissions and controller updates are conducted in an aperiodic way. Detailed stability analysis shows that the nonlinear system with the developed event-triggered optimal control policy is asymptotically stable. Simulation results on a single-link robot arm with different noise types have demonstrated the effectiveness of the proposed method.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Song, R Z, Xiao, W D, Sun, C Y. A new self-learning optimal control laws for a class of discrete-time nonlinear systems based on ESN architecture. Sci China Inf Sci, 2014, 57: 068202
Li, C H, Zhang, E C, Jiu L, et al. Optimal control on special Euclidean group via natural gradient algorithm. Sci China Inf Sci, 2016, 59: 112203
Wei, W N. Maximum principle for optimal control of neutral stochastic functional differential systems. Sci China Math, 2015, 58: 1265–1284
Lee, J H. Model predictive control: review of the three decades of development. Int J Control Autom Syst, 2011, 9: 415–424
Bequette, B W. Nonlinear control of chemical processes: a review. Ind Eng Chem Res, 1991, 30: 1391–1413
Mayne, D Q, Rawlings, J B, Rao, C V, et al. Constrained model predictive control: stability and optimality. Automatica, 2000, 36: 789–814
Hu, L S, Huang B, Cao, Y Y. Robust digital model predictive control for linear uncertain systems with saturations. IEEE Trans Automat Contr, 2004, 49: 792–796
Evans M, Cannon M, Kouvaritakis B. Robust and stochastic linear MPC for systems subject to multiplicative uncertainty. IFAC Proc Vol, 2012, 45: 335–341
Fang H, Shang, C S, Chen J. An optimization-based shared control framework with applications in multi-robot systems. Sci China Inf Sci, 2018, 61: 014201
Zhou, L M, Jia L, Wang, Y L. A robust integrated model predictive iterative learning control strategy for batch processes. Sci China Inf Sci, 2019, 62: 219202
Aguilera, R P, Lezana P, Quevedo, D E. Switched model predictive control for improved transient and steady-state performance. IEEE Trans Ind Inf, 2015, 11: 968–977
Venkat, A N, Hiskens, I A, Rawlings, J B, et al. Distributed MPC strategies with application to power system automatic generation control. IEEE Trans Contr Syst Technol, 2008, 16: 1192–1206
Bertsekas, D P. Dynamic programming and suboptimal control: a survey from ADP to MPC*. Eur J Control, 2005, 11: 310–334
Wei, Q L, Liu, D R. A novel policy iteration based deterministic Q-learning for discrete-time nonlinear systems. Sci China Inf Sci, 2015, 58: 122203
Wang D, Mu, C X. Developing nonlinear adaptive optimal regulators through an improved neural learning mechanism. Sci China Inf Sci, 2017, 60: 058201
Ding, D R, Wang, Z D, Han, Q L, et al. Neural-network-based output-feedback control under round-robin scheduling protocols. IEEE Trans Cybern, 2019, 49: 2372–2384
Ernst D, Glavic M, Capitanescu F, et al. Reinforcement learning versus model predictive control: a comparison on a power system problem. IEEE Trans Syst Man Cybern B, 2009, 39: 517–529
Görges D. Relations between model predictive control and reinforcement learning. IFAC-PapersOnLine, 2017, 50: 4920–4928
Lian, C Q, Xu X, Chen H, et al. Near-optimal tracking control of mobile robots via receding-horizon dual heuristic programming. IEEE Trans Cybern, 2016, 46: 2484–2496
Xu X, Chen H, Lian, C Q, et al. Learning-based predictive control for discrete-time nonlinear systems with stochastic disturbances. IEEE Trans Neural Netw Learn Syst, 2018, 29: 6202–6213
Dong L, Yan J, Yuan X, et al. Functional nonlinear model predictive control based on adaptive dynamic programming. IEEE Trans Cybern, 2019, 49: 4206–4218
Anta A, Tabuada P. To sample or not to sample: self-triggered control for nonlinear systems. IEEE Trans Automat Contr, 2010, 55: 2030–2042
Lemmon, M D. Event-triggered feedback in control, estimation, and optimization. In: Networked Control Systems. London: Springer, 2010. 293–358
Vamvoudakis, K G. Event-triggered optimal adaptive control algorithm for continuous-time nonlinear systems. IEEE/CAA J Autom Sin, 2014, 1: 282–293
Duan, G P, Xiao F, Wang L. Hybrid event- and time-triggered control for double-integrator heterogeneous networks. Sci China Inf Sci, 2019, 62: 022203
Zhang, X M, Han, Q L. Event-triggered dynamic output feedback control for networked control systems. IET Control Theor Appl, 2014, 8: 226–234
Zhang, B L, Han, Q L, Zhang, X M. Event-triggered H∞ reliable control for offshore structures in network environments. J Sound Vib, 2016, 368: 1–21
Zhang, X M, Han, Q L. A decentralized event-triggered dissipative control scheme for systems with multiple sensors to sample the system outputs. IEEE Trans Cybern, 2016, 46: 2745–2757
Liu, C X, Gao J, Li, H P, et al. Aperiodic robust model predictive control for constrained continuous-time nonlinear systems: an event-triggered approach. IEEE Trans Cybern, 2018, 48: 1397–1405
Li, H P, Shi Y. Event-triggered robust model predictive control of continuous-time nonlinear systems. Automatica, 2014, 50: 1507–1513
Ding, D R, Han, Q L, Wang, Z D, et al. A survey on model-based distributed control and filtering for industrial cyber-physical systems. IEEE Trans Ind Inf, 2019, 15: 2483–2499
Krichman M, Sontag, E D, Wang Y. Input-output-to-state stability. SIAM J Control Opt, 2001, 39: 1874–1928
Tabuada P. Event-triggered real-time scheduling of stabilizing control tasks. IEEE Trans Automat Contr, 2007, 52: 1680–1685
Heydari A, Balakrishnan, S N. Finite-horizon control-constrained nonlinear optimal control using single network adaptive critics. IEEE Trans Neural Netw Learn Syst, 2013, 24: 145–157
Khalil, H K. Nonlinear Systems. New Jersey: Prentice Hall, 2002
Dong L, Zhong, X N, Sun, C Y, et al. Event-triggered adaptive dynamic programming for continuous-time systems with control constraints. IEEE Trans Neural Netw Learn Syst, 2017, 28: 1941–1952
Zhong, X N, He, H B. An event-triggered ADP control approach for continuous-time system with unknown internal states. IEEE Trans Cybern, 2017, 47: 683–694
Chen H. Model Predictive Control. Beijing: Science Press, 2013
Acknowledgments
This work was supported in part by National Natural Science Foundation of China (Grant Nos. 61803085, 61921004, 61931020) and National Key R&D Program of China (Grant No. 2018AAA0101400).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Dong, L., Yuan, X. & Sun, C. Event-triggered receding horizon control via actor-critic design. Sci. China Inf. Sci. 63, 150210 (2020). https://doi.org/10.1007/s11432-019-2663-y
Received:
Revised:
Accepted:
Published:
DOI: https://doi.org/10.1007/s11432-019-2663-y