Abstract
This paper presents a Gaussian process based stochastic model predictive control method for linear time-invariant systems subject to bounded state-dependent additive uncertainties. Chance constraints are treated in analogy to tube-based MPC. To reduce the conservatism, the adaptive constraint tightening is performed by using the confidence region of the predicted uncertainty which is formulated based on the output of the Gaussian process model. Numerical simulations demonstrate the conservatism reducing advantage of the proposed Gaussian process based stochastic model predictive control algorithm in comparison with existing methods.
Access provided by Autonomous University of Puebla. Download conference paper PDF
Similar content being viewed by others
Keywords
1 Introduction
Model predictive control (MPC) is widely used in industry due to the ability of handling uncertainties and fulfilling constraints. However, the traditional nominal MPC may result in poor control quality on occasion in case that the serious disturbance occur, because it does not account for the uncertainty [1, 2]. The robust MPC, presuming the uncertainty is bounded, is capable to guarantee constraints satisfaction all the time by only considering the worst-case uncertainty. But it does not allow for the possible statistical properties of the uncertainty, despite the information is available in many cases. As a consequence, robust approaches may lead to overly conservative in algorithm design [3, 4]. In some real-world application cases, a certain probability of constraint violation is usually allowed. The stochastic model predictive control (SMPC) taking into account the a priori knowledge of the uncertainty and using the chance constraint will result in less conservatism in constraints satisfaction [5,6,7,8].
Majority of existing SMPC algorithms ensure closed loop constraint satisfaction typically rely on knowledge of worst case bounds corresponding to prescribed chance constraints. Although the offline computation of constraint tightening releases the computational complex, it causes intrinsic conservatism for ignoring past constraint violations and current uncertainty. To cope with this issue, several approaches have been proposed. In [9], the authors develop a recursively feasible MPC scheme by explicitly taking into account the past constraint violations to adaptively scale the tightening parameters. The work [10] exploits the observed constraint violations to adaptively scale the tightening parameters to eliminate the conservatism, and analyze the convergence of the amount of constraint violations rigorously using stochastic approximation. For linear systems under multiplicative and possibly unbounded model uncertainty, the work [11] presents a stochastic model predictive control algorithm. In which, the probabilistic constraints are reformulated in deterministic terms by means of the Cantelli inequality. A recursively feasible stochastic model predictive control scheme is designed by explicitly taking into account the past averaged-over-time amount of constraint violations when determining the current control input [12].
Another way to reduce the conservatism of MPC scheme is using the statistical machine learning methods to model the uncertainties based on prior knowledge [13,14,15]. Gaussian process (GP) regression is particularly attractive because it provides variance besides the mean of uncertainty, which can be incorporated into MPC to improve the performance [16,17,18,19].
In this paper, we propose a Gaussian process based SMPC (GP-SMPC) scheme for linear time-invariant (LTI) systems subject to bounded additive uncertainties. The uncertainties are state-dependent and bounded. The GP models for the uncertainties are trained offline on the base of previous collected data. The future mean and variance of the uncertainty can be predicted by the learned GP model on the condition of current state and input. The key contribution of this work is that the predicted information of uncertainty is used to adaptively scale the tightening parameters of the system constraints to achieve less conservatism.
he remainder of this paper is organized as follows. The time-varying tube-based SMPC is introduced in Sect. 2. Section 3 proposes the GP-SMPC scheme which mainly consists of the uncertainty modeling and constraint tightening. In Sect. 4, numerical simulations are given. Section 5 concludes the paper.
Notations
\({x}_{k|t}\) represents the \(k\)-step-ahead prediction of \(x\) at time \(t\).
\({\mathbb{R}}\) denotes the set of reals, \({\mathbb{N}}_{i}\) denotes the set of integers which equal or greater than \(i\),\({\mathbb{N}}_{i}^{j}\) denotes the set of consecutive integers \(\left\{i,\cdots ,j\right\}\).
\(\mathrm{Pr}(X)\) stands for the probability of an event \(X\).
The Minkowski sum is denoted by \(A\oplus B=\left\{a+b|a\in A, b\in B\right\}\).
The Pontryagin set difference is represented by \(A\ominus B=\left\{a\in A|a+b\in A, \forall b\in B\right\}\).
2 Time-Varying Tube-Based SMPC
Consider a discrete LTI system subject to additive uncertainties
where \({x}_{k}\in {\mathbb{R}}^{\mathrm{n}}\) and \({u}_{k}\in {\mathbb{R}}^{\mathrm{m}}\) are the state and input at time \(k\) respectively. The uncertainties \({w}_{k}\in {\mathbb{W}}\subset {\mathbb{R}}^{\mathrm{n}}\), which can be unmodeled nonlinearities and/or external disturbances, are bounded and state-dependent. Moreover, system (1) is subjected to the following constraints on states and inputs
The nominal system neglecting the uncertainty part is defined as
where the nominal state \({s}_{k}\in {\mathbb{R}}^{\mathrm{n}}\) and the nominal open loop input \({v}_{k}\in {\mathbb{R}}^{\mathrm{m}}\).
The error between observed state \({x}_{k}\) and nominal state \({s}_{k}\) is defined as
One of the commonly used control policies in robust tube MPC is
where the feedback gain \(K\) is obtained by LQR optimization for the nominal dynamics (4), such that \({A}_{cl}=A+BK\) is Schur stable.
Then the system dynamics in (1) can be decoupled into a nominal dynamics and an error dynamics as
The error dynamics (6b) will be used for constraint tightening.
Suppose that a polytope \(\mathcal{E}\subset {\mathbb{W}}\) is a confidence region of probability level \(1-\upepsilon\) for the uncertainty, that is.
where \(\upepsilon \in \left(0, 1\right)\).
Since the error dynamics in (6b) is linear, and the uncertainty \(w\in {\mathbb{W}}\), the propagation set of uncertainty \({e}_{k}\in {\mathcal{W}}_{\mathrm{k}}\) is evolved as
where \({\mathcal{W}}_{0}={\mathbb{W}}\). Then, it can be induced that \({\mathcal{W}}_{k}=\sum_{i=0}^{k}\oplus {A}_{cl}^{i}{\mathbb{W}}\), \(k\in {\mathbb{N}}_{0}\).
Construct the tightened propagation set of uncertainty as
Then
follows from (7), where \({\mathcal{D}}_{0}=\mathcal{E}\).
Define the time-varying tightened state constraint set as
If \({s}_{k}\in {\mathcal{C}}_{k}\), then \(\mathrm{Pr}\left({x}_{k}={s}_{k}+{e}_{k}\in {\mathbb{X}}\right)\ge 1-\upepsilon\) is satisfied, that is, the satisfaction of chance constraint (2a) is guaranteed by (10) and (11).
Define the tightened input constraint set
where \(\mathcal{Z}=\sum_{i=0}^{\infty }\oplus {A}_{cl}^{i}{\mathbb{W}}\) and \({e}_{k}\in \mathcal{Z}, k\in {\mathbb{N}}_{0}\). If \({v}_{k}\in \mathcal{V}\), then the hard constraint (2b) \({u}_{k}={v}_{k}+K{e}_{k}\in {\mathbb{U}}\) is guaranteed by (12).
Define terminal constraint set
The finite horizon optimal control problem to be solved at each time instant \(t\) is as follows:
3 SMPC Using Gaussian Process Regression
Since the uncertainty is state-dependent, it will be conservative if the confidence region \(\mathcal{E}\) formulated based on its maximum amplitude. In this section, a Gaussian process regression method is proposed to solve this issue.
3.1 Gaussian Process Regression
Considering a training set \(\left\{\left({x}_{i},{y}_{i}\right),i=\mathrm{1,2},\cdots ,M\right\}\), where \({x}_{i}\in {\mathbb{R}}^{d}\) and \({y}_{i}\in {\mathbb{R}}\). The GPR model learns a function \(f\left(x\right)\) mapping the input vector \(\mathrm{x}\) to the observed output value \(y\) given by \(y=f\left(x\right)+w\), where \(w\sim \mathcal{N}\left(0, {\sigma }_{n}^{2}\right)\). The observed output values are normally distributed \({\varvec{y}}\sim \mathcal{N}\left(\mu \left(X\right), K(X, X)\right)\), where the mean value vector \(\mu \left(X\right)={\left[\mu \left({x}_{1}\right),\cdots ,\mu \left({x}_{M}\right)\right]}^{T}\) and the covariance matrix
and \(c\left({x}_{i},{x}_{j}\right)\) is the covariance of \({x}_{i}\) \(\mathrm{and}\) \({x}_{j}\), which can be any positive definite function. A frequently used covariance function called Square Exponential Kernel function is defined as
where \(L=\mathrm{diag}\left(\left[{l}_{1},\cdots {l}_{d}\right]\right)\), \({\sigma }_{f}\) and \({\sigma }_{n}\) are the hyperparameters of the covariance function.
The training output \({\varvec{y}}\) and a predicted output \({y}^{*}\) corresponding to the test input \({x}^{*}\) are jointly Gaussian distribution
where \(k\left({x}^{*}, X\right)=\left[c\left({x}^{*},{x}_{1}\right), c\left({x}^{*},{x}_{2}\right),\cdots ,c\left({x}^{*},{x}_{M}\right)\right]\), \(k\left({x}^{*}, {x}^{*}\right)=c\left({x}^{*}, {x}^{*}\right)\).
Following the Bayesian modeling framework, the posterior distribution of \({y}^{*}\) can be obtained conditioned on the observations, and the resulting is still Gaussian with \({y}^{*}|{\varvec{y}}\sim \mathcal{N}\left(\mu \left({x}^{*}\right), {\sigma }^{2} \left({x}^{*}\right)\right)\) and
3.2 GP Model of Uncertainty
The learned GPR model depends on measurement data collected from previous experience. The model input and output are the state-control tuple \({z}_{k}=\left({x}_{k}; {u}_{k}\right)\) and corresponding uncertainty \({w}_{k}\), respectively. The uncertainty at time \(\mathrm{k}\).
The data pair \(\left({z}_{k},{w}_{k}\right)\) represents an individual experience. Given a well collected data pair set \(\mathfrak{D}=\left\{{\varvec{z}},{\varvec{w}}\right\}\) and a test data pair \(\left({z}^{*},{w}^{*}\right)\), the jointly Gaussian distribution is
The posterior distribution of \({w}^{*}\) is still Gaussian
with mean and variance as follows
The \(\mathrm{n}\) separate GP models are trained for each dimension in \(w\in {\mathbb{R}}^{n}\). We gain the optimal hyperparameters of each Gaussian model offline by maximizing the log marginal likelihood of collected data sets [20].
3.3 Adaptive Constraints
Define the prediction model as
where \({\tilde{x }}_{k}\) denotes the predicted state, \({\tilde{u }}_{k}\) the predicted input, and \({\tilde{w }}_{k}\) the predicted uncertainty. On the condition of trained GP models, the distribution of \({\tilde{w }}_{k}\) corresponding to \(\left({\tilde{x }}_{k}; {\tilde{u }}_{k}\right)\) can be obtained as
where \({\stackrel{\sim }{\mu }}_{k}\) and \({\stackrel{\sim }{\sigma }}_{k}^{2}\) are computed by (22a) and (22b).
Define the confidence region of the predicted uncertainty with the probability level \(1-\upepsilon\) as
where \(\mathrm{\alpha }\) is the quantile value corresponding to \(1-\upepsilon\).
According to (9), the more stringent propagation set of uncertainty is
Then
follows.
Construct the adaptively time-varying state constraint set as
If \({s}_{k}\in {\stackrel{\sim }{\mathcal{C}}}_{k}\), then the chance constraint \(\mathrm{Pr}\left({x}_{k}={s}_{k}+{e}_{k}\in {\mathbb{X}}\right)\ge 1-\upepsilon\) is satisfied.
Define the tightened input constraint set \(\mathcal{V}={\mathbb{U}}\ominus \mathrm{K}\mathcal{Z}\) as (12). If \({v}_{k}\in \mathcal{V}\), then the satisfaction of hard constraint \({u}_{k}={v}_{k}+K{e}_{k}\in {\mathbb{U}}\) is guaranteed.
3.4 Gaussian Process Based SMPC
On the basis of the time-varying tube-based SMPC, by combining the GP-based uncertainty prediction, the Gaussian process based stochastic optimal control problem to be solved at each time instant \(t\) is as follows:
The solution of the optimal control problem yields the optimal initial nominal state \({s}_{0|t}^{*}\) and input sequence.
The associate optimal state sequence for nominal system is
Using the first entry of the optimal input sequence and the optimal initial state, the optimal control law is designed as
Apply \({u}^{*}\left({x}_{t}\right)\) to the system (1) yields new state.
Based on the new state \({x}_{t+1}\), the entire process of GP based SMPC is repeated at time \(t+1\), yielding a receding horizon control strategy.
4 Numerical Simulation
In this section, the chance constraint satisfaction of the GP-SMPC scheme are compared with nominal MPC, robust MPC and time-varying tube-based SMPC. In the simulations, the polytopes \(\mathcal{C}\), \(\mathcal{V}\), \(\mathcal{D}\) and \(\mathcal{Z}\) are computed by using the MPT3 toolbox.
To show the constraint violation of the GP-SMPC scheme, a discrete LTI system subject to state-dependent additive uncertainty disturbed by a truncated normal distributed noise is designed as
The state and input constraints are \(\mathrm{Pr}\left({x}_{k}\in {\mathbb{X}}\right)\ge 0.8\) and \({u}_{k}\in {\mathbb{U}}\), respectively.
The uncertainty \({w}_{k}\in {\mathbb{W}}\) and
Design the weights of cost function \(Q={I}_{2}\) and \(R=1\). Compute \(K\) as the LQR feedback gain for the unconstrained optimal problem \(\left(A,B,Q,R\right)\). The prediction horizon is \(N=6\). The simulation step is \(N\mathrm{sim}=11\). The initial state \({x}_{0}={\left[-6.5, 10.5\right]}^{\mathrm{T}}\).
The state constraint violations of the nominal MPC, the robust MPC, the time-varying tube-based SMPC and the proposed GP-SMPC are illustrated in Figs. 1, 2, 3 and 4. In the up side of each figure, the closed-loop actual state trajectories of 100 realizations are demonstrated. On account of that the constraint violation occurs around the border at the first several steps, the details of the first 3 step trajectories is enlarged at the down side part of each figure. Table 1 presents the first three steps constraint violation ratios and the average ratios of 1000 realizations. From the figures and the table, it can be seen that: the average constraint violation at the first 3 steps of the nominal MPC is 100%, the constraint is break at all steps; the average constraint violation at the first 3 steps of the robust MPC is 0%, the constraint is satisfied with heavy-duty conservatism; the average constraint violation at the first 3 steps of the time-varying tube-based SMPC is 65.0%, the conservatism is relieved a bit; and that of the proposed GP-SMPC is 16.2%, which is close to 20% specified in advance, resulting in less conservatism and constraint satisfaction.
5 Conclusion
The proposed GP-SMPC scheme reduces the conservatism through tightening the constraints adaptively. Specifically, the stringent propagation set of uncertainty is obtained by using the time varying confidence region which is formulated on the basis of Gaussian process prediction. Numerical simulations validate that the chance constraint satisfaction of GP-SMPC is better than that of nominal MPC, robust MPC and time-varying tube-based SMPC.
References
Mayne, D.Q.: Model predictive control: recent developments and future promise. Automatica 50(12), 2967–2986 (2014)
Li, H., Yan, W., Shi, Y.: Triggering and control codesign in self-triggered model predictive control of constrained systems: with guaranteed performance. IEEE Trans. Autom. Control Autom. Control 63(11), 4008–4015 (2018)
Kouvaritakis, B., Cannon, M.: Model Predictive Control: Classical, Robust and Stochastic, 1st edn. Springer, Switzerland (2016)
Li, H., Shi, Y.: Robust receding horizon control for networked and distributed nonlinear systems. 1st edn. Springer (2017)
Mesbah, A.: Stochastic model predictive control: an overview and perspectives for future research. IEEE Control. Syst. 36(6), 30–44 (2016)
Lorenzen, M., Dabbene, F., Tempo, R., Allgöwer, F.: Stochastic MPC with offline uncertainty sampling. Automatica 81(1), 176–183 (2017)
Liu, X., Feng, L., Kong, X., Guo, S., Lee, K.Y.: Tube-based stochastic model predictive control with application to wind energy conversion system. IEEE Trans. Control Syst. Technol. https://doi.org/10.1109/TCST.2023.3291531(2023)
Arcari, E., Iannelli, A., Carron, A., Zeilinger, M.N.: Stochastic MPC with robustness to bounded parametric uncertainty. IEEE Trans. Autom. Control. https://doi.org/10.1109/TAC.2023.3294868(2023)
Kouvaritakis, B., Cannon, M., Raković, S., Cheng, Q.: Explicit use of probabilistic distributions in linear predictive control. Automatica 46(10), 1719–1724 (2010)
Munoz-Carpintero, D., Hu, G., Spanos, C.: Stochastic model predictive control with adaptive constraint tightening for non-conservative chance constraints satisfaction. Automatica 96, 32–39 (2018)
Farina, M., Scattolini, R.: Model predictive control of linear systems with multiplicative unbounded uncertainty and chance constraints. Automatica 70, 258–265 (2016)
Korda, M., Gondhalekar, R., Oldewurtel, F., Jones, C.N.: Stochastic MPC framework for controlling the average constraint violation. IEEE Trans. Autom. ControlAutom. Control 59(7), 1706–1721 (2014)
Li, F., Li, H., Li, S., He, Y.: Online learning stochastic model predictive control of linear uncertain systems. Int. J. Robust Nonlinear Control 32(17), 9275–9293 (2022)
Aswani, A., González, H., Sastry, S.S., Tomlin, C.: Provably safe and robust learning-based model predictive control. Automatica 49(5), 1216–1226 (2013)
Rosolia, U., Zhang, X., Borrelli, F.: A stochastic MPC approach with application to iterative learning. In: Proceedings of the 57th IEEE Conference on Decision and Control, pp. 5152–5157, Miami Beach, USA (2018)
Cao, G., Lai, E., Alam, F.: Gaussian process model predictive control of unmanned quadrotors. In: International Conference on Control, Automation and Robotics, pp. 200–206, Hong Kong, China (2016)
Li, F., Li, H., He, Y.: Adaptive stochastic model predictive control of linear systems using Gaussian process regression. IET Control Theory Appl. 15(5), 683–693 (2021)
Ostafew, C.J., Schoellig, A.P., Barfoot, T.D.: Robust constrained learning-based NMPC enabling reliable mobile robot path tracking. Int. J. Robot. Res. 35(13), 1547–1563 (2016)
Yang, X., Maciejowski, J.M.: Fault tolerant control using Gaussian processes and model predictive control. Int. J. Appl. Math. Comput. Sci.Comput. Sci. 25(1), 133–148 (2015)
Wang, Y., Ocampo-Martinez, C., Puig, V.: Stochastic model predictive control based on Gaussian processes applied to drinking water networks. IET Control Theory Appl. 10(8), 947–955 (2016)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Li, F., Song, L., Duan, X., Wu, C., Ji, X. (2024). Gaussian Process Based Stochastic Model Predictive Control of Linear System with Bounded Additive Uncertainty. In: Sun, F., Meng, Q., Fu, Z., Fang, B. (eds) Cognitive Systems and Information Processing. ICCSIP 2023. Communications in Computer and Information Science, vol 1918. Springer, Singapore. https://doi.org/10.1007/978-981-99-8018-5_20
Download citation
DOI: https://doi.org/10.1007/978-981-99-8018-5_20
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-8017-8
Online ISBN: 978-981-99-8018-5
eBook Packages: Computer ScienceComputer Science (R0)