Abstract
A review of characteristics-based approaches to optimal control computation is presented for nonlinear deterministic systems described by ordinary differential equations. We recall Pontryagin’s principle which gives necessary optimality conditions for open-loop control strategies. The related framework of generalized characteristics for first-order Hamilton–Jacobi–Bellman equations is discussed as a theoretical tool that bridges the gap between the necessary and sufficient optimality conditions. We point out widely used numerical techniques for obtaining optimal open-loop control strategies (in particular for state-constrained problems) and how indirect (characteristics based) and direct methods may reasonably be combined. A possible transition from open-loop to feedback constructions is also described. Moreover, we discuss approaches for attenuating the curse of dimensionality for certain classes of Hamilton–Jacobi–Bellman equations.
Similar content being viewed by others
Bibliography
Afanas’ev VN, Kolmanovskii VB, Nosov VR (1996) Mathematical theory of control systems design. Springer Science + Business Media, Dordrecht
Bardi M, Capuzzo-Dolcetta I (2008) Optimal control and viscosity solutions of Hamilton–Jacobi–Bellman equations. Birkhäuser, Boston
Basco V, Cannarsa P, Frankowska H (2018) Necessary conditions for infinite horizon optimal control problems with state constraints. Math Control Relat F 8(3&4):535–555
Bellman R (1957) Dynamic programming. Princeton University Press, Princeton
Benson DA, Huntington GT, Thorvaldsen TP, Rao AV (2006) Direct trajectory optimization and costate estimation via an orthogonal collocation method. J Guid Control Dyn 29 (6): 1435–1440
Bernhard P (1977) Singular surfaces in differential games: an introduction. In: Hagedorn P, Knobloch HW, Olsder GJ (eds) Differential games and applications. Lecture notes in control and information sciences, vol 3, pp 1–33. Springer, Berlin, Heidelberg
Bokanowski O, Desilles A, Zidani H, Zhao J (2017) User’s guide for the ROC-HJ solver: reachability, optimal control, and Hamilton–Jacobi equations, Version 2.3. http://uma.ensta-paristech.fr/soft/ROC-HJ/
Bonnard B, Faubourg L, Launay G, Trélat E (2003) Optimal control with state constraints and the space shuttle re-entry problem. J Dyn Control Syst 9 (2): 155–199
Cesari L (1983) Optimization—theory and applications, problems with ordinary differential equations. Applications of mathematics, vol 17. Springer, New York
Chow YT, Darbon J, Osher S, Yin W (2017) Algorithm for overcoming the curse of dimensionality for time-dependent non-convex Hamilton–Jacobi equations arising from optimal control and differential games problems. J Sci Comput 73 (2–3): 617–643
Clarke FH, Ledyaev YS, Stern RJ, Wolenski PR (1998) Nonsmooth analysis and control theory. Springer, New York
Cristiani E, Martinon P (2010) Initialization of the shooting method via the Hamilton–Jacobi–Bellman approach. J Optim Theory Appl 146 (2): 321–346
Darbon J, Osher S (2016) Algorithms for overcoming the curse of dimensionality for certain Hamilton–Jacobi equations arising in control theory and elsewhere. Res Math Sci 3: 19
Fleming WH, Soner HM (2006) Controlled Markov processes and viscosity solutions. Springer, New York
Grüne L (2014) Numerical methods for nonlinear optimal control problems. In: Baillieul J, Samad T (eds) Encyclopedia of systems and control. Springer, London
Hartl RF, Sethi SP, Vickson RG (1995) A survey of the maximum principles for optimal control problems with state constraints. SIAM Rev 37 (2): 181–218
Huntington GT, Benson DA, Rao AV (2007) Optimal configuration of tetrahedral spacecraft formations. J Aeronaut Sci 55 (2): 141–169
Jensen M, Smears I (2013) On the convergence of finite element methods for Hamilton–Jacobi–Bellman equations. SIAM J Numer Anal 51 (1): 137–162
Kang W, Wilcox LC (2017) Mitigating the curse of dimensionality: sparse grid characteristics method for optimal feedback control and HJB equations. Comput Optim Appl 68 (2): 289–315
Kushner HJ, Dupuis P (2001) Numerical methods for stochastic control problems in continuous time. Springer, New York
Maurer H (1977) On optimal control problems with bounded state variables and control appearing linearly. SIAM J Control Optim 15 (3): 345–362
McEneaney WM (2006) Max-plus methods in nonlinear control and estimation. Birkhäuser, Boston
McEneaney WM, Kluberg J (2009) Convergence rate for a curse-of-dimensionality-free method for a class of HJB PDEs. SIAM J Control Optim 48 (5): 3052–3079
Melikyan AA (1998) Generalized characteristics of first order PDEs: application in optimal control and differential games. Birkhäuser, Boston
Michalska H, Mayne DQ (1993) Robust receding horizon control of constrained nonlinear systems. IEEE Trans Automat Contr 38 (11): 1623–1633
Osher S, Fedkiw R (2003) Level set methods and dynamic implicit surfaces. Springer, New York
Pesch HJ (1994) A practical guide to the solution of real-life optimal control problems. Control Cybern 23 (1–2): 7–60
Pontryagin LS, Boltyansky VG, Gamkrelidze RV, Mishchenko EF (1964) The mathematical theory of optimal processes. Macmillan, New York
Rao AV (2010) A survey of numerical methods for optimal control. Adv Astron Sci 135 (1): 497–528
Schättler H (2014) Optimal control with state space constraints. In: Baillieul J, Samad T (eds) Encyclopedia of systems and control. Springer, London
Schättler H, Ledzewicz U (2015) Optimal control for mathematical models of cancer therapies: an application of geometric methods. Interdisciplinary applied mathematics, vol 42. Springer, New York
Subbotin AI (1995) Generalized solutions of first-order PDEs: the dynamical optimization perspective. Birkhäuser, Boston
Subbotina NN (2006) The method of characteristics for Hamilton–Jacobi equations and applications to dynamical optimization. J Math Sci 135 (3): 2955–3091
Trélat E (2012) Optimal control and applications to aerospace: some results and challenges. J Optim Theory Appl 154 (3): 713–758
Wang L (2009) Model predictive control system design and implementation using MATLAB. Springer, London
Yegorov I, Dower P (2018) Perspectives on characteristics based curse-of-dimensionality-free numerical approaches for solving Hamilton–Jacobi equations. Appl Math Opt. To appear. https://doi.org/10.1007/s00245-018-9509-6
Yong J, Zhou XY (1999) Stochastic controls: Hamiltonian systems and HJB equations. Springer, New York
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Section Editor information
Rights and permissions
Copyright information
© 2020 Springer-Verlag London Ltd., part of Springer Nature
About this entry
Cite this entry
Yegorov, I., Dower, P.M. (2020). Characteristics in Optimal Control Computation. In: Baillieul, J., Samad, T. (eds) Encyclopedia of Systems and Control. Springer, London. https://doi.org/10.1007/978-1-4471-5102-9_100056-1
Download citation
DOI: https://doi.org/10.1007/978-1-4471-5102-9_100056-1
Published:
Publisher Name: Springer, London
Print ISBN: 978-1-4471-5102-9
Online ISBN: 978-1-4471-5102-9
eBook Packages: Springer Reference EngineeringReference Module Computer Science and Engineering