Abstract
We consider an infinite horizon discounted optimal control problem and its time discretized approximation, and study the rate of convergence of the approximate solutions to the value function of the original problem. In particular we prove the rate is of order 1 as the discretization step tends to zero, provided a semiconcavity assumption is satisfied. We also characterize the limit of the optimal controls for the approximate problems within the framework of the theory of relaxed controls.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Berkovitz LD (1974) Optimal control theory. Springer-Verlag, New York
Capuzzo Dolcetta I (1983) On a discrete approximation of the Hamilton-Jacobi equation of dynamic programming. Appl Math Optim 10:367–377
Capuzzo Dolcetta I, Evans LC (To appear) Optimal switching for ordinary differential equations. SIAM J Control Optim
Capuzzo Dolcetta I, Matzeu M (1981) On the dynamic programming inequalities associated with the optimal stopping problem in discrete and continuous time. Numer Funct Anal Op 3:425–450
Capuzzo Dolcetta I, Matzeu M (To appear) A constructive approach to the deterministic stopping time problem. Control and Cybernetics
Capuzzo Dolcetta I, Matzeu M, Menaldi JL (To appear) On a system of first order quasi-variational inequalities connected with the optimal switching problem. Systems and Control Letters
Crandall MG, Evans LC, Lions PL (To appear) Some properties of viscosity solutions of Hamilton-Jacobi equations. Trans AMS
Crandall MG, Lions PL (1983) Viscosity solutions of Hamilton-Jacobi equations. Trans AMS 277:1–42
Crandall MG, Lions PL (To appear) Two approximations of solutions of Hamilton-Jacobi equations.
Cullum J (1969) Discrete approximations to continuous optimal control problems. SIAM J Control 7:32–49
Cullum J (1971) An explicit procedure for discretizing continuous optimal control problems. J Op Theory Appl 8(1):15–34
Douglis A (1961) The continuous dependence of generalized solutions of nonlinear partial differential equations upon initial data. Comm Pure Appl math 14:267–284
Edwards RE (1965) Functional analysis, theory and applications. Holt, Rinehart and Winston, New York
Fleming WH, Rishel R (1975) Deterministic and stochastic optimal control. Springer-Verlag, New York
Henrici P (1962) Discrete variable methods in ordinary differential equations. J. Wiley, New York
Hrustalev MM (9173) Necessary and sufficient optimality conditions in the form of Bellman's equation. Soviet Math Dokl 19:1262–1266
Kruzkov SN (1975) Generalized solutions of Hamilton-Jacobi equations of eikonal type. Math USSR Sbornik 27:406–446
Lee EB, Markus L (1967) Foundations of optimal control theory. J. Wiley, New York
Lions PL (1982) Generalized solutions of Hamilton-Jacobi equations. Pitman, London
Malanowski K (1979) On convergence of finite difference approximation to optimal control problems for systems with control appearing linearly. Archiwum Automatyki i Telemachaniki 24(2):155–170
Souganidis PE (1983) PhD thesis. University of Wisconsin
Souganidis PE (To appear) Existence of viscosity solutions of Hamilton-Jacobi equations
Tartar L (1979) Compensated compactness and applications to partial differential equations. In: Knops RJ (ed) Nonlinear analysis and mechanics. Heriot-Watt Symposium, vol. 4. Pitman, London
Warga J (1972) Optimal control of differential and functional equations. Academic Press, New York
Young LC (1969) Lectures on the calculus of variations and optimal control theory. W.B. Saunders, Philadelphia
Author information
Authors and Affiliations
Additional information
Communicated by W. Fleming
This work was done while the authors were visiting members of The Department of Mathematics of The University of Maryland at College Park.
Rights and permissions
About this article
Cite this article
Dolcetta, I.C., Ishii, H. Approximate solutions of the bellman equation of deterministic control theory. Appl Math Optim 11, 161–181 (1984). https://doi.org/10.1007/BF01442176
Accepted:
Issue Date:
DOI: https://doi.org/10.1007/BF01442176