Abstract
We consider a controlled stochastic system in presence of state-constraints. Under the assumption of exponential stabilizability of the system near a target set, we aim to characterize the set of points which can be asymptotically driven by an admissible control to the target with positive probability. We show that this set can be characterized as a level set of the optimal value function of a suitable unconstrained optimal control problem which in turn is the unique viscosity solution of a second order PDE which can thus be interpreted as a generalized Zubov equation.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Abu Hassan M., Storey C.: Numerical determination of domains of attraction for electrical power systems using the method of Zubov. Int. J. Control 34, 371–381 (1981)
Artstein Z.: Stabilization with relaxed controls. Nonlinear Anal. 7, 1163–1173 (1983)
Aulbach, B.: Asymptotic stability regions via extensions of Zubov’s method. I and II. Nonlinear Anal. Theory Methods Appl. 7, 1431–1440 and 1441–1454 (1983)
Bardi, M., Capuzzo Dolcetta, I.: Optimal Control and Viscosity Solutions of Hamilton-Jacobi-Bellman equations. In: Systems Control Found. Appl. Birkhäuser, Boston (1997)
Barles G., Burdeau J.: The Dirichlet problem for semilinear second-order degenerate elliptic equations and applications to stochastic exit time control problems. Commun. Partial Differ. Equ. 20(1-2), 129–178 (1995)
Barles G., Daher C., Romano M.: Optimal control on the \({l^\infty}\) norm of a diffusion process. SIAM J. Control Optim. 32(3), 612–634 (1994)
Bokanowski, O., Picarelli, A., Zidani, H.: Dynamic Programming and Error Estimates for Stochastic Control Problems with Maximum Cost. Appl. Math. Optim. 71(1), 125–163 (2015)
Bouchard B., Touzi N.: Weak dynamic programming principle for viscosity solutions. SIAM J. Control Optim. 49(3), 948–962 (2011)
Camilli F., Cesaroni A., Grüne L., Wirth F.: Stabilization of controlled diffusions via Zubov’s method. Stochast. Dyn. 6, 373–394 (2006)
Camilli F., Grüne L.: Characterizing attraction probabilities via the stochastic zubov method. Discrete Contin. Dyn. Syst. B 3, 457–468 (2003)
Camilli F., Grüne L., Wirth F.: A generalization of the Zubov’s equation to perturbed systems. SIAM J. Control Optim. 40, 496–515 (2002)
Camilli, F., Grüne, L., Wirth, F.: Characterizing controllability probabilities of stochastic control systems via Zubov’s method. In: Proceedings of the 16th International Symposium on Mathematical Theory of Networks and Systems (MTNS2004) (2004)
Camilli, F., Grüne, L., Wirth, F.: Control Lyapunov Functions and Zubov’s Method. SIAM J. Control Optim. 47, 301–326 (2008)
Camilli F., Loreti P.: A characterization of the domain of attraction for a locally exponentially stable stochastic system. NoDea 13, 205–222 (2006)
Cesaroni A.: Lyapunov stabilizability of controlled diffusions via a superoptimality principle for viscosity solutions. Appl. Math. Optim. 53(1), 1–29 (2006)
Crandall M.G., Ishii H., Lions P.L.: User’s guide to viscosity solutions of second order partial differential equations. Bull. Am. Math. Soc. 27(1), 1–67 (1992)
Dupuis P., Ishii H.: On oblique derivative problems for fully nonlinear second-order elliptic partial differential equations on nonsmooth domains. Nonlinear Anal. Theory Methods Appl. 15(12), 1123–1138 (1990)
Genesio R., Tartaglia M., Vicino A.: On the estimation of asymptotic stability regions: state of the art and new proposals. IEEE Trans. Autom. Control 30, 747–755 (1985)
Grüne, L., Zidani, H.: Zubov’s equation for state-constrained perturbed nonlinear systems. Math. Control Relat Fields 5(1), 55–71 (2014)
Hahn, W.: Stability of Motion (Translated from the German manuscript “Die Grundlehren der mathematischen Wissenschaften” Band 138, by Arne P. Baartz). Springer, New York (1967)
Hasminskii, R.Z.: Stochastic Stability of Differential equations. Sjithoff and Noordhoff International Publishers, Leyden (1980)
Haussmann U.G., Lepeltier J.P.: On the existence of optimal controls. SIAM J. Control Optim. 28(4), 851–902 (1990)
Ikeda, N., Watanabe, S.: Stochastic Differential Equations and Diffusion Processes, 1st edn. North Holland Publishing Company, Kodansha LTD, Amsterdam, Tokyo (1981)
Kirin N.E., Nelepin R.A., Bajdaev V.N.: Construction of the attraction region by Zubov’s method. Differ. Equ. 17, 871–880 (1982)
Kurzweil, Y.: On the inversion of the second theorem of Lyapunov on stability of motion. Czechoslovak Math. J. 81(6), 217–259, 455–473 (1956)
Kushner H.J.: Converse theorems for stochastic Lyapunov functions. SIAM J. Control Optim. 5, 228–233 (1967)
Kushner, H.J.: Stochastic stability and control. Academic Press, New York (1967)
Kushner, H.J.: Stochastic stability. In: Stability of stochastic dynamical systems (Proc. In- ternat. Sympos., Univ. Warwick, Coventry, 1972), Lecture Notes in Math., vol. 294. Springer, Berlin (1972)
Lyapunov, A.M.: The general problem of the stability of motion (English translation), Int. J Control. 55(3), 531–534 (1992)
Massera J.L.: On Lyapunov’s condition of stability. Ann. Math. 50, 705–721 (1949)
Sontag E.: A Lyapunov-like characterization of asymptotic controllability. SIAM J. Control Opt. 21, 462–471 (1983)
Touzi, N.: Optimal stochastic control, stochastic target problems, and backward SDE. In: Fields Institute Monographs, vol. 49. Springer, Berlin (2012)
Zubov, V.I.: Methods of A.M. Lyapunov and their Application. P. Noordhoff, Groningen (1964)
Author information
Authors and Affiliations
Corresponding author
Additional information
This work was partially supported by the EU under the 7th Framework Programme Marie Curie Initial Training Network “FP7-PEOPLE-2010-ITN”, SADCO project, GA number 264735-SADCO. Parts of the research for this paper were carried out while the second author was visiting the University of Bayreuth as part of her SADCO secondment.
Rights and permissions
About this article
Cite this article
Grüne, L., Picarelli, A. Zubov’s method for controlled diffusions with state constraints. Nonlinear Differ. Equ. Appl. 22, 1765–1799 (2015). https://doi.org/10.1007/s00030-015-0343-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00030-015-0343-0