Abstract
The optimal stopping and impulse control problems for a Markov-Feller process are considered when the controls are allowed only when a signal arrives. This is referred to as control problems with constraint. In [28, 29, 30], the HJB equation was solved and an optimal control (for the optimal stopping problem, the discounted impulse control problem and the ergodic impulse control problem, respectively) was obtained, under suitable conditions, including a setting on a compact metric state space. In this work, we extend most of the results to the situation where the state space of the Markov process is locally compact.
Access provided by Autonomous University of Puebla. Download to read the full chapter text
Chapter PDF
Similar content being viewed by others
Keywords
AMS Subject Classification
References
A. Bensoussan. Perturbation methods in optimal control. Gauthier-Villars, Montrouge, 1988.
A. Bensoussan. Dynamic programming and inventory control. IOS Press, Amsterdam, 2011.
A. Bensoussan and J.L. Lions. Applications des inéquations variationnelles en contrôle stochastique. Dunod, Paris, 1978.
A. Bensoussan and J.L. Lions. Contrôle impulsionnel et inéquations quasi-variationnelles. Gauthier-Villars, Paris, 1982.
A. Bensoussan and J.L. Menaldi. Hybrid control and dynamic programming. Dynam. Contin. Discrete Impuls. Systems, 3(4):395–442, 1997.
A. Bensoussan and J.L. Menaldi. Stochastic hybrid control. J. Math. Anal. Appl., 249(1):261–288, 2000. Special issue in honor of Richard Bellman.
P. Brémaud. Point Processes and Queues. Springer-Verlag, New York, 1981. Martingale dynamics.
P. Brémaud. Markov Chains. Springer-Verlag, New York, 1999. Gibbs fields, Monte Carlo simulation, and queues.
O.L.V. Costa, F. Dufour, and A.B. Piunovskiy. Constrained and unconstrained optimal discounted control of piecewise deterministic Markov processes. SIAM J. Control Optim., 54(3):1444–1474, 2016.
M.H.A. Davis. Markov models and optimization. Chapman & Hall, London, 1993.
P. Dupuis and H. Wang. Optimal stopping with random intervention times. Adv. in Appl. Probab., 34(1):141–157, 2002.
D. Egloff and M. Leippold. The valuation of American options with stochastic stopping time constraints. Appl. Math. Finance, 16(3-4):287–305, 2009.
S.N. Ethier and T.G. Kurtz. Markov processes. John Wiley & Sons Inc., New York, 1986. Characterization and convergence.
M.G. Garroni and J.L. Menaldi. Green functions for second order parabolic integro-differential problems. Longman Scientific & Technical, Harlow, 1992.
M.G. Garroni and J.L. Menaldi. Second order elliptic integro-differential problems. Chapman & Hall/CRC, Boca Raton, FL, 2002.
B. Hanouzet and J.L. Joly. Convergence uniforme des itérés définissant la solution d’une inéquation quasi variationnelle abstraite. C. R. Acad. Sci. Paris Sér. A-B, 286(17):A735–A738, 1978.
O. Hernández-Lerma. Adaptive Markov control processes. Springer-Verlag, New York, 1989.
O. Hernández-Lerma and J.B. Lasserre. Further topics on discrete-time Markov control processes. Springer-Verlag, New York, 1999.
A. Jaskiewicz. A fixed point approach to solve the average cost optimality equation for semi-Markov decision processes with Feller transition probabilities. Comm. Statist. Theory Meth- ods, 36(13-16):2559–2575, 2007.
H. Jasso-Fuentes, J.L. Menaldi, and M. Robin. Hybrid Control for Markov-Feller Processes. To appear, 2016.
M. Kurano. Semi-Markov decision processes and their applications in replacement models. Journal of the Operations Research, Society of Japan, 28(1):18–29, 1985.
M. Kurano. Semi-Markov decision processes with a reachable state-subset. Optimization, 20(3):305–315, 1989.
J. Lempa. Optimal stopping with information constraint. Appl. Math. Optim., 66(2):147–173, 2012.
J.P. Lepeltier and B. Marchal. Théorie générale du contrôle impulsionnel markovien. SIAM J. Control Optim., 22(4):645–665, 1984.
G. Liang. Stochastic control representations for penalized backward stochastic differential equations. SIAM J. Control Optim., 53(3):1440–1463, 2015.
G. Liang and W. Wei. Optimal switching at poisson random intervention times. Discrete and Continuous Dynamical Systems, Series B. To appear, see arXiv:1309.5608v2 [math.PR].
J.L. Menaldi. Stochastic hybrid optimal control models. In Stochastic models, II (Guanajuato, 2000), volume 16 of Aportaciones Mat. Investig., pages 205–250. Soc. Mat. Mexicana, México, 2001.
J.L. Menaldi and M. Robin. On some optimal stopping problems with constraint. SIAM J. Control Optim., 54(5):2650–2671, 2016.
J.L. Menaldi and M. Robin. On some impulse control problems with constraint. SIAM J. Control Optim., 55(5):3204–3225, 2017.
J.L. Menaldi and M. Robin. On some ergodic impulse control problems with constraint. SIAM J. Control Optim., 56(4):2690–2711, 2018.
J.L. Menaldi and M. Robin. Remarks on ergodic impulse control problems with constraint. In preparation, 2019.
S.P. Meyn and R.L. Tweedie. Stability of Markovian processes iii. Advances in Applied Probability, 25(3):518–548, 1993. Foster-Lyapunov criteria for continuous time processes.
S.P. Meyn and R.L. Tweedie. Markov chains and stochastic stability. Cambridge University Press, London, 2nd edition, 2009.
J. Palczewski and L. Stettner. Finite horizon optimal stopping of time-discontinuous functionals with applications to impulse control with delay. SIAM J. Control Optim., 48(8):4874–4909, 2010.
G. Peskir and A. Shiryaev. Optimal stopping and free-boundary problems. Birkhäuser Verlag, Basel, 2006.
M. Robin. Contrôle impulsionnel des processus de Markov. Thése d’état, 1978, 353 pp. Link at https://hal.archives-ouvertes.fr/tel-00735779/document .
L. Stettner. Discrete time adaptive impulsive control theory. Stochastic Process. Appl., 23(2):177–197, 1986.
L. Stettner. On ergodic impulsive control problems. Stochastics, 18(1):49–72, 1986.
H. Wang. Some control problems with random intervention times. Advances in Applied Probability, 33(2):404–422, 2001.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Menaldi, J.L., Robin, M. (2019). On Optimal Stopping and Impulse Control with Constraint. In: Yin, G., Zhang, Q. (eds) Modeling, Stochastic Control, Optimization, and Applications. The IMA Volumes in Mathematics and its Applications, vol 164. Springer, Cham. https://doi.org/10.1007/978-3-030-25498-8_18
Download citation
DOI: https://doi.org/10.1007/978-3-030-25498-8_18
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-25497-1
Online ISBN: 978-3-030-25498-8
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)