Abstract
In this paper we consider Markov Decision Processes with discounted cost and a random rate in Borel spaces. We establish the dynamic programming algorithm in finite and infinity horizon cases. We provide conditions for the existence of measurable selectors. And we show an example of consumption-investment problem.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Abbad M, Daoui C (2003) Hierarchical algorithms for discounted Markov decision processes. Math Methods Oper Res 58:237–245
Altman E (1999) Constrained Markov decision processes. Chapman & Hall/CRC, Boca Raton
Bertsekas DP (1995) Dynamic programming and optimal control. Athena Scientific, Belmont
Berument H, Kilinc Z, Ozlale U (2004) The effects of different inflation risk prepius on interest rate spreads. Physica A 333:317–324
Borkar VS (1991) Topics in controlled Markov chains. Longman, New York
Cai J, Dickson D, David CM (2004) Ruin probabilities with a Markov chain interest model. Insur Math Econ 35:513–525
Feinberg EA, Shwartz A (1994) Markov decision models with weighted discounted criteria. Math Oper Res 19:152–168
Feinberg EA, Shwartz A (1995) Constrained Markov decision models with weighted discounted rewards. Math Oper Res 20:302–320
Feinberg EA, Shwartz A (1999) Constrained dynamic programming with two discount factors: applications and an algorithm. IEEE Trans Autom Control 42:628–631
Gil A, Luis A (2004) Modelling the U.S. interest rate in terms of I(d) statistical models. Q Rev Econ Finance 44:475–486
Haberman S, Sung J (2005) Optimal pension funding dynamics over infinite control horizon when stochastic rates of return are stationary. Insur Math Econ 36:103–116
Hernández-Lerma O, González-Hernández J (2000) Constrained Markov control processes in Borel spaces: the discounted case. Math Meth Oper Res, 52:271–285
Hernández-Lerma O, Lasserre JB (1996) Discrete-time Markov control processes: basic optimality criteria. Springer, Berlin Heidelberg New York
Hinderer K (1970) Foundations of non-stationary dynamical programming with discrete time parameter Lecture notes in operation research, vol 33. Springer, Berlin Heidelberg New York
Hu Q (2003) Analysis for some properties of discrete time Markov decision processes. Optimization 52:495–505
Kurano M, Song J, Hosaka M, Huang Y (1998) Controlled Markov set-chains with discounting. J Appl Prob 35:293–302
Lee P, Rosenfield DB (2005) When to refinance a mortgage: a dynamic programming approach. Eur J Oper Res 166:266–277
Lettau M, Uhlig H (1999) Rules of thumb versus dynamic programming. Am Econ Rev 89(1):148–174
Liu K (1999) Weighted discounted Markov decision processes with perturbation. Acta Math Appl Sinica 15:183–189
López-Martínez RR, Hernández-Lerma O (2003) The Lagrange approach to constrained Markov processes: a survey and extension of results. Morfismos 7:1–26
Newell RG, Pizer WA (2003) Discounting the distant future: how much do uncertain rates increase valuation?. J Environ Econ Manage 46:52–71
Mao X, Piunovskiy AB (2000) Strategic measure in optimal control problems for stochastic sequences. Stoch Anal Appl 18:755–776
Michael K Ng (1999) A note on policy algorithms for discounted Markov decision problems. Oper Res Lett 25:195–197
Ogaki M, Santaella JA (2000) The exchange rate and the term structure of interest rates in México. J Dev Econ 63:135–155
Piunovskiy AB (1997) Optimal control of random sequences in problems with constraints. Kluwer, Boston
Puterman ML (1994) Markov decision processes: discrete stochastic dynamic programming. Wiley, New York
Sack B, Wieland V (2000) Interest-rate smoothing and optimal monetary policy: a review of recent empirical evidence. J Econ Bus 52:205–228
Shwartz A (2001) Death and discounting. IEEE Trans Autom Control 46:644–647
Stockey NL, Lucas RE Jr (1989) Recursive methods in economic dynamics. Harvard University Press, Cambridge
Author information
Authors and Affiliations
Corresponding author
Additional information
This research was partially supported by the PROMEP grant 103.5/05/40.
Rights and permissions
About this article
Cite this article
González-Hernández, J., López-Martínez, R.R. & Pérez-Hernández, J.R. Markov control processes with randomized discounted cost. Math Meth Oper Res 65, 27–44 (2007). https://doi.org/10.1007/s00186-006-0092-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00186-006-0092-2