Abstract
Risk-sensitive control is an area of significant current interest in stochastic control theory. It is a generalization of the classical, risk-neutral approach, whereby we seek to minimize an exponential of the sum of costs that depends not only on the expected cost, but on higher order moments as well.
Research supported in part by the National Science Foundation under grant EEC 9402384.
Research supported in part by a grant from the University of Arizona Foundation and the Office of the Vice President for Research; and in part by the National Science Foundation under grant NSF-INT 9201430.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Arapostathis, V.S. Borkar, E. Fernández-Gaucherand, M.K. Ghosh and S.I. Marcus. Discrete-time controlled Markov processes with average cost criterion: a survey. SIAM J. Control and Optim. 31 (1993), 282–344.
J.S. Baras and M.R. James. Robust and risk-sensitive output feedback control for finite state machines and hidden Markov models. To appear J. Math. Systems, Estimation and Control.
A.Bensoussan and J.H. Van Schuppen. Optimal control of partially observable stochastic systems with exponential of integral performance index. SIAM J. Control and Optim. 23 (1985), 599–613.
D.P. Bertsekas. Dynamic Programming: Deterministic and Stochastic Model. Englewood Cliffs, NJ: Prentice-Hall, 1987.
V.S. Borkar. On minimum cost per unit of time control of Markov chains. SIAM J. Cont. and Optim. 22 (1984), 965–978.
R. Cavazos-Cadena. Weak conditions for the existence of optimal stationary policies in average Markov decision chains with unbounded costs. Kybernetika 25 (1989), 145–156.
R. Cavazos-Cadena and L.I. Sennott. Comparing recent assumptions for the existence of average optimal stationary policies. Oper. Res. Lett. 11 (1992), 33–37.
K.-J. Chung and M.J. Sobel. Discounted MDP’s: Distribution functions and exponential utility maximization. SI AM J. Control and Optim. 25 (1987), 49–62.
S. Coraluppi and S.I. Marcus. Risk-sensitive control of Markov decision processes. Proc. 1996 Conf. on Information Science and Systems. 934–939.
P. Dupuis and R.S. Ellis. A Weak Convergence Approach to the Theory of Large Deviations. To be published by John Wiley & Sons.
J.N. Eagle, II. A Utility Criterion for the Markov Decision Process. Ph.D. Dissertation. Stanford University. Stanford, Caliornia. 1975.
C.-H. Fan, J.L. Speyer and C.R. Jaensch. Centralized and decentralized solutions fo the linear-exponential-Gaussian problem. IEEE Transactions on Automatic Control 39 (1994), 1986–2003.
E. Fernández-Gaucherand, A. Arapostathis, and S.I. Marcus. On the average cost optimality equation and the structure of optimal Policies for partially observable Markov decision processes. Annals of Operations Research 29 (1991), 439–470.
E. Fernández-Gaucherand, A. Arapostathis and S.I. Marcus. Analysis of an adaptive control scheme for a partially observed controlled Markov chain. IEEE Transactions on Automatic Control 38 (1993), 987–993.
E. Fernández-Gaucherand and S.I. Marcus. Risk-sensitive optimal control of hidden Markov models: structural results. To appear IEEE Transactions on Automatic Control.
W.H. Fleming and D. Hernández-Hernández. Risk sensitive control of finite state machines on an infinite horizon I. To appear SIAM J. Control and Optim.
W.H. Fleming and D. Hernández-Hernández. Risk sensitive control of finite state machines on an infinite horizon II. Technical Report. Division of Applied Mathematics. Brown University.
W.H. Fleming and W.M. McEneaney. Risk-sensitive control and differential games. Springer Lecture Notes in Control and Info. Sci. 184 (1992), 185–197.
W.H. Fleming and W.M. McEneaney. Risk-sensitive control on an infinite horizon. SIAM J. Control and Optim. 33 (1995), 1881–1915.
K. Glover and J.C. Doyle, State-space formulae for all stabilizing controllers that satisfy an H∞-norm bound and relations to risk sensitivit. Systems and Control Lett. 11 (1988), 167–172.
L.P. Hansen and T.J. Sargent. Discounted linear exponential quadratic Gaussian control. IEEE Transactions on Automatic Control 40 (1995), 968–971.
D. Hernández-Hernández and S.I. Marcus. Risk-sensitive control of Markov processes in countable state space. To appear Systems and Control Lett.
D. Hernández-Hernández and S.I. Marcus. Existence of risk sensitive optimal stationary policies for controlled Markov processes. Technical Report. Institute for Systems Research. University of Maryland.
D. Hernández-Hernández, S.I. Marcus, and P. Fard. Analysis of a risk sensitive control problem for hidden Markov chains. Technical Report. Institute for Systems Research. University of Maryland.
R.A. Howard and J.E. Matheson. Risk-sensitive Markov decision processes. Management Sci. 18 (1972), 356–369.
D.H. Jacobson. Optimal stochastic linear systems with exponential performance criteria and their relation to deterministic differential games. IEEE Transactions on Automatic Control 18 (1973), 124–131.
M.R. James, J.S. Baras and R.J. Elliott. Risk-sensitive control and dynamic games for partially observed discrete-time nonlinear systems. IEEE Transactions on Automatic Control 39 (1994), 780–792.
D.M. Kreps and E.L. Porteus. Temporal resolution of uncertainty and dynamic choice theory. Econometrica 46 (1978), 185–200.
P.R. Kumar and P. Varaiya. Stochastic Systems: Estimation, Identification and Adaptive Control. Englewood Cliffs, NJ: Prentice-Hall, 1986.
W.S. Lovejoy. On the convexity of policy regions in partially observed systems. Operations Research 35 (1987), 619–621.
T. Runolfsson. The equivalence between infinite horizon control of stochastic systems with exponential-of-integral performance index and stochastic differential games. IEEE Transactions on Automatic Control 39 (1994), 1551–1563.
R.D Smallwood and E.J. Sondik. The optimal control of partially observable Markov processes over a finite horizon. Operations Research 21 (1973), 1071–1088.
C.C. White. A Markov quality control process subject to partial observation. Management Science 23 (1977), 843–852.
P. Whittle. Risk-Sensitive Optimal Control New York: John Wiley & Sons, 1990.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1997 Springer Science+Business Media New York
About this paper
Cite this paper
Marcus, S.I., Fernández-Gaucherand, E., Hernández-Hernández, D., Coraluppi, S., Fard, P. (1997). Risk Sensitive Markov Decision Processes. In: Byrnes, C.I., Datta, B.N., Martin, C.F., Gilliam, D.S. (eds) Systems and Control in the Twenty-First Century. Systems & Control: Foundations & Applications, vol 22. Birkhäuser, Boston, MA. https://doi.org/10.1007/978-1-4612-4120-1_14
Download citation
DOI: https://doi.org/10.1007/978-1-4612-4120-1_14
Publisher Name: Birkhäuser, Boston, MA
Print ISBN: 978-1-4612-8662-2
Online ISBN: 978-1-4612-4120-1
eBook Packages: Springer Book Archive