Abstract
The paper deals with a class of semi-Markov control models with Borel state and control spaces and possibly unbounded costs, where the holding times distribution F depends on an unknown and possibly non-observable parameter which may change from stage to stage. The system is modeled as a game against nature, which is a particular case of a minimax control system. The objective is to show the existence of minimax strategies under the discounted and average cost criteria.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Altman, E., Hordijk, A.: Zero-sum Markov games and worst-case optimal control of queueing systems. Queueing Syst. Theory Appl. 21, 415–447 (1995)
Coraluppi, S.P., Marcus, S.I.: Mixed risk-neutral/minimax control of discrete-time finite state Markov decision process. IEEE Trans. Autom. Control 45, 528–532 (2000)
Dynkin, E.B., Yushkevich, A.A.: Controlled Markov Processes. Springer, New York (1979)
Federgruen, A., Tijms, H.C.: The optimality equation in average cost denumerable state semi-Markov decision problems. Recurrence conditions and algorithms. J. Appl. Probab. 15, 356–373 (1978)
Federgruen, A., Schweitzer, P.J., Tijms, H.C.: Denumerable undiscounted semi-Markov decision processes with unbounded rewards. Math. Oper. Res. 8, 298–313 (1983)
González-Trejo, T.J., Hernández-Lerma, O., Hoyos-Reyes, L.F.: Minimax control of discrete-time stochastic systems. SIAM J. Control Optim. 41, 1626–1659 (2003)
Hernández-Lerma, O., Lasserre, J.B.: Discrete-Time Markov Control Processes: Basic Optimality Criteria. Springer, New York (1996)
Hordijk, A., Passchier, O., Spieksma, F.M.: Optimal control against worst case admission policies: A multichained stochastic game. Math. Methods Oper. Res. 45, 281–301 (1997)
Jagannathan, R.: A minimax ordering policy for the infinite stage dynamic inventory problem. Manag. Sci. 24, 1138–1149 (1978)
Jaskiewicz, A.: An approximation approach to ergodic semi-Markov control processes. Math. Methods Oper. Res. 54, 1–19 (2001)
Jaskiewicz, A.: A fixed point approach to solve the average cost optimality equation for semi-Markov decision processes with Feller transition probabilities. Commun. Stat., Theory Methods 36, 2559–2575 (2007)
Kalyanasundaram, S., Chong, E.K.P., Shroff, N.B.: Markov decision processes with uncertain transition rates: sensitivity and max-min control. Asian J. Control 6(2), 253–269 (2004)
Küenle, H.-U.: Stochastiche Spiele und Entscheidungsmodelle. B. G. Teubner, Leipzig (1986)
Küenle, H.-U.: On the optimality of (s,S)-strategies in a minimax inventory model with average cost criterion. Optimization 22, 123–138 (1991)
Kurano, M.: Discrete-time Markovian decision processes with an unknown parameter-average return criterion. J. Oper. Res. Soc. Jpn. 15, 67–76 (1972)
Kurano, M.: Minimax strategies for average cost stochastic games with an application to inventory models. J. Oper. Res. Soc. Jpn. 30, 232–247 (1987)
Luque-Vásquez, F., Hernández-Lerma, O.: Semi-Markov models with average costs. Appl. Math. 26, 315–331 (1999)
Luque-Vásquez, F., Minjárez-Sosa, J.A.: Semi-Markov control processes with unknown holding times distribution under a discounted criterion. Math. Methods Oper. Res. 61, 455–468 (2005)
Luque-Vásquez, F., Minjárez-Sosa, J.A., Rosas-Rosas, L.C.: Semi-Markov control processes with unknown holding times distribution under an average cost criterion. Appl. Math. Optim. 61, 217–336 (2010)
Mandl, P.: Estimation and control in Markov chains. Adv. Appl. Probab. 6, 40–60 (1974)
Milliken, P., Marsh, C., Van Brunt, B.: Minimax controller design for a class of uncertain nonlinear systems. Automatica 35, 583–590 (1999)
Puterman, M.L.: Markov Decision Processes. Discrete Stochastic Dynamic Programming. Wiley, New York (1994)
Rieder, U.: Measurable selection theorems for optimization problems. Manuscripta Math. 115–131 (1978)
Savkin, A.V., Peterson, I.R.: Minimax optimal control of uncertain systems with structured uncertainty. Int. J. Robust Nonlinear Control 5, 119–137 (1995)
Schäl, M.: Conditions for optimality and for the limit on n-stage optimal policies to be optimal. Z. Wahrscheinlichkeitstheor. Verw. Geb. 32, 179–196 (1975)
Schweitzer, P.J.: Iterative solution of the functional equations of undiscounted Markov renewal programming. J. Math. Anal. Appl. 34, 495–501 (1971)
Vega-Amaya, O.: The average cost optimality equation: a fixed point approach. Bol. Soc. Mat. Mexicana 9, 185–195 (2003)
Yu, W., Guo, X.: Minimax controller design for discrete-time time-varying stochastic systems. In: Proceedings of the 41st IEEE CDC, Las Vegas, Nevada, USA, pp. 598–603 (2002)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Luque-Vásquez, F., Minjárez-Sosa, J.A. & Rosas-Rosas, L.d.C. Semi-Markov Control Models with Partially Known Holding Times Distribution: Discounted and Average Criteria. Acta Appl Math 114, 135–156 (2011). https://doi.org/10.1007/s10440-011-9605-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10440-011-9605-y
Keywords
- Semi-Markov control processes
- Discounted and average cost criteria
- Minimax control systems
- Games against nature