Abstract
We treat non-cooperative stochastic games with countable state space and with finitely many players each having finitely many moves available in a given state. As a function of the current state and move vector, each player incurs a nonnegative cost. Assumptions are given for the expected discounted cost game to have a Nash equilibrium randomized stationary strategy. These conditions hold for bounded costs, thereby generalizing Parthasarathy (1973) and Federgruen (1978). Assumptions are given for the long-run average expected cost game to have a Nash equilibrium randomized stationary strategy, under which each player has constant average cost. A flow control example illustrates the results. This paper complements the treatment of the zero-sum case in Sennott (1993a).
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Bertsekas DP (1987) Dynamic programming: Deterministic and stochastic models. Prentice-Hall Englewood Cliffs New Jersey
Borkar VS, Ghosh MK (1992) Denumerable state stochastic games with limiting average payoff. Preprint
Fan K (1952) Fixed-point and minimax theorems in locally convex topological linear spaces. Proc Nat Acad Sci USA 38:121–126
Federgruen A (1978) Onn-person stochastic games with denumerable state space. Adv Appl Probab 10:452–471
Fink AM (1964) Equilibrium in a stochasticN-person game. J Sci Hiroshima Univ, Series A-I 28:89–93
Glicksberg II (1952) A further generalization of the Kakutani fixed-point theorem, with application to Nash equilibrium points. Proc Amer Math Soc 3:170–174
Ghosh MK, Bagchi A (1992) Stochastic games with average payoff criterion. Preprint
Nowak AS (1992) Stationary equilibria for nonzero-sum average payoff ergodic stochastic games with general state space. Preprint
Parthasarathy T (1973) Discounted, positive, and noncooperative stochastic games. Int J Game Theory 2:25–37
Parathasarathy T, Stern M (1977) Markov games — a survey. Differential Games and Control Theory II, Roxin EO, Liu PI, Sternberg RL (Eds) Dekker 1–46
Raghavan TES, Filar JA (1991) Algorithms for stochastic games — a survey. Zeitschrift für Oper Res 35:437–472
Rogers PD (1969) Nonzero-sum stochastic games. PhD Thesis Univ of California at Berkeley Berkeley California
Ross SM (1983) Introduction to stochastic dynamic programming. Academic Press New York
Seneta E, Tweedie RL (1985) Moments for stationary and quasi-stationary distributions of Markov chains. J Appl Probab 22:148–155
Sennott LI (1986) A new condition for the existence of optimal stationary policies in average cost Markov decision prodesses. Oper Res Lett 5:17–23
Sennott LI (1989) Average cost optimal stationary policies in infinite state Markov decision processes with unbounded costs. Oper Res 37:626–633
Sennott LI (1994) Zero-sum stochastic games with unbounded costs: Discounted and average cost cases. ZOR 39:209–225
Sennott LI (1993b) Another set of conditions for average optimality in Markov control processes. To appear, Systems and Control Letters 23
Shapley LS (1953) Stochastic games. Proc Nat Acad Sci USA 39:1095–1100
Sobel MJ (1971) Non-cooperative stochastic games. Ann Math Stat 42:1930–1935
Sobel MJ (1973) Continuous stochastic games. J Appl Probab 10:597–604
Stern MA (1975) On stochastic games with limiting average payoff. Thesis Univ of Illinois Chicago
Takashashi M (1964) Equilibrium points of stochastic non-cooperativen-person games. J Sci Hiroshima Univ Series A-I, 28:95–99
Tweedie RL (1976) Criteria for classifying general Markov chains. Adv Appl Probab 8:737–771
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Sennott, L.I. Nonzero-sum stochastic games with unbounded costs: Discounted and average cost cases. ZOR - Methods and Models of Operations Research 40, 145–162 (1994). https://doi.org/10.1007/BF01432807
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF01432807