Abstract
Stochastic (a.k.a. Markov) Games pose many unsolved problems in Game Theory. One class of stochastic games that is better understood is that of Common Interest Stochastic Games (CISG). CISGs form an interesting class of multi-agent settings where the distributed nature of the systems, rather than adverserial behavior, is the main challenge to efficient learning. In this paper we examine three different approaches to RL in CISGs, embedded in the FriendQ, OAL, and Rmax algorithms. We show the performance of the above algorithms on some non-trivial games that illustrate the advantages and disadvantages of the different approaches.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)
Kaelbling, L.P., Littman, M.L., Moore, A.W.: Reinforcement learning: A survey. JAIR 4, 237–285 (1996)
Moore, A.W., Atkcson, C.G.: Prioritized sweeping: Reinforcement learning with less data and less time. Machine Learning 13, 103–130 (1993)
Claus, C., Boutilier, C.: The dynamics of reinforcement learning in cooperative multi-agent systems. In: Proc. Workshop on Multi-Agent Learning, pp. 602–608 (1997)
Wang, X., Sandholm, T.: Reinforcement learning to play an optimal nash equilibrium in team markov games. In: NIPS 2002 (2002)
Littman, M.L.: Friend-or-foe q-learning in general-sum games. In: Proc. ICML 2001 (2001)
Brafman, R.I., Tennenholtz, M.: R-max — a general polynomial time algorithm for near-optimal reinforcement learning. JMLR 3, 213–231 (2002)
Bertsekas, D., Tsitsiklis, J.: Neuri-Dynamic Programming. Athena Scientific, Belmont (1996)
Brafman, R.I., Tennenholtz, M.: Learning to coordinate efficiently: A model based approach. JAIR 19, 11–23 (2003)
Fudenberg, D., Levine, D.: The theory of learning in games. MIT Press, Cambridge (1998)
Dearden, R., Friedman, N., Andre, D.: Model based bayesian exploration. In: UAI 1999 (1999)
Chalkiadakis, G., Boutilier, C.: Coordination in multiagent reinforcement learning: A bayesian approach. In: AAMAS 2003 (2003)
Even-Dar, E., Mansour, Y.: Learning rates for Q-learning. In: Helmbold, D.P., Williamson, B. (eds.) COLT 2001 and EuroCOLT 2001. LNCS (LNAI), vol. 2111, p. 589. Springer, Heidelberg (2001)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Bab, A., Brafman, R. (2004). An Experimental Study of Different Approaches to Reinforcement Learning in Common Interest Stochastic Games. In: Boulicaut, JF., Esposito, F., Giannotti, F., Pedreschi, D. (eds) Machine Learning: ECML 2004. ECML 2004. Lecture Notes in Computer Science(), vol 3201. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-30115-8_10
Download citation
DOI: https://doi.org/10.1007/978-3-540-30115-8_10
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-23105-9
Online ISBN: 978-3-540-30115-8
eBook Packages: Springer Book Archive