Abstract
While a lot of papers on RoboCup’s robotic 2D soccer simulation have focused on the players’ offensive behavior, there are only a few papers that specifically address a team’s defense strategy. In this paper, we consider a defense scenario of crucial importance: We focus on situations where one of our players must interfere and disturb an opponent ball leading player in order to scotch the opponent team’s attack at an early stage and, even better, to eventually conquer the ball initiating a counter attack. We employ a reinforcement learning methodology that enables our players to autonomously acquire such an aggressive duel behavior, and we have embedded it into our soccer simulation team’s defensive strategy. Employing the learned NeuroHassle policy in our competition team, we were able to clearly improve the capabilities of our defense and, thus, to increase the performance of our team as a whole.
Chapter PDF
Similar content being viewed by others
Keywords
- Learning Agent
- Reinforcement Learning Method
- Multilayer Perceptron Neural Network
- Greedy Policy
- Opponent Team
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Kitano, H., Asada, M., Kuniyoshi, Y., Noda, I., Abd, H., Matsubara, E.O.: RoboCup: A Challenge Problem for AI. AI Magazine 18, 73–85 (1997)
Kyrylov, V., Greber, M., Bergman, D.: Multi-Criteria Optimization of Ball Passing in Simulated Soccer. Journal of Multi-Criteria Decision Analysis 13, 103–113 (2005)
Stone, P., Sutton, R., Kuhlmann, G.: Reinforcement Learning for RoboCup-Soccer Keepaway. Adaptive Bahvior 13, 165–188 (2005)
Dashti, H., Aghaeepour, N., Asadi, S., Bastani, M., Delafkar, Z., Disfani, F., Ghaderi, S., Kamali, S.: Dynamic Positioning Based on Voronoi Cells (DPVC). In: Bredenfeld, A., Jacoff, A., Noda, I., Takahashi, Y. (eds.) RoboCup 2005. LNCS, vol. 4020, pp. 219–229. Springer, Heidelberg (2006)
Reis, L., Lau, N., Oliveira, E.: Situation Based Strategic Positioning for Coordinating a Team of Homogeneous Agents. In: Hannebauer, M., Wendler, J., Pagello, E. (eds.) ECAI-WS 2000. LNCS, vol. 2103, pp. 175–197. Springer, Heidelberg (2001)
Riedmiller, M., Gabel, T.: On Experiences in a Complex and Competitive Gaming Domain: Reinforcement Learning Meets RoboCup. In: Proceedings of the 3rd IEEE Symposium on Computational Intelligence and Games (CIG 2007), Honolulu, USA, pp. 68–75. IEEE Press, Los Alamitos (2007)
Kalyanakrishnan, S., Liu, Y., Stone, P.: Half Field Offense in RoboCup Soccer: A Multiagent Reinforcement Learning Case Study. In: Lakemeyer, G., Sklar, E., Sorrenti, D.G., Takahashi, T. (eds.) RoboCup 2006: Robot Soccer World Cup X. LNCS, vol. 4434, pp. 72–85. Springer, Heidelberg (2007)
Kyrylov, V., Hou, E.: While the Ball in the Digital Soccer Is Rolling, Where the Non-Player Characters Should Go in a Defensive Situation?. In: Proceedings of Future Play, Toronto, Canada, pp. 13–17 (2007)
Sutton, R.S., Barto, A.G.: Reinforcement Learning. An Introduction. MIT Press/A Bradford Book, Cambridge (1998)
Gabel, T., Riedmiller, M.: Learning a Partial Behavior for a Competitive Robotic Soccer Agent. KI Zeitschrift 20, 18–23 (2006)
Bertsekas, D.P., Tsitsiklis, J.N.: Neuro Dynamic Programming. Athena Scientific, Belmont, USA (1996)
Noda, I., Matsubara, H., Hiraki, K., Frank, I.: Soccer Server: A Tool for Research on Multi-Agent Systems. Applied Artificial Intelligence 12, 233–250 (1998)
Sutton, R.S.: Learning to Predict by the Methods of Temporal Differences. Machine Learning 3, 9–44 (1988)
Riedmiller, M., Braun, H.: A Direct Adaptive Method for Faster Backpropagation Learning: The RPROP Algorithm. In: Ruspini, H. (ed.) Proceedings of the International Conference on Neural Networks (ICNN), San Francisco, pp. 586–591 (1993)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Gabel, T., Riedmiller, M., Trost, F. (2009). A Case Study on Improving Defense Behavior in Soccer Simulation 2D: The NeuroHassle Approach. In: Iocchi, L., Matsubara, H., Weitzenfeld, A., Zhou, C. (eds) RoboCup 2008: Robot Soccer World Cup XII. RoboCup 2008. Lecture Notes in Computer Science(), vol 5399. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-02921-9_6
Download citation
DOI: https://doi.org/10.1007/978-3-642-02921-9_6
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-02920-2
Online ISBN: 978-3-642-02921-9
eBook Packages: Computer ScienceComputer Science (R0)