Abstract
In recent years, the subject of learning autonomous robots has been widely discussed. Reinforcement learning (RL) is a popular method in this domain. However, its performance is quite sensitive to the discretization of state and action spaces. To overcome this problem, we have developed a new technique called Bayesian-discrimination-function-based RL (BRL). BRL has proven to be more effective than other standard RL algorithms in dealing with multi-robot system (MRS) problems. However, similar to most learning systems, BRL occasionally suffers from overfitting. This paper introduces an extension of BRL for improving the robustness of MRSs. Meta-learning based on the information entropy of firing rules is adopted for adaptively modifying its learning parameters. Physical experiments are conducted to verify the effectiveness of our proposed method.
Access provided by Autonomous University of Puebla. Download to read the full chapter text
Chapter PDF
Similar content being viewed by others
References
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)
Sutton, R.S.: Generalization in Reinforcement Learning: Successful Examples Using Sparse Coarse Coding. Advances in Neural Information Processing Systems 8, 1038–1044 (1996)
Morimoto, J., Doya, K.: Acquisition of Stand-Up Behavior by a Real Robot using Hierarchical Reinforcement Learning for Motion Learning: Learning ’Stand Up’ Trajectories. In: Proc. of International Conference on Machine Learning, pp. 623–630 (2000)
Lin, L.J.: Scaling Up Reinforcement Learning for Robot Control. In: Proc. of the 10th International Conference on Machine Learning, pp. 182–189 (1993)
Kolter, J.Z., Ng, A.Y.: Regularization and Feature Selection in Least-Squares Temporal Difference Learning. In: Proc. of the 26th International Conference on Machine Learning (2009)
Nouri, A., Littman, M.L.: Dimension Reduction and Its Application to Model-Based Exploration in Continuous Spaces. Machine Learning 81(1), 85–98 (2010)
Asada, M., Noda, S., Hosoda, K.: Action-Based Sensor Space Categorization for Robot Learning. In: Proc. of IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 1502–1509 (1996)
Takahashi, Y., Asada, M., Hosoda, K.: Reasonable Performance in Less Learning Time by Real Robot Based on Incremental State Space Segmentation. In: Proc. of IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 1502–1524 (1996)
Doya, K.: Reinforcement Learning in Continuous Time and Space. Neural Computation 12, 219–245 (2000)
Peters, J., Schaal, S.: Natural actor critic. Neurocomputing 71(7-9), 1180–1190 (2008)
Yasuda, T., Ohkura, K.: Autonomous Role Assignment in Homogeneous Multi-Robot Systems. Journal of Robotics and Mechatronics 17(5), 596–604 (2005)
Yasuda, T., Ohkura, K.: Improving Search Efficiency in the Action Space of an Instance-Based Reinforcement Learning. In: Almeida e Costa, F., Rocha, L.M., Costa, E., Harvey, I., Coutinho, A. (eds.) ECAL 2007. LNCS (LNAI), vol. 4648, pp. 325–334. Springer, Heidelberg (2007)
Yasuda, T., Ohkura, K.: Reinforcement Learning Technique with an Adaptive Action Generator for a Multi-Robot System. In: Asada, M., Hallam, J.C.T., Meyer, J.-A., Tani, J. (eds.) SAB 2008. LNCS (LNAI), vol. 5040, pp. 250–259. Springer, Heidelberg (2008)
Doya, K.: Metalearning and neuromodulation. Neural Networks 15(4-6), 495–506 (2002)
Schweighofer, N., Doya, K.: Meta-learning in Reinforcement Learning. Neural Networks 16(1), 5–9 (2003)
Elfwing, S., Uchibe, E., Doya, K., Chiristensen, H.I.: Co-evolution of Shaping Rewards and Meta-Parameters in Reinforcement Learning. Adaptive Behavior 16, 400–412 (2008)
Tenenbaum, J.B., de Sliva, V., Lagford, J.C.: A Global Geometric Framework for Nonlinear Dimensionality Reduction. Science 290(22), 2319–2323 (2000)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Yasuda, T., Wada, M., Ohkura, K. (2011). Instance-Based Reinforcement Learning Technique with a Meta-learning Mechanism for Robust Multi-Robot Systems. In: Groß, R., Alboul, L., Melhuish, C., Witkowski, M., Prescott, T.J., Penders, J. (eds) Towards Autonomous Robotic Systems. TAROS 2011. Lecture Notes in Computer Science(), vol 6856. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-23232-9_15
Download citation
DOI: https://doi.org/10.1007/978-3-642-23232-9_15
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-23231-2
Online ISBN: 978-3-642-23232-9
eBook Packages: Computer ScienceComputer Science (R0)