Abstract
We describe a reinforcement learning system that transfers skills from a previously learned source task to a related target task. The system uses inductive logic programming to analyze experience in the source task, and transfers rules for when to take actions. The target task learner accepts these rules through an advice-taking algorithm, which allows learners to benefit from outside guidance that may be imperfect. Our system accepts a human-provided mapping, which specifies the similarities between the source and target tasks and may also include advice about the differences between them. Using three tasks in the RoboCup simulated soccer domain, we demonstrate that this system can speed up reinforcement learning substantially.
This research is partially supported by DARPA grant HR0011-04-1-0007 and US Naval Research Laboratory grant N00173-06-1-G002.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Driessens, K., Dzeroski, S.: Integrating experimentation and guidance in relational reinforcement learning. In: Proc. ICML (2002)
Fung, G., Sandilya, S., Rao, B.: Rule extraction from linear support vector machines. In: Proc. KDD (2005)
Kuhlmann, G., Stone, P., Mooney, R., Shavlik, J.: Guiding a reinforcement learner with natural language advice: Initial results in RoboCup soccer. In: AAAI Workshop on Supervisory Control of Learning and Adaptive Systems (2004)
Maclin, R., Shavlik, J.: Creating advice-taking reinforcement learners. Machine Learning 22, 251–281 (1996)
Maclin, R., Shavlik, J., Torrey, L., Walker, T., Wild, E.: Giving advice about preferred actions to reinforcement learners via knowledge-based kernel regression. In: Proc. AAAI (2005)
Muggleton, S., De Raedt, L.: Inductive logic programming: Theory and methods. Journal of Logic Programming 19, 20, 629–679 (1994)
Singh, S.: Transfer of learning by composing solutions of elemental sequential tasks. Machine Learning 8(3-4), 323–339 (1992)
Srinivasan, A.: The Aleph manual (2001), http://web.comlab.ox.ac.uk/oucl/research/areas/machlearn/Aleph/aleph.html
Stone, P., Sutton, R.: Scaling reinforcement learning toward RoboCup soccer. In: Proc. ICML (2001)
Sun, R.: Knowledge extraction from reinforcement learning. New Learning Paradigms in Soft Computing, 170–180 (2002)
Sutton, R.: Learning to predict by the methods of temporal differences. Machine Learning 3, 9–44 (1988)
Sutton, R.: Generalization in reinforcement learning: Successful examples using sparse coarse coding. In: Proc. NIPS (1996)
Sutton, R., Barto, A.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)
Taylor, M., Stone, P.: Behavior transfer for value-function-based reinforcement learning. In: Proc. AAMAS (2005)
Torrey, L., Walker, T., Shavlik, J., Maclin, R.: Using advice to transfer knowledge acquired in one reinforcement learning task to another. In: Gama, J., Camacho, R., Brazdil, P.B., Jorge, A.M., Torgo, L. (eds.) ECML 2005. LNCS, vol. 3720, pp. 412–424. Springer, Heidelberg (2005)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Torrey, L., Shavlik, J., Walker, T., Maclin, R. (2006). Skill Acquisition Via Transfer Learning and Advice Taking. In: Fürnkranz, J., Scheffer, T., Spiliopoulou, M. (eds) Machine Learning: ECML 2006. ECML 2006. Lecture Notes in Computer Science(), vol 4212. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11871842_41
Download citation
DOI: https://doi.org/10.1007/11871842_41
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-45375-8
Online ISBN: 978-3-540-46056-5
eBook Packages: Computer ScienceComputer Science (R0)