Abstract
This paper addresses the problem of learning control skills from observation. In particular, we show how to infer a hierarchical, reactive program that reproduces and explains the observed actions of other agents, specifically the elements that are shared across multiple individuals. We infer these programs using a three-stage process that learns flat unordered rules, combines these rules into a classification hierarchy, and finally translates this structure into a hierarchical reactive program. The resulting program is concise and easy to understand, making it possible to view program induction as a practical technique for knowledge acquisition.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Anderson, C., Draper, B., & Peterson, D. (2000). Behavioral cloning of student pilots with modular neural networks. Proceedings of the Seventeenth International Conference on Machine Learning (pp. 25–32). Stanford: Morgan Kaufmann.
Brooks, R. (1986). A robust layered control system for a mobile robot. IEEE Journal of Robotics and Automation, 2, 14–23.
Clark, P., Boswell, R. (1991). Rule induction with CN2: Some recent improvements. Proceedings of the European Working Session on Learning (pp. 151–163). Porto.
Cypher, A. (Ed.). (1993). Watch what I do: Programming by demonstration. Cambridge, MA: MIT Press.
Firby, J. (1989). Adaptive execution in complex dynamic worlds. PhD Thesis, Department of Computer Science, Yale University, New Haven, CT.
George., M., Lansky, A., & Bessiere, P. (1985). A procedural logic. Proceedings of the Ninth International Joint Conference on Artificial Intelligence (pp. 516–523). Los Angeles: Morgan Kaufmann.
Kaelbling, L. P., Littman, L. M., & Moore, A. W. (1996). Reinforcement learning: A survey. Journal of Artificial Intelligence Research, 4, 237–285.
Langley, P., & Stromsten, S. (2000). Learning context-free grammars with a simplicity bias. Proceedings of the Eleventh European Conference on Machine Learning (pp. 220–228). Barcelona: Springer-Verlag.
Mitchell, T. M., Mahadevan, S., & Steinberg, L. (1985). Leap: A learning apprentice for VLSI design. Proceedings of the Ninth International Joint Conference on Artificial Intelligence (pp. 573–580). Los Angeles: Morgan Kaufmann.
Moriarty, D. E., Schultz, A. C., & Grefenstette, J. J. (1999). Evolutionary algorithms for reinforcement learning. Journal of Artificial Intelligence Research, 11, 241–276.
Nilsson, N. (1994). Teleoreactive programs for agent control. Journal of Artificial Intelligence Research, 1, 139–158.
Pomerleau, D. (1991). Rapidly adapting artificial neural networks for autonomous navigation. Advances in Neural Information Processing Systems 3 (pp. 429–435). San Francisco: Morgan Kaufmann.
Quinlan, J. R. (1993). C4.5: Programs for machine learning. San Francisco: Morgan Kaufmann.
Sammut, C. (1996). Automatic construction of reactive control systems using symbolic machine learning. Knowledge Engineering Review, 11, 27–42.
Schoppers, M. (1987). Universal plans for reactive robots in unpredictable environments. Proceedings of the Tenth International Joint Conference on Artificial Intelligence (pp. 1039–1046). Milan, Italy: Morgan Kaufmann.
Segre, A. (1987). A learning apprentice system for mechanical assembly. Proceedings of the Third IEEE Conference on AI for Applications (pp. 112–117).
Shapiro, D., Langley, P., & Shachter, R. (2001). Using background knowledge to speed reinforcement learning in physical agents. Proceedings of the Fifth International Conference on Autonomous Agents (pp. 254–261). Montreal: ACM Press.
Shapiro, D. (2001). Value-driven agents. PhD thesis, Department of Management Science and Engineering, Stanford University, Stanford, CA.
Shapiro, D., & Langley, P. (2002). Separating skills from preference: Using learning to program by reward. Proceedings of the Nineteenth International Conference on Machine Learning (pp. 570–577). Sydney: Morgan Kaufmann.
Urbancic, T., & Bratko, I. (1994). Reconstructing human skill with machine learning. Proceedings of the Eleventh European Conference on Artificial Intelligence (pp. 498–502). Amsterdam: John Wiley.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2002 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Ichise, R., Shapiro, D., Langley, P. (2002). Learning Hierarchical Skills from Observation. In: Lange, S., Satoh, K., Smith, C.H. (eds) Discovery Science. DS 2002. Lecture Notes in Computer Science, vol 2534. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-36182-0_22
Download citation
DOI: https://doi.org/10.1007/3-540-36182-0_22
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-00188-1
Online ISBN: 978-3-540-36182-4
eBook Packages: Springer Book Archive