Abstract
The present work surveys research that integrates successfully a number of complementary fields in Artificial Intelligence. Starting from integrations in Reinforcement Learning: Deep Reinforcement Learning and Relational Reinforcement Learning, we then present Neural-Symbolic Learning and Reasoning since it is applied to Deep Reinforcement Learning. Finally, we present integrations in Deep Reinforcement Learning, such as, Relational Deep Reinforcement Learning. We propose that this road is breaking through barriers in Reinforcement Learning and making us closer to Artificial General Intelligence, and we share views about the current challenges to get us further towards this goal.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Russell, S.J., Norvig, P.: Artificial Intelligence - A Modern Approach. Pearson Education, London (2010). Third International Edition
LeCun, Y., Bengio, Y., Hinton, G.E.: Deep learning. Nature 521, 436–444 (2015)
Sutton, R.S., Barto, A.G.: Reinforcement learning: an introduction. IEEE Trans. Neural Netw. 16, 285–286 (1988)
Kaelbling, L.P., Littman, M.L., Moore, A.W.: Reinforcement learning: a survey. ArXiv, cs.AI/9605103 (1996)
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A.A., Veness, J., Bellemare, M.G., Graves, A., Riedmiller, M.A., Fidjeland, A., Ostrovski, G., Petersen, S., Beattie, C., Sadik, A., Antonoglou, I., King, H., Kumaran, D., Wierstra, D., Legg, S., Hassabis, D.: Human-level control through deep reinforcement learning. Nature 518, 529–533 (2015)
Irpan, A.: Deep reinforcement learning doesn’t work yet (2018). https://www.alexirpan.com/2018/02/14/rl-hard.html
Domingos, P.: The Master Algorithm: How the Quest for the Ultimate Learning Machine Will Remake Our World. Basic Books, New York (2015)
Bishop, C.M.: Pattern Recognition and Machine Learning. Information Science and Statistics, 5th edn. Springer, Heidelberg (2007)
Domingos, P.M., Lowd, D.: Unifying logical and statistical AI with markov logic. Commun. ACM 62(7), 74–83 (2019)
Garnelo, M., Arulkumaran, K., Shanahan, M.: Towards deep symbolic reinforcement learning. ArXiv, abs/1609.05518 (2016)
Santoro, A., Raposo, D., Barrett, D.G.T., Malinowski, M., Pascanu, R., Battaglia, P.W., Lillicrap, T.P.: A simple neural network module for relational reasoning. In: NIPS (2017)
Zambaldi, V.F., Raposo, D., Santoro, A., Bapst, V., Li, Y., Babuschkin, I., Tuyls, K., Reichert, D.P., Lillicrap, T.P., Lockhart, E., Shanahan, M., Langston, V., Pascanu, R., Botvinick, M.M., Vinyals, O., Battaglia, P.W.: Relational deep reinforcement learning. ArXiv, abs/1806.01830 (2018)
Paes, A., Zaverucha, G., Costa, V.S.: On the use of stochastic local search techniques to revise first-order logic theories from examples. Mach. Learn. 106(2), 197–241 (2017)
Fitting, M.: First-Order Logic and Automated Theorem Proving. Graduate Texts in Computer Science, 2nd edn. Springer, Heidelberg (1996)
Christopher JCH Watkins and Peter Dayan: Q-learning. Mach. Learn. 8(3–4), 279–292 (1992)
Silver, D., Huang, A., Maddison, C.J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., Dieleman, S., Grewe, D., Nham, J., Kalchbrenner, N., Sutskever, I., Lillicrap, T.P., Leach, M., Kavukcuoglu, K., Graepel, T., Hassabis, D.: Mastering the game of Go with deep neural networks and tree search. Nature 529, 484–489 (2016)
Silver, D., Hubert, T., Schrittwieser, J., Antonoglou, I., Lai, M., Guez, A., Lanctot, M., Sifre, L., Kumaran, D., Graepel, T., Lillicrap, T.P., Simonyan, K., Hassabis, D.: A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play. Science 362, 1140–1144 (2018)
Ghazanfari, B., Afghah, F., Taylor, M.E.: Autonomous extraction of a hierarchical structure of tasks in reinforcement learning, a sequential associate rule mining approach. ArXiv, abs/1811.08275 (2018)
Ghazanfari, B., Taylor, M.E.: Autonomous extracting a hierarchical structure of tasks in reinforcement learning and multi-task reinforcement learning. ArXiv, abs/1709.04579 (2017)
El Bsat, S., Bou-Ammar, H., Taylor, M.E.: Scalable multitask policy gradient reinforcement learning. In: AAAI (2017)
Andrychowicz, M., Wolski, F., Ray, A., Schneider, J., Fong, R., Welinder, P., McGrew, B., Tobin, J., Abbeel, P., Zaremba, W.: Hindsight experience replay. In: Advances in Neural Information Processing Systems, pp. 5048–5058 (2017)
Sutton, R.S.: Dyna, an integrated architecture for learning, planning, and reacting. SIGART Bull. 2, 160–163 (1990)
Finn, C., Yu, T., Zhang, T., Abbeel, P., Levine, S.: One-shot visual imitation learning via meta-learning. ArXiv, abs/1709.04905 (2017)
Andreas, J., Klein, D., Levine, S.: Modular multitask reinforcement learning with policy sketches. ArXiv, abs/1611.01796 (2016)
Hussein, A., Gaber, M.M., Elyan, E., Jayne, C.: Imitation learning: a survey of learning methods. ACM Comput. Surv. 50, 21:1–21:35 (2017)
Gunning, D.: Explainable artificial intelligence (XAI). Defense Advanced Research Projects Agency (DARPA), nd Web, 2 (2017)
Ribeiro, M.T., Singh, S., Guestrin, C.: Why should i trust you?: explaining the predictions of any classifier. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1135–1144. ACM (2016)
Osband, I., Doron, Y., Hessel, M., Aslanides, J., Sezener, E., Saraiva, A., McKinney, K., Lattimore, T., Szepezvári, C., Singh, S., Van Roy, B., Sutton, R.S., Silver, D., van Hasselt, H.: Behaviour suite for reinforcement learning. ArXiv, abs/1908.03568 (2019)
Džeroski, S., De Raedt, L., Driessens, K.: Relational reinforcement learning. Machine learning 43(1–2), 7–52 (2001)
Tadepalli, P., Givan, R., Driessens, K.: Relational reinforcement learning: an overview. In: Proceedings of the ICML-2004 Workshop on Relational Reinforcement Learning, pp. 1–9 (2004)
Van Otterlo, M.: Relational representations in reinforcement learning: review and open problems. In: Proceedings of the ICML, vol. 2 (2002)
Quinlan, J.R.: Learning logical definitions from relations. Mach. Learn. 5, 239–266 (1990)
Morales, E.F.: Scaling up reinforcement learning with a relational representation. In: Proceedings of the Workshop on Adaptability in Multi-agent Systems, pp. 15–26 (2003)
Neural-symbolic integration. http://www.neural-symbolic.org
Evans, R., Grefenstette, E.: Learning explanatory rules from noisy data. J. Artif. Intell. Res. 61, 1–64 (2017)
Lloyd, J.W.: Foundations of logic programming. In: Symbolic Computation (1984)
Dong, H., Mao, J., Lin, T., Wang, C., Li, L., Zhou, D.: Neural logic machines. ArXiv, abs/1904.11694 (2019)
Trask, A., Hill, F., Reed, S.E., Rae, J.W., Dyer, C., Blunsom, P.: Neural arithmetic logic units. In: NeurIPS (2018)
Faris, W.G.: The number sense: how the mind creates mathematics by stanislas dehaene. Complexity 4(1), 46–48 (1998)
Gallistel, C.R.: Finding numbers in the brain. Philos. Trans. Roy. Soc. London Ser. B Biol. Sci. 373(1740) (2017)
Fodor, J.A., Pylyshyn, Z.W.: Connectionism and cognitive architecture: a critical analysis. Cognition 28, 3–71 (1988)
Marcus, G.F.: Integrating connectionism and cognitive science, The algebraic mind (2001)
Battaglia, P.W., Hamrick, J.B., Bapst, V., Sanchez-Gonzalez, A., Zambaldi, V.F., Malinowski, M., Tacchetti, A., Raposo, D., Santoro, A., Faulkner, R., Gülçehre, Ç., Francis Song, H., Ballard, A.J., Gilmer, J., Dahl, G.E., Vaswani, A., Allen, K.R., Nash, C., Langston, V., Dyer, C., Heess, N.M.O., Wierstra, D., Kohli, P., Botvinick, M.M., Vinyals, O., Li, Y., Pascanu, R.: Relational inductive biases, deep learning, and graph networks. ArXiv, abs/1806.01261 (2018)
Battaglia, P.W., Pascanu, R., Lai, M., Rezende, D.J., Kavukcuoglu, K.: Interaction networks for learning about objects, relations and physics. In: NIPS (2016)
Zambaldi, V.F., Raposo, D.C., Santoro, A., Bapst, V., Li, Y., Babuschkin, I., Tuyls, K., Reichert, D.P., Lillicrap, T.P., Lockhart, E., Shanahan, M., Langston, V., Pascanu, R., Botvinick, M.M., Vinyals, O., Battaglia, P.W.: Deep reinforcement learning with relational inductive biases. In: ICLR (2019)
Jiang, Z., Luo, S.: Neural logic reinforcement learning. ArXiv, abs/1904.10729 (2019)
Sutton, R.S., McAllester, D.A., Singh, S.P., Mansour, Y.: Policy gradient methods for reinforcement learning with function approximation. In: NIPS (1999)
Botvinick, M.M., Barrett, D.G.T., Battaglia, P.W., de Freitas, N., Kumaran, D., Leibo, J.Z., Lillicrap, T., Modayil, J., Mohamed, S., Rabinowitz, N.C., Rezende, D.J., Santoro, A., Schaul, T., Summerfield, C., Wayne, G., Weber, T., Wierstra, D., Legg, S., Hassabis, D.: Building machines that learn and think for themselves: commentary on lake et al., behavioral and brain sciences, 2017. Behavioral Brain Sci. 40, e255 (2017)
Lake, B.M., Ullman, T.D., Tenenbaum, J.B., Gershman, S.J.: Building machines that learn and think like people. Behav. Brain Sci. 40, e253 (2017)
Acknowledgements
We thank the institutions CRACS/INESCTEC\(^4\) and LIACC/UP\(^3\), for the support and contribution of its members, that was very valuable, for the research behind this paper and its presentation.
This work is financed by National Funds through the Portuguese funding agency, FCT - Fundação para a Ciência e a Tecnologia within projects: UID/EEA/50014 and UID/EEA/00027.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Rocha, F.M., Costa, V.S., Reis, L.P. (2020). From Reinforcement Learning Towards Artificial General Intelligence. In: Rocha, Á., Adeli, H., Reis, L., Costanzo, S., Orovic, I., Moreira, F. (eds) Trends and Innovations in Information Systems and Technologies. WorldCIST 2020. Advances in Intelligent Systems and Computing, vol 1160. Springer, Cham. https://doi.org/10.1007/978-3-030-45691-7_37
Download citation
DOI: https://doi.org/10.1007/978-3-030-45691-7_37
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-45690-0
Online ISBN: 978-3-030-45691-7
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)